Simple proxy checker script via CURL
While working on the ScrapeNinja scraping solution, I often need to verify if particular proxy is alive and if it is performing well. Since I don't want to use various online services, especially for private proxies with user&password authentication, I have written a simple bash script which is much more concise than typing all the commands to terminal CURL manually:
#!/bin/bash
# download, do chmod +x and copy to /usr/bin/local via ln -s /downloaded-dir/pcheck.sh /usr/local/bin/pcheck
# then launch as: pcheck http://user:pw@proxy-addr:port (to view ip and geo via lumtest.com website)
# or pcheck http://user:pw@proxy-addr:port https://www.google.com/ (to download specific website)
curl -w @- -sS -H "user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36" --proxy "$1" ${2:-"https://lumtest.com/myip.json"} <<'EOF'
\n
time_namelookup: %{time_namelookup}\n
time_connect: %{time_connect}\n
time_appconnect: %{time_appconnect}\n
time_pretransfer: %{time_pretransfer}\n
time_redirect: %{time_redirect}\n
time_starttransfer: %{time_starttransfer}\n
----------\n
time_total: %{time_total}\n
EOF
It is also available for download via github gists: https://gist.github.com/restyler/69ceb2284be4ed54c340b3123fb71df4
There are two ways of using the script:
Check if proxy is alive and view its ip and geo
This is a default mode I use 90% of time. Basically, to check if proxy is alive and working fine, all I need to do is to type pcheck http://user:pw@proxy-addr:port
and see the output which looks like this:
As you see on the screenshot, the script also outputs total connection and download time, which is very helpful.
Better speed test for the proxy
pcheck http://user:pw@proxy-addr:port https://file-examples.com/storage/fe79dd59726252a2a9bc9b8/2017/10/file-example_PDF_500_kB.pdf
500KB download is a good way for a real performance test for a proxy. It's better to launch the script a few times to see if everything works good. 1mb pdf file is also available: https://file-examples.com/storage/fe79dd59726252a2a9bc9b8/2017/10/file-example_PDF_1MB.pdf
Checking multiple proxies concurrently
If you need to perform a multiple proxies check, it's probably a better idea to use specialized node.js script like https://github.com/restyler/proxy-checker-cli which accepts a list of proxy addresses and also provides a simple way to verify if proxy response contains an expected string of text.