Simple proxy checker script via CURL

Simple proxy checker script via CURL

Table of Contents

While working on the ScrapeNinja scraping solution, I often need to verify if particular proxy is alive and if it is performing well. Since I don't want to use various online services, especially for private proxies with user&password authentication, I have written a simple bash script which is much more concise than typing all the commands to terminal CURL manually:

#!/bin/bash

# download, do chmod +x and copy to /usr/bin/local via ln -s /downloaded-dir/pcheck.sh /usr/local/bin/pcheck
# then launch as: pcheck http://user:pw@proxy-addr:port (to view ip and geo via lumtest.com website) 
# or pcheck http://user:pw@proxy-addr:port https://www.google.com/ (to download specific website)

curl -w @- -sS -H "user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36"  --proxy  "$1" ${2:-"https://lumtest.com/myip.json"} <<'EOF'
\n
    time_namelookup:  %{time_namelookup}\n
       time_connect:  %{time_connect}\n
    time_appconnect:  %{time_appconnect}\n
   time_pretransfer:  %{time_pretransfer}\n
      time_redirect:  %{time_redirect}\n
 time_starttransfer:  %{time_starttransfer}\n
                    ----------\n
         time_total:  %{time_total}\n
EOF

It is also available for download via github gists: https://gist.github.com/restyler/69ceb2284be4ed54c340b3123fb71df4

There are two ways of using the script:

Check if proxy is alive and view its ip and geo

This is a default mode I use 90% of time. Basically, to check if proxy is alive and working fine, all I need to do is to type pcheck http://user:pw@proxy-addr:port and see the output which looks like this:

As you see on the screenshot, the script also outputs total connection and download time, which is very helpful.

Better speed test for the proxy

pcheck http://user:pw@proxy-addr:port  https://file-examples.com/storage/fe79dd59726252a2a9bc9b8/2017/10/file-example_PDF_500_kB.pdf

500KB download is a good way for a real performance test for a proxy. It's better to launch the script a few times to see if everything works good. 1mb pdf file is also available: https://file-examples.com/storage/fe79dd59726252a2a9bc9b8/2017/10/file-example_PDF_1MB.pdf

Checking multiple proxies concurrently

If you need to perform a multiple proxies check, it's probably a better idea to use specialized node.js script like https://github.com/restyler/proxy-checker-cli which accepts a list of proxy addresses and also provides a simple way to verify if proxy response contains an expected string of text.