ScraperAPI

ScraperAPI features free plan (which says 1000 API credits but in fact they give you 5K API credits in the account) and offers 3 modes out of the box:

1. Basic web scraping API:

curl "http://api.scraperapi.com?api_key=6615f8e5aac0ace727f46faaebfd6aa0&url=http://httpbin.org/ip"

2. Async mode:

In this mode, ScraperAPI returns a job ID and you need to poll the API to get the result.

curl -X POST -H "Content-Type: application/json" -d '{"apiKey": "6615f8e5aac0ace727f46faaebfd6aa0", "url": "http://httpbin.org/ip"}' "https://async.scraperapi.com/jobs"

3. Proxy mode:

In this mode, you just use ScraperAPI as a proxy server and it will forward your requests to the target website. It's important to note that -k flag is required to disable SSL certificate verification (so ScraperAPI effectively decodes and analyzes the content which goes through proxy - this allows to make "proxy" become "smart", for example, this allows the proxy to transparently retry the particular HTTP request).

curl -x "http://scraperapi:APIKEY@proxy-server.scraperapi.com:8001" -k "http://httpbin.org/ip"