Pixeljets
  • Home
  • Blog
  • Vue.js posts
  • Author
Anthony Sidashin

Anthony Sidashin

Software engineer with 15+ years of experience. Solo founder of ScrapeNinja.net web scraping SaaS. Passionate about building and growing SaaS products. https://twitter.com/pixeljets

Slovenia •
56 posts •

How to web scrape Zillow using ScrapeNinja and JavaScript

Web scraping is a popular technique that allows developers to quickly and easily extract data from websites. It's especially useful for extracting real estate information, such as property listings and median home prices.

  • Anthony Sidashin
    Anthony Sidashin
6 min read

Low Code Web Scraping Recipe: track Apple.com for refurbished iPhones and get push alert on specific model

There is a number of projects which allow website monitoring, but I needed a pretty custom one - I wanted to check Apple.com refurbished section for iphone 12 models and get push

  • Anthony Sidashin
    Anthony Sidashin
7 min read
node.js

How to set proxy in Puppeteer: 3 ways

Puppeteer is an incredibly useful tool for automating web browsers. It allows to run headless (or non-headless) Chrome instances, automatically interacting with websites and pages in ways that would normally require manual input

  • Anthony Sidashin
    Anthony Sidashin
5 min read
node.js

How to set proxy in node-fetch

Executing http(s) request via proxy might be helpful in a lot of cases, this helps to make your http request look like it was executed from a different country or location. Setting

  • Anthony Sidashin
    Anthony Sidashin
2 min read
node.js

Web scraping in Javascript: node-fetch vs axios vs got vs superagent

There is a number of ways to perform web requests in Node.js: node-fetch, axios, got, superagentNode.js can perform HTTP requests without additional packagesWhile I don't ever use this approach because of

  • Anthony Sidashin
    Anthony Sidashin
5 min read

cURL examples: requests with proxy, set user agent, send POST JSON request, and more

cURL is a small *nix utility to perform network requests.This is a quick cheat sheet on how cURL can be used for web scraping or any other cases when you need to

  • Anthony Sidashin
    Anthony Sidashin
3 min read
I have tested out Zapier, Make.com and Pipedream.com from a developer perspective

I have tested out Zapier, Make.com and Pipedream.com from a developer perspective

A few days ago, I took a deep dive into integrating my ScrapeNinja web scrapers into Zapier, Pipedream.com, and Integromat (Make.com) to better understand the market situation among low-code and no-code

  • Anthony Sidashin
    Anthony Sidashin
13 min read

Web scraping in Google Sheets: ImportXML & alternatives

In case you want to import some random website data into Google Sheets, the obvious way to start this exciting adventure is to use importXML() function. The main advantage is that this function

  • Anthony Sidashin
    Anthony Sidashin
3 min read
Running untrusted JavaScript in Node.js

Running untrusted JavaScript in Node.js

ScrapeNinja Scraping API recently got an exciting feature called Extractors. Extractors are pieces of user-supplied Javascript code which are executed in ScrapeNinja backend so ScrapeNinja returns pure JSON with data, from any HTML

  • Anthony Sidashin
    Anthony Sidashin
11 min read
Cheerio: parse HTML in Javascript. Playground and cheatsheet

Cheerio: parse HTML in Javascript. Playground and cheatsheet

Cheerio is a de-facto standard to parse HTML in a server-side Javascript (Node.js) now. It is a fast, flexible, and lean implementation of jQuery-like syntax designed specifically for the server.Github: https:

  • Anthony Sidashin
    Anthony Sidashin
3 min read
What are USDT transfer fees now in BEP20, ERC20, TRC20?
crypto

What are USDT transfer fees now in BEP20, ERC20, TRC20?

Have you ever wondered what is the best blockchain to send USDT and why Ethereum blockchain is so expensive (is it?) in terms of gas fees, and what is the best alternative? I

  • Anthony Sidashin
    Anthony Sidashin
4 min read

How to remove background from a signature: 3 tools

Let's say you want to sign some PDF with you "real" human signature. Of course you can draw your signature using your mouse or touchpad, but this "fully digital" signature usually turns out

  • Anthony Sidashin
    Anthony Sidashin
4 min read
Puppeteer: click an element and get raw JSON from XHR/AJAX response
puppeteer

Puppeteer: click an element and get raw JSON from XHR/AJAX response

This lately became a pretty popular question when scraping with Puppeteer: let's say you want to interact with the page (e.g. click the button) and retrieve the raw ajax response (usually, JSON)

  • Anthony Sidashin
    Anthony Sidashin
5 min read
Puppeteer API service for web scraping
puppeteer

Puppeteer API service for web scraping

Okay, let's admit it - web scraping via Puppeteer and Playwright is the most versatile and flexible way of web scraping nowadays. Unfortunately it's also the most cumbersome, time consuming way of scraping,

  • Anthony Sidashin
    Anthony Sidashin
10 min read
Morning sports is my happiness magic pill
sports

Morning sports is my happiness magic pill

I am an indie hacker and CTO in my mid-thirties, and my life improved so much a few years ago when morning activity magic clicked for me. After a period of burnout, reduced

  • Anthony Sidashin
    Anthony Sidashin
6 min read
php

How to do web scraping in PHP

Web scraping is a big and hot topic now, and PHP is a pretty fast language which is  convenient for rapid prototyping and is wildly popular across web developers. I have pretty extensive

  • Anthony Sidashin
    Anthony Sidashin
7 min read
scrapeninja

ScrapeNinja: never handle retries and proxies in your code again

I am glad to announce that ScrapeNinja scraping solution just received major update and got  new features: RetriesRetries are must have for every scraping project. Proxies fail to process your request, the target

  • Anthony Sidashin
    Anthony Sidashin
2 min read
Simple proxy checker script via CURL

Simple proxy checker script via CURL

While working on the ScrapeNinja scraping solution, I often need to verify if particular proxy is alive and if it is performing well. Since I don't want to use various online services, especially

  • Anthony Sidashin
    Anthony Sidashin
2 min read
Sending Requests in Web Scraping: cURL, Chrome, Firefox, REST.client, netcat

Sending Requests in Web Scraping: cURL, Chrome, Firefox, REST.client, netcat

Contents:Chrome Dev ToolsCopy as cURLcURL options: proxy, show only headersFirefox: edit&resend; multi-account containerscURL to Python scraper converterVS Code REST.client extensionHTTP server one-liner for debuggingWhile working with scraping, I have

  • Anthony Sidashin
    Anthony Sidashin
6 min read

Making PDF look like scanned. Top 4 tools to apply scanner effect, reviewed.

Some bigger companies still require wet signatures on documents, which was a source of constant hassle for me during recent years. My workflow was:Receive email with the PDF documentDownload the documentPrint the

  • Anthony Sidashin
    Anthony Sidashin
5 min read
How to bypass CloudFlare 403 (code:1020) errors [UPDATED 2023]
scraping

How to bypass CloudFlare 403 (code:1020) errors [UPDATED 2023]

I've recently started getting Cloudflare 1020 (403) errors when scraping some random e-commerce website. At first, I thought that the website didn't like my scraper IP address, but changing IP addresses to clean

  • Anthony Sidashin
    Anthony Sidashin
7 min read

VS Code Remote for Node.js caveat: dealing with detached nodemon process

I develop all my new projects on a remote Hetzner Cloud machine, using wonderful and almost too-good-to-be-true VS Code Remote. I recommend this setup for everyone who does not like spinning fans of

  • Anthony Sidashin
    Anthony Sidashin
2 min read

Clickhouse as an alternative to ElasticSearch and MySQL, for log storage and analysis, in 2021

In 2018, I've written an article about Clickhouse, this piece of content is still pretty popular across the internet, and even was translated a few times. More than two years have passed since,

  • Anthony Sidashin
    Anthony Sidashin
10 min read

inWidget proxified: Free Instagram widget for your website, in 2021

For one of my projects, I needed a widget which would render posts from an Instagram hashtag. It turned out to be very cumbersome to implement nowadays, because Instagram shut down its legacy

  • Anthony Sidashin
    Anthony Sidashin
1 min read

Best way to daemonize node.js process in 2021: forever, pm2, nodemon, docker, supervisor, systemd and what to choose

During my development career I used a lot of different solutions to daemonize processes (mostly, node.js scripts), and I decided to do a quick writeup with very short description of each approach

  • Anthony Sidashin
    Anthony Sidashin
4 min read
Pixeljets © 2023
Latest Posts Ghost