node.js - Pixeljets

In-depth comparison of Lovable.dev and Bolt.new

I extensively use AI tools for coding - primarily Claude Sonnet 3.5 in VS Code Copilot and the OpenAI ChatGPT macOS app (using the 01 and 40 models) as of December 2024. While these tools, which felt groundbreaking just months ago, have become an integral part of my daily workflow, I see significant room for improvement in AI code tools, particularly in UX and approach. Despite progress, I still spend a lot of time copying and pasting between tools. While VS Code Copilot’s code edits and AI aut

How to set proxy in Playwright

In this article I will describe how to set a proxy in Playwright (Node.js version of Playwright). Playwright is obviously one of the best and most modern solutions to automate browsers in 2024. It uses the CDP protocol to send commands to browsers and supports Chromium, Chrome and Firefox browsers out of the box. It is open source and very well maintained. It's main use case is UI test automation and web scraping. Setting up proxies is useful for both of these use cases - especially for web scr

How do download PDF in Playwright

In the ever-evolving world of web scraping, I often come across hurdles that require creative solutions and some quick code workarounds and hacks - and oh boy! this is especially true when I am working with programmatically driven browsers, which I happen to do a lot lately. Today, I'd like to share a challenge I faced while trying to download PDF files using Playwright, and how I managed to overcome it. The Unexpected Twist with Chromium and Playwright Initially, after quickly browsing Playw

How to set proxy in Puppeteer: 3 ways

Puppeteer is an incredibly useful tool for automating web browsers. It allows to run headless (or non-headless) Chrome instances, automatically interacting with websites and pages in ways that would normally require manual input from a user or other scripts. In a lot of cases (particularly in web scraping tasks) it is required for HTTP requests to look like they originate from different IPs or networks than your server running Puppeteer – and this is where proxies come into play. In this blog po

How to set proxy in node-fetch

Executing http(s) request via proxy might be helpful in a lot of cases, this helps to make your http request look like it was executed from a different country or location. Setting proxy in node-fetch Node.js package is not simple as in Axios (where we can set a proxy by passing simple JS object with options), in node-fetch we need to pass an Agent with proxy set up, so it is a bit more manual work. But, this is also a good thing, because we can use latest and greatest proxy package from npm fo

Web scraping in Javascript: node-fetch vs axios vs got vs superagent

There is a number of ways to perform web requests in Node.js: node-fetch, axios, got, superagent Node.js can perform HTTP requests without additional packages While I don't ever use this approach because of it's poor developer ergonomics (using EventEmitter to collect the response data is just too verbose for me), Node.js is perfectly capable of sending HTTP requests without any libraries from npm! const https = require('https'); https.get('https://example.com/some-page', (resp) => { let