The Web Scraping Club | Pierluigi Vinciguerra | Substack

Can invisible_playwright Beat Bot Detection? A Hands-On Test and Review

Have we finally found a good replacement for playwright-stealth, or even a strong alternative to Camoufox or Patchright?

READ THE LATEST

Most Popular

THE LAB #1: Scraping data from an app

Sep 4, 2022 • Pierluigi Vinciguerra

THE LAB #3: Scraping Cloudflare protected websites

Sep 27, 2022 • Pierluigi Vinciguerra

THE LAB #73: How to Bypass Cloudflare in 2025

Jan 23, 2025 • Pierluigi Vinciguerra

Ten years of web scraping: a personal perspective about selling web data

Mar 24, 2024 • Pierluigi Vinciguerra

Platinum Partner

The Great Web Unblocker Benchmark - Cloudflare Edition

Testing different web unblockers against Indeed.com

Sep 22, 2024 • Pierluigi Vinciguerra

The Web Unblocker Cost Benchmark

Price comparison between the most well-known web unblockers on the market

Dec 24, 2023 • Pierluigi Vinciguerra

Recent posts

What breaks in a long-running authenticated scraper session, and how to keep it logged in

Most scrapers work fine for a few hours, but not if you keep one logged in for a few weeks.

Sep 29 • Joel Griffith

How to Use Scraping to Analyze Social Media Trends

A deep dive int how to build a multi-platform social media scraping pipeline that turns posts, hashtags, and engagement data into meaningful trend…

Sep 27 • Federico Trotta

The LAB #116: Warming an Imperva reese84 cookie, and replaying it without a browser

A tenth of a cent buys thirteen minutes of scraping, from any IP I tried

Sep 24 • Pierluigi Vinciguerra

AI

Why Your RAG Pipelines Are Only as Good as Your Data Acquisition Layer

A deep dive into what RAG actually requires, where web search integration falls short, and why scraping is the only retrieval layer that scales.

Aug 2 • Federico Trotta

Build an AI Agent for Scraping and Analyzing Research Papers

Let’s build an AI agent in Python for research paper scraping and analysis.

Dec 14, 2025 • Antonello Zanini

Using NLP for Entity Extraction From Scraped Data

From theory to practice: how to extract entities from textual scraped data using NLP

Dec 7, 2025 • Federico Trotta

Using AI to Detect Patterns in Scraped Data

A practical guide on finding patterns in scraped data with advanced techniques

Nov 23, 2025 • Federico Trotta

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts