Archive - The Web Scraping Club

THE LAB #113: Obscura browser against Akamai

A 30MB browser and a Radisson deal finder during Oktoberfest

16 hrs ago • Pierluigi Vinciguerra

Four years of The Web Scraping Club

On August 2022 The Web Scraping Club born. 4 years later the industry looks another world.

Aug 16 • Pierluigi Vinciguerra

Prepare Web Content and Documents for LLM Ingestion Using MarkItDown

Markdown is the language of AI. Learn how to convert different data formats into it!

Aug 9 • Antonello Zanini

Tool of the week #1: Bright Data Scraper Studio, From Prompt to Production Scraper in Minutes

Explore how Bright Data's Scraper Studio bridges the gap between AI-assisted building and full IDE-level control without locking you into either.

Aug 4 • Federico Trotta

Why Your RAG Pipelines Are Only as Good as Your Data Acquisition Layer

A deep dive into what RAG actually requires, where web search integration falls short, and why scraping is the only retrieval layer that scales.

Aug 2 • Federico Trotta

July 2026

THE LAB #112: Intercepting a Website's Internal API Calls

How I pull a store's catalog from its internal API, by hand and in code.

Jul 30 • Pierluigi Vinciguerra

Crawl of Fame - July 2026

The rollercoaster of the proxy industry and a a sigh of relief for the industry

Jul 27 • Pierluigi Vinciguerra

Hands-On Guide to SearchApi's Web Search MCP Server

Enable AI agents to access data from major search engines via MCP with SearchApi!

Jul 26 • Antonello Zanini

THE LAB #111: The OSS browser automation stealth benchmark

Fifteen tools against two live bot detectors, and only four passed the strict one.

Jul 23 • Pierluigi Vinciguerra

The Engineer's Guide to Residential Proxies: From IP Fundamentals to Exit Node Sourcing

Everything an engineer needs to know about residential proxies, ranging from how they work, their types, and where those IPs actually come from

Jul 19 • Federico Trotta

THE LAB #110: Why Shopee is so hard to scrape

Every route into Shopee hits a different wall. I mapped all of them and got past none.

Jul 16 • Pierluigi Vinciguerra

Browser Fingerprinting Deep Dive

How Detection Works and How Scrapers Adapt

Jul 14 • Dennis Kriukiov

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts