This year I’m on a panel instead of giving a talk. It’s titled “The Adaptive Web: Turning Industry Challenges into Competitive Advantage”, and I’ll share the stage with Andreea Stroe from Adobe, Hocine Amrane from NielsenIQ, and Andrey Gourine from Limy AI. Forty minutes with people who look at web data from very different seats than mine, so I expect to learn more than I speak.
The full agenda has 10 sessions and 18 speakers. A few I’m curious about:
Etienne Ellie from ScrapingBee on “Web Hydration: Loading Dynamic Content Without a Browser”. Getting data without rendering the page is a topic I keep coming back to in The Lab.
“We Made Our Web Search Worse on Purpose” by Emilis Strimaitis from Hostinger, if only for the title.
The panel “Is the Web Closing? What It Could Mean for Businesses, Journalists, and Researchers” with Carl Miller, Paul Bradshaw and Krishna Sood. The pay-per-crawl discussion I return to below is the same question seen from the other side of the fence.
Tadas Gedgaudas from topYappers closing with “Best Open-Source Scraping Repositories of 2026”, which is only for live viewers.
As I did with the 2025 edition, I’ll publish a wrap-up right after the event.
FuelTurin
The second date is an event I’m organizing together with DataFuel, an Italian company that launched its brand and scraping API this year. On Wednesday, October 7, during the Wave by Vento week in Turin, we’re hosting a networking aperitivo for people who work with web data and AI.
Wave by Vento is the new name of Italian Tech Week. It runs from October 7 to 9, with great speakers (Jony Ive, Luca Ferrari, CEO of Bending Spoons, Nik Storonsky, CEO of Revolut, and others) on the main stage, and it brings many people from our industry to Turin on those days. A rooftop aperitivo felt like the simplest way to get around 40 of them in the same room: founders, web data experts, AI and agent builders.
The format is simple. You bring what you work on; we cover drinks and food. No pitches, no slides, no demos, no badge scanning. Registration is by approval, so tell us what you do when you sign up.
What the SERP API vendors did about Google’s goto links
On August 30 I published what I found about Google’s goto links, the change that replaced every destination URL in the search results with a google.com/goto?url= token. Two findings are worth repeating here, because they explain the answers that came from the vendors.
The comments under that article added a detail I couldn’t reproduce on my side. BigBry gets the go-to links in a human-controlled incognito window, with no CDP involved, and in Firefox when logged out. Michael Vederman drove Chrome and Edge with three different stacks and wrote that “it all boiled down to using a signed-in profile or not”, although after a few more hits Firefox gave him the wrappers logged in or not. In my runs, a private window never showed a wrapper, so the observations don’t line up perfectly. My reading is that the signed-in state weighs more than my two windows suggested, and that the classifier looks at more than one signal. If you run your own test, check the account state before you check the flags.
Resolving the tokens is cheap. A plain GET with no cookies, no proxy and no TLS impersonation returns a 302 with the destination in the Location header. I ran 1,200 resolutions from one home IP at concurrency 20 and got 110 requests per second sustained, with zero 429s. Tokens captured one day still resolve the next morning. Two traps remain: HEAD answers 200 without a Location header, so it looks like a success and gives you nothing, and the same result gets a new token on every impression, so there is no lookup table to build.
DataForSEO published on August 27 that it resolves the organic goto URLs for 99.99% of returned SERPs, with the remaining cases concentrated in AI Overviews and local packs. SerpApi announced on September 5 that direct URLs are back in its responses, and told PPC Land that the change “required updates to how Google result links were decoded and normalized”, without saying whether that means following the redirects or how much it costs them. In the same article, Johannes Beus from SISTRIX described the goto links as “a change requiring an added-cost workaround”.
So everyone who sells Google’s index reacted quickly to this update, and the impact on the SERP scraping ecosystem was minimal.
The Web Scraping Club is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
For this month, that’s it; see you in October with the next edition of the Crawl of Fame.
Great write-up, this matches what I've seen too. In my experience logged-in sessions consistently give cleaner, more stable SERPs to scrape, since Google leans harder on the redirect-wrapper and other friction for anonymous/logged-out traffic. Kameleo actually makes this pretty painless: you log into an account once, save that browser profile persistently in storage, and then reload it on demand while keeping full fingerprint stealth. So you get the benefits of a logged-in session without having to babysit logins on every run.
Great write-up, this matches what I've seen too. In my experience logged-in sessions consistently give cleaner, more stable SERPs to scrape, since Google leans harder on the redirect-wrapper and other friction for anonymous/logged-out traffic. Kameleo actually makes this pretty painless: you log into an account once, save that browser profile persistently in storage, and then reload it on demand while keeping full fingerprint stealth. So you get the benefits of a logged-in session without having to babysit logins on every run.