Google is encrypting its search result URLs. I'm still recovering all of them in one pass.
Google has started replacing the destination URL on search results with a link to google.com/goto and an encrypted blob. A Google spokesperson confirmed it to Search Engine Roundtable (photo credits to Barry Schwartz) as an anti-scraping measure.
Every rank tracker and SERP API you use reads those URLs out of the HTML. Encrypting them breaks that.
You’ve probably not seen it. It only happens when you’re logged out. Log in and your results are still clean links, so this has spread through scraper people and nobody else.
I spent today taking it apart.
They’ve done the encryption properly. The blob decodes to protobuf. There’s a version byte, then what looks like a key ID, then the ciphertext. The key ID is identical on every result I pulled, across different queries and different sessions. When Google rotates the key, that value is what changes.
The ciphertext is randomised on every render. I fetched the same query twice, seconds apart, and got eight different blobs at eight identical lengths. So you can’t build a lookup table of blob to URL and reuse it. Every one is single use.
The ping attribute used to expose the clean URL in plaintext. It doesn’t now. It’s the same ciphertext, plus a new uoh flag.
curl is finished too. It comes back with HTTP 200 and about 91KB, no captcha or “enable JavaScript” wall, just a JavaScript redirect shell with no results in it.
Natzir Turrado has posted a workaround that uses Google’s own translation links. That one’s public and he joked how John Mueller has already liked his post. So this method might be patched soon.
I found a much better way. Mine works differently. One pass over the page Google already served, no redirect following or extra requests, and it gave me every organic result in rank order on every query I tried, Reddit included.
I’m not publishing it because I don’t want this patched. If you work on SERP data and you want it, message me on socials or use my contact form.
I will only share this with people I know in the industry. Thanks for understanding.