[GH-ISSUE #1252] [BUG] Whoogle broken again. Way Forward #738

Closed
opened 2026-02-25 20:36:27 +03:00 by kerem · 17 comments
Owner

Originally created by @Don-Swanson on GitHub (Oct 3, 2025).
Original GitHub issue: https://github.com/benbusby/whoogle-search/issues/1252

Originally assigned to: @Don-Swanson on GitHub.

Hey All

I just noticed it. I've been trying a bunch of other UA Strings, but Whoogle might be fully dead in the water this time. I'll have to take some time to contemplate if there is another easy way around this.

I'm open to ideas.

Potential thoughts are to scrape the Mullvad Leta results instead. They pay for Google's API so it's still google on the backend, but their results are cached, so not sure of any effects of that.

Let me know your thoughts.

Originally created by @Don-Swanson on GitHub (Oct 3, 2025). Original GitHub issue: https://github.com/benbusby/whoogle-search/issues/1252 Originally assigned to: @Don-Swanson on GitHub. Hey All I just noticed it. I've been trying a bunch of other UA Strings, but Whoogle might be fully dead in the water this time. I'll have to take some time to contemplate if there is another easy way around this. I'm open to ideas. Potential thoughts are to scrape the Mullvad Leta results instead. They pay for Google's API so it's still google on the backend, but their results are cached, so not sure of any effects of that. Let me know your thoughts.
kerem 2026-02-25 20:36:27 +03:00
Author
Owner

@blackletum commented on GitHub (Oct 3, 2025):

I was just about to open an issue because I was having problems earlier today and completely nuked my install and set it up fresh and still had the same issue.

I had a few UA strings I had tested and all worked until today. Google seems really adamant to kill all of this.

How does SearX/XNG/etc and its forks handle google results?

Looking forward to what others come up with as well. I love Whoogle and have hosted it for half a decade now and I'd hate to have to replace it.

<!-- gh-comment-id:3363838321 --> @blackletum commented on GitHub (Oct 3, 2025): I was just about to open an issue because I was having problems earlier today and completely nuked my install and set it up fresh and still had the same issue. I had a few UA strings I had tested and all worked until today. Google seems really adamant to kill all of this. How does SearX/XNG/etc and its forks handle google results? Looking forward to what others come up with as well. I love Whoogle and have hosted it for half a decade now and I'd hate to have to replace it.
Author
Owner

@mclardass commented on GitHub (Oct 3, 2025):

The days of "Don't be evil" was such a quaint time.. Good luck @Don-Swanson, I don't have any recommendations but appreciate the efforts.

<!-- gh-comment-id:3363859476 --> @mclardass commented on GitHub (Oct 3, 2025): The days of "Don't be evil" was such a quaint time.. Good luck @Don-Swanson, I don't have any recommendations but appreciate the efforts.
Author
Owner

@blackletum commented on GitHub (Oct 3, 2025):

I do know too that the folks over at Lynx have been having a heck of a time with Google as well. I did see this as well as others but it's too late for me to go through them, but I do wonder if they have any fixes over there as well.

git.sr.ht/~bptato/chawan@6b57aef32e

edit:

saw these threads too, but i've got an early start tomorrow. hopefully some helpful info.

https://lists.nongnu.org/archive/html/lynx-dev/2025-10/

<!-- gh-comment-id:3363869425 --> @blackletum commented on GitHub (Oct 3, 2025): I do know too that the folks over at Lynx have been having a heck of a time with Google as well. I did see this as well as others but it's too late for me to go through them, but I do wonder if they have any fixes over there as well. https://git.sr.ht/~bptato/chawan/commit/6b57aef32e1d17758f429ff521bc9857f9d3edd3?__goaway_challenge=meta-refresh&__goaway_id=82b3711115197373daab2c29099e4c21 edit: saw these threads too, but i've got an early start tomorrow. hopefully some helpful info. https://lists.nongnu.org/archive/html/lynx-dev/2025-10/
Author
Owner

@inittux111 commented on GitHub (Oct 3, 2025):

Hello @Don-Swanson That's too bad! I have no idea. I've been using Kagi as my main search engine for a few months now and had been using Whoogle when on my mobile phone still. I would be sad to see Whoogle end but if there's not way around then there's no way around. No matter what happens many thanks and also to the previous developer of Whoogle.

<!-- gh-comment-id:3365284493 --> @inittux111 commented on GitHub (Oct 3, 2025): Hello @Don-Swanson That's too bad! I have no idea. I've been using [Kagi](https://kagi.com) as my main search engine for a few months now and had been using Whoogle when on my mobile phone still. I would be sad to see Whoogle end but if there's not way around then there's no way around. No matter what happens many thanks and also to the previous developer of Whoogle.
Author
Owner

@aibosss commented on GitHub (Oct 3, 2025):

Anyone with a working UA, please just post that youve got one working so that we know to continue searching..
I tried ~30 diff text-based browsers UA and all need to UPGRADE.

LibreY still work with Google..

Is there no way for the backend to scrape Google's javascript ?

<!-- gh-comment-id:3365459332 --> @aibosss commented on GitHub (Oct 3, 2025): Anyone with a working UA, please just post that youve got one working so that we know to continue searching.. I tried ~30 diff text-based browsers UA and all need to UPGRADE. LibreY still work with Google.. Is there no way for the backend to scrape Google's javascript ?
Author
Owner

@si1en7 commented on GitHub (Oct 3, 2025):

Is there no way for the backend to scrape Google's javascript ?

The problem with javascript is you need it to run #and/or render it. Whoogle doesn't currently provide that sort of functionality. Whoogle would need to be adapted to take advantage of either automated browsing or headless rendering, which requires additional resources and is slow.

<!-- gh-comment-id:3365942178 --> @si1en7 commented on GitHub (Oct 3, 2025): > Is there no way for the backend to scrape Google's javascript ? The problem with javascript is you need it to run #and/or render it. Whoogle doesn't currently provide that sort of functionality. Whoogle would need to be adapted to take advantage of either automated browsing or headless rendering, which requires additional resources and is slow.
Author
Owner

@aibosss commented on GitHub (Oct 3, 2025):

I'd say not many additional resources as there are plenty of light libraries providing just that now, but yes it would be a bit slower, but I think text-based agents were already slowed down by Google for 1-2 months (longer page load time), so it would be similar speed as Whoogle was in the end.

<!-- gh-comment-id:3365957685 --> @aibosss commented on GitHub (Oct 3, 2025): I'd say not many additional resources as there are plenty of light libraries providing just that now, but yes it would be a bit slower, but I think text-based agents were already slowed down by Google for 1-2 months (longer page load time), so it would be similar speed as Whoogle was in the end.
Author
Owner

@Don-Swanson commented on GitHub (Oct 3, 2025):

Well @aibosss Don't hold out on us, if you know of a better way that doesn't use many additional resources, please do share those thoughts.

<!-- gh-comment-id:3365971203 --> @Don-Swanson commented on GitHub (Oct 3, 2025): Well @aibosss Don't hold out on us, if you know of a better way that doesn't use many additional resources, please do share those thoughts.
Author
Owner

@aibosss commented on GitHub (Oct 3, 2025):

Some ways here https://roundproxies.com/blog/scrape-google-search-results/
and especially https://github.com/web-agent-master/google-search

<!-- gh-comment-id:3366017057 --> @aibosss commented on GitHub (Oct 3, 2025): Some ways here https://roundproxies.com/blog/scrape-google-search-results/ and especially https://github.com/web-agent-master/google-search
Author
Owner

@Don-Swanson commented on GitHub (Oct 3, 2025):

@aibosss
Thanks for the enthusiasm, however it feels like you are just doing AI searches and throwing the findings here without testing or consideration of the project and its constraints. Using Playwright (like your second link suggests and like several of the examples in the first link allude to) is a non starter. As identified by @si1en7 It would severely balloon this lightweight simple project into something with an incredibly complex backend with 300-400MB of additional dependencies, this goes fundamentally against Whoogle's purpose (This might work better as a separate project with different goals that I'd love to see you develop).

The lightweight anti-detection methods your links suggest either appear to no longer work (likely due to the latest changes made by google) or cost money, which is not something this project will fund, and most end users will find another free project before spending money.

If you find something that you have tested to actually work, I invite you to share it along with your results. (Better yet, to create a PR with the required changes so that all can share in your discovery)


For everyone else, I'm going to try tonight and make a stopgap with a version that leverages leta results temporarily while I investigate how SearXNG maintains their google searches and see if it's feasible to implement.

<!-- gh-comment-id:3366195664 --> @Don-Swanson commented on GitHub (Oct 3, 2025): @aibosss Thanks for the enthusiasm, however it feels like you are just doing AI searches and throwing the findings here without testing or consideration of the project and its constraints. Using Playwright (like your second link suggests and like several of the examples in the first link allude to) is a non starter. As identified by @si1en7 It would severely balloon this lightweight simple project into something with an incredibly complex backend with 300-400MB of additional dependencies, this goes fundamentally against Whoogle's purpose (This might work better as a separate project with different goals that I'd love to see you develop). The lightweight anti-detection methods your links suggest either appear to no longer work (likely due to the latest changes made by google) or cost money, which is not something this project will fund, and most end users will find another free project before spending money. If you find something that you have tested to actually work, I invite you to share it along with your results. (Better yet, to create a PR with the required changes so that all can share in your discovery) ------ For everyone else, I'm going to try tonight and make a stopgap with a version that leverages leta results temporarily while I investigate how SearXNG maintains their google searches and see if it's feasible to implement.
Author
Owner

@Don-Swanson commented on GitHub (Oct 3, 2025):

Hey All,
New Beta Release (AKA Stopgap) is in the pipeline, builds are happening now, and should be done soon.
https://github.com/benbusby/whoogle-search/releases/tag/v1.1.0-beta

<!-- gh-comment-id:3367560025 --> @Don-Swanson commented on GitHub (Oct 3, 2025): Hey All, New Beta Release (AKA Stopgap) is in the pipeline, builds are happening now, and should be done soon. https://github.com/benbusby/whoogle-search/releases/tag/v1.1.0-beta
Author
Owner

@inittux111 commented on GitHub (Oct 4, 2025):

@Don-Swanson Thanks so much! When will the container image for this beta be available or will it only be built once it's officially been released?

<!-- gh-comment-id:3367958653 --> @inittux111 commented on GitHub (Oct 4, 2025): @Don-Swanson Thanks so much! When will the container image for this beta be available or will it only be built once it's officially been released?
Author
Owner

@MadWalnut commented on GitHub (Oct 4, 2025):

When will the container image for this beta be available or will it only be built once it's officially been released?

You can use benbusby/whoogle-search:1.1.0-beta, see release notes.

<!-- gh-comment-id:3368118667 --> @MadWalnut commented on GitHub (Oct 4, 2025): > When will the container image for this beta be available or will it only be built once it's officially been released? You can use `benbusby/whoogle-search:1.1.0-beta`, see [release notes](https://github.com/benbusby/whoogle-search/releases/tag/v1.1.0-beta).
Author
Owner

@Don-Swanson commented on GitHub (Oct 4, 2025):

This has also been turned into a regular release.

Test this release and let us know of any CRITICAL bugs. As this is intended as a stopgap only, please do not report minor bugs related to this release—Major/Critical bugs only that must be fixed to keep the stopgap alive.

If you have genuine ideas on ways forward (no generic AI suggestions or "here's what I found in a search"), please bring them to the way forward discussion.

<!-- gh-comment-id:3368362916 --> @Don-Swanson commented on GitHub (Oct 4, 2025): This has also been turned into a regular release. Test this release and let us know of any CRITICAL bugs. As this is intended as a stopgap only, please do not report minor bugs related to this release—Major/Critical bugs only that must be fixed to keep the stopgap alive. If you have genuine ideas on ways forward (no generic AI suggestions or "here's what I found in a search"), please bring them to the [way forward discussion](https://github.com/benbusby/whoogle-search/discussions/1253).
Author
Owner

@pmonck commented on GitHub (Oct 4, 2025):

Anyone with a working UA, please just post that youve got one working so that we know to continue searching.. I tried ~30 diff text-based browsers UA and all need to UPGRADE.

LibreY still work with Google..

Is there no way for the backend to scrape Google's javascript ?

I have a working UA.

<!-- gh-comment-id:3368609165 --> @pmonck commented on GitHub (Oct 4, 2025): > Anyone with a working UA, please just post that youve got one working so that we know to continue searching.. I tried ~30 diff text-based browsers UA and all need to UPGRADE. > > LibreY still work with Google.. > > Is there no way for the backend to scrape Google's javascript ? I have a working UA.
Author
Owner

@boognish-rising commented on GitHub (Oct 5, 2025):

Anyone with a working UA, please just post that youve got one working so that we know to continue searching.. I tried ~30 diff text-based browsers UA and all need to UPGRADE.
LibreY still work with Google..
Is there no way for the backend to scrape Google's javascript ?

I have a working UA.

bit of a head scratcher since you posted immediately following @Don-Swanson's call for folks with helpful contributions, but in case you missed it, your input would be valuable in the way forward discussion. The project needs all the help it can get atm, so thanks in advance!

PS - disregard if you've already made your way over there.

<!-- gh-comment-id:3368632262 --> @boognish-rising commented on GitHub (Oct 5, 2025): > > Anyone with a working UA, please just post that youve got one working so that we know to continue searching.. I tried ~30 diff text-based browsers UA and all need to UPGRADE. > > LibreY still work with Google.. > > Is there no way for the backend to scrape Google's javascript ? > > I have a working UA. bit of a head scratcher since you posted immediately following @Don-Swanson's call for folks with helpful contributions, but in case you missed it, your input would be valuable in the [way forward discussion](https://github.com/benbusby/whoogle-search/discussions/1253). The project needs all the help it can get atm, so thanks in advance! PS - disregard if you've already made your way over there.
Author
Owner

@Don-Swanson commented on GitHub (Oct 8, 2025):

Closing as everyone should have realized by now that their instance has stopped and the current fix.

<!-- gh-comment-id:3383524040 --> @Don-Swanson commented on GitHub (Oct 8, 2025): Closing as everyone should have realized by now that their instance has stopped and the current fix.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/whoogle-search#738
No description provided.