mirror of
https://github.com/benbusby/whoogle-search.git
synced 2026-04-25 12:15:50 +03:00
[GH-ISSUE #42] [BUG] Searching results in page stating "Our systems have detected unusual traffic from your computer network." #26
Labels
No labels
Fixed (Pending PR Merge)
Stale
bug
enhancement
enhancement
good first issue
help wanted
keep-open
needs more info
pull-request
question
theme
unfortunate
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/whoogle-search#26
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @timespacedecay on GitHub (May 13, 2020).
Original GitHub issue: https://github.com/benbusby/whoogle-search/issues/42
Describe the bug
Searching results in the following message:
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Google results.
Desktop (please complete the following information):
Additional info:
Going to Google.com proper and searching "namemesh" provides expected results page.
Whoogle is deployed via Docker on my home network (on a QNAP); the Whoogle search and Google search originate from the same WAN IP, though different LAN IP. So it does not appear to be a simple flag against my WAN IP as my desktop PC can make valid google.com searches.
To be clear, this error occurs with any search query now, not just "namemesh".
@benbusby commented on GitHub (May 13, 2020):
I've noticed that Google treats non-javascript searches (
gbv=1) with more suspicion than regular searches, so that could be the reason why you're seeing the "unusual traffic" page through Whoogle and not through regular Google searches.That being said, I have a similar setup and haven't encountered this yet, except for when I'm on a VPN (in which case I get the same error if I'm using a relatively popular VPN). I'll try looking into this a bit more though, it'll become especially pertinent when adding in Tor routing as a feature, as those addresses are universally flagged it seems.
@apmechev commented on GitHub (May 15, 2020):
I have the same issue on a VPS (Hetzner). When running
curl -sA "Chrome"on the same host, it returns the .html file correctly. Also search works in Firefox@esturniolo commented on GitHub (May 18, 2020):
Same here. With docker container on OVH VPS and fresh install. I can't do any search, it always shows that "...unusual traffic..." and bla bla.
I've another web services work in the same VPS with no issues.
@Chluz commented on GitHub (May 21, 2020):
Same for me, also on OVH. I don't get any issues when use the same machine and external IP as a sock proxy
@benbusby commented on GitHub (May 21, 2020):
Can anyone verify if running a full, exact curl request from the machine to the search address is working? Running
curl -LA "Slowzilla/5.0 (X11; Finux x86_64; rv:75.0) Gecko/20100101 Squierfox/75.0" "https://google.com/search?q=test&gbv=1"should match the Whoogle request structure closely enough.If it returns the same error message, there's really not much I can do to personally debug this, since that indicates that Google has somehow managed to flag the machine either by its address or some combo of Whoogle request + machine address. Normal google queries would still run fine in this case, but queries through Whoogle might be problematic because of how the User Agent is formed, or the fact that non-js results are being requested, or because there aren't any google cookies available in the request header, or something else entirely.
Alternatively, it's possible that Google is specifically blocking search requests matching OVH/Hetzner/etc CIDR blocks. I don't have any setups on these services, but I don't have any issues on DigitalOcean, Linode, or Vultr.
If it doesn't return the same error message, then I'm not sure what the best next step would be. I know this will be an issue with Tor support, since the majority of Tor exit nodes are blocked by Google as well and result in this "unusual traffic" page, but I haven't thought of a good solution for that either yet.
@timespacedecay commented on GitHub (May 21, 2020):
@benbusby I just want to add that since the day I posted this originally, I have not had an issue; in fact, the issue/error from Google went away within an hour or so of posting.
@grigsby commented on GitHub (May 22, 2020):
I'm having the same problem. I tried running the curl command above, and it returns the same error "Our systems have detected unusual traffic..." Bummer. I don't think it's a VPN-related issue because I'm on a VPN and my "normal" Google searches are not being blocked. It is admittedly very hard to outwit Google!
@apmechev commented on GitHub (May 22, 2020):
I've delved a bit deeper and it looks like the
curlcommands I've been running work with 'http://google.com' but not with 'https://google.com'. The http request works even with the spoofed Agent.I tried changing the request.py to use an http url, but it looks like it still redirects to https, which causes the error. Will test on linode and migrate to my instance there if needed
@esturniolo commented on GitHub (May 22, 2020):
Oh crap...
This is from OVH.
@grigsby commented on GitHub (May 22, 2020):
Well, now it's working again! I'll keep updating here if I can find a pattern.
@esturniolo commented on GitHub (May 22, 2020):
Did you do something or just change the behaviour alone?
Where do you have the instance active?
@Chluz commented on GitHub (May 22, 2020):
I am getting the same as @esturniolo when running the curl command on my OVH server.
The funny thing is that I just setup a machine with a visual browser to go through the same IP as my OVH server with OpenVPN, i don't get the issues. I have tried through OpenVPN with chrome, both with and without user agent set to the same as the curl command, and I also ran the curl command in a terminal. I could never generate the issue.
@grigsby commented on GitHub (May 26, 2020):
Alas, it's still happening, both behind a VPN and with no VPN. The error/rejection page normally has a CAPTCHA on it that is not getting relayed by whoogle. "This page appears when Google automatically detects requests coming from your computer network which appear to be in violation of the Terms of Service. The block will expire shortly after those requests stop. In the meantime, solving the above CAPTCHA will let you continue to use our services." (I've gotten this page, for example, when using Private Internet Access, and Google detects a lot of searches coming from the same PIA IP address. Solving the CAPTCHA effectively whitelists the requests.)
Is it possible to pass the CAPTCHA through whoogle in order to get the request whitelisted?
@XNinety9 commented on GitHub (Jun 28, 2020):
Same problem here, OVH server too :(
EDIT: changed country to another one, it works now!
@benbusby commented on GitHub (Jul 2, 2020):
I'm going to close this issue for now, since I'm not sure if there really is a solution that Whoogle could implement. The CAPTCHA can't be passed through to Whoogle since CAPTCHAs enforce having the same host as the issuer. I'll keep thinking about possible solutions as I continue working on the project, but for now it's not seeming like there is a decent workaround (at least not an obvious one). I've encountered the issue a few times in the last month or so, and my solution has just been to destroy/rebuild a server (and get a new IP in the process). With a script this is pretty quick, so it hasn't bothered me too much, but still annoying nonetheless.
@naveenjohnsonv commented on GitHub (Nov 8, 2020):
I've noticed that if I gave a custom location to search config, I get flagged much more often.
@gitthangbaby commented on GitHub (Nov 13, 2020):
i get flagged so often, there's no way out. anyone hosting whoogle?
@squromiv commented on GitHub (Apr 30, 2021):
Same problem. Is there any chance to fix the issue?
@squromiv commented on GitHub (Apr 30, 2021):
I changed two private VPN`s in different countries. First attempt resulted in the issue. The second config worked in about 10 minutes, then I had the same error, without CAPTCHA. Ordinary gogle search works as it should. So whoogle-search is totally useless for me. It's a pity!
@bigshans commented on GitHub (Oct 13, 2021):
Taking a new complete head from local and put it in my code is worked for me.