starred/whoogle-search
1
0
Fork 0
mirror of https://github.com/benbusby/whoogle-search.git synced 2026-04-25 12:15:50 +03:00
Code Issues 9 Projects Releases Packages Wiki Activity

[GH-ISSUE #1163] [BUG] frequent 504 behind NPM #705

New issue
Closed
opened 2026-02-25 20:36:20 +03:00 by kerem · 2 comments
kerem commented 2026-02-25 20:36:20 +03:00
Owner
Copy link

Originally created by @working-name on GitHub (Jun 26, 2024).
Original GitHub issue: https://github.com/benbusby/whoogle-search/issues/1163

Describe the bug
For some reason I find myself doing docker restart the more I use search throughout the day. The container is just sitting pretty there, no load. Nothing in NPM's logs other than timeout connecting to whoogle container. Private instance, I see nothing in NPM logs to make me think someone anyone else is hitting it outside of me.

2024/06/26 13:24:21 [error] 878044#878044: *597529 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 1.1.1.1, server: whoogle.domain, request: "GET /search?q=Lumen HTTP/2.0", upstream: "http://172.17.0.1:18084/search?q=Lumen", host: "whoogle.domain"

Container sitting pretty, not choked, all the while ignoring incoming requests. Normally PIDS are about 8 when idle, so there's a few hanging for sure.

CONTAINER ID   NAME             CPU %     MEM USAGE / LIMIT   MEM %     NET I/O        BLOCK I/O       PIDS
4be21cef5885   whoogle-search   0.15%     128.6MiB / 1GiB     12.56%    102MB / 75MB   108MB / 168kB   15

To Reproduce
Steps to reproduce the behavior:

  1. Run whoogle via docker, place it behind NPM
  2. Use as you would normally
  3. Bump into 504 errors

Deployment Method

  • Heroku (one-click deploy)
  • Docker
  • run executable
  • pip/pipx
  • Other: [describe setup]

Version of Whoogle Search

  • Latest build from [Docker Hub, benbusby/whoogle-search]
  • Version [Whoogle Search v0.8.4]

Desktop (please complete the following information):

  • OS: [Debian GNU/Linux 12 (bookworm)]
  • Browser [any]
  • Version [any]

Smartphone (please complete the following information): NOT applicable.

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
The container either recovers by itself after a while (is it single threaded and it waits for something to time out?), or it never does and I have to manually restart it.

It's not NPM, I checked logs, it passes the request along. I SSHed into the box, and ran a curl against whoogle container's locally exposed port, and it times out. It's 100% whoogle.

There's no useful errors logs in docker logs except this repeating issue with simple-theme.css:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1348, in handle_user_exception
    return self.ensure_sync(handler)(e)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/whoogle/app/routes.py", line 624, in internal_error
    return render_template(
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/templating.py", line 151, in render_template
    return _render(app, template, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/templating.py", line 132, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/whoogle/app/templates/error.html", line 8, in top-level template code
    <link rel="stylesheet" href="{{ cb_url(config.theme + '-theme.css') }}"/>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/whoogle/app/__init__.py", line 179, in <lambda>
    cb_url=lambda f: app.config['CACHE_BUSTING_MAP'][f])
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
KeyError: 'simple-theme.css'
Originally created by @working-name on GitHub (Jun 26, 2024). Original GitHub issue: https://github.com/benbusby/whoogle-search/issues/1163 **Describe the bug** For some reason I find myself doing docker restart the more I use search throughout the day. The container is just sitting pretty there, no load. Nothing in NPM's logs other than timeout connecting to whoogle container. Private instance, I see nothing in NPM logs to make me think someone anyone else is hitting it outside of me. ```js 2024/06/26 13:24:21 [error] 878044#878044: *597529 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 1.1.1.1, server: whoogle.domain, request: "GET /search?q=Lumen HTTP/2.0", upstream: "http://172.17.0.1:18084/search?q=Lumen", host: "whoogle.domain" ``` Container sitting pretty, not choked, all the while ignoring incoming requests. Normally PIDS are about 8 when idle, so there's a few hanging for sure. ```py CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 4be21cef5885 whoogle-search 0.15% 128.6MiB / 1GiB 12.56% 102MB / 75MB 108MB / 168kB 15 ``` **To Reproduce** Steps to reproduce the behavior: 1. Run whoogle via docker, place it behind NPM 2. Use as you would normally 3. Bump into 504 errors **Deployment Method** - [ ] Heroku (one-click deploy) - [x] Docker - [ ] `run` executable - [ ] pip/pipx - [ ] Other: [describe setup] **Version of Whoogle Search** - [x] Latest build from [Docker Hub, benbusby/whoogle-search] - [x] Version [Whoogle Search v0.8.4] **Desktop (please complete the following information):** - OS: [Debian GNU/Linux 12 (bookworm)] - Browser [any] - Version [any] ~~**Smartphone (please complete the following information):**~~ NOT applicable. - Device: [e.g. iPhone6] - OS: [e.g. iOS8.1] - Browser [e.g. stock browser, safari] - Version [e.g. 22] **Additional context** The container either recovers by itself after a while (is it single threaded and it waits for something to time out?), or it never does and I have to manually restart it. It's not NPM, I checked logs, it passes the request along. I SSHed into the box, and ran a curl against whoogle container's locally exposed port, and it times out. It's 100% whoogle. There's no useful errors logs in docker logs except this repeating issue with `simple-theme.css`: ```js During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1348, in handle_user_exception return self.ensure_sync(handler)(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/whoogle/app/routes.py", line 624, in internal_error return render_template( ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask/templating.py", line 151, in render_template return _render(app, template, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask/templating.py", line 132, in _render rv = template.render(context) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render self.environment.handle_exception() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception raise rewrite_traceback_stack(source=source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/whoogle/app/templates/error.html", line 8, in top-level template code <link rel="stylesheet" href="{{ cb_url(config.theme + '-theme.css') }}"/> ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/whoogle/app/__init__.py", line 179, in <lambda> cb_url=lambda f: app.config['CACHE_BUSTING_MAP'][f]) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^ KeyError: 'simple-theme.css' ```
kerem 2026-02-25 20:36:20 +03:00
  • closed this issue
  • added the
    bug
    Stale
         labels
kerem commented 2026-02-25 20:36:20 +03:00
Author
Owner
Copy link

@working-name commented on GitHub (Jun 27, 2024):

Found it dead again. This time the container says unhealthy. I just issued a killall -9 curl to kill all the health check processes that hung. Here's what's running on it:

$ docker compose exec whoogle-search ps aux
PID   USER     TIME  COMMAND
    1 whoogle   0:00 {run} /bin/sh ./run
    7 whoogle   0:00 {start-tor.sh} /bin/sh misc/tor/start-tor.sh
   11 whoogle   0:12 tor -f /etc/tor/torrc
   15 whoogle   0:59 python3 -um app --host 0.0.0.0 --port 5000
12776 whoogle   0:00 ps aux

Trying to curl from within the container results in the same behavior: timeout. Does this have to do with Tor? I don't remember enabling it.

$ docker compose exec whoogle-search curl localhost:5000 -vvv -m 2
* Host localhost:5000 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:5000...
* Immediate connect fail for ::1: Address not available
*   Trying 127.0.0.1:5000...
* Connected to localhost (127.0.0.1) port 5000
> GET / HTTP/1.1
> Host: localhost:5000
> User-Agent: curl/8.5.0
> Accept: */*
>
* Operation timed out after 2002 milliseconds with 0 bytes received
* Closing connection
curl: (28) Operation timed out after 2002 milliseconds with 0 bytes received

Okay, let's rebuild the container.

$ docker compose exec whoogle-search curl localhost:5000 -vvv -m 2; docker compose up -d --force-recreate && sleep 10 && docker compose exec whoogle-search curl localhost:5000 -vvv -m 2
* Host localhost:5000 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:5000...
* Immediate connect fail for ::1: Address not available
*   Trying 127.0.0.1:5000...
* Connected to localhost (127.0.0.1) port 5000
> GET / HTTP/1.1
> Host: localhost:5000
> User-Agent: curl/8.5.0
> Accept: */*
>
* Operation timed out after 2002 milliseconds with 0 bytes received
* Closing connection
curl: (28) Operation timed out after 2002 milliseconds with 0 bytes received
[+] Running 1/1
 ✔ Container whoogle-search  Started                                                                                                                                            20.2s
* Host localhost:5000 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:5000...
* Immediate connect fail for ::1: Address not available
*   Trying 127.0.0.1:5000...
* Connected to localhost (127.0.0.1) port 5000
> GET / HTTP/1.1
> Host: localhost:5000
> User-Agent: curl/8.5.0
> Accept: */*
>
* Operation timed out after 2002 milliseconds with 0 bytes received
* Closing connection
curl: (28) Operation timed out after 2002 milliseconds with 0 bytes received
### 10 seconds was not enough for the process to be ready...
containme@geek23:~/apps/whoogle-search$ docker compose exec whoogle-search curl localhost:5000 -vvv -m 2
* Host localhost:5000 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:5000...
* Immediate connect fail for ::1: Address not available
*   Trying 127.0.0.1:5000...
* Connected to localhost (127.0.0.1) port 5000
> GET / HTTP/1.1
> Host: localhost:5000
> User-Agent: curl/8.5.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: max-age=86400
< Content-Length: 12765
< Content-Type: text/html; charset=utf-8
< Date: Thu, 27 Jun 2024 03:21:07 GMT
< Server: waitress
< Set-Cookie: __Secure-session=.eJw1zMsKgkAUANBfibt2wEc-ElrmokDQRoU2cZ1H2ugU5kAl_nuzaXdWZ4HrU0wjaqFnSOfJCAfQzN3f7KFlf4N0WR1Q4mMBmxZSyN2waLMBc3_omrv6nvXwLrxOtqouK3rpyoDneKhPdcPCagxfNDhONMtURd092MuYntuGyZh7AhOCPjKy3blIEtm6JGERjzGIrD1Yf9UWNKw.Znzaow.JVGZZBm2hG4RsYpAPSdyjLnUzgU; Expires=Fri, 27 Jun 2025 03:21:07 GMT; Secure; HttpOnly; Path=/; SameSite=Lax
< Vary: Cookie
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
<
<html style="background: #000;">
<head>
... and so on. Normal response.
<!-- gh-comment-id:2193657632 --> @working-name commented on GitHub (Jun 27, 2024): Found it dead again. This time the container says unhealthy. I just issued a `killall -9 curl` to kill all the health check processes that hung. Here's what's running on it: ```py $ docker compose exec whoogle-search ps aux PID USER TIME COMMAND 1 whoogle 0:00 {run} /bin/sh ./run 7 whoogle 0:00 {start-tor.sh} /bin/sh misc/tor/start-tor.sh 11 whoogle 0:12 tor -f /etc/tor/torrc 15 whoogle 0:59 python3 -um app --host 0.0.0.0 --port 5000 12776 whoogle 0:00 ps aux ``` Trying to curl from within the container results in the same behavior: timeout. Does this have to do with Tor? I don't remember enabling it. ```py $ docker compose exec whoogle-search curl localhost:5000 -vvv -m 2 * Host localhost:5000 was resolved. * IPv6: ::1 * IPv4: 127.0.0.1 * Trying [::1]:5000... * Immediate connect fail for ::1: Address not available * Trying 127.0.0.1:5000... * Connected to localhost (127.0.0.1) port 5000 > GET / HTTP/1.1 > Host: localhost:5000 > User-Agent: curl/8.5.0 > Accept: */* > * Operation timed out after 2002 milliseconds with 0 bytes received * Closing connection curl: (28) Operation timed out after 2002 milliseconds with 0 bytes received ``` Okay, let's rebuild the container. ```py $ docker compose exec whoogle-search curl localhost:5000 -vvv -m 2; docker compose up -d --force-recreate && sleep 10 && docker compose exec whoogle-search curl localhost:5000 -vvv -m 2 * Host localhost:5000 was resolved. * IPv6: ::1 * IPv4: 127.0.0.1 * Trying [::1]:5000... * Immediate connect fail for ::1: Address not available * Trying 127.0.0.1:5000... * Connected to localhost (127.0.0.1) port 5000 > GET / HTTP/1.1 > Host: localhost:5000 > User-Agent: curl/8.5.0 > Accept: */* > * Operation timed out after 2002 milliseconds with 0 bytes received * Closing connection curl: (28) Operation timed out after 2002 milliseconds with 0 bytes received [+] Running 1/1 ✔ Container whoogle-search Started 20.2s * Host localhost:5000 was resolved. * IPv6: ::1 * IPv4: 127.0.0.1 * Trying [::1]:5000... * Immediate connect fail for ::1: Address not available * Trying 127.0.0.1:5000... * Connected to localhost (127.0.0.1) port 5000 > GET / HTTP/1.1 > Host: localhost:5000 > User-Agent: curl/8.5.0 > Accept: */* > * Operation timed out after 2002 milliseconds with 0 bytes received * Closing connection curl: (28) Operation timed out after 2002 milliseconds with 0 bytes received ### 10 seconds was not enough for the process to be ready... containme@geek23:~/apps/whoogle-search$ docker compose exec whoogle-search curl localhost:5000 -vvv -m 2 * Host localhost:5000 was resolved. * IPv6: ::1 * IPv4: 127.0.0.1 * Trying [::1]:5000... * Immediate connect fail for ::1: Address not available * Trying 127.0.0.1:5000... * Connected to localhost (127.0.0.1) port 5000 > GET / HTTP/1.1 > Host: localhost:5000 > User-Agent: curl/8.5.0 > Accept: */* > < HTTP/1.1 200 OK < Cache-Control: max-age=86400 < Content-Length: 12765 < Content-Type: text/html; charset=utf-8 < Date: Thu, 27 Jun 2024 03:21:07 GMT < Server: waitress < Set-Cookie: __Secure-session=.eJw1zMsKgkAUANBfibt2wEc-ElrmokDQRoU2cZ1H2ugU5kAl_nuzaXdWZ4HrU0wjaqFnSOfJCAfQzN3f7KFlf4N0WR1Q4mMBmxZSyN2waLMBc3_omrv6nvXwLrxOtqouK3rpyoDneKhPdcPCagxfNDhONMtURd092MuYntuGyZh7AhOCPjKy3blIEtm6JGERjzGIrD1Yf9UWNKw.Znzaow.JVGZZBm2hG4RsYpAPSdyjLnUzgU; Expires=Fri, 27 Jun 2025 03:21:07 GMT; Secure; HttpOnly; Path=/; SameSite=Lax < Vary: Cookie < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < <html style="background: #000;"> <head> ... and so on. Normal response. ```
kerem commented 2026-02-25 20:36:20 +03:00
Author
Owner
Copy link

@github-actions[bot] commented on GitHub (Sep 15, 2025):

This issue has been automatically marked as stale due to inactivity. If it is still valid please comment within 7 days or it will be auto-closed.

<!-- gh-comment-id:3291516711 --> @github-actions[bot] commented on GitHub (Sep 15, 2025): This issue has been automatically marked as stale due to inactivity. If it is still valid please comment within 7 days or it will be auto-closed.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
dependabot/pip/python-dotenv-1.2.2
v1.2.4
v1.2.3
v1.2.2
v1.2.1-test
v1.1.2
v1.1.1-beta
v1.1.0
v1.1.0-beta
v1.0.0-beta
v0.9.4
v0.9.3
v0.9.2
v0.9.1
v0.9.0
v0.8.4
v0.8.3
v0.8.2
v0.8.1
v0.8.0
v0.7.4
v0.7.3
v0.7.2
v0.7.1
v0.7.0
v0.6.0
v0.5.4
v0.5.3
v0.5.2
v0.5.1
v0.5.0
v0.4.1
v0.4.0
v0.3.2
v0.3.1
v0.3.0
v0.2.1
0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
v0.0.1
Labels
Clear labels   
Fixed (Pending PR Merge)

   
Stale

   
bug

   
enhancement

   
enhancement

   
good first issue

   
help wanted

   
keep-open

   
needs more info

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
theme

   
unfortunate
No labels
Fixed (Pending PR Merge)
Stale
bug
enhancement
enhancement
good first issue
help wanted
keep-open
needs more info
pull-request
question
theme
unfortunate
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/whoogle-search#705
No description provided.