[GH-ISSUE #1050] [BUG] 500 Internal Server Error #642

New issue

Closed

opened 2026-02-25 20:36:12 +03:00 by kerem · 1 comment

kerem commented

2026-02-25 20:36:12 +03:00

Owner

Originally created by @seirdotexe on GitHub (Aug 10, 2023).
Original GitHub issue: https://github.com/benbusby/whoogle-search/issues/1050

Describe the bug
When you search for Mathias Normann, you get error code 500. It doesn't happen when you search for Mathias Normannaaa.

To Reproduce
Steps to reproduce the behavior:

Search on your Whoogle instance for Mathias Normann.

Deployment Method

Heroku (one-click deploy)
Docker
run executable
pip/pipx
Other: [describe setup]

Version of Whoogle Search

Latest build from [source] (i.e. GitHub, Docker Hub, pip, etc)
Version 0.8.2
Not sure

Desktop (please complete the following information):

OS: Windows 11 Pro
Browser: FireFox Developer Edition
Version 117.0b5 (64-bit)

Additional context

08/10/2023 4:21:21 PM
ERROR:app:Exception on /search [POST]
08/10/2023 4:21:21 PM
Traceback (most recent call last):
08/10/2023 4:21:21 PM
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app
08/10/2023 4:21:21 PM
    response = self.full_dispatch_request()
08/10/2023 4:21:21 PM
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
08/10/2023 4:21:21 PM
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request
08/10/2023 4:21:21 PM
    rv = self.handle_user_exception(e)
08/10/2023 4:21:21 PM
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
08/10/2023 4:21:21 PM
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request
08/10/2023 4:21:21 PM
    rv = self.dispatch_request()
08/10/2023 4:21:21 PM
         ^^^^^^^^^^^^^^^^^^^^^^^
08/10/2023 4:21:21 PM
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request
08/10/2023 4:21:21 PM
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
08/10/2023 4:21:21 PM
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
08/10/2023 4:21:21 PM
  File "/whoogle/app/routes.py", line 122, in decorated
08/10/2023 4:21:21 PM
    return f(*args, **kwargs)
08/10/2023 4:21:21 PM
           ^^^^^^^^^^^^^^^^^^
08/10/2023 4:21:21 PM
  File "/whoogle/app/routes.py", line 57, in decorated
08/10/2023 4:21:21 PM
    return f(*args, **kwargs)
08/10/2023 4:21:21 PM
           ^^^^^^^^^^^^^^^^^^
08/10/2023 4:21:21 PM
  File "/whoogle/app/routes.py", line 314, in search
08/10/2023 4:21:21 PM
    response = search_util.generate_response()
08/10/2023 4:21:21 PM
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
08/10/2023 4:21:21 PM
  File "/whoogle/app/utils/search.py", line 165, in generate_response
08/10/2023 4:21:21 PM
    formatted_results = content_filter.clean(html_soup)
08/10/2023 4:21:21 PM
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
08/10/2023 4:21:21 PM
  File "/whoogle/app/filter.py", line 170, in clean
08/10/2023 4:21:21 PM
    self.update_link(link)
08/10/2023 4:21:21 PM
    ^^^^^^^^^^^^^^^^^^^^^^
08/10/2023 4:21:21 PM
  File "/whoogle/app/filter.py", line 496, in update_link
08/10/2023 4:21:21 PM
    q = extract_q(result_link.query, href)
08/10/2023 4:21:21 PM
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
08/10/2023 4:21:21 PM
  File "/whoogle/app/filter.py", line 47, in extract_q
08/10/2023 4:21:21 PM
    return parse_qs(q_str)['q'][0] if ('&q=' in href or '?q=' in href) else ''
08/10/2023 4:21:21 PM
           ~~~~~~~~~~~~~~~^^^^^
08/10/2023 4:21:21 PM
KeyError: 'q'

Originally created by @seirdotexe on GitHub (Aug 10, 2023). Original GitHub issue: https://github.com/benbusby/whoogle-search/issues/1050 **Describe the bug** When you search for **Mathias Normann**, you get error code 500. It doesn't happen when you search for **Mathias Normannaaa**. **To Reproduce** Steps to reproduce the behavior: 1. Search on your Whoogle instance for **Mathias Normann**. **Deployment Method** - [ ] Heroku (one-click deploy) - [x] Docker - [ ] `run` executable - [ ] pip/pipx - [ ] Other: [describe setup] **Version of Whoogle Search** - [x] Latest build from [source] (i.e. GitHub, Docker Hub, pip, etc) - [ ] Version 0.8.2 - [ ] Not sure **Desktop (please complete the following information):** - OS: Windows 11 Pro - Browser: FireFox Developer Edition - Version 117.0b5 (64-bit) **Additional context** ``` 08/10/2023 4:21:21 PM ERROR:app:Exception on /search [POST] 08/10/2023 4:21:21 PM Traceback (most recent call last): 08/10/2023 4:21:21 PM File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app 08/10/2023 4:21:21 PM response = self.full_dispatch_request() 08/10/2023 4:21:21 PM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 08/10/2023 4:21:21 PM File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request 08/10/2023 4:21:21 PM rv = self.handle_user_exception(e) 08/10/2023 4:21:21 PM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 08/10/2023 4:21:21 PM File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request 08/10/2023 4:21:21 PM rv = self.dispatch_request() 08/10/2023 4:21:21 PM ^^^^^^^^^^^^^^^^^^^^^^^ 08/10/2023 4:21:21 PM File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request 08/10/2023 4:21:21 PM return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) 08/10/2023 4:21:21 PM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 08/10/2023 4:21:21 PM File "/whoogle/app/routes.py", line 122, in decorated 08/10/2023 4:21:21 PM return f(*args, **kwargs) 08/10/2023 4:21:21 PM ^^^^^^^^^^^^^^^^^^ 08/10/2023 4:21:21 PM File "/whoogle/app/routes.py", line 57, in decorated 08/10/2023 4:21:21 PM return f(*args, **kwargs) 08/10/2023 4:21:21 PM ^^^^^^^^^^^^^^^^^^ 08/10/2023 4:21:21 PM File "/whoogle/app/routes.py", line 314, in search 08/10/2023 4:21:21 PM response = search_util.generate_response() 08/10/2023 4:21:21 PM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 08/10/2023 4:21:21 PM File "/whoogle/app/utils/search.py", line 165, in generate_response 08/10/2023 4:21:21 PM formatted_results = content_filter.clean(html_soup) 08/10/2023 4:21:21 PM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 08/10/2023 4:21:21 PM File "/whoogle/app/filter.py", line 170, in clean 08/10/2023 4:21:21 PM self.update_link(link) 08/10/2023 4:21:21 PM ^^^^^^^^^^^^^^^^^^^^^^ 08/10/2023 4:21:21 PM File "/whoogle/app/filter.py", line 496, in update_link 08/10/2023 4:21:21 PM q = extract_q(result_link.query, href) 08/10/2023 4:21:21 PM ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 08/10/2023 4:21:21 PM File "/whoogle/app/filter.py", line 47, in extract_q 08/10/2023 4:21:21 PM return parse_qs(q_str)['q'][0] if ('&q=' in href or '?q=' in href) else '' 08/10/2023 4:21:21 PM ~~~~~~~~~~~~~~~^^^^^ 08/10/2023 4:21:21 PM KeyError: 'q' ```

kerem

2026-02-25 20:36:12 +03:00

closed this issue
added the
bug
label

kerem commented

2026-02-25 20:36:12 +03:00

Author

Owner

@ahmad-alkadri commented on GitHub (Aug 13, 2023):

Confirmed getting the same bug on my docker instance.

It seems that the problem's coming from the parsing of the query. Curious if there's other instances of search queries producing this bug.

EDIT: Found the problem. This function:

github.com/benbusby/whoogle-search@18fc14dc5b/app/filter.py (L35-L47)

will return the first element of an array in ['q'] key of the result of the parse_qs function. The problem was there could be an instance where the value would be blank. And the parse_qs is from the urllib module, which is a built-in python module.

Luckily according to the doc of the function we could keep the blank value instead of throwing it. I'll try it and try to make a PR asap if successful.

@ahmad-alkadri commented on GitHub (Aug 13, 2023): Confirmed getting the same bug on my docker instance. It seems that the problem's coming from the parsing of the query. Curious if there's other instances of search queries producing this bug. EDIT: Found the problem. This function: https://github.com/benbusby/whoogle-search/blob/18fc14dc5b6d898975d1e83134441c9976fc4f29/app/filter.py#L35-L47 will return the first element of an array in `['q']` key of the result of the `parse_qs` function. The problem was there could be an instance where the value would be blank. And the `parse_qs` is from the `urllib` module, which is a built-in python module. Luckily according to the doc of the function we could keep the blank value instead of throwing it. I'll try it and try to make a PR asap if successful.

kerem referenced this issue

2026-02-25 20:37:10 +03:00

[PR #654] [MERGED] Fix 'collapse_sections' for 'MINIMAL_MODE' #937