mirror of
https://github.com/benbusby/whoogle-search.git
synced 2026-04-25 12:15:50 +03:00
[GH-ISSUE #17] [BUG] Save config http only #11
Labels
No labels
Fixed (Pending PR Merge)
Stale
bug
enhancement
enhancement
good first issue
help wanted
keep-open
needs more info
pull-request
question
theme
unfortunate
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/whoogle-search#11
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @gripped on GitHub (May 10, 2020).
Original GitHub issue: https://github.com/benbusby/whoogle-search/issues/17
Originally assigned to: @benbusby on GitHub.
Describe the bug
I've installed this both in an AWS free instance manually and also using your heroku quick deploy
At first, in the case of AWS, saving the config settings worked because I'd opened port 80 to get ssl certs with certbot. Once I closed port 80 again the config changes won't save . No biggie I can open port 80 ! My nginx settings redirect back to https anyway.
However with heroku I start with a https page eg https://mywhoogle.herokuapp.com. Saving the config settings take me to the http page http://mywhoogle.herokuapp.com where it will stay for any subsequent searches
To Reproduce
Steps to reproduce the behavior: AWS
Steps to reproduce the behavior: Heroku
Expected behavior
AWS: allow save via https
heroku: remain on https after save if that's what we we on before saving.
Nice work :)
@benbusby commented on GitHub (May 10, 2020):
I pushed a new update that should address this issue. The config section on the main page now has an option to set the instance's root url. When you get a chance, try pulling in the new changes and make sure the app url is using https before saving your config.
@gripped commented on GitHub (May 11, 2020):
This does work to allow saving via https. If the Root URL: in the settings is changed to https://
However in the first instance, when first visiting via https:// the Root URL: is pre-populated with the http:// form of the url. I tested several times by restarting the heroku dyno.
Therefore someone not knowing they had to edit this to https:// does still get the hanging page on the AWS install (with port 80 blocked), when they click save. But on the heroku install the save will complete, and the session will be http until manually reverting to https.
So fixed for me, able to save via https, but not closing as I don't think it's fixed entirely ? A user shouldn't be getting dropped down to http because they didn't know to add an 's' to a url before saving. (not that I'll have any users, private search)
Still great work :)
@gripped commented on GitHub (May 11, 2020):
Sorry more http woes. Adding whoogle as a search engine in firefox and either using it as the default engine, or from the search choices displayed in the address bar dropdown results in an http search.
In ~/.mozilla/firefox/xxxxxxxx.default/search.json.mozlz4
Strangely editing the http's to https's in this file not working for me. Still http search from the added engine ?
All my other added search engines seem to default to https.
@benbusby commented on GitHub (May 11, 2020):
So this is a bit of a complicated issue. As far as the Flask server is aware, it's only running http (which is why the root url field is prepopulated the way it is -- this is technically how the Flask server is actually being run), even though requests are being proxied through https either through heroku, aws, etc. Another weird thing is that the past 4 out of 5 heroku instances I've spun up have handled https redirects properly, but one did not. So with the 4 that did work, POST requests to
/configroutes over http were properly forwarded to https and I never encountered any problems.Beyond what I already pushed for a partial fix, I suppose I could add a section to the README with a note about how https traffic is occasionally not properly rerouted, so ensure that the root url under config matches the protocol you'd like to use. I can't do a blanket fix for rerouting all http to https, since this would break instances that aren't being proxied. Just adding a README fix also doesn't quite feel like a good solution, so I'm still trying to think of a better alternative. One possible alternative is just using javascript to send the POST request instead of relying on just the HTML form, where it'd be easier to specify the full url (with https protocol) as the proper endpoint.
Regarding your issues with setting the default search engine, I've had issues in the past where Firefox caches the opensearch template even when I change some small aspect about it. In the past I've just removed the Whoogle search template from the list in search preferences, and sometimes it's required clearing data in Firefox before it'll accept the new template. This may or may not be the issue you're seeing, just wanted to offer what I've experienced in the past.
@gripped commented on GitHub (May 12, 2020):
Thanks for your explanation.
I think you probably should make this limitation clear in the readme. I expect most people making the effort to use a privacy enhanced google search will want to be using https all of the time.
With that in mind my own opinion is that long term you should make the app https only.
With the case of heroru that should be seamless as a certificate is in place automatically. But a bit harder to automatically setup with the other deployment methods. I get that you want this to be easy to install. Thinking outside the box maybe caddy instead of flask ?
Anyhow I'm glad I came across this project on reddit, as it's inspired me to finally also get my own private searx instance going. But I intend to use both going forward . I like the way whoogle gives me my own frontend to results very similar to what I'd get going straight through google.com
@benbusby commented on GitHub (May 12, 2020):
Agreed. I have a couple ideas for configuring https enforcement at runtime, but need to try them out and see how well they integrate with the rest of the codebase. Will let you know once I reach a proper solution.
@benbusby commented on GitHub (May 15, 2020):
Fixed in #48 (pending merge). HTTPS is now enforced in all Docker instances unless otherwise specified, and there's an easy
--https-onlyflag for instances running through pip/pipx. Let me know if you want to take a look or not, otherwise I'll probably merge this in the next few hours. Thanks again for the discussion, it's helpful to get feedback on stuff like this.@gripped commented on GitHub (May 15, 2020):
Testing it now. Back to you soon
@gripped commented on GitHub (May 15, 2020):
Sorry multiple issues.
In the PR the readme says add --https-only to the command line
This has the effect of causing config.json to get stored in whoogle-search/--https-only/static/ instead of whoogle-search/app/static/ (more on this in a bit)
Tried instead
But still wasn't working
And maybe I've lost the plot but isn't the logic wrong here
route.py line 24
Shouldn't that be True ? I've tried both though and no effect with either
( --debug set by default in whoogle-search )
back to config.json
It's probable I'm missing something but where multiple users are using the same instance aren't they overwriting each others settings ?
Maybe I've done something wrong ?
is the code I was testing.
@benbusby commented on GitHub (May 15, 2020):
Regarding the logic in routes on line 24, it's falling back to false if that environment variable isn't set (which is what it should default to, only the Dockerfile build arg should ever set that to True).
Also in the readme changes, I should update that to be a bit more clear. The
--https-onlyflag should only be applied directly to the run command for the Flask app, not the executable itself. I will make a note to update the executable name to reduce some confusion -- when it's installed through pip, the command to run the server is the same as the executable, but they function differently. Just unfortunate naming conventions on my part, but easy to fix.Regarding the use case of multiple users using the same instance, that is an issue, but the (currently) intended use case is only ever supposed to be individually run instances only. With the introduction of basic auth in #51, I could start to visit the idea of separating config directories to be dependent on the active user, but otherwise the project would need a not-insignificant redesign to be used by multiple people with separate config files.
For testing the feature, I've been using the pip installed method of running the app, but the same effect can be accomplished by updating the executable to run with the
--https-onlyflag at the end of thepython3 -um app ...command.@gripped commented on GitHub (May 15, 2020):
I'll take another look tomorrow. I was tired when testing before.
Though I believe I did also try --https-only at the end of python3 -um app ...
The way it's written up in the readme is a definite red herring though.
Tried lots of things and it didn't seem to work.
Deployed on heroku as well.
This in the build log implies the env was setup correctly I think ?
But https was not forced. Same behaviour as before.
@benbusby commented on GitHub (May 15, 2020):
Okay so (presumably) final solution, which is all on
masternow:runto avoid confusion with the pip installedwhoogle-searchexecutableheroku-app) that by default enforces HTTPS, since the deployments there are guaranteed to always support HTTPS.I'm going to close this issue for now, but we can keep discussing here if you run into any issues with the new changes. I believe this is now a good compromise for giving the option to enforce HTTPS for some deployments, and not completely break other deployments that rely on HTTPS through a reverse proxy setup (where Flask should remain in HTTP anyways).
Just to make sure, I deployed two different Heroku versions of the app and manually verified that all routes are being served over HTTPS only, and that the opensearch template is correctly populated with the HTTPS version of the url.