mirror of
https://github.com/flyimg/flyimg.git
synced 2026-04-25 09:45:50 +03:00
[GH-ISSUE #465] flyimg apache reverse proxy crashing #174
Labels
No labels
Docs
Docs
Docs
Security
UnitTest
bug
dependencies
duplicate
enhancement
enhancement
enhancement
hacktoberfest
help wanted
invalid
pull-request
question
stale
version 1
version 2
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/flyimg#174
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @srsquare on GitHub (Jun 4, 2024).
Original GitHub issue: https://github.com/flyimg/flyimg/issues/465
Originally assigned to: @sadok-f on GitHub.
I have flyimg running on my main server, via apache reverse proxy. It used to run fine, but now after an hour or two apache crashes. The server is running debian 11, has 8 vCPUs, 30gb of ram, and 800gb drive. The problem appears that memory is going out of bounds and thus apache crashes. If flyimg docker is stopped, apache does not crash.
This is the apache reverse proxy config:
AllowEncodedSlashes NoDecode <Location /i> ProxyPass http://127.0.0.1:3380 nocanon </Location>flyimg is the most current version, I just updated it in the last few days. The only custom change, is I had to alter the s3 integration to work with Wasabi, but that also has been working fine for years.
I have attached a screenshot of btop, so you can see main server health, at the bottom. This is without flyimg running.
Mostly, I use it to set a width and quality, so nothing overly fancy. This kind of thing: /upload/w_800,q_80/https://
So far, I have had no luck tracing and resolving the issue. Hoping someone here can help me. Let me know if there is more info that might be helpful, etc. I am not sure how to check flyimg to determine the amount of traffic it's handling, and ensure no one is using it besides me, since I never set up the domain restrictions.
This is the parameters file:
@sadok-f commented on GitHub (Jun 4, 2024):
Thank you @srsquare for opening this issue.
can you check the Flyimg container logs? can you paste here the outputs especially when Apache is crashing.
is Flyimg container continue to work normally? after you restart Apache?
also what command are you using to run Flyimg (Im interested to see the mount volume used)
Thank you !
@srsquare commented on GitHub (Jun 4, 2024):
Normally I have to restart both apache and flyimg, however if I just restart them they crash with in hours again. If I reboot the server, it will take longer before crashing. I will have to start flyimg while I am working on the flipside, and capture the apache error logs for you then. I am not sure how to get the logs for you from flyimg, other than what's in portainer under log. Not sure if thats what you want.
docker run -itd -p 3380:80 --name flyimg -v /var/www/docker/flyimg/parameters.yml flyimg/flyimg
@sadok-f commented on GitHub (Jun 4, 2024):
can you check your Apache access logs, to see how frequent the requests you are getting for Flyimg?
for Flyimg contain, when it crashes you can get the contain logs via
docker logs flyimg@srsquare commented on GitHub (Jun 4, 2024):
Thanks for the help and guidance. At the moment I dont have a way to check usage for flyimg, but I could turn on awstats for it and see if that helps.
when I run this command, I get a ton of the below errors.
docker logs flyimg@sadok-f commented on GitHub (Jun 4, 2024):
interesting,
which arch platform is your server? and which flyimg version are you running?
the config file you're using is a bit outdated. can you update it to the latest version?
EDIT: I suspect that the issue is coming from AVIF format generation, to disable it can you change the setting enable_avif to false?
enable_avif: false@srsquare commented on GitHub (Jun 4, 2024):
I pulled the latest flyimg container on 2024-05-31 so it should be the most current, I would assume.
Yes, I will update the parameter file tomorrow night. I work nights, so I really should be heading to bed before the sun comes up in the next 30 - 60 mins lol. Will provide farther updates once I do that and run it to see if I can get any other log data that will help. I also turned on awstats, so will be interesting to see what the stats are.
@srsquare commented on GitHub (Jun 4, 2024):
If it helps the server is from Contabo with 8 vCPUs, 30gb of ram, and 800gb drive.
@srsquare commented on GitHub (Jun 5, 2024):
As an update, I have updated the parameter yml file and have it running again. When/if it crashes I will try and capture what logs I can. Maybe I will be lucky and its just because the parameter file is out of date.
I did check and still getting errors via dockers log flyimg
Edit: so far images are working fine, and everything loading faster than it has been in a while. The only error im seeing is the one above.
@sadok-f commented on GitHub (Jun 5, 2024):
It would be great if you track down the source image that caused this error?
idealy with the full parameters such as
/uploads/w_150,q_80/IMAGE_SOURCEif not, maybe you can download the image
/var/www/html/var/tmp/original-fd86ef9a8f9a26a531557252ff6067b6from the container and attach it here.@srsquare commented on GitHub (Jun 5, 2024):
That path is within docker container, so I did
docker exec -it flyimg bashAs you can see, the file is empty. I also did a cat on it, and there was nothing.
Is there a way to have flyimg return the whole URL of the image causing the error?
So far its still running, everything working, and has not crashed. I have been watching access log off and on and seems like it avgs between maybe 2 to 8 a min to only a few over 10 mins... just depending on traffic on my sites, none of which get a ton of traffic.
I see in the error is still referencing avif even though I set avif to false in the parameter file, here is that part of the settings.
@srsquare commented on GitHub (Jun 5, 2024):
I have tracked down a few issues, one isnt really a bug on flyimg side. For some reason when one of my assistants created content on my site they named it like this
cil_ - Copy 2.pngWhich is causing fly image to do this
cil_): Failed to open stream: HTTP request failed!as you will notice incomplete url, which ill have my VA work on fixing, but sharing encase you want to add logic to handle that in some way.
@srsquare commented on GitHub (Jun 5, 2024):
I am seeing this error, and I guess ill share the whole url since its just a site about my dog lol... the logo is valid and loading on the site, so no idea why this error is coming up.
Message: Error occurred while trying to read the file Url : https:/wacky3leggedjack.com/assets/WackyEagleTech_400px.png, error : 2,fopen(https:/wacky3leggedjack.com/assets/WackyEagleTech_400px.png): Failed to open stream: No such file or directory,/var/www/html/src/Core/Entity/Image/InputImage.php,113@sadok-f commented on GitHub (Jun 5, 2024):
is it because of the slash
/missing afterhttps:/?shouldn't be
https://wacky3leggedjack.com/assets/WackyEagleTech_400px.pnginstead ofhttps:/wacky3leggedjack.com/assets/WackyEagleTech_400px.png?@srsquare commented on GitHub (Jun 5, 2024):
Good catch, but no idea where its coming from with only 1 slash. I just tried greping most of my sites code with no luck.
@sadok-f commented on GitHub (Jun 5, 2024):
also I think you're not mounting correctly the config file.
as you mentioned here:
but it should be like this:
@srsquare commented on GitHub (Jun 5, 2024):
hmmm surprised I didnt catch that. I see endpoint has been added for s3 storage. Is there a way I can use that to do something like this, without having to modify the code now?
https://s3.'.$s3Params['region'].'.wasabisys.com'@sadok-f commented on GitHub (Jun 5, 2024):
can you try out in
config/parameters.ymlfile to addendpointoption toaws_s3, like this:@srsquare commented on GitHub (Jun 5, 2024):
now when I test a gallery page I get this php error
@srsquare commented on GitHub (Jun 5, 2024):
I made some tweaks to php settings in www.conf and got that under better control. At the moment server cpu, memory, etc. all looks good. Will let you know if I have more issues, over the next few days. Maybe, crossing fingers, it will behave better now.
@sadok-f commented on GitHub (Jun 5, 2024):
could you please put here the changes you did in the www.conf file?
when did you start getting the pool www error message?
can you share some details about your gallery page? like how many images per page, in which calling Flyimg endpoint.
thanks!
@srsquare commented on GitHub (Jun 5, 2024):
The gallery page is https://wacky3leggedjack.com/gallery.php, its just a site from when my dog got attacked and lost his leg. If I counted right, it's about 70 images. The error messages came up after the page started loading, on force reload so it wouldn't use browser cache. The changes were to the pm settings.
(note: I had to install nano, use whichever cli editor you like)
I should note I still get some pm errors, but I choose a balance, as the gallery page is one of the ones I have with the most images on it. So its a good load test and check for errors.
@srsquare commented on GitHub (Jun 5, 2024):
currently btop shows my server as being pretty happy
@sadok-f commented on GitHub (Jun 5, 2024):
interesting, thanks for sharing the details.
please keep me updated about the performance.
@sadok-f commented on GitHub (Jun 12, 2024):
Hi @srsquare
any update on this issue? can we close it?
Thanks!
@srsquare commented on GitHub (Jun 13, 2024):
I am still having issues with Apache/php crashing but not sure if that's flyimg, might be load, might be issues with other apps, etc.
@sadok-f commented on GitHub (Jun 17, 2024):
sorry I can't help here without knowing the whole setup details as you seem to have more applications running on the same server.
@srsquare commented on GitHub (Jun 17, 2024):
No worries, It's worked for the last 2 or 3 days without crashing, that's a huge improvement. I think elasticsearch was also causing issues. Yes, I have a number of different sites, apps, and tools running on the server, complicating things.
@sadok-f commented on GitHub (Jun 18, 2024):
Closing the issue, as it seems not related to Flyimg.
Thanks for reporting it anyway.