starred/pictshare
1
0
Fork 0
mirror of https://github.com/HaschekSolutions/pictshare.git synced 2026-04-26 07:25:55 +03:00
Code Issues 34 Projects Releases Packages Wiki Activity

[GH-ISSUE #120] Very high CPU and swap space usage. #96

New issue
Closed
opened 2026-02-25 23:40:40 +03:00 by kerem · 8 comments
kerem commented 2026-02-25 23:40:40 +03:00
Owner
Copy link

Originally created by @dessalines on GitHub (Jun 5, 2020).
Original GitHub issue: https://github.com/HaschekSolutions/pictshare/issues/120

I'm getting really high CPU usage, and swap space being used up on my server all of the sudden from pictshare (I'm using the docker image). pictshare has about 5000 pictures, and has had no issues until now. Might be a memory leak?

7416  php-fpm        335072 kB
7415  php-fpm        334936 kB
7417  php-fpm        334376 kB
7428  php-fpm        334328 kB
7424  php-fpm        333968 kB
7426  php-fpm        231788 kB
7513  php-fpm        31904 kB
7422  php-fpm        12692 kB
7973  php-fpm        8356 kB
7396  php-fpm        3372 kB

I just checked and no new files are being added to the pictshare folder, so no uploads are taking place either.

Originally created by @dessalines on GitHub (Jun 5, 2020). Original GitHub issue: https://github.com/HaschekSolutions/pictshare/issues/120 I'm getting really high CPU usage, and swap space being used up on my server all of the sudden from pictshare (I'm using the docker image). pictshare has about 5000 pictures, and has had no issues until now. Might be a memory leak? ``` 7416 php-fpm 335072 kB 7415 php-fpm 334936 kB 7417 php-fpm 334376 kB 7428 php-fpm 334328 kB 7424 php-fpm 333968 kB 7426 php-fpm 231788 kB 7513 php-fpm 31904 kB 7422 php-fpm 12692 kB 7973 php-fpm 8356 kB 7396 php-fpm 3372 kB ``` I just checked and no new files are being added to the pictshare folder, so no uploads are taking place either.
kerem closed this issue 2026-02-25 23:40:40 +03:00
kerem commented 2026-02-25 23:40:40 +03:00
Author
Owner
Copy link

@Nutomic commented on GitHub (Jun 5, 2020):

Some interesting log lines (I can send you the full log by email).

WARNING: [pool www] child 1456, script '/usr/share/nginx/html/index.php' (request: "GET /index.php?url
=/webp/192/zlr3vh.jpg") execution timed out (618.597039 sec), terminating
[error] 261#261: *161002 upstream timed out (110: Operation timed out) while reading response header from
upstream, client: 172.19.0.1, server: _, request: "GET /webp/96/ed9ej7.jpg HTTP/1.0", upstream: "fastcgi://127.0.0.1:9000", host: "dev.lemmy.ml", referr
er: "https://dev.lemmy.ml/"
WARNING: [pool www] server reached pm.max_children setting (9), consider raising it

Edit: log sent to the contact email on https://haschek.solutions/

<!-- gh-comment-id:639566102 --> @Nutomic commented on GitHub (Jun 5, 2020): Some interesting log lines (I can send you the full log by email). ``` WARNING: [pool www] child 1456, script '/usr/share/nginx/html/index.php' (request: "GET /index.php?url =/webp/192/zlr3vh.jpg") execution timed out (618.597039 sec), terminating [error] 261#261: *161002 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 172.19.0.1, server: _, request: "GET /webp/96/ed9ej7.jpg HTTP/1.0", upstream: "fastcgi://127.0.0.1:9000", host: "dev.lemmy.ml", referr er: "https://dev.lemmy.ml/" WARNING: [pool www] server reached pm.max_children setting (9), consider raising it ``` Edit: log sent to the contact email on https://haschek.solutions/
kerem commented 2026-02-25 23:40:40 +03:00
Author
Owner
Copy link

@dessalines commented on GitHub (Jun 5, 2020):

Here's a stackoverflow answer for a fix: https://serverfault.com/questions/479443/php5-fpm-server-reached-pm-max-children

But I don't know if that's the issue, since it seems like the webp conversion might be leaking memory or something.

<!-- gh-comment-id:639568524 --> @dessalines commented on GitHub (Jun 5, 2020): Here's a stackoverflow answer for a fix: https://serverfault.com/questions/479443/php5-fpm-server-reached-pm-max-children But I don't know if that's the issue, since it seems like the webp conversion might be leaking memory or something.
kerem commented 2026-02-25 23:40:40 +03:00
Author
Owner
Copy link

@dessalines commented on GitHub (Jun 5, 2020):

Some error messages:

^[[36mpictshare_1  |^[[0m 172.19.0.1 - - [05/Jun/2020:14:16:36 +0000] "GET /webp/192/zlr3vh.jpg HTTP/1.0" 504 569 "https://dev.lemmy.ml/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36 OPR/68.0.3618.125"
^[[36mpictshare_1  |^[[0m [05-Jun-2020 14:26:01] WARNING: [pool www] child 277 said into stderr: "NOTICE: PHP message: PHP Fatal erro
r:  gd-webp encoding failed in /usr/share/nginx/html/content-controllers/image/image.controller.php on line 256"
^[[36mpictshare_1  |^[[0m [05-Jun-2020 14:27:26] WARNING: [pool www] child 276, script '/usr/share/nginx/html/index.php' (request: "GET /index.php?url=/webp/192/zlr3vh.jpg") execution timed out (713.932975 sec), terminating
^[[36mpictshare_1  |^[[0m [05-Jun-2020 14:27:27] WARNING: [pool www] child 275, script '/usr/share/nginx/html/index.php' (request: "GET /index.php?url=/192/zlr3vh.jpg") execution timed out (713.929932 sec), terminating

^[[36mpictshare_1  |^[[0m 2020/06/05 14:31:06 [error] 261#261: *241 upstream timed out (110: Operation timed out) while reading respo
nse header from upstream, client: 172.19.0.1, server: _, request: "GET /webp/192/zlr3vh.jpg HTTP/1.0", upstream: "fastcgi://127.0.0.1:9000", host: "dev.lemmy.ml", referrer: "https://dev.lemmy.ml/"

LOTS of these ones:

^[[36mpictshare_1  |^[[0m 2020/06/05 12:32:08 [error] 261#261: *155304 FastCGI sent in stderr: "PHP message: PHP Warning:  imagecreatefromstring(): Empty string or invalid image in /usr/share/nginx/html/content-controllers/image/image.controller.php on line 225PHP message: PHP Warning:  imagesx() expects parameter 1 to be resource, bool given in /usr/share/nginx/html/content-controllers/image/resize.php on line 82PHP message: PHP Warning:  imagesy() expects parameter 1 to be resource, bool given in /usr/share/nginx/html/content-controllers/image/resize.php on line 83PHP message: PHP Warning:  Division by zero in /usr/share/nginx/html/content-controllers/image/resize.php on line 99PHP message: PHP Warning:  imagecreatetruecolor() expects parameter 2 to be int, float given in /usr/share/nginx/html/content-controllers/image/resize.php on line 104PHP message: PHP Warning:  imagecolorstotal() expects parameter 1 to be resource, bool given in /usr/share/nginx/html/content-controllers/image/resize.php on line 106PHP message: PHP Warning:  imagefill() expects parameter 1 to be resource, null given in /usr/share/nginx/html/content-controllers/image/resize.php on line 113PHP message: PHP Warning:  imagesavealpha() expects parameter 1 to be resource, null given in /usr/share/nginx/html/content-controllers/image/resize.php on line 114PHP message: PHP Warning:  imagealphablending() expects parameter 1 to be resource, null given in /usr/share/nginx/html/content-controllers/image/resize.php on line 115PHP message: PHP Warning:  imagecopyresampled() expects parameter 1 to be resource, null given in /usr/share/nginx/html/content-controllers/image/resize.php on line 117PHP message: PHP Warning:  imagewebp() expects parameter 1 to be resource, null given in /usr/share/nginx/html/content-controllers/image/image.controller.php on line 256PHP message: PHP Warning:  filemtime(): stat failed for /usr/share/nginx/html/data/y595n6.jpg/4a26594eabeeb128e09d7cf85cdbfb6b_y595n6.jpg in /usr/share/nginx/html/content-controllers/image/image.controller.php on line 216PHP
<!-- gh-comment-id:639578925 --> @dessalines commented on GitHub (Jun 5, 2020): Some error messages: ``` ^[[36mpictshare_1 |^[[0m 172.19.0.1 - - [05/Jun/2020:14:16:36 +0000] "GET /webp/192/zlr3vh.jpg HTTP/1.0" 504 569 "https://dev.lemmy.ml/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36 OPR/68.0.3618.125" ^[[36mpictshare_1 |^[[0m [05-Jun-2020 14:26:01] WARNING: [pool www] child 277 said into stderr: "NOTICE: PHP message: PHP Fatal erro r: gd-webp encoding failed in /usr/share/nginx/html/content-controllers/image/image.controller.php on line 256" ^[[36mpictshare_1 |^[[0m [05-Jun-2020 14:27:26] WARNING: [pool www] child 276, script '/usr/share/nginx/html/index.php' (request: "GET /index.php?url=/webp/192/zlr3vh.jpg") execution timed out (713.932975 sec), terminating ^[[36mpictshare_1 |^[[0m [05-Jun-2020 14:27:27] WARNING: [pool www] child 275, script '/usr/share/nginx/html/index.php' (request: "GET /index.php?url=/192/zlr3vh.jpg") execution timed out (713.929932 sec), terminating ``` ``` ^[[36mpictshare_1 |^[[0m 2020/06/05 14:31:06 [error] 261#261: *241 upstream timed out (110: Operation timed out) while reading respo nse header from upstream, client: 172.19.0.1, server: _, request: "GET /webp/192/zlr3vh.jpg HTTP/1.0", upstream: "fastcgi://127.0.0.1:9000", host: "dev.lemmy.ml", referrer: "https://dev.lemmy.ml/" ``` LOTS of these ones: ``` ^[[36mpictshare_1 |^[[0m 2020/06/05 12:32:08 [error] 261#261: *155304 FastCGI sent in stderr: "PHP message: PHP Warning: imagecreatefromstring(): Empty string or invalid image in /usr/share/nginx/html/content-controllers/image/image.controller.php on line 225PHP message: PHP Warning: imagesx() expects parameter 1 to be resource, bool given in /usr/share/nginx/html/content-controllers/image/resize.php on line 82PHP message: PHP Warning: imagesy() expects parameter 1 to be resource, bool given in /usr/share/nginx/html/content-controllers/image/resize.php on line 83PHP message: PHP Warning: Division by zero in /usr/share/nginx/html/content-controllers/image/resize.php on line 99PHP message: PHP Warning: imagecreatetruecolor() expects parameter 2 to be int, float given in /usr/share/nginx/html/content-controllers/image/resize.php on line 104PHP message: PHP Warning: imagecolorstotal() expects parameter 1 to be resource, bool given in /usr/share/nginx/html/content-controllers/image/resize.php on line 106PHP message: PHP Warning: imagefill() expects parameter 1 to be resource, null given in /usr/share/nginx/html/content-controllers/image/resize.php on line 113PHP message: PHP Warning: imagesavealpha() expects parameter 1 to be resource, null given in /usr/share/nginx/html/content-controllers/image/resize.php on line 114PHP message: PHP Warning: imagealphablending() expects parameter 1 to be resource, null given in /usr/share/nginx/html/content-controllers/image/resize.php on line 115PHP message: PHP Warning: imagecopyresampled() expects parameter 1 to be resource, null given in /usr/share/nginx/html/content-controllers/image/resize.php on line 117PHP message: PHP Warning: imagewebp() expects parameter 1 to be resource, null given in /usr/share/nginx/html/content-controllers/image/image.controller.php on line 256PHP message: PHP Warning: filemtime(): stat failed for /usr/share/nginx/html/data/y595n6.jpg/4a26594eabeeb128e09d7cf85cdbfb6b_y595n6.jpg in /usr/share/nginx/html/content-controllers/image/image.controller.php on line 216PHP ```
kerem commented 2026-02-25 23:40:40 +03:00
Author
Owner
Copy link

@geek-at commented on GitHub (Jun 5, 2020):

hmm yes looks like a problem with the jpg to webp conversion. pictshare.net is running the same docker image with 6m hits a week and 30k images and I have no such issues.

Your "Operation timed out" error seems to confirm that something is running that uses a large amount of ressources. How much traffic are you seeing?

<!-- gh-comment-id:639610537 --> @geek-at commented on GitHub (Jun 5, 2020): hmm yes looks like a problem with the jpg to webp conversion. pictshare.net is running the same docker image with 6m hits a week and 30k images and I have no such issues. Your "Operation timed out" error seems to confirm that something is running that uses a large amount of ressources. How much traffic are you seeing?
kerem commented 2026-02-25 23:40:40 +03:00
Author
Owner
Copy link

@Nutomic commented on GitHub (Jun 5, 2020):

Not sure about the traffic, especially because pictshare is down now and we dont have any proper monitoring. My guess is less than 10 requests/s on average (with spikes when a user first opens the page).

<!-- gh-comment-id:639626112 --> @Nutomic commented on GitHub (Jun 5, 2020): Not sure about the traffic, especially because pictshare is down now and we dont have any proper monitoring. My guess is less than 10 requests/s on average (with spikes when a user first opens the page).
kerem commented 2026-02-25 23:40:40 +03:00
Author
Owner
Copy link

@dessalines commented on GitHub (Jun 5, 2020):

One thing, I removed a large picture, (~20 MB), and it stabilized A LOT (down from using 2GB of swap, to 20 MB of swap), but there's still a memory leak somewhere, the swap usage is still rising.

<!-- gh-comment-id:639627436 --> @dessalines commented on GitHub (Jun 5, 2020): One thing, I removed a large picture, (~20 MB), and it stabilized A LOT (down from using 2GB of swap, to 20 MB of swap), but there's still a memory leak somewhere, the swap usage is still rising.
kerem commented 2026-02-25 23:40:41 +03:00
Author
Owner
Copy link

@Nutomic commented on GitHub (Jun 5, 2020):

This image might be the one that broke it: https://commons.wikimedia.org/wiki/File:Pizigani_1367_Chart_10MB.jpg

<!-- gh-comment-id:639762830 --> @Nutomic commented on GitHub (Jun 5, 2020): This image might be the one that broke it: https://commons.wikimedia.org/wiki/File:Pizigani_1367_Chart_10MB.jpg
kerem commented 2026-02-25 23:40:41 +03:00
Author
Owner
Copy link

@geek-at commented on GitHub (Nov 24, 2021):

The reason seems to be that behind the scenes the php-gd library is mapping the images pixel by pixel (basically BMP in RAM) which blows up on larger images. Takes a over a gig of ram to convert the 10mb jpeg you linked. Until we have a better conversion rate.

I also tested cwebp on a server it took about two minutes and used half a gigabyte of RAM to convert the same photo to webp. Seems the only way to go to make it work is increased RAM and higher timeouts

<!-- gh-comment-id:978349818 --> @geek-at commented on GitHub (Nov 24, 2021): The reason seems to be that behind the scenes the php-gd library is mapping the images pixel by pixel (basically BMP in RAM) which blows up on larger images. Takes a over a gig of ram to convert the 10mb jpeg you linked. Until we have a better conversion rate. I also tested [cwebp](https://developers.google.com/speed/webp/docs/cwebp) on a server it took about two minutes and used half a gigabyte of RAM to convert the same photo to webp. Seems the only way to go to make it work is increased RAM and higher timeouts
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
dependabot/composer/src/lib/phpunit/phpunit-12.5.22
v3
v2
v3.0.7
v3.0.6
v3.0.5
v3.0.4
v3.0.3
v3.0.2
v3.0.1
v3.0.0
v2.0.3
v2.0.2
v2.0.1
v2.0.0
1.5
v1.1
1.0
Labels
Clear labels   
Feature request

   
Feature request

   
bug

   
cant reproduce

   
enhancement

   
help wanted

   
pull-request

Mirrored from GitHub Pull Request

  
wontfix
No labels
Feature request
Feature request
bug
cant reproduce
enhancement
help wanted
pull-request
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/pictshare#96
No description provided.