[GH-ISSUE #1036] Performance with ~50k+ Albums / 650000 songs? #618

Closed
opened 2026-02-26 02:33:43 +03:00 by kerem · 10 comments
Owner

Originally created by @ghost on GitHub (Aug 2, 2019).
Original GitHub issue: https://github.com/koel/koel/issues/1036

I like the interface and all but before I jump on the train and waste my time. With how many songs/albums is this server tested? Currently using Plex and have to create a lot of different libraries in order to keep the performance scrolling through albums or artists. Search in Plex is pretty good, else I would not use it, but browsing is a problem.

Originally created by @ghost on GitHub (Aug 2, 2019). Original GitHub issue: https://github.com/koel/koel/issues/1036 I like the interface and all but before I jump on the train and waste my time. With how many songs/albums is this server tested? Currently using Plex and have to create a lot of different libraries in order to keep the performance scrolling through albums or artists. Search in Plex is pretty good, else I would not use it, but browsing is a problem.
Author
Owner

@BrookeDot commented on GitHub (Apr 19, 2020):

I am going to close out this issue as it is stale. My install currently has 5,822 songs. If anyone with a larger library would like to comment please feel free to add your experiences with Koel to this issue.

<!-- gh-comment-id:616209703 --> @BrookeDot commented on GitHub (Apr 19, 2020): I am going to close out this issue as it is stale. My install currently has 5,822 songs. If anyone with a larger library would like to comment please feel free to add your experiences with Koel to this issue.
Author
Owner

@phanan commented on GitHub (Apr 19, 2020):

Let's keep it open for easier discovery. Implementing support for huge libraries is actually something on Koel's roadmap, though I'm not sure when I'll have time for it yet.

<!-- gh-comment-id:616234376 --> @phanan commented on GitHub (Apr 19, 2020): Let's keep it open for easier discovery. Implementing support for huge libraries is actually something on Koel's roadmap, though I'm not sure when I'll have time for it yet.
Author
Owner

@livejq commented on GitHub (Jul 20, 2020):

Has anyone ever encountered this mistake?
`root@8ebc0d05f82c:/var/www/html# php artisan koel:sync
Syncing media from /music

1/5517 [░░░░░░░░░░░░░░░░░░░░░░░░░░░░]   0%

In AbstractDecoder.php line 236:

Allowed memory size of 134217728 bytes exhausted (tried to allocate 7171431 bytes)
`
and I have modified the params 'memory_limit = 128M', but not worked,can't it scan in batches?

<!-- gh-comment-id:660859410 --> @livejq commented on GitHub (Jul 20, 2020): Has anyone ever encountered this mistake? `root@8ebc0d05f82c:/var/www/html# php artisan koel:sync Syncing media from /music 1/5517 [░░░░░░░░░░░░░░░░░░░░░░░░░░░░] 0% In AbstractDecoder.php line 236: Allowed memory size of 134217728 bytes exhausted (tried to allocate 7171431 bytes) ` and I have modified the params 'memory_limit = 128M', but not worked,can't it scan in batches?
Author
Owner

@livejq commented on GitHub (Jul 20, 2020):

when i try to scan an album , it worked finally...so I knew that koel can't scan a huge music library,how depressing it is 😢

<!-- gh-comment-id:660875041 --> @livejq commented on GitHub (Jul 20, 2020): when i try to scan an album , it worked finally...so I knew that koel can't scan a huge music library,how depressing it is :cry:
Author
Owner

@phanan commented on GitHub (Jul 20, 2020):

You allocated a mere 128MB to scan 650,000 songs? I don't think it's a Koel issue 😬

<!-- gh-comment-id:660888958 --> @phanan commented on GitHub (Jul 20, 2020): You allocated a mere 128MB to scan 650,000 songs? I don't think it's a Koel issue 😬
Author
Owner

@livejq commented on GitHub (Jul 23, 2020):

oh~ '128M' is the default value. 'modified' means that I change it to '4096M' but also not worked.It's a hyzual's docker. So sorry hhh~ it's not exactly a Koel issue. They all not use the latest version for Koel. I just wanna find some solutions here. I use another one(alexphillips's docker) then all problems has gone😄~

<!-- gh-comment-id:662847559 --> @livejq commented on GitHub (Jul 23, 2020): oh~ '128M' is the default value. 'modified' means that I change it to '4096M' but also not worked.It's a hyzual's docker. So sorry hhh~ it's not exactly a Koel issue. They all not use the latest version for Koel. I just wanna find some solutions here. I use another one(alexphillips's docker) then all problems has gone:smile:~
Author
Owner

@fionera commented on GitHub (Mar 14, 2021):

I wanted to give koel another try, but because it loads all Artists, Songs and Albums, my Chrome and the Webserver crashes/is overloaded. Why doesnt it just load a basic overview and then only the requested stuff via AJAX?

<!-- gh-comment-id:798907540 --> @fionera commented on GitHub (Mar 14, 2021): I wanted to give koel another try, but because it loads all Artists, Songs and Albums, my Chrome and the Webserver crashes/is overloaded. Why doesnt it just load a basic overview and then only the requested stuff via AJAX?
Author
Owner

@phanan commented on GitHub (Mar 14, 2021):

TBH it was because Koel was built to scratch my own itch and I never had 50k
songs. But dealing with huge libraries is now on the list, just that I
don’t know when I will have time for it.

On Sun 14. Mar 2021 at 14:33 Fionera @.***> wrote:

I wanted to give koel another try, but because it loads all Artists, Songs
and Albums, my Chrome and the Webserver crashes/is overloaded. Why doesnt
it just load a basic overview and then only the requested stuff via AJAX?


You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/koel/koel/issues/1036#issuecomment-798907540, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AB5O3UX3SHB5CE5IQD2I723TDS3JXANCNFSM4II765RQ
.

<!-- gh-comment-id:798922210 --> @phanan commented on GitHub (Mar 14, 2021): TBH it was because Koel was built to scratch my own itch and I never had 50k songs. But dealing with huge libraries is now on the list, just that I don’t know when I will have time for it. On Sun 14. Mar 2021 at 14:33 Fionera ***@***.***> wrote: > I wanted to give koel another try, but because it loads all Artists, Songs > and Albums, my Chrome and the Webserver crashes/is overloaded. Why doesnt > it just load a basic overview and then only the requested stuff via AJAX? > > — > You are receiving this because you modified the open/close state. > Reply to this email directly, view it on GitHub > <https://github.com/koel/koel/issues/1036#issuecomment-798907540>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AB5O3UX3SHB5CE5IQD2I723TDS3JXANCNFSM4II765RQ> > . >
Author
Owner

@vesubian commented on GitHub (Jan 17, 2022):

Hello,
First of all a big thank you and congrats to @phanan for the cool project.

I am starting to use it with my personal library of about 40k songs and its working well.
Scanning took about 2 hours in cli.

Would be interested in knowing if significantly larger libraries still work fine.

<!-- gh-comment-id:1014646716 --> @vesubian commented on GitHub (Jan 17, 2022): Hello, First of all a big thank you and congrats to @phanan for the cool project. I am starting to use it with my personal library of about 40k songs and its working well. Scanning took about 2 hours in cli. Would be interested in knowing if significantly larger libraries still work fine.
Author
Owner

@LesKent commented on GitHub (Mar 10, 2022):

I tried to see how koel would work with a large library (765K songs, 58K albums, and 13.7K artists) and it does not work for me after scanning in the library. I had to increase PHP memory limit to 10GB or it would report running out of memory in the laravel log. I upped the PHP max_execution_time and max_input_time to 1200 yet it still would not load any pages and it falls back to the login screen after 5-10 minutes of "thinking". During this initial page loading I would see memory usage go up to around 8GB. Syncing the library initially took over a day, but that does not bother me as that's a one time task. I'm not sure what more I can do to get it working at this point.

As a comparison, this same large library works very fast with Subsonic, showing snappy performance and an suprisingly low memory footprint, but it suffered from database corruption sometimes. With Airsonic Advanced it also works pretty good, albeit a little slower than Subsonic but it uses a ton of memory (often 7+GB). Plex is super fast for navigation and searching and also has a very low memory usage yet library scanning is hit or miss and its hard to debug or understand why. Navidrome is the most miserly with memory usage and is pretty fast overall. All these applications are running concurrently on a fast server with dual E5-2690 v4 processors, 192GB of RAM and all SSD storage under Windows Server 2019.

Hopefully there are some improvements with koel to deal with large libraries in the future as it matures.

<!-- gh-comment-id:1063697068 --> @LesKent commented on GitHub (Mar 10, 2022): I tried to see how koel would work with a large library (765K songs, 58K albums, and 13.7K artists) and it does not work for me after scanning in the library. I had to increase PHP memory limit to 10GB or it would report running out of memory in the laravel log. I upped the PHP max_execution_time and max_input_time to 1200 yet it still would not load any pages and it falls back to the login screen after 5-10 minutes of "thinking". During this initial page loading I would see memory usage go up to around 8GB. Syncing the library initially took over a day, but that does not bother me as that's a one time task. I'm not sure what more I can do to get it working at this point. As a comparison, this same large library works very fast with Subsonic, showing snappy performance and an suprisingly low memory footprint, but it suffered from database corruption sometimes. With Airsonic Advanced it also works pretty good, albeit a little slower than Subsonic but it uses a ton of memory (often 7+GB). Plex is super fast for navigation and searching and also has a very low memory usage yet library scanning is hit or miss and its hard to debug or understand why. Navidrome is the most miserly with memory usage and is pretty fast overall. All these applications are running concurrently on a fast server with dual E5-2690 v4 processors, 192GB of RAM and all SSD storage under Windows Server 2019. Hopefully there are some improvements with koel to deal with large libraries in the future as it matures.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/koel-koel#618
No description provided.