[GH-ISSUE #301] Serverless version of Koel

kerem commented

2026-02-26 02:32:28 +03:00

Owner

Originally created by @eladg on GitHub (Apr 15, 2016).
Original GitHub issue: https://github.com/koel/koel/issues/301

Hey @phanan, thanks for creating Koel, its really an amazing well thought eco system for music lovers.

For me, Koel's pricing is a big blocker for installing it as a service. EC2 prices starts at $15 a month and hosting my music library will cost even more... and all of that is per month(!) which makes this solution not very cost effective comparing to popular streaming services.

Theoretical question here:

I understand Koel is based on laravel on the server side and vuejs on the client side. Also I understand koel:sync is the main task to generate the songs/albums/files database records... which makes me think, will it be possible to create a Serverless version of Koel where a user will only have to pay for storage & streaming?

Cons:

Offline Management/Interaction (uploading songs, configuring settings, etc etc...)
Disabled Features:
- Playlists
- Interactions will be disabled (can't like / play counts etc etc)
Higher loading time (say on launch) where the client caches the library metadata on lunch.

Pros:

Extremely cheap monthly bill considering hosting on static storage host service such as S3
Reducing barriers for new users and going BELOW the popular $10 fee for a self hosted service.

On a Serverless installation, considering the following:

1GB of storage/data traffic on AWS S3 costs ~ $0.15.
60min of 192Kpbs aac is 85MB
we are talking about 800 Hours of streaming/hosting for $10 which can spread over weeks or even months, depends on your listening habits.

Hosting the files on the server is something you already have to do with the current model, removing the server out of the equation will make this solution very cost effective and easy for newcomers.

Would love to hear some thoughts about this.

References:
https://aws.amazon.com/s3/pricing/
http://audio-rescue.com/file-size/
http://ec2price.com/

Originally created by @eladg on GitHub (Apr 15, 2016). Original GitHub issue: https://github.com/koel/koel/issues/301 Hey @phanan, thanks for creating Koel, its really an amazing well thought eco system for music lovers. For me, Koel's pricing is a big blocker for installing it as a service. EC2 prices starts at $15 a month and hosting my music library will cost even more... and all of that is per month(!) which makes this solution not very cost effective comparing to popular streaming services. Theoretical question here: I understand Koel is based on [laravel](https://laravel.com/) on the server side and [vuejs](http://vuejs.org/) on the client side. Also I understand `koel:sync` is the main task to generate the songs/albums/files database records... which makes me think, will it be possible to create a Serverless version of Koel where a user will only have to pay for storage & streaming? Cons: - Offline Management/Interaction (uploading songs, configuring settings, etc etc...) - Disabled Features: - Playlists - Interactions will be disabled (can't like / play counts etc etc) - Higher loading time (say on launch) where the client caches the library metadata on lunch. Pros: - Extremely cheap monthly bill considering hosting on static storage host service such as S3 - Reducing barriers for new users and going BELOW the popular $10 fee for a self hosted service. On a Serverless installation, considering the following: - 1GB of storage/data traffic on AWS S3 costs ~ $0.15. - 60min of 192Kpbs aac is ~85MB we are talking about 800~ Hours of streaming/hosting for $10 which can spread over weeks or even months, depends on your listening habits. Hosting the files on the server is something you already have to do with the current model, removing the server out of the equation will make this solution very cost effective and easy for newcomers. Would love to hear some thoughts about this. References: https://aws.amazon.com/s3/pricing/ http://audio-rescue.com/file-size/ http://ec2price.com/

kerem closed this issue

2026-02-26 02:32:28 +03:00

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@phanan commented on GitHub (Apr 15, 2016):

That will require a remote scanning logic, i.e. your PHP scripts are local, but your media files are on S3. Such is not supported natively by Koel as discussed in #71, but you can try @hayarne's guide here: https://gist.github.com/heyarne/2823f0c1eac9ad41c7ecfb21f2bec9cb.

@phanan commented on GitHub (Apr 15, 2016): That will require a remote scanning logic, i.e. your PHP scripts are local, but your media files are on S3. Such is not supported natively by Koel as discussed in #71, but you can try @hayarne's guide here: https://gist.github.com/heyarne/2823f0c1eac9ad41c7ecfb21f2bec9cb.

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@eladg commented on GitHub (Apr 16, 2016):

Supporting s3 is something I think Koel need to do out of the box, don't you agree @phanan? If so, we can start there and move forward, no?

@eladg commented on GitHub (Apr 16, 2016): Supporting s3 is something I think Koel need to do out of the box, don't you agree @phanan? If so, we can start there and move forward, no?

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@phanan commented on GitHub (Apr 16, 2016):

If you read the linked issue you'll see why such a thing isn't as simple as
it seems.

On Sat, Apr 16, 2016 at 8:35 AM, Elad Gariany notifications@github.com
wrote:

Supporting s3 is something I think Koel need to do out of the box, don't
you agree @phanan https://github.com/phanan? If so, we can start there
and move forward, no?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/phanan/koel/issues/301#issuecomment-210697926

@phanan commented on GitHub (Apr 16, 2016): If you read the linked issue you'll see why such a thing isn't as simple as it seems. On Sat, Apr 16, 2016 at 8:35 AM, Elad Gariany notifications@github.com wrote: > Supporting s3 is something I think Koel need to do out of the box, don't > you agree @phanan https://github.com/phanan? If so, we can start there > and move forward, no? > > — > You are receiving this because you were mentioned. > Reply to this email directly or view it on GitHub > https://github.com/phanan/koel/issues/301#issuecomment-210697926

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@eladg commented on GitHub (Apr 16, 2016):

The main problem I see is this, currently Tags/metadata is coupled together with the files. So processes like uploading a new file to a server blocking streaming a file that already exists on the datastore.

This is a flaw in the technical design not a issue that can not be solved with the right software solution.

Exactly like you are saying here:

After further exploration, I can't figure out a way to make this work. Streaming is not a problem, scanning is. In order to read the ID3 tags, you'll need to download each and every song with every scan, which is too expensive IMO.

For example, I've noticed (sorry, can't find reference at the moment) Koel is saving cover photos outside of mp3 files. This is important for the client wishing to present a cover photo without actually getting the file. Why isn't the same practice apples for scanning and streaming files?

Decoupling metadata (in our case ID3) and files is really important for the future growth of the project.

@eladg commented on GitHub (Apr 16, 2016): The main problem I see is this, currently Tags/metadata is coupled together with the files. So processes like uploading a new file to a server blocking streaming a file that already exists on the datastore. This is a flaw in the technical design not a issue that can not be solved with the right software solution. Exactly like you are saying here: ``` After further exploration, I can't figure out a way to make this work. Streaming is not a problem, scanning is. In order to read the ID3 tags, you'll need to download each and every song with every scan, which is too expensive IMO. ``` For example, I've noticed (sorry, can't find reference at the moment) Koel is saving cover photos outside of mp3 files. This is important for the client wishing to present a cover photo without actually getting the file. Why isn't the same practice apples for scanning and streaming files? Decoupling metadata (in our case ID3) and files is really important for the future growth of the project.

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@eladg commented on GitHub (Apr 16, 2016):

Also, regarding pricing:
If you're hosting your service on EC2 and using S3, there are no costs involving in copying data within the same region (see: Data Transfer IN To Amazon S3 section).

Regarding complexity of downloading everything on a scan:
Koel should support incremental scans out of the box, this is true if you're hosting your files remotely or if your library is 40GBs - the complexity is still there.

@eladg commented on GitHub (Apr 16, 2016): Also, regarding pricing: If you're hosting your service on EC2 and using S3, there are **no costs** involving in copying data within the same region (see: [Data Transfer IN To Amazon S3](https://aws.amazon.com/s3/pricing/) section). Regarding complexity of downloading everything on a scan: Koel should support incremental scans out of the box, this is true if you're hosting your files remotely or if your library is 40GBs - the complexity is still there.

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@phanan commented on GitHub (Apr 17, 2016):

Well I don't mean to be defensive, but most of your points are inaccurate or confusing. Correct me if I'm wrong, but I don't think you have really looked at Koel's code and have an exact idea how it works.

The main problem I see is this, currently Tags/metadata is coupled together with the files.

Tags are embedded within audio files, so extracting them requires reading the files. This process is done once for each file, unless the file is modified. Even so, for an remote object storage, the initial scan will be painfully slow. For example, in @hayarne's guide, 511 songs at 4.6GB takes about 40 minutes. That's far from efficient. And I'm pretty sure an average user has thousands, if not tens of thousands, of songs.

So processes like uploading a new file to a server blocking streaming a file that already exists on the datastore.

Koel doesn't handle uploading, so I'm not sure what you mean by this statement as well as what it contributes to our problem.

For example, I've noticed (sorry, can't find reference at the moment) Koel is saving cover photos outside of mp3 files. This is important for the client wishing to present a cover photo without actually getting the file. Why isn't the same practice apples for scanning and streaming files?

After scanning, tags are saved into the database and remain unchanged till the next file modification. So again, I'm not sure what you mean. Not to mention that photos and actual media files serve totally different purpose obviously.

If you're hosting your service on EC2 and using S3, there are no costs involving in copying data within the same region

What does this have to do with Koel?

Koel should support incremental scans out of the box

And it does. What it doesn't, at least at this moment, is Object Storage, for the reasons I've been trying to explain.

To conclude, I would love to support Object Storage, but as of now this feature is way easier said than done. If you have a proper technical solution – which I can't find in your comments – I'm more than happy to hear. Otherwise, phrases like "you should do this," "you need to do that," and "this is important to the project growth," when the request is clearly first and foremost for your personal needs, can sound pretty demanding don't you think?

@phanan commented on GitHub (Apr 17, 2016): Well I don't mean to be defensive, but most of your points are inaccurate or confusing. Correct me if I'm wrong, but I don't think you have really looked at Koel's code and have an exact idea how it works. > The main problem I see is this, currently Tags/metadata is coupled together with the files. Tags are embedded _within_ audio files, so extracting them _requires_ reading the files. This process is done once for each file, unless the file is modified. Even so, for an remote object storage, the _initial_ scan will be painfully slow. For example, in @hayarne's guide, 511 songs at 4.6GB takes about 40 minutes. That's far from efficient. And I'm pretty sure an average user has thousands, if not tens of thousands, of songs. > So processes like uploading a new file to a server blocking streaming a file that already exists on the datastore. Koel doesn't handle uploading, so I'm not sure what you mean by this statement as well as what it contributes to our problem. > For example, I've noticed (sorry, can't find reference at the moment) Koel is saving cover photos outside of mp3 files. This is important for the client wishing to present a cover photo without actually getting the file. Why isn't the same practice apples for scanning and streaming files? After scanning, tags are saved into the database and remain unchanged till the next file modification. So again, I'm not sure what you mean. Not to mention that photos and actual media files serve totally different purpose obviously. > If you're hosting your service on EC2 and using S3, there are no costs involving in copying data within the same region What does this have to do with Koel? > Koel should support incremental scans out of the box And it does. What it doesn't, at least at this moment, is Object Storage, for the reasons I've been trying to explain. To conclude, I would _love_ to support Object Storage, but as of now this feature is way easier said than done. If you have a proper technical solution – which I can't find in your comments – I'm more than happy to hear. Otherwise, phrases like "you should do this," "you need to do that," and "this is important to the project growth," when the request is clearly first and foremost for your personal needs, can sound pretty demanding don't you think?

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@eladg commented on GitHub (Apr 20, 2016):

Hey @phanan, appreciate your response. You don't need to be defensive, I can assure you, I'm here TO HELP (not to waste your time) and for the best of Koel. Lets keep the discussion positive and productive (or not talk at all).

I think we come from slightly different methodologies - me from the Micro Services world and you from Web Applications methodology, it's a common gap when discussing architectural changes and that's completely ok. This is a architectural discussion/suggestion and not a feature request. If we can agree on the right approach, I can promise I will PERSONALLY work on the PR (and please, take my word for it :) )

Yet, this is not for my personal needs, no idea how you came to that conclusions (and sorry, that's kinda rude to assume). I explicitly listed the benefits of having a serverless solution, starting with reducing the costs of hosting every instance of Koel. If currently hosting Koel costs about $10-25 a month (my calculation above) and a serverless solution may cost $3-10 a month (if you listen to music 24/7!), that's for everyone's benefit, not me.

If you're taking this number lightly or just couldn't careless about this barrier for new users, I think there is nothing else to discuss here.

If you would like, I can come up with a propose technical document, with the design of supporting object storage and serverless installation(!). I'm doing it because I would love to have a personal music streaming service I can control and that does not cost a fortune...

This is an MIT based opensource project and my intensions are good :)

@eladg commented on GitHub (Apr 20, 2016): Hey @phanan, appreciate your response. You don't need to be defensive, I can assure you, I'm here TO HELP (not to waste your time) and for the best of Koel. Lets keep the discussion positive and productive (or not talk at all). I think we come from slightly different methodologies - me from the Micro Services world and you from Web Applications methodology, it's a common gap when discussing architectural changes and that's completely ok. This is a architectural discussion/suggestion and not a feature request. If we can agree on the right approach, I can promise I will PERSONALLY work on the PR (and please, take my word for it :) ) Yet, this is not for my personal needs, no idea how you came to that conclusions (and sorry, that's kinda rude to assume). I explicitly listed the benefits of having a serverless solution, starting with reducing the costs of hosting every instance of Koel. If currently hosting Koel costs about $10-25 a month (my calculation above) and a serverless solution may cost $3-10 a month (if you listen to music 24/7!), that's for everyone's benefit, not me. If you're taking this number lightly or just couldn't careless about this barrier for new users, I think there is nothing else to discuss here. If you would like, I can come up with a propose technical document, with the design of supporting object storage and serverless installation(!). I'm doing it because I would love to have a personal music streaming service I can control and that does not cost a fortune... This is an MIT based opensource project and my intensions are good :)

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@phanan commented on GitHub (Apr 21, 2016):

I'd agree, my assume was kinda rude. My apologies :)

Now, I don't take the number lightly, but Koel was developed on the
presumption that the user – as a web developer – already has his own server
(even a shared host will do) to share with Koel, and not that he will need
to rent a whole server just for Koel. The latter wouldn't make much sense,
especially when we think about how cheap streaming services like Spotify
and Apple Music are nowadays.

That said, and as shared before, the ability to use object storage is
something I'd love to see supported out of the box. Still, assuming that
you know how Koel currently scans and updates the library, implementation
is a challenge, for all the reasons listed above. Yet again, if you have a
solution, I'm all ears :)

On Thu, Apr 21, 2016 at 7:18 AM, Elad Gariany notifications@github.com
wrote:

Hey @phanan https://github.com/phanan, appreciate your response. You
don't need to be defensive, I can assure you, I'm here TO HELP (not to
waste your time) and for the best of Koel. Lets keep the discussion
positive and productive (or not talk at all).

I think we come from slightly different methodologies - me from the Micro
Services world and you from Web Applications methodology, it's a common gap
when discussing architectural changes and that's completely ok. This is a
architectural discussion/suggestion and not a feature request. If we can
agree on the right approach, I can promise I will PERSONALLY work on the PR
(and please, take my word for it :) )

Yet, this is not for my personal needs, no idea how you came to that
conclusions (and sorry, that's kinda rude the assume). I explicitly listed
the benefits of having a serverless solution, starting with reducing the
costs of hosting every instance of Koel. If currently hosting Koel costs
about $10-25 a month (my calculation above) and a serverless solution may
cost $3-10 a month (if you listen to music 24/7!), that's for everyone's
benefit not me.

If you're taking this number lightly or just couldn't careless about this
barrier for new users, I think there is nothing else to discuss here.

If you would like, I can come up with a propose technical document, with
the design of supporting object storage and serverless installation(!). I'm
doing it because I would love to have a personal music streaming service I
can control and that does not cost a fortune...

This is an MIT based opensource project and my intensions are good :)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/phanan/koel/issues/301#issuecomment-212649279

@phanan commented on GitHub (Apr 21, 2016): I'd agree, my assume was kinda rude. My apologies :) Now, I don't take the number lightly, but Koel was developed on the presumption that the user – as a web developer – already has his own server (even a shared host will do) to share with Koel, and not that he will need to rent a whole server just for Koel. The latter wouldn't make much sense, especially when we think about how cheap streaming services like Spotify and Apple Music are nowadays. That said, and as shared before, the ability to use object storage is something I'd love to see supported out of the box. Still, assuming that you know how Koel currently scans and updates the library, implementation is a challenge, for all the reasons listed above. Yet again, if you have a solution, I'm all ears :) On Thu, Apr 21, 2016 at 7:18 AM, Elad Gariany notifications@github.com wrote: > Hey @phanan https://github.com/phanan, appreciate your response. You > don't need to be defensive, I can assure you, I'm here TO HELP (not to > waste your time) and for the best of Koel. Lets keep the discussion > positive and productive (or not talk at all). > > I think we come from slightly different methodologies - me from the Micro > Services world and you from Web Applications methodology, it's a common gap > when discussing architectural changes and that's completely ok. This is a > architectural discussion/suggestion and not a feature request. If we can > agree on the right approach, I can promise I will PERSONALLY work on the PR > (and please, take my word for it :) ) > > Yet, this is not for my personal needs, no idea how you came to that > conclusions (and sorry, that's kinda rude the assume). I explicitly listed > the benefits of having a serverless solution, starting with reducing the > costs of hosting every instance of Koel. If currently hosting Koel costs > about $10-25 a month (my calculation above) and a serverless solution may > cost $3-10 a month (if you listen to music 24/7!), that's for everyone's > benefit not me. > > If you're taking this number lightly or just couldn't careless about this > barrier for new users, I think there is nothing else to discuss here. > > If you would like, I can come up with a propose technical document, with > the design of supporting object storage and serverless installation(!). I'm > doing it because I would love to have a personal music streaming service I > can control and that does not cost a fortune... > > This is an MIT based opensource project and my intensions are good :) > > — > You are receiving this because you were mentioned. > Reply to this email directly or view it on GitHub > https://github.com/phanan/koel/issues/301#issuecomment-212649279

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@phanan commented on GitHub (Jun 10, 2016):

I think I've figured out a way to do this for Amazon S3. Reopening to remind myself.

@phanan commented on GitHub (Jun 10, 2016): I think I've figured out a way to do this for Amazon S3. Reopening to remind myself.

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@phanan commented on GitHub (Jun 13, 2016):

I've come up with a package (and guide) to use Amazon S3 with Koel. Check it out: https://github.com/phanan/koel-aws.

@phanan commented on GitHub (Jun 13, 2016): I've come up with a package (and guide) to use Amazon S3 with Koel. Check it out: https://github.com/phanan/koel-aws.

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@eladg commented on GitHub (Jun 16, 2016):

That's great news @phanan, thank you! 👍 🎉

Would you still consider a Serverless architecture for Koel in the future? Maybe it will be 2 years from now and there is no immediate call to action at this point, but I think the Pros can really use as an alternative to popular music services...

If you be willing to consider, I can come up with a list of changes that will be needed to support it.

(For example, lambda process to update a JSON/whatever data structure with the list of files on the datastore etc etc...)

@eladg commented on GitHub (Jun 16, 2016): That's great news @phanan, thank you! 👍 🎉 Would you still consider a Serverless architecture for Koel in the future? Maybe it will be 2 years from now and there is no immediate call to action at this point, but I think the Pros can really use as an alternative to popular music services... If you be willing to consider, I can come up with a list of changes that will be needed to support it. (For example, lambda process to update a JSON/whatever data structure with the list of files on the datastore etc etc...)

kerem commented

2026-02-26 02:32:29 +03:00

Author

Owner

@phanan commented on GitHub (Jun 16, 2016):

@eladg Yes, for sure I'd be happy to consider whatever's good for the project and its users ;)

@phanan commented on GitHub (Jun 16, 2016): @eladg Yes, for sure I'd be happy to consider whatever's good for the project and its users ;)

Rows
Columns

[GH-ISSUE #301] Serverless version of Koel #217