[GH-ISSUE #34] Limit cache size #22

Closed
opened 2026-02-27 19:28:22 +03:00 by kerem · 14 comments
Owner

Originally created by @sashahilton00 on GitHub (Jan 29, 2018).
Original GitHub issue: https://github.com/librespot-org/librespot/issues/34

Issue by giannello
Friday Jul 29, 2016 at 19:50 GMT
Originally opened as https://github.com/plietar/librespot/issues/105


For people running librespot on hardware with limited resources, it would be nice to have an option to limit the cache size.

Originally created by @sashahilton00 on GitHub (Jan 29, 2018). Original GitHub issue: https://github.com/librespot-org/librespot/issues/34 <a href="https://github.com/giannello"><img src="https://avatars0.githubusercontent.com/u/551789?v=4" align="left" width="96" height="96" hspace="10"></img></a> **Issue by [giannello](https://github.com/giannello)** _Friday Jul 29, 2016 at 19:50 GMT_ _Originally opened as https://github.com/plietar/librespot/issues/105_ ---- For people running librespot on hardware with limited resources, it would be nice to have an option to limit the cache size.
kerem 2026-02-27 19:28:22 +03:00
Author
Owner

@sashahilton00 commented on GitHub (Jan 29, 2018):

Comment by plietar
Friday Jul 29, 2016 at 20:04 GMT


This is definitely something I'd like to have, but requires quite a bit of work.
At the moment, caching is just a matter of placing the file in the cache directory, and trying to open it when loading.

This requires some bookkeeping to keep track of what files are in the cache, their size, along with an eviction algorithm.

These could be used as inspiration, although librespot doesn't need something as sophisticated :
https://wiki.mozilla.org/Necko/Cache
https://www.chromium.org/developers/design-documents/network-stack/disk-cache

<!-- gh-comment-id:361258862 --> @sashahilton00 commented on GitHub (Jan 29, 2018): <a href="https://github.com/plietar"><img src="https://avatars0.githubusercontent.com/u/1489775?v=4" align="left" width="48" height="48" hspace="10"></img></a> **Comment by [plietar](https://github.com/plietar)** _Friday Jul 29, 2016 at 20:04 GMT_ ---- This is definitely something I'd like to have, but requires quite a bit of work. At the moment, caching is just a matter of placing the file in the cache directory, and trying to open it when loading. This requires some bookkeeping to keep track of what files are in the cache, their size, along with an eviction algorithm. These could be used as inspiration, although librespot doesn't need something as sophisticated : https://wiki.mozilla.org/Necko/Cache https://www.chromium.org/developers/design-documents/network-stack/disk-cache
Author
Owner

@sashahilton00 commented on GitHub (Jan 29, 2018):

Comment by mfonville
Tuesday Nov 29, 2016 at 14:16 GMT


For the moment, users that face issues with it, they could make a simple cron-job that checks the atime of files tp clean-up.

<!-- gh-comment-id:361258881 --> @sashahilton00 commented on GitHub (Jan 29, 2018): <a href="https://github.com/mfonville"><img src="https://avatars0.githubusercontent.com/u/6221544?v=4" align="left" width="48" height="48" hspace="10"></img></a> **Comment by [mfonville](https://github.com/mfonville)** _Tuesday Nov 29, 2016 at 14:16 GMT_ ---- For the moment, users that face issues with it, they could make a simple cron-job that checks the atime of files tp clean-up.
Author
Owner

@sashahilton00 commented on GitHub (Jan 29, 2018):

Comment by juriansluiman
Wednesday Feb 22, 2017 at 13:22 GMT


@mfonville how is this possible? If I run find /tmp/librespot/files/* -atime +2 -exec rm {} \; it crashes librespot. The only way I can get it to work is to stop the librespot service, clear up the files and start it again. If I do this, the music will stop when playing, so it doesn't seem to be the most ideal solution yet :)

<!-- gh-comment-id:361258900 --> @sashahilton00 commented on GitHub (Jan 29, 2018): <a href="https://github.com/juriansluiman"><img src="https://avatars2.githubusercontent.com/u/705925?v=4" align="left" width="48" height="48" hspace="10"></img></a> **Comment by [juriansluiman](https://github.com/juriansluiman)** _Wednesday Feb 22, 2017 at 13:22 GMT_ ---- @mfonville how is this possible? If I run `find /tmp/librespot/files/* -atime +2 -exec rm {} \;` it crashes librespot. The only way I can get it to work is to stop the librespot service, clear up the files and start it again. If I do this, the music will stop when playing, so it doesn't seem to be the most ideal solution yet :)
Author
Owner

@sashahilton00 commented on GitHub (Jan 29, 2018):

Comment by plietar
Wednesday Feb 22, 2017 at 14:33 GMT


@juriansluiman Deleting files while librespot is running should work.
Can you run librespot with RUST_BACKTRACE=1 and --verbose and post the log please ? What platform are you running on ?

<!-- gh-comment-id:361258930 --> @sashahilton00 commented on GitHub (Jan 29, 2018): <a href="https://github.com/plietar"><img src="https://avatars0.githubusercontent.com/u/1489775?v=4" align="left" width="48" height="48" hspace="10"></img></a> **Comment by [plietar](https://github.com/plietar)** _Wednesday Feb 22, 2017 at 14:33 GMT_ ---- @juriansluiman Deleting files while librespot is running should work. Can you run librespot with `RUST_BACKTRACE=1` and `--verbose` and post the log please ? What platform are you running on ?
Author
Owner

@sashahilton00 commented on GitHub (Jan 29, 2018):

Comment by juriansluiman
Wednesday Feb 22, 2017 at 15:10 GMT


@plietar sorry, you get forget my comment. I was having another issue with librespot so it stopped working, but just tried again and it does continue to play. My bad!

<!-- gh-comment-id:361258951 --> @sashahilton00 commented on GitHub (Jan 29, 2018): <a href="https://github.com/juriansluiman"><img src="https://avatars2.githubusercontent.com/u/705925?v=4" align="left" width="48" height="48" hspace="10"></img></a> **Comment by [juriansluiman](https://github.com/juriansluiman)** _Wednesday Feb 22, 2017 at 15:10 GMT_ ---- @plietar sorry, you get forget my comment. I was having another issue with librespot so it stopped working, but just tried again and it does continue to play. My bad!
Author
Owner

@sashahilton00 commented on GitHub (Jan 29, 2018):

Comment by michaelherger
Tuesday May 02, 2017 at 04:34 GMT


I've modified librespot to disable the audio file-cache. See pull request #181.

<!-- gh-comment-id:361258979 --> @sashahilton00 commented on GitHub (Jan 29, 2018): <a href="https://github.com/michaelherger"><img src="https://avatars3.githubusercontent.com/u/2789989?v=4" align="left" width="48" height="48" hspace="10"></img></a> **Comment by [michaelherger](https://github.com/michaelherger)** _Tuesday May 02, 2017 at 04:34 GMT_ ---- I've modified librespot to disable the audio file-cache. See pull request #181.
Author
Owner

@shawaj commented on GitHub (Jan 31, 2021):

Any progress on this @sashahilton00 ?

<!-- gh-comment-id:770324830 --> @shawaj commented on GitHub (Jan 31, 2021): Any progress on this @sashahilton00 ?
Author
Owner

@Sojiro84 commented on GitHub (Feb 13, 2021):

Wondering the same. I just discovered that librespot is taking up 41.8GB of data.

That is a lot for something that should just stream audio!

<!-- gh-comment-id:778581883 --> @Sojiro84 commented on GitHub (Feb 13, 2021): Wondering the same. I just discovered that librespot is taking up 41.8GB of data. That is a lot for something that should just stream audio!
Author
Owner

@debdrup commented on GitHub (Feb 16, 2021):

For a simple algorithm, would it make sense to evict the oldest cached files, either by creation or modification date, when the cache grows beyond a certain size?
As long some code gets added to fix this issue, I'm not sure it needs to be optimal or perfect.

It's definitely better than having a library be responsible for taking up any available diskspace on a long enough timescale.

<!-- gh-comment-id:779821165 --> @debdrup commented on GitHub (Feb 16, 2021): For a simple algorithm, would it make sense to evict the oldest cached files, either by creation or modification date, when the cache grows beyond a certain size? As long some code gets added to fix this issue, I'm not sure it needs to be optimal or perfect. It's definitely better than having a library be responsible for taking up any available diskspace on a long enough timescale.
Author
Owner

@sashahilton00 commented on GitHub (Feb 16, 2021):

Nope, no progress. We did discuss it a little in another issue, I can't remember which one exactly, will check when I get home. Ideally we would separate the cache logic out, as there are a few use cases/requirements, eg. Limiting cache by size and/or date, or disable the cache all together.

I can appreciate that in the meantime though it would be better to have something rather than nothing, so if someone does want to throw together a PR that implements a basic cache eviction function, we'll look at getting it merged. If no one wants to, I'll try and give it a go at a later date, but currently I have limited time, and there are a couple of other features that take priority imo, such as retrieving files from the CDN or session reconnection.

<!-- gh-comment-id:779850335 --> @sashahilton00 commented on GitHub (Feb 16, 2021): Nope, no progress. We did discuss it a little in another issue, I can't remember which one exactly, will check when I get home. Ideally we would separate the cache logic out, as there are a few use cases/requirements, eg. Limiting cache by size and/or date, or disable the cache all together. I can appreciate that in the meantime though it would be better to have something rather than nothing, so if someone does want to throw together a PR that implements a basic cache eviction function, we'll look at getting it merged. If no one wants to, I'll try and give it a go at a later date, but currently I have limited time, and there are a couple of other features that take priority imo, such as retrieving files from the CDN or session reconnection.
Author
Owner

@ashthespy commented on GitHub (Feb 16, 2021):

I have an old branch with a simple LRU based file cache implementation with a prescribed limit, can hunt things up if someone would like to work on it..

<!-- gh-comment-id:779855940 --> @ashthespy commented on GitHub (Feb 16, 2021): I have an old branch with a simple LRU based file cache implementation with a prescribed limit, can hunt things up if someone would like to work on it..
Author
Owner

@woodruffw commented on GitHub (Feb 21, 2021):

I'm completely new to this project, but I'm happy to take a stab if someone has a partial implementation.

<!-- gh-comment-id:782887390 --> @woodruffw commented on GitHub (Feb 21, 2021): I'm completely new to this project, but I'm happy to take a stab if someone has a partial implementation.
Author
Owner

@azrdev commented on GitHub (Feb 21, 2021):

I have deployed the following systemd units:

# /etc/systemd/system/spotifyd-cache-clean.timer
[Unit]
Description=Daily clean audio file cache of spotifyd/librespot

[Timer]
OnCalendar=4:00
Persistent=true

[Install]
WantedBy=timers.target
# /etc/systemd/system/spotifyd-cache-clean.service
[Unit]
Description=Clean audio file cache of spotifyd/librespot

[Service]
User=spotifyd
ExecStart=/usr/bin/find /var/cache/spotifyd/files -type f -atime +30 -print -delete

Obviously depends on systemd and the filesystem correctly tracking access times,
then deletes all cached files not accessed for the last 30 days

<!-- gh-comment-id:782904445 --> @azrdev commented on GitHub (Feb 21, 2021): I have deployed the following systemd units: ~~~ini # /etc/systemd/system/spotifyd-cache-clean.timer [Unit] Description=Daily clean audio file cache of spotifyd/librespot [Timer] OnCalendar=4:00 Persistent=true [Install] WantedBy=timers.target ~~~ ~~~ini # /etc/systemd/system/spotifyd-cache-clean.service [Unit] Description=Clean audio file cache of spotifyd/librespot [Service] User=spotifyd ExecStart=/usr/bin/find /var/cache/spotifyd/files -type f -atime +30 -print -delete ~~~ Obviously depends on systemd and the filesystem correctly tracking access times, then deletes all cached files not accessed for the last 30 days
Author
Owner

@debdrup commented on GitHub (Feb 21, 2021):

While that's a good temporary hack for some, it's not really a solution.

Firstly, because it isn't cross-platform (even if it were to be included with librespot, which exists where systemd does not) and secondly, because it's not a POSIX-compatible invocation of find.

It's also not okay for libraries or programs to take up arbitrary disk space without ways of cleaning up after themselves.

<!-- gh-comment-id:782932178 --> @debdrup commented on GitHub (Feb 21, 2021): While that's a good temporary hack for some, it's not really a solution. Firstly, because it isn't cross-platform (even if it were to be included with librespot, which exists where systemd does not) and secondly, because [it's not a POSIX-compatible invocation of find](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html#top). It's also not okay for libraries or programs to take up arbitrary disk space without ways of cleaning up after themselves.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/librespot#22
No description provided.