mirror of
https://github.com/go-shiori/shiori.git
synced 2026-04-25 06:25:54 +03:00
[GH-ISSUE #546] Docker image cannot be used with SQLite in a mounted volume #301
Labels
No labels
component:backend
component:builds
component:builds
component:extension
component:frontend
component:readability
database
database:mysql
database:postgres
database:sqlite
feature:ebooks
github_actions
good first issue
hacktoberfest
note:duplicate?
note:fixed?
note:out-of-scope?
os:windows
priority:high
priority:low
pull-request
resolution:as-intended
resolution:cant-reproduce
resolution:duplicate
resolution:fixed
resolution:wontfix
tag:TBD
tag:big-task
tag:help-wanted
tag:huge-data
tag:meta
tag:more-info
tag:next
tag:no-stale
tag:requires-migrations
tag:research
tag:security 🛡️
tag:stale
tag:waiting-for-assignee
type:bug
type:documentation
type:enhancement
type:meta
type:ux
user:cli
user:web
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/shiori#301
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @schlamar on GitHub (Dec 31, 2022).
Original GitHub issue: https://github.com/go-shiori/shiori/issues/546
Originally assigned to: @fmartingr on GitHub.
Data
Describe the bug / actual behavior
If you run the Docker image with /shiori as mounted volume the shiori process cannot write the database. It crashes with "out of memory" (see stacktrace below).
Expected behavior
Shiori with SQLite can be used with a mounted volume (running without mounted volume doesn't make sense for production deployment).
To Reproduce
Create a compose.yaml
Running
docker compose upresults in:Notes
This is probably a permission issue. I don't think you should create an extra user in a Docker container to run an application in default use cases (disclaimer: no Docker expert here, I'm just guessing). Just run mkdir /shiori instead of addgroup/adduser probably fixes this issue.
@schlamar commented on GitHub (Dec 31, 2022):
Next issue on manual build, there is no alpine 3.16 on ghcr.io registry:
Probably you should use the official Docker hub here.
@schlamar commented on GitHub (Dec 31, 2022):
This Dockerfile worked for me
@schlamar commented on GitHub (Dec 31, 2022):
A workaround is to create the "data" directory before running docker. It is only broken if the data directory is created by docker / the container.
The documented docker run from Usage.md works because there is no subdirectory involved.
@fmartingr commented on GitHub (Jan 10, 2023):
Hey @schlamar, have you tried this with the
devtag? It works correctly on my machine (though is a Mac), but it automatically migrates the database and creates the appropriate folders:Though I tried with
latestand it works as well in my case (and I have it setup on a server with a volume too, have been working for a while now). Which host operating system are you using?@therysin commented on GitHub (Jan 11, 2023):
Also having the same issue with both the latest and dev builds. I’m on Ubuntu 22.04.1 arm (Oracle Cloud).
Seemed like an awesome project but the container won’t start unfortunately (same out of memory error ).
I should also note that I’m using docker-compose(same file as OP to test), and it’s running behind traefik.
If the mounted volume lines are commented or removed from the compose file, shiori works fine (without data persistence).
@schlamar commented on GitHub (Jan 11, 2023):
Dev fails, too:
Running Arch Linux with current docker release:
@Denow commented on GitHub (Jan 18, 2023):
I have the same problem with Ubuntu 20.04.5 LTS (x86).
@fmartingr commented on GitHub (Jan 18, 2023):
Are you running ubuntu over oracle cloud as well? Do you have the exact same error as @schlamar ?
@Denow commented on GitHub (Jan 18, 2023):
No, I am running Ubuntu in a vm on an onprem esxi host. The error I am receiving is the same as @schlamar:
Failed to open database: unable to open database file: out of memory (14)The container keeps restarting after this error message and the data dir is empty so it does look like a permission issue.
@Denow commented on GitHub (Jan 18, 2023):
Nevermind I was mounting the wrong volume...
@therysin commented on GitHub (Jan 18, 2023):
Can you expand further on what was wrong/ what volume you mounted to fix?
@Denow commented on GitHub (Jan 18, 2023):
I was using the following tutorial for deploying shiori: https://noted.lol/tutorial-setting-up-shiori.
The docker-compose file in this tutorial changes the shiori data dir with an environment variable to SHIORI_DIR=/data and creates a volume for /data. Using the same docker-compose file gave me the
Failed to open database: unable to open database file: out of memory (14)error.I removed the environment variable so shiori would write the db to the default location /shiori and also changed the volume path to the same location, now everything is working great.
@therysin commented on GitHub (Jan 18, 2023):
Thanks. Was hoping it would be something I overlooked, but I’m already using the default shiori location 🥲. I’m using the exact same compose file as the OP.
@therysin commented on GitHub (Jan 19, 2023):
I ended up using a regular docker volume for persistence and not a bind mount.
Probably a temp fix, but works out in the short term.
@fmartingr commented on GitHub (Jan 21, 2023):
I have been trying to run and connect to an Oracle Cloud ubuntu instance for 15 minutes now. I wanted to try over there just in case, but this works on my mac/linux machines. You should be able to create a mounted volume if you have enough permissions over the folder being mounted. I'll leave this open for now, in case someone else can chip in here.
@danielnbalasoiu commented on GitHub (Feb 25, 2023):
I had the same error and I fixed it by creating the local directory and then set the correct permissions expected by shiori.
@milosimpson commented on GitHub (Oct 23, 2023):
Also running into this issue trying to get shiori running in docker on a Synology NAS.
If I don't specify a data volume, I can get v.1.5.5 to boot because shiori will make a sqlite db in the
/shioridir inside the docker container, which works but isn't great as there is no upgrade path as the db is "trapped" inside the container.When I try to give it an external volume/mount, it fails to be able to make the sqllite db file because user 1000/shiori doesn't have write permissions to the mounted directory.
I am guessing you want to use a non-root user for security reasons since this service exposes an http service.
I think the fix might be to: while booting, as root chown the SHIORI_DIR to the uid 1000/shiori user, then switch to the uid 1000/shiori user.
I tried to test this by pre-populating a
shiori.dbfile in the mount volumn/directory, to get past the boot failure so that I could shell into the container and try to runsudo chown -R 1000:1000on the mounted dir to see if that would temporarily make things work, but the image does not containsudo.Shelled into the container (v1.5.5) to look at who owns /shiori/shiori.db on a setup where there is no external volume mount.

User
shioriowns the file.Shelled into the container with an external volume mount with pre-existing shiori.db file (v1.5.3 that it doesn't try to migrate db on boot). User root owns that file/directory which means it isn't usable.

@GeekyNinja2003 commented on GitHub (Oct 23, 2023):
Hi ,
Thank you for reaching out regarding the Shiori Docker issue on Synology
NAS. I appreciate your detailed explanation of the problem.
It seems like you’re encountering a permissions issue when trying to use an
external volume for Shiori in Docker. You’re correct in assuming that we
aim to use a non-root user for security reasons, especially for services
that expose HTTP. To address this issue, your suggested approach could be a
viable solution.
Here’s a step-by-step breakdown: 1. While booting as root: • Change
ownership of the SHIORI_DIR to the UID 1000/shiori user. • Switch to the
UID 1000/shiori user. This should resolve the permission problem and allow
Shiori to create the SQLite DB file on the external volume. Unfortunately,
the image does not contain sudo, as you’ve mentioned. In this case, you
might need to use other methods to change ownership, such as using the
chown command with the --no-dereference option.
Feel free to test this approach and let us know how it works for you. If
you have any further questions or encounter any issues during this process,
please don’t hesitate to reach out for additional assistance.
Best regards, Jerry;-)
On Mon, 23 Oct 2023 at 02:18 Milo Simpson @.***> wrote:
@fmartingr commented on GitHub (Oct 29, 2023):
I need to unify both Dockerfiles, since we use one for the CI (production builds) and one for development, and that makes no sense right now. That said, Shiori should work using a mounted volume without much issue as I have been running it like that for some time now.
Maybe allowing to customize the user and group IDs on the image via environment variables would work well, but that something I need to investigate and make backwards compatible, so it will be after we release 1.6.0, since that already contains enough changes as it is.
@milosimpson commented on GitHub (Nov 12, 2023):
Figured out my problem. It is a Synology NAS specific issue. The fix was to create a specific non-root user with access to the mount directory ala https://drfrankenstein.co.uk/step-2-setting-up-a-restricted-docker-user-and-obtaining-ids/ and then use that user with Docker Compose. The Synology "Container Manager" app is basically Synology's Portainer, and it can run docker compose yaml files.
It seemed like most suggestions on the internet were to make user and group that the docker image wants on the host OS, which is easy to do on Linux systems, but not on the Synology.
The catch is that it seems like only compose and the yaml file approach can set the user and group id, not the "docker" command line or simple docker run UIs in the Synology.
The one nice thing that Shiroi could do is take UID and PID to use as environment variables, that way it could be run w/out compose.
@danielyrovas commented on GitHub (Apr 8, 2024):
I ran into this issue using podman running in rootless mode. I manged to solve it by building a container as below:
@fmartingr commented on GitHub (May 12, 2024):
I just tested the image on PR #907 on a linux laptop and it required no further modifications in order to mount a local path: