[GH-ISSUE #546] Docker image cannot be used with SQLite in a mounted volume #301

Closed
opened 2026-02-25 23:33:54 +03:00 by kerem · 22 comments
Owner

Originally created by @schlamar on GitHub (Dec 31, 2022).
Original GitHub issue: https://github.com/go-shiori/shiori/issues/546

Originally assigned to: @fmartingr on GitHub.

Data

  • Shiori version: latest docker image (1.5.3)
  • Database Engine: SQLite
  • Operating system: N/A
  • CLI/Web interface/Web Extension: N/A

Describe the bug / actual behavior

If you run the Docker image with /shiori as mounted volume the shiori process cannot write the database. It crashes with "out of memory" (see stacktrace below).

Expected behavior

Shiori with SQLite can be used with a mounted volume (running without mounted volume doesn't make sense for production deployment).

To Reproduce

Create a compose.yaml

services:
  shiori:
    container_name: shiori
    image: "ghcr.io/go-shiori/shiori"
    restart: unless-stopped
    volumes:
      - ./data:/shiori  
      - /etc/localtime:/etc/localtime:ro
    ports:
      - 8080:8080

Running docker compose up results in:

shiori  | panic: unable to open database file: out of memory (14)
shiori  |
shiori  | goroutine 1 [running]:
shiori  | github.com/jmoiron/sqlx.MustConnect(...)
shiori  |       /home/runner/go/pkg/mod/github.com/jmoiron/sqlx@v1.3.5/sqlx.go:654
shiori  | github.com/go-shiori/shiori/internal/database.OpenSQLiteDatabase({0xc0000ca438, 0x15fc350})
shiori  |       /home/runner/work/shiori/shiori/internal/database/sqlite.go:27 +0xaa
shiori  | github.com/go-shiori/shiori/internal/cmd.openSQLiteDatabase()
shiori  |       /home/runner/work/shiori/shiori/internal/cmd/root.go:117 +0x6a
shiori  | github.com/go-shiori/shiori/internal/cmd.openDatabase()
shiori  |       /home/runner/work/shiori/shiori/internal/cmd/root.go:111 +0x79
shiori  | github.com/go-shiori/shiori/internal/cmd.preRunRootHandler(0xc00021cc80, {0xbbd9a8, 0x0, 0x0})
shiori  |       /home/runner/work/shiori/shiori/internal/cmd/root.go:64 +0x116
shiori  | github.com/spf13/cobra.(*Command).execute(0xc00021cc80, {0x15fbd98, 0x0, 0x0})
shiori  |       /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:840 +0x50a
shiori  | github.com/spf13/cobra.(*Command).ExecuteC(0xc0001a3400)
shiori  |       /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:974 +0x3bc
shiori  | github.com/spf13/cobra.(*Command).Execute(...)
shiori  |       /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:902
shiori  | main.main()
shiori  |       /home/runner/work/shiori/shiori/main.go:19 +0x1e

Notes

This is probably a permission issue. I don't think you should create an extra user in a Docker container to run an application in default use cases (disclaimer: no Docker expert here, I'm just guessing). Just run mkdir /shiori instead of addgroup/adduser probably fixes this issue.

Originally created by @schlamar on GitHub (Dec 31, 2022). Original GitHub issue: https://github.com/go-shiori/shiori/issues/546 Originally assigned to: @fmartingr on GitHub. ## Data - **Shiori version**: latest docker image (1.5.3) - **Database Engine**: SQLite - **Operating system**: N/A - **CLI/Web interface/Web Extension**: N/A ## Describe the bug / actual behavior If you run the Docker image with /shiori as mounted volume the shiori process cannot write the database. It crashes with "out of memory" (see stacktrace below). ## Expected behavior Shiori with SQLite can be used with a mounted volume (running without mounted volume doesn't make sense for production deployment). ## To Reproduce Create a compose.yaml ```yaml services: shiori: container_name: shiori image: "ghcr.io/go-shiori/shiori" restart: unless-stopped volumes: - ./data:/shiori - /etc/localtime:/etc/localtime:ro ports: - 8080:8080 ``` Running `docker compose up` results in: ``` shiori | panic: unable to open database file: out of memory (14) shiori | shiori | goroutine 1 [running]: shiori | github.com/jmoiron/sqlx.MustConnect(...) shiori | /home/runner/go/pkg/mod/github.com/jmoiron/sqlx@v1.3.5/sqlx.go:654 shiori | github.com/go-shiori/shiori/internal/database.OpenSQLiteDatabase({0xc0000ca438, 0x15fc350}) shiori | /home/runner/work/shiori/shiori/internal/database/sqlite.go:27 +0xaa shiori | github.com/go-shiori/shiori/internal/cmd.openSQLiteDatabase() shiori | /home/runner/work/shiori/shiori/internal/cmd/root.go:117 +0x6a shiori | github.com/go-shiori/shiori/internal/cmd.openDatabase() shiori | /home/runner/work/shiori/shiori/internal/cmd/root.go:111 +0x79 shiori | github.com/go-shiori/shiori/internal/cmd.preRunRootHandler(0xc00021cc80, {0xbbd9a8, 0x0, 0x0}) shiori | /home/runner/work/shiori/shiori/internal/cmd/root.go:64 +0x116 shiori | github.com/spf13/cobra.(*Command).execute(0xc00021cc80, {0x15fbd98, 0x0, 0x0}) shiori | /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:840 +0x50a shiori | github.com/spf13/cobra.(*Command).ExecuteC(0xc0001a3400) shiori | /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:974 +0x3bc shiori | github.com/spf13/cobra.(*Command).Execute(...) shiori | /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:902 shiori | main.main() shiori | /home/runner/work/shiori/shiori/main.go:19 +0x1e ``` ## Notes This is probably a permission issue. I don't think you should create an extra user in a Docker container to run an application in default use cases (disclaimer: no Docker expert here, I'm just guessing). Just run mkdir /shiori instead of addgroup/adduser probably fixes this issue.
kerem 2026-02-25 23:33:54 +03:00
Author
Owner

@schlamar commented on GitHub (Dec 31, 2022):

Next issue on manual build, there is no alpine 3.16 on ghcr.io registry:

failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: ghcr.io/ghcri/alpine:3.16: not found

Probably you should use the official Docker hub here.

<!-- gh-comment-id:1368197134 --> @schlamar commented on GitHub (Dec 31, 2022): Next issue on manual build, there is no alpine 3.16 on ghcr.io registry: ``` failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: ghcr.io/ghcri/alpine:3.16: not found ``` Probably you should use the official Docker hub here.
Author
Owner

@schlamar commented on GitHub (Dec 31, 2022):

This Dockerfile worked for me

# build stage
FROM golang:1.19-alpine3.16 AS builder
WORKDIR /src
COPY . .
RUN go build -ldflags '-s -w'

# server image

FROM alpine:3.16
LABEL org.opencontainers.image.source https://github.com/go-shiori/shiori
COPY --from=builder /src/shiori /usr/bin/
RUN mkdir /shiori
WORKDIR /shiori
EXPOSE 8080
ENV SHIORI_DIR /shiori/
ENTRYPOINT ["/usr/bin/shiori"]
CMD ["serve"]
<!-- gh-comment-id:1368198162 --> @schlamar commented on GitHub (Dec 31, 2022): This Dockerfile worked for me ```Dockerfile # build stage FROM golang:1.19-alpine3.16 AS builder WORKDIR /src COPY . . RUN go build -ldflags '-s -w' # server image FROM alpine:3.16 LABEL org.opencontainers.image.source https://github.com/go-shiori/shiori COPY --from=builder /src/shiori /usr/bin/ RUN mkdir /shiori WORKDIR /shiori EXPOSE 8080 ENV SHIORI_DIR /shiori/ ENTRYPOINT ["/usr/bin/shiori"] CMD ["serve"] ```
Author
Owner

@schlamar commented on GitHub (Dec 31, 2022):

A workaround is to create the "data" directory before running docker. It is only broken if the data directory is created by docker / the container.

The documented docker run from Usage.md works because there is no subdirectory involved.

<!-- gh-comment-id:1368199065 --> @schlamar commented on GitHub (Dec 31, 2022): A workaround is to create the "data" directory before running docker. It is only broken if the data directory is created by docker / the container. The documented docker run from Usage.md works because there is no subdirectory involved.
Author
Owner

@fmartingr commented on GitHub (Jan 10, 2023):

Hey @schlamar, have you tried this with the dev tag? It works correctly on my machine (though is a Mac), but it automatically migrates the database and creates the appropriate folders:

docker run -v "$(pwd)/data:/shiori" -p 8080:8080 ghcr.io/go-shiori/shiori:dev

Though I tried with latest and it works as well in my case (and I have it setup on a server with a volume too, have been working for a while now). Which host operating system are you using?

<!-- gh-comment-id:1377793463 --> @fmartingr commented on GitHub (Jan 10, 2023): Hey @schlamar, have you tried this with the `dev` tag? It works correctly on my machine (though is a Mac), but it automatically migrates the database and creates the appropriate folders: ``` docker run -v "$(pwd)/data:/shiori" -p 8080:8080 ghcr.io/go-shiori/shiori:dev ``` Though I tried with `latest` and it works as well in my case (and I have it setup on a server with a volume too, have been working for a while now). Which host operating system are you using?
Author
Owner

@therysin commented on GitHub (Jan 11, 2023):

Hey @schlamar, have you tried this with the dev tag? It works correctly on my machine (though is a Mac), but it automatically migrates the database and creates the appropriate folders:

docker run -v "$(pwd)/data:/shiori" -p 8080:8080 ghcr.io/go-shiori/shiori:dev

Though I tried with latest and it works as well in my case (and I have it setup on a server with a volume too, have been working for a while now). Which host operating system are you using?

Also having the same issue with both the latest and dev builds. I’m on Ubuntu 22.04.1 arm (Oracle Cloud).

Seemed like an awesome project but the container won’t start unfortunately (same out of memory error ).

I should also note that I’m using docker-compose(same file as OP to test), and it’s running behind traefik.

If the mounted volume lines are commented or removed from the compose file, shiori works fine (without data persistence).

<!-- gh-comment-id:1378905385 --> @therysin commented on GitHub (Jan 11, 2023): > Hey @schlamar, have you tried this with the `dev` tag? It works correctly on my machine (though is a Mac), but it automatically migrates the database and creates the appropriate folders: > > ``` > docker run -v "$(pwd)/data:/shiori" -p 8080:8080 ghcr.io/go-shiori/shiori:dev > ``` > > Though I tried with `latest` and it works as well in my case (and I have it setup on a server with a volume too, have been working for a while now). Which host operating system are you using? Also having the same issue with both the latest and dev builds. I’m on Ubuntu 22.04.1 arm (Oracle Cloud). Seemed like an awesome project but the container won’t start unfortunately (same out of memory error ). I should also note that I’m using docker-compose(same file as OP to test), and it’s running behind traefik. If the mounted volume lines are commented or removed from the compose file, shiori works fine (without data persistence).
Author
Owner

@schlamar commented on GitHub (Jan 11, 2023):

Dev fails, too:

$ sudo docker run -v "$(pwd)/data:/shiori" -p 8080:8080 ghcr.io/go-shiori/shiori:dev
Unable to find image 'ghcr.io/go-shiori/shiori:dev' locally
dev: Pulling from go-shiori/shiori
...
Status: Downloaded newer image for ghcr.io/go-shiori/shiori:dev
Failed to open database: unable to open database file: out of memory (14)

Running Arch Linux with current docker release:

$ docker --version
Docker version 20.10.22, build 3a2c30b63a
<!-- gh-comment-id:1379372334 --> @schlamar commented on GitHub (Jan 11, 2023): Dev fails, too: ```console $ sudo docker run -v "$(pwd)/data:/shiori" -p 8080:8080 ghcr.io/go-shiori/shiori:dev Unable to find image 'ghcr.io/go-shiori/shiori:dev' locally dev: Pulling from go-shiori/shiori ... Status: Downloaded newer image for ghcr.io/go-shiori/shiori:dev Failed to open database: unable to open database file: out of memory (14) ``` Running Arch Linux with current docker release: ```console $ docker --version Docker version 20.10.22, build 3a2c30b63a ```
Author
Owner

@Denow commented on GitHub (Jan 18, 2023):

I have the same problem with Ubuntu 20.04.5 LTS (x86).

$ docker --version
Docker version 20.10.22, build 3a2c30b
<!-- gh-comment-id:1386803363 --> @Denow commented on GitHub (Jan 18, 2023): I have the same problem with Ubuntu 20.04.5 LTS (x86). ``` $ docker --version Docker version 20.10.22, build 3a2c30b ```
Author
Owner

@fmartingr commented on GitHub (Jan 18, 2023):

I have the same problem with Ubuntu 20.04.5 LTS (x86).

$ docker --version
Docker version 20.10.22, build 3a2c30b

Are you running ubuntu over oracle cloud as well? Do you have the exact same error as @schlamar ?

<!-- gh-comment-id:1386885639 --> @fmartingr commented on GitHub (Jan 18, 2023): > I have the same problem with Ubuntu 20.04.5 LTS (x86). > > ``` > $ docker --version > Docker version 20.10.22, build 3a2c30b > ``` Are you running ubuntu over oracle cloud as well? Do you have the exact same error as @schlamar ?
Author
Owner

@Denow commented on GitHub (Jan 18, 2023):

No, I am running Ubuntu in a vm on an onprem esxi host. The error I am receiving is the same as @schlamar:

Failed to open database: unable to open database file: out of memory (14)

The container keeps restarting after this error message and the data dir is empty so it does look like a permission issue.

<!-- gh-comment-id:1386897015 --> @Denow commented on GitHub (Jan 18, 2023): No, I am running Ubuntu in a vm on an onprem esxi host. The error I am receiving is the same as @schlamar: `Failed to open database: unable to open database file: out of memory (14)` The container keeps restarting after this error message and the data dir is empty so it does look like a permission issue.
Author
Owner

@Denow commented on GitHub (Jan 18, 2023):

Nevermind I was mounting the wrong volume...

<!-- gh-comment-id:1386912205 --> @Denow commented on GitHub (Jan 18, 2023): Nevermind I was mounting the wrong volume...
Author
Owner

@therysin commented on GitHub (Jan 18, 2023):

Nevermind I was mounting the wrong volume...

Can you expand further on what was wrong/ what volume you mounted to fix?

<!-- gh-comment-id:1386994648 --> @therysin commented on GitHub (Jan 18, 2023): > Nevermind I was mounting the wrong volume... Can you expand further on what was wrong/ what volume you mounted to fix?
Author
Owner

@Denow commented on GitHub (Jan 18, 2023):

I was using the following tutorial for deploying shiori: https://noted.lol/tutorial-setting-up-shiori.
The docker-compose file in this tutorial changes the shiori data dir with an environment variable to SHIORI_DIR=/data and creates a volume for /data. Using the same docker-compose file gave me the Failed to open database: unable to open database file: out of memory (14) error.

version: "3"
services:
  shiori:
    image: ghcr.io/go-shiori/shiori
    container_name: shiori
    environment:
      - PUID=1000
      - PGID=1000
      - SHIORI_DIR=/data
    ports:
      - 8080:8080
    restart: unless-stopped
    volumes:
      - /data/shiori:/data

I removed the environment variable so shiori would write the db to the default location /shiori and also changed the volume path to the same location, now everything is working great.

<!-- gh-comment-id:1387079953 --> @Denow commented on GitHub (Jan 18, 2023): I was using the following tutorial for deploying shiori: https://noted.lol/tutorial-setting-up-shiori. The docker-compose file in this tutorial changes the shiori data dir with an environment variable to SHIORI_DIR=/data and creates a volume for /data. Using the same docker-compose file gave me the `Failed to open database: unable to open database file: out of memory (14)` error. ``` version: "3" services: shiori: image: ghcr.io/go-shiori/shiori container_name: shiori environment: - PUID=1000 - PGID=1000 - SHIORI_DIR=/data ports: - 8080:8080 restart: unless-stopped volumes: - /data/shiori:/data ``` I removed the environment variable so shiori would write the db to the default location /shiori and also changed the volume path to the same location, now everything is working great.
Author
Owner

@therysin commented on GitHub (Jan 18, 2023):

I was using the following tutorial for deploying shiori: https://noted.lol/tutorial-setting-up-shiori. The docker-compose file in this tutorial changes the shiori data dir with an environment variable to SHIORI_DIR=/data and creates a volume for /data. Using the same docker-compose file gave me the Failed to open database: unable to open database file: out of memory (14) error.

version: "3"
services:
  shiori:
    image: ghcr.io/go-shiori/shiori
    container_name: shiori
    environment:
      - PUID=1000
      - PGID=1000
      - SHIORI_DIR=/data
    ports:
      - 8080:8080
    restart: unless-stopped
    volumes:
      - /data/shiori:/data

I removed the environment variable so shiori would write the db to the default location /shiori and also changed the volume path to the same location, now everything is working great.

Thanks. Was hoping it would be something I overlooked, but I’m already using the default shiori location 🥲. I’m using the exact same compose file as the OP.

<!-- gh-comment-id:1387093643 --> @therysin commented on GitHub (Jan 18, 2023): > I was using the following tutorial for deploying shiori: https://noted.lol/tutorial-setting-up-shiori. The docker-compose file in this tutorial changes the shiori data dir with an environment variable to SHIORI_DIR=/data and creates a volume for /data. Using the same docker-compose file gave me the `Failed to open database: unable to open database file: out of memory (14)` error. > > ``` > version: "3" > services: > shiori: > image: ghcr.io/go-shiori/shiori > container_name: shiori > environment: > - PUID=1000 > - PGID=1000 > - SHIORI_DIR=/data > ports: > - 8080:8080 > restart: unless-stopped > volumes: > - /data/shiori:/data > ``` > > I removed the environment variable so shiori would write the db to the default location /shiori and also changed the volume path to the same location, now everything is working great. Thanks. Was hoping it would be something I overlooked, but I’m already using the default shiori location 🥲. I’m using the exact same compose file as the OP.
Author
Owner

@therysin commented on GitHub (Jan 19, 2023):

I ended up using a regular docker volume for persistence and not a bind mount.

Probably a temp fix, but works out in the short term.

<!-- gh-comment-id:1396260891 --> @therysin commented on GitHub (Jan 19, 2023): I ended up using a regular docker volume for persistence and not a bind mount. Probably a temp fix, but works out in the short term.
Author
Owner

@fmartingr commented on GitHub (Jan 21, 2023):

I have been trying to run and connect to an Oracle Cloud ubuntu instance for 15 minutes now. I wanted to try over there just in case, but this works on my mac/linux machines. You should be able to create a mounted volume if you have enough permissions over the folder being mounted. I'll leave this open for now, in case someone else can chip in here.

<!-- gh-comment-id:1399209110 --> @fmartingr commented on GitHub (Jan 21, 2023): I have been trying to run and connect to an Oracle Cloud ubuntu instance for 15 minutes now. I wanted to try over there just in case, but this works on my mac/linux machines. You should be able to create a mounted volume if you have enough permissions over the folder being mounted. I'll leave this open for now, in case someone else can chip in here.
Author
Owner

@danielnbalasoiu commented on GitHub (Feb 25, 2023):

I had the same error and I fixed it by creating the local directory and then set the correct permissions expected by shiori.

mkdir ./shiori-data
chown -R 1000:1000 ./shiori-data
<!-- gh-comment-id:1445040642 --> @danielnbalasoiu commented on GitHub (Feb 25, 2023): I had the same error and I fixed it by creating the local directory and then set the correct permissions [expected by shiori](https://github.com/go-shiori/shiori/blob/master/Dockerfile#L12). ```shell mkdir ./shiori-data chown -R 1000:1000 ./shiori-data ```
Author
Owner

@milosimpson commented on GitHub (Oct 23, 2023):

Also running into this issue trying to get shiori running in docker on a Synology NAS.

If I don't specify a data volume, I can get v.1.5.5 to boot because shiori will make a sqlite db in the/shiori dir inside the docker container, which works but isn't great as there is no upgrade path as the db is "trapped" inside the container.

When I try to give it an external volume/mount, it fails to be able to make the sqllite db file because user 1000/shiori doesn't have write permissions to the mounted directory.

I am guessing you want to use a non-root user for security reasons since this service exposes an http service.

I think the fix might be to: while booting, as root chown the SHIORI_DIR to the uid 1000/shiori user, then switch to the uid 1000/shiori user.

I tried to test this by pre-populating a shiori.db file in the mount volumn/directory, to get past the boot failure so that I could shell into the container and try to run sudo chown -R 1000:1000 on the mounted dir to see if that would temporarily make things work, but the image does not contain sudo.

Shelled into the container (v1.5.5) to look at who owns /shiori/shiori.db on a setup where there is no external volume mount.
User shiori owns the file.
Screenshot 2023-10-22 at 10 04 41 PM

Shelled into the container with an external volume mount with pre-existing shiori.db file (v1.5.3 that it doesn't try to migrate db on boot). User root owns that file/directory which means it isn't usable.
Screenshot 2023-10-22 at 10 21 53 PM

<!-- gh-comment-id:1774368245 --> @milosimpson commented on GitHub (Oct 23, 2023): Also running into this issue trying to get shiori running in docker on a Synology NAS. If I don't specify a data volume, I can get v.1.5.5 to boot because shiori will make a sqlite db in the`/shiori` dir inside the docker container, which works but isn't great as there is no upgrade path as the db is "trapped" inside the container. When I try to give it an external volume/mount, it fails to be able to make the sqllite db file because user 1000/shiori doesn't have write permissions to the mounted directory. I am guessing you want to use a non-root user for security reasons since this service exposes an http service. I think the fix might be to: while booting, as root chown the SHIORI_DIR to the uid 1000/shiori user, then switch to the uid 1000/shiori user. I tried to test this by pre-populating a `shiori.db` file in the mount volumn/directory, to get past the boot failure so that I could shell into the container and try to run `sudo chown -R 1000:1000` on the mounted dir to see if that would temporarily make things work, but the image does not contain `sudo`. Shelled into the container (v1.5.5) to look at who owns /shiori/shiori.db on a setup where there is no external volume mount. User `shiori` owns the file. ![Screenshot 2023-10-22 at 10 04 41 PM](https://github.com/go-shiori/shiori/assets/1203207/2deb8e58-4788-4dc2-8f9d-f1572ee93ec0) Shelled into the container with an external volume mount with pre-existing shiori.db file (v1.5.3 that it doesn't try to migrate db on boot). User root owns that file/directory which means it isn't usable. ![Screenshot 2023-10-22 at 10 21 53 PM](https://github.com/go-shiori/shiori/assets/1203207/8aecdd5d-78d2-4f69-ab24-800c1d9724ce)
Author
Owner

@GeekyNinja2003 commented on GitHub (Oct 23, 2023):

Hi ,
Thank you for reaching out regarding the Shiori Docker issue on Synology
NAS. I appreciate your detailed explanation of the problem.

It seems like you’re encountering a permissions issue when trying to use an
external volume for Shiori in Docker. You’re correct in assuming that we
aim to use a non-root user for security reasons, especially for services
that expose HTTP. To address this issue, your suggested approach could be a
viable solution.

Here’s a step-by-step breakdown: 1. While booting as root: • Change
ownership of the SHIORI_DIR to the UID 1000/shiori user. • Switch to the
UID 1000/shiori user. This should resolve the permission problem and allow
Shiori to create the SQLite DB file on the external volume. Unfortunately,
the image does not contain sudo, as you’ve mentioned. In this case, you
might need to use other methods to change ownership, such as using the
chown command with the --no-dereference option.

Feel free to test this approach and let us know how it works for you. If
you have any further questions or encounter any issues during this process,
please don’t hesitate to reach out for additional assistance.

Best regards, Jerry;-)

On Mon, 23 Oct 2023 at 02:18 Milo Simpson @.***> wrote:

Also running into this issue trying to get shiori running in docker on a
Synology NAS.

If I don't specify a data volume, I can get v.1.5.5 to boot because shiori
will make a sqlite db in the/shiori dir inside the docker container,
which works but isn't great as there is no upgrade path as the db is
"trapped" inside the container.

When I try to give it an external volume/mount, it fails to be able to
make the sqllite db file because user 1000/shiori doesn't have permissions
to the mount.

I am guessing you want to use a non-root user for security reasons / since
this service exposes an http service.

I think the fix might be to: while booting as root, chown the SHIORI_DIR
to the uid 1000/shiori user, then switch to the uid 1000/shiori user.

I tried to test this by pre-populating a shiori.db file in the mount dir,
to at least get past the boot failure on v1.5.5 so that I could shell into
the container and try to run sudo chown -R 1000:1000 on the mounted dir
to see if that would temporarily make things work, but the image does not
contain sudo.


Reply to this email directly, view it on GitHub
https://github.com/go-shiori/shiori/issues/546#issuecomment-1774368245,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A7OXXSAQUYIE3QUV7HG6PU3YAXOZ3AVCNFSM6AAAAAATNPG3L6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZUGM3DQMRUGU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

<!-- gh-comment-id:1774951559 --> @GeekyNinja2003 commented on GitHub (Oct 23, 2023): Hi , Thank you for reaching out regarding the Shiori Docker issue on Synology NAS. I appreciate your detailed explanation of the problem. It seems like you’re encountering a permissions issue when trying to use an external volume for Shiori in Docker. You’re correct in assuming that we aim to use a non-root user for security reasons, especially for services that expose HTTP. To address this issue, your suggested approach could be a viable solution. Here’s a step-by-step breakdown: 1. While booting as root: • Change ownership of the SHIORI_DIR to the UID 1000/shiori user. • Switch to the UID 1000/shiori user. This should resolve the permission problem and allow Shiori to create the SQLite DB file on the external volume. Unfortunately, the image does not contain sudo, as you’ve mentioned. In this case, you might need to use other methods to change ownership, such as using the chown command with the --no-dereference option. Feel free to test this approach and let us know how it works for you. If you have any further questions or encounter any issues during this process, please don’t hesitate to reach out for additional assistance. Best regards, Jerry;-) On Mon, 23 Oct 2023 at 02:18 Milo Simpson ***@***.***> wrote: > Also running into this issue trying to get shiori running in docker on a > Synology NAS. > > If I don't specify a data volume, I can get v.1.5.5 to boot because shiori > will make a sqlite db in the/shiori dir inside the docker container, > which works but isn't great as there is no upgrade path as the db is > "trapped" inside the container. > > When I try to give it an external volume/mount, it fails to be able to > make the sqllite db file because user 1000/shiori doesn't have permissions > to the mount. > > I am guessing you want to use a non-root user for security reasons / since > this service exposes an http service. > > I think the fix might be to: while booting as root, chown the SHIORI_DIR > to the uid 1000/shiori user, then switch to the uid 1000/shiori user. > > I tried to test this by pre-populating a shiori.db file in the mount dir, > to at least get past the boot failure on v1.5.5 so that I could shell into > the container and try to run sudo chown -R 1000:1000 on the mounted dir > to see if that would temporarily make things work, but the image does not > contain sudo. > > — > Reply to this email directly, view it on GitHub > <https://github.com/go-shiori/shiori/issues/546#issuecomment-1774368245>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/A7OXXSAQUYIE3QUV7HG6PU3YAXOZ3AVCNFSM6AAAAAATNPG3L6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZUGM3DQMRUGU> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> >
Author
Owner

@fmartingr commented on GitHub (Oct 29, 2023):

I need to unify both Dockerfiles, since we use one for the CI (production builds) and one for development, and that makes no sense right now. That said, Shiori should work using a mounted volume without much issue as I have been running it like that for some time now.

Maybe allowing to customize the user and group IDs on the image via environment variables would work well, but that something I need to investigate and make backwards compatible, so it will be after we release 1.6.0, since that already contains enough changes as it is.

<!-- gh-comment-id:1784054754 --> @fmartingr commented on GitHub (Oct 29, 2023): I need to unify both Dockerfiles, since we use one for the CI (production builds) and one for development, and that makes no sense right now. That said, Shiori should work using a mounted volume without much issue as I have been running it like that for some time now. Maybe allowing to customize the user and group IDs on the image via environment variables would work well, but that something I need to investigate and make backwards compatible, so it will be after we release 1.6.0, since that already contains enough changes as it is.
Author
Owner

@milosimpson commented on GitHub (Nov 12, 2023):

Figured out my problem. It is a Synology NAS specific issue. The fix was to create a specific non-root user with access to the mount directory ala https://drfrankenstein.co.uk/step-2-setting-up-a-restricted-docker-user-and-obtaining-ids/ and then use that user with Docker Compose. The Synology "Container Manager" app is basically Synology's Portainer, and it can run docker compose yaml files.

version: '3.3'
services:
  shiori:
    image: ghcr.io/go-shiori/shiori
    container_name: shiori
    user: UID:GID   // this is the important part; set the user and group that have access to  /volume2/docker/shiori
    ports:
      - 8080:8080
    restart: unless-stopped
    volumes:
      - /volume2/docker/shiori:/shiori           
      // note you have to specify the the "real" path on host not the nice Synology share name

It seemed like most suggestions on the internet were to make user and group that the docker image wants on the host OS, which is easy to do on Linux systems, but not on the Synology.

The catch is that it seems like only compose and the yaml file approach can set the user and group id, not the "docker" command line or simple docker run UIs in the Synology.

The one nice thing that Shiroi could do is take UID and PID to use as environment variables, that way it could be run w/out compose.

<!-- gh-comment-id:1806989347 --> @milosimpson commented on GitHub (Nov 12, 2023): Figured out my problem. It is a Synology NAS specific issue. The fix was to create a specific non-root user with access to the mount directory ala https://drfrankenstein.co.uk/step-2-setting-up-a-restricted-docker-user-and-obtaining-ids/ and then use that user with Docker Compose. The Synology "Container Manager" app is basically Synology's Portainer, and it can run docker compose yaml files. ``` version: '3.3' services: shiori: image: ghcr.io/go-shiori/shiori container_name: shiori user: UID:GID // this is the important part; set the user and group that have access to /volume2/docker/shiori ports: - 8080:8080 restart: unless-stopped volumes: - /volume2/docker/shiori:/shiori // note you have to specify the the "real" path on host not the nice Synology share name ``` It seemed like most suggestions on the internet were to make user and group that the docker image wants on the host OS, which is easy to do on Linux systems, but not on the Synology. The catch is that it seems like only compose and the yaml file approach can set the user and group id, not the "docker" command line or simple docker run UIs in the Synology. The one nice thing that Shiroi could do is take UID and PID to use as environment variables, that way it could be run w/out compose.
Author
Owner

@danielyrovas commented on GitHub (Apr 8, 2024):

I ran into this issue using podman running in rootless mode. I manged to solve it by building a container as below:

FROM ghcr.io/go-shiori/shiori:latest AS source

FROM docker.io/alpine:3.19
LABEL org.opencontainers.image.source https://github.com/go-shiori/shiori
COPY --from=source /usr/bin/shiori /usr/bin/

EXPOSE 8080
WORKDIR /shiori
ENV SHIORI_DIR /shiori/
ENTRYPOINT ["/usr/bin/shiori"]
CMD ["server"]
<!-- gh-comment-id:2041708635 --> @danielyrovas commented on GitHub (Apr 8, 2024): I ran into this issue using podman running in rootless mode. I manged to solve it by building a container as below: ```Containerfile FROM ghcr.io/go-shiori/shiori:latest AS source FROM docker.io/alpine:3.19 LABEL org.opencontainers.image.source https://github.com/go-shiori/shiori COPY --from=source /usr/bin/shiori /usr/bin/ EXPOSE 8080 WORKDIR /shiori ENV SHIORI_DIR /shiori/ ENTRYPOINT ["/usr/bin/shiori"] CMD ["server"] ```
Author
Owner

@fmartingr commented on GitHub (May 12, 2024):

I just tested the image on PR #907 on a linux laptop and it required no further modifications in order to mount a local path:

$ cd $(mktemp -d)
$ docker run -v ./data:/shiori -p 8080:8080 ghcr.io/go-shiori/shiori:pr-907
... server running, add bookmarks
^C
$ ls data
shiori.db
$ sqlite3 data/shiori.db "SELECT COUNT(*) FROM bookmark;
1
<!-- gh-comment-id:2106293676 --> @fmartingr commented on GitHub (May 12, 2024): I just tested the image on PR #907 on a linux laptop and it required no further modifications in order to mount a local path: ``` $ cd $(mktemp -d) $ docker run -v ./data:/shiori -p 8080:8080 ghcr.io/go-shiori/shiori:pr-907 ... server running, add bookmarks ^C $ ls data shiori.db $ sqlite3 data/shiori.db "SELECT COUNT(*) FROM bookmark; 1 ```
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/shiori#301
No description provided.