[GH-ISSUE #172] How to backup/restore the database?

kerem commented

2026-02-27 08:15:04 +03:00

Owner

Originally created by @tetricky on GitHub (May 28, 2022).
Original GitHub issue: https://github.com/lldap/lldap/issues/172

Originally assigned to: @tetricky on GitHub.

Is there a process defined for backing up (restoring) the database? It's awkward for me to automate stopping the container as part of my backup process (I use a scripted borgmatic), and on the server I'm using I can't use LVM storage snapshots. I'd like a way to backup and restore without possible borkage if a user is mid-password reset or something that is writing (I'm using authelia as a frontend).

I think the database is sqlite? The docker (although I'm running under podman) container doesn't appear to have sqlite tools built in. A postgres storage backend would solve this as part of my current workflow (borgmatic automates database dumps as part of it's backup), and I know there is some work towards that, but until then I would appreciate some guidance on an appropriate backup process.

By the way, the level of functionality, and ease of use, of this project absolutely hits the sweet spot for my needs. Very much appreciated.

Originally created by @tetricky on GitHub (May 28, 2022). Original GitHub issue: https://github.com/lldap/lldap/issues/172 Originally assigned to: @tetricky on GitHub. Is there a process defined for backing up (restoring) the database? It's awkward for me to automate stopping the container as part of my backup process (I use a scripted borgmatic), and on the server I'm using I can't use LVM storage snapshots. I'd like a way to backup and restore without possible borkage if a user is mid-password reset or something that is writing (I'm using authelia as a frontend). I think the database is sqlite? The docker (although I'm running under podman) container doesn't appear to have sqlite tools built in. A postgres storage backend would solve this as part of my current workflow (borgmatic automates database dumps as part of it's backup), and I know there is some work towards that, but until then I would appreciate some guidance on an appropriate backup process. By the way, the level of functionality, and ease of use, of this project absolutely hits the sweet spot for my needs. Very much appreciated.

kerem added the

documentation

question

labels

2026-02-27 08:15:04 +03:00

kerem commented

2026-02-27 08:15:04 +03:00

Author

Owner

@nitnelave commented on GitHub (May 28, 2022):

The proper way to backup the database would be to run sqlite3 users.db ".backup 'backup_users.db'" (source).

However, as you said, the provided docker image doesn't have sqlite3 or cron installed. It should be fairly straightforward to install them from the provided docker image. A bit trickier would be to make the cron configurable with env variables for other users.

I would start by making your own docker image based on nitnelave/lldap with the right tools installed, and if you feel like it also make a configurable cron job. That would be a great addition to the repository, I would welcome a contribution! Then I can auto build and deploy some nitnelave/lldap_sqlite_backup image.

I'm glad the project is useful!

@nitnelave commented on GitHub (May 28, 2022): The proper way to backup the database would be to run `sqlite3 users.db ".backup 'backup_users.db'"` ([source](https://stackoverflow.com/questions/25675314/how-to-backup-sqlite-database)). However, as you said, the provided docker image doesn't have `sqlite3` or `cron` installed. It should be fairly straightforward to install them from the provided docker image. A bit trickier would be to make the cron configurable with env variables for other users. I would start by making your own docker image based on `nitnelave/lldap` with the right tools installed, and if you feel like it also make a configurable cron job. That would be a great addition to the repository, I would welcome a contribution! Then I can auto build and deploy some `nitnelave/lldap_sqlite_backup` image. I'm glad the project is useful!

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (May 29, 2022):

I've used this before on other projects.It's very comprehensive (many different database types and options eg. S3 storage), but suffers from being equally big (circa 150MB).
tiredofit/docker-db-backup

This is looks like it might be a better candidate for this project (circa 4MB) - sqlite only, and a lot more limited options (only db dump to local filesystem, so needs a separate backup system), and I'll try to have a go at setting it up, when I get around to that bit, and report back.
efrecon/sqlite-backup

I would suggest this approach of a container with the tools to make dumps/backups (mount the sqlite.db location to both containers), as it keeps your image small and tightly focused. I've not really used sqlite in production before though, so I'll need to do some testing of this approach.

@tetricky commented on GitHub (May 29, 2022): I've used this before on other projects.It's very comprehensive (many different database types and options eg. S3 storage), but suffers from being equally big (circa 150MB). [tiredofit/docker-db-backup](https://github.com/tiredofit/docker-db-backup) This is looks like it might be a better candidate for this project (circa 4MB) - sqlite only, and a lot more limited options (only db dump to local filesystem, so needs a separate backup system), and I'll try to have a go at setting it up, when I get around to that bit, and report back. [efrecon/sqlite-backup](https://github.com/efrecon/sqlite-backup) I would suggest this approach of a container with the tools to make dumps/backups (mount the sqlite.db location to both containers), as it keeps your image small and tightly focused. I've not really used sqlite in production before though, so I'll need to do some testing of this approach.

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@nitnelave commented on GitHub (May 29, 2022):

That sounds like a nice approach, separating the concerns. I might end up making an image with both built in for ease of use, but let's start with independent images.

@nitnelave commented on GitHub (May 29, 2022): That sounds like a nice approach, separating the concerns. I might end up making an image with both built in for ease of use, but let's start with independent images.

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (Jun 16, 2022):

I haven't ignored this. I've actually done a bit of work on various possible approaches. The conclusion I have reached, is that you don't want this stuff in your image.....it will add size and a potential overhead, and will either be mostly redundant (or not meet the use case) in most cases.
I have made a gist that is part guide, part discussion around the issue.
sqlite_backup.md

@tetricky commented on GitHub (Jun 16, 2022): I haven't ignored this. I've actually done a bit of work on various possible approaches. The conclusion I have reached, is that you don't want this stuff in your image.....it will add size and a potential overhead, and will either be mostly redundant (or not meet the use case) in most cases. I have made a gist that is part guide, part discussion around the issue. [sqlite_backup.md](https://gist.github.com/tetricky/d2b177bfea1757a36db2d94412fc3474)

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@nitnelave commented on GitHub (Jun 16, 2022):

Thanks, that guide is very useful!
In order to simplify deployment, would you be able to provide an example docker-compose config with LLDAP + dockron running sqlite-backup every day? That's something that I can easily include in the README, and link to your guide for more details.

@nitnelave commented on GitHub (Jun 16, 2022): Thanks, that guide is very useful! In order to simplify deployment, would you be able to provide an example docker-compose config with LLDAP + dockron running sqlite-backup every day? That's something that I can easily include in the README, and link to your guide for more details.

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (Jun 16, 2022):

Possibly, but maybe not. I don't actually use docker any more, and I avoid docker-compose (even though supported in recent versions of podman). Instead I use podman pods, and automate podman services with systemd services/timers. For a number of reasons, philosophical and practical (for example podman can generate deployment manifests to k8s).

I could knock a guide up to deploy the equivalent in a podman/pod with a timer service?

@tetricky commented on GitHub (Jun 16, 2022): Possibly, but maybe not. I don't actually use docker any more, and I avoid docker-compose (even though supported in recent versions of podman). Instead I use podman pods, and automate podman services with systemd services/timers. For a number of reasons, philosophical and practical (for example podman can generate deployment manifests to k8s). I could knock a guide up to deploy the equivalent in a podman/pod with a timer service?

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@nitnelave commented on GitHub (Jun 16, 2022):

That would be useful, yes. Thanks!

@nitnelave commented on GitHub (Jun 16, 2022): That would be useful, yes. Thanks!

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@nitnelave commented on GitHub (Aug 2, 2022):

@tetricky Do you have a guide for podman for backup? I can adapt it to docker.

@nitnelave commented on GitHub (Aug 2, 2022): @tetricky Do you have a guide for podman for backup? I can adapt it to docker.

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (Aug 2, 2022):

It's on my todo list, but not at the top (I have a working backup based on host level calls and borgmatic).

I am looking at ttionya/vaultwarden-backup, modified to default to saving backup locally, or remotely (by modifying the rclone config). I am planning on adding in a notify by xmpp as well as smtp option (this suits my use case, and to make the exercise worthwhile for me).

You'll have to bear with me though, I made a start, but then other priorities intervened.

@tetricky commented on GitHub (Aug 2, 2022): It's on my todo list, but not at the top (I have a working backup based on host level calls and borgmatic). I am looking at [ttionya/vaultwarden-backup](https://github.com/ttionya/vaultwarden-backup), modified to default to saving backup locally, or remotely (by modifying the rclone config). I am planning on adding in a notify by xmpp as well as smtp option (this suits my use case, and to make the exercise worthwhile for me). You'll have to bear with me though, I made a start, but then other priorities intervened.

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@nitnelave commented on GitHub (Aug 2, 2022):

No worries, we're all doing this on our free time :) I was just checking if you still wanted to with on it or if I should try to take it over. I'm happy leaving it in your competent hands!

@nitnelave commented on GitHub (Aug 2, 2022): No worries, we're all doing this on our free time :) I was just checking if you still wanted to with on it or if I should try to take it over. I'm happy leaving it in your competent hands!

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (Aug 2, 2022):

My intention is to complete it in the mid term - a couple more weeks. (it's always a couple more weeks!)

@tetricky commented on GitHub (Aug 2, 2022): My intention is to complete it in the mid term - a couple more weeks. (it's always a couple more weeks!)

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (Oct 2, 2022):

I've spent some time on this over the last week. I'm pretty close to something that I'm happy with. I just have some testing to do. I would hope to publish this coming week.

@tetricky commented on GitHub (Oct 2, 2022): I've spent some time on this over the last week. I'm pretty close to something that I'm happy with. I just have some testing to do. I would hope to publish this coming week.

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (Oct 3, 2022):

I have now completed a provisional container that might be useful for lldap backup. This is set up by default to dump the sqlite database to the local filesystem, but can accept any rclone storage backend. Notifications are possible by email or xmpp. CRON handles automatic backups within the container.
On github: tetricky/sqlite-backup. On docker hub: tetricky/sqlite-backup.

@tetricky commented on GitHub (Oct 3, 2022): I have now completed a provisional container that might be useful for lldap backup. This is set up by default to dump the sqlite database to the local filesystem, but can accept any rclone storage backend. Notifications are possible by email or xmpp. CRON handles automatic backups within the container. On github: [tetricky/sqlite-backup](https://github.com/tetricky/sqlite-backup). On docker hub: [tetricky/sqlite-backup](https://hub.docker.com/r/tetricky/sqlite-backup).

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@nitnelave commented on GitHub (Oct 4, 2022):

Nice! Do you want to do a PR linking your container from the README?

@nitnelave commented on GitHub (Oct 4, 2022): Nice! Do you want to do a PR linking your container from the README?

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (Oct 4, 2022):

I've added a variant that also backs up the lldap_config.toml and private_key. In the github branch lldap.
On docker hub: tetricky/lldap-backup

@tetricky commented on GitHub (Oct 4, 2022): I've added a variant that also backs up the `lldap_config.toml` and `private_key`. In the github branch lldap. On docker hub: [tetricky/lldap-backup](https://hub.docker.com/r/tetricky/lldap-backup)

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (Oct 4, 2022):

....sorry, you replied while I was doing that. I'm new to collaboration on github. not sure how to do that.

@tetricky commented on GitHub (Oct 4, 2022): ....sorry, you replied while I was doing that. I'm new to collaboration on github. not sure how to do that.

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@nitnelave commented on GitHub (Oct 4, 2022):

If possible it would be nice to have a configurable list of files to backup, in case people want to name their config differently or back up a jwt secret that was written to a file. If you don't have a simple way to do that, that's fine.

As for collaboration on GH, the common way is to fork my repo, push a commit on your fork with your changes (can be on the main branch or another one) and then create a PR merging your branch to my main branch (you'll see that if you push a commit to your branch, you'll have a "Contribute" button on the top right of your main repo view).

Or if you want a quick and "dirty" way of doing a simple change, you can just click on the "edit" button on the README file, and it'll do most of what I said above at once for you.

@nitnelave commented on GitHub (Oct 4, 2022): If possible it would be nice to have a configurable list of files to backup, in case people want to name their config differently or back up a jwt secret that was written to a file. If you don't have a simple way to do that, that's fine. As for collaboration on GH, the common way is to fork my repo, push a commit on your fork with your changes (can be on the main branch or another one) and then create a PR merging your branch to my main branch (you'll see that if you push a commit to your branch, you'll have a "Contribute" button on the top right of your main repo view). Or if you want a quick and "dirty" way of doing a simple change, you can just click on the "edit" button on the README file, and it'll do most of what I said above at once for you.

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (Oct 4, 2022):

...being able to rename the files is do-able. Because of the way the pruning of the database files from the rclone storage backend works the key is to know how many files there could be, what default names are, and what can have their names changed. I'm vaguely aware that there maybe certificate files for a possible ssl (?) connection to the lldap backend. I don't really use that, so I'm not au fait with what the possible configuration files are.

It would seem sensible for me to address those, before drawing others attention to it.

@tetricky commented on GitHub (Oct 4, 2022): ...being able to rename the files is do-able. Because of the way the pruning of the database files from the rclone storage backend works the key is to know how many files there could be, what default names are, and what can have their names changed. I'm vaguely aware that there maybe certificate files for a possible ssl (?) connection to the lldap backend. I don't really use that, so I'm not au fait with what the possible configuration files are. It would seem sensible for me to address those, before drawing others attention to it.

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@tetricky commented on GitHub (Oct 4, 2022):

I've added an image that allows for renaming the config files that I am aware of. It's in the lldap branch on github, and I've also pushed containers to docker hub.
On github: tetricky/sqlite-backup. On docker hub: tetricky/lldap-backup

@tetricky commented on GitHub (Oct 4, 2022): I've added an image that allows for renaming the config files that I am aware of. It's in the lldap branch on github, and I've also pushed containers to docker hub. On github: [tetricky/sqlite-backup](https://github.com/tetricky/sqlite-backup). On docker hub: [tetricky/lldap-backup](https://hub.docker.com/r/tetricky/lldap-backup)

kerem commented

2026-02-27 08:15:05 +03:00

Author

Owner

@nitnelave commented on GitHub (Aug 3, 2023):

@tetricky do you think you could add your documentation to the README? So the next users can more easily find it.

@nitnelave commented on GitHub (Aug 3, 2023): @tetricky do you think you could add your documentation to the README? So the next users can more easily find it.

kerem referenced this issue

2026-02-27 08:17:51 +03:00

[PR #69] [MERGED] workflows: cache dependency builds and get code coverage #522

Rows
Columns

[GH-ISSUE #172] How to backup/restore the database? #69