[GH-ISSUE #75] Feature: Import / Export for backup purposes #69

Open
opened 2026-03-02 11:46:14 +03:00 by kerem · 18 comments
Owner

Originally created by @georgzoeller on GitHub (Apr 3, 2024).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/75

It would be nice to be able to backup export the saved content and restore it (potentially on another machine).

Originally created by @georgzoeller on GitHub (Apr 3, 2024). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/75 It would be nice to be able to backup export the saved content and restore it (potentially on another machine).
Author
Owner

@MohamedBassem commented on GitHub (Apr 3, 2024):

Do you want to do this per user or as a system backup? If it's a system backup, just archiving the 'DATA_DIR' is good enough. Redis data is ephemeral, and the search index can be reconstructed from the admin panel.

<!-- gh-comment-id:2033989436 --> @MohamedBassem commented on GitHub (Apr 3, 2024): Do you want to do this per user or as a system backup? If it's a system backup, just archiving the 'DATA_DIR' is good enough. Redis data is ephemeral, and the search index can be reconstructed from the admin panel.
Author
Owner

@streusselhirni commented on GitHub (Apr 28, 2024):

For me, a system-wide backup would be enough. I think for less technical users, it would be nice to have some documentation about how to backup the DATA_DIR.

<!-- gh-comment-id:2081552324 --> @streusselhirni commented on GitHub (Apr 28, 2024): For me, a system-wide backup would be enough. I think for less technical users, it would be nice to have some documentation about how to backup the `DATA_DIR`.
Author
Owner

@talentedmrripley commented on GitHub (May 31, 2024):

For me, a system-wide backup would be enough. I think for less technical users, it would be nice to have some documentation about how to backup the DATA_DIR.

This feels like it will be specific to each users setup and their preferred backup methods and locations. Or are you just thinking of some mention in documentation that backing up the data is enough for a restore or transfer?

<!-- gh-comment-id:2143050134 --> @talentedmrripley commented on GitHub (May 31, 2024): > For me, a system-wide backup would be enough. I think for less technical users, it would be nice to have some documentation about how to backup the `DATA_DIR`. This feels like it will be specific to each users setup and their preferred backup methods and locations. Or are you just thinking of some mention in documentation that backing up the data is enough for a restore or transfer?
Author
Owner

@hongruilin commented on GitHub (Jul 29, 2024):

Do you want to do this per user or as a system backup? If it's a system backup, just archiving the 'DATA_DIR' is good enough. Redis data is ephemeral, and the search index can be reconstructed from the admin panel.

Hello, do you mean DATA_DIR is hoarder_data /var/lib/docker/volumes/hoarder_data/_data?

<!-- gh-comment-id:2256813349 --> @hongruilin commented on GitHub (Jul 29, 2024): > Do you want to do this per user or as a system backup? If it's a system backup, just archiving the 'DATA_DIR' is good enough. Redis data is ephemeral, and the search index can be reconstructed from the admin panel. Hello, do you mean DATA_DIR is hoarder_data /var/lib/docker/volumes/hoarder_data/_data?
Author
Owner

@lnicola commented on GitHub (Jul 30, 2024):

It would be nice to have a better solution. I didn't check the contents of that directory, but copying data files on the fly isn't necessarily a great idea, see e.g. https://theorangeone.net/posts/database-backups/ or https://www.sqlite.org/howtocorrupt.html#_backup_or_restore_while_a_transaction_is_active.

<!-- gh-comment-id:2257495398 --> @lnicola commented on GitHub (Jul 30, 2024): It would be nice to have a better solution. I didn't check the contents of that directory, but copying data files on the fly isn't necessarily a great idea, see e.g. https://theorangeone.net/posts/database-backups/ or https://www.sqlite.org/howtocorrupt.html#_backup_or_restore_while_a_transaction_is_active.
Author
Owner

@kamtschatka commented on GitHub (Jul 30, 2024):

You simply have to turn off your docker containers, do a backup and then restart them after the backup. I am not sure how many users your hoarder instance has, but I am guessing it should be easy enough to find a timeframe, where people are sleeping. And in the worst case, you have a 5 minute downtime.

<!-- gh-comment-id:2257982945 --> @kamtschatka commented on GitHub (Jul 30, 2024): You simply have to turn off your docker containers, do a backup and then restart them after the backup. I am not sure how many users your hoarder instance has, but I am guessing it should be easy enough to find a timeframe, where people are sleeping. And in the worst case, you have a 5 minute downtime.
Author
Owner

@MRDGH2821 commented on GitHub (Aug 17, 2024):

Per user backup would also be nice. This can be like sqlite db file containing query results of current user 🤔
Considering I'm single user, DATA_DIR is fine for me.

<!-- gh-comment-id:2294871849 --> @MRDGH2821 commented on GitHub (Aug 17, 2024): Per user backup would also be nice. This can be like sqlite db file containing query results of current user :thinking: Considering I'm single user, `DATA_DIR` is fine for me.
Author
Owner

@raviwarrier commented on GitHub (Sep 8, 2024):

Do you want to do this per user or as a system backup? If it's a system backup, just archiving the 'DATA_DIR' is good enough. Redis data is ephemeral, and the search index can be reconstructed from the admin panel.

This may seem simple for people who can and are comfortable working on the backend. But for those who are not engineers or coders - like people who use my instance of hoarder to take their backups. It would definitely improve the UX for non-technical users.

<!-- gh-comment-id:2336523323 --> @raviwarrier commented on GitHub (Sep 8, 2024): > Do you want to do this per user or as a system backup? If it's a system backup, just archiving the 'DATA_DIR' is good enough. Redis data is ephemeral, and the search index can be reconstructed from the admin panel. This may seem simple for people who can and are comfortable working on the backend. But for those who are not engineers or coders - like people who use my instance of hoarder to take their backups. It would definitely improve the UX for non-technical users.
Author
Owner

@alexkutsan commented on GitHub (Oct 3, 2024):

Agree, backup and restore would be a good feature. At least as HTML file (without cached content and screenshots).

<!-- gh-comment-id:2391226726 --> @alexkutsan commented on GitHub (Oct 3, 2024): Agree, backup and restore would be a good feature. At least as HTML file (without cached content and screenshots).
Author
Owner

@MohamedBassem commented on GitHub (Oct 5, 2024):

I have added support for a per user export of links and notes which is going to be available in the next release (github.com/hoarder-app/hoarder@c8a3c1ee02). It's not a backup though, it's for the sake of migrating to another platform if you want to.

<!-- gh-comment-id:2395081125 --> @MohamedBassem commented on GitHub (Oct 5, 2024): I have added support for a per user export of links and notes which is going to be available in the next release (https://github.com/hoarder-app/hoarder/commit/c8a3c1ee02e917b2e553d403b7bf94cbc736f51d). It's not a backup though, it's for the sake of migrating to another platform if you want to.
Author
Owner

@raviwarrier commented on GitHub (Oct 5, 2024):

That's good. Even export is fine. It just means indexing and inferencing might need to be done again, but as long as one can get a copy of all the links, it's good. :)

Keep up the good work buddy. 👍🏽

<!-- gh-comment-id:2395115295 --> @raviwarrier commented on GitHub (Oct 5, 2024): That's good. Even export is fine. It just means indexing and inferencing might need to be done again, but as long as one can get a copy of all the links, it's good. :) Keep up the good work buddy. 👍🏽
Author
Owner

@mbuotidem commented on GitHub (Nov 1, 2024):

Another use-case. I'm planning to run this on Google Cloud Run or Amazon Lightsail (container version, not the vps version) but I wouldn't feel safe without a backup mechanism in the event of container crashes. So I'd like to be able to periodically save the DATA_DIR (or just the export data) to the respective cloud's object storage. Is this something you'd be open to accepting a PR for?

<!-- gh-comment-id:2451248822 --> @mbuotidem commented on GitHub (Nov 1, 2024): Another use-case. I'm planning to run this on Google Cloud Run or Amazon Lightsail (container version, not the vps version) but I wouldn't feel safe without a backup mechanism in the event of container crashes. So I'd like to be able to periodically save the DATA_DIR (or just the export data) to the respective cloud's object storage. Is this something you'd be open to accepting a PR for?
Author
Owner

@Godferdom commented on GitHub (Jan 2, 2025):

It would be nice to propose an automatic (daily/monthly) backup instead of an export.

<!-- gh-comment-id:2568443287 --> @Godferdom commented on GitHub (Jan 2, 2025): It would be nice to propose an automatic (daily/monthly) backup instead of an export.
Author
Owner

@thiswillbeyourgithub commented on GitHub (May 19, 2025):

FYI my karakeep-python-api can be used to download all lists, bookmarks and highlights, etc to a text file via its API, as well as recreate it. Only per user though (the admin can't query from the other users afaik). If there's enough interest (and if @MohamedBassem does not have the time to spare to write the import/export code) I could write a script to makes at least the backup part easily. It would pair nicely with cron/systemd too. If interested create a feature request on the tool's repo: https://github.com/thiswillbeyourgithub/karakeep_python_api/

Also I think #1403 would be needed before someone else than @MohamedBassem can write a proper backup/restore script.

<!-- gh-comment-id:2890122700 --> @thiswillbeyourgithub commented on GitHub (May 19, 2025): FYI my [karakeep-python-api](https://github.com/thiswillbeyourgithub/karakeep_python_api/) can be used to download all lists, bookmarks and highlights, etc to a text file via its API, as well as recreate it. Only per user though (the admin can't query from the other users afaik). If there's enough interest (and if @MohamedBassem does not have the time to spare to write the import/export code) I could write a script to makes at least the backup part easily. It would pair nicely with cron/systemd too. If interested create a feature request on the tool's repo: https://github.com/thiswillbeyourgithub/karakeep_python_api/ Also I think #1403 would be needed before someone else than @MohamedBassem can write a proper backup/restore script.
Author
Owner

@MohamedBassem commented on GitHub (Sep 14, 2025):

FYI, I've just released a new version of the CLI that allows you to migrate all your data between servers, and to also take a full dump of your account into an archive (including assets).

You can find the docs for the server migration here: https://docs.karakeep.app/next/guides/server-migration

This is very experimental, so use with care.

Image
<!-- gh-comment-id:3289546960 --> @MohamedBassem commented on GitHub (Sep 14, 2025): FYI, I've just released a new version of the CLI that allows you to migrate all your data between servers, and to also take a full dump of your account into an archive (including assets). You can find the docs for the server migration here: https://docs.karakeep.app/next/guides/server-migration This is very experimental, so use with care. <img width="1772" height="1248" alt="Image" src="https://github.com/user-attachments/assets/a0ad9fe4-945f-489e-96bd-792b641338bb" />
Author
Owner

@MohamedBassem commented on GitHub (Sep 14, 2025):

And I'm planning to implement automatic backups as well.

<!-- gh-comment-id:3289547577 --> @MohamedBassem commented on GitHub (Sep 14, 2025): And I'm planning to implement automatic backups as well.
Author
Owner

@Depechie commented on GitHub (Sep 16, 2025):

The CLI option works like a charm!

<!-- gh-comment-id:3300341657 --> @Depechie commented on GitHub (Sep 16, 2025): The CLI option works like a charm!
Author
Owner

@juanmanavarro commented on GitHub (Nov 18, 2025):

Hi.

Migration gives me this error

Migrating user settings …
Error: Failed migrating user settings. Reason: No "query"-procedure on path "users.settings"
✗
Error: Migration failed. Reason: No "query"-procedure on path "users.settings"

Any idea?
Thanks

<!-- gh-comment-id:3546986004 --> @juanmanavarro commented on GitHub (Nov 18, 2025): Hi. Migration gives me this error ``` Migrating user settings … Error: Failed migrating user settings. Reason: No "query"-procedure on path "users.settings" ✗ Error: Migration failed. Reason: No "query"-procedure on path "users.settings" ``` Any idea? Thanks
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/karakeep#69
No description provided.