[GH-ISSUE #722] Question: Docker on Windows archiving to an SMB path that doesn't support FSYNC

kerem commented

2026-03-01 14:43:44 +03:00

Owner

Originally created by @aluhrs13 on GitHub (Apr 24, 2021).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/722

I've been fighting with this long enough to hope that someone with more ArchiveBox and/or Docker knowledge can spot something I'm doing wrong. I'm attempting to run ArchiveBox on Windows following the docker-compose directions, with my Volume on an SMB path (my NAS, not my server).

First I make the volume:
docker volume create --driver local --opt type=cifs --opt device=//<ipaddress>/ArchiveBox --opt o=username=<username>,password=<pass> ArchiveBox

Then I run the first command in the directions:
docker-compose run archivebox init --setup

And hit errors:

Creating network "archivebox3_default" with the default driver
Creating archivebox3_archivebox_run ... done
[i] [2021-04-24 19:07:10] ArchiveBox v0.6.2: archivebox init --setup
    > /data

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/logging/config.py", line 564, in configure
    handler = self.configure_handler(handlers[name])
  File "/usr/local/lib/python3.9/logging/config.py", line 745, in configure_handler
    result = factory(**kwargs)
  File "/usr/local/lib/python3.9/logging/handlers.py", line 153, in __init__
    BaseRotatingHandler.__init__(self, filename, mode, encoding=encoding,
  File "/usr/local/lib/python3.9/logging/handlers.py", line 58, in __init__
    logging.FileHandler.__init__(self, filename, mode=mode,
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1146, in __init__
    StreamHandler.__init__(self, self._open())
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1175, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding,
PermissionError: [Errno 13] Permission denied: '/data/logs/errors.log'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/archivebox", line 33, in <module>
    sys.exit(load_entry_point('archivebox', 'console_scripts', 'archivebox')())
  File "/app/archivebox/cli/__init__.py", line 140, in main
    run_subcommand(
  File "/app/archivebox/cli/__init__.py", line 74, in run_subcommand
    setup_django(in_memory_db=subcommand in fake_db, check_db=cmd_requires_db and not init_pending)
  File "/app/archivebox/config.py", line 1136, in setup_django
    django.setup()
  File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 19, in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/usr/local/lib/python3.9/site-packages/django/utils/log.py", line 75, in configure_logging
    logging_config_func(logging_settings)
  File "/usr/local/lib/python3.9/logging/config.py", line 809, in dictConfig
    dictConfigClass(config).configure()
  File "/usr/local/lib/python3.9/logging/config.py", line 571, in configure
    raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'logfile'
ERROR: 1

A folder named "logs" is created, but nothing else. Obviously seems to be a permissions issue of some sort, but I don't know enough to figure out what it is. Any suggestions or alternative approaches?

Originally created by @aluhrs13 on GitHub (Apr 24, 2021). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/722 I've been fighting with this long enough to hope that someone with more ArchiveBox and/or Docker knowledge can spot something I'm doing wrong. I'm attempting to run ArchiveBox on Windows following the docker-compose directions, with my Volume on an SMB path (my NAS, not my server). First I make the volume: `docker volume create --driver local --opt type=cifs --opt device=//<ipaddress>/ArchiveBox --opt o=username=<username>,password=<pass> ArchiveBox` Then I run the first command in the directions: `docker-compose run archivebox init --setup` And hit errors: ``` Creating network "archivebox3_default" with the default driver Creating archivebox3_archivebox_run ... done [i] [2021-04-24 19:07:10] ArchiveBox v0.6.2: archivebox init --setup > /data Traceback (most recent call last): File "/usr/local/lib/python3.9/logging/config.py", line 564, in configure handler = self.configure_handler(handlers[name]) File "/usr/local/lib/python3.9/logging/config.py", line 745, in configure_handler result = factory(**kwargs) File "/usr/local/lib/python3.9/logging/handlers.py", line 153, in __init__ BaseRotatingHandler.__init__(self, filename, mode, encoding=encoding, File "/usr/local/lib/python3.9/logging/handlers.py", line 58, in __init__ logging.FileHandler.__init__(self, filename, mode=mode, File "/usr/local/lib/python3.9/logging/__init__.py", line 1146, in __init__ StreamHandler.__init__(self, self._open()) File "/usr/local/lib/python3.9/logging/__init__.py", line 1175, in _open return open(self.baseFilename, self.mode, encoding=self.encoding, PermissionError: [Errno 13] Permission denied: '/data/logs/errors.log' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/bin/archivebox", line 33, in <module> sys.exit(load_entry_point('archivebox', 'console_scripts', 'archivebox')()) File "/app/archivebox/cli/__init__.py", line 140, in main run_subcommand( File "/app/archivebox/cli/__init__.py", line 74, in run_subcommand setup_django(in_memory_db=subcommand in fake_db, check_db=cmd_requires_db and not init_pending) File "/app/archivebox/config.py", line 1136, in setup_django django.setup() File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 19, in setup configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) File "/usr/local/lib/python3.9/site-packages/django/utils/log.py", line 75, in configure_logging logging_config_func(logging_settings) File "/usr/local/lib/python3.9/logging/config.py", line 809, in dictConfig dictConfigClass(config).configure() File "/usr/local/lib/python3.9/logging/config.py", line 571, in configure raise ValueError('Unable to configure handler ' ValueError: Unable to configure handler 'logfile' ERROR: 1 ``` A folder named "logs" is created, but nothing else. Obviously seems to be a permissions issue of some sort, but I don't know enough to figure out what it is. Any suggestions or alternative approaches?

kerem

2026-03-01 14:43:44 +03:00

closed this issue
added the
touches: data/schema/architecture
label

kerem commented

2026-03-01 14:43:45 +03:00

Author

Owner

@pirate commented on GitHub (Apr 24, 2021):

Try adding a :z flag to your volume:

services:
    archivebox:
        volumes:
            - ArchiveBox:/data:z

Network drives throw red-herring permissions errors sometimes because we use atomic_write to FSYNC all writes, and some filesystems don't support it. The :z flag seems to often fix that quirk by having docker middleman the writes (which seems to hide the lack of FSYNC support and fix the error as a side effect).

@pirate commented on GitHub (Apr 24, 2021): Try adding a `:z` flag to your volume: ```yaml services: archivebox: volumes: - ArchiveBox:/data:z ``` Network drives throw red-herring permissions errors sometimes because we use `atomic_write` to FSYNC all writes, and some filesystems don't support it. The `:z` flag seems to often fix that quirk by having docker middleman the writes (which seems to hide the lack of FSYNC support and fix the error as a side effect).

kerem commented

2026-03-01 14:43:45 +03:00

Author

Owner

@aluhrs13 commented on GitHub (Apr 24, 2021):

No dice 😢, here's the contents of my docker-compose.yml in case it's something there.

version: '2.4'

volumes:
  vol:
    external: true
    name: ArchiveBox

services:
    archivebox:
        image: ${DOCKER_IMAGE:-archivebox/archivebox:latest}     
        command: server --quick-init 0.0.0.0:8000
        ports:
            - 3081:8000
        environment:
            - ALLOWED_HOSTS=*
            - MEDIA_MAX_SIZE=750m
            - SUBMIT_ARCHIVE_DOT_ORG=False
        volumes:
            - vol:/data:z

@aluhrs13 commented on GitHub (Apr 24, 2021): No dice 😢, here's the contents of my docker-compose.yml in case it's something there. ``` version: '2.4' volumes: vol: external: true name: ArchiveBox services: archivebox: image: ${DOCKER_IMAGE:-archivebox/archivebox:latest} command: server --quick-init 0.0.0.0:8000 ports: - 3081:8000 environment: - ALLOWED_HOSTS=* - MEDIA_MAX_SIZE=750m - SUBMIT_ARCHIVE_DOT_ORG=False volumes: - vol:/data:z ```

kerem commented

2026-03-01 14:43:45 +03:00

Author

Owner

@pirate commented on GitHub (Apr 25, 2021):

docker-compose run --rm archivebox /bin/ls -lah /data
docker-compose run --rm archivebox /bin/ls -lah /data/logs

Can you post the output of these ^

@pirate commented on GitHub (Apr 25, 2021): ```bash docker-compose run --rm archivebox /bin/ls -lah /data docker-compose run --rm archivebox /bin/ls -lah /data/logs ``` Can you post the output of these ^

kerem commented

2026-03-01 14:43:45 +03:00

Author

Owner

@aluhrs13 commented on GitHub (Apr 25, 2021):

Here you go, thanks for the help!

C:\Users\aluhr\Docker\ArchiveBox3>docker-compose run --rm archivebox /bin/ls -lah /data
Creating archivebox3_archivebox_run ... done
total 4.0K
drwxr-xr-x 2 root root    0 Apr 14 08:57 .
drwxr-xr-x 1 root root 4.0K Apr 25 01:05 ..
drwxr-xr-x 2 root root    0 Apr 14 08:57 logs

C:\Users\aluhr\Docker\ArchiveBox3>docker-compose run --rm archivebox /bin/ls -lah /data/logs
Creating archivebox3_archivebox_run ... done
total 0
drwxr-xr-x 2 root root 0 Apr 14 08:57 .
drwxr-xr-x 2 root root 0 Apr 14 08:57 ..

@aluhrs13 commented on GitHub (Apr 25, 2021): Here you go, thanks for the help! ``` C:\Users\aluhr\Docker\ArchiveBox3>docker-compose run --rm archivebox /bin/ls -lah /data Creating archivebox3_archivebox_run ... done total 4.0K drwxr-xr-x 2 root root 0 Apr 14 08:57 . drwxr-xr-x 1 root root 4.0K Apr 25 01:05 .. drwxr-xr-x 2 root root 0 Apr 14 08:57 logs C:\Users\aluhr\Docker\ArchiveBox3>docker-compose run --rm archivebox /bin/ls -lah /data/logs Creating archivebox3_archivebox_run ... done total 0 drwxr-xr-x 2 root root 0 Apr 14 08:57 . drwxr-xr-x 2 root root 0 Apr 14 08:57 .. ```

kerem commented

2026-03-01 14:43:45 +03:00

Author

Owner

@pirate commented on GitHub (Apr 25, 2021):

Can you try giving all permissions on the dir? chmod -R 777 data (or whatever the windows equivalent of chmod is).

You may also have to change the settings on the cifs cluster to allow read/write for all users, as often permissions are enforced on the server side of the fileshare.

@pirate commented on GitHub (Apr 25, 2021): Can you try giving all permissions on the dir? `chmod -R 777 data` (or whatever the windows equivalent of `chmod` is). You may also have to change the settings on the cifs cluster to allow read/write for all users, as often permissions are enforced on the server side of the fileshare.

kerem commented

2026-03-01 14:43:45 +03:00

Author

Owner

@aluhrs13 commented on GitHub (Apr 25, 2021):

The permissions definitely changed according to SSH and Synology's dashboard, but re-running the the last commands still show the same permissions from Docker's perspective and I get the same error.

I wonder if I should re-think my approach entirely. Is there any way to change what folder archived data is saved to after initial setup? I didn't see an environment variable to configure that. Or maybe I finally go nuclear and swap this server to Linux like I should've done from the start 😅.

@aluhrs13 commented on GitHub (Apr 25, 2021): The permissions definitely changed according to SSH and Synology's dashboard, but re-running the the last commands still show the same permissions from Docker's perspective and I get the same error. I wonder if I should re-think my approach entirely. Is there any way to change what folder archived data is saved to after initial setup? I didn't see an environment variable to configure that. Or maybe I finally go nuclear and swap this server to Linux like I should've done from the start 😅.

kerem commented

2026-03-01 14:43:45 +03:00

Author

Owner

@aluhrs13 commented on GitHub (Apr 25, 2021):

Alright I've bit a partial bullet and opted to just run ArchiveBox on my Synology instead of trying to fight this more. Feel free to close this out, unless there's something here you want to investigate further, I'm happy to continue trying troubleshooting steps.

@aluhrs13 commented on GitHub (Apr 25, 2021): Alright I've bit a partial bullet and opted to just run ArchiveBox on my Synology instead of trying to fight this more. Feel free to close this out, unless there's something here you want to investigate further, I'm happy to continue trying troubleshooting steps.

kerem commented

2026-03-01 14:43:45 +03:00

Author

Owner

@pirate commented on GitHub (Apr 12, 2022):

Note I've added a new DB/filesystem troubleshooting area to the wiki that may help people arriving here from Google: https://github.com/ArchiveBox/ArchiveBox/wiki/Upgrading-or-Merging-Archives#database-troubleshooting

Contributions/suggestions welcome there.

@pirate commented on GitHub (Apr 12, 2022): Note I've added a new DB/filesystem troubleshooting area to the wiki that may help people arriving here from Google: https://github.com/ArchiveBox/ArchiveBox/wiki/Upgrading-or-Merging-Archives#database-troubleshooting Contributions/suggestions welcome there.

kerem referenced this issue

2026-03-01 17:53:52 +03:00

[GH-ISSUE #456] Support for network drives or filesystems that don't implement FSYNC #1812

kerem referenced this issue

2026-03-14 22:06:50 +03:00

[GH-ISSUE #456] Support for network drives or filesystems that don't implement FSYNC #3323

Rows
Columns

[GH-ISSUE #722] Question: Docker on Windows archiving to an SMB path that doesn't support FSYNC #456