[GH-ISSUE #762] Question: properly save PDF from URLs without .pdf extension #483

New issue

Closed

opened 2026-03-01 14:44:01 +03:00 by kerem · 1 comment

kerem commented

2026-03-01 14:44:01 +03:00

Owner

Originally created by @philippemilink on GitHub (Jun 5, 2021).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/762

(perhaps this is a feature request)

If I save a web page serving a PDF file (such as https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial), the "wget > HTML" preview displays the listing of the folder containing the save. I have to go through the "See all files..." and browse the folder tree to get the file. Moreover, since the file has no extension, the browser considers it as application/octet-stream file, and doesn't open it directly in the browser as it does normally.

With URL including the .pdf extension, the preview works fine.

Is there a way to make it work for all URLs serving PDF files (with and without the extension) ?

If not, may I suggest a feature adding the pdf extension to the saved file (and the saved URL) if it appears the saved file is a PDF ? (it could be done by looking at the Content-Type header)

Originally created by @philippemilink on GitHub (Jun 5, 2021). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/762 (perhaps this is a feature request) If I save a web page serving a PDF file (such as https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial), the "wget > HTML" preview displays the listing of the folder containing the save. I have to go through the "See all files..." and browse the folder tree to get the file. Moreover, since the file has no extension, the browser considers it as `application/octet-stream` file, and doesn't open it directly in the browser as it does normally. With URL including the `.pdf` extension, the preview works fine. Is there a way to make it work for all URLs serving PDF files (with and without the extension) ? If not, may I suggest a feature adding the `pdf` extension to the saved file (and the saved URL) if it appears the saved file is a PDF ? (it could be done by looking at the `Content-Type` header)

kerem

2026-03-01 14:44:01 +03:00

closed this issue
added the
status: wontfix
label

kerem commented

2026-03-01 14:44:02 +03:00

Author

Owner

@pirate commented on GitHub (Jun 10, 2021):

This is a known shortcoming of how static files are saved and replayed by the WGET extractor, see here: https://github.com/ArchiveBox/ArchiveBox/blob/dev/archivebox/extractors/wget.py#L179

Likely not going to fix this, as we want to avoid rewriting the URLs to be different in the filesystem than how they are served (as much as possible). The one place we do rewrite URLs is to add .html on the end, and that's already one of the gnarliest most difficult-to-maintain pieces of code in the entire codebase, so I'm not going to add another edge case there.

We also don't serve MIME headers with wget-downloaded content, so we aren't able to replicate they way they serve PDF mime content from a non .pdf URL extension. We can't safely replicate that yet, so any non-HTML content is served as application/octet-stream.

Debian should serve .pdf on the end of their URL or redirect to a .pdf URL from the non-pdf one instead of serving PDF mime content from a non-PDF extension. Bending over backwards to support that type of weirdness will introduce too much complexity and break other areas of ArchiveBox.

@pirate commented on GitHub (Jun 10, 2021): This is a known shortcoming of how static files are saved and replayed by the WGET extractor, see here: https://github.com/ArchiveBox/ArchiveBox/blob/dev/archivebox/extractors/wget.py#L179 Likely not going to fix this, as we want to avoid rewriting the URLs to be different in the filesystem than how they are served (as much as possible). The one place we do rewrite URLs is to add `.html` on the end, and that's already one of the gnarliest most difficult-to-maintain [pieces of code](https://github.com/ArchiveBox/ArchiveBox/blob/dev/archivebox/extractors/wget.py#L129) in the entire codebase, so I'm not going to add another edge case there. We also don't serve MIME headers with wget-downloaded content, so we aren't able to replicate they way they serve PDF mime content from a non `.pdf` URL extension. We can't safely replicate that yet, so any non-HTML content is served as `application/octet-stream`. Debian should serve `.pdf` on the end of their URL or redirect to a `.pdf` URL from the non-pdf one instead of serving PDF mime content from a non-PDF extension. Bending over backwards to support that type of weirdness will introduce too much complexity and break other areas of ArchiveBox.

kerem referenced this issue

2026-03-01 14:48:46 +03:00

[PR #486] [MERGED] fix: Wget absolute path generating issues #1187

kerem referenced this issue

2026-03-01 14:48:47 +03:00

[PR #488] [MERGED] fix: Remove trailing slash from public index #1188

kerem referenced this issue

2026-03-01 17:54:01 +03:00

[GH-ISSUE #483] Bugfix: docker iframe adds /data/ to url incorrectly #1827

kerem referenced this issue

2026-03-01 18:00:27 +03:00

[PR #488] [MERGED] fix: Remove trailing slash from public index #2694