[GH-ISSUE #1568] Bug: archived YouTube videos aren't accessible #2446

Open
opened 2026-03-01 17:59:05 +03:00 by kerem · 10 comments
Owner

Originally created by @arielelkin on GitHub (Oct 25, 2024).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1568

Describe the bug

Adding a youtube video apparently archives it but isn't accessible via the UI.

Steps to reproduce

  1. Click Add +
  2. insert youtube video URL: https://www.youtube.com/watch?v=gzodB0Sp6ZI

Snapshot shows as "Pending" permanently:
Image

Impossible to access actual archived video. When clicking wget, cookie message won't go away:
Image

Screenshots or log output

Image

Log
Not Found: /youtubei/v1/log_event

"POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179

Not Found: /generate_204

"HEAD /generate_204 HTTP/1.1" 404 179

"GET /admin/core/archiveresult/?o=-1 HTTP/1.1" 200 63927

"GET /archive/1729852304.684918/index.html HTTP/1.1" 200 241231

Not Found: /archive/1729852304.684918/screenshot.png

"GET /archive/1729852304.684918/screenshot.png HTTP/1.1" 404 1242

"GET /archive/1729852304.684918/media/ HTTP/1.1" 200 401

"GET /archive/1729852304.684918/output.html HTTP/1.1" 200 0

"GET /archive/1729852304.684918/mercury/content.html HTTP/1.1" 200 164

Not Found: /archive/1729852304.684918/readability/content.html

"GET /archive/1729852304.684918/readability/content.html HTTP/1.1" 404 1252

Not Found: /archive/1729852304.684918/singlefile.html

"GET /archive/1729852304.684918/singlefile.html HTTP/1.1" 404 1243

"GET /archive/1729852304.684918/www.youtube.com/watch%40v%3DbuOo3C4CmSA.html HTTP/1.1" 200 858714

Not Found: /archive/1729852304.684918/output.pdf

"GET /archive/1729852304.684918/output.pdf HTTP/1.1" 404 1238

"GET /archive/1729852304.684918/headers.json HTTP/1.1" 200 2573

Not Found: /archive/1729852304.684918/git/

"GET /archive/1729852304.684918/git/ HTTP/1.1" 404 1232

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/webcomponents-sd.vflset/webcomponents-sd.js HTTP/1.1" 200 78674

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/intersection-observer.min.vflset/intersection-observer.min.js HTTP/1.1" 200 5547

"GET /archive/1729852304.684918/i.ytimg.com/vi/buOo3C4CmSA/hqdefault.jpg HTTP/1.1" 200 27727

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/custom-elements-es5-adapter.vflset/custom-elements-es5-adapter.js HTTP/1.1" 200 2051

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/web-animations-next-lite.min.vflset/web-animations-next-lite.min.js HTTP/1.1" 200 50864

"GET /archive/1729852304.684918/www.youtube.com/s/player/fb725ac8/player_ias.vflset/de_DE/base.js HTTP/1.1" 200 2467475

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/desktop_polymer.vflset/desktop_polymer.js HTTP/1.1" 200 8593455

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/scheduler.vflset/scheduler.js HTTP/1.1" 200 9713

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/cssbin/www-main-desktop-player-skeleton.css HTTP/1.1" 200 2741

"GET /archive/1729852304.684918/www.youtube.com/s//ytmainappweb//ss/k=ytmainappweb.kevlar_base.CsHs8scMMtg.L.B1.O/am=AACCBA/d=0/rs=AGKMywFhf542l67Xb-Ga16Te278BjvrbGA.css HTTP/1.1" 200 2490567

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/www-i18n-constants-de_DE.vflset/www-i18n-constants.js HTTP/1.1" 200 7360

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/cssbin/www-main-desktop-watch-page-skeleton.css HTTP/1.1" 200 8555

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/cssbin/www-onepick.css HTTP/1.1" 200 739

"GET /archive/1729852304.684918/fonts.googleapis.com/css2@family=Roboto%253Awght@300%3B400%3B500%3B700&family=YouTube+Sans%253Awght@300..900&display=swap.css HTTP/1.1" 200 74374

"GET /archive/1729852304.684918/www.youtube.com/s/player/fb725ac8/www-player.css HTTP/1.1" 200 392635

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/network.vflset/network.js HTTP/1.1" 200 14335

"GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/spf.vflset/spf.js HTTP/1.1" 200 38401

Not Found: /s/player/fb725ac8/player_ias.vflset/de_DE/offline.js

"GET /s/player/fb725ac8/player_ias.vflset/de_DE/offline.js HTTP/1.1" 404 179

Not Found: /s/player/fb725ac8/player_ias.vflset/de_DE/remote.js

"GET /s/player/fb725ac8/player_ias.vflset/de_DE/remote.js HTTP/1.1" 404 179

Not Found: /s/player/fb725ac8/player_ias.vflset/de_DE/miniplayer.js

"GET /s/player/fb725ac8/player_ias.vflset/de_DE/miniplayer.js HTTP/1.1" 404 179

"GET /archive/1729852304.684918/accounts.google.com/ServiceLogin@service=youtube&uilel=3&passive=true&continue=https%253A%252F%252Fwww.youtube.com%252Fsignin%253Faction_handle_signin=true&app=desktop&hl=de&next=%252Fsignin_passive&feature=passive&hl=de.html HTTP/1.1" 200 742246

"GET /archive/1729852304.684918/fonts.gstatic.com/s/roboto/v32/KFOmCnqEu92Fr1Mu4mxKKTU1Kg.woff2 HTTP/1.1" 200 13388

"GET /archive/1729852304.684918/www.gstatic.com//mss/boq-identity//js/k=boq-identity.AccountsSignInUi.de.ThjYv1ztrgk.es5.O/am=5A5GkWEbgYD4hKcBvQFFAiEHAAAAAAAAAABtAACAHgY/d=1/excm=_b,tp,identifierview/ed=1/dg=0/wt=2/ujg=1/rs=AOaEmlE411wcUYF9zUGYxvo-iJ0FAU3Ew/m=_b,_tp HTTP/1.1" 200 750991

Not Found: /youtubei/v1/att/get

"POST /youtubei/v1/att/get?prettyPrint=false HTTP/1.1" 404 179

Not Found: /youtubei/v1/log_event

"POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179

Not Found: /s/player/fb725ac8/www-player.css

"GET /s/player/fb725ac8/www-player.css HTTP/1.1" 404 179

"GET /archive/1729852304.684918/fonts.gstatic.com/s/roboto/v32/KFOlCnqEu92Fr1MmEU9fBBc4AMP6lQ.woff2 HTTP/1.1" 200 13432

"GET /archive/1729852304.684918/fonts.gstatic.com/s/youtubesans/v30/Qw38ZQNGEDjaO2m6tqIqX5E-AVS5_rSejo46_PCTRspJ0OosolrBEJL3HO_T7fHoCVHx.woff2 HTTP/1.1" 200 32256

"GET /archive/1729852304.684918/fonts.gstatic.com/s/roboto/v32/KFOlCnqEu92Fr1MmWUlfBBc4AMP6lQ.woff2 HTTP/1.1" 200 13408

Not Found: /sw.js

"GET /sw.js HTTP/1.1" 404 179

Not Found: /youtubei/v1/att/get

"POST /youtubei/v1/att/get?prettyPrint=false HTTP/1.1" 404 179

Not Found: /s/search/audio/no_input.mp3

"GET /s/search/audio/no_input.mp3 HTTP/1.1" 404 179

Not Found: /s/search/audio/open.mp3

"GET /s/search/audio/open.mp3 HTTP/1.1" 404 179

Not Found: /s/search/audio/success.mp3

"GET /s/search/audio/success.mp3 HTTP/1.1" 404 179

Not Found: /s/search/audio/failure.mp3

"GET /s/search/audio/failure.mp3 HTTP/1.1" 404 179

Not Found: /youtubei/v1/att/get

"POST /youtubei/v1/att/get?prettyPrint=false HTTP/1.1" 404 179

Not Found: /youtubei/v1/log_event

"POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179

Not Found: /generate_204

"HEAD /generate_204 HTTP/1.1" 404 179

Not Found: /youtubei/v1/att/get

"POST /youtubei/v1/att/get?prettyPrint=false HTTP/1.1" 404 179

Not Found: /youtubei/v1/log_event

"POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179

Not Found: /generate_204

"HEAD /generate_204 HTTP/1.1" 404 179

Not Found: /youtubei/v1/log_event

"POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179

Not Found: /generate_204

"HEAD /generate_204 HTTP/1.1" 404 179

Not Found: /youtubei/v1/log_event

"POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179

Not Found: /generate_204

"HEAD /generate_204 HTTP/1.1" 404 179

Not Found: /youtubei/v1/att/get

"POST /youtubei/v1/att/get?prettyPrint=false HTTP/1.1" 404 179

Not Found: /youtubei/v1/log_event

"POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179

"GET /index.html HTTP/1.1" 302 0

"GET / HTTP/1.1" 302 0

"GET /admin/core/snapshot/ HTTP/1.1" 200 26778

"GET /admin/jsi18n/ HTTP/1.1" 200 3191

"GET /admin/core/snapshot/add/ HTTP/1.1" 302 0

"GET /add/ HTTP/1.1" 200 7296

[+] Adding URL: https://www.youtube.com/watch?v=gzodB0Sp6ZI

ArchiveBox version

version 0.7.2

Originally created by @arielelkin on GitHub (Oct 25, 2024). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1568 #### Describe the bug Adding a youtube video apparently archives it but isn't accessible via the UI. #### Steps to reproduce 1. Click Add + 2. insert youtube video URL: https://www.youtube.com/watch?v=gzodB0Sp6ZI Snapshot shows as "Pending" permanently: ![Image](https://github.com/user-attachments/assets/289ec44c-67f0-4d58-88c9-e2a4b0f160e8) Impossible to access actual archived video. When clicking wget, cookie message won't go away: ![Image](https://github.com/user-attachments/assets/ebd0edd0-8854-4a32-8906-7db8f00af6dd) #### Screenshots or log output ![Image](https://github.com/user-attachments/assets/64ae3c47-ca8c-4b3f-90aa-29209d698677) <details> <summary>Log</summary> <pre> Not Found: /youtubei/v1/log_event "POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179 Not Found: /generate_204 "HEAD /generate_204 HTTP/1.1" 404 179 "GET /admin/core/archiveresult/?o=-1 HTTP/1.1" 200 63927 "GET /archive/1729852304.684918/index.html HTTP/1.1" 200 241231 Not Found: /archive/1729852304.684918/screenshot.png "GET /archive/1729852304.684918/screenshot.png HTTP/1.1" 404 1242 "GET /archive/1729852304.684918/media/ HTTP/1.1" 200 401 "GET /archive/1729852304.684918/output.html HTTP/1.1" 200 0 "GET /archive/1729852304.684918/mercury/content.html HTTP/1.1" 200 164 Not Found: /archive/1729852304.684918/readability/content.html "GET /archive/1729852304.684918/readability/content.html HTTP/1.1" 404 1252 Not Found: /archive/1729852304.684918/singlefile.html "GET /archive/1729852304.684918/singlefile.html HTTP/1.1" 404 1243 "GET /archive/1729852304.684918/www.youtube.com/watch%40v%3DbuOo3C4CmSA.html HTTP/1.1" 200 858714 Not Found: /archive/1729852304.684918/output.pdf "GET /archive/1729852304.684918/output.pdf HTTP/1.1" 404 1238 "GET /archive/1729852304.684918/headers.json HTTP/1.1" 200 2573 Not Found: /archive/1729852304.684918/git/ "GET /archive/1729852304.684918/git/ HTTP/1.1" 404 1232 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/webcomponents-sd.vflset/webcomponents-sd.js HTTP/1.1" 200 78674 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/intersection-observer.min.vflset/intersection-observer.min.js HTTP/1.1" 200 5547 "GET /archive/1729852304.684918/i.ytimg.com/vi/buOo3C4CmSA/hqdefault.jpg HTTP/1.1" 200 27727 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/custom-elements-es5-adapter.vflset/custom-elements-es5-adapter.js HTTP/1.1" 200 2051 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/web-animations-next-lite.min.vflset/web-animations-next-lite.min.js HTTP/1.1" 200 50864 "GET /archive/1729852304.684918/www.youtube.com/s/player/fb725ac8/player_ias.vflset/de_DE/base.js HTTP/1.1" 200 2467475 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/desktop_polymer.vflset/desktop_polymer.js HTTP/1.1" 200 8593455 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/scheduler.vflset/scheduler.js HTTP/1.1" 200 9713 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/cssbin/www-main-desktop-player-skeleton.css HTTP/1.1" 200 2741 "GET /archive/1729852304.684918/www.youtube.com/s/_/ytmainappweb/_/ss/k=ytmainappweb.kevlar_base.CsHs8scMMtg.L.B1.O/am=AACCBA/d=0/rs=AGKMywFhf542l67Xb-Ga16Te278BjvrbGA.css HTTP/1.1" 200 2490567 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/www-i18n-constants-de_DE.vflset/www-i18n-constants.js HTTP/1.1" 200 7360 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/cssbin/www-main-desktop-watch-page-skeleton.css HTTP/1.1" 200 8555 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/cssbin/www-onepick.css HTTP/1.1" 200 739 "GET /archive/1729852304.684918/fonts.googleapis.com/css2@family=Roboto%253Awght@300%3B400%3B500%3B700&family=YouTube+Sans%253Awght@300..900&display=swap.css HTTP/1.1" 200 74374 "GET /archive/1729852304.684918/www.youtube.com/s/player/fb725ac8/www-player.css HTTP/1.1" 200 392635 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/network.vflset/network.js HTTP/1.1" 200 14335 "GET /archive/1729852304.684918/www.youtube.com/s/desktop/d7bac1f2/jsbin/spf.vflset/spf.js HTTP/1.1" 200 38401 Not Found: /s/player/fb725ac8/player_ias.vflset/de_DE/offline.js "GET /s/player/fb725ac8/player_ias.vflset/de_DE/offline.js HTTP/1.1" 404 179 Not Found: /s/player/fb725ac8/player_ias.vflset/de_DE/remote.js "GET /s/player/fb725ac8/player_ias.vflset/de_DE/remote.js HTTP/1.1" 404 179 Not Found: /s/player/fb725ac8/player_ias.vflset/de_DE/miniplayer.js "GET /s/player/fb725ac8/player_ias.vflset/de_DE/miniplayer.js HTTP/1.1" 404 179 "GET /archive/1729852304.684918/accounts.google.com/ServiceLogin@service=youtube&uilel=3&passive=true&continue=https%253A%252F%252Fwww.youtube.com%252Fsignin%253Faction_handle_signin=true&app=desktop&hl=de&next=%252Fsignin_passive&feature=passive&hl=de.html HTTP/1.1" 200 742246 "GET /archive/1729852304.684918/fonts.gstatic.com/s/roboto/v32/KFOmCnqEu92Fr1Mu4mxKKTU1Kg.woff2 HTTP/1.1" 200 13388 "GET /archive/1729852304.684918/www.gstatic.com/_/mss/boq-identity/_/js/k=boq-identity.AccountsSignInUi.de.ThjYv1ztrgk.es5.O/am=5A5GkWEbgYD4hKcBvQFFAiEHAAAAAAAAAABtAACAHgY/d=1/excm=_b,_tp,identifierview/ed=1/dg=0/wt=2/ujg=1/rs=AOaEmlE411wcUYF9zUGYxvo_-iJ0FAU3Ew/m=_b,_tp HTTP/1.1" 200 750991 Not Found: /youtubei/v1/att/get "POST /youtubei/v1/att/get?prettyPrint=false HTTP/1.1" 404 179 Not Found: /youtubei/v1/log_event "POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179 Not Found: /s/player/fb725ac8/www-player.css "GET /s/player/fb725ac8/www-player.css HTTP/1.1" 404 179 "GET /archive/1729852304.684918/fonts.gstatic.com/s/roboto/v32/KFOlCnqEu92Fr1MmEU9fBBc4AMP6lQ.woff2 HTTP/1.1" 200 13432 "GET /archive/1729852304.684918/fonts.gstatic.com/s/youtubesans/v30/Qw38ZQNGEDjaO2m6tqIqX5E-AVS5_rSejo46_PCTRspJ0OosolrBEJL3HO_T7fHoCVHx.woff2 HTTP/1.1" 200 32256 "GET /archive/1729852304.684918/fonts.gstatic.com/s/roboto/v32/KFOlCnqEu92Fr1MmWUlfBBc4AMP6lQ.woff2 HTTP/1.1" 200 13408 Not Found: /sw.js "GET /sw.js HTTP/1.1" 404 179 Not Found: /youtubei/v1/att/get "POST /youtubei/v1/att/get?prettyPrint=false HTTP/1.1" 404 179 Not Found: /s/search/audio/no_input.mp3 "GET /s/search/audio/no_input.mp3 HTTP/1.1" 404 179 Not Found: /s/search/audio/open.mp3 "GET /s/search/audio/open.mp3 HTTP/1.1" 404 179 Not Found: /s/search/audio/success.mp3 "GET /s/search/audio/success.mp3 HTTP/1.1" 404 179 Not Found: /s/search/audio/failure.mp3 "GET /s/search/audio/failure.mp3 HTTP/1.1" 404 179 Not Found: /youtubei/v1/att/get "POST /youtubei/v1/att/get?prettyPrint=false HTTP/1.1" 404 179 Not Found: /youtubei/v1/log_event "POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179 Not Found: /generate_204 "HEAD /generate_204 HTTP/1.1" 404 179 Not Found: /youtubei/v1/att/get "POST /youtubei/v1/att/get?prettyPrint=false HTTP/1.1" 404 179 Not Found: /youtubei/v1/log_event "POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179 Not Found: /generate_204 "HEAD /generate_204 HTTP/1.1" 404 179 Not Found: /youtubei/v1/log_event "POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179 Not Found: /generate_204 "HEAD /generate_204 HTTP/1.1" 404 179 Not Found: /youtubei/v1/log_event "POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179 Not Found: /generate_204 "HEAD /generate_204 HTTP/1.1" 404 179 Not Found: /youtubei/v1/att/get "POST /youtubei/v1/att/get?prettyPrint=false HTTP/1.1" 404 179 Not Found: /youtubei/v1/log_event "POST /youtubei/v1/log_event?alt=json HTTP/1.1" 404 179 "GET /index.html HTTP/1.1" 302 0 "GET / HTTP/1.1" 302 0 "GET /admin/core/snapshot/ HTTP/1.1" 200 26778 "GET /admin/jsi18n/ HTTP/1.1" 200 3191 "GET /admin/core/snapshot/add/ HTTP/1.1" 302 0 "GET /add/ HTTP/1.1" 200 7296 [+] Adding URL: https://www.youtube.com/watch?v=gzodB0Sp6ZI </pre> </details> #### ArchiveBox version version 0.7.2
Author
Owner

@pirate commented on GitHub (Oct 25, 2024):

Please post the full output of archivebox version.

<!-- gh-comment-id:2438230131 --> @pirate commented on GitHub (Oct 25, 2024): Please post the full output of `archivebox version`.
Author
Owner

@arielelkin commented on GitHub (Oct 28, 2024):

<!-- gh-comment-id:2441955067 --> @arielelkin commented on GitHub (Oct 28, 2024): ![](https://d33v4339jhl8k0.cloudfront.net/inline/156271/f37315f47f49413d416d61e1f38713112c8e6616/ea6443859db570ddf272753a0cb266c6c8992d6f/Screenshot-2024-10-28-at-9-40-46-PM.png)
Author
Owner

@pirate commented on GitHub (Oct 28, 2024):

Ah ok you're using the old docker image, it's over a year old at this point so yt-dlp isn't on the latest verison and is likely failing due to changes to Youtube.

You're welcome to try the latest BETA archivebox/archivebox:dev, upgrade yt-dlp within the container with apt manually, or wait for the upcoming v0.9.0 stable release to arrive.

<!-- gh-comment-id:2442556601 --> @pirate commented on GitHub (Oct 28, 2024): Ah ok you're using the old docker image, it's over a year old at this point so `yt-dlp` isn't on the latest verison and is likely failing due to changes to Youtube. You're welcome to try the latest BETA `archivebox/archivebox:dev`, upgrade `yt-dlp` within the container with `apt` manually, or wait for the upcoming v0.9.0 stable release to arrive.
Author
Owner

@arielelkin commented on GitHub (Oct 29, 2024):

Im using archivebox via pikapod.net, so I've no control over the container and will wait for the next stable release.

Thanks!

<!-- gh-comment-id:2444009062 --> @arielelkin commented on GitHub (Oct 29, 2024): Im using archivebox via pikapod.net, so I've no control over the container and will wait for the next stable release. Thanks!
Author
Owner

@minosimo commented on GitHub (Nov 6, 2024):

I am seeing the same behavior on the dev docker image.

Image

Image

<!-- gh-comment-id:2459811894 --> @minosimo commented on GitHub (Nov 6, 2024): I am seeing the same behavior on the dev docker image. ![Image](https://github.com/user-attachments/assets/ecf64e2d-4300-4a6a-9233-53575395cca6) ![Image](https://github.com/user-attachments/assets/678e7ad5-fd60-4999-b8f8-7b6e4ecdcc97)
Author
Owner

@pirate commented on GitHub (Nov 6, 2024):

@minosimo can you confirm you're not seeing any videos in the data/archive/<timestamp>/media/ folders of snapshots of youtube.com urls? Can you share the data/archive/<timestamp>/index.json from one of those captures that you're expecting to see videos in?

To be clear: youtube videos are never playable inside the native youtube UI in the captures, they're extracted out as .mp4 files and are visible labelled as Media with the 📼 icon in the UI and are findable under the media/ folder in the filesystem.

<!-- gh-comment-id:2460694268 --> @pirate commented on GitHub (Nov 6, 2024): @minosimo can you confirm you're not seeing any videos in the `data/archive/<timestamp>/media/` folders of snapshots of youtube.com urls? Can you share the `data/archive/<timestamp>/index.json` from one of those captures that you're expecting to see videos in? To be clear: youtube videos are never playable inside the native youtube UI in the captures, they're extracted out as `.mp4` files and are visible labelled as `Media` with the 📼 icon in the UI and are findable under the `media/` folder in the filesystem.
Author
Owner

@minosimo commented on GitHub (Nov 7, 2024):

Yes, the media folder is empty. I tried with several youtube urls but it looks like the yt-dlp command fails.

index.json

<!-- gh-comment-id:2461663504 --> @minosimo commented on GitHub (Nov 7, 2024): Yes, the media folder is empty. I tried with several youtube urls but it looks like the yt-dlp command fails. [index.json](https://github.com/user-attachments/files/17658811/index.json)
Author
Owner

@nguyenmp commented on GitHub (Nov 11, 2024):

I personally ran into issues with yt-dlp as well and ran the plugin directly to see what the problem was. You can find the full yt-dlp command in your index.json file but you can run it through docker with:

docker compose exec -it --user archivebox archivebox /usr/local/bin/yt-dlp https://youtu.be/sVjk5nrb_lI

I found it I was hitting https://github.com/yt-dlp/yt-dlp/issues/10128 because I was running ArchiveBox on DigitalOcean and YouTube seems to be blocking their whole IP range now. My workaround was to set up a proxy and route yt-dlp traffic through that.

Try running the command directly, you might be hitting the same issue, or maybe something different.

<!-- gh-comment-id:2467111077 --> @nguyenmp commented on GitHub (Nov 11, 2024): I personally ran into issues with `yt-dlp` as well and ran the plugin directly to see what the problem was. You can find the full `yt-dlp` command in your index.json file but you can run it through docker with: ``` docker compose exec -it --user archivebox archivebox /usr/local/bin/yt-dlp https://youtu.be/sVjk5nrb_lI ``` I found it I was hitting https://github.com/yt-dlp/yt-dlp/issues/10128 because I was running ArchiveBox on DigitalOcean and YouTube seems to be blocking their whole IP range now. My workaround was to set up a proxy and route yt-dlp traffic through that. Try running the command directly, you might be hitting the same issue, or maybe something different.
Author
Owner

@nguyenmp commented on GitHub (Nov 11, 2024):

Also, might be worth showing the "standard error" when a plugin command fails. It would make debugging a lot easier.

<!-- gh-comment-id:2467114936 --> @nguyenmp commented on GitHub (Nov 11, 2024): Also, might be worth showing the "standard error" when a plugin command fails. It would make debugging a lot easier.
Author
Owner

@pirate commented on GitHub (Nov 11, 2024):

We used to show stdout/stderr when an extractor failed but even trying to summarize it was quite noisy, and too many people would open issues because they didn't understand that some errors are inevitable with some URLs.

Now we just show the command needed to run to get the fullstdout/stderr, I find it's easer for people to debug that way and many people solve issues on their own when they see the command is wrong / when there is some environment issue.

<!-- gh-comment-id:2467168560 --> @pirate commented on GitHub (Nov 11, 2024): We used to show `stdout`/`stderr` when an extractor failed but even trying to summarize it was quite noisy, and too many people would open issues because they didn't understand that some errors are inevitable with some URLs. Now we just show the command needed to run to get the full`stdout`/`stderr`, I find it's easer for people to debug that way and many people solve issues on their own when they see the command is wrong / when there is some environment issue.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/ArchiveBox#2446
No description provided.