starred/karakeep
1
0
Fork 0
mirror of https://github.com/karakeep-app/karakeep.git synced 2026-04-25 16:06:04 +03:00
Code Issues 569 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #1367] YouTube video not downloading - despite set .env variables #876

New issue
Closed
opened 2026-03-02 11:53:25 +03:00 by kerem · 3 comments
kerem commented 2026-03-02 11:53:25 +03:00
Owner
Copy link

Originally created by @ballerbude on GitHub (May 7, 2025).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/1367

The following 3 env variable are set in the .env:

CRAWLER_VIDEO_DOWNLOAD=true
CRAWLER_VIDEO_DOWNLOAD_MAX_SIZE=-1
CRAWLER_VIDEO_DOWNLOAD_TIMEOUT_SEC=7200

This is the full container log what happens when the URL is pasted in karakeep and save button is pressed:

web-1          | 2025-05-07T11:50:29.805Z info: [search][29] Attempting to index bookmark with id xg1xac3cliwbxjr32v348spi ...
web-1          | 2025-05-07T11:50:29.845Z info: [Crawler][27] Will crawl "https://www.youtube.com/watch?v=i1HgN7u7w_w" for link with id "xg1xac3cliwbxjr32v348spi"
web-1          | 2025-05-07T11:50:29.845Z info: [Crawler][27] Attempting to determine the content-type for the url https://www.youtube.com/watch?v=i1HgN7u7w_w
meilisearch-1  | 2025-05-07T11:50:29.869689Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.97ms time.idle=12.4ms
meilisearch-1  | 2025-05-07T11:50:29.875890Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=166µs time.idle=1.80ms
meilisearch-1  | 2025-05-07T11:50:29.933339Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=148µs time.idle=414µs
meilisearch-1  | 2025-05-07T11:50:30.013462Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=172µs time.idle=829µs
meilisearch-1  | 2025-05-07T11:50:30.071702Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=154µs time.idle=285µs
meilisearch-1  | 2025-05-07T11:50:30.132808Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=185µs time.idle=715µs
meilisearch-1  | 2025-05-07T11:50:30.194398Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=208µs time.idle=1.27ms
web-1          | 2025-05-07T11:50:30.278Z info: [webhook][30] Starting a webhook job for bookmark with id "xg1xac3cliwbxjr32v348spi for operation "created"
web-1          | 2025-05-07T11:50:30.280Z info: [webhook][30] Completed successfully
web-1          | 2025-05-07T11:50:30.345Z info: [ruleEngine][28] Completed successfully
meilisearch-1  | 2025-05-07T11:50:30.368682Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=500µs time.idle=518µs
meilisearch-1  | 2025-05-07T11:50:30.435675Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=909µs time.idle=239µs
meilisearch-1  | 2025-05-07T11:50:30.462496Z  INFO index_scheduler::scheduler::process_index_operation: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 2 } processed_in=600.436154ms
meilisearch-1  | 2025-05-07T11:50:30.472425Z  INFO index_scheduler::scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
meilisearch-1  | 2025-05-07T11:50:30.499710Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=2.74ms time.idle=84.2µs
web-1          | 2025-05-07T11:50:30.552Z info: [search][29] Completed successfully
web-1          | 2025-05-07T11:50:30.693Z info: [Crawler][27] Content-type for the url https://www.youtube.com/watch?v=i1HgN7u7w_w is "text/html; charset=utf-8"
chrome-1       | [0507/115030.888259:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
chrome-1       | [0507/115032.161403:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
chrome-1       | [0507/115037.142370:WARNING:audio_manager_linux.cc(53)] Falling back to ALSA for audio output. PulseAudio is not available or could not be initialized.
chrome-1       | ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
chrome-1       | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
chrome-1       | ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
chrome-1       | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
chrome-1       | ALSA lib confmisc.c:1342:(snd_func_refer) error evaluating name
chrome-1       | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
chrome-1       | ALSA lib conf.c:5727:(snd_config_expand) Evaluate error: No such file or directory
chrome-1       | ALSA lib pcm.c:2675:(snd_pcm_open_noupdate) Unknown PCM default
chrome-1       | [0507/115037.232443:ERROR:alsa_util.cc(204)] PcmOpen: default,No such file or directory
chrome-1       | ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
chrome-1       | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
chrome-1       | ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
chrome-1       | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
chrome-1       | ALSA lib confmisc.c:1342:(snd_func_refer) error evaluating name
chrome-1       | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
chrome-1       | ALSA lib conf.c:5727:(snd_config_expand) Evaluate error: No such file or directory
chrome-1       | ALSA lib pcm.c:2675:(snd_pcm_open_noupdate) Unknown PCM default
chrome-1       | [0507/115037.233018:ERROR:alsa_util.cc(204)] PcmOpen: plug:default,No such file or directory
chrome-1       | [0507/115037.279803:ERROR:web_contents_delegate.cc(260)] WebContentsDelegate::CheckMediaAccessPermission: Not supported.
chrome-1       | [0507/115037.279832:ERROR:web_contents_delegate.cc(260)] WebContentsDelegate::CheckMediaAccessPermission: Not supported.
web-1          | 2025-05-07T11:50:37.485Z info: [Crawler][27] Successfully navigated to "https://www.youtube.com/watch?v=i1HgN7u7w_w". Waiting for the page to load ...
web-1          | 2025-05-07T11:50:39.044Z info: [Crawler][27] Finished waiting for the page to load.
web-1          | 2025-05-07T11:50:39.823Z info: [Crawler][27] Successfully fetched the page content.
web-1          | 2025-05-07T11:50:42.076Z info: [Crawler][27] Finished capturing page content and a screenshot. FullPageScreenshot: true
web-1          | 2025-05-07T11:50:42.091Z info: [Crawler][27] Will attempt to extract metadata from page ...
web-1          | 2025-05-07T11:50:48.922Z info: [Crawler][27] Will attempt to extract readable content ...
web-1          | 2025-05-07T11:50:54.135Z info: [Crawler][27] Done extracting readable content.
web-1          | 2025-05-07T11:50:56.236Z info: [Crawler][27] Stored the screenshot as assetId: d08e1304-e2de-4c3d-92e9-2695ad8a1fdc
web-1          | 2025-05-07T11:50:56.264Z info: [Crawler][27] Done extracting metadata from the page.
web-1          | 2025-05-07T11:50:56.264Z info: [Crawler][27] Downloading image from "https://yt3.ggpht.com/OdKb6lPyN0jP_NAqbSy2CsSOoRk9obh4EleQQEi4PwwmmrP3FGznZuTJ8Qle3On2s-r_9cMe=s48-c-k-c0x00ffffff-no-rj"
web-1          | 2025-05-07T11:50:56.330Z info: [Crawler][27] Downloaded image as assetId: 9cc7f37f-091c-435f-9004-4c75388b8890
web-1          | 2025-05-07T11:50:56.572Z info: [Crawler][27] Will attempt to archive page ...
web-1          | 2025-05-07T11:50:57.524Z info: [webhook][34] Starting a webhook job for bookmark with id "xg1xac3cliwbxjr32v348spi for operation "crawled"
web-1          | 2025-05-07T11:50:57.527Z info: [webhook][34] Completed successfully
web-1          | 2025-05-07T11:50:57.764Z info: [search][32] Attempting to index bookmark with id xg1xac3cliwbxjr32v348spi ...
web-1          | 2025-05-07T11:50:57.807Z debug: [inference][31] No inference client configured, nothing to do now
web-1          | 2025-05-07T11:50:57.807Z info: [inference][31] Completed successfully
web-1          | 2025-05-07T11:50:57.876Z info: [VideoCrawler][33] Attempting to download a file from "https://www.youtube.com/watch?v=i1HgN7u7w_w" to "/tmp/video_downloads/c9e11679-a038-43ec-b873-472168e612a0" using the following arguments: "https://www.youtube.com/watch?v=i1HgN7u7w_w,-o,/tmp/video_downloads/c9e11679-a038-43ec-b873-472168e612a0,--no-playlist"
meilisearch-1  | 2025-05-07T11:50:57.908547Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=700µs time.idle=11.7ms
meilisearch-1  | 2025-05-07T11:50:57.933682Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=1.09ms time.idle=2.78ms
meilisearch-1  | 2025-05-07T11:50:58.012629Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=226µs time.idle=4.81ms
meilisearch-1  | 2025-05-07T11:50:58.126315Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=159µs time.idle=385µs
meilisearch-1  | 2025-05-07T11:50:58.185126Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=348µs time.idle=74.8µs
meilisearch-1  | 2025-05-07T11:50:58.219836Z  INFO index_scheduler::scheduler::process_index_operation: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 2 } processed_in=310.649311ms
meilisearch-1  | 2025-05-07T11:50:58.250859Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=172µs time.idle=2.58ms
meilisearch-1  | 2025-05-07T11:50:58.261674Z  INFO index_scheduler::scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
meilisearch-1  | 2025-05-07T11:50:58.322197Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=2.68ms time.idle=82.0µs
web-1          | 2025-05-07T11:50:58.360Z info: [search][32] Completed successfully
web-1          | 2025-05-07T11:51:06.378Z info: [Crawler][27] Done archiving the page as assetId: 12c3848d-d89a-43d3-9d15-75eac7ec5909
web-1          | 2025-05-07T11:51:06.503Z info: [Crawler][27] Completed successfully
web-1          | 2025-05-07T11:51:08.321Z error: [VideoCrawler][33] Failed to download a file from "https://www.youtube.com/watch?v=i1HgN7u7w_w" to "/tmp/video_downloads/c9e11679-a038-43ec-b873-472168e612a0"
web-1          | 2025-05-07T11:51:08.361Z info: [VideoCrawler][33] Video Download Completed successfully

It seems to affect all YouTube videos. Am I missing something? Any additional configuration required?

Image

Originally created by @ballerbude on GitHub (May 7, 2025). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/1367 The following 3 env variable are set in the .env: ``` CRAWLER_VIDEO_DOWNLOAD=true CRAWLER_VIDEO_DOWNLOAD_MAX_SIZE=-1 CRAWLER_VIDEO_DOWNLOAD_TIMEOUT_SEC=7200 ``` This is the full container log what happens when the URL is pasted in karakeep and save button is pressed: ``` web-1 | 2025-05-07T11:50:29.805Z info: [search][29] Attempting to index bookmark with id xg1xac3cliwbxjr32v348spi ... web-1 | 2025-05-07T11:50:29.845Z info: [Crawler][27] Will crawl "https://www.youtube.com/watch?v=i1HgN7u7w_w" for link with id "xg1xac3cliwbxjr32v348spi" web-1 | 2025-05-07T11:50:29.845Z info: [Crawler][27] Attempting to determine the content-type for the url https://www.youtube.com/watch?v=i1HgN7u7w_w meilisearch-1 | 2025-05-07T11:50:29.869689Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.97ms time.idle=12.4ms meilisearch-1 | 2025-05-07T11:50:29.875890Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=166µs time.idle=1.80ms meilisearch-1 | 2025-05-07T11:50:29.933339Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=148µs time.idle=414µs meilisearch-1 | 2025-05-07T11:50:30.013462Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=172µs time.idle=829µs meilisearch-1 | 2025-05-07T11:50:30.071702Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=154µs time.idle=285µs meilisearch-1 | 2025-05-07T11:50:30.132808Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=185µs time.idle=715µs meilisearch-1 | 2025-05-07T11:50:30.194398Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=208µs time.idle=1.27ms web-1 | 2025-05-07T11:50:30.278Z info: [webhook][30] Starting a webhook job for bookmark with id "xg1xac3cliwbxjr32v348spi for operation "created" web-1 | 2025-05-07T11:50:30.280Z info: [webhook][30] Completed successfully web-1 | 2025-05-07T11:50:30.345Z info: [ruleEngine][28] Completed successfully meilisearch-1 | 2025-05-07T11:50:30.368682Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=500µs time.idle=518µs meilisearch-1 | 2025-05-07T11:50:30.435675Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=909µs time.idle=239µs meilisearch-1 | 2025-05-07T11:50:30.462496Z INFO index_scheduler::scheduler::process_index_operation: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 2 } processed_in=600.436154ms meilisearch-1 | 2025-05-07T11:50:30.472425Z INFO index_scheduler::scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. meilisearch-1 | 2025-05-07T11:50:30.499710Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/11 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=2.74ms time.idle=84.2µs web-1 | 2025-05-07T11:50:30.552Z info: [search][29] Completed successfully web-1 | 2025-05-07T11:50:30.693Z info: [Crawler][27] Content-type for the url https://www.youtube.com/watch?v=i1HgN7u7w_w is "text/html; charset=utf-8" chrome-1 | [0507/115030.888259:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. chrome-1 | [0507/115032.161403:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. chrome-1 | [0507/115037.142370:WARNING:audio_manager_linux.cc(53)] Falling back to ALSA for audio output. PulseAudio is not available or could not be initialized. chrome-1 | ALSA lib confmisc.c:855:(parse_card) cannot find card '0' chrome-1 | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory chrome-1 | ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings chrome-1 | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory chrome-1 | ALSA lib confmisc.c:1342:(snd_func_refer) error evaluating name chrome-1 | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory chrome-1 | ALSA lib conf.c:5727:(snd_config_expand) Evaluate error: No such file or directory chrome-1 | ALSA lib pcm.c:2675:(snd_pcm_open_noupdate) Unknown PCM default chrome-1 | [0507/115037.232443:ERROR:alsa_util.cc(204)] PcmOpen: default,No such file or directory chrome-1 | ALSA lib confmisc.c:855:(parse_card) cannot find card '0' chrome-1 | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory chrome-1 | ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings chrome-1 | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory chrome-1 | ALSA lib confmisc.c:1342:(snd_func_refer) error evaluating name chrome-1 | ALSA lib conf.c:5204:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory chrome-1 | ALSA lib conf.c:5727:(snd_config_expand) Evaluate error: No such file or directory chrome-1 | ALSA lib pcm.c:2675:(snd_pcm_open_noupdate) Unknown PCM default chrome-1 | [0507/115037.233018:ERROR:alsa_util.cc(204)] PcmOpen: plug:default,No such file or directory chrome-1 | [0507/115037.279803:ERROR:web_contents_delegate.cc(260)] WebContentsDelegate::CheckMediaAccessPermission: Not supported. chrome-1 | [0507/115037.279832:ERROR:web_contents_delegate.cc(260)] WebContentsDelegate::CheckMediaAccessPermission: Not supported. web-1 | 2025-05-07T11:50:37.485Z info: [Crawler][27] Successfully navigated to "https://www.youtube.com/watch?v=i1HgN7u7w_w". Waiting for the page to load ... web-1 | 2025-05-07T11:50:39.044Z info: [Crawler][27] Finished waiting for the page to load. web-1 | 2025-05-07T11:50:39.823Z info: [Crawler][27] Successfully fetched the page content. web-1 | 2025-05-07T11:50:42.076Z info: [Crawler][27] Finished capturing page content and a screenshot. FullPageScreenshot: true web-1 | 2025-05-07T11:50:42.091Z info: [Crawler][27] Will attempt to extract metadata from page ... web-1 | 2025-05-07T11:50:48.922Z info: [Crawler][27] Will attempt to extract readable content ... web-1 | 2025-05-07T11:50:54.135Z info: [Crawler][27] Done extracting readable content. web-1 | 2025-05-07T11:50:56.236Z info: [Crawler][27] Stored the screenshot as assetId: d08e1304-e2de-4c3d-92e9-2695ad8a1fdc web-1 | 2025-05-07T11:50:56.264Z info: [Crawler][27] Done extracting metadata from the page. web-1 | 2025-05-07T11:50:56.264Z info: [Crawler][27] Downloading image from "https://yt3.ggpht.com/OdKb6lPyN0jP_NAqbSy2CsSOoRk9obh4EleQQEi4PwwmmrP3FGznZuTJ8Qle3On2s-r_9cMe=s48-c-k-c0x00ffffff-no-rj" web-1 | 2025-05-07T11:50:56.330Z info: [Crawler][27] Downloaded image as assetId: 9cc7f37f-091c-435f-9004-4c75388b8890 web-1 | 2025-05-07T11:50:56.572Z info: [Crawler][27] Will attempt to archive page ... web-1 | 2025-05-07T11:50:57.524Z info: [webhook][34] Starting a webhook job for bookmark with id "xg1xac3cliwbxjr32v348spi for operation "crawled" web-1 | 2025-05-07T11:50:57.527Z info: [webhook][34] Completed successfully web-1 | 2025-05-07T11:50:57.764Z info: [search][32] Attempting to index bookmark with id xg1xac3cliwbxjr32v348spi ... web-1 | 2025-05-07T11:50:57.807Z debug: [inference][31] No inference client configured, nothing to do now web-1 | 2025-05-07T11:50:57.807Z info: [inference][31] Completed successfully web-1 | 2025-05-07T11:50:57.876Z info: [VideoCrawler][33] Attempting to download a file from "https://www.youtube.com/watch?v=i1HgN7u7w_w" to "/tmp/video_downloads/c9e11679-a038-43ec-b873-472168e612a0" using the following arguments: "https://www.youtube.com/watch?v=i1HgN7u7w_w,-o,/tmp/video_downloads/c9e11679-a038-43ec-b873-472168e612a0,--no-playlist" meilisearch-1 | 2025-05-07T11:50:57.908547Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=700µs time.idle=11.7ms meilisearch-1 | 2025-05-07T11:50:57.933682Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=1.09ms time.idle=2.78ms meilisearch-1 | 2025-05-07T11:50:58.012629Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=226µs time.idle=4.81ms meilisearch-1 | 2025-05-07T11:50:58.126315Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=159µs time.idle=385µs meilisearch-1 | 2025-05-07T11:50:58.185126Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=348µs time.idle=74.8µs meilisearch-1 | 2025-05-07T11:50:58.219836Z INFO index_scheduler::scheduler::process_index_operation: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 2 } processed_in=310.649311ms meilisearch-1 | 2025-05-07T11:50:58.250859Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=172µs time.idle=2.58ms meilisearch-1 | 2025-05-07T11:50:58.261674Z INFO index_scheduler::scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. meilisearch-1 | 2025-05-07T11:50:58.322197Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/12 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=2.68ms time.idle=82.0µs web-1 | 2025-05-07T11:50:58.360Z info: [search][32] Completed successfully web-1 | 2025-05-07T11:51:06.378Z info: [Crawler][27] Done archiving the page as assetId: 12c3848d-d89a-43d3-9d15-75eac7ec5909 web-1 | 2025-05-07T11:51:06.503Z info: [Crawler][27] Completed successfully web-1 | 2025-05-07T11:51:08.321Z error: [VideoCrawler][33] Failed to download a file from "https://www.youtube.com/watch?v=i1HgN7u7w_w" to "/tmp/video_downloads/c9e11679-a038-43ec-b873-472168e612a0" web-1 | 2025-05-07T11:51:08.361Z info: [VideoCrawler][33] Video Download Completed successfully ``` It seems to affect all YouTube videos. Am I missing something? Any additional configuration required? ![Image](https://github.com/user-attachments/assets/36cbfe3e-c963-448a-b288-82e78999374a)
kerem closed this issue 2026-03-02 11:53:26 +03:00
kerem commented 2026-03-02 11:53:26 +03:00
Author
Owner
Copy link

@vhsdream commented on GitHub (May 7, 2025):

You might need to also add some yt-dl arguments to your config. I was having the same issue and the following helped with some (but not all) of the Youtube videos:

CRAWLER_YTDLP_ARGS="--max-filesize=500M"

Adjust the max filesize to suit your needs of course.

<!-- gh-comment-id:2858376983 --> @vhsdream commented on GitHub (May 7, 2025): You might need to also add some yt-dl arguments to your config. I was having the same issue and the following helped with some (but not all) of the Youtube videos: `CRAWLER_YTDLP_ARGS="--max-filesize=500M"` Adjust the max filesize to suit your needs of course.
kerem commented 2026-03-02 11:53:26 +03:00
Author
Owner
Copy link

@ballerbude commented on GitHub (May 7, 2025):

Thank you for the quick response. Unfortunately, that doesn't help. I see the argument being passed, but the result is still the same:

web-1          | 2025-05-07T12:54:58.185Z info: [VideoCrawler][25] Attempting to download a file from "https://www.youtube.com/watch?v=3G6I46efADo" to "/tmp/video_downloads/d0be9a31-d0b1-4ec6-b90c-f52db8f9bdcc" using the following arguments: "https://www.youtube.com/watch?v=3G6I46efADo,--max-filesize=999M,-o,/tmp/video_downloads/d0be9a31-d0b1-4ec6-b90c-f52db8f9bdcc,--no-playlist"
meilisearch-1  | 2025-05-07T12:54:58.489222Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=5.28ms time.idle=13.9ms
meilisearch-1  | 2025-05-07T12:54:58.524947Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/10 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=4.48ms time.idle=296µs
meilisearch-1  | 2025-05-07T12:54:58.647487Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/10 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=181µs time.idle=786µs
meilisearch-1  | 2025-05-07T12:54:58.722324Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/10 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=180µs time.idle=2.41ms
meilisearch-1  | 2025-05-07T12:54:58.811855Z  INFO index_scheduler::scheduler::process_index_operation: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 1 } processed_in=321.004113ms
meilisearch-1  | 2025-05-07T12:54:58.849277Z  INFO index_scheduler::scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
meilisearch-1  | 2025-05-07T12:54:58.899883Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/10 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=1.94ms time.idle=105µs
meilisearch-1  | 2025-05-07T12:54:59.213341Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/10 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=197µs time.idle=9.46ms
web-1          | 2025-05-07T12:54:59.272Z info: [search][24] Completed successfully
web-1          | 2025-05-07T12:55:12.347Z error: [VideoCrawler][25] Failed to download a file from "https://www.youtube.com/watch?v=3G6I46efADo" to "/tmp/video_downloads/d0be9a31-d0b1-4ec6-b90c-f52db8f9bdcc"
web-1          | 2025-05-07T12:55:12.358Z info: [VideoCrawler][25] Video Download Completed successfully
web-1          | 2025-05-07T12:55:14.575Z info: [Crawler][19] Done archiving the page as assetId: 3ce2818f-c978-42f9-b433-77755b8eb736
web-1          | 2025-05-07T12:55:14.711Z info: [Crawler][19] Completed successfully
<!-- gh-comment-id:2858492425 --> @ballerbude commented on GitHub (May 7, 2025): Thank you for the quick response. Unfortunately, that doesn't help. I see the argument being passed, but the result is still the same: ``` web-1 | 2025-05-07T12:54:58.185Z info: [VideoCrawler][25] Attempting to download a file from "https://www.youtube.com/watch?v=3G6I46efADo" to "/tmp/video_downloads/d0be9a31-d0b1-4ec6-b90c-f52db8f9bdcc" using the following arguments: "https://www.youtube.com/watch?v=3G6I46efADo,--max-filesize=999M,-o,/tmp/video_downloads/d0be9a31-d0b1-4ec6-b90c-f52db8f9bdcc,--no-playlist" meilisearch-1 | 2025-05-07T12:54:58.489222Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=5.28ms time.idle=13.9ms meilisearch-1 | 2025-05-07T12:54:58.524947Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/10 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=4.48ms time.idle=296µs meilisearch-1 | 2025-05-07T12:54:58.647487Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/10 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=181µs time.idle=786µs meilisearch-1 | 2025-05-07T12:54:58.722324Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/10 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=180µs time.idle=2.41ms meilisearch-1 | 2025-05-07T12:54:58.811855Z INFO index_scheduler::scheduler::process_index_operation: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 1 } processed_in=321.004113ms meilisearch-1 | 2025-05-07T12:54:58.849277Z INFO index_scheduler::scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. meilisearch-1 | 2025-05-07T12:54:58.899883Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/10 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=1.94ms time.idle=105µs meilisearch-1 | 2025-05-07T12:54:59.213341Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/10 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=197µs time.idle=9.46ms web-1 | 2025-05-07T12:54:59.272Z info: [search][24] Completed successfully web-1 | 2025-05-07T12:55:12.347Z error: [VideoCrawler][25] Failed to download a file from "https://www.youtube.com/watch?v=3G6I46efADo" to "/tmp/video_downloads/d0be9a31-d0b1-4ec6-b90c-f52db8f9bdcc" web-1 | 2025-05-07T12:55:12.358Z info: [VideoCrawler][25] Video Download Completed successfully web-1 | 2025-05-07T12:55:14.575Z info: [Crawler][19] Done archiving the page as assetId: 3ce2818f-c978-42f9-b433-77755b8eb736 web-1 | 2025-05-07T12:55:14.711Z info: [Crawler][19] Completed successfully ```
kerem commented 2026-03-02 11:53:26 +03:00
Author
Owner
Copy link

@ballerbude commented on GitHub (May 8, 2025):

Okay, it was previously installed on a Ubuntu 22.04 Server VM, now I used the same docker-compose.yml and the same .env on another VM with Ubuntu 24.04 and YouTube downloads are working. I have no clue why, but I guess it's fixed.

<!-- gh-comment-id:2862319431 --> @ballerbude commented on GitHub (May 8, 2025): Okay, it was previously installed on a Ubuntu 22.04 Server VM, now I used the same docker-compose.yml and the same .env on another VM with Ubuntu 24.04 and YouTube downloads are working. I have no clue why, but I guess it's fixed.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
refactor/use-npm-singlefile
onetab
claude/issue-2596-20260321-1401
claude/fix-docs-button-responsive-V3aBQ
claude/review-import-backpressure-D4ArJ
claude/fix-archived-bookmarks-mobile-P9OJW
claude/issue-1189-20260211-1601
claude/fix-nested-smart-lists-3uFkt
claude/issue-2298-20251223-1704
feat/import-v3
claude/add-cli-search-subcommand-6kIe0
claude/add-bookmark-indexing-timestamps-96bPj
claude/auto-disable-failing-feeds-fkDhP
claude/add-tag-search-aliases-HzESD
feat/docker-compose-dev
claude/add-attachedby-tags-endpoint-01WYfemMGHJJjXsPYLvUJAno
claude/fix-crawler-memory-leaks-NE7Ct
bookmark-debugger
claude/issue-2352-20260106-1120
claude/issue-1977-20260102-2348
claude/add-banner-rendering-JeLUk
claude/add-descendant-qualifier-cUm26
claude/skip-metadata-refresh-archives-CAo4Y
claude/fix-archive-pending-banner-pAyGM
claude/add-embeddings-support-h2swV
claude/nested-manage-lists-QVV85
claude/privacy-type-system-MG1bT
claude/add-action-menu-icons-6hNKw
claude/issue-2299-20251223-1711
claude/bookmark-indexing-progress-QwZSI
claude/migrate-bookmark-attachments-3O2te
claude/add-2025-wrapped-feature-tIUIh
claude/improve-ai-settings-design-639tq
claude/add-youtube-metascraper-plugin-0lWC7
claude/add-problem-reporting-gSSEV
claude/add-mobile-list-menus-spcS7
claude/shadcn-bookmark-cards-WWHzP
claude/add-extensions-link-HTeXc
claude/add-onboarding-screens-hsYMO
claude/fix-settings-switch-overflow-nlzM4
claude/clamp-bookmark-titles-diAEz
claude/port-stats-mobile-expo-MuXAn
claude/whats-new-base-version-vrv8C
claude/fix-settings-auth-checks-jgyD8
claude/add-server-version-display-3sGa2
claude/fix-tag-editor-scrolling-rzdbG
claude/add-company-pricing-card-y5mHY
claude/audit-optimize-transactions-xpDVc
codex/ensure-consistent-ui-experience-across-app-pages
claude/plan-opentelemetry-integration-01Jx183mz1Ev8h8JoYj97Auw
libsql
db-indicies
claude/export-import-lists-01UuCWwdaqduAd35NppvjnMD
claude/configurable-worker-timeout-0198GQh6YrrRzqG62xnogyrz
claude/check-import-quota-01CPdxTpHp18Ba62bYcBTVbA
claude/scraper-worker-thread-01FEHen6MGrQHmdBstJSuiyA
claude/customize-dialog-styling-01CVjEv2KgyZJSpCg3mqkvR7
claude/add-asset-cache-headers-0175WhNcqwiwurrmjj52jnLT
claude/add-db-search-plugin-017Xxd4Jq3MfjWT788vgfbaq
benchmarks-2
claude/add-filtered-deletion-01DTxWNcg3hhqdNpeNLa3s6L
claude/actionbutton-loading-spinner-015DY5ZTvgPgFAXTZz3UGaYv
claude/add-broken-links-qualifier-01S31X1LsKiYb9gE1dXTKvi3
claude/docker-release-tag-trigger-01UmzFXEumhK2jdmRGtMcueo
claude/spread-feed-fetch-scheduling-01EihUtmZSyqeE1HfRMessxW
restate-idempotency
claude/align-android-ios-colors-01GJfkhEyZVBReohVioPa8ok
claude/improve-mobile-app-colors-0155LzHfkd5HyJr6YyZMsus5
codex/add-autocomplete-for-search-query-language
claude/add-bookmark-backups-016L2A8Z94n7tDgDdMPdFuAd
claude/restrict-binary-user-permissions-01FSGyy2RXGZvE26YbAejzGi
effect-ts
claude/prepare-trpc-npm-publish-0193EjfwpxSNVNcLXqXjs6Ln
shared-list-sidebar
claude/lazy-load-tiktoken-017UTNpJPTcMMQvNEBa1aFwo
codex/fix-asset-pre-processing-worker-abort-signals
add-groupid
claude/add-bookmark-list-button-01VF7uXYNLsVDzqdozWMXP5M
claude/extract-shared-ui-components-01DSVfaCr6WRqAyx1vJTZk9r
claude/migrate-shadcn-sidebar-01DKjpg9MD5PJ2potemSnbvW
claude/add-collaborators-rate-limits-01VjXyRWWPUkGQKa8d8D8qKj
claude/modernize-dark-mode-01FRfE81PAY5C44pFu1cYocf
claude/add-signed-url-bookmark-01PjYT1ZhvLK2FPJNTAhJsWf
restate-group-id
claude/add-highlights-page-012vhHpn8fVNp3gf7gBeW14s
claude/disable-shared-bookmark-features-01B9fiGUdu6NyWaxSQFsQBxP
claude/mobile-bookmark-grid-layouts-018cGBBMhPJVq6PJVRBpqT2r
claude/add-mobile-bookmark-summary-01494LYoh4sJW5Fj4GPm62Vj
claude/add-mobile-tags-screen-01WRADt4ZzvXVew1Y9vqF8SV
claude/add-highlight-notes-01LpanRLS4a2YMnT1qB5GTqX
claude/add-search-bar-014k2ngaqjwYRVSvqmbuECqr
claude/hide-collaborator-emails-01TQrkkMupC7CR9BTuDkireg
claude/list-invitation-approval-0129V89M1riXW6JqmoF74VfM
claude/add-bookmark-archive-sort-018VbGPGvtmsGgXFEERoAX7B
claude/add-mobile-smart-lists-01251tYo9u1SywE6XFezAv9e
claude/bookmark-drag-drop-01DmWq286ogHpDGHKcXjKr3z
claude/add-rss-import-01DH1Q2axcDeq8nQJR5MWjPJ
claude/mobile-inapp-browser-auth-01KiT6bwyntRPQ1X4oTtAveC
claude/offline-mode-react-query-01D1rE2bdBEPw2teGqunr5Gd
claude/add-singlefile-extension-support-01BEB9QQZABzwfZDvR9Bz5b2
claude/custom-list-slugs-01VxcfkNUXZ97FNpNVURopMq
claude/issue-2148-20251118-1133
claude/add-groupid-queue-fairness-011CV1r8Wb46HuGAg5o95i3m
claude/hide-viewer-shared-lists-01Fst6NBvdxrXXnDhUmjsNDP
claude/collaborative-lists-013AvDvMqkoszDVcSoCYgBcM
claude/implement-feature-01LT5XzGsbEhZkYXNEjEwdui
claude/fix-bookmark-loading-state-01AgF4H2drxwuTCJDB2Xgiu4
claude/admin-user-edit-013tbiRmb1KX2fhSYqmGKCu8
claude/expose-all-api-01YTruEW72WQYMtq4iZoaPkA
claude/add-doc-link-main-016NYLxShpKuH6R8XCBgeZtc
claude/fix-issue-2133-019JLvdSRAUbU4FtjQztcM6S
claude/explore-effect-ts-integration-01F7xb1dWwP1ma4LnLbFGfDD
claude/optimize-dockerfile-build-011CV5gDnPZbdbbVSPDofC4e
claude/add-custom-headers-guide-011CV249t16aWDRb1mCrzQdC
claude/mobile-app-signup-011CUxPtCXgU6U3T8GShTR2Q
claude/crawler-worker-fetch-browser-011CUvcRc24XEr9DTWDW6MX8
claude/fix-issue-784-011CUvubQrcZHG9S3KjpCKbK
codex/add-user-settings-for-inference-language-and-screenshots
claude/fix-mobile-signin-server-address-011CUnaUWwY2Fhq5Xbwhgr8H
better-auth-2
claude/issue-2028-20251012-1429
claude/issue-1010-20251012-1154
codex/update-feed-refresh-job-idempotency-key
restate
import-v2
fix-public-lists
recurse-delete-list
abort-dangling-processing
tag-pagination
ratelimit-plugin
claude/issue-1937-20250914-0912
codex/implement-title-search-query-qualifier
copilot/add-edit-button-for-notes
cookie-path
ai-tag-cleanup
codex/add-allowlist-and-blocklist-env-variables
mobile-retheme
expo-next-upgrade
opencode/issue1788-20250727215611
fix-trailing-slash-deduplication
edit-bookmark-dialog
bookmark-embeddings
rag
nextjs-15
bookmark-hover-bar
sapling-pr-archive-MohamedBassem
track-bookmark-assets
json-cli
admin-settings
mobile-dark-mode
android/v1.9.2-0
ios/v1.9.1-1
android/v1.9.1-0
ios/v1.9.1-0
ios/v1.9.0-2
ios/v1.9.0-1
android/v1.9.0-1
extension/v1.2.9
cli/v0.31.0
sdk/v0.31.0
mcp/v0.31.0
android/v1.9.0-0
ios/v1.9.0-0
v0.31.0
android/v1.8.5-0
cli/v0.30.0
sdk/v0.30.0
ios/v1.8.4-0
android/v1.8.4-0
v0.30.0
cli/v0.29.1
v0.29.3
v0.29.2
v0.29.1
sdk/v0.29.0
cli/v0.29.0
mcp/v0.29.0
ios/v1.8.3-0
android/v1.8.3-0
extension/v1.2.8
v0.29.0
android/v1.8.2-2
android/v1.8.2-1
ios/v1.8.2-0
android/v1.8.2-0
extension/v1.2.7
android/v1.8.1-0
ios/v1.8.1-0
v0.28.0
cli/v0.27.1
cli/v0.27.0
v0.27.1
sdk/v0.27.0
v0.27.0
android/v1.8.0-1
ios/v1.8.0-1
mcp/v0.26.0
sdk/v0.26.0
v0.26.0
cli/v0.25.0
ios/v1.7.0-1
mcp/v0.25.0
v0.25.0
extension/v1.2.6
ios/v1.7.0-0
android/v1.7.0-0
v0.24.1
v0.24.0
mcp/v0.23.10
mcp/v0.23.9
mcp/v0.23.8
extension/v1.2.5
mcp/v0.23.7
mcp/v0.23.6
mcp/v0.23.5
mcp/v0.23.4
sdk/v0.23.2
cli/v0.23.0
extension/v1.2.4
android/v1.6.9-1
ios/v1.6.9-1
v0.23.2
v0.23.1
sdk/v0.23.0
v0.23.0
ios/v1.6.9-0
sdk/v0.22.0
v0.22.0
android/v1.6.8-0
ios/v1.6.8-0
sdk/v0.21.2
sdk/v0.21.1
sdk/v0.21.0
v0.21.0
cli/v0.20.0
v0.20.0
ios/v1.6.7-4
android/v1.6.7-4
ios/v1.6.7-3
android/v1.6.7-3
android/v1.6.7-2
ios/v1.6.7-2
android/v1.6.7-1
ios/v1.6.7-1
ios/v1.6.7-0
android/v1.6.7-0
v0.19.0
android/v1.6.6-0
android/v1.6.5-0
ios/v1.6.5-0
ios/v1.6.4-0
android/v1.6.4-0
v0.18.0
v0.17.1
v0.17.0
ios/v1.6.3-0
android/v1.6.3-0
extension/v1.2.3
ios/v1.6.2-1
android/v1.6.2-1
ios/v1.6.2-0
android/v1.6.2-0
v0.16.0
ios/v1.6.1-3
android/v1.6.1-3
ios/v1.6.1-2
android/v1.6.1-2
android/v1.6.1-1
ios/v1.6.1-1
android/v1.6.1-0
ios/v1.6.1-0
extension/v1.2.2
android/v1.6.0-1
ios/v1.6.0-1
ios/v1.6.0
android/v1.6.0
cli/v0.13.7
cli/v0.13.6
v0.15.0
cli/v0.13.5
extension/v1.2.1
v0.14.0
cli/v0.13.3
cli/v0.13.2
cli/v0.13.1
cli/v0.13.0
v0.13.1
v0.13.0
mobile-v1.5.0
mobile-v1.4.0
v0.12.2
v0.12.1
v0.12.0
v0.11.1
v0.11.0
v0.10.1
v0.10.0
v0.9.0
v0.8.0
v0.7.0
v0.6.0
v0.5.0
v0.4.1
v.0.4.0
v.0.3.1
v0.3.0
v0.2.2
v0.2.1
v0.2.0
v0.1.0
Labels
Clear labels   
UI/UX

   
android

   
bug

   
dependencies

   
documentation

   
documentation

   
extension

   
feature request

   
feature request

   
good first issue

   
ios

   
long-term

   
performance

   
pri/high

   
pri/low

   
pri/medium

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
status/approved

   
status/icebox

   
status/pending_clarification

   
status/untriaged
No labels
UI/UX
android
bug
dependencies
documentation
documentation
extension
feature request
feature request
good first issue
ios
long-term
performance
pri/high
pri/low
pri/medium
pull-request
question
status/approved
status/icebox
status/pending_clarification
status/untriaged
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/karakeep#876
No description provided.