starred/karakeep
1
0
Fork 0
mirror of https://github.com/karakeep-app/karakeep.git synced 2026-04-25 07:56:05 +03:00
Code Issues 569 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #978] Link Banners Not Loading #647

New issue
Open
opened 2026-03-02 11:51:37 +03:00 by kerem · 12 comments
kerem commented 2026-03-02 11:51:37 +03:00
Owner
Copy link

Originally created by @lmalmoreno on GitHub (Feb 4, 2025).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/978

Describe the Bug

Hello,

I am a self-hosting enthusiast and could hardly believe it when I found this project. ChatGPT recommended it to me, and it's exactly what I need. That said, I want to express my deep gratitude for the project and the time you've invested in it.

This past weekend, I decided to get it running on my system. I followed all the instructions in the installation manual (docker-compose), and it promptly started running on my small Linux server (DietPi, PC x86). Although everything seemed fine, I'm encountering some issues and couldn't find any solutions in the manuals or on Discord.

The problem is that the link banners don't load, displaying a "card banner" message instead (this only happens on the web page; in the iOS app, everything works fine). Another issue occurs in Edge/Chrome: after some time of use, a message appears saying, "Application error: a client-side exception has occurred (see the browser console for more information)," and it's no longer possible to access the interface. Several errors are also generated in the browser console. So far, I haven't had this problem in Firefox, but errors do appear in the browser console.

I also tried running the system in Docker on Windows 10 but experienced the same behavior. I've attached screenshots, container logs, and my configuration files. I'm new to Docker, so if I'm making any mistakes, I apologize in advance.

I'm currently running version 0.22.0, but this behavior also occurred in the previous version.

Thank you very much for this incredible project!

docker-compose:

services:
  web:
    image: ghcr.io/hoarder-app/hoarder:${HOARDER_VERSION:-release}
    restart: unless-stopped
    volumes:
      - data:/data
    ports:
      - 3000:3000
    env_file:
      - .env
    environment:
      MEILI_ADDR: http://meilisearch:7700
      BROWSER_WEB_URL: http://chrome:9222
      # OPENAI_API_KEY: ...
      DATA_DIR: /data
  chrome:
    image: gcr.io/zenika-hub/alpine-chrome:123
    restart: unless-stopped
    command:
      - --no-sandbox
      - --disable-gpu
      - --disable-dev-shm-usage
      - --remote-debugging-address=0.0.0.0
      - --remote-debugging-port=9222
      - --hide-scrollbars
  meilisearch:
    image: getmeili/meilisearch:v1.11.1
    restart: unless-stopped
    env_file:
      - .env
    environment:
      MEILI_NO_ANALYTICS: "true"
    volumes:
      - meilisearch:/meili_data

volumes:
  meilisearch:
  data:

.env:

HOARDER_VERSION=release
NEXTAUTH_SECRET=*************************
MEILI_MASTER_KEY=************************
NEXTAUTH_URL=http://192.168.0.2:3000/
OPENAI_API_KEY=**************************
CRAWLER_FULL_PAGE_ARCHIVE=true
OCR_LANGS=eng,por
CRAWLER_DOWNLOAD_BANNER_IMAGE=true
CRAWLER_STORE_SCREENSHOT=true

chrome:

[0203/031223.179528:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/031223.214397:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/031223.214841:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/031223.231575:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig.
[0203/031223.418146:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
[0203/031223.418263:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
[0203/031223.556772:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable.
DevTools listening on ws://0.0.0.0:9222/devtools/browser/96b49013-8b36-4107-9dc0-bdb2082f859f
[0203/031223.714719:WARNING:sandbox_linux.cc(418)] InitializeSandbox() called with multiple threads in process gpu-process.
[0203/031224.047573:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig.
[0203/122601.492487:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/122602.718145:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/122604.207074:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/122604.405135:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/122611.999333:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/122635.562933:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/122640.439652:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/122640.439834:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/122640.440503:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/122640.440592:WARNING:property.cc(144)] DaemonVersion: GetAndBlock: failed.
[0203/122640.440942:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/122640.441242:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/122642.742224:ERROR:web_contents_delegate.cc(260)] WebContentsDelegate::CheckMediaAccessPermission: Not supported.
[0203/122642.742310:ERROR:web_contents_delegate.cc(260)] WebContentsDelegate::CheckMediaAccessPermission: Not supported.
[0203/122643.612672:WARNING:audio_manager_linux.cc(53)] Falling back to ALSA for audio output. PulseAudio is not available or could not be initialized.
[0203/122652.433558:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/122800.763669:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/122900.610360:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/123000.647172:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/123100.644500:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/123107.552855:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
[0203/123107.552978:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
[0203/123203.195990:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/123948.729055:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/123948.734221:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig.
[0203/123948.743611:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/123948.750325:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[0203/123948.811181:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
[0203/123948.811379:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
DevTools listening on ws://0.0.0.0:9222/devtools/browser/78afe4f8-20eb-4917-9703-929dca21e8fe
[0203/123948.863655:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable.
[0203/123948.973638:WARNING:sandbox_linux.cc(418)] InitializeSandbox() called with multiple threads in process gpu-process.
[0203/123949.335887:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig.
[0203/125448.964372:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
[0203/125448.964555:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
[0203/130949.064689:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
[0203/130949.064808:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
[0203/132449.082211:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
[0203/132449.082304:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
[0203/132749.032800:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/132903.239891:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/132919.254303:WARNING:audio_manager_linux.cc(53)] Falling back to ALSA for audio output. PulseAudio is not available or could not be initialized.
[0203/133004.985697:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/133105.016812:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/133205.066476:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/133305.072982:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
[0203/133949.182923:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
[0203/133949.183046:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended

meilisearch:

888b     d888          d8b 888 d8b                                            888
8888b   d8888          Y8P 888 Y8P                                            888
88888b.d88888              888                                                888
888Y88888P888  .d88b.  888 888 888 .d8888b   .d88b.   8888b.  888d888 .d8888b 88888b.
888 Y888P 888 d8P  Y8b 888 888 888 88K      d8P  Y8b     "88b 888P"  d88P"    888 "88b
888  Y8P  888 88888888 888 888 888 "Y8888b. 88888888 .d888888 888    888      888  888
888   "   888 Y8b.     888 888 888      X88 Y8b.     888  888 888    Y88b.    888  888
888       888  "Y8888  888 888 888  88888P'  "Y8888  "Y888888 888     "Y8888P 888  888
Config file path:  "none"
Database path:    "./data.ms"
Server listening on:  "http://0.0.0.0:7700"
Environment:    "development"
Commit SHA:    "13025594a8303b3445df89399a75db9d951031f3"
Commit date:    "2024-11-06T11:35:26Z"
Package version:  "1.11.1"
Anonymous telemetry:  "Disabled"
A master key has been set. Requests to Meilisearch won't be authorized unless you provide an authentication key.
Check out Meilisearch Cloud!  https://www.meilisearch.com/cloud?utm_campaign=oss&utm_source=engine&utm_medium=cli
Documentation:      https://www.meilisearch.com/docs
Source code:      https://github.com/meilisearch/meilisearch
Discord:      https://discord.meilisearch.com
2025-02-03T12:42:03.066843Z  INFO actix_server::builder: starting 2 workers
2025-02-03T12:42:03.067020Z  INFO actix_server::server: Actix runtime found; starting in Actix runtime
2025-02-03T13:27:34.841688Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=2.55ms time.idle=3.74ms
2025-02-03T13:27:34.864775Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/bookmarks/settings query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=790µs time.idle=715µs
2025-02-03T13:27:34.892698Z  INFO HTTP request{method=DELETE host="meilisearch:7700" route=/indexes/bookmarks/documents/l4m6nbquhcex7v3pg5ux878y query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=500µs time.idle=4.54ms
2025-02-03T13:27:34.920500Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/28 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=510µs time.idle=737µs
2025-02-03T13:27:34.995140Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/28 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=645µs time.idle=584µs
2025-02-03T13:27:35.068196Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/28 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=440µs time.idle=669µs
2025-02-03T13:27:35.136574Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/28 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=448µs time.idle=706µs
2025-02-03T13:27:35.199093Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
2025-02-03T13:27:35.215458Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/28 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=540µs time.idle=948µs
2025-02-03T13:27:48.438772Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.71ms time.idle=6.07ms
2025-02-03T13:27:48.465287Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/29 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=709µs time.idle=913µs
2025-02-03T13:27:48.530045Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/29 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=451µs time.idle=574µs
2025-02-03T13:27:48.584434Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=141.750581ms
2025-02-03T13:27:48.594951Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/29 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=398µs time.idle=460µs
2025-02-03T13:27:48.598723Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
2025-02-03T13:27:48.655471Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/29 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=367µs time.idle=448µs
2025-02-03T13:32:04.315237Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.28ms time.idle=5.27ms
2025-02-03T13:32:04.367459Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/30 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=468µs time.idle=629µs
2025-02-03T13:32:04.429874Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/30 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=335µs time.idle=340µs
2025-02-03T13:32:04.485226Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=169.229771ms
2025-02-03T13:32:04.490210Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/30 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=309µs time.idle=340µs
2025-02-03T13:32:04.495215Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
2025-02-03T13:32:04.550783Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/30 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=331µs time.idle=340µs
2025-02-03T13:32:06.639100Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.84ms time.idle=5.41ms
2025-02-03T13:32:06.654667Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/31 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=516µs time.idle=674µs
2025-02-03T13:32:06.721220Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/31 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=1.49ms time.idle=1.24ms
2025-02-03T13:32:06.817393Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/31 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=463µs time.idle=763µs
2025-02-03T13:32:06.913829Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/31 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=786µs time.idle=496µs
2025-02-03T13:32:06.934847Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=295.014825ms
2025-02-03T13:32:06.952301Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
2025-02-03T13:32:06.977492Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/31 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=410µs time.idle=525µs
2025-02-03T13:32:59.518897Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.37ms time.idle=13.4ms
2025-02-03T13:32:59.593036Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/32 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=890µs time.idle=909µs
2025-02-03T13:32:59.599946Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=80.405386ms
2025-02-03T13:32:59.607750Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
2025-02-03T13:32:59.655066Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/32 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=281µs time.idle=327µs
2025-02-03T13:33:01.731282Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=900µs time.idle=4.37ms
2025-02-03T13:33:01.742423Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/33 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=485µs time.idle=343µs
2025-02-03T13:33:01.803383Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/33 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=922µs time.idle=1.14ms
2025-02-03T13:33:01.865759Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/33 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=301µs time.idle=370µs
2025-02-03T13:33:01.887845Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=156.136523ms
2025-02-03T13:33:01.899339Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
2025-02-03T13:33:01.925042Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/33 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=279µs time.idle=380µs
2025-02-03T13:34:03.284712Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.23ms time.idle=5.06ms
2025-02-03T13:34:03.316049Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/34 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=461µs time.idle=701µs
2025-02-03T13:34:03.368621Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=82.981119ms
2025-02-03T13:34:03.373548Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
2025-02-03T13:34:03.379246Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/34 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=281µs time.idle=323µs
2025-02-03T13:34:04.447868Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.64ms time.idle=4.54ms
2025-02-03T13:34:04.466921Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/35 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=409µs time.idle=571µs
2025-02-03T13:34:04.532082Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/35 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=421µs time.idle=427µs
2025-02-03T13:34:04.593212Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/35 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=446µs time.idle=402µs
2025-02-03T13:34:04.629022Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=177.107994ms
2025-02-03T13:34:04.638147Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
2025-02-03T13:34:04.652510Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/35 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=285µs time.idle=421µs

hoarder:

2025-02-03T13:31:04.930Z error: [Crawler][68] Failed to download and store image: AbortError: The operation was aborted.
2025-02-03T13:31:04.966Z info: [Crawler][68] Content-type for the url https://www.uol.com.br is "text/html;charset=UTF-8"
2025-02-03T13:31:16.107Z info: [Crawler][68] Successfully navigated to "https://www.uol.com.br". Waiting for the page to load ...
2025-02-03T13:31:21.112Z info: [Crawler][68] Finished waiting for the page to load.
2025-02-03T13:31:21.255Z info: [Crawler][68] Successfully fetched the page content.
2025-02-03T13:31:22.666Z info: [Crawler][68] Finished capturing page content and a screenshot. FullPageScreenshot: false
2025-02-03T13:31:22.692Z info: [Crawler][68] Will attempt to extract metadata from page ...
2025-02-03T13:31:40.726Z info: [Crawler][68] Will attempt to extract readable content ...
2025-02-03T13:32:02.940Z info: [Crawler][68] Done extracting readable content.
2025-02-03T13:32:03.037Z info: [Crawler][68] Stored the screenshot as assetId: f5b62d13-69f1-499d-bf7c-822675cfb578
2025-02-03T13:32:03.394Z info: [Crawler][68] Done extracting metadata from the page.
2025-02-03T13:32:03.395Z info: [Crawler][68] Downloading image from "https://conteudo.imguol.com.br/c/home/interacao/facebook/compartilhe.png"
2025-02-03T13:32:03.531Z info: [Crawler][68] Downloaded image as assetId: 7a4974be-4aad-4873-bdf7-f545a66342ba
2025-02-03T13:32:03.639Z info: [Crawler][68] Will attempt to archive page ...
2025-02-03T13:32:04.057Z info: [webhook][74] Starting a webhook job for bookmark with id "tspj5ffls8k3y3e0nlo538ez"
2025-02-03T13:32:04.058Z info: [webhook][74] Completed successfully
2025-02-03T13:32:04.091Z info: [search][72] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ...
2025-02-03T13:32:04.171Z info: [inference][71] Starting an inference job for bookmark with id "tspj5ffls8k3y3e0nlo538ez"
2025-02-03T13:32:04.255Z info: [VideoCrawler][73] Skipping video download from "https://www.uol.com.br", because it is disabled in the config.
2025-02-03T13:32:04.256Z info: [VideoCrawler][73] Video Download Completed successfully
2025-02-03T13:32:04.557Z info: [search][72] Completed successfully
2025-02-03T13:32:04.903Z error: [Crawler][68] Crawling job failed: Error: Timed-out after 60 secs
Error: Timed-out after 60 secs
    at Timeout._onTimeout (/app/apps/workers/utils.ts:2:1025)
    at listOnTimeout (node:internal/timers:594:17)
    at process.processTimers (node:internal/timers:529:7)
2025-02-03T13:32:04.938Z info: [Crawler][68] Will crawl "https://www.uol.com.br" for link with id "tspj5ffls8k3y3e0nlo538ez"
2025-02-03T13:32:04.939Z info: [Crawler][68] Attempting to determine the content-type for the url https://www.uol.com.br
2025-02-03T13:32:05.010Z info: [Crawler][68] Content-type for the url https://www.uol.com.br is "text/html;charset=UTF-8"
2025-02-03T13:32:05.999Z info: [inference][71] Inferring tag for bookmark "tspj5ffls8k3y3e0nlo538ez" used 798 tokens and inferred: Brazilian News,Digital Services,Politics,Technology,Entertainment
2025-02-03T13:32:06.051Z info: [inference][71] Completed successfully
2025-02-03T13:32:06.600Z info: [search][75] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ...
2025-02-03T13:32:06.984Z info: [search][75] Completed successfully
2025-02-03T13:32:15.313Z info: [Crawler][68] Successfully navigated to "https://www.uol.com.br". Waiting for the page to load ...
2025-02-03T13:32:16.413Z info: [Crawler][68] Finished waiting for the page to load.
2025-02-03T13:32:16.978Z info: [Crawler][68] Successfully fetched the page content.
2025-02-03T13:32:18.297Z info: [Crawler][68] Finished capturing page content and a screenshot. FullPageScreenshot: false
2025-02-03T13:32:18.349Z info: [Crawler][68] Will attempt to extract metadata from page ...
2025-02-03T13:32:36.213Z info: [Crawler][68] Will attempt to extract readable content ...
2025-02-03T13:32:58.179Z info: [Crawler][68] Done extracting readable content.
2025-02-03T13:32:58.263Z info: [Crawler][68] Stored the screenshot as assetId: a862aa60-ae1d-4f9b-b2df-2cd63e463abd
2025-02-03T13:32:58.623Z info: [Crawler][68] Done extracting metadata from the page.
2025-02-03T13:32:58.624Z info: [Crawler][68] Downloading image from "https://conteudo.imguol.com.br/c/home/interacao/facebook/compartilhe.png"
2025-02-03T13:32:58.757Z info: [Crawler][68] Downloaded image as assetId: 80df0aa6-e320-4380-834f-6fd6bde42251
2025-02-03T13:32:58.908Z info: [Crawler][68] Will attempt to archive page ...
2025-02-03T13:32:59.267Z info: [search][77] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ...
2025-02-03T13:32:59.346Z info: [inference][76] Starting an inference job for bookmark with id "tspj5ffls8k3y3e0nlo538ez"
2025-02-03T13:32:59.411Z info: [webhook][79] Starting a webhook job for bookmark with id "tspj5ffls8k3y3e0nlo538ez"
2025-02-03T13:32:59.411Z info: [webhook][79] Completed successfully
2025-02-03T13:32:59.472Z info: [VideoCrawler][78] Skipping video download from "https://www.uol.com.br", because it is disabled in the config.
2025-02-03T13:32:59.473Z info: [VideoCrawler][78] Video Download Completed successfully
2025-02-03T13:32:59.662Z info: [search][77] Completed successfully
2025-02-03T13:33:00.714Z info: [inference][76] Inferring tag for bookmark "tspj5ffls8k3y3e0nlo538ez" used 798 tokens and inferred: Brazil,Digital Services,News Media,Technology,UOL
2025-02-03T13:33:00.755Z info: [inference][76] Completed successfully
2025-02-03T13:33:01.706Z info: [search][80] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ...
2025-02-03T13:33:01.930Z info: [search][80] Completed successfully
2025-02-03T13:33:04.928Z error: [Crawler][68] Crawling job failed: Error: Timed-out after 60 secs
Error: Timed-out after 60 secs
    at Timeout._onTimeout (/app/apps/workers/utils.ts:2:1025)
    at listOnTimeout (node:internal/timers:594:17)
    at process.processTimers (node:internal/timers:529:7)
2025-02-03T13:33:04.960Z info: [Crawler][68] Will crawl "https://www.uol.com.br" for link with id "tspj5ffls8k3y3e0nlo538ez"
2025-02-03T13:33:04.960Z info: [Crawler][68] Attempting to determine the content-type for the url https://www.uol.com.br
2025-02-03T13:33:05.016Z info: [Crawler][68] Content-type for the url https://www.uol.com.br is "text/html;charset=UTF-8"
2025-02-03T13:33:16.662Z info: [Crawler][68] Successfully navigated to "https://www.uol.com.br". Waiting for the page to load ...
2025-02-03T13:33:21.667Z info: [Crawler][68] Finished waiting for the page to load.
2025-02-03T13:33:21.829Z info: [Crawler][68] Successfully fetched the page content.
2025-02-03T13:33:23.267Z info: [Crawler][68] Finished capturing page content and a screenshot. FullPageScreenshot: false
2025-02-03T13:33:23.294Z info: [Crawler][68] Will attempt to extract metadata from page ...
2025-02-03T13:33:40.967Z info: [Crawler][68] Will attempt to extract readable content ...
2025-02-03T13:34:02.042Z info: [Crawler][68] Done extracting readable content.
2025-02-03T13:34:02.145Z info: [Crawler][68] Stored the screenshot as assetId: cf47a918-afcd-4186-9c3f-75680c142404
2025-02-03T13:34:02.492Z info: [Crawler][68] Done extracting metadata from the page.
2025-02-03T13:34:02.493Z info: [Crawler][68] Downloading image from "https://conteudo.imguol.com.br/c/home/interacao/facebook/compartilhe.png"
2025-02-03T13:34:02.632Z info: [Crawler][68] Downloaded image as assetId: e457cb9a-3f9d-4262-a6c3-649da98242da
2025-02-03T13:34:02.731Z info: [Crawler][68] Will attempt to archive page ...
2025-02-03T13:34:03.138Z info: [webhook][84] Starting a webhook job for bookmark with id "tspj5ffls8k3y3e0nlo538ez"
2025-02-03T13:34:03.139Z info: [webhook][84] Completed successfully
2025-02-03T13:34:03.181Z info: [VideoCrawler][83] Skipping video download from "https://www.uol.com.br", because it is disabled in the config.
2025-02-03T13:34:03.182Z info: [VideoCrawler][83] Video Download Completed successfully
2025-02-03T13:34:03.219Z info: [inference][81] Starting an inference job for bookmark with id "tspj5ffls8k3y3e0nlo538ez"
2025-02-03T13:34:03.251Z info: [search][82] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ...
2025-02-03T13:34:03.384Z info: [search][82] Completed successfully
2025-02-03T13:34:04.304Z info: [inference][81] Inferring tag for bookmark "tspj5ffls8k3y3e0nlo538ez" used 797 tokens and inferred: Brazilian News,Politics,Entertainment,Technology,Sports
2025-02-03T13:34:04.365Z info: [inference][81] Completed successfully
2025-02-03T13:34:04.420Z info: [search][85] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ...
2025-02-03T13:34:04.657Z info: [search][85] Completed successfully
2025-02-03T13:34:04.952Z error: [Crawler][68] Crawling job failed: Error: Timed-out after 60 secs
Error: Timed-out after 60 secs
    at Timeout._onTimeout (/app/apps/workers/utils.ts:2:1025)
    at listOnTimeout (node:internal/timers:594:17)
    at process.processTimers (node:internal/timers:529:7)
  ▲ Next.js 14.2.21
  - Local:        http://localhost:3000
  - Network:      http://0.0.0.0:3000
 ✓ Starting...
 ✓ Ready in 1890ms
(node:1126) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead.
(Use node --trace-deprecation ... to show where the warning was created)

Steps to Reproduce

I’m not exactly sure; I just built the Docker image and tried to use it,.
I even tried using the links from the demo, but they didn't work either.

Expected Behaviour

Ensure that the banner images load just like they do in the app demo.

Screenshots or Additional Context

Image
Image
Image

Device Details

DietPi PC / Windows 10 / Firefox / Edge

Exact Hoarder Version

0.22.0

Have you checked the troubleshooting guide?

  • I have checked the troubleshooting guide and I haven't found a solution to my problem
Originally created by @lmalmoreno on GitHub (Feb 4, 2025). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/978 ### Describe the Bug Hello, I am a self-hosting enthusiast and could hardly believe it when I found this project. ChatGPT recommended it to me, and it's exactly what I need. That said, I want to express my deep gratitude for the project and the time you've invested in it. This past weekend, I decided to get it running on my system. I followed all the instructions in the installation manual (docker-compose), and it promptly started running on my small Linux server (DietPi, PC x86). Although everything seemed fine, I'm encountering some issues and couldn't find any solutions in the manuals or on Discord. The problem is that the link banners don't load, displaying a "card banner" message instead (this only happens on the web page; in the iOS app, everything works fine). Another issue occurs in Edge/Chrome: after some time of use, a message appears saying, "Application error: a client-side exception has occurred (see the browser console for more information)," and it's no longer possible to access the interface. Several errors are also generated in the browser console. So far, I haven't had this problem in Firefox, but errors do appear in the browser console. I also tried running the system in Docker on Windows 10 but experienced the same behavior. I've attached screenshots, container logs, and my configuration files. I'm new to Docker, so if I'm making any mistakes, I apologize in advance. I'm currently running version 0.22.0, but this behavior also occurred in the previous version. Thank you very much for this incredible project! docker-compose: ```yaml services: web: image: ghcr.io/hoarder-app/hoarder:${HOARDER_VERSION:-release} restart: unless-stopped volumes: - data:/data ports: - 3000:3000 env_file: - .env environment: MEILI_ADDR: http://meilisearch:7700 BROWSER_WEB_URL: http://chrome:9222 # OPENAI_API_KEY: ... DATA_DIR: /data chrome: image: gcr.io/zenika-hub/alpine-chrome:123 restart: unless-stopped command: - --no-sandbox - --disable-gpu - --disable-dev-shm-usage - --remote-debugging-address=0.0.0.0 - --remote-debugging-port=9222 - --hide-scrollbars meilisearch: image: getmeili/meilisearch:v1.11.1 restart: unless-stopped env_file: - .env environment: MEILI_NO_ANALYTICS: "true" volumes: - meilisearch:/meili_data volumes: meilisearch: data: ``` .env: ```env HOARDER_VERSION=release NEXTAUTH_SECRET=************************* MEILI_MASTER_KEY=************************ NEXTAUTH_URL=http://192.168.0.2:3000/ OPENAI_API_KEY=************************** CRAWLER_FULL_PAGE_ARCHIVE=true OCR_LANGS=eng,por CRAWLER_DOWNLOAD_BANNER_IMAGE=true CRAWLER_STORE_SCREENSHOT=true ``` chrome: ```log [0203/031223.179528:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/031223.214397:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/031223.214841:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/031223.231575:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig. [0203/031223.418146:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed [0203/031223.418263:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended [0203/031223.556772:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable. DevTools listening on ws://0.0.0.0:9222/devtools/browser/96b49013-8b36-4107-9dc0-bdb2082f859f [0203/031223.714719:WARNING:sandbox_linux.cc(418)] InitializeSandbox() called with multiple threads in process gpu-process. [0203/031224.047573:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig. [0203/122601.492487:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/122602.718145:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/122604.207074:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/122604.405135:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/122611.999333:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/122635.562933:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/122640.439652:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/122640.439834:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/122640.440503:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/122640.440592:WARNING:property.cc(144)] DaemonVersion: GetAndBlock: failed. [0203/122640.440942:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/122640.441242:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/122642.742224:ERROR:web_contents_delegate.cc(260)] WebContentsDelegate::CheckMediaAccessPermission: Not supported. [0203/122642.742310:ERROR:web_contents_delegate.cc(260)] WebContentsDelegate::CheckMediaAccessPermission: Not supported. [0203/122643.612672:WARNING:audio_manager_linux.cc(53)] Falling back to ALSA for audio output. PulseAudio is not available or could not be initialized. [0203/122652.433558:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/122800.763669:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/122900.610360:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/123000.647172:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/123100.644500:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/123107.552855:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed [0203/123107.552978:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended [0203/123203.195990:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/123948.729055:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/123948.734221:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig. [0203/123948.743611:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/123948.750325:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [0203/123948.811181:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed [0203/123948.811379:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended DevTools listening on ws://0.0.0.0:9222/devtools/browser/78afe4f8-20eb-4917-9703-929dca21e8fe [0203/123948.863655:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable. [0203/123948.973638:WARNING:sandbox_linux.cc(418)] InitializeSandbox() called with multiple threads in process gpu-process. [0203/123949.335887:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig. [0203/125448.964372:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed [0203/125448.964555:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended [0203/130949.064689:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed [0203/130949.064808:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended [0203/132449.082211:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed [0203/132449.082304:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended [0203/132749.032800:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/132903.239891:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/132919.254303:WARNING:audio_manager_linux.cc(53)] Falling back to ALSA for audio output. PulseAudio is not available or could not be initialized. [0203/133004.985697:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/133105.016812:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/133205.066476:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/133305.072982:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. [0203/133949.182923:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed [0203/133949.183046:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended ``` meilisearch: ```log 888b d888 d8b 888 d8b 888 8888b d8888 Y8P 888 Y8P 888 88888b.d88888 888 888 888Y88888P888 .d88b. 888 888 888 .d8888b .d88b. 8888b. 888d888 .d8888b 88888b. 888 Y888P 888 d8P Y8b 888 888 888 88K d8P Y8b "88b 888P" d88P" 888 "88b 888 Y8P 888 88888888 888 888 888 "Y8888b. 88888888 .d888888 888 888 888 888 888 " 888 Y8b. 888 888 888 X88 Y8b. 888 888 888 Y88b. 888 888 888 888 "Y8888 888 888 888 88888P' "Y8888 "Y888888 888 "Y8888P 888 888 Config file path: "none" Database path: "./data.ms" Server listening on: "http://0.0.0.0:7700" Environment: "development" Commit SHA: "13025594a8303b3445df89399a75db9d951031f3" Commit date: "2024-11-06T11:35:26Z" Package version: "1.11.1" Anonymous telemetry: "Disabled" A master key has been set. Requests to Meilisearch won't be authorized unless you provide an authentication key. Check out Meilisearch Cloud! https://www.meilisearch.com/cloud?utm_campaign=oss&utm_source=engine&utm_medium=cli Documentation: https://www.meilisearch.com/docs Source code: https://github.com/meilisearch/meilisearch Discord: https://discord.meilisearch.com 2025-02-03T12:42:03.066843Z INFO actix_server::builder: starting 2 workers 2025-02-03T12:42:03.067020Z INFO actix_server::server: Actix runtime found; starting in Actix runtime 2025-02-03T13:27:34.841688Z INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=2.55ms time.idle=3.74ms 2025-02-03T13:27:34.864775Z INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/bookmarks/settings query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=790µs time.idle=715µs 2025-02-03T13:27:34.892698Z INFO HTTP request{method=DELETE host="meilisearch:7700" route=/indexes/bookmarks/documents/l4m6nbquhcex7v3pg5ux878y query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=500µs time.idle=4.54ms 2025-02-03T13:27:34.920500Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/28 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=510µs time.idle=737µs 2025-02-03T13:27:34.995140Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/28 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=645µs time.idle=584µs 2025-02-03T13:27:35.068196Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/28 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=440µs time.idle=669µs 2025-02-03T13:27:35.136574Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/28 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=448µs time.idle=706µs 2025-02-03T13:27:35.199093Z INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. 2025-02-03T13:27:35.215458Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/28 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=540µs time.idle=948µs 2025-02-03T13:27:48.438772Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.71ms time.idle=6.07ms 2025-02-03T13:27:48.465287Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/29 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=709µs time.idle=913µs 2025-02-03T13:27:48.530045Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/29 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=451µs time.idle=574µs 2025-02-03T13:27:48.584434Z INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=141.750581ms 2025-02-03T13:27:48.594951Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/29 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=398µs time.idle=460µs 2025-02-03T13:27:48.598723Z INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. 2025-02-03T13:27:48.655471Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/29 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=367µs time.idle=448µs 2025-02-03T13:32:04.315237Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.28ms time.idle=5.27ms 2025-02-03T13:32:04.367459Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/30 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=468µs time.idle=629µs 2025-02-03T13:32:04.429874Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/30 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=335µs time.idle=340µs 2025-02-03T13:32:04.485226Z INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=169.229771ms 2025-02-03T13:32:04.490210Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/30 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=309µs time.idle=340µs 2025-02-03T13:32:04.495215Z INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. 2025-02-03T13:32:04.550783Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/30 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=331µs time.idle=340µs 2025-02-03T13:32:06.639100Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.84ms time.idle=5.41ms 2025-02-03T13:32:06.654667Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/31 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=516µs time.idle=674µs 2025-02-03T13:32:06.721220Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/31 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=1.49ms time.idle=1.24ms 2025-02-03T13:32:06.817393Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/31 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=463µs time.idle=763µs 2025-02-03T13:32:06.913829Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/31 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=786µs time.idle=496µs 2025-02-03T13:32:06.934847Z INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=295.014825ms 2025-02-03T13:32:06.952301Z INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. 2025-02-03T13:32:06.977492Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/31 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=410µs time.idle=525µs 2025-02-03T13:32:59.518897Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.37ms time.idle=13.4ms 2025-02-03T13:32:59.593036Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/32 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=890µs time.idle=909µs 2025-02-03T13:32:59.599946Z INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=80.405386ms 2025-02-03T13:32:59.607750Z INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. 2025-02-03T13:32:59.655066Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/32 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=281µs time.idle=327µs 2025-02-03T13:33:01.731282Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=900µs time.idle=4.37ms 2025-02-03T13:33:01.742423Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/33 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=485µs time.idle=343µs 2025-02-03T13:33:01.803383Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/33 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=922µs time.idle=1.14ms 2025-02-03T13:33:01.865759Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/33 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=301µs time.idle=370µs 2025-02-03T13:33:01.887845Z INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=156.136523ms 2025-02-03T13:33:01.899339Z INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. 2025-02-03T13:33:01.925042Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/33 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=279µs time.idle=380µs 2025-02-03T13:34:03.284712Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.23ms time.idle=5.06ms 2025-02-03T13:34:03.316049Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/34 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=461µs time.idle=701µs 2025-02-03T13:34:03.368621Z INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=82.981119ms 2025-02-03T13:34:03.373548Z INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. 2025-02-03T13:34:03.379246Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/34 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=281µs time.idle=323µs 2025-02-03T13:34:04.447868Z INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/bookmarks/documents query_parameters=primaryKey=id user_agent=node status_code=202}: meilisearch: close time.busy=1.64ms time.idle=4.54ms 2025-02-03T13:34:04.466921Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/35 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=409µs time.idle=571µs 2025-02-03T13:34:04.532082Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/35 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=421µs time.idle=427µs 2025-02-03T13:34:04.593212Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/35 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=446µs time.idle=402µs 2025-02-03T13:34:04.629022Z INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 } processed_in=177.107994ms 2025-02-03T13:34:04.638147Z INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks. 2025-02-03T13:34:04.652510Z INFO HTTP request{method=GET host="meilisearch:7700" route=/tasks/35 query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=285µs time.idle=421µs ``` hoarder: ```log 2025-02-03T13:31:04.930Z error: [Crawler][68] Failed to download and store image: AbortError: The operation was aborted. 2025-02-03T13:31:04.966Z info: [Crawler][68] Content-type for the url https://www.uol.com.br is "text/html;charset=UTF-8" 2025-02-03T13:31:16.107Z info: [Crawler][68] Successfully navigated to "https://www.uol.com.br". Waiting for the page to load ... 2025-02-03T13:31:21.112Z info: [Crawler][68] Finished waiting for the page to load. 2025-02-03T13:31:21.255Z info: [Crawler][68] Successfully fetched the page content. 2025-02-03T13:31:22.666Z info: [Crawler][68] Finished capturing page content and a screenshot. FullPageScreenshot: false 2025-02-03T13:31:22.692Z info: [Crawler][68] Will attempt to extract metadata from page ... 2025-02-03T13:31:40.726Z info: [Crawler][68] Will attempt to extract readable content ... 2025-02-03T13:32:02.940Z info: [Crawler][68] Done extracting readable content. 2025-02-03T13:32:03.037Z info: [Crawler][68] Stored the screenshot as assetId: f5b62d13-69f1-499d-bf7c-822675cfb578 2025-02-03T13:32:03.394Z info: [Crawler][68] Done extracting metadata from the page. 2025-02-03T13:32:03.395Z info: [Crawler][68] Downloading image from "https://conteudo.imguol.com.br/c/home/interacao/facebook/compartilhe.png" 2025-02-03T13:32:03.531Z info: [Crawler][68] Downloaded image as assetId: 7a4974be-4aad-4873-bdf7-f545a66342ba 2025-02-03T13:32:03.639Z info: [Crawler][68] Will attempt to archive page ... 2025-02-03T13:32:04.057Z info: [webhook][74] Starting a webhook job for bookmark with id "tspj5ffls8k3y3e0nlo538ez" 2025-02-03T13:32:04.058Z info: [webhook][74] Completed successfully 2025-02-03T13:32:04.091Z info: [search][72] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ... 2025-02-03T13:32:04.171Z info: [inference][71] Starting an inference job for bookmark with id "tspj5ffls8k3y3e0nlo538ez" 2025-02-03T13:32:04.255Z info: [VideoCrawler][73] Skipping video download from "https://www.uol.com.br", because it is disabled in the config. 2025-02-03T13:32:04.256Z info: [VideoCrawler][73] Video Download Completed successfully 2025-02-03T13:32:04.557Z info: [search][72] Completed successfully 2025-02-03T13:32:04.903Z error: [Crawler][68] Crawling job failed: Error: Timed-out after 60 secs Error: Timed-out after 60 secs at Timeout._onTimeout (/app/apps/workers/utils.ts:2:1025) at listOnTimeout (node:internal/timers:594:17) at process.processTimers (node:internal/timers:529:7) 2025-02-03T13:32:04.938Z info: [Crawler][68] Will crawl "https://www.uol.com.br" for link with id "tspj5ffls8k3y3e0nlo538ez" 2025-02-03T13:32:04.939Z info: [Crawler][68] Attempting to determine the content-type for the url https://www.uol.com.br 2025-02-03T13:32:05.010Z info: [Crawler][68] Content-type for the url https://www.uol.com.br is "text/html;charset=UTF-8" 2025-02-03T13:32:05.999Z info: [inference][71] Inferring tag for bookmark "tspj5ffls8k3y3e0nlo538ez" used 798 tokens and inferred: Brazilian News,Digital Services,Politics,Technology,Entertainment 2025-02-03T13:32:06.051Z info: [inference][71] Completed successfully 2025-02-03T13:32:06.600Z info: [search][75] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ... 2025-02-03T13:32:06.984Z info: [search][75] Completed successfully 2025-02-03T13:32:15.313Z info: [Crawler][68] Successfully navigated to "https://www.uol.com.br". Waiting for the page to load ... 2025-02-03T13:32:16.413Z info: [Crawler][68] Finished waiting for the page to load. 2025-02-03T13:32:16.978Z info: [Crawler][68] Successfully fetched the page content. 2025-02-03T13:32:18.297Z info: [Crawler][68] Finished capturing page content and a screenshot. FullPageScreenshot: false 2025-02-03T13:32:18.349Z info: [Crawler][68] Will attempt to extract metadata from page ... 2025-02-03T13:32:36.213Z info: [Crawler][68] Will attempt to extract readable content ... 2025-02-03T13:32:58.179Z info: [Crawler][68] Done extracting readable content. 2025-02-03T13:32:58.263Z info: [Crawler][68] Stored the screenshot as assetId: a862aa60-ae1d-4f9b-b2df-2cd63e463abd 2025-02-03T13:32:58.623Z info: [Crawler][68] Done extracting metadata from the page. 2025-02-03T13:32:58.624Z info: [Crawler][68] Downloading image from "https://conteudo.imguol.com.br/c/home/interacao/facebook/compartilhe.png" 2025-02-03T13:32:58.757Z info: [Crawler][68] Downloaded image as assetId: 80df0aa6-e320-4380-834f-6fd6bde42251 2025-02-03T13:32:58.908Z info: [Crawler][68] Will attempt to archive page ... 2025-02-03T13:32:59.267Z info: [search][77] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ... 2025-02-03T13:32:59.346Z info: [inference][76] Starting an inference job for bookmark with id "tspj5ffls8k3y3e0nlo538ez" 2025-02-03T13:32:59.411Z info: [webhook][79] Starting a webhook job for bookmark with id "tspj5ffls8k3y3e0nlo538ez" 2025-02-03T13:32:59.411Z info: [webhook][79] Completed successfully 2025-02-03T13:32:59.472Z info: [VideoCrawler][78] Skipping video download from "https://www.uol.com.br", because it is disabled in the config. 2025-02-03T13:32:59.473Z info: [VideoCrawler][78] Video Download Completed successfully 2025-02-03T13:32:59.662Z info: [search][77] Completed successfully 2025-02-03T13:33:00.714Z info: [inference][76] Inferring tag for bookmark "tspj5ffls8k3y3e0nlo538ez" used 798 tokens and inferred: Brazil,Digital Services,News Media,Technology,UOL 2025-02-03T13:33:00.755Z info: [inference][76] Completed successfully 2025-02-03T13:33:01.706Z info: [search][80] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ... 2025-02-03T13:33:01.930Z info: [search][80] Completed successfully 2025-02-03T13:33:04.928Z error: [Crawler][68] Crawling job failed: Error: Timed-out after 60 secs Error: Timed-out after 60 secs at Timeout._onTimeout (/app/apps/workers/utils.ts:2:1025) at listOnTimeout (node:internal/timers:594:17) at process.processTimers (node:internal/timers:529:7) 2025-02-03T13:33:04.960Z info: [Crawler][68] Will crawl "https://www.uol.com.br" for link with id "tspj5ffls8k3y3e0nlo538ez" 2025-02-03T13:33:04.960Z info: [Crawler][68] Attempting to determine the content-type for the url https://www.uol.com.br 2025-02-03T13:33:05.016Z info: [Crawler][68] Content-type for the url https://www.uol.com.br is "text/html;charset=UTF-8" 2025-02-03T13:33:16.662Z info: [Crawler][68] Successfully navigated to "https://www.uol.com.br". Waiting for the page to load ... 2025-02-03T13:33:21.667Z info: [Crawler][68] Finished waiting for the page to load. 2025-02-03T13:33:21.829Z info: [Crawler][68] Successfully fetched the page content. 2025-02-03T13:33:23.267Z info: [Crawler][68] Finished capturing page content and a screenshot. FullPageScreenshot: false 2025-02-03T13:33:23.294Z info: [Crawler][68] Will attempt to extract metadata from page ... 2025-02-03T13:33:40.967Z info: [Crawler][68] Will attempt to extract readable content ... 2025-02-03T13:34:02.042Z info: [Crawler][68] Done extracting readable content. 2025-02-03T13:34:02.145Z info: [Crawler][68] Stored the screenshot as assetId: cf47a918-afcd-4186-9c3f-75680c142404 2025-02-03T13:34:02.492Z info: [Crawler][68] Done extracting metadata from the page. 2025-02-03T13:34:02.493Z info: [Crawler][68] Downloading image from "https://conteudo.imguol.com.br/c/home/interacao/facebook/compartilhe.png" 2025-02-03T13:34:02.632Z info: [Crawler][68] Downloaded image as assetId: e457cb9a-3f9d-4262-a6c3-649da98242da 2025-02-03T13:34:02.731Z info: [Crawler][68] Will attempt to archive page ... 2025-02-03T13:34:03.138Z info: [webhook][84] Starting a webhook job for bookmark with id "tspj5ffls8k3y3e0nlo538ez" 2025-02-03T13:34:03.139Z info: [webhook][84] Completed successfully 2025-02-03T13:34:03.181Z info: [VideoCrawler][83] Skipping video download from "https://www.uol.com.br", because it is disabled in the config. 2025-02-03T13:34:03.182Z info: [VideoCrawler][83] Video Download Completed successfully 2025-02-03T13:34:03.219Z info: [inference][81] Starting an inference job for bookmark with id "tspj5ffls8k3y3e0nlo538ez" 2025-02-03T13:34:03.251Z info: [search][82] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ... 2025-02-03T13:34:03.384Z info: [search][82] Completed successfully 2025-02-03T13:34:04.304Z info: [inference][81] Inferring tag for bookmark "tspj5ffls8k3y3e0nlo538ez" used 797 tokens and inferred: Brazilian News,Politics,Entertainment,Technology,Sports 2025-02-03T13:34:04.365Z info: [inference][81] Completed successfully 2025-02-03T13:34:04.420Z info: [search][85] Attempting to index bookmark with id tspj5ffls8k3y3e0nlo538ez ... 2025-02-03T13:34:04.657Z info: [search][85] Completed successfully 2025-02-03T13:34:04.952Z error: [Crawler][68] Crawling job failed: Error: Timed-out after 60 secs Error: Timed-out after 60 secs at Timeout._onTimeout (/app/apps/workers/utils.ts:2:1025) at listOnTimeout (node:internal/timers:594:17) at process.processTimers (node:internal/timers:529:7) ▲ Next.js 14.2.21 - Local: http://localhost:3000 - Network: http://0.0.0.0:3000 ✓ Starting... ✓ Ready in 1890ms (node:1126) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead. (Use node --trace-deprecation ... to show where the warning was created) ``` ### Steps to Reproduce I’m not exactly sure; I just built the Docker image and tried to use it,. I even tried using the links from the demo, but they didn't work either. ### Expected Behaviour Ensure that the banner images load just like they do in the app demo. ### Screenshots or Additional Context ![Image](https://github.com/user-attachments/assets/4452721f-66b5-49ad-82d6-53b864f29007) ![Image](https://github.com/user-attachments/assets/eb04075a-e2c8-4bfa-81b4-7dd2ffeb6039) ![Image](https://github.com/user-attachments/assets/fdeb7fea-fed6-43fc-8e5b-c3bf689cd4ef) ### Device Details DietPi PC / Windows 10 / Firefox / Edge ### Exact Hoarder Version 0.22.0 ### Have you checked the troubleshooting guide? - [x] I have checked the troubleshooting guide and I haven't found a solution to my problem
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@ocean-haiyang commented on GitHub (Feb 15, 2025):

I would also like to know.

<!-- gh-comment-id:2661110707 --> @ocean-haiyang commented on GitHub (Feb 15, 2025): I would also like to know.
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@MohamedBassem commented on GitHub (Feb 17, 2025):

From the console logs, it seems that you're facing a lot of issues connecting to your server from the browser. You said that it works fine on the iOS app which is interesting. What browser are you using? Are you hitting the same issues on other browsers?

<!-- gh-comment-id:2662577191 --> @MohamedBassem commented on GitHub (Feb 17, 2025): From the console logs, it seems that you're facing a lot of issues connecting to your server from the browser. You said that it works fine on the iOS app which is interesting. What browser are you using? Are you hitting the same issues on other browsers?
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@lmalmoreno commented on GitHub (Feb 17, 2025):

From the console logs, it seems that you're facing a lot of issues connecting to your server from the browser. You said that it works fine on the iOS app which is interesting. What browser are you using? Are you hitting the same issues on other browsers?

After some more testing, I noticed a pattern causing these errors. First, I performed a clean installation of Hoarder, setting up a new docker-compose installation. Right after creating the user, I had the Hoarder interface working perfectly, but obviously without any saved links. At this point, if I saved just one or a few links, everything worked fine. However, what "breaks" my installation is importing around 27,000 saved links I have in my Chrome browser.

To perform this import, I followed the procedure of exporting the links from the browser in .html format and then importing this .html into Hoarder. Up to a certain point, everything worked perfectly, but at some moment, the entire system broke. Once broken, restarting Docker doesn’t help—it comes back with the same problems shown in the screenshots. The only solution is to delete the container and set up a new one from scratch.

Right now, I’ve set up a new installation from scratch, and I’m gradually importing the links I have saved in Chrome. This has actually been helpful since a lot of the things I had saved no longer make sense or are even offline now, so I’m doing some cleaning. 😃

Regarding the browser that shows the issue, it’s mainly Chrome. It displays this "client-side" error, and from that moment, it’s no longer possible to access the web interface. I also tested it in Firefox, and while it does show errors in the logs, the web interface remains usable—the only issue is that the banners are broken.

The error occurred both when running Docker on a mini PC with DietPi and on my desktop with Windows 10. I tested this to ensure that the problem wasn’t caused by my mini PC lacking processing power, but that doesn’t seem to be the case.

The only issue I noticed breaking Hoarder is the import of my bookmarks from Chrome. I just don’t understand why, even with the web interface broken, everything works perfectly on iOS. 😂

In any case, Hoarder is currently working, and I’m gradually organizing my link collection. Thank you so much for this incredible project!

<!-- gh-comment-id:2663305618 --> @lmalmoreno commented on GitHub (Feb 17, 2025): > From the console logs, it seems that you're facing a lot of issues connecting to your server from the browser. You said that it works fine on the iOS app which is interesting. What browser are you using? Are you hitting the same issues on other browsers? After some more testing, I noticed a pattern causing these errors. First, I performed a clean installation of Hoarder, setting up a new docker-compose installation. Right after creating the user, I had the Hoarder interface working perfectly, but obviously without any saved links. At this point, if I saved just one or a few links, everything worked fine. However, what "breaks" my installation is importing around 27,000 saved links I have in my Chrome browser. To perform this import, I followed the procedure of exporting the links from the browser in .html format and then importing this .html into Hoarder. Up to a certain point, everything worked perfectly, but at some moment, the entire system broke. Once broken, restarting Docker doesn’t help—it comes back with the same problems shown in the screenshots. The only solution is to delete the container and set up a new one from scratch. Right now, I’ve set up a new installation from scratch, and I’m gradually importing the links I have saved in Chrome. This has actually been helpful since a lot of the things I had saved no longer make sense or are even offline now, so I’m doing some cleaning. 😃 Regarding the browser that shows the issue, it’s mainly Chrome. It displays this "client-side" error, and from that moment, it’s no longer possible to access the web interface. I also tested it in Firefox, and while it does show errors in the logs, the web interface remains usable—the only issue is that the banners are broken. The error occurred both when running Docker on a mini PC with DietPi and on my desktop with Windows 10. I tested this to ensure that the problem wasn’t caused by my mini PC lacking processing power, but that doesn’t seem to be the case. The only issue I noticed breaking Hoarder is the import of my bookmarks from Chrome. I just don’t understand why, even with the web interface broken, everything works perfectly on iOS. 😂 In any case, Hoarder is currently working, and I’m gradually organizing my link collection. Thank you so much for this incredible project!
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@NoahShomette commented on GitHub (Feb 25, 2025):

I too am running into the same issue.

I am moving my hoarder install to a different computer so I've done the below steps:

  1. Saved the old data folder
  2. Moved it to my new pc
  3. Started a new docker container with the above data folder

The android app works fine as far as I can tell but firefox/chrome doesnt have working banners and is unstable (every once in a while crashes with the same crash the OP got + frequently reloads and stuff).

HOARDER_VERSION=0.20.0
DATA_DIR=/data
NEXTAUTH_URL=***
NEXTAUTH_SECRET=**


// Docker file
services:
  web:
    image: ghcr.io/hoarder-app/hoarder:${HOARDER_VERSION:-release}
    restart: unless-stopped
    volumes:
      - /srv/hoarder/data:/data
    ports:
      - 3002:3000
    env_file:
      - .env
    environment:
      MEILI_ADDR: http://meilisearch:7700
      BROWSER_WEB_URL: http://chrome:9222
      # OPENAI_API_KEY: ...
      DATA_DIR: /data
  chrome:
    image: gcr.io/zenika-hub/alpine-chrome:123
    restart: unless-stopped
    command:
      - --no-sandbox
      - --disable-gpu
      - --disable-dev-shm-usage
      - --remote-debugging-address=0.0.0.0
      - --remote-debugging-port=9222
      - --hide-scrollbars
  meilisearch:
    image: getmeili/meilisearch:v1.11.1
    restart: unless-stopped
    env_file:
      - .env
    environment:
      MEILI_NO_ANALYTICS: "true"
    volumes:
      - /srv/hoarder/meilisearch:/meili_data

// Chrome errors

chrome-1  | [0225/022302.423305:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
chrome-1  | [0225/022302.424491:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
chrome-1  | [0225/022302.424541:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
chrome-1  | [0225/022302.425210:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig.
chrome-1  | [0225/022302.495893:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
chrome-1  | [0225/022302.495918:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
chrome-1  | 
chrome-1  | DevTools listening on ws://0.0.0.0:9222/devtools/browser/1b7d8d01-a85f-4f31-905a-2eba6f75436e
chrome-1  | [0225/022302.501436:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable.
chrome-1  | [0225/022302.523846:WARNING:sandbox_linux.cc(418)] InitializeSandbox() called with multiple threads in process gpu-process.
chrome-1  | [0225/022302.589560:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig.
chrome-1  | [0225/023802.563757:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
chrome-1  | [0225/023802.563836:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended

// hoarder errors
eb-1  | (node:458) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
web-1  | (Use `node --trace-deprecation ...` to show where the warning was created)
web-1  |   ▲ Next.js 14.2.15
web-1  |   - Local:        http://localhost:3000
web-1  |   - Network:      http://0.0.0.0:3000
web-1  | 
web-1  |  ✓ Starting...
web-1  |  ✓ Ready in 390ms
<!-- gh-comment-id:2680256755 --> @NoahShomette commented on GitHub (Feb 25, 2025): I too am running into the same issue. I am moving my hoarder install to a different computer so I've done the below steps: 1. Saved the old data folder 2. Moved it to my new pc 3. Started a new docker container with the above data folder The android app works fine as far as I can tell but firefox/chrome doesnt have working banners and is unstable (every once in a while crashes with the same crash the OP got + frequently reloads and stuff). ``` HOARDER_VERSION=0.20.0 DATA_DIR=/data NEXTAUTH_URL=*** NEXTAUTH_SECRET=** ``` ``` // Docker file services: web: image: ghcr.io/hoarder-app/hoarder:${HOARDER_VERSION:-release} restart: unless-stopped volumes: - /srv/hoarder/data:/data ports: - 3002:3000 env_file: - .env environment: MEILI_ADDR: http://meilisearch:7700 BROWSER_WEB_URL: http://chrome:9222 # OPENAI_API_KEY: ... DATA_DIR: /data chrome: image: gcr.io/zenika-hub/alpine-chrome:123 restart: unless-stopped command: - --no-sandbox - --disable-gpu - --disable-dev-shm-usage - --remote-debugging-address=0.0.0.0 - --remote-debugging-port=9222 - --hide-scrollbars meilisearch: image: getmeili/meilisearch:v1.11.1 restart: unless-stopped env_file: - .env environment: MEILI_NO_ANALYTICS: "true" volumes: - /srv/hoarder/meilisearch:/meili_data ``` ``` // Chrome errors chrome-1 | [0225/022302.423305:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory chrome-1 | [0225/022302.424491:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory chrome-1 | [0225/022302.424541:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory chrome-1 | [0225/022302.425210:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig. chrome-1 | [0225/022302.495893:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed chrome-1 | [0225/022302.495918:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended chrome-1 | chrome-1 | DevTools listening on ws://0.0.0.0:9222/devtools/browser/1b7d8d01-a85f-4f31-905a-2eba6f75436e chrome-1 | [0225/022302.501436:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable. chrome-1 | [0225/022302.523846:WARNING:sandbox_linux.cc(418)] InitializeSandbox() called with multiple threads in process gpu-process. chrome-1 | [0225/022302.589560:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig. chrome-1 | [0225/023802.563757:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed chrome-1 | [0225/023802.563836:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended ``` ``` // hoarder errors eb-1 | (node:458) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. web-1 | (Use `node --trace-deprecation ...` to show where the warning was created) web-1 | ▲ Next.js 14.2.15 web-1 | - Local: http://localhost:3000 web-1 | - Network: http://0.0.0.0:3000 web-1 | web-1 | ✓ Starting... web-1 | ✓ Ready in 390ms ```
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@imamouse111 commented on GitHub (Mar 26, 2025):

I'm also having this issue. I never tried importing any bookmarks as OP describes though, just manually added stuff (both via the Android app and browser). My instance is running on OMV (Debian). Images load in the Android app just fine. Images that have been uploaded directly within the Android app and via the browser come up only when clicking on the item. And images within a bookmarked article are displayed fine when viewing in Hoarder. Tested in Chrome, Firefox, and Brave; same result, same console errors. I even set up a Cloudflare tunnel and images still did not display. I also confirmed it wasn't something on my network blocking images from loading by disabling WiFi on my phone as well as disabling all custom DNS on my network and browser. Really hoping a fix comes eventually but even without images this app is fantastic. Exactly what I've been looking for.

<!-- gh-comment-id:2753031244 --> @imamouse111 commented on GitHub (Mar 26, 2025): I'm also having this issue. I never tried importing any bookmarks as OP describes though, just manually added stuff (both via the Android app and browser). My instance is running on OMV (Debian). Images load in the Android app just fine. Images that have been uploaded directly within the Android app and via the browser come up only when clicking on the item. And images within a bookmarked article are displayed fine when viewing in Hoarder. Tested in Chrome, Firefox, and Brave; same result, same console errors. I even set up a Cloudflare tunnel and images still did not display. I also confirmed it wasn't something on my network blocking images from loading by disabling WiFi on my phone as well as disabling all custom DNS on my network and browser. Really hoping a fix comes eventually but even without images this app is fantastic. Exactly what I've been looking for.
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@imamouse111 commented on GitHub (Mar 26, 2025):

Had the thought last night that maybe it's a certificate issue. I'm also working on setting up an instance of AFFiNE and in their FAQ page they mention an issue with copy/paste not working due to browser restrictions, and their fix is to either proxy traffic or to set up your own CA. Has anybody tried either of these fixes?

Maybe this is the completely wrong rabbit hole, but I'm going to try attacking this by creating a certificate in Cloudflare. If that doesn't work maybe I'll try setting up NPM again (tried this a year ago but never got it working properly). I'll report back on the results.

<!-- gh-comment-id:2754399568 --> @imamouse111 commented on GitHub (Mar 26, 2025): Had the thought last night that maybe it's a certificate issue. I'm also working on setting up an instance of AFFiNE and in their FAQ page they mention an issue with copy/paste not working due to browser restrictions, and their fix is to either proxy traffic or to set up your own CA. Has anybody tried either of these fixes? Maybe this is the completely wrong rabbit hole, but I'm going to try attacking this by creating a certificate in Cloudflare. If that doesn't work maybe I'll try setting up NPM again (tried this a year ago but never got it working properly). I'll report back on the results.
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@jeffgoh commented on GitHub (Jun 7, 2025):

mine looked to be working fine at first, and then I imported 3000+ links from pocket

The images that don't work look like this when inspected:

<img alt="card banner" decoding="async" data-nimg="fill" 
class="h-56 min-h-56 w-full object-cover rounded-t-lg" 
style="position:absolute;height:100%;width:100%;left:0;top:0;right:0;bottom:0;color:transparent" 
src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAA1JREFUGFdj+P///38ACfsD/QVDRcoAAAAASUVORK5CYII=">

whereas the ones that work fine look like this

<img alt="card banner" loading="lazy" decoding="async" data-nimg="fill" 
class="h-56 min-h-56 w-full object-cover rounded-t-lg"
style="position:absolute;height:100%;width:100%;left:0;top:0;right:0;bottom:0;color:transparent" sizes="100vw"
srcset="/_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=640&amp;q=75 640w, 
/_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=750&amp;q=75 750w,
/_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=828&amp;q=75 828w,
/_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=1080&amp;q=75 1080w,
/_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=1200&amp;q=75 1200w,
/_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=1920&amp;q=75 1920w, /_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=2048&amp;q=75 2048w,
/_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=3840&amp;q=75 3840w"
src="/_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=3840&amp;q=75">
<!-- gh-comment-id:2952192295 --> @jeffgoh commented on GitHub (Jun 7, 2025): mine looked to be working fine at first, and then I imported 3000+ links from pocket The images that don't work look like this when inspected: ``` <img alt="card banner" decoding="async" data-nimg="fill" class="h-56 min-h-56 w-full object-cover rounded-t-lg" style="position:absolute;height:100%;width:100%;left:0;top:0;right:0;bottom:0;color:transparent" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAA1JREFUGFdj+P///38ACfsD/QVDRcoAAAAASUVORK5CYII="> ``` whereas the ones that work fine look like this ``` <img alt="card banner" loading="lazy" decoding="async" data-nimg="fill" class="h-56 min-h-56 w-full object-cover rounded-t-lg" style="position:absolute;height:100%;width:100%;left:0;top:0;right:0;bottom:0;color:transparent" sizes="100vw" srcset="/_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=640&amp;q=75 640w, /_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=750&amp;q=75 750w, /_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=828&amp;q=75 828w, /_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=1080&amp;q=75 1080w, /_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=1200&amp;q=75 1200w, /_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=1920&amp;q=75 1920w, /_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=2048&amp;q=75 2048w, /_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=3840&amp;q=75 3840w" src="/_next/image?url=%2Fapi%2Fassets%2F7c60f860-b0f6-4a5f-a410-d99f8108a7e3&amp;w=3840&amp;q=75"> ```
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@jeffgoh commented on GitHub (Jun 7, 2025):

@MohamedBassem

As near as I can tell, fetching the banner image takes about a minute per link. So, for my 3000+ link import from Pocket, 3000 minutes = 50 hours? I appreciate the low resource usage, but it would be great to have a way to speed this up.

There should probably be a way to make "refresh" of a link fix the banner image by jumping to the front of the queue. Also, prioritising links that are being displayed or closer to being displayed (in wordpress optimization terms, lazy loading images) would help fix the perception that things are slow/broken.

I'm also thinking that Facebook and Telegram use some kind of graph API to pick up the intended "cover" image, which would be much faster than directly scraping (but maybe at the expense of some privacy). Scraping and/or user uploaded can be a fallback when the provided image is somehow broken, but I think graph API will significantly reduce the work needed to complete an import (lots of people exiting Pocket = lots of people exploring alternatives).

Right now, I'm doing this to keep track of how close it is to completing banner image generation for all my links:

ls -l /data/asset/<profilename>/ | wc -l

but that is clumsy at best. Sorry if this isn't the right place to put my thoughts on making link banners work better/faster, but it seemed relevant to the issue topic.

<!-- gh-comment-id:2952285861 --> @jeffgoh commented on GitHub (Jun 7, 2025): @MohamedBassem As near as I can tell, fetching the banner image takes about a minute per link. So, for my 3000+ link import from Pocket, 3000 minutes = 50 hours? I appreciate the low resource usage, but it would be great to have a way to speed this up. There should probably be a way to make "refresh" of a link fix the banner image by jumping to the front of the queue. Also, prioritising links that are being displayed or closer to being displayed (in wordpress optimization terms, lazy loading images) would help fix the perception that things are slow/broken. I'm also thinking that Facebook and Telegram use some kind of graph API to pick up the intended "cover" image, which would be much faster than directly scraping (but maybe at the expense of some privacy). Scraping and/or user uploaded can be a fallback when the provided image is somehow broken, but I think graph API will significantly reduce the work needed to complete an import (lots of people exiting Pocket = lots of people exploring alternatives). Right now, I'm doing this to keep track of how close it is to completing banner image generation for all my links: `ls -l /data/asset/<profilename>/ | wc -l` but that is clumsy at best. Sorry if this isn't the right place to put my thoughts on making link banners work better/faster, but it seemed relevant to the issue topic.
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@MohamedBassem commented on GitHub (Jun 7, 2025):

@jeffgoh

  • if you have the resources for it, you can increase the parallelism for the crawler worker with CRAWLER_NUM_WORKERS=<num_workers>.
  • If you don't care about screenshots or full fledged crawling, you can unset BROWSER_WEB_URL and it'll be fetching only the OG banner images (same as facebook and telegram).
  • If you want to track how close you are in, there's an admin page for background jobs which tells you how many pending jobs there are.
<!-- gh-comment-id:2952327279 --> @MohamedBassem commented on GitHub (Jun 7, 2025): @jeffgoh * if you have the resources for it, you can increase the parallelism for the crawler worker with `CRAWLER_NUM_WORKERS=<num_workers>`. * If you don't care about screenshots or full fledged crawling, you can unset `BROWSER_WEB_URL` and it'll be fetching only the OG banner images (same as facebook and telegram). * If you want to track how close you are in, there's an admin page for background jobs which tells you how many pending jobs there are.
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@jeffgoh commented on GitHub (Jun 11, 2025):

@jeffgoh

  • if you have the resources for it, you can increase the parallelism for the crawler worker with CRAWLER_NUM_WORKERS=<num_workers>.
  • If you don't care about screenshots or full fledged crawling, you can unset BROWSER_WEB_URL and it'll be fetching only the OG banner images (same as facebook and telegram).
  • If you want to track how close you are in, there's an admin page for background jobs which tells you how many pending jobs there are.

Thanks for quick reply. Great work, btw. Loving the android app. It's obvious a lot of love went into this project.

Crawling is great re: having a personal archive of the data (in case website goes dark), so I don't know that I want to disable that. Can I have it crawl the page, but use OG banner by default? Some kind of drop down to select OG banner vs crawled vs custom. I read somewhere that you already put in a way to override the fetched image so that my karakeep contents are aesthetic, so it would just be one more option.

I did also find that importing from Pocket was less useful than you would think since many links are dead. I'm now writing a link verifier to clean up pocket export before importing the next batch (I used split to do 100 link chunks)

<!-- gh-comment-id:2961096120 --> @jeffgoh commented on GitHub (Jun 11, 2025): > [@jeffgoh](https://github.com/jeffgoh) > > * if you have the resources for it, you can increase the parallelism for the crawler worker with `CRAWLER_NUM_WORKERS=<num_workers>`. > * If you don't care about screenshots or full fledged crawling, you can unset `BROWSER_WEB_URL` and it'll be fetching only the OG banner images (same as facebook and telegram). > * If you want to track how close you are in, there's an admin page for background jobs which tells you how many pending jobs there are. Thanks for quick reply. Great work, btw. Loving the android app. It's obvious a lot of love went into this project. Crawling is great re: having a personal archive of the data (in case website goes dark), so I don't know that I want to disable that. Can I have it crawl the page, but use OG banner by default? Some kind of drop down to select OG banner vs crawled vs custom. I read somewhere that you already put in a way to override the fetched image so that my karakeep contents are aesthetic, so it would just be one more option. I did also find that importing from Pocket was less useful than you would think since many links are dead. I'm now writing a link verifier to clean up pocket export before importing the next batch (I used split to do 100 link chunks)
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@imamouse111 commented on GitHub (Jun 21, 2025):

Has anybody been able to solve this? I'm on v0.25.0 and still unable to get banner images to load in browser. They load fine on the Android app, but most of what I want to use Karakeep for is in the browser.

<!-- gh-comment-id:2993770723 --> @imamouse111 commented on GitHub (Jun 21, 2025): Has anybody been able to solve this? I'm on v0.25.0 and still unable to get banner images to load in browser. They load fine on the Android app, but most of what I want to use Karakeep for is in the browser.
kerem commented 2026-03-02 11:51:38 +03:00
Author
Owner
Copy link

@qmarquez commented on GitHub (Aug 22, 2025):

HI team! Im facing this same issue! the next image optimization it´s throwing errors when the image api is called. On a first attemp I attack the cloudflare tunnel I use to access the selfhost server, then I try to access just locally (changing the next auth URL to the local ip and face the same issues).
There are some clue about what could be happening here?

<!-- gh-comment-id:3214971328 --> @qmarquez commented on GitHub (Aug 22, 2025): HI team! Im facing this same issue! the next image optimization it´s throwing errors when the image api is called. On a first attemp I attack the cloudflare tunnel I use to access the selfhost server, then I try to access just locally (changing the next auth URL to the local ip and face the same issues). There are some clue about what could be happening here?
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
refactor/use-npm-singlefile
onetab
claude/issue-2596-20260321-1401
claude/fix-docs-button-responsive-V3aBQ
claude/review-import-backpressure-D4ArJ
claude/fix-archived-bookmarks-mobile-P9OJW
claude/issue-1189-20260211-1601
claude/fix-nested-smart-lists-3uFkt
claude/issue-2298-20251223-1704
feat/import-v3
claude/add-cli-search-subcommand-6kIe0
claude/add-bookmark-indexing-timestamps-96bPj
claude/auto-disable-failing-feeds-fkDhP
claude/add-tag-search-aliases-HzESD
feat/docker-compose-dev
claude/add-attachedby-tags-endpoint-01WYfemMGHJJjXsPYLvUJAno
claude/fix-crawler-memory-leaks-NE7Ct
bookmark-debugger
claude/issue-2352-20260106-1120
claude/issue-1977-20260102-2348
claude/add-banner-rendering-JeLUk
claude/add-descendant-qualifier-cUm26
claude/skip-metadata-refresh-archives-CAo4Y
claude/fix-archive-pending-banner-pAyGM
claude/add-embeddings-support-h2swV
claude/nested-manage-lists-QVV85
claude/privacy-type-system-MG1bT
claude/add-action-menu-icons-6hNKw
claude/issue-2299-20251223-1711
claude/bookmark-indexing-progress-QwZSI
claude/migrate-bookmark-attachments-3O2te
claude/add-2025-wrapped-feature-tIUIh
claude/improve-ai-settings-design-639tq
claude/add-youtube-metascraper-plugin-0lWC7
claude/add-problem-reporting-gSSEV
claude/add-mobile-list-menus-spcS7
claude/shadcn-bookmark-cards-WWHzP
claude/add-extensions-link-HTeXc
claude/add-onboarding-screens-hsYMO
claude/fix-settings-switch-overflow-nlzM4
claude/clamp-bookmark-titles-diAEz
claude/port-stats-mobile-expo-MuXAn
claude/whats-new-base-version-vrv8C
claude/fix-settings-auth-checks-jgyD8
claude/add-server-version-display-3sGa2
claude/fix-tag-editor-scrolling-rzdbG
claude/add-company-pricing-card-y5mHY
claude/audit-optimize-transactions-xpDVc
codex/ensure-consistent-ui-experience-across-app-pages
claude/plan-opentelemetry-integration-01Jx183mz1Ev8h8JoYj97Auw
libsql
db-indicies
claude/export-import-lists-01UuCWwdaqduAd35NppvjnMD
claude/configurable-worker-timeout-0198GQh6YrrRzqG62xnogyrz
claude/check-import-quota-01CPdxTpHp18Ba62bYcBTVbA
claude/scraper-worker-thread-01FEHen6MGrQHmdBstJSuiyA
claude/customize-dialog-styling-01CVjEv2KgyZJSpCg3mqkvR7
claude/add-asset-cache-headers-0175WhNcqwiwurrmjj52jnLT
claude/add-db-search-plugin-017Xxd4Jq3MfjWT788vgfbaq
benchmarks-2
claude/add-filtered-deletion-01DTxWNcg3hhqdNpeNLa3s6L
claude/actionbutton-loading-spinner-015DY5ZTvgPgFAXTZz3UGaYv
claude/add-broken-links-qualifier-01S31X1LsKiYb9gE1dXTKvi3
claude/docker-release-tag-trigger-01UmzFXEumhK2jdmRGtMcueo
claude/spread-feed-fetch-scheduling-01EihUtmZSyqeE1HfRMessxW
restate-idempotency
claude/align-android-ios-colors-01GJfkhEyZVBReohVioPa8ok
claude/improve-mobile-app-colors-0155LzHfkd5HyJr6YyZMsus5
codex/add-autocomplete-for-search-query-language
claude/add-bookmark-backups-016L2A8Z94n7tDgDdMPdFuAd
claude/restrict-binary-user-permissions-01FSGyy2RXGZvE26YbAejzGi
effect-ts
claude/prepare-trpc-npm-publish-0193EjfwpxSNVNcLXqXjs6Ln
shared-list-sidebar
claude/lazy-load-tiktoken-017UTNpJPTcMMQvNEBa1aFwo
codex/fix-asset-pre-processing-worker-abort-signals
add-groupid
claude/add-bookmark-list-button-01VF7uXYNLsVDzqdozWMXP5M
claude/extract-shared-ui-components-01DSVfaCr6WRqAyx1vJTZk9r
claude/migrate-shadcn-sidebar-01DKjpg9MD5PJ2potemSnbvW
claude/add-collaborators-rate-limits-01VjXyRWWPUkGQKa8d8D8qKj
claude/modernize-dark-mode-01FRfE81PAY5C44pFu1cYocf
claude/add-signed-url-bookmark-01PjYT1ZhvLK2FPJNTAhJsWf
restate-group-id
claude/add-highlights-page-012vhHpn8fVNp3gf7gBeW14s
claude/disable-shared-bookmark-features-01B9fiGUdu6NyWaxSQFsQBxP
claude/mobile-bookmark-grid-layouts-018cGBBMhPJVq6PJVRBpqT2r
claude/add-mobile-bookmark-summary-01494LYoh4sJW5Fj4GPm62Vj
claude/add-mobile-tags-screen-01WRADt4ZzvXVew1Y9vqF8SV
claude/add-highlight-notes-01LpanRLS4a2YMnT1qB5GTqX
claude/add-search-bar-014k2ngaqjwYRVSvqmbuECqr
claude/hide-collaborator-emails-01TQrkkMupC7CR9BTuDkireg
claude/list-invitation-approval-0129V89M1riXW6JqmoF74VfM
claude/add-bookmark-archive-sort-018VbGPGvtmsGgXFEERoAX7B
claude/add-mobile-smart-lists-01251tYo9u1SywE6XFezAv9e
claude/bookmark-drag-drop-01DmWq286ogHpDGHKcXjKr3z
claude/add-rss-import-01DH1Q2axcDeq8nQJR5MWjPJ
claude/mobile-inapp-browser-auth-01KiT6bwyntRPQ1X4oTtAveC
claude/offline-mode-react-query-01D1rE2bdBEPw2teGqunr5Gd
claude/add-singlefile-extension-support-01BEB9QQZABzwfZDvR9Bz5b2
claude/custom-list-slugs-01VxcfkNUXZ97FNpNVURopMq
claude/issue-2148-20251118-1133
claude/add-groupid-queue-fairness-011CV1r8Wb46HuGAg5o95i3m
claude/hide-viewer-shared-lists-01Fst6NBvdxrXXnDhUmjsNDP
claude/collaborative-lists-013AvDvMqkoszDVcSoCYgBcM
claude/implement-feature-01LT5XzGsbEhZkYXNEjEwdui
claude/fix-bookmark-loading-state-01AgF4H2drxwuTCJDB2Xgiu4
claude/admin-user-edit-013tbiRmb1KX2fhSYqmGKCu8
claude/expose-all-api-01YTruEW72WQYMtq4iZoaPkA
claude/add-doc-link-main-016NYLxShpKuH6R8XCBgeZtc
claude/fix-issue-2133-019JLvdSRAUbU4FtjQztcM6S
claude/explore-effect-ts-integration-01F7xb1dWwP1ma4LnLbFGfDD
claude/optimize-dockerfile-build-011CV5gDnPZbdbbVSPDofC4e
claude/add-custom-headers-guide-011CV249t16aWDRb1mCrzQdC
claude/mobile-app-signup-011CUxPtCXgU6U3T8GShTR2Q
claude/crawler-worker-fetch-browser-011CUvcRc24XEr9DTWDW6MX8
claude/fix-issue-784-011CUvubQrcZHG9S3KjpCKbK
codex/add-user-settings-for-inference-language-and-screenshots
claude/fix-mobile-signin-server-address-011CUnaUWwY2Fhq5Xbwhgr8H
better-auth-2
claude/issue-2028-20251012-1429
claude/issue-1010-20251012-1154
codex/update-feed-refresh-job-idempotency-key
restate
import-v2
fix-public-lists
recurse-delete-list
abort-dangling-processing
tag-pagination
ratelimit-plugin
claude/issue-1937-20250914-0912
codex/implement-title-search-query-qualifier
copilot/add-edit-button-for-notes
cookie-path
ai-tag-cleanup
codex/add-allowlist-and-blocklist-env-variables
mobile-retheme
expo-next-upgrade
opencode/issue1788-20250727215611
fix-trailing-slash-deduplication
edit-bookmark-dialog
bookmark-embeddings
rag
nextjs-15
bookmark-hover-bar
sapling-pr-archive-MohamedBassem
track-bookmark-assets
json-cli
admin-settings
mobile-dark-mode
android/v1.9.2-0
ios/v1.9.1-1
android/v1.9.1-0
ios/v1.9.1-0
ios/v1.9.0-2
ios/v1.9.0-1
android/v1.9.0-1
extension/v1.2.9
cli/v0.31.0
sdk/v0.31.0
mcp/v0.31.0
android/v1.9.0-0
ios/v1.9.0-0
v0.31.0
android/v1.8.5-0
cli/v0.30.0
sdk/v0.30.0
ios/v1.8.4-0
android/v1.8.4-0
v0.30.0
cli/v0.29.1
v0.29.3
v0.29.2
v0.29.1
sdk/v0.29.0
cli/v0.29.0
mcp/v0.29.0
ios/v1.8.3-0
android/v1.8.3-0
extension/v1.2.8
v0.29.0
android/v1.8.2-2
android/v1.8.2-1
ios/v1.8.2-0
android/v1.8.2-0
extension/v1.2.7
android/v1.8.1-0
ios/v1.8.1-0
v0.28.0
cli/v0.27.1
cli/v0.27.0
v0.27.1
sdk/v0.27.0
v0.27.0
android/v1.8.0-1
ios/v1.8.0-1
mcp/v0.26.0
sdk/v0.26.0
v0.26.0
cli/v0.25.0
ios/v1.7.0-1
mcp/v0.25.0
v0.25.0
extension/v1.2.6
ios/v1.7.0-0
android/v1.7.0-0
v0.24.1
v0.24.0
mcp/v0.23.10
mcp/v0.23.9
mcp/v0.23.8
extension/v1.2.5
mcp/v0.23.7
mcp/v0.23.6
mcp/v0.23.5
mcp/v0.23.4
sdk/v0.23.2
cli/v0.23.0
extension/v1.2.4
android/v1.6.9-1
ios/v1.6.9-1
v0.23.2
v0.23.1
sdk/v0.23.0
v0.23.0
ios/v1.6.9-0
sdk/v0.22.0
v0.22.0
android/v1.6.8-0
ios/v1.6.8-0
sdk/v0.21.2
sdk/v0.21.1
sdk/v0.21.0
v0.21.0
cli/v0.20.0
v0.20.0
ios/v1.6.7-4
android/v1.6.7-4
ios/v1.6.7-3
android/v1.6.7-3
android/v1.6.7-2
ios/v1.6.7-2
android/v1.6.7-1
ios/v1.6.7-1
ios/v1.6.7-0
android/v1.6.7-0
v0.19.0
android/v1.6.6-0
android/v1.6.5-0
ios/v1.6.5-0
ios/v1.6.4-0
android/v1.6.4-0
v0.18.0
v0.17.1
v0.17.0
ios/v1.6.3-0
android/v1.6.3-0
extension/v1.2.3
ios/v1.6.2-1
android/v1.6.2-1
ios/v1.6.2-0
android/v1.6.2-0
v0.16.0
ios/v1.6.1-3
android/v1.6.1-3
ios/v1.6.1-2
android/v1.6.1-2
android/v1.6.1-1
ios/v1.6.1-1
android/v1.6.1-0
ios/v1.6.1-0
extension/v1.2.2
android/v1.6.0-1
ios/v1.6.0-1
ios/v1.6.0
android/v1.6.0
cli/v0.13.7
cli/v0.13.6
v0.15.0
cli/v0.13.5
extension/v1.2.1
v0.14.0
cli/v0.13.3
cli/v0.13.2
cli/v0.13.1
cli/v0.13.0
v0.13.1
v0.13.0
mobile-v1.5.0
mobile-v1.4.0
v0.12.2
v0.12.1
v0.12.0
v0.11.1
v0.11.0
v0.10.1
v0.10.0
v0.9.0
v0.8.0
v0.7.0
v0.6.0
v0.5.0
v0.4.1
v.0.4.0
v.0.3.1
v0.3.0
v0.2.2
v0.2.1
v0.2.0
v0.1.0
Labels
Clear labels   
UI/UX

   
android

   
bug

   
dependencies

   
documentation

   
documentation

   
extension

   
feature request

   
feature request

   
good first issue

   
ios

   
long-term

   
performance

   
pri/high

   
pri/low

   
pri/medium

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
status/approved

   
status/icebox

   
status/pending_clarification

   
status/untriaged
No labels
UI/UX
android
bug
dependencies
documentation
documentation
extension
feature request
feature request
good first issue
ios
long-term
performance
pri/high
pri/low
pri/medium
pull-request
question
status/approved
status/icebox
status/pending_clarification
status/untriaged
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/karakeep#647
No description provided.