starred/karakeep
1
0
Fork 0
mirror of https://github.com/karakeep-app/karakeep.git synced 2026-04-25 16:06:04 +03:00
Code Issues 569 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #879] Unable to cache images locally, and unable to capture a full-page screenshot. #573

New issue
Closed
opened 2026-03-02 11:50:58 +03:00 by kerem · 4 comments
kerem commented 2026-03-02 11:50:58 +03:00
Owner
Copy link

Originally created by @xzdbadboy on GitHub (Jan 14, 2025).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/879

Describe the Bug

I have already set CRAWLER_DOWNLOAD_BANNER_IMAGE, CRAWLER_STORE_SCREENSHOT, and CRAWLER_FULL_PAGE_SCREENSHOT to true, but it seems like they are not working.

During the crawling process, some pages' code and iamges was lost, and I have taken screenshots.

The screenshots are incomplete; only one screen's worth was captured.

Steps to Reproduce

hoarder-web logs

2025/01/14 10:30:04 | stdout | 2025-01-14T02:30:04.463Z info: [search][228] Completed successfully
2025/01/14 10:30:04 | stdout | 2025-01-14T02:30:04.281Z info: [search][228] Attempting to index bookmark with id bwj64p6bok5o0ylklkapgh0m ...
2025/01/14 10:30:04 | stdout | 2025-01-14T02:30:04.076Z info: [inference][225] Completed successfully
2025/01/14 10:30:03 | stdout | 2025-01-14T02:30:03.970Z info: [inference][225] Inferring tag for bookmark "bwj64p6bok5o0ylklkapgh0m" used 996 tokens and inferred: QNAP NAS,Docker,网络代理,技术教程,什么值得买
2025/01/14 10:30:02 | stdout | 2025-01-14T02:30:02.189Z info: [search][226] Completed successfully
2025/01/14 10:30:01 | stdout | 2025-01-14T02:30:01.115Z info: [VideoCrawler][227] Video Download Completed successfully
2025/01/14 10:30:01 | stdout | 2025-01-14T02:30:01.115Z info: [VideoCrawler][227] Skipping video download from "https://post.smzdm.com/p/awopqn7g/", because it is disabled in the config.
2025/01/14 10:30:01 | stdout | 2025-01-14T02:30:01.091Z info: [inference][225] Starting an inference job for bookmark with id "bwj64p6bok5o0ylklkapgh0m"
2025/01/14 10:30:01 | stdout | 2025-01-14T02:30:01.066Z info: [search][226] Attempting to index bookmark with id bwj64p6bok5o0ylklkapgh0m ...
2025/01/14 10:30:00 | stdout | 2025-01-14T02:30:00.675Z info: [Crawler][219] Completed successfully
2025/01/14 10:30:00 | stdout | 2025-01-14T02:30:00.580Z info: [Crawler][219] Downloaded image as assetId: d3503dd1-d802-4562-b06d-a225adbc9981
2025/01/14 10:29:58 | stdout | 2025-01-14T02:29:58.525Z info: [Crawler][219] Downloading image from "https://a.zdmimg.com/202407/05/6687a7416056e4638.png_fo742.jpg"
2025/01/14 10:29:58 | stdout | 2025-01-14T02:29:58.524Z info: [Crawler][219] Done extracting metadata from the page.
2025/01/14 10:29:55 | stdout | 2025-01-14T02:29:55.026Z info: [Crawler][219] Stored the screenshot as assetId: b082351a-a9a3-4c60-9bbe-e4fc85edcb38
2025/01/14 10:29:54 | stdout | 2025-01-14T02:29:54.994Z info: [Crawler][219] Done extracting readable content.
2025/01/14 10:29:52 | stdout | 2025-01-14T02:29:52.605Z info: [Crawler][219] Will attempt to extract readable content ...
2025/01/14 10:29:50 | stdout | 2025-01-14T02:29:50.706Z info: [Crawler][219] Will attempt to extract metadata from page ...
2025/01/14 10:29:50 | stdout | 2025-01-14T02:29:50.699Z info: [Crawler][219] Finished capturing page content and a screenshot. FullPageScreenshot: false
2025/01/14 10:29:50 | stdout | 2025-01-14T02:29:50.374Z info: [Crawler][219] Successfully fetched the page content.
2025/01/14 10:29:50 | stdout | 2025-01-14T02:29:50.304Z info: [Crawler][219] Finished waiting for the page to load.
2025/01/14 10:29:49 | stdout | 2025-01-14T02:29:49.254Z info: [Crawler][219] Successfully navigated to "https://post.smzdm.com/p/awopqn7g/". Waiting for the page to load ...
2025/01/14 10:29:47 | stdout | 2025-01-14T02:29:47.994Z info: [search][224] Completed successfully
2025/01/14 10:29:47 | stdout | 2025-01-14T02:29:47.758Z info: [search][224] Attempting to index bookmark with id atkzrc8zccyzz12ybpffs3z3 ...
2025/01/14 10:29:47 | stdout | 2025-01-14T02:29:47.256Z info: [inference][221] Completed successfully
2025/01/14 10:29:47 | stdout | 2025-01-14T02:29:47.163Z info: [inference][221] Inferring tag for bookmark "atkzrc8zccyzz12ybpffs3z3" used 2261 tokens and inferred: 人工智能,数据科学,本地RAG服务,Docker,Open WebUI,Ollama,Qwen2.5,文档检索,生成式AI
2025/01/14 10:29:45 | stdout | 2025-01-14T02:29:45.044Z info: [Crawler][219] Content-type for the url https://post.smzdm.com/p/awopqn7g/ is "text/html; charset=UTF-8"
2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.705Z info: [search][222] Completed successfully
2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.275Z info: [VideoCrawler][223] Video Download Completed successfully
2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.275Z info: [VideoCrawler][223] Skipping video download from "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/", because it is disabled in the config.
2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.239Z info: [inference][221] Starting an inference job for bookmark with id "atkzrc8zccyzz12ybpffs3z3"
2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.211Z info: [search][222] Attempting to index bookmark with id atkzrc8zccyzz12ybpffs3z3 ...
2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.088Z info: [Crawler][219] Attempting to determine the content-type for the url https://post.smzdm.com/p/awopqn7g/
2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.088Z info: [Crawler][219] Will crawl "https://post.smzdm.com/p/awopqn7g/" for link with id "bwj64p6bok5o0ylklkapgh0m"
2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.038Z info: [Crawler][217] Completed successfully
2025/01/14 10:29:43 | stdout | 2025-01-14T02:29:43.886Z info: [Crawler][217] Downloaded image as assetId: 3c11ee08-634a-412c-9ad3-a695dadd36e9
2025/01/14 10:29:42 | stdout | 2025-01-14T02:29:42.370Z info: [Crawler][217] Downloading image from "https://cloud.cuterwrite.fun/img/2024-06-29_116903285_p0_master1200.webp"
2025/01/14 10:29:42 | stdout | 2025-01-14T02:29:42.370Z info: [Crawler][217] Done extracting metadata from the page.
2025/01/14 10:29:42 | stdout | 2025-01-14T02:29:42.203Z info: [Crawler][217] Stored the screenshot as assetId: adb0dcfa-ab3b-4ed4-bc87-c4fcb6dfe2d7
2025/01/14 10:29:42 | stdout | 2025-01-14T02:29:42.128Z info: [Crawler][217] Done extracting readable content.
2025/01/14 10:29:41 | stdout | 2025-01-14T02:29:41.428Z info: [Crawler][217] Will attempt to extract readable content ...
2025/01/14 10:29:40 | stdout | 2025-01-14T02:29:40.642Z info: [Crawler][217] Will attempt to extract metadata from page ...
2025/01/14 10:29:40 | stdout | 2025-01-14T02:29:40.632Z info: [Crawler][217] Finished capturing page content and a screenshot. FullPageScreenshot: false
2025/01/14 10:29:40 | stdout | 2025-01-14T02:29:40.199Z info: [Crawler][217] Successfully fetched the page content.
2025/01/14 10:29:40 | stdout | 2025-01-14T02:29:40.189Z info: [Crawler][217] Finished waiting for the page to load.
2025/01/14 10:29:39 | stdout | 2025-01-14T02:29:39.677Z info: [search][220] Completed successfully
2025/01/14 10:29:39 | stdout | 2025-01-14T02:29:39.584Z info: [search][220] Attempting to index bookmark with id bwj64p6bok5o0ylklkapgh0m ...
2025/01/14 10:29:37 | stdout | 2025-01-14T02:29:37.513Z info: [Crawler][217] Successfully navigated to "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/". Waiting for the page to load ...
2025/01/14 10:29:35 | stdout | 2025-01-14T02:29:35.809Z info: [Crawler][217] Content-type for the url https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/ is "text/html; charset=utf-8"
2025/01/14 10:29:34 | stdout | 2025-01-14T02:29:34.538Z info: [search][218] Completed successfully
2025/01/14 10:29:34 | stdout | 2025-01-14T02:29:34.370Z info: [search][218] Attempting to index bookmark with id atkzrc8zccyzz12ybpffs3z3 ...
2025/01/14 10:29:34 | stdout | 2025-01-14T02:29:34.244Z info: [Crawler][217] Attempting to determine the content-type for the url https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/
2025/01/14 10:29:34 | stdout | 2025-01-14T02:29:34.244Z info: [Crawler][217] Will crawl "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/" for link with id "atkzrc8zccyzz12ybpffs3z3"
2025/01/14 10:29:28 | stderr | (Use `node --trace-deprecation ...` to show where the warning was created)
2025/01/14 10:29:28 | stderr | (node:69) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.100Z info: Starting asset preprocessing worker ...
2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.100Z info: Starting feed worker ...
2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.099Z info: Starting video worker ...
2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.099Z info: Starting tidy assets worker ...
2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.098Z info: Starting search indexing worker ...
2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.098Z info: Starting inference worker ...
2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.097Z info: Starting crawler worker ...
2025/01/14 10:28:26 | stdout | 2025-01-14T02:28:26.946Z info: [Crawler] Successfully resolved IP address, new address: http://192.168.96.4:9222/
2025/01/14 10:28:26 | stdout | 2025-01-14T02:28:26.938Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222
2025/01/14 10:28:26 | stderr | (Use `node --trace-deprecation ...` to show where the warning was created)
2025/01/14 10:28:26 | stderr | (node:121) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
2025/01/14 10:28:26 | stdout | 2025-01-14T02:28:26.835Z info: [crawler] Loading adblocker ...
2025/01/14 10:28:26 | stdout | 2025-01-14T02:28:26.820Z info: Workers version: nightly
2025/01/14 10:28:22 | stdout |  
2025/01/14 10:28:22 | stdout | > tsx index.ts
2025/01/14 10:28:22 | stdout | > @hoarder/workers@0.1.0 start:prod /app/apps/workers
2025/01/14 10:28:22 | stdout |  
2025/01/14 10:28:22 | stdout | ✓ Ready in 662ms
2025/01/14 10:28:21 | stdout | ✓ Starting...
2025/01/14 10:28:21 | stdout |  
2025/01/14 10:28:21 | stdout | - Network:      http://0.0.0.0:3000
2025/01/14 10:28:21 | stdout | - Local:        http://localhost:3000
2025/01/14 10:28:21 | stdout | ▲ Next.js 14.2.21
2025/01/14 10:28:21 | stderr | s6-rc: info: service legacy-services successfully started
2025/01/14 10:28:21 | stderr | s6-rc: info: service legacy-services: starting
2025/01/14 10:28:21 | stderr | s6-rc: info: service svc-web successfully started
2025/01/14 10:28:21 | stderr | s6-rc: info: service svc-workers successfully started
2025/01/14 10:28:21 | stderr | s6-rc: info: service svc-web: starting
2025/01/14 10:28:21 | stderr | s6-rc: info: service svc-workers: starting
2025/01/14 10:28:21 | stderr | s6-rc: info: service init-db-migration successfully started
2025/01/14 10:28:21 | stderr | s6-rc: info: service legacy-cont-init successfully started
2025/01/14 10:28:21 | stderr | s6-rc: info: service legacy-cont-init: starting
2025/01/14 10:28:21 | stderr | s6-rc: info: service fix-attrs successfully started
2025/01/14 10:28:21 | stdout | Running db migration script
2025/01/14 10:28:21 | stderr | s6-rc: info: service init-db-migration: starting
2025/01/14 10:28:21 | stderr | s6-rc: info: service fix-attrs: starting
2025/01/14 10:28:21 | stderr | s6-rc: info: service s6rc-oneshot-runner successfully started
2025/01/14 10:28:21 | stderr | s6-rc: info: service s6rc-oneshot-runner: starting
2025/01/14 10:28:15 | stdout | 2025-01-14T02:28:15.112Z info: [Crawler] Successfully resolved IP address, new address: http://192.168.96.4:9222/
2025/01/14 10:28:15 | stdout | 2025-01-14T02:28:15.112Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222
2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.131Z info: Shutting down crawler, openai, tidyAssets, video, feed, assetPreprocessing and search workers ...
2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.131Z info: Received SIGTERM, shutting down ...
2025/01/14 10:28:10 | stderr | s6-rc: info: service svc-web successfully stopped
2025/01/14 10:28:10 | stdout | at async /app/apps/workers/crawlerWorker.ts:2:4664
2025/01/14 10:28:10 | stdout | at async PuppeteerExtra.connect (/app/apps/workers/node_modules/.pnpm/puppeteer-extra@3.3.6_puppeteer@22.3.0_typescript@5.3.3_/node_modules/puppeteer-extra/dist/index.cjs.js:151:25)
2025/01/14 10:28:10 | stdout | at async _connectToBrowser (/app/apps/workers/node_modules/.pnpm/puppeteer-core@22.3.0/node_modules/puppeteer-core/lib/cjs/puppeteer/common/BrowserConnector.js:50:50)
2025/01/14 10:28:10 | stdout | at async getConnectionTransport (/app/apps/workers/node_modules/.pnpm/puppeteer-core@22.3.0/node_modules/puppeteer-core/lib/cjs/puppeteer/common/BrowserConnector.js:81:31)
2025/01/14 10:28:10 | stdout | at async getWSEndpoint (/app/apps/workers/node_modules/.pnpm/puppeteer-core@22.3.0/node_modules/puppeteer-core/lib/cjs/puppeteer/common/BrowserConnector.js:94:24)
2025/01/14 10:28:10 | stdout | at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
2025/01/14 10:28:10 | stdout | at node:internal/deps/undici/undici:13484:13
2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.108Z error: [Crawler] Failed to connect to the browser instance, will retry in 5 secs: TypeError: Failed to fetch browser webSocket URL from http://192.168.96.4:9222/json/version: fetch failed
2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.100Z info: [Crawler] Successfully resolved IP address, new address: http://192.168.96.4:9222/
2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.097Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222
2025/01/14 10:28:10 | stderr | s6-rc: info: service fix-attrs successfully stopped
2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.095Z info: [Crawler] The puppeteer browser got disconnected. Will attempt to launch it again.
2025/01/14 10:28:10 | stderr | s6-rc: info: service fix-attrs: stopping
2025/01/14 10:28:10 | stderr | s6-rc: info: service legacy-cont-init successfully stopped
2025/01/14 10:28:10 | stderr | s6-rc: info: service svc-web: stopping
2025/01/14 10:28:10 | stderr | s6-rc: info: service svc-workers: stopping
2025/01/14 10:28:10 | stderr | s6-rc: info: service legacy-cont-init: stopping
2025/01/14 10:28:10 | stderr | s6-rc: info: service legacy-services successfully stopped
2025/01/14 10:28:10 | stderr | s6-rc: info: service legacy-services: stopping
2025/01/14 10:28:06 | stdout | 2025-01-14T02:28:06.001Z info: [search][216] Completed successfully
2025/01/14 10:28:05 | stdout | 2025-01-14T02:28:05.874Z info: [search][216] Attempting to index bookmark with id ua33wgcehsgv7wls1jk79l39 ...
2025/01/14 10:28:04 | stderr | }
2025/01/14 10:28:04 | stderr | [cause]: undefined
2025/01/14 10:28:04 | stderr | code: 'NOT_FOUND',
2025/01/14 10:28:04 | stderr | at async Promise.all (index 1) {
2025/01/14 10:28:04 | stderr | at async /app/apps/web/.next/server/app/api/trpc/[trpc]/route.js:1:4379
2025/01/14 10:28:04 | stderr | at async t (/app/apps/web/.next/server/chunks/480.js:4:46248)
2025/01/14 10:28:04 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909)
2025/01/14 10:28:04 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909)
2025/01/14 10:28:04 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909)
2025/01/14 10:28:04 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909)
2025/01/14 10:28:04 | stderr | at async t (/app/apps/web/.next/server/chunks/480.js:4:45282)
2025/01/14 10:28:04 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909)
2025/01/14 10:28:04 | stderr | at /app/apps/web/.next/server/chunks/6815.js:1:26219
2025/01/14 10:28:04 | stderr | p [TRPCError]: Bookmark not found
2025/01/14 10:28:03 | stdout | 2025-01-14T02:28:03.831Z info: [search][215] Completed successfully
2025/01/14 10:28:03 | stdout | 2025-01-14T02:28:03.672Z info: [search][215] Attempting to index bookmark with id sw3gz67beg5aopukjn1saaic ...
2025/01/14 10:28:02 | stderr | }
2025/01/14 10:28:02 | stderr | [cause]: undefined
2025/01/14 10:28:02 | stderr | code: 'NOT_FOUND',
2025/01/14 10:28:02 | stderr | at async Promise.all (index 1) {
2025/01/14 10:28:02 | stderr | at async /app/apps/web/.next/server/app/api/trpc/[trpc]/route.js:1:4379
2025/01/14 10:28:02 | stderr | at async t (/app/apps/web/.next/server/chunks/480.js:4:46248)
2025/01/14 10:28:02 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909)
2025/01/14 10:28:02 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909)
2025/01/14 10:28:02 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909)
2025/01/14 10:28:02 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909)
2025/01/14 10:28:02 | stderr | at async t (/app/apps/web/.next/server/chunks/480.js:4:45282)
2025/01/14 10:28:02 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909)
2025/01/14 10:28:02 | stderr | at /app/apps/web/.next/server/chunks/6815.js:1:26219
2025/01/14 10:28:02 | stderr | p [TRPCError]: Bookmark not found
2025/01/14 10:27:28 | stdout | 2025-01-14T02:27:28.567Z info: [search][214] Completed successfully
2025/01/14 10:27:28 | stdout | 2025-01-14T02:27:28.432Z info: [search][214] Attempting to index bookmark with id ua33wgcehsgv7wls1jk79l39 ...
2025/01/14 10:27:27 | stdout | 2025-01-14T02:27:27.897Z info: [inference][211] Completed successfully
2025/01/14 10:27:27 | stdout | 2025-01-14T02:27:27.852Z info: [inference][211] Inferring tag for bookmark "ua33wgcehsgv7wls1jk79l39" used 2261 tokens and inferred: 人工智能,数据科学,本地RAG服务,Open WebUI,Ollama,Qwen2.5,Docker,文档检索,生成式AI
2025/01/14 10:27:22 | stdout | 2025-01-14T02:27:22.379Z info: [search][212] Completed successfully
2025/01/14 10:27:22 | stdout | 2025-01-14T02:27:22.241Z info: [search][212] Attempting to index bookmark with id ua33wgcehsgv7wls1jk79l39 ...
2025/01/14 10:27:22 | stdout | 2025-01-14T02:27:22.216Z info: [inference][211] Starting an inference job for bookmark with id "ua33wgcehsgv7wls1jk79l39"
2025/01/14 10:27:22 | stdout | 2025-01-14T02:27:22.179Z info: [VideoCrawler][213] Video Download Completed successfully
2025/01/14 10:27:22 | stdout | 2025-01-14T02:27:22.179Z info: [VideoCrawler][213] Skipping video download from "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/", because it is disabled in the config.
2025/01/14 10:27:21 | stdout | 2025-01-14T02:27:21.922Z info: [Crawler][205] Completed successfully
2025/01/14 10:27:21 | stdout | 2025-01-14T02:27:21.833Z info: [Crawler][205] Downloaded image as assetId: 0df8f5c5-252f-4f0b-8f2f-4f199c5ff0d3
2025/01/14 10:27:20 | stdout | 2025-01-14T02:27:20.344Z info: [Crawler][205] Downloading image from "https://cloud.cuterwrite.fun/img/2024-06-29_116903285_p0_master1200.webp"
2025/01/14 10:27:20 | stdout | 2025-01-14T02:27:20.344Z info: [Crawler][205] Done extracting metadata from the page.
2025/01/14 10:27:18 | stdout | 2025-01-14T02:27:18.154Z info: [Crawler][205] Stored the screenshot as assetId: d066ed42-e62c-4690-b0f3-5ae3b92beea9
2025/01/14 10:27:18 | stdout | 2025-01-14T02:27:18.130Z info: [Crawler][205] Done extracting readable content.
2025/01/14 10:27:17 | stdout | 2025-01-14T02:27:17.645Z info: [Crawler][205] Will attempt to extract readable content ...
2025/01/14 10:27:16 | stdout | 2025-01-14T02:27:16.965Z info: [Crawler][205] Will attempt to extract metadata from page ...
2025/01/14 10:27:16 | stdout | 2025-01-14T02:27:16.958Z info: [Crawler][205] Finished capturing page content and a screenshot. FullPageScreenshot: false
2025/01/14 10:27:16 | stdout | 2025-01-14T02:27:16.531Z info: [Crawler][205] Successfully fetched the page content.
2025/01/14 10:27:16 | stdout | 2025-01-14T02:27:16.522Z info: [Crawler][205] Finished waiting for the page to load.
2025/01/14 10:27:13 | stdout | 2025-01-14T02:27:13.809Z info: [Crawler][205] Successfully navigated to "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/". Waiting for the page to load ...
2025/01/14 10:27:13 | stdout | 2025-01-14T02:27:13.587Z info: [search][210] Completed successfully
2025/01/14 10:27:13 | stdout | 2025-01-14T02:27:13.431Z info: [search][210] Attempting to index bookmark with id sw3gz67beg5aopukjn1saaic ...
2025/01/14 10:27:13 | stdout | 2025-01-14T02:27:13.367Z info: [inference][207] Completed successfully
2025/01/14 10:27:13 | stdout | 2025-01-14T02:27:13.307Z info: [inference][207] Inferring tag for bookmark "sw3gz67beg5aopukjn1saaic" used 996 tokens and inferred: QNAP NAS,Docker,网络代理,技术教程,什么值得买
2025/01/14 10:27:11 | stdout | 2025-01-14T02:27:11.540Z info: [Crawler][205] Content-type for the url https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/ is "text/html; charset=utf-8"
2025/01/14 10:27:11 | stdout | 2025-01-14T02:27:11.314Z info: [search][208] Completed successfully
2025/01/14 10:27:10 | stdout | 2025-01-14T02:27:10.265Z info: [VideoCrawler][209] Video Download Completed successfully
2025/01/14 10:27:10 | stdout | 2025-01-14T02:27:10.265Z info: [VideoCrawler][209] Skipping video download from "https://post.smzdm.com/p/awopqn7g/", because it is disabled in the config.
2025/01/14 10:27:10 | stdout | 2025-01-14T02:27:10.227Z info: [inference][207] Starting an inference job for bookmark with id "sw3gz67beg5aopukjn1saaic"
2025/01/14 10:27:10 | stdout | 2025-01-14T02:27:10.197Z info: [search][208] Attempting to index bookmark with id sw3gz67beg5aopukjn1saaic ...
2025/01/14 10:27:09 | stdout | 2025-01-14T02:27:09.821Z info: [Crawler][205] Attempting to determine the content-type for the url https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/
2025/01/14 10:27:09 | stdout | 2025-01-14T02:27:09.821Z info: [Crawler][205] Will crawl "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/" for link with id "ua33wgcehsgv7wls1jk79l39"
2025/01/14 10:27:09 | stdout | 2025-01-14T02:27:09.772Z info: [Crawler][203] Completed successfully
2025/01/14 10:27:09 | stdout | 2025-01-14T02:27:09.637Z info: [Crawler][203] Downloaded image as assetId: 6f0b7a19-2453-4a23-a633-860d17c1f5bc
2025/01/14 10:26:50 | stdout | 2025-01-14T02:26:50.670Z info: [Crawler][203] Downloading image from "https://a.zdmimg.com/202407/05/6687a7416056e4638.png_fo742.jpg"
2025/01/14 10:26:50 | stdout | 2025-01-14T02:26:50.670Z info: [Crawler][203] Done extracting metadata from the page.
2025/01/14 10:26:50 | stdout | 2025-01-14T02:26:50.131Z info: [Crawler][203] Stored the screenshot as assetId: 6e53e387-aec4-440b-94ff-b2a27fa8a21b
2025/01/14 10:26:50 | stdout | 2025-01-14T02:26:50.055Z info: [Crawler][203] Done extracting readable content.
2025/01/14 10:26:47 | stdout | 2025-01-14T02:26:47.410Z info: [Crawler][203] Will attempt to extract readable content ...
2025/01/14 10:26:44 | stdout | 2025-01-14T02:26:44.890Z info: [Crawler][203] Will attempt to extract metadata from page ...
2025/01/14 10:26:44 | stdout | 2025-01-14T02:26:44.884Z info: [Crawler][203] Finished capturing page content and a screenshot. FullPageScreenshot: false
2025/01/14 10:26:44 | stdout | 2025-01-14T02:26:44.560Z info: [Crawler][203] Successfully fetched the page content.
2025/01/14 10:26:44 | stdout | 2025-01-14T02:26:44.493Z info: [Crawler][203] Finished waiting for the page to load.
2025/01/14 10:26:43 | stdout | 2025-01-14T02:26:43.436Z info: [Crawler][203] Successfully navigated to "https://post.smzdm.com/p/awopqn7g/". Waiting for the page to load ...
2025/01/14 10:26:38 | stdout | 2025-01-14T02:26:38.260Z info: [Crawler][203] Content-type for the url https://post.smzdm.com/p/awopqn7g/ is "text/html; charset=UTF-8"
2025/01/14 10:26:38 | stdout | 2025-01-14T02:26:38.170Z info: [search][206] Completed successfully
2025/01/14 10:26:38 | stdout | 2025-01-14T02:26:38.030Z info: [search][206] Attempting to index bookmark with id ua33wgcehsgv7wls1jk79l39 ...
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.997Z info: [search][204] Completed successfully
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.822Z info: [search][204] Attempting to index bookmark with id sw3gz67beg5aopukjn1saaic ...
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.808Z info: [Crawler][203] Attempting to determine the content-type for the url https://post.smzdm.com/p/awopqn7g/
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.807Z info: [Crawler][203] Will crawl "https://post.smzdm.com/p/awopqn7g/" for link with id "sw3gz67beg5aopukjn1saaic"
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.720Z info: Starting asset preprocessing worker ...
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.720Z info: Starting feed worker ...
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.719Z info: Starting video worker ...
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.719Z info: Starting tidy assets worker ...
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.718Z info: Starting search indexing worker ...
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.718Z info: Starting inference worker ...
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.717Z info: Starting crawler worker ...
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.576Z info: [Crawler] Successfully resolved IP address, new address: http://192.168.96.4:9222/
2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.575Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222
2025/01/14 10:26:09 | stderr | (Use `node --trace-deprecation ...` to show where the warning was created)
2025/01/14 10:26:09 | stderr | (node:69) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
2025/01/14 10:25:44 | stderr | (Use `node --trace-deprecation ...` to show where the warning was created)
2025/01/14 10:25:44 | stderr | (node:121) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
2025/01/14 10:25:44 | stdout | 2025-01-14T02:25:44.447Z info: [crawler] Loading adblocker ...
2025/01/14 10:25:44 | stdout | 2025-01-14T02:25:44.432Z info: Workers version: nightly
2025/01/14 10:25:39 | stdout |  
2025/01/14 10:25:39 | stdout | > tsx index.ts
2025/01/14 10:25:39 | stdout | > @hoarder/workers@0.1.0 start:prod /app/apps/workers
2025/01/14 10:25:39 | stdout |  
2025/01/14 10:25:39 | stdout | ✓ Ready in 709ms
2025/01/14 10:25:39 | stdout | ✓ Starting...
2025/01/14 10:25:39 | stdout |  
2025/01/14 10:25:39 | stdout | - Network:      http://0.0.0.0:3000
2025/01/14 10:25:39 | stdout | - Local:        http://localhost:3000
2025/01/14 10:25:39 | stdout | ▲ Next.js 14.2.21
2025/01/14 10:25:38 | stderr | s6-rc: info: service legacy-services successfully started
2025/01/14 10:25:38 | stderr | s6-rc: info: service legacy-services: starting
2025/01/14 10:25:38 | stderr | s6-rc: info: service svc-workers successfully started
2025/01/14 10:25:38 | stderr | s6-rc: info: service svc-web successfully started
2025/01/14 10:25:38 | stderr | s6-rc: info: service svc-web: starting
2025/01/14 10:25:38 | stderr | s6-rc: info: service svc-workers: starting
2025/01/14 10:25:38 | stderr | s6-rc: info: service init-db-migration successfully started
2025/01/14 10:25:38 | stderr | s6-rc: info: service legacy-cont-init successfully started
2025/01/14 10:25:38 | stderr | s6-rc: info: service legacy-cont-init: starting
2025/01/14 10:25:38 | stderr | s6-rc: info: service fix-attrs successfully started
2025/01/14 10:25:38 | stdout | Running db migration script
2025/01/14 10:25:38 | stderr | s6-rc: info: service init-db-migration: starting
2025/01/14 10:25:38 | stderr | s6-rc: info: service fix-attrs: starting
2025/01/14 10:25:38 | stderr | s6-rc: info: service s6rc-oneshot-runner successfully started
2025/01/14 10:25:38 | stderr | s6-rc: info: service s6rc-oneshot-runner: starting
2025/01/14 10:25:28 | stdout | 2025-01-14T02:25:28.672Z info: [Crawler] We're shutting down so won't retry.
2025/01/14 10:25:28 | stdout | at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:internal/dns/promises:86:17)
2025/01/14 10:25:28 | stdout | 2025-01-14T02:25:28.672Z error: [Crawler] Failed to connect to the browser instance, will retry in 5 secs: Error: getaddrinfo ENOTFOUND chrome
2025/01/14 10:25:28 | stdout | 2025-01-14T02:25:28.178Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222
2025/01/14 10:25:23 | stdout | 2025-01-14T02:25:23.204Z info: Shutting down crawler, openai, tidyAssets, video, feed, assetPreprocessing and search workers ...
2025/01/14 10:25:23 | stdout | 2025-01-14T02:25:23.204Z info: Received SIGTERM, shutting down ...
2025/01/14 10:25:23 | stderr | s6-rc: info: service svc-web successfully stopped
2025/01/14 10:25:23 | stderr | s6-rc: info: service fix-attrs successfully stopped
2025/01/14 10:25:23 | stderr | s6-rc: info: service fix-attrs: stopping
2025/01/14 10:25:23 | stderr | s6-rc: info: service legacy-cont-init successfully stopped

hoarder-chrome logs

2025/01/14 10:29:48 | stderr | [0114/022948.756462:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -202
2025/01/14 10:29:48 | stderr |  
2025/01/14 10:29:48 | stderr |  
2025/01/14 10:29:48 | stderr | ERROR: No matching issuer found
2025/01/14 10:29:48 | stderr | ----- Certificate i=1 (CN=Surge Generated CA A1C13D18,O=Surge Generated CA A1C13D18) -----
2025/01/14 10:29:48 | stderr | [0114/022948.755992:ERROR:cert_verify_proc_builtin.cc(878)] CertVerifyProcBuiltin for api.smzdm.com failed:
2025/01/14 10:29:45 | stderr | [0114/022945.061864:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
2025/01/14 10:29:36 | stderr | [0114/022936.344877:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -100
2025/01/14 10:29:36 | stderr | [0114/022936.335518:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -100
2025/01/14 10:29:35 | stderr | [0114/022935.828587:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
2025/01/14 10:28:11 | stderr | [0114/022811.610233:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig.
2025/01/14 10:28:11 | stderr | [0114/022811.545431:WARNING:sandbox_linux.cc(418)] InitializeSandbox() called with multiple threads in process gpu-process.
2025/01/14 10:28:11 | stderr | [0114/022811.520109:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable.
2025/01/14 10:28:11 | stderr | DevTools listening on ws://0.0.0.0:9222/devtools/browser/69681a36-29e2-494e-bf3c-ea520dd35361
2025/01/14 10:28:11 | stderr |  
2025/01/14 10:28:11 | stderr | [0114/022811.513500:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
2025/01/14 10:28:11 | stderr | [0114/022811.513466:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
2025/01/14 10:28:11 | stderr | [0114/022811.499503:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig.
2025/01/14 10:28:11 | stderr | [0114/022811.498824:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2025/01/14 10:28:11 | stderr | [0114/022811.498740:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2025/01/14 10:28:11 | stderr | [0114/022811.497433:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2025/01/14 10:27:12 | stderr | [0114/022712.546254:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -100
2025/01/14 10:27:12 | stderr | [0114/022712.537223:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -100
2025/01/14 10:27:11 | stderr | [0114/022711.556801:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
2025/01/14 10:26:41 | stderr | [0114/022641.972590:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -202
2025/01/14 10:26:41 | stderr |  
2025/01/14 10:26:41 | stderr |  
2025/01/14 10:26:41 | stderr | ERROR: No matching issuer found
2025/01/14 10:26:41 | stderr | ----- Certificate i=1 (CN=Surge Generated CA A1C13D18,O=Surge Generated CA A1C13D18) -----
2025/01/14 10:26:41 | stderr | [0114/022641.970387:ERROR:cert_verify_proc_builtin.cc(878)] CertVerifyProcBuiltin for api.smzdm.com failed:
2025/01/14 10:26:38 | stderr | [0114/022638.280699:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
2025/01/14 10:25:38 | stderr | [0114/022538.853559:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig.
2025/01/14 10:25:38 | stderr | [0114/022538.746891:WARNING:sandbox_linux.cc(418)] InitializeSandbox() called with multiple threads in process gpu-process.
2025/01/14 10:25:38 | stderr | DevTools listening on ws://0.0.0.0:9222/devtools/browser/fc7cb16d-c9e1-4211-981f-dd332ac689ff
2025/01/14 10:25:38 | stderr |  
2025/01/14 10:25:38 | stderr | [0114/022538.704116:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable.
2025/01/14 10:25:38 | stderr | [0114/022538.697959:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
2025/01/14 10:25:38 | stderr | [0114/022538.697813:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
2025/01/14 10:25:38 | stderr | [0114/022538.678026:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig.
2025/01/14 10:25:38 | stderr | [0114/022538.676951:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2025/01/14 10:25:38 | stderr | [0114/022538.676663:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2025/01/14 10:25:38 | stderr | [0114/022538.674718:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2025/01/14 10:24:42 | stderr | [0114/022442.342215:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
2025/01/14 10:24:42 | stderr | [0114/022442.342176:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed
2025/01/14 10:09:42 | stderr | [0114/020942.248779:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended
2025/01/14 10:09:42 | stderr | [0114/020942.248710:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed

hoarder-meilisearch logs

2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/130 HTTP/1.1" 200 339 "-" "node" 0.000280
-- | -- | --
2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  index_scheduler] A batch of tasks was successfully completed.
2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/130 HTTP/1.1" 200 303 "-" "node" 0.000281
2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/130 HTTP/1.1" 200 303 "-" "node" 0.000269
2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 8 }
2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/130 HTTP/1.1" 200 303 "-" "node" 0.000314
2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 138 "-" "node" 0.011559
2025/01/14 10:30:02 | stderr | [2025-01-14T02:30:02Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 340 "-" "node" 0.000315
2025/01/14 10:30:02 | stderr | [2025-01-14T02:30:02Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 274 "-" "node" 0.000317
2025/01/14 10:30:02 | stderr | [2025-01-14T02:30:02Z INFO  index_scheduler] A batch of tasks was successfully completed.
2025/01/14 10:30:02 | stderr | [2025-01-14T02:30:02Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000318
2025/01/14 10:30:02 | stderr | [2025-01-14T02:30:02Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000291
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 8 }
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000365
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000504
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000351
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000446
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000410
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000382
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000307
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000347
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000328
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000277
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000345
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000327
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000281
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000444
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000379
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000535
2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 138 "-" "node" 0.014785
2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/128 HTTP/1.1" 200 340 "-" "node" 0.000266
2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/128 HTTP/1.1" 200 274 "-" "node" 0.000267
2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  index_scheduler] A batch of tasks was successfully completed.
2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/128 HTTP/1.1" 200 304 "-" "node" 0.000269
2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/128 HTTP/1.1" 200 304 "-" "node" 0.000328
2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 8 }
2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/128 HTTP/1.1" 200 304 "-" "node" 0.000278
2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 138 "-" "node" 0.013246
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 340 "-" "node" 0.000311
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 274 "-" "node" 0.000274
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  index_scheduler] A batch of tasks was successfully completed.
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000277
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 8 }
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000278
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000387
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000564
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000371
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000530
2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 138 "-" "node" 0.015483
2025/01/14 10:29:39 | stderr | [2025-01-14T02:29:39Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/126 HTTP/1.1" 200 339 "-" "node" 0.000364
2025/01/14 10:29:39 | stderr | [2025-01-14T02:29:39Z INFO  index_scheduler] A batch of tasks was successfully completed.
2025/01/14 10:29:39 | stderr | [2025-01-14T02:29:39Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 8 }
2025/01/14 10:29:39 | stderr | [2025-01-14T02:29:39Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/126 HTTP/1.1" 200 303 "-" "node" 0.000641
2025/01/14 10:29:39 | stderr | [2025-01-14T02:29:39Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 137 "-" "node" 0.014483
2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/125 HTTP/1.1" 200 340 "-" "node" 0.000349
2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  index_scheduler] A batch of tasks was successfully completed.
2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/125 HTTP/1.1" 200 304 "-" "node" 0.000452
2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 7 }
2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/125 HTTP/1.1" 200 304 "-" "node" 0.000789
2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 138 "-" "node" 0.016186
2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /indexes/bookmarks/settings HTTP/1.1" 200 585 "-" "node" 0.000433
2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /indexes HTTP/1.1" 200 173 "-" "node" 0.001020
2025/01/14 10:28:11 | stderr | [2025-01-14T02:28:11Z INFO  actix_server::server] Actix runtime found; starting in Actix runtime
2025/01/14 10:28:11 | stderr | [2025-01-14T02:28:11Z INFO  actix_server::builder] starting 4 workers
2025/01/14 10:28:11 | stderr |  
2025/01/14 10:28:11 | stderr | Discord: https://discord.meilisearch.com
2025/01/14 10:28:11 | stderr | Source code: https://github.com/meilisearch/meilisearch
2025/01/14 10:28:11 | stderr | Documentation: https://www.meilisearch.com/docs
2025/01/14 10:28:11 | stderr | Check out Meilisearch Cloud! https://www.meilisearch.com/cloud?utm_campaign=oss&utm_source=engine&utm_medium=cli
2025/01/14 10:28:11 | stderr |  
2025/01/14 10:28:11 | stderr | A master key has been set. Requests to Meilisearch won't be authorized unless you provide an authentication key.
2025/01/14 10:28:11 | stderr |  
2025/01/14 10:28:11 | stderr | Anonymous telemetry: "Disabled"
2025/01/14 10:28:11 | stderr | Package version: "1.6.2"
2025/01/14 10:28:11 | stderr | Commit date: "2024-02-13T14:47:34+00:00"
2025/01/14 10:28:11 | stderr | Commit SHA: "1a083d54fc2840ac59530e5395397211cace35be"
2025/01/14 10:28:11 | stderr | Environment: "development"
2025/01/14 10:28:11 | stderr | Server listening on: "http://0.0.0.0:7700"
2025/01/14 10:28:11 | stderr | Database path: "./data.ms"
2025/01/14 10:28:11 | stderr | Config file path: "none"
2025/01/14 10:28:11 | stderr |  
2025/01/14 10:28:11 | stderr | 888       888  "Y8888  888 888 888  88888P'  "Y8888  "Y888888 888     "Y8888P 888  888
2025/01/14 10:28:11 | stderr | 888   "   888 Y8b.     888 888 888      X88 Y8b.     888  888 888    Y88b.    888  888
2025/01/14 10:28:11 | stderr | 888  Y8P  888 88888888 888 888 888 "Y8888b. 88888888 .d888888 888    888      888  888
2025/01/14 10:28:11 | stderr | 888 Y888P 888 d8P  Y8b 888 888 888 88K      d8P  Y8b     "88b 888P"  d88P"    888 "88b
2025/01/14 10:28:11 | stderr | 888Y88888P888  .d88b.  888 888 888 .d8888b   .d88b.   8888b.  888d888 .d8888b 88888b.
2025/01/14 10:28:11 | stderr | 88888b.d88888              888                                                888
2025/01/14 10:28:11 | stderr | 8888b   d8888          Y8P 888 Y8P                                            888
2025/01/14 10:28:11 | stderr | 888b     d888          d8b 888 d8b                                            888
2025/01/14 10:28:11 | stderr |  

对比
截图
照片丢失

Expected Behaviour

I hope you can help me solve this issue. If you need any information from me, please let me know directly. Thank you very much!

Screenshots or Additional Context

docker-compose.yml

version: "3.8"
services:
  web:
    image: ghcr.io/hoarder-app/hoarder:latest
    restart: unless-stopped
    volumes:
      - /volume1/docker/hoarder/data:/data
    ports:
      - 3004:3000
    env_file:
      - .env
    environment:
      MEILI_ADDR: http://meilisearch:7700 #  http://192.168.96.4:7700
      BROWSER_WEB_URL: http://chrome:9222  #http://192.168.96.3:9222
      # OPENAI_API_KEY: ...
      DATA_DIR: /data

  chrome:
    image: gcr.io/zenika-hub/alpine-chrome:123
    restart: unless-stopped
    command:
      - --no-sandbox
      - --disable-gpu
      - --disable-dev-shm-usage
      - --remote-debugging-address=0.0.0.0
      - --remote-debugging-port=9222
      - --hide-scrollbars
      - --proxy-server=http://ip:port
      - --proxy-bypass-list='*.local,127.0.0.1,10.10.10.0/24'

  meilisearch:
    image: getmeili/meilisearch:v1.6
    restart: unless-stopped
    env_file:
      - .env
    environment:
      MEILI_NO_ANALYTICS: "true"
    volumes:
      - /volume1/docker/hoarder/meilisearch:/meili_data

.env

NEXTAUTH_SECRET=***********
MEILI_MASTER_KEY=***********
NEXTAUTH_URL=http://localhost:3004
CRAWLER_DOWNLOAD_BANNER_IMAGE=true
CRAWLER_STORE_SCREENSHOT=true
CRAWLER_FULL_PAGE_SCREENSHOP=true
CRAWLER_ENABLE_ADBLOCKER=true
OPENAI_BASE_URL=https://api.deepseek.com/v1
OPENAI_API_KEY=****************
INFERENCE_LANG=chinese
INFERENCE_TEXT_MODEL=deepseek-chat

network environment

Public IP, using a Mac mini with Surge Mac as the gateway, with DHCP service enabled, and using Surge's HTTP proxy within Synology.

Device Details

Device：Synology1821+ DSM 7.2.1-69057 Update 5

Exact Hoarder Version

tag:latest 服务器版本 nightly(0.21.0 ⬆️)

Have you checked the troubleshooting guide?

  • I have checked the troubleshooting guide and I haven't found a solution to my problem
Originally created by @xzdbadboy on GitHub (Jan 14, 2025). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/879 ### Describe the Bug ### I have already set CRAWLER_DOWNLOAD_BANNER_IMAGE, CRAWLER_STORE_SCREENSHOT, and CRAWLER_FULL_PAGE_SCREENSHOT to true, but it seems like they are not working. ### During the crawling process, some pages' code and iamges was lost, and I have taken screenshots. ### The screenshots are incomplete; only one screen's worth was captured. ### Steps to Reproduce ### hoarder-web logs ``` 2025/01/14 10:30:04 | stdout | 2025-01-14T02:30:04.463Z info: [search][228] Completed successfully 2025/01/14 10:30:04 | stdout | 2025-01-14T02:30:04.281Z info: [search][228] Attempting to index bookmark with id bwj64p6bok5o0ylklkapgh0m ... 2025/01/14 10:30:04 | stdout | 2025-01-14T02:30:04.076Z info: [inference][225] Completed successfully 2025/01/14 10:30:03 | stdout | 2025-01-14T02:30:03.970Z info: [inference][225] Inferring tag for bookmark "bwj64p6bok5o0ylklkapgh0m" used 996 tokens and inferred: QNAP NAS,Docker,网络代理,技术教程,什么值得买 2025/01/14 10:30:02 | stdout | 2025-01-14T02:30:02.189Z info: [search][226] Completed successfully 2025/01/14 10:30:01 | stdout | 2025-01-14T02:30:01.115Z info: [VideoCrawler][227] Video Download Completed successfully 2025/01/14 10:30:01 | stdout | 2025-01-14T02:30:01.115Z info: [VideoCrawler][227] Skipping video download from "https://post.smzdm.com/p/awopqn7g/", because it is disabled in the config. 2025/01/14 10:30:01 | stdout | 2025-01-14T02:30:01.091Z info: [inference][225] Starting an inference job for bookmark with id "bwj64p6bok5o0ylklkapgh0m" 2025/01/14 10:30:01 | stdout | 2025-01-14T02:30:01.066Z info: [search][226] Attempting to index bookmark with id bwj64p6bok5o0ylklkapgh0m ... 2025/01/14 10:30:00 | stdout | 2025-01-14T02:30:00.675Z info: [Crawler][219] Completed successfully 2025/01/14 10:30:00 | stdout | 2025-01-14T02:30:00.580Z info: [Crawler][219] Downloaded image as assetId: d3503dd1-d802-4562-b06d-a225adbc9981 2025/01/14 10:29:58 | stdout | 2025-01-14T02:29:58.525Z info: [Crawler][219] Downloading image from "https://a.zdmimg.com/202407/05/6687a7416056e4638.png_fo742.jpg" 2025/01/14 10:29:58 | stdout | 2025-01-14T02:29:58.524Z info: [Crawler][219] Done extracting metadata from the page. 2025/01/14 10:29:55 | stdout | 2025-01-14T02:29:55.026Z info: [Crawler][219] Stored the screenshot as assetId: b082351a-a9a3-4c60-9bbe-e4fc85edcb38 2025/01/14 10:29:54 | stdout | 2025-01-14T02:29:54.994Z info: [Crawler][219] Done extracting readable content. 2025/01/14 10:29:52 | stdout | 2025-01-14T02:29:52.605Z info: [Crawler][219] Will attempt to extract readable content ... 2025/01/14 10:29:50 | stdout | 2025-01-14T02:29:50.706Z info: [Crawler][219] Will attempt to extract metadata from page ... 2025/01/14 10:29:50 | stdout | 2025-01-14T02:29:50.699Z info: [Crawler][219] Finished capturing page content and a screenshot. FullPageScreenshot: false 2025/01/14 10:29:50 | stdout | 2025-01-14T02:29:50.374Z info: [Crawler][219] Successfully fetched the page content. 2025/01/14 10:29:50 | stdout | 2025-01-14T02:29:50.304Z info: [Crawler][219] Finished waiting for the page to load. 2025/01/14 10:29:49 | stdout | 2025-01-14T02:29:49.254Z info: [Crawler][219] Successfully navigated to "https://post.smzdm.com/p/awopqn7g/". Waiting for the page to load ... 2025/01/14 10:29:47 | stdout | 2025-01-14T02:29:47.994Z info: [search][224] Completed successfully 2025/01/14 10:29:47 | stdout | 2025-01-14T02:29:47.758Z info: [search][224] Attempting to index bookmark with id atkzrc8zccyzz12ybpffs3z3 ... 2025/01/14 10:29:47 | stdout | 2025-01-14T02:29:47.256Z info: [inference][221] Completed successfully 2025/01/14 10:29:47 | stdout | 2025-01-14T02:29:47.163Z info: [inference][221] Inferring tag for bookmark "atkzrc8zccyzz12ybpffs3z3" used 2261 tokens and inferred: 人工智能,数据科学,本地RAG服务,Docker,Open WebUI,Ollama,Qwen2.5,文档检索,生成式AI 2025/01/14 10:29:45 | stdout | 2025-01-14T02:29:45.044Z info: [Crawler][219] Content-type for the url https://post.smzdm.com/p/awopqn7g/ is "text/html; charset=UTF-8" 2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.705Z info: [search][222] Completed successfully 2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.275Z info: [VideoCrawler][223] Video Download Completed successfully 2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.275Z info: [VideoCrawler][223] Skipping video download from "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/", because it is disabled in the config. 2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.239Z info: [inference][221] Starting an inference job for bookmark with id "atkzrc8zccyzz12ybpffs3z3" 2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.211Z info: [search][222] Attempting to index bookmark with id atkzrc8zccyzz12ybpffs3z3 ... 2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.088Z info: [Crawler][219] Attempting to determine the content-type for the url https://post.smzdm.com/p/awopqn7g/ 2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.088Z info: [Crawler][219] Will crawl "https://post.smzdm.com/p/awopqn7g/" for link with id "bwj64p6bok5o0ylklkapgh0m" 2025/01/14 10:29:44 | stdout | 2025-01-14T02:29:44.038Z info: [Crawler][217] Completed successfully 2025/01/14 10:29:43 | stdout | 2025-01-14T02:29:43.886Z info: [Crawler][217] Downloaded image as assetId: 3c11ee08-634a-412c-9ad3-a695dadd36e9 2025/01/14 10:29:42 | stdout | 2025-01-14T02:29:42.370Z info: [Crawler][217] Downloading image from "https://cloud.cuterwrite.fun/img/2024-06-29_116903285_p0_master1200.webp" 2025/01/14 10:29:42 | stdout | 2025-01-14T02:29:42.370Z info: [Crawler][217] Done extracting metadata from the page. 2025/01/14 10:29:42 | stdout | 2025-01-14T02:29:42.203Z info: [Crawler][217] Stored the screenshot as assetId: adb0dcfa-ab3b-4ed4-bc87-c4fcb6dfe2d7 2025/01/14 10:29:42 | stdout | 2025-01-14T02:29:42.128Z info: [Crawler][217] Done extracting readable content. 2025/01/14 10:29:41 | stdout | 2025-01-14T02:29:41.428Z info: [Crawler][217] Will attempt to extract readable content ... 2025/01/14 10:29:40 | stdout | 2025-01-14T02:29:40.642Z info: [Crawler][217] Will attempt to extract metadata from page ... 2025/01/14 10:29:40 | stdout | 2025-01-14T02:29:40.632Z info: [Crawler][217] Finished capturing page content and a screenshot. FullPageScreenshot: false 2025/01/14 10:29:40 | stdout | 2025-01-14T02:29:40.199Z info: [Crawler][217] Successfully fetched the page content. 2025/01/14 10:29:40 | stdout | 2025-01-14T02:29:40.189Z info: [Crawler][217] Finished waiting for the page to load. 2025/01/14 10:29:39 | stdout | 2025-01-14T02:29:39.677Z info: [search][220] Completed successfully 2025/01/14 10:29:39 | stdout | 2025-01-14T02:29:39.584Z info: [search][220] Attempting to index bookmark with id bwj64p6bok5o0ylklkapgh0m ... 2025/01/14 10:29:37 | stdout | 2025-01-14T02:29:37.513Z info: [Crawler][217] Successfully navigated to "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/". Waiting for the page to load ... 2025/01/14 10:29:35 | stdout | 2025-01-14T02:29:35.809Z info: [Crawler][217] Content-type for the url https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/ is "text/html; charset=utf-8" 2025/01/14 10:29:34 | stdout | 2025-01-14T02:29:34.538Z info: [search][218] Completed successfully 2025/01/14 10:29:34 | stdout | 2025-01-14T02:29:34.370Z info: [search][218] Attempting to index bookmark with id atkzrc8zccyzz12ybpffs3z3 ... 2025/01/14 10:29:34 | stdout | 2025-01-14T02:29:34.244Z info: [Crawler][217] Attempting to determine the content-type for the url https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/ 2025/01/14 10:29:34 | stdout | 2025-01-14T02:29:34.244Z info: [Crawler][217] Will crawl "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/" for link with id "atkzrc8zccyzz12ybpffs3z3" 2025/01/14 10:29:28 | stderr | (Use `node --trace-deprecation ...` to show where the warning was created) 2025/01/14 10:29:28 | stderr | (node:69) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. 2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.100Z info: Starting asset preprocessing worker ... 2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.100Z info: Starting feed worker ... 2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.099Z info: Starting video worker ... 2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.099Z info: Starting tidy assets worker ... 2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.098Z info: Starting search indexing worker ... 2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.098Z info: Starting inference worker ... 2025/01/14 10:28:27 | stdout | 2025-01-14T02:28:27.097Z info: Starting crawler worker ... 2025/01/14 10:28:26 | stdout | 2025-01-14T02:28:26.946Z info: [Crawler] Successfully resolved IP address, new address: http://192.168.96.4:9222/ 2025/01/14 10:28:26 | stdout | 2025-01-14T02:28:26.938Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222 2025/01/14 10:28:26 | stderr | (Use `node --trace-deprecation ...` to show where the warning was created) 2025/01/14 10:28:26 | stderr | (node:121) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. 2025/01/14 10:28:26 | stdout | 2025-01-14T02:28:26.835Z info: [crawler] Loading adblocker ... 2025/01/14 10:28:26 | stdout | 2025-01-14T02:28:26.820Z info: Workers version: nightly 2025/01/14 10:28:22 | stdout |   2025/01/14 10:28:22 | stdout | > tsx index.ts 2025/01/14 10:28:22 | stdout | > @hoarder/workers@0.1.0 start:prod /app/apps/workers 2025/01/14 10:28:22 | stdout |   2025/01/14 10:28:22 | stdout | ✓ Ready in 662ms 2025/01/14 10:28:21 | stdout | ✓ Starting... 2025/01/14 10:28:21 | stdout |   2025/01/14 10:28:21 | stdout | - Network:      http://0.0.0.0:3000 2025/01/14 10:28:21 | stdout | - Local:        http://localhost:3000 2025/01/14 10:28:21 | stdout | ▲ Next.js 14.2.21 2025/01/14 10:28:21 | stderr | s6-rc: info: service legacy-services successfully started 2025/01/14 10:28:21 | stderr | s6-rc: info: service legacy-services: starting 2025/01/14 10:28:21 | stderr | s6-rc: info: service svc-web successfully started 2025/01/14 10:28:21 | stderr | s6-rc: info: service svc-workers successfully started 2025/01/14 10:28:21 | stderr | s6-rc: info: service svc-web: starting 2025/01/14 10:28:21 | stderr | s6-rc: info: service svc-workers: starting 2025/01/14 10:28:21 | stderr | s6-rc: info: service init-db-migration successfully started 2025/01/14 10:28:21 | stderr | s6-rc: info: service legacy-cont-init successfully started 2025/01/14 10:28:21 | stderr | s6-rc: info: service legacy-cont-init: starting 2025/01/14 10:28:21 | stderr | s6-rc: info: service fix-attrs successfully started 2025/01/14 10:28:21 | stdout | Running db migration script 2025/01/14 10:28:21 | stderr | s6-rc: info: service init-db-migration: starting 2025/01/14 10:28:21 | stderr | s6-rc: info: service fix-attrs: starting 2025/01/14 10:28:21 | stderr | s6-rc: info: service s6rc-oneshot-runner successfully started 2025/01/14 10:28:21 | stderr | s6-rc: info: service s6rc-oneshot-runner: starting 2025/01/14 10:28:15 | stdout | 2025-01-14T02:28:15.112Z info: [Crawler] Successfully resolved IP address, new address: http://192.168.96.4:9222/ 2025/01/14 10:28:15 | stdout | 2025-01-14T02:28:15.112Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222 2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.131Z info: Shutting down crawler, openai, tidyAssets, video, feed, assetPreprocessing and search workers ... 2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.131Z info: Received SIGTERM, shutting down ... 2025/01/14 10:28:10 | stderr | s6-rc: info: service svc-web successfully stopped 2025/01/14 10:28:10 | stdout | at async /app/apps/workers/crawlerWorker.ts:2:4664 2025/01/14 10:28:10 | stdout | at async PuppeteerExtra.connect (/app/apps/workers/node_modules/.pnpm/puppeteer-extra@3.3.6_puppeteer@22.3.0_typescript@5.3.3_/node_modules/puppeteer-extra/dist/index.cjs.js:151:25) 2025/01/14 10:28:10 | stdout | at async _connectToBrowser (/app/apps/workers/node_modules/.pnpm/puppeteer-core@22.3.0/node_modules/puppeteer-core/lib/cjs/puppeteer/common/BrowserConnector.js:50:50) 2025/01/14 10:28:10 | stdout | at async getConnectionTransport (/app/apps/workers/node_modules/.pnpm/puppeteer-core@22.3.0/node_modules/puppeteer-core/lib/cjs/puppeteer/common/BrowserConnector.js:81:31) 2025/01/14 10:28:10 | stdout | at async getWSEndpoint (/app/apps/workers/node_modules/.pnpm/puppeteer-core@22.3.0/node_modules/puppeteer-core/lib/cjs/puppeteer/common/BrowserConnector.js:94:24) 2025/01/14 10:28:10 | stdout | at process.processTicksAndRejections (node:internal/process/task_queues:105:5) 2025/01/14 10:28:10 | stdout | at node:internal/deps/undici/undici:13484:13 2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.108Z error: [Crawler] Failed to connect to the browser instance, will retry in 5 secs: TypeError: Failed to fetch browser webSocket URL from http://192.168.96.4:9222/json/version: fetch failed 2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.100Z info: [Crawler] Successfully resolved IP address, new address: http://192.168.96.4:9222/ 2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.097Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222 2025/01/14 10:28:10 | stderr | s6-rc: info: service fix-attrs successfully stopped 2025/01/14 10:28:10 | stdout | 2025-01-14T02:28:10.095Z info: [Crawler] The puppeteer browser got disconnected. Will attempt to launch it again. 2025/01/14 10:28:10 | stderr | s6-rc: info: service fix-attrs: stopping 2025/01/14 10:28:10 | stderr | s6-rc: info: service legacy-cont-init successfully stopped 2025/01/14 10:28:10 | stderr | s6-rc: info: service svc-web: stopping 2025/01/14 10:28:10 | stderr | s6-rc: info: service svc-workers: stopping 2025/01/14 10:28:10 | stderr | s6-rc: info: service legacy-cont-init: stopping 2025/01/14 10:28:10 | stderr | s6-rc: info: service legacy-services successfully stopped 2025/01/14 10:28:10 | stderr | s6-rc: info: service legacy-services: stopping 2025/01/14 10:28:06 | stdout | 2025-01-14T02:28:06.001Z info: [search][216] Completed successfully 2025/01/14 10:28:05 | stdout | 2025-01-14T02:28:05.874Z info: [search][216] Attempting to index bookmark with id ua33wgcehsgv7wls1jk79l39 ... 2025/01/14 10:28:04 | stderr | } 2025/01/14 10:28:04 | stderr | [cause]: undefined 2025/01/14 10:28:04 | stderr | code: 'NOT_FOUND', 2025/01/14 10:28:04 | stderr | at async Promise.all (index 1) { 2025/01/14 10:28:04 | stderr | at async /app/apps/web/.next/server/app/api/trpc/[trpc]/route.js:1:4379 2025/01/14 10:28:04 | stderr | at async t (/app/apps/web/.next/server/chunks/480.js:4:46248) 2025/01/14 10:28:04 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909) 2025/01/14 10:28:04 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909) 2025/01/14 10:28:04 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909) 2025/01/14 10:28:04 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909) 2025/01/14 10:28:04 | stderr | at async t (/app/apps/web/.next/server/chunks/480.js:4:45282) 2025/01/14 10:28:04 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909) 2025/01/14 10:28:04 | stderr | at /app/apps/web/.next/server/chunks/6815.js:1:26219 2025/01/14 10:28:04 | stderr | p [TRPCError]: Bookmark not found 2025/01/14 10:28:03 | stdout | 2025-01-14T02:28:03.831Z info: [search][215] Completed successfully 2025/01/14 10:28:03 | stdout | 2025-01-14T02:28:03.672Z info: [search][215] Attempting to index bookmark with id sw3gz67beg5aopukjn1saaic ... 2025/01/14 10:28:02 | stderr | } 2025/01/14 10:28:02 | stderr | [cause]: undefined 2025/01/14 10:28:02 | stderr | code: 'NOT_FOUND', 2025/01/14 10:28:02 | stderr | at async Promise.all (index 1) { 2025/01/14 10:28:02 | stderr | at async /app/apps/web/.next/server/app/api/trpc/[trpc]/route.js:1:4379 2025/01/14 10:28:02 | stderr | at async t (/app/apps/web/.next/server/chunks/480.js:4:46248) 2025/01/14 10:28:02 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909) 2025/01/14 10:28:02 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909) 2025/01/14 10:28:02 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909) 2025/01/14 10:28:02 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909) 2025/01/14 10:28:02 | stderr | at async t (/app/apps/web/.next/server/chunks/480.js:4:45282) 2025/01/14 10:28:02 | stderr | at async a (/app/apps/web/.next/server/chunks/480.js:4:45909) 2025/01/14 10:28:02 | stderr | at /app/apps/web/.next/server/chunks/6815.js:1:26219 2025/01/14 10:28:02 | stderr | p [TRPCError]: Bookmark not found 2025/01/14 10:27:28 | stdout | 2025-01-14T02:27:28.567Z info: [search][214] Completed successfully 2025/01/14 10:27:28 | stdout | 2025-01-14T02:27:28.432Z info: [search][214] Attempting to index bookmark with id ua33wgcehsgv7wls1jk79l39 ... 2025/01/14 10:27:27 | stdout | 2025-01-14T02:27:27.897Z info: [inference][211] Completed successfully 2025/01/14 10:27:27 | stdout | 2025-01-14T02:27:27.852Z info: [inference][211] Inferring tag for bookmark "ua33wgcehsgv7wls1jk79l39" used 2261 tokens and inferred: 人工智能,数据科学,本地RAG服务,Open WebUI,Ollama,Qwen2.5,Docker,文档检索,生成式AI 2025/01/14 10:27:22 | stdout | 2025-01-14T02:27:22.379Z info: [search][212] Completed successfully 2025/01/14 10:27:22 | stdout | 2025-01-14T02:27:22.241Z info: [search][212] Attempting to index bookmark with id ua33wgcehsgv7wls1jk79l39 ... 2025/01/14 10:27:22 | stdout | 2025-01-14T02:27:22.216Z info: [inference][211] Starting an inference job for bookmark with id "ua33wgcehsgv7wls1jk79l39" 2025/01/14 10:27:22 | stdout | 2025-01-14T02:27:22.179Z info: [VideoCrawler][213] Video Download Completed successfully 2025/01/14 10:27:22 | stdout | 2025-01-14T02:27:22.179Z info: [VideoCrawler][213] Skipping video download from "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/", because it is disabled in the config. 2025/01/14 10:27:21 | stdout | 2025-01-14T02:27:21.922Z info: [Crawler][205] Completed successfully 2025/01/14 10:27:21 | stdout | 2025-01-14T02:27:21.833Z info: [Crawler][205] Downloaded image as assetId: 0df8f5c5-252f-4f0b-8f2f-4f199c5ff0d3 2025/01/14 10:27:20 | stdout | 2025-01-14T02:27:20.344Z info: [Crawler][205] Downloading image from "https://cloud.cuterwrite.fun/img/2024-06-29_116903285_p0_master1200.webp" 2025/01/14 10:27:20 | stdout | 2025-01-14T02:27:20.344Z info: [Crawler][205] Done extracting metadata from the page. 2025/01/14 10:27:18 | stdout | 2025-01-14T02:27:18.154Z info: [Crawler][205] Stored the screenshot as assetId: d066ed42-e62c-4690-b0f3-5ae3b92beea9 2025/01/14 10:27:18 | stdout | 2025-01-14T02:27:18.130Z info: [Crawler][205] Done extracting readable content. 2025/01/14 10:27:17 | stdout | 2025-01-14T02:27:17.645Z info: [Crawler][205] Will attempt to extract readable content ... 2025/01/14 10:27:16 | stdout | 2025-01-14T02:27:16.965Z info: [Crawler][205] Will attempt to extract metadata from page ... 2025/01/14 10:27:16 | stdout | 2025-01-14T02:27:16.958Z info: [Crawler][205] Finished capturing page content and a screenshot. FullPageScreenshot: false 2025/01/14 10:27:16 | stdout | 2025-01-14T02:27:16.531Z info: [Crawler][205] Successfully fetched the page content. 2025/01/14 10:27:16 | stdout | 2025-01-14T02:27:16.522Z info: [Crawler][205] Finished waiting for the page to load. 2025/01/14 10:27:13 | stdout | 2025-01-14T02:27:13.809Z info: [Crawler][205] Successfully navigated to "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/". Waiting for the page to load ... 2025/01/14 10:27:13 | stdout | 2025-01-14T02:27:13.587Z info: [search][210] Completed successfully 2025/01/14 10:27:13 | stdout | 2025-01-14T02:27:13.431Z info: [search][210] Attempting to index bookmark with id sw3gz67beg5aopukjn1saaic ... 2025/01/14 10:27:13 | stdout | 2025-01-14T02:27:13.367Z info: [inference][207] Completed successfully 2025/01/14 10:27:13 | stdout | 2025-01-14T02:27:13.307Z info: [inference][207] Inferring tag for bookmark "sw3gz67beg5aopukjn1saaic" used 996 tokens and inferred: QNAP NAS,Docker,网络代理,技术教程,什么值得买 2025/01/14 10:27:11 | stdout | 2025-01-14T02:27:11.540Z info: [Crawler][205] Content-type for the url https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/ is "text/html; charset=utf-8" 2025/01/14 10:27:11 | stdout | 2025-01-14T02:27:11.314Z info: [search][208] Completed successfully 2025/01/14 10:27:10 | stdout | 2025-01-14T02:27:10.265Z info: [VideoCrawler][209] Video Download Completed successfully 2025/01/14 10:27:10 | stdout | 2025-01-14T02:27:10.265Z info: [VideoCrawler][209] Skipping video download from "https://post.smzdm.com/p/awopqn7g/", because it is disabled in the config. 2025/01/14 10:27:10 | stdout | 2025-01-14T02:27:10.227Z info: [inference][207] Starting an inference job for bookmark with id "sw3gz67beg5aopukjn1saaic" 2025/01/14 10:27:10 | stdout | 2025-01-14T02:27:10.197Z info: [search][208] Attempting to index bookmark with id sw3gz67beg5aopukjn1saaic ... 2025/01/14 10:27:09 | stdout | 2025-01-14T02:27:09.821Z info: [Crawler][205] Attempting to determine the content-type for the url https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/ 2025/01/14 10:27:09 | stdout | 2025-01-14T02:27:09.821Z info: [Crawler][205] Will crawl "https://cuterwrite.top/p/integrate-open-webui-ollama-qwen25-local-rag/" for link with id "ua33wgcehsgv7wls1jk79l39" 2025/01/14 10:27:09 | stdout | 2025-01-14T02:27:09.772Z info: [Crawler][203] Completed successfully 2025/01/14 10:27:09 | stdout | 2025-01-14T02:27:09.637Z info: [Crawler][203] Downloaded image as assetId: 6f0b7a19-2453-4a23-a633-860d17c1f5bc 2025/01/14 10:26:50 | stdout | 2025-01-14T02:26:50.670Z info: [Crawler][203] Downloading image from "https://a.zdmimg.com/202407/05/6687a7416056e4638.png_fo742.jpg" 2025/01/14 10:26:50 | stdout | 2025-01-14T02:26:50.670Z info: [Crawler][203] Done extracting metadata from the page. 2025/01/14 10:26:50 | stdout | 2025-01-14T02:26:50.131Z info: [Crawler][203] Stored the screenshot as assetId: 6e53e387-aec4-440b-94ff-b2a27fa8a21b 2025/01/14 10:26:50 | stdout | 2025-01-14T02:26:50.055Z info: [Crawler][203] Done extracting readable content. 2025/01/14 10:26:47 | stdout | 2025-01-14T02:26:47.410Z info: [Crawler][203] Will attempt to extract readable content ... 2025/01/14 10:26:44 | stdout | 2025-01-14T02:26:44.890Z info: [Crawler][203] Will attempt to extract metadata from page ... 2025/01/14 10:26:44 | stdout | 2025-01-14T02:26:44.884Z info: [Crawler][203] Finished capturing page content and a screenshot. FullPageScreenshot: false 2025/01/14 10:26:44 | stdout | 2025-01-14T02:26:44.560Z info: [Crawler][203] Successfully fetched the page content. 2025/01/14 10:26:44 | stdout | 2025-01-14T02:26:44.493Z info: [Crawler][203] Finished waiting for the page to load. 2025/01/14 10:26:43 | stdout | 2025-01-14T02:26:43.436Z info: [Crawler][203] Successfully navigated to "https://post.smzdm.com/p/awopqn7g/". Waiting for the page to load ... 2025/01/14 10:26:38 | stdout | 2025-01-14T02:26:38.260Z info: [Crawler][203] Content-type for the url https://post.smzdm.com/p/awopqn7g/ is "text/html; charset=UTF-8" 2025/01/14 10:26:38 | stdout | 2025-01-14T02:26:38.170Z info: [search][206] Completed successfully 2025/01/14 10:26:38 | stdout | 2025-01-14T02:26:38.030Z info: [search][206] Attempting to index bookmark with id ua33wgcehsgv7wls1jk79l39 ... 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.997Z info: [search][204] Completed successfully 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.822Z info: [search][204] Attempting to index bookmark with id sw3gz67beg5aopukjn1saaic ... 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.808Z info: [Crawler][203] Attempting to determine the content-type for the url https://post.smzdm.com/p/awopqn7g/ 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.807Z info: [Crawler][203] Will crawl "https://post.smzdm.com/p/awopqn7g/" for link with id "sw3gz67beg5aopukjn1saaic" 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.720Z info: Starting asset preprocessing worker ... 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.720Z info: Starting feed worker ... 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.719Z info: Starting video worker ... 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.719Z info: Starting tidy assets worker ... 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.718Z info: Starting search indexing worker ... 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.718Z info: Starting inference worker ... 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.717Z info: Starting crawler worker ... 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.576Z info: [Crawler] Successfully resolved IP address, new address: http://192.168.96.4:9222/ 2025/01/14 10:26:37 | stdout | 2025-01-14T02:26:37.575Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222 2025/01/14 10:26:09 | stderr | (Use `node --trace-deprecation ...` to show where the warning was created) 2025/01/14 10:26:09 | stderr | (node:69) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. 2025/01/14 10:25:44 | stderr | (Use `node --trace-deprecation ...` to show where the warning was created) 2025/01/14 10:25:44 | stderr | (node:121) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. 2025/01/14 10:25:44 | stdout | 2025-01-14T02:25:44.447Z info: [crawler] Loading adblocker ... 2025/01/14 10:25:44 | stdout | 2025-01-14T02:25:44.432Z info: Workers version: nightly 2025/01/14 10:25:39 | stdout |   2025/01/14 10:25:39 | stdout | > tsx index.ts 2025/01/14 10:25:39 | stdout | > @hoarder/workers@0.1.0 start:prod /app/apps/workers 2025/01/14 10:25:39 | stdout |   2025/01/14 10:25:39 | stdout | ✓ Ready in 709ms 2025/01/14 10:25:39 | stdout | ✓ Starting... 2025/01/14 10:25:39 | stdout |   2025/01/14 10:25:39 | stdout | - Network:      http://0.0.0.0:3000 2025/01/14 10:25:39 | stdout | - Local:        http://localhost:3000 2025/01/14 10:25:39 | stdout | ▲ Next.js 14.2.21 2025/01/14 10:25:38 | stderr | s6-rc: info: service legacy-services successfully started 2025/01/14 10:25:38 | stderr | s6-rc: info: service legacy-services: starting 2025/01/14 10:25:38 | stderr | s6-rc: info: service svc-workers successfully started 2025/01/14 10:25:38 | stderr | s6-rc: info: service svc-web successfully started 2025/01/14 10:25:38 | stderr | s6-rc: info: service svc-web: starting 2025/01/14 10:25:38 | stderr | s6-rc: info: service svc-workers: starting 2025/01/14 10:25:38 | stderr | s6-rc: info: service init-db-migration successfully started 2025/01/14 10:25:38 | stderr | s6-rc: info: service legacy-cont-init successfully started 2025/01/14 10:25:38 | stderr | s6-rc: info: service legacy-cont-init: starting 2025/01/14 10:25:38 | stderr | s6-rc: info: service fix-attrs successfully started 2025/01/14 10:25:38 | stdout | Running db migration script 2025/01/14 10:25:38 | stderr | s6-rc: info: service init-db-migration: starting 2025/01/14 10:25:38 | stderr | s6-rc: info: service fix-attrs: starting 2025/01/14 10:25:38 | stderr | s6-rc: info: service s6rc-oneshot-runner successfully started 2025/01/14 10:25:38 | stderr | s6-rc: info: service s6rc-oneshot-runner: starting 2025/01/14 10:25:28 | stdout | 2025-01-14T02:25:28.672Z info: [Crawler] We're shutting down so won't retry. 2025/01/14 10:25:28 | stdout | at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:internal/dns/promises:86:17) 2025/01/14 10:25:28 | stdout | 2025-01-14T02:25:28.672Z error: [Crawler] Failed to connect to the browser instance, will retry in 5 secs: Error: getaddrinfo ENOTFOUND chrome 2025/01/14 10:25:28 | stdout | 2025-01-14T02:25:28.178Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222 2025/01/14 10:25:23 | stdout | 2025-01-14T02:25:23.204Z info: Shutting down crawler, openai, tidyAssets, video, feed, assetPreprocessing and search workers ... 2025/01/14 10:25:23 | stdout | 2025-01-14T02:25:23.204Z info: Received SIGTERM, shutting down ... 2025/01/14 10:25:23 | stderr | s6-rc: info: service svc-web successfully stopped 2025/01/14 10:25:23 | stderr | s6-rc: info: service fix-attrs successfully stopped 2025/01/14 10:25:23 | stderr | s6-rc: info: service fix-attrs: stopping 2025/01/14 10:25:23 | stderr | s6-rc: info: service legacy-cont-init successfully stopped ``` ### hoarder-chrome logs ``` 2025/01/14 10:29:48 | stderr | [0114/022948.756462:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -202 2025/01/14 10:29:48 | stderr |   2025/01/14 10:29:48 | stderr |   2025/01/14 10:29:48 | stderr | ERROR: No matching issuer found 2025/01/14 10:29:48 | stderr | ----- Certificate i=1 (CN=Surge Generated CA A1C13D18,O=Surge Generated CA A1C13D18) ----- 2025/01/14 10:29:48 | stderr | [0114/022948.755992:ERROR:cert_verify_proc_builtin.cc(878)] CertVerifyProcBuiltin for api.smzdm.com failed: 2025/01/14 10:29:45 | stderr | [0114/022945.061864:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. 2025/01/14 10:29:36 | stderr | [0114/022936.344877:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -100 2025/01/14 10:29:36 | stderr | [0114/022936.335518:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -100 2025/01/14 10:29:35 | stderr | [0114/022935.828587:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. 2025/01/14 10:28:11 | stderr | [0114/022811.610233:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig. 2025/01/14 10:28:11 | stderr | [0114/022811.545431:WARNING:sandbox_linux.cc(418)] InitializeSandbox() called with multiple threads in process gpu-process. 2025/01/14 10:28:11 | stderr | [0114/022811.520109:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable. 2025/01/14 10:28:11 | stderr | DevTools listening on ws://0.0.0.0:9222/devtools/browser/69681a36-29e2-494e-bf3c-ea520dd35361 2025/01/14 10:28:11 | stderr |   2025/01/14 10:28:11 | stderr | [0114/022811.513500:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended 2025/01/14 10:28:11 | stderr | [0114/022811.513466:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed 2025/01/14 10:28:11 | stderr | [0114/022811.499503:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig. 2025/01/14 10:28:11 | stderr | [0114/022811.498824:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory 2025/01/14 10:28:11 | stderr | [0114/022811.498740:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory 2025/01/14 10:28:11 | stderr | [0114/022811.497433:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory 2025/01/14 10:27:12 | stderr | [0114/022712.546254:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -100 2025/01/14 10:27:12 | stderr | [0114/022712.537223:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -100 2025/01/14 10:27:11 | stderr | [0114/022711.556801:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. 2025/01/14 10:26:41 | stderr | [0114/022641.972590:ERROR:ssl_client_socket_impl.cc(970)] handshake failed; returned -1, SSL error code 1, net_error -202 2025/01/14 10:26:41 | stderr |   2025/01/14 10:26:41 | stderr |   2025/01/14 10:26:41 | stderr | ERROR: No matching issuer found 2025/01/14 10:26:41 | stderr | ----- Certificate i=1 (CN=Surge Generated CA A1C13D18,O=Surge Generated CA A1C13D18) ----- 2025/01/14 10:26:41 | stderr | [0114/022641.970387:ERROR:cert_verify_proc_builtin.cc(878)] CertVerifyProcBuiltin for api.smzdm.com failed: 2025/01/14 10:26:38 | stderr | [0114/022638.280699:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition. 2025/01/14 10:25:38 | stderr | [0114/022538.853559:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig. 2025/01/14 10:25:38 | stderr | [0114/022538.746891:WARNING:sandbox_linux.cc(418)] InitializeSandbox() called with multiple threads in process gpu-process. 2025/01/14 10:25:38 | stderr | DevTools listening on ws://0.0.0.0:9222/devtools/browser/fc7cb16d-c9e1-4211-981f-dd332ac689ff 2025/01/14 10:25:38 | stderr |   2025/01/14 10:25:38 | stderr | [0114/022538.704116:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable. 2025/01/14 10:25:38 | stderr | [0114/022538.697959:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended 2025/01/14 10:25:38 | stderr | [0114/022538.697813:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed 2025/01/14 10:25:38 | stderr | [0114/022538.678026:WARNING:dns_config_service_linux.cc(427)] Failed to read DnsConfig. 2025/01/14 10:25:38 | stderr | [0114/022538.676951:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory 2025/01/14 10:25:38 | stderr | [0114/022538.676663:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory 2025/01/14 10:25:38 | stderr | [0114/022538.674718:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory 2025/01/14 10:24:42 | stderr | [0114/022442.342215:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended 2025/01/14 10:24:42 | stderr | [0114/022442.342176:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed 2025/01/14 10:09:42 | stderr | [0114/020942.248779:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping recommended platform policies because no policy file was found at: /etc/chromium/policies/recommended 2025/01/14 10:09:42 | stderr | [0114/020942.248710:INFO:policy_logger.cc(145)] :components/policy/core/common/config_dir_policy_loader.cc(118) Skipping mandatory platform policies because no policy file was found at: /etc/chromium/policies/managed ``` ### hoarder-meilisearch logs ``` 2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/130 HTTP/1.1" 200 339 "-" "node" 0.000280 -- | -- | -- 2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  index_scheduler] A batch of tasks was successfully completed. 2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/130 HTTP/1.1" 200 303 "-" "node" 0.000281 2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/130 HTTP/1.1" 200 303 "-" "node" 0.000269 2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 8 } 2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/130 HTTP/1.1" 200 303 "-" "node" 0.000314 2025/01/14 10:30:04 | stderr | [2025-01-14T02:30:04Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 138 "-" "node" 0.011559 2025/01/14 10:30:02 | stderr | [2025-01-14T02:30:02Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 340 "-" "node" 0.000315 2025/01/14 10:30:02 | stderr | [2025-01-14T02:30:02Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 274 "-" "node" 0.000317 2025/01/14 10:30:02 | stderr | [2025-01-14T02:30:02Z INFO  index_scheduler] A batch of tasks was successfully completed. 2025/01/14 10:30:02 | stderr | [2025-01-14T02:30:02Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000318 2025/01/14 10:30:02 | stderr | [2025-01-14T02:30:02Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000291 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 8 } 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000365 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000504 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000351 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000446 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000410 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000382 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000307 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000347 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000328 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000277 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000345 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000327 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000281 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000444 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000379 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/129 HTTP/1.1" 200 304 "-" "node" 0.000535 2025/01/14 10:30:01 | stderr | [2025-01-14T02:30:01Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 138 "-" "node" 0.014785 2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/128 HTTP/1.1" 200 340 "-" "node" 0.000266 2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/128 HTTP/1.1" 200 274 "-" "node" 0.000267 2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  index_scheduler] A batch of tasks was successfully completed. 2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/128 HTTP/1.1" 200 304 "-" "node" 0.000269 2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/128 HTTP/1.1" 200 304 "-" "node" 0.000328 2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 8 } 2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/128 HTTP/1.1" 200 304 "-" "node" 0.000278 2025/01/14 10:29:47 | stderr | [2025-01-14T02:29:47Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 138 "-" "node" 0.013246 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 340 "-" "node" 0.000311 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 274 "-" "node" 0.000274 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  index_scheduler] A batch of tasks was successfully completed. 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000277 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 8 } 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000278 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000387 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000564 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000371 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/127 HTTP/1.1" 200 304 "-" "node" 0.000530 2025/01/14 10:29:44 | stderr | [2025-01-14T02:29:44Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 138 "-" "node" 0.015483 2025/01/14 10:29:39 | stderr | [2025-01-14T02:29:39Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/126 HTTP/1.1" 200 339 "-" "node" 0.000364 2025/01/14 10:29:39 | stderr | [2025-01-14T02:29:39Z INFO  index_scheduler] A batch of tasks was successfully completed. 2025/01/14 10:29:39 | stderr | [2025-01-14T02:29:39Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 8 } 2025/01/14 10:29:39 | stderr | [2025-01-14T02:29:39Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/126 HTTP/1.1" 200 303 "-" "node" 0.000641 2025/01/14 10:29:39 | stderr | [2025-01-14T02:29:39Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 137 "-" "node" 0.014483 2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/125 HTTP/1.1" 200 340 "-" "node" 0.000349 2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  index_scheduler] A batch of tasks was successfully completed. 2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/125 HTTP/1.1" 200 304 "-" "node" 0.000452 2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 7 } 2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /tasks/125 HTTP/1.1" 200 304 "-" "node" 0.000789 2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/bookmarks/documents?primaryKey=id HTTP/1.1" 202 138 "-" "node" 0.016186 2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /indexes/bookmarks/settings HTTP/1.1" 200 585 "-" "node" 0.000433 2025/01/14 10:29:34 | stderr | [2025-01-14T02:29:34Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /indexes HTTP/1.1" 200 173 "-" "node" 0.001020 2025/01/14 10:28:11 | stderr | [2025-01-14T02:28:11Z INFO  actix_server::server] Actix runtime found; starting in Actix runtime 2025/01/14 10:28:11 | stderr | [2025-01-14T02:28:11Z INFO  actix_server::builder] starting 4 workers 2025/01/14 10:28:11 | stderr |   2025/01/14 10:28:11 | stderr | Discord: https://discord.meilisearch.com 2025/01/14 10:28:11 | stderr | Source code: https://github.com/meilisearch/meilisearch 2025/01/14 10:28:11 | stderr | Documentation: https://www.meilisearch.com/docs 2025/01/14 10:28:11 | stderr | Check out Meilisearch Cloud! https://www.meilisearch.com/cloud?utm_campaign=oss&utm_source=engine&utm_medium=cli 2025/01/14 10:28:11 | stderr |   2025/01/14 10:28:11 | stderr | A master key has been set. Requests to Meilisearch won't be authorized unless you provide an authentication key. 2025/01/14 10:28:11 | stderr |   2025/01/14 10:28:11 | stderr | Anonymous telemetry: "Disabled" 2025/01/14 10:28:11 | stderr | Package version: "1.6.2" 2025/01/14 10:28:11 | stderr | Commit date: "2024-02-13T14:47:34+00:00" 2025/01/14 10:28:11 | stderr | Commit SHA: "1a083d54fc2840ac59530e5395397211cace35be" 2025/01/14 10:28:11 | stderr | Environment: "development" 2025/01/14 10:28:11 | stderr | Server listening on: "http://0.0.0.0:7700" 2025/01/14 10:28:11 | stderr | Database path: "./data.ms" 2025/01/14 10:28:11 | stderr | Config file path: "none" 2025/01/14 10:28:11 | stderr |   2025/01/14 10:28:11 | stderr | 888       888  "Y8888  888 888 888  88888P'  "Y8888  "Y888888 888     "Y8888P 888  888 2025/01/14 10:28:11 | stderr | 888   "   888 Y8b.     888 888 888      X88 Y8b.     888  888 888    Y88b.    888  888 2025/01/14 10:28:11 | stderr | 888  Y8P  888 88888888 888 888 888 "Y8888b. 88888888 .d888888 888    888      888  888 2025/01/14 10:28:11 | stderr | 888 Y888P 888 d8P  Y8b 888 888 888 88K      d8P  Y8b     "88b 888P"  d88P"    888 "88b 2025/01/14 10:28:11 | stderr | 888Y88888P888  .d88b.  888 888 888 .d8888b   .d88b.   8888b.  888d888 .d8888b 88888b. 2025/01/14 10:28:11 | stderr | 88888b.d88888              888                                                888 2025/01/14 10:28:11 | stderr | 8888b   d8888          Y8P 888 Y8P                                            888 2025/01/14 10:28:11 | stderr | 888b     d888          d8b 888 d8b                                            888 2025/01/14 10:28:11 | stderr |   ``` ![对比](https://github.com/user-attachments/assets/86edbd0f-8d65-4474-a12d-c936278d52a0) ![截图](https://github.com/user-attachments/assets/04adb51a-3f6c-4c3c-9da6-679fe3b6dcf8) ![照片丢失](https://github.com/user-attachments/assets/fcd14c1b-8312-4a2f-ba0d-91e788c3fa6f) ### Expected Behaviour I hope you can help me solve this issue. If you need any information from me, please let me know directly. Thank you very much! ### Screenshots or Additional Context ### docker-compose.yml ``` version: "3.8" services: web: image: ghcr.io/hoarder-app/hoarder:latest restart: unless-stopped volumes: - /volume1/docker/hoarder/data:/data ports: - 3004:3000 env_file: - .env environment: MEILI_ADDR: http://meilisearch:7700 # http://192.168.96.4:7700 BROWSER_WEB_URL: http://chrome:9222 #http://192.168.96.3:9222 # OPENAI_API_KEY: ... DATA_DIR: /data chrome: image: gcr.io/zenika-hub/alpine-chrome:123 restart: unless-stopped command: - --no-sandbox - --disable-gpu - --disable-dev-shm-usage - --remote-debugging-address=0.0.0.0 - --remote-debugging-port=9222 - --hide-scrollbars - --proxy-server=http://ip:port - --proxy-bypass-list='*.local,127.0.0.1,10.10.10.0/24' meilisearch: image: getmeili/meilisearch:v1.6 restart: unless-stopped env_file: - .env environment: MEILI_NO_ANALYTICS: "true" volumes: - /volume1/docker/hoarder/meilisearch:/meili_data ``` ### .env ``` NEXTAUTH_SECRET=*********** MEILI_MASTER_KEY=*********** NEXTAUTH_URL=http://localhost:3004 CRAWLER_DOWNLOAD_BANNER_IMAGE=true CRAWLER_STORE_SCREENSHOT=true CRAWLER_FULL_PAGE_SCREENSHOP=true CRAWLER_ENABLE_ADBLOCKER=true OPENAI_BASE_URL=https://api.deepseek.com/v1 OPENAI_API_KEY=**************** INFERENCE_LANG=chinese INFERENCE_TEXT_MODEL=deepseek-chat ```` ### network environment Public IP, using a Mac mini with Surge Mac as the gateway, with DHCP service enabled, and using Surge's HTTP proxy within Synology. ### Device Details Device：Synology1821+ DSM 7.2.1-69057 Update 5 ### Exact Hoarder Version tag:latest 服务器版本 nightly(0.21.0 ⬆️) ### Have you checked the troubleshooting guide? - [X] I have checked the troubleshooting guide and I haven't found a solution to my problem
kerem 2026-03-02 11:50:58 +03:00
  • closed this issue
  • added the
    question
         label
kerem commented 2026-03-02 11:50:59 +03:00
Author
Owner
Copy link

@kamtschatka commented on GitHub (Jan 14, 2025):

the .env file says "CRAWLER_FULL_PAGE_SCREENSHOP" (note: the P at the end, should obviously be a T)

<!-- gh-comment-id:2589809305 --> @kamtschatka commented on GitHub (Jan 14, 2025): the .env file says "CRAWLER_FULL_PAGE_SCREENSHOP" (note: the P at the end, should obviously be a T)
kerem commented 2026-03-02 11:50:59 +03:00
Author
Owner
Copy link

@yellowbikewithbluehandles commented on GitHub (Jan 30, 2025):

Hello! Sorry new user for docker container manager here but how did you get .env files into container manager? where do you actually put .env files into? Its confusing me because for other docker compose.yaml I directly put the .env environment variables into the yaml.

<!-- gh-comment-id:2623821656 --> @yellowbikewithbluehandles commented on GitHub (Jan 30, 2025): Hello! Sorry new user for docker container manager here but how did you get .env files into container manager? where do you actually put .env files into? Its confusing me because for other docker compose.yaml I directly put the .env environment variables into the yaml.
kerem commented 2026-03-02 11:50:59 +03:00
Author
Owner
Copy link

@ocean-haiyang commented on GitHub (Feb 15, 2025):

Hello! Sorry new user for docker container manager here but how did you get .env files into container manager? where do you actually put .env files into? Its confusing me because for other docker compose.yaml I directly put the .env environment variables into the yaml.

do you mean in portainer?

<!-- gh-comment-id:2661111226 --> @ocean-haiyang commented on GitHub (Feb 15, 2025): > Hello! Sorry new user for docker container manager here but how did you get .env files into container manager? where do you actually put .env files into? Its confusing me because for other docker compose.yaml I directly put the .env environment variables into the yaml. do you mean in portainer?
kerem commented 2026-03-02 11:50:59 +03:00
Author
Owner
Copy link

@MohamedBassem commented on GitHub (Feb 16, 2025):

Closing this as the original issue seems to have been rootcaused

<!-- gh-comment-id:2661633251 --> @MohamedBassem commented on GitHub (Feb 16, 2025): Closing this as the original issue seems to have been rootcaused
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
refactor/use-npm-singlefile
onetab
claude/issue-2596-20260321-1401
claude/fix-docs-button-responsive-V3aBQ
claude/review-import-backpressure-D4ArJ
claude/fix-archived-bookmarks-mobile-P9OJW
claude/issue-1189-20260211-1601
claude/fix-nested-smart-lists-3uFkt
claude/issue-2298-20251223-1704
feat/import-v3
claude/add-cli-search-subcommand-6kIe0
claude/add-bookmark-indexing-timestamps-96bPj
claude/auto-disable-failing-feeds-fkDhP
claude/add-tag-search-aliases-HzESD
feat/docker-compose-dev
claude/add-attachedby-tags-endpoint-01WYfemMGHJJjXsPYLvUJAno
claude/fix-crawler-memory-leaks-NE7Ct
bookmark-debugger
claude/issue-2352-20260106-1120
claude/issue-1977-20260102-2348
claude/add-banner-rendering-JeLUk
claude/add-descendant-qualifier-cUm26
claude/skip-metadata-refresh-archives-CAo4Y
claude/fix-archive-pending-banner-pAyGM
claude/add-embeddings-support-h2swV
claude/nested-manage-lists-QVV85
claude/privacy-type-system-MG1bT
claude/add-action-menu-icons-6hNKw
claude/issue-2299-20251223-1711
claude/bookmark-indexing-progress-QwZSI
claude/migrate-bookmark-attachments-3O2te
claude/add-2025-wrapped-feature-tIUIh
claude/improve-ai-settings-design-639tq
claude/add-youtube-metascraper-plugin-0lWC7
claude/add-problem-reporting-gSSEV
claude/add-mobile-list-menus-spcS7
claude/shadcn-bookmark-cards-WWHzP
claude/add-extensions-link-HTeXc
claude/add-onboarding-screens-hsYMO
claude/fix-settings-switch-overflow-nlzM4
claude/clamp-bookmark-titles-diAEz
claude/port-stats-mobile-expo-MuXAn
claude/whats-new-base-version-vrv8C
claude/fix-settings-auth-checks-jgyD8
claude/add-server-version-display-3sGa2
claude/fix-tag-editor-scrolling-rzdbG
claude/add-company-pricing-card-y5mHY
claude/audit-optimize-transactions-xpDVc
codex/ensure-consistent-ui-experience-across-app-pages
claude/plan-opentelemetry-integration-01Jx183mz1Ev8h8JoYj97Auw
libsql
db-indicies
claude/export-import-lists-01UuCWwdaqduAd35NppvjnMD
claude/configurable-worker-timeout-0198GQh6YrrRzqG62xnogyrz
claude/check-import-quota-01CPdxTpHp18Ba62bYcBTVbA
claude/scraper-worker-thread-01FEHen6MGrQHmdBstJSuiyA
claude/customize-dialog-styling-01CVjEv2KgyZJSpCg3mqkvR7
claude/add-asset-cache-headers-0175WhNcqwiwurrmjj52jnLT
claude/add-db-search-plugin-017Xxd4Jq3MfjWT788vgfbaq
benchmarks-2
claude/add-filtered-deletion-01DTxWNcg3hhqdNpeNLa3s6L
claude/actionbutton-loading-spinner-015DY5ZTvgPgFAXTZz3UGaYv
claude/add-broken-links-qualifier-01S31X1LsKiYb9gE1dXTKvi3
claude/docker-release-tag-trigger-01UmzFXEumhK2jdmRGtMcueo
claude/spread-feed-fetch-scheduling-01EihUtmZSyqeE1HfRMessxW
restate-idempotency
claude/align-android-ios-colors-01GJfkhEyZVBReohVioPa8ok
claude/improve-mobile-app-colors-0155LzHfkd5HyJr6YyZMsus5
codex/add-autocomplete-for-search-query-language
claude/add-bookmark-backups-016L2A8Z94n7tDgDdMPdFuAd
claude/restrict-binary-user-permissions-01FSGyy2RXGZvE26YbAejzGi
effect-ts
claude/prepare-trpc-npm-publish-0193EjfwpxSNVNcLXqXjs6Ln
shared-list-sidebar
claude/lazy-load-tiktoken-017UTNpJPTcMMQvNEBa1aFwo
codex/fix-asset-pre-processing-worker-abort-signals
add-groupid
claude/add-bookmark-list-button-01VF7uXYNLsVDzqdozWMXP5M
claude/extract-shared-ui-components-01DSVfaCr6WRqAyx1vJTZk9r
claude/migrate-shadcn-sidebar-01DKjpg9MD5PJ2potemSnbvW
claude/add-collaborators-rate-limits-01VjXyRWWPUkGQKa8d8D8qKj
claude/modernize-dark-mode-01FRfE81PAY5C44pFu1cYocf
claude/add-signed-url-bookmark-01PjYT1ZhvLK2FPJNTAhJsWf
restate-group-id
claude/add-highlights-page-012vhHpn8fVNp3gf7gBeW14s
claude/disable-shared-bookmark-features-01B9fiGUdu6NyWaxSQFsQBxP
claude/mobile-bookmark-grid-layouts-018cGBBMhPJVq6PJVRBpqT2r
claude/add-mobile-bookmark-summary-01494LYoh4sJW5Fj4GPm62Vj
claude/add-mobile-tags-screen-01WRADt4ZzvXVew1Y9vqF8SV
claude/add-highlight-notes-01LpanRLS4a2YMnT1qB5GTqX
claude/add-search-bar-014k2ngaqjwYRVSvqmbuECqr
claude/hide-collaborator-emails-01TQrkkMupC7CR9BTuDkireg
claude/list-invitation-approval-0129V89M1riXW6JqmoF74VfM
claude/add-bookmark-archive-sort-018VbGPGvtmsGgXFEERoAX7B
claude/add-mobile-smart-lists-01251tYo9u1SywE6XFezAv9e
claude/bookmark-drag-drop-01DmWq286ogHpDGHKcXjKr3z
claude/add-rss-import-01DH1Q2axcDeq8nQJR5MWjPJ
claude/mobile-inapp-browser-auth-01KiT6bwyntRPQ1X4oTtAveC
claude/offline-mode-react-query-01D1rE2bdBEPw2teGqunr5Gd
claude/add-singlefile-extension-support-01BEB9QQZABzwfZDvR9Bz5b2
claude/custom-list-slugs-01VxcfkNUXZ97FNpNVURopMq
claude/issue-2148-20251118-1133
claude/add-groupid-queue-fairness-011CV1r8Wb46HuGAg5o95i3m
claude/hide-viewer-shared-lists-01Fst6NBvdxrXXnDhUmjsNDP
claude/collaborative-lists-013AvDvMqkoszDVcSoCYgBcM
claude/implement-feature-01LT5XzGsbEhZkYXNEjEwdui
claude/fix-bookmark-loading-state-01AgF4H2drxwuTCJDB2Xgiu4
claude/admin-user-edit-013tbiRmb1KX2fhSYqmGKCu8
claude/expose-all-api-01YTruEW72WQYMtq4iZoaPkA
claude/add-doc-link-main-016NYLxShpKuH6R8XCBgeZtc
claude/fix-issue-2133-019JLvdSRAUbU4FtjQztcM6S
claude/explore-effect-ts-integration-01F7xb1dWwP1ma4LnLbFGfDD
claude/optimize-dockerfile-build-011CV5gDnPZbdbbVSPDofC4e
claude/add-custom-headers-guide-011CV249t16aWDRb1mCrzQdC
claude/mobile-app-signup-011CUxPtCXgU6U3T8GShTR2Q
claude/crawler-worker-fetch-browser-011CUvcRc24XEr9DTWDW6MX8
claude/fix-issue-784-011CUvubQrcZHG9S3KjpCKbK
codex/add-user-settings-for-inference-language-and-screenshots
claude/fix-mobile-signin-server-address-011CUnaUWwY2Fhq5Xbwhgr8H
better-auth-2
claude/issue-2028-20251012-1429
claude/issue-1010-20251012-1154
codex/update-feed-refresh-job-idempotency-key
restate
import-v2
fix-public-lists
recurse-delete-list
abort-dangling-processing
tag-pagination
ratelimit-plugin
claude/issue-1937-20250914-0912
codex/implement-title-search-query-qualifier
copilot/add-edit-button-for-notes
cookie-path
ai-tag-cleanup
codex/add-allowlist-and-blocklist-env-variables
mobile-retheme
expo-next-upgrade
opencode/issue1788-20250727215611
fix-trailing-slash-deduplication
edit-bookmark-dialog
bookmark-embeddings
rag
nextjs-15
bookmark-hover-bar
sapling-pr-archive-MohamedBassem
track-bookmark-assets
json-cli
admin-settings
mobile-dark-mode
android/v1.9.2-0
ios/v1.9.1-1
android/v1.9.1-0
ios/v1.9.1-0
ios/v1.9.0-2
ios/v1.9.0-1
android/v1.9.0-1
extension/v1.2.9
cli/v0.31.0
sdk/v0.31.0
mcp/v0.31.0
android/v1.9.0-0
ios/v1.9.0-0
v0.31.0
android/v1.8.5-0
cli/v0.30.0
sdk/v0.30.0
ios/v1.8.4-0
android/v1.8.4-0
v0.30.0
cli/v0.29.1
v0.29.3
v0.29.2
v0.29.1
sdk/v0.29.0
cli/v0.29.0
mcp/v0.29.0
ios/v1.8.3-0
android/v1.8.3-0
extension/v1.2.8
v0.29.0
android/v1.8.2-2
android/v1.8.2-1
ios/v1.8.2-0
android/v1.8.2-0
extension/v1.2.7
android/v1.8.1-0
ios/v1.8.1-0
v0.28.0
cli/v0.27.1
cli/v0.27.0
v0.27.1
sdk/v0.27.0
v0.27.0
android/v1.8.0-1
ios/v1.8.0-1
mcp/v0.26.0
sdk/v0.26.0
v0.26.0
cli/v0.25.0
ios/v1.7.0-1
mcp/v0.25.0
v0.25.0
extension/v1.2.6
ios/v1.7.0-0
android/v1.7.0-0
v0.24.1
v0.24.0
mcp/v0.23.10
mcp/v0.23.9
mcp/v0.23.8
extension/v1.2.5
mcp/v0.23.7
mcp/v0.23.6
mcp/v0.23.5
mcp/v0.23.4
sdk/v0.23.2
cli/v0.23.0
extension/v1.2.4
android/v1.6.9-1
ios/v1.6.9-1
v0.23.2
v0.23.1
sdk/v0.23.0
v0.23.0
ios/v1.6.9-0
sdk/v0.22.0
v0.22.0
android/v1.6.8-0
ios/v1.6.8-0
sdk/v0.21.2
sdk/v0.21.1
sdk/v0.21.0
v0.21.0
cli/v0.20.0
v0.20.0
ios/v1.6.7-4
android/v1.6.7-4
ios/v1.6.7-3
android/v1.6.7-3
android/v1.6.7-2
ios/v1.6.7-2
android/v1.6.7-1
ios/v1.6.7-1
ios/v1.6.7-0
android/v1.6.7-0
v0.19.0
android/v1.6.6-0
android/v1.6.5-0
ios/v1.6.5-0
ios/v1.6.4-0
android/v1.6.4-0
v0.18.0
v0.17.1
v0.17.0
ios/v1.6.3-0
android/v1.6.3-0
extension/v1.2.3
ios/v1.6.2-1
android/v1.6.2-1
ios/v1.6.2-0
android/v1.6.2-0
v0.16.0
ios/v1.6.1-3
android/v1.6.1-3
ios/v1.6.1-2
android/v1.6.1-2
android/v1.6.1-1
ios/v1.6.1-1
android/v1.6.1-0
ios/v1.6.1-0
extension/v1.2.2
android/v1.6.0-1
ios/v1.6.0-1
ios/v1.6.0
android/v1.6.0
cli/v0.13.7
cli/v0.13.6
v0.15.0
cli/v0.13.5
extension/v1.2.1
v0.14.0
cli/v0.13.3
cli/v0.13.2
cli/v0.13.1
cli/v0.13.0
v0.13.1
v0.13.0
mobile-v1.5.0
mobile-v1.4.0
v0.12.2
v0.12.1
v0.12.0
v0.11.1
v0.11.0
v0.10.1
v0.10.0
v0.9.0
v0.8.0
v0.7.0
v0.6.0
v0.5.0
v0.4.1
v.0.4.0
v.0.3.1
v0.3.0
v0.2.2
v0.2.1
v0.2.0
v0.1.0
Labels
Clear labels   
UI/UX

   
android

   
bug

   
dependencies

   
documentation

   
documentation

   
extension

   
feature request

   
feature request

   
good first issue

   
ios

   
long-term

   
performance

   
pri/high

   
pri/low

   
pri/medium

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
status/approved

   
status/icebox

   
status/pending_clarification

   
status/untriaged
No labels
UI/UX
android
bug
dependencies
documentation
documentation
extension
feature request
feature request
good first issue
ios
long-term
performance
pri/high
pri/low
pri/medium
pull-request
question
status/approved
status/icebox
status/pending_clarification
status/untriaged
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/karakeep#573
No description provided.