starred/karakeep
1
0
Fork 0
mirror of https://github.com/karakeep-app/karakeep.git synced 2026-04-25 07:56:05 +03:00
Code Issues 569 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #537] URL seems to get hoarder stuck in a crawl loop on v0.18.0 #347

New issue
Open
opened 2026-03-02 11:49:04 +03:00 by kerem · 8 comments
kerem commented 2026-03-02 11:49:04 +03:00
Owner
Copy link

Originally created by @dimatx on GitHub (Oct 14, 2024).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/537

I seem to have a URL that gets hoarder stuck in a loop where it tries to crawl, then recrawls, etc. It only stops when I delete the bookmark.

Please let me know if you need any more info than what I provided.

2024-10-14T16:50:49.322Z info: [search][890] Attempting to index bookmark with id q10606sx8ev4xhstqbjw5gaq ...
2024-10-14T16:50:49.340Z info: [inference][889] Starting an inference job for bookmark with id "q10606sx8ev4xhstqbjw5gaq"
2024-10-14T16:50:49.511Z info: [search][890] Completed successfully
2024-10-14T16:50:50.949Z info: [inference][889] Inferring tag for bookmark "q10606sx8ev4xhstqbjw5gaq" used 2936 tokens and inferred: LineageOS,Lenovo ThinkSmart View,Home Automation,Android Installation,Open Source
2024-10-14T16:50:51.001Z info: [inference][889] Completed successfully
2024-10-14T16:50:51.556Z info: [search][891] Attempting to index bookmark with id q10606sx8ev4xhstqbjw5gaq ...
2024-10-14T16:50:51.673Z info: [search][891] Completed successfully
2024-10-14T16:54:23.815Z info: [Crawler][892] Will crawl "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/" for link with id "k1l8zj5ixpgj9hugbvmibqfc"
2024-10-14T16:54:23.815Z info: [Crawler][892] Attempting to determine the content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/
2024-10-14T16:54:23.882Z info: [Crawler][892] Content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ is "text/html; charset=UTF-8"
2024-10-14T16:54:23.907Z info: [search][893] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ...
2024-10-14T16:54:23.975Z info: [search][893] Completed successfully
2024-10-14T16:54:26.944Z info: [Crawler][892] Successfully navigated to "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/". Waiting for the page to load ...
2024-10-14T16:54:29.602Z info: [Crawler][892] Finished waiting for the page to load.
2024-10-14T16:54:29.833Z info: [Crawler][892] Finished capturing page content and a screenshot. FullPageScreenshot: false
2024-10-14T16:54:29.842Z info: [Crawler][892] Will attempt to extract metadata from page ...
2024-10-14T16:54:30.699Z info: [Crawler][892] Will attempt to extract readable content ...
2024-10-14T16:54:31.384Z info: [Crawler][892] Done extracting readable content.
2024-10-14T16:54:31.396Z info: [Crawler][892] Stored the screenshot as assetId: 249e0a78-90b8-4e26-b051-17730c928aae
2024-10-14T16:54:31.443Z info: [Crawler][892] Done extracting metadata from the page.
2024-10-14T16:54:31.443Z info: [Crawler][892] Downloading image from "https://odsonfinance.com/wp-content/uploads/2024/01/How-to-do-a-Backdoor-Roth-IRA-1.png"
2024-10-14T16:54:31.553Z info: [Crawler][892] Downloaded image as assetId: c6acee98-fa08-4076-8f7c-8e05becff000
2024-10-14T16:54:31.612Z info: [Crawler][892] Completed successfully
2024-10-14T16:54:32.419Z info: [search][895] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ...
2024-10-14T16:54:32.437Z info: [inference][894] Starting an inference job for bookmark with id "k1l8zj5ixpgj9hugbvmibqfc"
2024-10-14T16:54:32.554Z info: [search][895] Completed successfully
2024-10-14T16:54:33.930Z info: [inference][894] Inferring tag for bookmark "k1l8zj5ixpgj9hugbvmibqfc" used 2122 tokens and inferred: Roth IRA,Backdoor Roth,Fidelity,Personal Finance,Investing
2024-10-14T16:54:33.971Z info: [inference][894] Completed successfully
2024-10-14T16:54:34.587Z info: [search][896] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ...
2024-10-14T16:54:34.652Z info: [search][896] Completed successfully
2024-10-14T16:55:03.684Z info: [Crawler][897] Will crawl "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/" for link with id "k1l8zj5ixpgj9hugbvmibqfc"
2024-10-14T16:55:03.684Z info: [Crawler][897] Attempting to determine the content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/
2024-10-14T16:55:03.757Z info: [Crawler][897] Content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ is "text/html; charset=UTF-8"
2024-10-14T16:55:07.264Z info: [Crawler][897] Successfully navigated to "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/". Waiting for the page to load ...
2024-10-14T16:55:11.589Z info: [Crawler][897] Finished waiting for the page to load.
2024-10-14T16:55:11.803Z info: [Crawler][897] Finished capturing page content and a screenshot. FullPageScreenshot: false
2024-10-14T16:55:11.810Z info: [Crawler][897] Will attempt to extract metadata from page ...
2024-10-14T16:55:12.468Z info: [Crawler][897] Will attempt to extract readable content ...
2024-10-14T16:55:13.130Z info: [Crawler][897] Done extracting readable content.
2024-10-14T16:55:13.141Z info: [Crawler][897] Stored the screenshot as assetId: 9a16e763-4619-46d3-8e9e-281e2280acec
2024-10-14T16:55:13.181Z info: [Crawler][897] Done extracting metadata from the page.
2024-10-14T16:55:13.182Z info: [Crawler][897] Downloading image from "https://odsonfinance.com/wp-content/uploads/2024/01/How-to-do-a-Backdoor-Roth-IRA-1.png"
2024-10-14T16:55:13.289Z info: [Crawler][897] Downloaded image as assetId: 76a7a90b-ccab-462e-9f06-9afa6799cf93
2024-10-14T16:55:13.381Z info: [Crawler][897] Will attempt to archive page ...
2024-10-14T16:55:14.169Z info: [inference][898] Starting an inference job for bookmark with id "k1l8zj5ixpgj9hugbvmibqfc"
2024-10-14T16:55:14.186Z info: [search][899] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ...
2024-10-14T16:55:14.249Z info: [search][899] Completed successfully
2024-10-14T16:55:16.005Z info: [inference][898] Inferring tag for bookmark "k1l8zj5ixpgj9hugbvmibqfc" used 2122 tokens and inferred: Roth IRA,Backdoor Roth,Fidelity,Investing,Personal Finance
2024-10-14T16:55:16.041Z info: [inference][898] Completed successfully
2024-10-14T16:55:16.287Z info: [search][900] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ...
2024-10-14T16:55:16.348Z info: [search][900] Completed successfully
2024-10-14T16:56:03.719Z info: [Crawler][897] Will crawl "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/" for link with id "k1l8zj5ixpgj9hugbvmibqfc"
2024-10-14T16:56:03.719Z info: [Crawler][897] Attempting to determine the content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/
2024-10-14T16:56:03.771Z info: [Crawler][897] Content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ is "text/html; charset=UTF-8"
2024-10-14T16:56:06.978Z info: [Crawler][897] Successfully navigated to "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/". Waiting for the page to load ...
2024-10-14T16:56:09.540Z info: [Crawler][897] Finished waiting for the page to load.
2024-10-14T16:56:09.737Z info: [Crawler][897] Finished capturing page content and a screenshot. FullPageScreenshot: false
2024-10-14T16:56:09.747Z info: [Crawler][897] Will attempt to extract metadata from page ...
2024-10-14T16:56:10.481Z info: [Crawler][897] Will attempt to extract readable content ...
2024-10-14T16:56:11.025Z info: [Crawler][897] Done extracting readable content.
2024-10-14T16:56:11.038Z info: [Crawler][897] Stored the screenshot as assetId: 72382d5e-3a19-4382-83d2-1e1cec207c1e
2024-10-14T16:56:11.086Z info: [Crawler][897] Done extracting metadata from the page.
2024-10-14T16:56:11.086Z info: [Crawler][897] Downloading image from "https://odsonfinance.com/wp-content/uploads/2024/01/How-to-do-a-Backdoor-Roth-IRA-1.png"
2024-10-14T16:56:11.215Z info: [Crawler][897] Downloaded image as assetId: 6d054521-87bd-4e25-9b9a-f12c81784706
2024-10-14T16:56:11.312Z info: [Crawler][897] Will attempt to archive page ...
2024-10-14T16:56:12.066Z info: [inference][901] Starting an inference job for bookmark with id "k1l8zj5ixpgj9hugbvmibqfc"
2024-10-14T16:56:12.082Z info: [search][902] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ...
2024-10-14T16:56:12.146Z info: [search][902] Completed successfully
2024-10-14T16:56:15.205Z info: [inference][901] Inferring tag for bookmark "k1l8zj5ixpgj9hugbvmibqfc" used 2123 tokens and inferred: Backdoor Roth IRA,Fidelity,Retirement Planning,Personal Finance,Investing
2024-10-14T16:56:15.258Z info: [inference][901] Completed successfully
2024-10-14T16:56:16.187Z info: [search][903] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ...
2024-10-14T16:56:16.301Z info: [search][903] Completed successfully
2024-10-14T16:57:03.761Z info: [Crawler][897] Will crawl "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/" for link with id "k1l8zj5ixpgj9hugbvmibqfc"
2024-10-14T16:57:03.761Z info: [Crawler][897] Attempting to determine the content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/
2024-10-14T16:57:03.813Z info: [Crawler][897] Content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ is "text/html; charset=UTF-8"
2024-10-14T16:57:08.863Z info: [Crawler][897] Successfully navigated to "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/". Waiting for the page to load ...
2024-10-14T16:57:11.373Z info: [Crawler][897] Finished waiting for the page to load.
2024-10-14T16:57:11.577Z info: [Crawler][897] Finished capturing page content and a screenshot. FullPageScreenshot: false
2024-10-14T16:57:11.586Z info: [Crawler][897] Will attempt to extract metadata from page ...
2024-10-14T16:57:12.127Z info: [Crawler][897] Will attempt to extract readable content ...
2024-10-14T16:57:12.678Z info: [Crawler][897] Done extracting readable content.
2024-10-14T16:57:12.690Z info: [Crawler][897] Stored the screenshot as assetId: c9b9223a-8cfc-4061-aab7-362035ec162e
2024-10-14T16:57:12.733Z info: [Crawler][897] Done extracting metadata from the page.
2024-10-14T16:57:12.733Z info: [Crawler][897] Downloading image from "https://odsonfinance.com/wp-content/uploads/2024/01/How-to-do-a-Backdoor-Roth-IRA-1.png"
2024-10-14T16:57:12.850Z info: [Crawler][897] Downloaded image as assetId: 88122b54-c814-439b-96bf-f265809f2cbe
2024-10-14T16:57:12.945Z info: [Crawler][897] Will attempt to archive page ...
2024-10-14T16:57:13.718Z info: [inference][904] Starting an inference job for bookmark with id "k1l8zj5ixpgj9hugbvmibqfc"
2024-10-14T16:57:13.735Z info: [search][905] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ...
2024-10-14T16:57:13.803Z info: [search][905] Completed successfully
2024-10-14T16:57:15.744Z info: [inference][904] Inferring tag for bookmark "k1l8zj5ixpgj9hugbvmibqfc" used 2124 tokens and inferred: Backdoor Roth IRA,Fidelity,Personal Finance,Retirement Planning,Investing Strategies
2024-10-14T16:57:15.780Z info: [inference][904] Completed successfully
2024-10-14T16:57:15.830Z info: [search][906] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ...
2024-10-14T16:57:15.945Z info: [search][906] Completed successfully
Originally created by @dimatx on GitHub (Oct 14, 2024). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/537 I seem to have a URL that gets hoarder stuck in a loop where it tries to crawl, then recrawls, etc. It only stops when I delete the bookmark. Please let me know if you need any more info than what I provided. ``` 2024-10-14T16:50:49.322Z info: [search][890] Attempting to index bookmark with id q10606sx8ev4xhstqbjw5gaq ... 2024-10-14T16:50:49.340Z info: [inference][889] Starting an inference job for bookmark with id "q10606sx8ev4xhstqbjw5gaq" 2024-10-14T16:50:49.511Z info: [search][890] Completed successfully 2024-10-14T16:50:50.949Z info: [inference][889] Inferring tag for bookmark "q10606sx8ev4xhstqbjw5gaq" used 2936 tokens and inferred: LineageOS,Lenovo ThinkSmart View,Home Automation,Android Installation,Open Source 2024-10-14T16:50:51.001Z info: [inference][889] Completed successfully 2024-10-14T16:50:51.556Z info: [search][891] Attempting to index bookmark with id q10606sx8ev4xhstqbjw5gaq ... 2024-10-14T16:50:51.673Z info: [search][891] Completed successfully 2024-10-14T16:54:23.815Z info: [Crawler][892] Will crawl "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/" for link with id "k1l8zj5ixpgj9hugbvmibqfc" 2024-10-14T16:54:23.815Z info: [Crawler][892] Attempting to determine the content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ 2024-10-14T16:54:23.882Z info: [Crawler][892] Content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ is "text/html; charset=UTF-8" 2024-10-14T16:54:23.907Z info: [search][893] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ... 2024-10-14T16:54:23.975Z info: [search][893] Completed successfully 2024-10-14T16:54:26.944Z info: [Crawler][892] Successfully navigated to "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/". Waiting for the page to load ... 2024-10-14T16:54:29.602Z info: [Crawler][892] Finished waiting for the page to load. 2024-10-14T16:54:29.833Z info: [Crawler][892] Finished capturing page content and a screenshot. FullPageScreenshot: false 2024-10-14T16:54:29.842Z info: [Crawler][892] Will attempt to extract metadata from page ... 2024-10-14T16:54:30.699Z info: [Crawler][892] Will attempt to extract readable content ... 2024-10-14T16:54:31.384Z info: [Crawler][892] Done extracting readable content. 2024-10-14T16:54:31.396Z info: [Crawler][892] Stored the screenshot as assetId: 249e0a78-90b8-4e26-b051-17730c928aae 2024-10-14T16:54:31.443Z info: [Crawler][892] Done extracting metadata from the page. 2024-10-14T16:54:31.443Z info: [Crawler][892] Downloading image from "https://odsonfinance.com/wp-content/uploads/2024/01/How-to-do-a-Backdoor-Roth-IRA-1.png" 2024-10-14T16:54:31.553Z info: [Crawler][892] Downloaded image as assetId: c6acee98-fa08-4076-8f7c-8e05becff000 2024-10-14T16:54:31.612Z info: [Crawler][892] Completed successfully 2024-10-14T16:54:32.419Z info: [search][895] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ... 2024-10-14T16:54:32.437Z info: [inference][894] Starting an inference job for bookmark with id "k1l8zj5ixpgj9hugbvmibqfc" 2024-10-14T16:54:32.554Z info: [search][895] Completed successfully 2024-10-14T16:54:33.930Z info: [inference][894] Inferring tag for bookmark "k1l8zj5ixpgj9hugbvmibqfc" used 2122 tokens and inferred: Roth IRA,Backdoor Roth,Fidelity,Personal Finance,Investing 2024-10-14T16:54:33.971Z info: [inference][894] Completed successfully 2024-10-14T16:54:34.587Z info: [search][896] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ... 2024-10-14T16:54:34.652Z info: [search][896] Completed successfully 2024-10-14T16:55:03.684Z info: [Crawler][897] Will crawl "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/" for link with id "k1l8zj5ixpgj9hugbvmibqfc" 2024-10-14T16:55:03.684Z info: [Crawler][897] Attempting to determine the content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ 2024-10-14T16:55:03.757Z info: [Crawler][897] Content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ is "text/html; charset=UTF-8" 2024-10-14T16:55:07.264Z info: [Crawler][897] Successfully navigated to "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/". Waiting for the page to load ... 2024-10-14T16:55:11.589Z info: [Crawler][897] Finished waiting for the page to load. 2024-10-14T16:55:11.803Z info: [Crawler][897] Finished capturing page content and a screenshot. FullPageScreenshot: false 2024-10-14T16:55:11.810Z info: [Crawler][897] Will attempt to extract metadata from page ... 2024-10-14T16:55:12.468Z info: [Crawler][897] Will attempt to extract readable content ... 2024-10-14T16:55:13.130Z info: [Crawler][897] Done extracting readable content. 2024-10-14T16:55:13.141Z info: [Crawler][897] Stored the screenshot as assetId: 9a16e763-4619-46d3-8e9e-281e2280acec 2024-10-14T16:55:13.181Z info: [Crawler][897] Done extracting metadata from the page. 2024-10-14T16:55:13.182Z info: [Crawler][897] Downloading image from "https://odsonfinance.com/wp-content/uploads/2024/01/How-to-do-a-Backdoor-Roth-IRA-1.png" 2024-10-14T16:55:13.289Z info: [Crawler][897] Downloaded image as assetId: 76a7a90b-ccab-462e-9f06-9afa6799cf93 2024-10-14T16:55:13.381Z info: [Crawler][897] Will attempt to archive page ... 2024-10-14T16:55:14.169Z info: [inference][898] Starting an inference job for bookmark with id "k1l8zj5ixpgj9hugbvmibqfc" 2024-10-14T16:55:14.186Z info: [search][899] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ... 2024-10-14T16:55:14.249Z info: [search][899] Completed successfully 2024-10-14T16:55:16.005Z info: [inference][898] Inferring tag for bookmark "k1l8zj5ixpgj9hugbvmibqfc" used 2122 tokens and inferred: Roth IRA,Backdoor Roth,Fidelity,Investing,Personal Finance 2024-10-14T16:55:16.041Z info: [inference][898] Completed successfully 2024-10-14T16:55:16.287Z info: [search][900] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ... 2024-10-14T16:55:16.348Z info: [search][900] Completed successfully 2024-10-14T16:56:03.719Z info: [Crawler][897] Will crawl "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/" for link with id "k1l8zj5ixpgj9hugbvmibqfc" 2024-10-14T16:56:03.719Z info: [Crawler][897] Attempting to determine the content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ 2024-10-14T16:56:03.771Z info: [Crawler][897] Content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ is "text/html; charset=UTF-8" 2024-10-14T16:56:06.978Z info: [Crawler][897] Successfully navigated to "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/". Waiting for the page to load ... 2024-10-14T16:56:09.540Z info: [Crawler][897] Finished waiting for the page to load. 2024-10-14T16:56:09.737Z info: [Crawler][897] Finished capturing page content and a screenshot. FullPageScreenshot: false 2024-10-14T16:56:09.747Z info: [Crawler][897] Will attempt to extract metadata from page ... 2024-10-14T16:56:10.481Z info: [Crawler][897] Will attempt to extract readable content ... 2024-10-14T16:56:11.025Z info: [Crawler][897] Done extracting readable content. 2024-10-14T16:56:11.038Z info: [Crawler][897] Stored the screenshot as assetId: 72382d5e-3a19-4382-83d2-1e1cec207c1e 2024-10-14T16:56:11.086Z info: [Crawler][897] Done extracting metadata from the page. 2024-10-14T16:56:11.086Z info: [Crawler][897] Downloading image from "https://odsonfinance.com/wp-content/uploads/2024/01/How-to-do-a-Backdoor-Roth-IRA-1.png" 2024-10-14T16:56:11.215Z info: [Crawler][897] Downloaded image as assetId: 6d054521-87bd-4e25-9b9a-f12c81784706 2024-10-14T16:56:11.312Z info: [Crawler][897] Will attempt to archive page ... 2024-10-14T16:56:12.066Z info: [inference][901] Starting an inference job for bookmark with id "k1l8zj5ixpgj9hugbvmibqfc" 2024-10-14T16:56:12.082Z info: [search][902] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ... 2024-10-14T16:56:12.146Z info: [search][902] Completed successfully 2024-10-14T16:56:15.205Z info: [inference][901] Inferring tag for bookmark "k1l8zj5ixpgj9hugbvmibqfc" used 2123 tokens and inferred: Backdoor Roth IRA,Fidelity,Retirement Planning,Personal Finance,Investing 2024-10-14T16:56:15.258Z info: [inference][901] Completed successfully 2024-10-14T16:56:16.187Z info: [search][903] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ... 2024-10-14T16:56:16.301Z info: [search][903] Completed successfully 2024-10-14T16:57:03.761Z info: [Crawler][897] Will crawl "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/" for link with id "k1l8zj5ixpgj9hugbvmibqfc" 2024-10-14T16:57:03.761Z info: [Crawler][897] Attempting to determine the content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ 2024-10-14T16:57:03.813Z info: [Crawler][897] Content-type for the url https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/ is "text/html; charset=UTF-8" 2024-10-14T16:57:08.863Z info: [Crawler][897] Successfully navigated to "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/". Waiting for the page to load ... 2024-10-14T16:57:11.373Z info: [Crawler][897] Finished waiting for the page to load. 2024-10-14T16:57:11.577Z info: [Crawler][897] Finished capturing page content and a screenshot. FullPageScreenshot: false 2024-10-14T16:57:11.586Z info: [Crawler][897] Will attempt to extract metadata from page ... 2024-10-14T16:57:12.127Z info: [Crawler][897] Will attempt to extract readable content ... 2024-10-14T16:57:12.678Z info: [Crawler][897] Done extracting readable content. 2024-10-14T16:57:12.690Z info: [Crawler][897] Stored the screenshot as assetId: c9b9223a-8cfc-4061-aab7-362035ec162e 2024-10-14T16:57:12.733Z info: [Crawler][897] Done extracting metadata from the page. 2024-10-14T16:57:12.733Z info: [Crawler][897] Downloading image from "https://odsonfinance.com/wp-content/uploads/2024/01/How-to-do-a-Backdoor-Roth-IRA-1.png" 2024-10-14T16:57:12.850Z info: [Crawler][897] Downloaded image as assetId: 88122b54-c814-439b-96bf-f265809f2cbe 2024-10-14T16:57:12.945Z info: [Crawler][897] Will attempt to archive page ... 2024-10-14T16:57:13.718Z info: [inference][904] Starting an inference job for bookmark with id "k1l8zj5ixpgj9hugbvmibqfc" 2024-10-14T16:57:13.735Z info: [search][905] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ... 2024-10-14T16:57:13.803Z info: [search][905] Completed successfully 2024-10-14T16:57:15.744Z info: [inference][904] Inferring tag for bookmark "k1l8zj5ixpgj9hugbvmibqfc" used 2124 tokens and inferred: Backdoor Roth IRA,Fidelity,Personal Finance,Retirement Planning,Investing Strategies 2024-10-14T16:57:15.780Z info: [inference][904] Completed successfully 2024-10-14T16:57:15.830Z info: [search][906] Attempting to index bookmark with id k1l8zj5ixpgj9hugbvmibqfc ... 2024-10-14T16:57:15.945Z info: [search][906] Completed successfully ```
kerem commented 2026-03-02 11:49:05 +03:00
Author
Owner
Copy link

@raviwarrier commented on GitHub (Oct 15, 2024):

I have a similar problem. really old bookmarks from now defunct websites or apps. But when I try to search their URL in hoarder, I get no results and so, I have no easy way to find and delete them.

<!-- gh-comment-id:2412684340 --> @raviwarrier commented on GitHub (Oct 15, 2024): I have a similar problem. really old bookmarks from now defunct websites or apps. But when I try to search their URL in hoarder, I get no results and so, I have no easy way to find and delete them.
kerem commented 2026-03-02 11:49:06 +03:00
Author
Owner
Copy link

@kamtschatka commented on GitHub (Oct 15, 2024):

I tried adding the URL "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/" and everything works just fine.
Are you on the latest version? How are you deploying hoarder?

<!-- gh-comment-id:2414372455 --> @kamtschatka commented on GitHub (Oct 15, 2024): I tried adding the URL "https://odsonfinance.com/chapter-2b-how-to-do-a-backdoor-roth-with-fidelity-step-by-step-instructions/" and everything works just fine. Are you on the latest version? How are you deploying hoarder?
kerem commented 2026-03-02 11:49:06 +03:00
Author
Owner
Copy link

@dimatx commented on GitHub (Oct 16, 2024):

Docker compose and on the latest version. Any other info I can help provide for troubleshooting, assuming I can reproduce?

<!-- gh-comment-id:2415720555 --> @dimatx commented on GitHub (Oct 16, 2024): Docker compose and on the latest version. Any other info I can help provide for troubleshooting, assuming I can reproduce?
kerem commented 2026-03-02 11:49:06 +03:00
Author
Owner
Copy link

@kamtschatka commented on GitHub (Oct 16, 2024):

any environment variables you have set?

<!-- gh-comment-id:2417287162 --> @kamtschatka commented on GitHub (Oct 16, 2024): any environment variables you have set?
kerem commented 2026-03-02 11:49:06 +03:00
Author
Owner
Copy link

@dimatx commented on GitHub (Oct 16, 2024):

Did you try downloading the full page archive? that seems to be what is causing the loop, the process never seems to finish.

Here's my docker compose and .env.

version: "3.8"
services:
  web:
    image: ghcr.io/hoarder-app/hoarder:${HOARDER_VERSION:-release}
    restart: unless-stopped
    volumes:
      - data:/data
    ports:
      - 3200:3000
    env_file:
      - .env
    environment:
      MEILI_ADDR: http://meilisearch:7700
      BROWSER_WEB_URL: http://chrome:9222
      OPENAI_API_KEY: *********************************
      DATA_DIR: /data
  chrome:
    image: gcr.io/zenika-hub/alpine-chrome:123
    restart: unless-stopped
    command:
      - --no-sandbox
      - --disable-gpu
      - --disable-dev-shm-usage
      - --remote-debugging-address=0.0.0.0
      - --remote-debugging-port=9222
      - --hide-scrollbars
  meilisearch:
    image: getmeili/meilisearch:v1.6
    restart: unless-stopped
    env_file:
      - .env
    environment:
      MEILI_NO_ANALYTICS: "true"
    volumes:
      - meilisearch:/meili_data
volumes:
  meilisearch: null
  data: null
networks: {}

HOARDER_VERSION=release
NEXTAUTH_SECRET=*******************
MEILI_MASTER_KEY=*******************
NEXTAUTH_URL=http://*******************:3200
<!-- gh-comment-id:2417384352 --> @dimatx commented on GitHub (Oct 16, 2024): Did you try downloading the full page archive? that seems to be what is causing the loop, the process never seems to finish. Here's my docker compose and .env. ``` version: "3.8" services: web: image: ghcr.io/hoarder-app/hoarder:${HOARDER_VERSION:-release} restart: unless-stopped volumes: - data:/data ports: - 3200:3000 env_file: - .env environment: MEILI_ADDR: http://meilisearch:7700 BROWSER_WEB_URL: http://chrome:9222 OPENAI_API_KEY: ********************************* DATA_DIR: /data chrome: image: gcr.io/zenika-hub/alpine-chrome:123 restart: unless-stopped command: - --no-sandbox - --disable-gpu - --disable-dev-shm-usage - --remote-debugging-address=0.0.0.0 - --remote-debugging-port=9222 - --hide-scrollbars meilisearch: image: getmeili/meilisearch:v1.6 restart: unless-stopped env_file: - .env environment: MEILI_NO_ANALYTICS: "true" volumes: - meilisearch:/meili_data volumes: meilisearch: null data: null networks: {} ``` ``` HOARDER_VERSION=release NEXTAUTH_SECRET=******************* MEILI_MASTER_KEY=******************* NEXTAUTH_URL=http://*******************:3200 ```
kerem commented 2026-03-02 11:49:06 +03:00
Author
Owner
Copy link

@kamtschatka commented on GitHub (Oct 19, 2024):

try increasing the CRAWLER_JOB_TIMEOUT_SEC. The default is 60 seconds, if the full page archival takes too long, it might cause this behavior.

<!-- gh-comment-id:2423624728 --> @kamtschatka commented on GitHub (Oct 19, 2024): try increasing the CRAWLER_JOB_TIMEOUT_SEC. The default is 60 seconds, if the full page archival takes too long, it might cause this behavior.
kerem commented 2026-03-02 11:49:06 +03:00
Author
Owner
Copy link

@dimatx commented on GitHub (Oct 19, 2024):

Made it 300 seconds, issue persists. Isn't it strange that there is a loop despite no errors/failures in the logs? It also uses OpenAI credits over and over according to the logs, so could run up a bill for someone without a low budget set in OpenAI.

<!-- gh-comment-id:2424266617 --> @dimatx commented on GitHub (Oct 19, 2024): Made it 300 seconds, issue persists. Isn't it strange that there is a loop despite no errors/failures in the logs? It also uses OpenAI credits over and over according to the logs, so could run up a bill for someone without a low budget set in OpenAI.
kerem commented 2026-03-02 11:49:06 +03:00
Author
Owner
Copy link

@fifty-six commented on GitHub (Nov 13, 2024):

I have a similar issue with "https://www.npopov.com/2022/12/20/This-year-in-LLVM-2022.html", if that's any help.

<!-- gh-comment-id:2475043135 --> @fifty-six commented on GitHub (Nov 13, 2024): I have a similar issue with "https://www.npopov.com/2022/12/20/This-year-in-LLVM-2022.html", if that's any help.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
refactor/use-npm-singlefile
onetab
claude/issue-2596-20260321-1401
claude/fix-docs-button-responsive-V3aBQ
claude/review-import-backpressure-D4ArJ
claude/fix-archived-bookmarks-mobile-P9OJW
claude/issue-1189-20260211-1601
claude/fix-nested-smart-lists-3uFkt
claude/issue-2298-20251223-1704
feat/import-v3
claude/add-cli-search-subcommand-6kIe0
claude/add-bookmark-indexing-timestamps-96bPj
claude/auto-disable-failing-feeds-fkDhP
claude/add-tag-search-aliases-HzESD
feat/docker-compose-dev
claude/add-attachedby-tags-endpoint-01WYfemMGHJJjXsPYLvUJAno
claude/fix-crawler-memory-leaks-NE7Ct
bookmark-debugger
claude/issue-2352-20260106-1120
claude/issue-1977-20260102-2348
claude/add-banner-rendering-JeLUk
claude/add-descendant-qualifier-cUm26
claude/skip-metadata-refresh-archives-CAo4Y
claude/fix-archive-pending-banner-pAyGM
claude/add-embeddings-support-h2swV
claude/nested-manage-lists-QVV85
claude/privacy-type-system-MG1bT
claude/add-action-menu-icons-6hNKw
claude/issue-2299-20251223-1711
claude/bookmark-indexing-progress-QwZSI
claude/migrate-bookmark-attachments-3O2te
claude/add-2025-wrapped-feature-tIUIh
claude/improve-ai-settings-design-639tq
claude/add-youtube-metascraper-plugin-0lWC7
claude/add-problem-reporting-gSSEV
claude/add-mobile-list-menus-spcS7
claude/shadcn-bookmark-cards-WWHzP
claude/add-extensions-link-HTeXc
claude/add-onboarding-screens-hsYMO
claude/fix-settings-switch-overflow-nlzM4
claude/clamp-bookmark-titles-diAEz
claude/port-stats-mobile-expo-MuXAn
claude/whats-new-base-version-vrv8C
claude/fix-settings-auth-checks-jgyD8
claude/add-server-version-display-3sGa2
claude/fix-tag-editor-scrolling-rzdbG
claude/add-company-pricing-card-y5mHY
claude/audit-optimize-transactions-xpDVc
codex/ensure-consistent-ui-experience-across-app-pages
claude/plan-opentelemetry-integration-01Jx183mz1Ev8h8JoYj97Auw
libsql
db-indicies
claude/export-import-lists-01UuCWwdaqduAd35NppvjnMD
claude/configurable-worker-timeout-0198GQh6YrrRzqG62xnogyrz
claude/check-import-quota-01CPdxTpHp18Ba62bYcBTVbA
claude/scraper-worker-thread-01FEHen6MGrQHmdBstJSuiyA
claude/customize-dialog-styling-01CVjEv2KgyZJSpCg3mqkvR7
claude/add-asset-cache-headers-0175WhNcqwiwurrmjj52jnLT
claude/add-db-search-plugin-017Xxd4Jq3MfjWT788vgfbaq
benchmarks-2
claude/add-filtered-deletion-01DTxWNcg3hhqdNpeNLa3s6L
claude/actionbutton-loading-spinner-015DY5ZTvgPgFAXTZz3UGaYv
claude/add-broken-links-qualifier-01S31X1LsKiYb9gE1dXTKvi3
claude/docker-release-tag-trigger-01UmzFXEumhK2jdmRGtMcueo
claude/spread-feed-fetch-scheduling-01EihUtmZSyqeE1HfRMessxW
restate-idempotency
claude/align-android-ios-colors-01GJfkhEyZVBReohVioPa8ok
claude/improve-mobile-app-colors-0155LzHfkd5HyJr6YyZMsus5
codex/add-autocomplete-for-search-query-language
claude/add-bookmark-backups-016L2A8Z94n7tDgDdMPdFuAd
claude/restrict-binary-user-permissions-01FSGyy2RXGZvE26YbAejzGi
effect-ts
claude/prepare-trpc-npm-publish-0193EjfwpxSNVNcLXqXjs6Ln
shared-list-sidebar
claude/lazy-load-tiktoken-017UTNpJPTcMMQvNEBa1aFwo
codex/fix-asset-pre-processing-worker-abort-signals
add-groupid
claude/add-bookmark-list-button-01VF7uXYNLsVDzqdozWMXP5M
claude/extract-shared-ui-components-01DSVfaCr6WRqAyx1vJTZk9r
claude/migrate-shadcn-sidebar-01DKjpg9MD5PJ2potemSnbvW
claude/add-collaborators-rate-limits-01VjXyRWWPUkGQKa8d8D8qKj
claude/modernize-dark-mode-01FRfE81PAY5C44pFu1cYocf
claude/add-signed-url-bookmark-01PjYT1ZhvLK2FPJNTAhJsWf
restate-group-id
claude/add-highlights-page-012vhHpn8fVNp3gf7gBeW14s
claude/disable-shared-bookmark-features-01B9fiGUdu6NyWaxSQFsQBxP
claude/mobile-bookmark-grid-layouts-018cGBBMhPJVq6PJVRBpqT2r
claude/add-mobile-bookmark-summary-01494LYoh4sJW5Fj4GPm62Vj
claude/add-mobile-tags-screen-01WRADt4ZzvXVew1Y9vqF8SV
claude/add-highlight-notes-01LpanRLS4a2YMnT1qB5GTqX
claude/add-search-bar-014k2ngaqjwYRVSvqmbuECqr
claude/hide-collaborator-emails-01TQrkkMupC7CR9BTuDkireg
claude/list-invitation-approval-0129V89M1riXW6JqmoF74VfM
claude/add-bookmark-archive-sort-018VbGPGvtmsGgXFEERoAX7B
claude/add-mobile-smart-lists-01251tYo9u1SywE6XFezAv9e
claude/bookmark-drag-drop-01DmWq286ogHpDGHKcXjKr3z
claude/add-rss-import-01DH1Q2axcDeq8nQJR5MWjPJ
claude/mobile-inapp-browser-auth-01KiT6bwyntRPQ1X4oTtAveC
claude/offline-mode-react-query-01D1rE2bdBEPw2teGqunr5Gd
claude/add-singlefile-extension-support-01BEB9QQZABzwfZDvR9Bz5b2
claude/custom-list-slugs-01VxcfkNUXZ97FNpNVURopMq
claude/issue-2148-20251118-1133
claude/add-groupid-queue-fairness-011CV1r8Wb46HuGAg5o95i3m
claude/hide-viewer-shared-lists-01Fst6NBvdxrXXnDhUmjsNDP
claude/collaborative-lists-013AvDvMqkoszDVcSoCYgBcM
claude/implement-feature-01LT5XzGsbEhZkYXNEjEwdui
claude/fix-bookmark-loading-state-01AgF4H2drxwuTCJDB2Xgiu4
claude/admin-user-edit-013tbiRmb1KX2fhSYqmGKCu8
claude/expose-all-api-01YTruEW72WQYMtq4iZoaPkA
claude/add-doc-link-main-016NYLxShpKuH6R8XCBgeZtc
claude/fix-issue-2133-019JLvdSRAUbU4FtjQztcM6S
claude/explore-effect-ts-integration-01F7xb1dWwP1ma4LnLbFGfDD
claude/optimize-dockerfile-build-011CV5gDnPZbdbbVSPDofC4e
claude/add-custom-headers-guide-011CV249t16aWDRb1mCrzQdC
claude/mobile-app-signup-011CUxPtCXgU6U3T8GShTR2Q
claude/crawler-worker-fetch-browser-011CUvcRc24XEr9DTWDW6MX8
claude/fix-issue-784-011CUvubQrcZHG9S3KjpCKbK
codex/add-user-settings-for-inference-language-and-screenshots
claude/fix-mobile-signin-server-address-011CUnaUWwY2Fhq5Xbwhgr8H
better-auth-2
claude/issue-2028-20251012-1429
claude/issue-1010-20251012-1154
codex/update-feed-refresh-job-idempotency-key
restate
import-v2
fix-public-lists
recurse-delete-list
abort-dangling-processing
tag-pagination
ratelimit-plugin
claude/issue-1937-20250914-0912
codex/implement-title-search-query-qualifier
copilot/add-edit-button-for-notes
cookie-path
ai-tag-cleanup
codex/add-allowlist-and-blocklist-env-variables
mobile-retheme
expo-next-upgrade
opencode/issue1788-20250727215611
fix-trailing-slash-deduplication
edit-bookmark-dialog
bookmark-embeddings
rag
nextjs-15
bookmark-hover-bar
sapling-pr-archive-MohamedBassem
track-bookmark-assets
json-cli
admin-settings
mobile-dark-mode
android/v1.9.2-0
ios/v1.9.1-1
android/v1.9.1-0
ios/v1.9.1-0
ios/v1.9.0-2
ios/v1.9.0-1
android/v1.9.0-1
extension/v1.2.9
cli/v0.31.0
sdk/v0.31.0
mcp/v0.31.0
android/v1.9.0-0
ios/v1.9.0-0
v0.31.0
android/v1.8.5-0
cli/v0.30.0
sdk/v0.30.0
ios/v1.8.4-0
android/v1.8.4-0
v0.30.0
cli/v0.29.1
v0.29.3
v0.29.2
v0.29.1
sdk/v0.29.0
cli/v0.29.0
mcp/v0.29.0
ios/v1.8.3-0
android/v1.8.3-0
extension/v1.2.8
v0.29.0
android/v1.8.2-2
android/v1.8.2-1
ios/v1.8.2-0
android/v1.8.2-0
extension/v1.2.7
android/v1.8.1-0
ios/v1.8.1-0
v0.28.0
cli/v0.27.1
cli/v0.27.0
v0.27.1
sdk/v0.27.0
v0.27.0
android/v1.8.0-1
ios/v1.8.0-1
mcp/v0.26.0
sdk/v0.26.0
v0.26.0
cli/v0.25.0
ios/v1.7.0-1
mcp/v0.25.0
v0.25.0
extension/v1.2.6
ios/v1.7.0-0
android/v1.7.0-0
v0.24.1
v0.24.0
mcp/v0.23.10
mcp/v0.23.9
mcp/v0.23.8
extension/v1.2.5
mcp/v0.23.7
mcp/v0.23.6
mcp/v0.23.5
mcp/v0.23.4
sdk/v0.23.2
cli/v0.23.0
extension/v1.2.4
android/v1.6.9-1
ios/v1.6.9-1
v0.23.2
v0.23.1
sdk/v0.23.0
v0.23.0
ios/v1.6.9-0
sdk/v0.22.0
v0.22.0
android/v1.6.8-0
ios/v1.6.8-0
sdk/v0.21.2
sdk/v0.21.1
sdk/v0.21.0
v0.21.0
cli/v0.20.0
v0.20.0
ios/v1.6.7-4
android/v1.6.7-4
ios/v1.6.7-3
android/v1.6.7-3
android/v1.6.7-2
ios/v1.6.7-2
android/v1.6.7-1
ios/v1.6.7-1
ios/v1.6.7-0
android/v1.6.7-0
v0.19.0
android/v1.6.6-0
android/v1.6.5-0
ios/v1.6.5-0
ios/v1.6.4-0
android/v1.6.4-0
v0.18.0
v0.17.1
v0.17.0
ios/v1.6.3-0
android/v1.6.3-0
extension/v1.2.3
ios/v1.6.2-1
android/v1.6.2-1
ios/v1.6.2-0
android/v1.6.2-0
v0.16.0
ios/v1.6.1-3
android/v1.6.1-3
ios/v1.6.1-2
android/v1.6.1-2
android/v1.6.1-1
ios/v1.6.1-1
android/v1.6.1-0
ios/v1.6.1-0
extension/v1.2.2
android/v1.6.0-1
ios/v1.6.0-1
ios/v1.6.0
android/v1.6.0
cli/v0.13.7
cli/v0.13.6
v0.15.0
cli/v0.13.5
extension/v1.2.1
v0.14.0
cli/v0.13.3
cli/v0.13.2
cli/v0.13.1
cli/v0.13.0
v0.13.1
v0.13.0
mobile-v1.5.0
mobile-v1.4.0
v0.12.2
v0.12.1
v0.12.0
v0.11.1
v0.11.0
v0.10.1
v0.10.0
v0.9.0
v0.8.0
v0.7.0
v0.6.0
v0.5.0
v0.4.1
v.0.4.0
v.0.3.1
v0.3.0
v0.2.2
v0.2.1
v0.2.0
v0.1.0
Labels
Clear labels   
UI/UX

   
android

   
bug

   
dependencies

   
documentation

   
documentation

   
extension

   
feature request

   
feature request

   
good first issue

   
ios

   
long-term

   
performance

   
pri/high

   
pri/low

   
pri/medium

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
status/approved

   
status/icebox

   
status/pending_clarification

   
status/untriaged
No labels
UI/UX
android
bug
dependencies
documentation
documentation
extension
feature request
feature request
good first issue
ios
long-term
performance
pri/high
pri/low
pri/medium
pull-request
question
status/approved
status/icebox
status/pending_clarification
status/untriaged
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/karakeep#347
No description provided.