[GH-ISSUE #2252] Separate AI tagging for images

kerem commented

2026-03-02 11:56:52 +03:00

Owner

Originally created by @Fmstrat on GitHub (Dec 13, 2025).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/2252

Describe the feature you'd like

When sharing an image, or a URL to an image, you end up with the image and whatever filename the image has. Often from social sites this is something like ajzJruYtTT2H.png. You're never finding that again.

There are better models now, like Qwen3 VL, that are great at extracting content from images. It would be great of there were settings to specify AI use for image tagging/description independent from standard use.

Describe the benefits this would bring to existing Karakeep users

Actually having relevant search content for saved images.

Can the goal of this request already be achieved via other means?

Not really.

Have you searched for an existing open/closed issue?

I have searched for existing issues and none cover my fundamental request

Additional context

No response

Originally created by @Fmstrat on GitHub (Dec 13, 2025). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/2252 ### Describe the feature you'd like When sharing an image, or a URL to an image, you end up with the image and whatever filename the image has. Often from social sites this is something like `ajzJruYtTT2H.png`. You're never finding that again. There are better models now, like Qwen3 VL, that are great at extracting content from images. It would be great of there were settings to specify AI use for image tagging/description independent from standard use. ### Describe the benefits this would bring to existing Karakeep users Actually having relevant search content for saved images. ### Can the goal of this request already be achieved via other means? Not really. ### Have you searched for an existing open/closed issue? - [x] I have searched for existing issues and none cover my fundamental request ### Additional context _No response_

kerem

2026-03-02 11:56:52 +03:00

closed this issue
added the
feature request

status/untriaged
labels

kerem commented

2026-03-02 11:56:53 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Dec 13, 2025):

You can specify a separate model for images by setting INFERENCE_IMAGE_MODEL, right?

@MohamedBassem commented on GitHub (Dec 13, 2025): You can specify a separate model for images by setting `INFERENCE_IMAGE_MODEL`, right?

kerem commented

2026-03-02 11:56:53 +03:00

Author

Owner

@Fmstrat commented on GitHub (Dec 13, 2025):

Hmm, guess its been a while since I looked at the config. I will take a look. I also just noticed Tesseract support, too. Is that supposed to fire by default?

@Fmstrat commented on GitHub (Dec 13, 2025): Hmm, guess its been a while since I looked at the config. I will take a look. I also just noticed Tesseract support, too. Is that supposed to fire by default?

kerem commented

2026-03-02 11:56:53 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Dec 13, 2025):

yes

@MohamedBassem commented on GitHub (Dec 13, 2025): yes

kerem commented

2026-03-02 11:56:53 +03:00

Author

Owner

@Fmstrat commented on GitHub (Dec 13, 2025):

Hmm, there didn't appear to be any OCR, but setting the INFERENCE_IMAGE_MODEL worked great, I now have tags! Thank you. Is it possible to have Karakeep rewrite the title of the bookmark for images? If not, I'll revamp this into a request for that, otherwise I will close. Thanks again.

@Fmstrat commented on GitHub (Dec 13, 2025): Hmm, there didn't appear to be any OCR, but setting the `INFERENCE_IMAGE_MODEL` worked great, I now have tags! Thank you. Is it possible to have Karakeep rewrite the title of the bookmark for images? If not, I'll revamp this into a request for that, otherwise I will close. Thanks again.

kerem commented

2026-03-02 11:56:53 +03:00

Author

Owner

@Fmstrat commented on GitHub (Dec 13, 2025):

And may have run into a separate issue where it thinks webp are videos and can't tag them:

|archiver-karakeep | 2025-12-13T14:32:27.944152187Z 2025-12-13T14:32:27.943Z info: [Crawler][17839] Downloading image from "https://lemmy.dbzer0.com/pictrs/image/e0eb1e5d-3d67-4a3e-86ed-6108a385f341.webp"
|archiver-karakeep | 2025-12-13T14:32:28.467180447Z 2025-12-13T14:32:28.466Z info: [Crawler][17839] Downloaded image as assetId: 28df8064-44a9-46bf-a747-2c415e09d378
|archiver-karakeep | 2025-12-13T14:32:28.576927028Z 2025-12-13T14:32:28.576Z info: [Crawler][17839] Will attempt to archive page ...
|archiver-karakeep | 2025-12-13T14:32:28.735852694Z 2025-12-13T14:32:28.735Z info: [inference][17840] Starting an inference job for bookmark with id "fjvhr0rdaguhrl52x1lqfb8h"
|archiver-karakeep | 2025-12-13T14:32:28.737179392Z 2025-12-13T14:32:28.736Z info: [inference] No content found for link "fjvhr0rdaguhrl52x1lqfb8h". Skipping tagging.
|archiver-karakeep | 2025-12-13T14:32:28.737232944Z 2025-12-13T14:32:28.737Z info: [inference][17840] Skipping tagging for bookmark "fjvhr0rdaguhrl52x1lqfb8h" due to missing content.
|archiver-karakeep | 2025-12-13T14:32:28.737292256Z 2025-12-13T14:32:28.737Z info: [inference][17840] Completed successfully
|archiver-karakeep | 2025-12-13T14:32:28.754446257Z 2025-12-13T14:32:28.753Z info: [inference][17841] Starting a summary job for bookmark with id "fjvhr0rdaguhrl52x1lqfb8h"
|archiver-karakeep | 2025-12-13T14:32:28.755410396Z 2025-12-13T14:32:28.755Z info: [inference] No content found for link "fjvhr0rdaguhrl52x1lqfb8h". Skipping summary.
|archiver-karakeep | 2025-12-13T14:32:28.755464940Z 2025-12-13T14:32:28.755Z info: [inference][17841] Completed successfully

@Fmstrat commented on GitHub (Dec 13, 2025): And may have run into a separate issue where it thinks `webp` are videos and can't tag them: ``` |archiver-karakeep | 2025-12-13T14:32:27.944152187Z 2025-12-13T14:32:27.943Z info: [Crawler][17839] Downloading image from "https://lemmy.dbzer0.com/pictrs/image/e0eb1e5d-3d67-4a3e-86ed-6108a385f341.webp" |archiver-karakeep | 2025-12-13T14:32:28.467180447Z 2025-12-13T14:32:28.466Z info: [Crawler][17839] Downloaded image as assetId: 28df8064-44a9-46bf-a747-2c415e09d378 |archiver-karakeep | 2025-12-13T14:32:28.576927028Z 2025-12-13T14:32:28.576Z info: [Crawler][17839] Will attempt to archive page ... |archiver-karakeep | 2025-12-13T14:32:28.735852694Z 2025-12-13T14:32:28.735Z info: [inference][17840] Starting an inference job for bookmark with id "fjvhr0rdaguhrl52x1lqfb8h" |archiver-karakeep | 2025-12-13T14:32:28.737179392Z 2025-12-13T14:32:28.736Z info: [inference] No content found for link "fjvhr0rdaguhrl52x1lqfb8h". Skipping tagging. |archiver-karakeep | 2025-12-13T14:32:28.737232944Z 2025-12-13T14:32:28.737Z info: [inference][17840] Skipping tagging for bookmark "fjvhr0rdaguhrl52x1lqfb8h" due to missing content. |archiver-karakeep | 2025-12-13T14:32:28.737292256Z 2025-12-13T14:32:28.737Z info: [inference][17840] Completed successfully |archiver-karakeep | 2025-12-13T14:32:28.754446257Z 2025-12-13T14:32:28.753Z info: [inference][17841] Starting a summary job for bookmark with id "fjvhr0rdaguhrl52x1lqfb8h" |archiver-karakeep | 2025-12-13T14:32:28.755410396Z 2025-12-13T14:32:28.755Z info: [inference] No content found for link "fjvhr0rdaguhrl52x1lqfb8h". Skipping summary. |archiver-karakeep | 2025-12-13T14:32:28.755464940Z 2025-12-13T14:32:28.755Z info: [inference][17841] Completed successfully ```

kerem commented

2026-03-02 11:56:53 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Dec 13, 2025):

Glad that the image model worked. And currently, we don't support renaming images unfortunately. Not sure I understand the issue that you're facing with webp as the logs look normal to me.

@MohamedBassem commented on GitHub (Dec 13, 2025): Glad that the image model worked. And currently, we don't support renaming images unfortunately. Not sure I understand the issue that you're facing with `webp` as the logs look normal to me.

kerem commented

2026-03-02 11:56:53 +03:00

Author

Owner

@Fmstrat commented on GitHub (Dec 13, 2025):

It's the skipping inference part. Just had it happen with jpeg, too:

|archiver-karakeep | 2025-12-13T14:41:09.565198245Z 2025-12-13T14:41:09.565Z info: [Crawler][17860:0] Downloading image from "https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg"
|archiver-karakeep | 2025-12-13T14:41:09.881939959Z 2025-12-13T14:41:09.881Z info: <-- GET /api/trpc/bookmarks.getBookmark?batch=1&input=%7B%220%22%3A%7B%22json%22%3A%7B%22bookmarkId%22%3A%22lrzhr94v0jesd1xk5yca1ah6%22%7D%7D%7D
|archiver-karakeep | 2025-12-13T14:41:09.886922424Z 2025-12-13T14:41:09.886Z info: --> GET /api/trpc/bookmarks.getBookmark?batch=1&input=%7B%220%22%3A%7B%22json%22%3A%7B%22bookmarkId%22%3A%22lrzhr94v0jesd1xk5yca1ah6%22%7D%7D%7D 200 5ms
|archiver-karakeep | 2025-12-13T14:41:10.156196534Z 2025-12-13T14:41:10.155Z info: [Crawler][17860:0] Downloaded image as assetId: 15a1b188-ab69-45f8-8369-0c0cc94b882a (180887 bytes)
|archiver-karakeep | 2025-12-13T14:41:10.210402204Z 2025-12-13T14:41:10.210Z info: [Crawler][17860:0] Will attempt to archive page ...
|archiver-karakeep | 2025-12-13T14:41:10.257434000Z 2025-12-13T14:41:10.257Z info: [search][17867] Attempting to index bookmark with id lrzhr94v0jesd1xk5yca1ah6 ...
|archiver-karakeep | 2025-12-13T14:41:10.470287437Z 2025-12-13T14:41:10.470Z info: [search][17867] Completed successfully
|archiver-karakeep | 2025-12-13T14:41:10.908336771Z 2025-12-13T14:41:10.908Z info: <-- GET /api/trpc/bookmarks.getBookmark?batch=1&input=%7B%220%22%3A%7B%22json%22%3A%7B%22bookmarkId%22%3A%22lrzhr94v0jesd1xk5yca1ah6%22%7D%7D%7D
|archiver-karakeep | 2025-12-13T14:41:10.913520989Z 2025-12-13T14:41:10.913Z info: --> GET /api/trpc/bookmarks.getBookmark?batch=1&input=%7B%220%22%3A%7B%22json%22%3A%7B%22bookmarkId%22%3A%22lrzhr94v0jesd1xk5yca1ah6%22%7D%7D%7D 200 5ms
|archiver-karakeep | 2025-12-13T14:41:10.934748089Z 2025-12-13T14:41:10.934Z info: <-- GET /api/assets/15a1b188-ab69-45f8-8369-0c0cc94b882a
|archiver-karakeep | 2025-12-13T14:41:10.940295095Z 2025-12-13T14:41:10.940Z info: --> GET /api/assets/15a1b188-ab69-45f8-8369-0c0cc94b882a 200 5ms
|archiver-karakeep | 2025-12-13T14:41:11.049586384Z 2025-12-13T14:41:11.049Z info: [inference][17865] Starting an inference job for bookmark with id "lrzhr94v0jesd1xk5yca1ah6"
|archiver-karakeep | 2025-12-13T14:41:11.050072507Z 2025-12-13T14:41:11.050Z info: [inference] No content found for link "lrzhr94v0jesd1xk5yca1ah6". Skipping tagging.
|archiver-karakeep | 2025-12-13T14:41:11.050089269Z 2025-12-13T14:41:11.050Z info: [inference][17865] Skipping tagging for bookmark "lrzhr94v0jesd1xk5yca1ah6" due to missing content.
|archiver-karakeep | 2025-12-13T14:41:11.050138592Z 2025-12-13T14:41:11.050Z info: [inference][17865] Completed successfully

Specifically the No content found for link "lrzhr94v0jesd1xk5yca1ah6". Skipping tagging..

To reproduce:

Doesn't add tags:
- Add: https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg
Does add tags:
- Download https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg
- Drag and drop the JPG into the browser to add it

It seems some file extensions aren't working when coming in via links. The content-type in the 200 response looks fine.

@Fmstrat commented on GitHub (Dec 13, 2025): It's the skipping inference part. Just had it happen with `jpeg`, too: ``` |archiver-karakeep | 2025-12-13T14:41:09.565198245Z 2025-12-13T14:41:09.565Z info: [Crawler][17860:0] Downloading image from "https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg" |archiver-karakeep | 2025-12-13T14:41:09.881939959Z 2025-12-13T14:41:09.881Z info: <-- GET /api/trpc/bookmarks.getBookmark?batch=1&input=%7B%220%22%3A%7B%22json%22%3A%7B%22bookmarkId%22%3A%22lrzhr94v0jesd1xk5yca1ah6%22%7D%7D%7D |archiver-karakeep | 2025-12-13T14:41:09.886922424Z 2025-12-13T14:41:09.886Z info: --> GET /api/trpc/bookmarks.getBookmark?batch=1&input=%7B%220%22%3A%7B%22json%22%3A%7B%22bookmarkId%22%3A%22lrzhr94v0jesd1xk5yca1ah6%22%7D%7D%7D 200 5ms |archiver-karakeep | 2025-12-13T14:41:10.156196534Z 2025-12-13T14:41:10.155Z info: [Crawler][17860:0] Downloaded image as assetId: 15a1b188-ab69-45f8-8369-0c0cc94b882a (180887 bytes) |archiver-karakeep | 2025-12-13T14:41:10.210402204Z 2025-12-13T14:41:10.210Z info: [Crawler][17860:0] Will attempt to archive page ... |archiver-karakeep | 2025-12-13T14:41:10.257434000Z 2025-12-13T14:41:10.257Z info: [search][17867] Attempting to index bookmark with id lrzhr94v0jesd1xk5yca1ah6 ... |archiver-karakeep | 2025-12-13T14:41:10.470287437Z 2025-12-13T14:41:10.470Z info: [search][17867] Completed successfully |archiver-karakeep | 2025-12-13T14:41:10.908336771Z 2025-12-13T14:41:10.908Z info: <-- GET /api/trpc/bookmarks.getBookmark?batch=1&input=%7B%220%22%3A%7B%22json%22%3A%7B%22bookmarkId%22%3A%22lrzhr94v0jesd1xk5yca1ah6%22%7D%7D%7D |archiver-karakeep | 2025-12-13T14:41:10.913520989Z 2025-12-13T14:41:10.913Z info: --> GET /api/trpc/bookmarks.getBookmark?batch=1&input=%7B%220%22%3A%7B%22json%22%3A%7B%22bookmarkId%22%3A%22lrzhr94v0jesd1xk5yca1ah6%22%7D%7D%7D 200 5ms |archiver-karakeep | 2025-12-13T14:41:10.934748089Z 2025-12-13T14:41:10.934Z info: <-- GET /api/assets/15a1b188-ab69-45f8-8369-0c0cc94b882a |archiver-karakeep | 2025-12-13T14:41:10.940295095Z 2025-12-13T14:41:10.940Z info: --> GET /api/assets/15a1b188-ab69-45f8-8369-0c0cc94b882a 200 5ms |archiver-karakeep | 2025-12-13T14:41:11.049586384Z 2025-12-13T14:41:11.049Z info: [inference][17865] Starting an inference job for bookmark with id "lrzhr94v0jesd1xk5yca1ah6" |archiver-karakeep | 2025-12-13T14:41:11.050072507Z 2025-12-13T14:41:11.050Z info: [inference] No content found for link "lrzhr94v0jesd1xk5yca1ah6". Skipping tagging. |archiver-karakeep | 2025-12-13T14:41:11.050089269Z 2025-12-13T14:41:11.050Z info: [inference][17865] Skipping tagging for bookmark "lrzhr94v0jesd1xk5yca1ah6" due to missing content. |archiver-karakeep | 2025-12-13T14:41:11.050138592Z 2025-12-13T14:41:11.050Z info: [inference][17865] Completed successfully ``` Specifically the `No content found for link "lrzhr94v0jesd1xk5yca1ah6". Skipping tagging.`. To reproduce: - Doesn't add tags: - Add: https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg - Does add tags: - Download https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg - Drag and drop the JPG into the browser to add it It seems some file extensions aren't working when coming in via links. The content-type in the 200 response looks fine.

kerem commented

2026-03-02 11:56:53 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Dec 13, 2025):

[Crawler][6713:0] Content-type for the url https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg is "null"

We rely on the website we're crawling telling us the content type of the asset. In here, it's an image but they responded with null so we treated it as link instead of an image. So this is a website error, karakeep works as intended here.

@MohamedBassem commented on GitHub (Dec 13, 2025): > [Crawler][6713:0] Content-type for the url https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg is "null" We rely on the website we're crawling telling us the content type of the asset. In here, it's an image but they responded with `null` so we treated it as link instead of an image. So this is a website error, karakeep works as intended here.

kerem commented

2026-03-02 11:56:53 +03:00

Author

Owner

@Fmstrat commented on GitHub (Dec 13, 2025):

Weird.. When grabbing the image in a browser it has a content type of image/jpeg?

@Fmstrat commented on GitHub (Dec 13, 2025): Weird.. When grabbing the image in a browser it has a content type of `image/jpeg`? <img width="1252" height="575" alt="Image" src="https://github.com/user-attachments/assets/da794123-c614-4563-80cb-f035a5ae2344" />

kerem commented

2026-03-02 11:56:54 +03:00

Author

Owner

@Fmstrat commented on GitHub (Dec 13, 2025):

After some more investigation and testing using node-fetch it appears to be a problem with specifying a method of HEAD instead of GET. It appears pretty common for proxy services to block HEAD requests with 403, especially when not using signed URLs.

This example, replicating what Karakeep is doing, fails with a 403 and thus no content-type header:

import fetch, { Headers } from "node-fetch";

async function main() {
    const response = await fetch(
        'https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg',
        {
            method: 'HEAD',
        },
    );
    console.log(response)
}

main();

While this returns a 200 and the content-type header just fine:

import fetch, { Headers } from "node-fetch";

async function main() {
    const response = await fetch(
        'https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg',
    );
    console.log(response)
}

main();

@Fmstrat commented on GitHub (Dec 13, 2025): After some more investigation and testing using `node-fetch` it appears to be a problem with specifying a method of `HEAD` instead of `GET`. It appears pretty common for proxy services to block HEAD requests with 403, especially when not using signed URLs. This example, replicating what Karakeep is doing, fails with a 403 and thus no content-type header: ```js import fetch, { Headers } from "node-fetch"; async function main() { const response = await fetch( 'https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg', { method: 'HEAD', }, ); console.log(response) } main(); ``` While this returns a 200 and the content-type header just fine: ```js import fetch, { Headers } from "node-fetch"; async function main() { const response = await fetch( 'https://feddit.org/pictrs/image/23e19a63-25c8-4a91-a746-a3705f04f61e.jpeg', ); console.log(response) } main(); ```

kerem commented

2026-03-02 11:56:54 +03:00

Author

Owner

@Fmstrat commented on GitHub (Dec 13, 2025):

As a note, this would likely happen on any item hosted on an S3 service, like AWS Cloudflare, etc, due to the difference in GET vs HEAD imposed by them. Since this was posted on Lemmy, which can be configured to host images on an S3 compatible service, this makes sense.

@Fmstrat commented on GitHub (Dec 13, 2025): As a note, this would likely happen on any item hosted on an S3 service, like AWS Cloudflare, etc, due to the difference in `GET` vs `HEAD` imposed by them. Since this was posted on Lemmy, which can be configured to host images on an S3 compatible service, this makes sense.

kerem commented

2026-03-02 11:56:54 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Dec 13, 2025):

thanks for the investigation! I don't see a reason why we can't do GET requests for those requests (and just not read the body)

@MohamedBassem commented on GitHub (Dec 13, 2025): thanks for the investigation! I don't see a reason why we can't do GET requests for those requests (and just not read the body)

kerem commented

2026-03-02 11:56:54 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Dec 13, 2025):

Done

And I confirm that it works. Thanks for the investigation!

@MohamedBassem commented on GitHub (Dec 13, 2025): [Done](https://github.com/karakeep-app/karakeep/commit/d6dd8ebdb614d39890810ff9fbc8d71d35af4f03) And I confirm that it works. Thanks for the investigation!

kerem commented

2026-03-02 11:56:54 +03:00

Author

Owner

@Fmstrat commented on GitHub (Dec 13, 2025):

No problem. It might be more efficient to hold the body if it's not a video, though, that way the crawler isn't making 2 hits every time.

@Fmstrat commented on GitHub (Dec 13, 2025): No problem. It might be more efficient to hold the body if it's not a video, though, that way the crawler isn't making 2 hits every time.

Rows
Columns

[GH-ISSUE #2252] Separate AI tagging for images #1376

Describe the feature you'd like

Describe the benefits this would bring to existing Karakeep users

Can the goal of this request already be achieved via other means?

Have you searched for an existing open/closed issue?

Additional context