starred/karakeep

Fork 0

mirror of https://github.com/karakeep-app/karakeep.git synced 2026-04-27 00:46:03 +03:00

[GH-ISSUE #94] [Feature Request] Add a token cutoff #81

New issue

Closed

opened 2026-03-02 11:46:22 +03:00 by kerem · 11 comments

kerem commented

2026-03-02 11:46:22 +03:00

Owner

Originally created by @Reledia on GitHub (Apr 10, 2024).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/94

Hello. Sometimes, some pages I save contains essays or stories that can span aboce 20k+ tokens. This does not only used much more credits/money when doing a request, but it consumes a lot of time (from 4s to >50s) and the prompt get lost in the way.

Would it be possible to add an env variable so we can set the max number of tokens to send in a single request?

Originally created by @Reledia on GitHub (Apr 10, 2024). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/94 Hello. Sometimes, some pages I save contains essays or stories that can span aboce 20k+ tokens. This does not only used much more credits/money when doing a request, but it consumes a lot of time (from 4s to >50s) and the prompt get lost in the way. Would it be possible to add an env variable so we can set the max number of tokens to send in a single request?

kerem

2026-03-02 11:46:22 +03:00

closed this issue
added the
feature request

good first issue
labels

kerem commented

2026-03-02 11:46:23 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Apr 11, 2024):

Hi, there's already a hardcoded token cutoff of 1500 tokens (to fit in the context window of some models). But making this configurable makes a ton of sense :) Should be easy to implement and include in the next release. Thanks for the suggestion!

@MohamedBassem commented on GitHub (Apr 11, 2024): Hi, there's already a hardcoded token cutoff of 1500 tokens (to fit in the context window of some models). But making this configurable makes a ton of sense :) Should be easy to implement and include in the next release. Thanks for the suggestion!

kerem commented

2026-03-02 11:46:23 +03:00

Author

Owner

@Reledia commented on GitHub (Apr 11, 2024):

I'm happy to hear this was already implemented, but I observe a different thing in my logs

My version of hoarder should be the latest available

@Reledia commented on GitHub (Apr 11, 2024): I'm happy to hear this was already implemented, but I observe a different thing in my logs ![Screenshot_20240411_125943_Vivaldi.jpg](https://github.com/MohamedBassem/hoarder-app/assets/82183718/92e1c732-39ec-4061-a27d-25b016f1ebf3) My version of hoarder should be the latest available

kerem commented

2026-03-02 11:46:23 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Apr 11, 2024):

Oh wow, that's a lot of tokens. Hmmm, I have a guess. I'm currently approximating tokens to words. So technically speaking I'm truncating the content to 1500 words not tokens. Maybe that's where the difference is coming from? Do you have a link that you are willing to share for me to debug this? Thanks!

@MohamedBassem commented on GitHub (Apr 11, 2024): Oh wow, that's a lot of tokens. Hmmm, I have a guess. I'm currently approximating `tokens` to `words`. So technically speaking I'm truncating the content to 1500 words not tokens. Maybe that's where the difference is coming from? Do you have a link that you are willing to share for me to debug this? Thanks!

kerem commented

2026-03-02 11:46:23 +03:00

Author

Owner

@Reledia commented on GitHub (Apr 11, 2024):

I don't wanna share the actual link because it's very explicit, but it was a chapter of a fanfiction from AO3, so very heavy on words

@Reledia commented on GitHub (Apr 11, 2024): I don't wanna share the actual link because it's very explicit, but it was a chapter of a fanfiction from AO3, so very heavy on words

kerem commented

2026-03-02 11:46:24 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Apr 15, 2024):

Turned out, I had a bug in the content truncating logic. Sending a fix now.

@MohamedBassem commented on GitHub (Apr 15, 2024): Turned out, I had a bug in the content truncating logic. Sending a fix now.

kerem commented

2026-03-02 11:46:24 +03:00

Author

Owner

@Reledia commented on GitHub (Apr 15, 2024):

By looking at the commit, was the truncate function sending words only after the 1500th one?

@Reledia commented on GitHub (Apr 15, 2024): By looking at the commit, was the truncate function sending words only after the 1500th one?

kerem commented

2026-03-02 11:46:24 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Apr 15, 2024):

Yeah, noob mistake :)

@MohamedBassem commented on GitHub (Apr 15, 2024): Yeah, noob mistake :)

kerem commented

2026-03-02 11:46:24 +03:00

Author

Owner

@Reledia commented on GitHub (Apr 15, 2024):

Dont worry, it happens to everyone :)
But maybe in the next release it would be better to inform users to re-process all previous bookmarks?

@Reledia commented on GitHub (Apr 15, 2024): Dont worry, it happens to everyone :) But maybe in the next release it would be better to inform users to re-process all previous bookmarks?

kerem commented

2026-03-02 11:46:24 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Apr 15, 2024):

The bug only affects bookmarks with content larger than 1500 though, so it's not all bad. I'll mention it in the release note and let people decide whether they want to re-process or not (given that re-processing can be expensive for those using openai for example).

@MohamedBassem commented on GitHub (Apr 15, 2024): The bug only affects bookmarks with content larger than 1500 though, so it's not all bad. I'll mention it in the release note and let people decide whether they want to re-process or not (given that re-processing can be expensive for those using openai for example).

kerem commented

2026-03-02 11:46:24 +03:00

Author

Owner

@Reledia commented on GitHub (Apr 15, 2024):

Seems like the best solution. Thanks for the time

@Reledia commented on GitHub (Apr 15, 2024): Seems like the best solution. Thanks for the time

kerem commented

2026-03-02 11:46:24 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Apr 15, 2024):

Hotfix 0.12.1 released with the fix. Thanks for the report!

@MohamedBassem commented on GitHub (Apr 15, 2024): Hotfix `0.12.1` released with the fix. Thanks for the report!

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/karakeep#81

No description provided.

Rows
Columns