[GH-ISSUE #1139] [FR] Rate Limit setting for inference #745

New issue

Open

opened 2026-03-02 11:52:23 +03:00 by kerem · 1 comment

kerem commented

2026-03-02 11:52:23 +03:00

Owner

Originally created by @DIGist on GitHub (Mar 19, 2025).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/1139

Describe the feature you'd like

Hi, whether using Ollama, or an Openai endpoint like openrouter (since there are some free llms there), it would be good to have a setting for setting a rate limit for inference jobs.
Something like:
INFERENCE_RATE_LIMIT_PER_MINUTE=3
CONCURRENT_INFERENCE_JOBS=1

Describe the benefits this would bring to existing Hoarder users

Allow lowend ollama users and people using free apis to leverage them efficiently for their inference tasks.

Can the goal of this request already be achieved via other means?

Have you searched for an existing open/closed issue?

I have searched for existing issues and none cover my fundamental request

Additional context

No response

Originally created by @DIGist on GitHub (Mar 19, 2025). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/1139 ### Describe the feature you'd like Hi, whether using Ollama, or an Openai endpoint like openrouter (since there are some free llms there), it would be good to have a setting for setting a rate limit for inference jobs. Something like: INFERENCE_RATE_LIMIT_PER_MINUTE=3 CONCURRENT_INFERENCE_JOBS=1 ### Describe the benefits this would bring to existing Hoarder users Allow lowend ollama users and people using free apis to leverage them efficiently for their inference tasks. ### Can the goal of this request already be achieved via other means? No ### Have you searched for an existing open/closed issue? - [x] I have searched for existing issues and none cover my fundamental request ### Additional context _No response_

kerem added the

feature request

status/icebox

labels

2026-03-02 11:52:23 +03:00

kerem commented

2026-03-02 11:52:24 +03:00

Author

Owner

@sk33ny commented on GitHub (May 31, 2025):

I have also run into this issue and would love to see a feature like this implemented.

@sk33ny commented on GitHub (May 31, 2025): I have also run into this issue and would love to see a feature like this implemented.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/karakeep#745

No description provided.

Rows
Columns