[GH-ISSUE #1890] [Doc] INFERENCE_NUM_WORKERS is bottlenecked by SEARCH_NUM_WORKERS #1175

New issue

Open

opened 2026-03-02 11:55:31 +03:00 by kerem · 0 comments

kerem commented

2026-03-02 11:55:31 +03:00

Owner

Originally created by @mratsim on GitHub (Sep 5, 2025).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/1890

I am trying the new INFERENCE_NUM_WORKERS=16 flag to schedule 16 inference jobs in parallel.

However when looking in the logs of my inference server (vLLM), I could only see 4 simultaneous queries in flight.

It seems like it was because the rest of my config was

CRAWLER_NUM_WORKERS=4                                                                                                                                                                                                                                                     
SEARCH_NUM_WORKERS=4                                                                                                                                                                                                                                                      
ASSET_PREPROCESSING_NUM_WORKERS=4

I changed SEARCH_NUM_WORKERS=16 and now I do get 15 (not 16) inference requests in parallel.

I think the fact that SEARCH_NUM_WORKERS should be bigger than INFERENCE_NUM_WORKERS should be added to the documentation.

Originally created by @mratsim on GitHub (Sep 5, 2025). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/1890 I am trying the new `INFERENCE_NUM_WORKERS=16` flag to schedule 16 inference jobs in parallel. However when looking in the logs of my inference server (vLLM), I could only see 4 simultaneous queries in flight. It seems like it was because the rest of my config was ``` CRAWLER_NUM_WORKERS=4 SEARCH_NUM_WORKERS=4 ASSET_PREPROCESSING_NUM_WORKERS=4 ``` I changed `SEARCH_NUM_WORKERS=16` and now I do get 15 (not 16) inference requests in parallel. I think the fact that SEARCH_NUM_WORKERS should be bigger than INFERENCE_NUM_WORKERS should be added to the documentation.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/karakeep#1175

No description provided.

Rows
Columns