[GH-ISSUE #389] Is it necessary for the `LREM` argument count = 0 ? #1185

New issue

Closed

opened 2026-03-07 22:06:56 +03:00 by kerem · 1 comment

kerem commented

2026-03-07 22:06:56 +03:00

Owner

Originally created by @ryan961 on GitHub (Jan 28, 2022).
Original GitHub issue: https://github.com/hibiken/asynq/issues/389

Originally assigned to: @hibiken on GitHub.

LREM: Removes the first count occurrences of elements equal to element from the list stored at key
....
count = 0: Remove all elements equal to element.

In the case of a certain amount of data, it will cause a lot of slow queries, which will cause Redis CPU to explode, and this will enter a vicious cycle that can't be handled.
github.com/hibiken/asynq@dff2e3a336/internal/rdb/rdb.go (L312)
github.com/hibiken/asynq@dff2e3a336/internal/rdb/rdb.go (L344)
...

Originally created by @ryan961 on GitHub (Jan 28, 2022). Original GitHub issue: https://github.com/hibiken/asynq/issues/389 Originally assigned to: @hibiken on GitHub. > `LREM`: Removes the first count occurrences of elements equal to element from the list stored at key .... > count = 0: Remove all elements equal to element. In the case of a certain amount of data, it will cause a lot of slow queries, which will cause Redis CPU to explode, and this will enter a vicious cycle that can't be handled. https://github.com/hibiken/asynq/blob/dff2e3a33639338c648599bf6ea83f835feec4bf/internal/rdb/rdb.go#L312 https://github.com/hibiken/asynq/blob/dff2e3a33639338c648599bf6ea83f835feec4bf/internal/rdb/rdb.go#L344 ...

kerem

2026-03-07 22:06:56 +03:00

closed this issue
added the
question
label

kerem commented

2026-03-07 22:06:57 +03:00

Author

Owner

@hibiken commented on GitHub (Jan 28, 2022):

@ryalu Thank you for reporting this issue!

These LREM operations are executed against the asynq:{qname}:active list which can only grow up to the number specified by Config.Concurrency so should be bounded to smallish number (e.g. ~100 or so).

But you made a good point, that this count may only need to be set to 1 since we only need to remove just one element from the list during normal operation. I didn't put too much thoughts there but I wanted to guarantee that we remove the id from the list. We can run benchmark tests with different concurrency settings, and see if changing this to 1 would make it more performant.

@hibiken commented on GitHub (Jan 28, 2022): @ryalu Thank you for reporting this issue! These `LREM` operations are executed against the `asynq:{qname}:active` list which can only grow up to the number specified by `Config.Concurrency` so should be bounded to smallish number (e.g. ~100 or so). But you made a good point, that this count may only need to be set to 1 since we only need to remove just one element from the list during normal operation. I didn't put too much thoughts there but I wanted to guarantee that we remove the id from the list. We can run benchmark tests with different concurrency settings, and see if changing this to 1 would make it more performant.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/asynq#1185

No description provided.

Rows
Columns

[GH-ISSUE #389] Is it necessary for the LREM argument count = 0 ? #1185

[GH-ISSUE #389] Is it necessary for the `LREM` argument count = 0 ? #1185