[GH-ISSUE #506] [FEATURE REQUEST] Ability to control/speed up sleep when processing empty queue

kerem commented

2026-03-02 05:19:48 +03:00

Owner

Originally created by @yousifh on GitHub (Jul 1, 2022).
Original GitHub issue: https://github.com/hibiken/asynq/issues/506

Originally assigned to: @hibiken on GitHub.

Is your feature request related to a problem? Please describe.
I have been benchmarking this library and measuring the latency of time to execution. The time between when a task is enqueued and when it gets picked up by a worker. I'm using the following parameters

7 workers processes
32 concurrency per worker
Enqueuing 40k tasks/min
Workers combined have ability to process 45k tasks/min

One thing I noticed is that the average time to execution latency was about 190ms which seemed quite high. Digging into the code the issue seems to be about when the processor sleeps when it hits an empty queue
github.com/hibiken/asynq@c70ff6a335/processor.go (L176-L184)

Since globally all the workers can process tasks at faster rate than they are being enqueued, they will frequently hit this 1 sec sleep period and increase the latency of picking up tasks.

Describe the solution you'd like
I prototyped couple of solutions to see if there are ways to lower the latency.

First solution was to make the sleep period configurable. Then I set it to 100ms. That seemed reasonable enough given that Redis operations by Asynq run very fast and 100ms gives it enough time for other background stuff to run like the scheduler.

Running the benchmark again, the latency now dropped to an average of 30ms. Looking at Redis metrics, on idle with 7 workers before the change the redis.cpu.user metric was around 2ms while after doing the change, it increased to 5ms. It doubled but still seems reasonable.

Of course this is idle conditions, but during normal operations the impact on the Redis server is negligible. So for example during the benchmark in both cases the redis.cpu.user metric was averaging 94ms, but the lower sleep period case had lower latency as an advantage.

Describe alternatives you've considered
I tested an alternative solution to have the sleep period for empty queues dynamically change between 1 sec and 100ms based on backoff of when it sees empty vs non-empty. So starts at 1 sec, decreases to 100ms for every successive call of non-empty queue. And keeps increasing all way back to 1 sec for calls when the queue is empty. But that just seemed to add more complexity to the code.

Additional context
If you think the first approach is reasonable, I can push a PR to allow users to set the sleep period while letting it default to 1 sec. So if an app doesn't need lower latency they can ignore it, while apps that need low latency can set that option at the cost of higher CPU utilization of the Redis server while it's idle.

If there are other solutions to explore, I'm more than happy to prototype and open a PR.

Originally created by @yousifh on GitHub (Jul 1, 2022). Original GitHub issue: https://github.com/hibiken/asynq/issues/506 Originally assigned to: @hibiken on GitHub. **Is your feature request related to a problem? Please describe.** I have been benchmarking this library and measuring the latency of time to execution. The time between when a task is enqueued and when it gets picked up by a worker. I'm using the following parameters - 7 workers processes - 32 concurrency per worker - Enqueuing 40k tasks/min - Workers combined have ability to process 45k tasks/min One thing I noticed is that the average time to execution latency was about `190ms` which seemed quite high. Digging into the code the issue seems to be about when the processor sleeps when it hits an empty queue https://github.com/hibiken/asynq/blob/c70ff6a335eed05735427567e8e9549745f43204/processor.go#L176-L184 Since globally all the workers can process tasks at faster rate than they are being enqueued, they will frequently hit this 1 sec sleep period and increase the latency of picking up tasks. **Describe the solution you'd like** I prototyped couple of solutions to see if there are ways to lower the latency. First solution was to make the sleep period configurable. Then I set it to `100ms`. That seemed reasonable enough given that Redis operations by Asynq run very fast and `100ms` gives it enough time for other background stuff to run like the scheduler. Running the benchmark again, the latency now dropped to an average of `30ms`. Looking at Redis metrics, on idle with 7 workers before the change the `redis.cpu.user` metric was around `2ms` while after doing the change, it increased to `5ms`. It doubled but still seems reasonable. Of course this is idle conditions, but during normal operations the impact on the Redis server is negligible. So for example during the benchmark in both cases the `redis.cpu.user` metric was averaging `94ms`, but the lower sleep period case had lower latency as an advantage. **Describe alternatives you've considered** I tested an alternative solution to have the sleep period for empty queues dynamically change between 1 sec and 100ms based on backoff of when it sees empty vs non-empty. So starts at 1 sec, decreases to 100ms for every successive call of non-empty queue. And keeps increasing all way back to 1 sec for calls when the queue is empty. But that just seemed to add more complexity to the code. **Additional context** If you think the first approach is reasonable, I can push a PR to allow users to set the sleep period while letting it default to 1 sec. So if an app doesn't need lower latency they can ignore it, while apps that need low latency can set that option at the cost of higher CPU utilization of the Redis server while it's idle. If there are other solutions to explore, I'm more than happy to prototype and open a PR.

kerem added the

enhancement

designing

labels

2026-03-02 05:19:48 +03:00

kerem commented

2026-03-02 05:19:49 +03:00

Author

Owner

@hibiken commented on GitHub (Jul 2, 2022):

@yousifh Thank you for creating this issue! And thanks for providing this insight, very helpful.

In an ideal world, Redis provides a command to do just what we want, but not likely(context: https://github.com/redis/redis/issues/1785).

Make the sleep time configurable sounds ok to me, but maybe we should brainstorm other alternatives first.
Let me get back to you in a week or so. In the meantime, if you have some other ideas, please let us know in this thread!

@hibiken commented on GitHub (Jul 2, 2022): @yousifh Thank you for creating this issue! And thanks for providing this insight, very helpful. In an ideal world, Redis provides a command to do just what we want, but not likely(context: https://github.com/redis/redis/issues/1785). Make the sleep time configurable sounds ok to me, but maybe we should brainstorm other alternatives first. Let me get back to you in a week or so. In the meantime, if you have some other ideas, please let us know in this thread!

kerem commented

2026-03-02 05:19:49 +03:00

Author

Owner

@yousifh commented on GitHub (Jul 11, 2022):

I was reading the issue linked and does MBRPOPLPUSH help if the dequeue happens in a Lua script? Wouldn't the Lua script itself still block the entire server while this command is waiting? This comment mentions it is planned but I can't find that mentioned in the docs

The other alternative approach is something like this

The idea being if the processor finds messages it resets the sleep to a baseline (100ms for example) Then if it doesn't find any messages it keeps incrementing it till it hits the max of 1 sec sleep. So while the server is usually busy it will try to fetch messages faster if it happens to get an empty queue error every once in a while. And while the server is idle, it will have the same old behaviour of 1 sec sleep.

The numbers of base sleep and increments can be tweaked. I just went for something simple. During my benchmark tests this made the average time to execution latency be 75ms which is an improvement over the default 190ms

It tries to do an adaptive flow control, but I tried to keep it as basic as possible to not complicate the code

@yousifh commented on GitHub (Jul 11, 2022): I was reading the issue linked and does `MBRPOPLPUSH` help if the dequeue happens in a Lua script? Wouldn't the Lua script itself still block the entire server while this command is waiting? This [comment](https://github.com/redis/redis/issues/1785#issuecomment-45205956) mentions it is planned but I can't find that mentioned in the docs The other alternative approach is something like [this](https://github.com/hibiken/asynq/compare/master...yousifh:asynq:faster-sleep) The idea being if the processor finds messages it resets the sleep to a baseline (100ms for example) Then if it doesn't find any messages it keeps incrementing it till it hits the max of 1 sec sleep. So while the server is usually busy it will try to fetch messages faster if it happens to get an empty queue error every once in a while. And while the server is idle, it will have the same old behaviour of 1 sec sleep. The numbers of base sleep and increments can be tweaked. I just went for something simple. During my benchmark tests this made the average time to execution latency be `75ms` which is an improvement over the default `190ms` It tries to do an adaptive flow control, but I tried to keep it as basic as possible to not complicate the code

kerem commented

2026-03-02 05:19:49 +03:00

Author

Owner

@HenryvanderVegte commented on GitHub (Dec 9, 2022):

Hi @hibiken ,

is there any progress on this? I'm also experiencing latency in execution time because of this. Both of the solutions @yousifh described (making the sleep time configurable or having a backoff on sleep) would work for my scenario I think.
Thanks!

@HenryvanderVegte commented on GitHub (Dec 9, 2022): Hi @hibiken , is there any progress on this? I'm also experiencing latency in execution time because of this. Both of the solutions @yousifh described (making the sleep time configurable or having a backoff on sleep) would work for my scenario I think. Thanks!

kerem referenced this issue

2026-03-02 05:54:31 +03:00

[PR #231] [MERGED] Fix scheduler #675

kerem referenced this issue

2026-03-07 22:11:55 +03:00

[PR #231] [MERGED] Fix scheduler #1682

kerem referenced this issue

2026-03-15 20:58:15 +03:00

[PR #231] [MERGED] Fix scheduler #2691

Rows
Columns

[GH-ISSUE #506] [FEATURE REQUEST] Ability to control/speed up sleep when processing empty queue #231