[GH-ISSUE #1046] [BUG] Scheduler Tasks Frequently Skipped When Large Number of Handlers Registered #497

New issue

Open

opened 2026-03-02 05:21:35 +03:00 by kerem · 0 comments

kerem commented

2026-03-02 05:21:35 +03:00

Owner

Originally created by @lingcoder on GitHub (May 7, 2025).
Original GitHub issue: https://github.com/hibiken/asynq/issues/1046

Originally assigned to: @hibiken, @kamikazechaser on GitHub.

Describe the bug
We have encountered a critical issue where scheduled tasks managed by the Scheduler are often skipped and not executed as expected. Our application registers around 25 Scheduler tasks and about 30 regular worker handlers. The problem occurs even though we are running a single asynq instance with asynq.Config.Concurrency set to 20,000.

Environment (please complete the following information):

OS: Ubuntu 22.04
asynq package version: v0.25.1
Redis/Valkey version: 7.0.11

To Reproduce
Steps to reproduce the behavior:

Setup background processing with a single asynq instance, configuring asynq.Config.Concurrency to 20,000.
Register approximately 25 Scheduler tasks and 30 regular worker handlers.
Observe the execution over time.
Notice that several scheduled tasks are skipped and do not get executed as expected.

Expected behavior
All Scheduler tasks should be executed according to their schedule without being skipped.

Screenshots
If applicable, add screenshots or logs here to help explain your problem.

Additional context
We have tried adjusting the concurrency and monitoring resource usage, but the issue persists. This bug is causing critical scheduled jobs to be missed, which impacts our application's reliability.
We suspect that this issue may be related to the load on Redis or the application instance, and possibly the high number of registered scheduled tasks and task handlers. We did not experience this problem when the number of scheduled tasks was smaller, but it started to appear as the number increased recently. This is especially problematic for tasks that are scheduled to run only once per day—if they are skipped, it has a significant impact on our business.

Originally created by @lingcoder on GitHub (May 7, 2025). Original GitHub issue: https://github.com/hibiken/asynq/issues/1046 Originally assigned to: @hibiken, @kamikazechaser on GitHub. **Describe the bug** We have encountered a critical issue where scheduled tasks managed by the Scheduler are often skipped and not executed as expected. Our application registers around 25 Scheduler tasks and about 30 regular worker handlers. The problem occurs even though we are running a single asynq instance with `asynq.Config.Concurrency` set to 20,000. **Environment (please complete the following information):** - OS: Ubuntu 22.04 - `asynq` package version: v0.25.1 - Redis/Valkey version: 7.0.11 **To Reproduce** Steps to reproduce the behavior: 1. Setup background processing with a single asynq instance, configuring `asynq.Config.Concurrency` to 20,000. 2. Register approximately 25 Scheduler tasks and 30 regular worker handlers. 3. Observe the execution over time. 4. Notice that several scheduled tasks are skipped and do not get executed as expected. **Expected behavior** All Scheduler tasks should be executed according to their schedule without being skipped. **Screenshots** If applicable, add screenshots or logs here to help explain your problem. **Additional context** We have tried adjusting the concurrency and monitoring resource usage, but the issue persists. This bug is causing critical scheduled jobs to be missed, which impacts our application's reliability. We suspect that this issue may be related to the load on Redis or the application instance, and possibly the high number of registered scheduled tasks and task handlers. We did not experience this problem when the number of scheduled tasks was smaller, but it started to appear as the number increased recently. This is especially problematic for tasks that are scheduled to run only once per day—if they are skipped, it has a significant impact on our business.