starred/asynq

Fork 0

mirror of https://github.com/hibiken/asynq.git synced 2026-04-25 23:15:51 +03:00

[GH-ISSUE #907] [BUG] Asynq task didn't got processed by the worker even after many hours of getting picked up #2475

New issue

Open

opened 2026-03-15 20:36:37 +03:00 by kerem · 5 comments

kerem commented

2026-03-15 20:36:37 +03:00

Owner

Originally created by @gautam-tiwari on GitHub (Aug 1, 2024).
Original GitHub issue: https://github.com/hibiken/asynq/issues/907

Originally assigned to: @hibiken on GitHub.

Describe the bug

I have build an application in golang with uses asynq.
I have around 80-100 workers and 4 queues.
For the past few weeks I have been encountering an issue in which the asynq worker picks up the tasks(as the state of the task is active) but it doesn't get processed.
Even when the workers are free it doesn't gets picked up.

To Reproduce
Steps to reproduce the behavior (Code snippets if applicable):

Implement 4 different queues and about 80-100 workers.
Set the priority of the queues as 5, 7, 10 and 10.
Tasks should be such that they would take an average of 5 minutes per task.
Enqueue a lot of tasks(around 5000) related to the least priority queue.
Enqueue lesser tasks(around 1000) releated to high priority queue.
Then observe that a few tasks of the low priority queue won't even get picked up.

Expected behavior

Workers should pick up the task even it is of low priority after a certain amount of time.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

OS:Linux
Version of asynq package v0.24.1

Additional context

One of the other possiblities would be that the workers picked up the task processed it but the redis key didn't got updated.

Originally created by @gautam-tiwari on GitHub (Aug 1, 2024). Original GitHub issue: https://github.com/hibiken/asynq/issues/907 Originally assigned to: @hibiken on GitHub. **Describe the bug** - I have build an application in golang with uses asynq. - I have around 80-100 workers and 4 queues. - For the past few weeks I have been encountering an issue in which the asynq worker picks up the tasks(as the state of the task is `active`) but it doesn't get processed. - Even when the workers are free it doesn't gets picked up. **To Reproduce** Steps to reproduce the behavior (Code snippets if applicable): 1. Implement 4 different queues and about 80-100 workers. 2. Set the priority of the queues as 5, 7, 10 and 10. 3. Tasks should be such that they would take an average of 5 minutes per task. 4. Enqueue a lot of tasks(around 5000) related to the least priority queue. 5. Enqueue lesser tasks(around 1000) releated to high priority queue. 6. Then observe that a few tasks of the low priority queue won't even get picked up. **Expected behavior** - Workers should pick up the task even it is of low priority after a certain amount of time. **Screenshots** If applicable, add screenshots to help explain your problem. **Environment (please complete the following information):** - OS:Linux - Version of `asynq` package v0.24.1 **Additional context** - One of the other possiblities would be that the workers picked up the task processed it but the redis key didn't got updated.

kerem added the

bug

label

2026-03-15 20:36:37 +03:00

kerem commented

2026-03-15 20:36:49 +03:00

Author

Owner

@of89013 commented on GitHub (Aug 22, 2024):

我估计我的错误也是这个情况

@of89013 commented on GitHub (Aug 22, 2024): 我估计我的错误也是这个情况

kerem commented

2026-03-15 20:36:54 +03:00

Author

Owner

@gautam-tiwari commented on GitHub (Aug 22, 2024):

Hi @of89013 How is that so?

@gautam-tiwari commented on GitHub (Aug 22, 2024): Hi @of89013 How is that so?

kerem commented

2026-03-15 20:36:59 +03:00

Author

Owner

@gautam-tiwari commented on GitHub (Aug 22, 2024):

There also exists one issue where the "msg" subkey inside the redis asynq key map gets lost somehow. I'll attach a ss for reference.

Invalid asynq key(missing msg field)

Valid asynq key

@gautam-tiwari commented on GitHub (Aug 22, 2024): There also exists one issue where the "msg" subkey inside the redis asynq key map gets lost somehow. I'll attach a ss for reference. # Invalid asynq key(missing msg field) ![Screenshot 2024-08-16 at 1 35 40 PM](https://github.com/user-attachments/assets/aeff2dba-a5e2-46b7-bde6-8deac6b16a07) # Valid asynq key <img width="1471" alt="Screenshot 2024-08-22 at 10 45 37 PM" src="https://github.com/user-attachments/assets/1960321a-67e3-42c7-8b18-c673d778e1c4">

kerem commented

2026-03-15 20:37:04 +03:00

Author

Owner

@pior commented on GitHub (Oct 31, 2024):

Are you using Redis Cluster?

@pior commented on GitHub (Oct 31, 2024): Are you using Redis Cluster?

kerem commented

2026-03-15 20:37:10 +03:00

Author

Owner

@cobain commented on GitHub (Nov 1, 2024):

it should the same issue with #952

@cobain commented on GitHub (Nov 1, 2024): it should the same issue with #952

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/asynq#2475

No description provided.

Rows
Columns