starred/asynq

Fork 0

mirror of https://github.com/hibiken/asynq.git synced 2026-04-26 07:25:56 +03:00

[GH-ISSUE #586] [BUG] Unable to route the task to the correct worker #1299

New issue

Closed

opened 2026-03-07 22:08:24 +03:00 by kerem · 2 comments

kerem commented

2026-03-07 22:08:24 +03:00

Owner

Originally created by @s0und0fs1lence on GitHub (Dec 14, 2022).
Original GitHub issue: https://github.com/hibiken/asynq/issues/586

Originally assigned to: @hibiken on GitHub.

Describe the bug
Hi, i'm using asynq with 3 worker servers, and all work fine.
Now i'm trying to add a 4° server to the cluster, but this server process different type of tasks.
I'm noticing that sometimes (actually really often), when i enqueue a new task that could be handled only by the new worker, the task actually goes scheduled by one of the "old" workers; this is problematic because i want this new tasks to go only in the servers that can handle them.
Also the problem is also in the opposite direction because sometimes, a "old" task goes into new server and the problem is the same;

To Reproduce
Steps to reproduce the behavior (Code snippets if applicable):

Setup a cluster with 2 workers, 1 that can handle task:A and 1 that can handle task:B
Enqueue multiple time task:A
See that the task A sometimes is scheduled on server that only handle task:B

Expected behavior
The task should be scheduled only on the servers that can handle the task. This could be achieved by labeling the workers and giving the ability to the client to specifiy a worker label when enqueuing, or by trying all the servers in the cluster untill one accept the task or all refuse it.

Screenshots

Environment (please complete the following information):

OS: [e.g. MacOS, Linux] Linux
Version of asynq package v0.23.0

Originally created by @s0und0fs1lence on GitHub (Dec 14, 2022). Original GitHub issue: https://github.com/hibiken/asynq/issues/586 Originally assigned to: @hibiken on GitHub. **Describe the bug** Hi, i'm using asynq with 3 worker servers, and all work fine. Now i'm trying to add a 4° server to the cluster, but this server process different type of tasks. I'm noticing that sometimes (actually really often), when i enqueue a new task that could be handled only by the new worker, the task actually goes scheduled by one of the "old" workers; this is problematic because i want this new tasks to go only in the servers that can handle them. Also the problem is also in the opposite direction because sometimes, a "old" task goes into new server and the problem is the same; ![image](https://user-images.githubusercontent.com/60502183/207567060-6c1428c4-d4e7-436e-aa12-83f51904eed5.png) **To Reproduce** Steps to reproduce the behavior (Code snippets if applicable): 1. Setup a cluster with 2 workers, 1 that can handle task:A and 1 that can handle task:B 2. Enqueue multiple time task:A 3. See that the task A sometimes is scheduled on server that only handle task:B **Expected behavior** The task should be scheduled only on the servers that can handle the task. This could be achieved by labeling the workers and giving the ability to the client to specifiy a worker label when enqueuing, or by trying all the servers in the cluster untill one accept the task or all refuse it. **Screenshots** ![image](https://user-images.githubusercontent.com/60502183/207567789-8d5cde5b-947b-402f-a28c-c1b5dff61220.png) **Environment (please complete the following information):** - OS: [e.g. MacOS, Linux] Linux - Version of `asynq` package v0.23.0

kerem

2026-03-07 22:08:24 +03:00

closed this issue
added the
bug
label

kerem commented

2026-03-07 22:08:25 +03:00

Author

Owner

@hibiken commented on GitHub (Dec 18, 2022):

You should create a separate queue for each task type and configure worker server to subscribe to appropriate queue(s).

Example:

// Enqueue taskA to queue called "foo"
_, err := client.Enqueue(taskA, asynq.Queue("foo"))

// Enqueue taskB to queue called "bar"
_, err := client.Enqueue(taskB, asynq.Queue("bar"))

On the worker server side, you configure it accordingly:

srv := asynq.NewServer(redisConnOpt, asynq.Config{
     Queues: map[int]string{ "foo": 1 }, // this server will only process tasks enqueued to a queue called "foo"
})

Please let me know if you have additional questions :)

@hibiken commented on GitHub (Dec 18, 2022): You should create a separate queue for each task type and configure worker server to subscribe to appropriate queue(s). Example: ```go // Enqueue taskA to queue called "foo" _, err := client.Enqueue(taskA, asynq.Queue("foo")) // Enqueue taskB to queue called "bar" _, err := client.Enqueue(taskB, asynq.Queue("bar")) ``` On the worker server side, you configure it accordingly: ```go srv := asynq.NewServer(redisConnOpt, asynq.Config{ Queues: map[int]string{ "foo": 1 }, // this server will only process tasks enqueued to a queue called "foo" }) ``` Please let me know if you have additional questions :)

kerem commented

2026-03-07 22:08:25 +03:00

Author

Owner

@s0und0fs1lence commented on GitHub (Dec 20, 2022):

Thank you,
whit this approach actually i solved my problem;

Is it possible to add this information in the wiki? if it was there i could have figured this out without opening an issue.

Thanks, and congratulation for the awesome tool you developed!

@s0und0fs1lence commented on GitHub (Dec 20, 2022): Thank you, whit this approach actually i solved my problem; Is it possible to add this information in the wiki? if it was there i could have figured this out without opening an issue. Thanks, and congratulation for the awesome tool you developed!

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/asynq#1299

No description provided.

Rows
Columns