starred/asynq

Fork 0

mirror of https://github.com/hibiken/asynq.git synced 2026-04-26 07:25:56 +03:00

[GH-ISSUE #1040] [BUG] Handler not found in multi server #1509

New issue

Closed

opened 2026-03-07 22:10:46 +03:00 by kerem · 3 comments

kerem commented

2026-03-07 22:10:46 +03:00

Owner

Originally created by @newarifrh on GitHub (Mar 28, 2025).
Original GitHub issue: https://github.com/hibiken/asynq/issues/1040

Originally assigned to: @hibiken, @kamikazechaser on GitHub.

Describe the bug
I have a microservices-based application where each service runs an Asynq server using the same Redis instance. I chose the same Redis instance across all services because I need to enqueue tasks from Service A to be processed by Service B.

However, when Service A enqueues a task for Service B, the task is successfully sent. But Service B does not receive the task.

Checking Asynq monitoring, the task details show "Handler not found". Interestingly, when I manually retry the task via the retry tab, it eventually runs successfully after multiple attempts.

Each service is running in a Docker container, and I am using Docker Swarm as the orchestrator.

Environment (please complete the following information):

OS: Linux
v0.25.1
Redis

Screenshots

Originally created by @newarifrh on GitHub (Mar 28, 2025). Original GitHub issue: https://github.com/hibiken/asynq/issues/1040 Originally assigned to: @hibiken, @kamikazechaser on GitHub. **Describe the bug** I have a microservices-based application where each service runs an Asynq server using the same Redis instance. I chose the same Redis instance across all services because I need to enqueue tasks from Service A to be processed by Service B. However, when Service A enqueues a task for Service B, the task is successfully sent. But Service B does not receive the task. Checking Asynq monitoring, the task details show "Handler not found". Interestingly, when I manually retry the task via the retry tab, it eventually runs successfully after multiple attempts. Each service is running in a Docker container, and I am using Docker Swarm as the orchestrator. **Environment (please complete the following information):** - OS: Linux - v0.25.1 - Redis **Screenshots** <img width="1352" alt="Image" src="https://github.com/user-attachments/assets/0e694775-ec10-430a-aee3-f7b79dfb8cd2" /> <img width="1296" alt="Image" src="https://github.com/user-attachments/assets/f11594ba-bcf9-4d5f-aa5f-c8df16652507" />

kerem

2026-03-07 22:10:46 +03:00

closed this issue
added the
bug
label

kerem commented

2026-03-07 22:10:49 +03:00

Author

Owner

@newarifrh commented on GitHub (Mar 29, 2025):

I have multiple Asynq server instances using the same Redis instance.

For example:
Server Alfa has handlers for Task A and Task B.
Server Beta only has a handler for Task C.

When I enqueue Task C from Server Alfa, Asynq tries to find the handler across all connected servers using the same Redis instance. This results in a "Handler not found" error because Server Alfa does not have a handler for Task C. The task keeps retrying until it eventually gets picked up by Server Beta, which actually has the correct handler.

As a solution, I decided to use separate Redis instances for each server to ensure tasks are only consumed by the intended server.

Is there a way to explicitly specify which server should process a task? For example, by setting the target server’s host or PID during enqueue?

@newarifrh commented on GitHub (Mar 29, 2025): I have multiple Asynq server instances using the same Redis instance. For example: Server Alfa has handlers for Task A and Task B. Server Beta only has a handler for Task C. When I enqueue Task C from Server Alfa, Asynq tries to find the handler across all connected servers using the same Redis instance. This results in a "Handler not found" error because Server Alfa does not have a handler for Task C. The task keeps retrying until it eventually gets picked up by Server Beta, which actually has the correct handler. As a solution, I decided to use separate Redis instances for each server to ensure tasks are only consumed by the intended server. Is there a way to explicitly specify which server should process a task? For example, by setting the target server’s host or PID during enqueue?

kerem commented

2026-03-07 22:10:49 +03:00

Author

Owner

@mty2015 commented on GitHub (Apr 5, 2025):

You can specify different Server to execute different queues to achieve the effect you want, for example:

Server Alfa Code:

	srvA := asynq.NewServer(
		asynq.Config{
			Queues: map[string]int{
				"queue-a": 1,
				"queue-b":  1,
			},
		},
	)

Server Beta Code:

	srvB := asynq.NewServer(
		asynq.Config{
			Queues: map[string]int{
				"queue-c": 1
			},
		},
	)

However, it is not recommended that you do this. The Queues option is used to control the priority of queues. From the business scenario you described, your server A and server B should be two different services and should use two independent Redis instances.

@mty2015 commented on GitHub (Apr 5, 2025): You can specify different `Server` to execute different queues to achieve the effect you want, for example: Server Alfa Code: ```go srvA := asynq.NewServer( asynq.Config{ Queues: map[string]int{ "queue-a": 1, "queue-b": 1, }, }, ) ``` Server Beta Code: ```go srvB := asynq.NewServer( asynq.Config{ Queues: map[string]int{ "queue-c": 1 }, }, ) ``` However, it is not recommended that you do this. The Queues option is used to control the priority of queues. From the business scenario you described, your server A and server B should be two different services and should use two independent Redis instances.

kerem commented

2026-03-07 22:10:49 +03:00

Author

Owner

@newarifrh commented on GitHub (Apr 10, 2025):

Thank you @mty2015, I already implement independent redis instances

@newarifrh commented on GitHub (Apr 10, 2025): Thank you @mty2015, I already implement independent redis instances

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/asynq#1509

No description provided.

Rows
Columns