[GH-ISSUE #332] [QUESTION] Does Asynq support multiple servers and processes? Is there a locking mechanism? #147

New issue

Closed

opened 2026-03-02 05:19:07 +03:00 by kerem · 3 comments

kerem commented

2026-03-02 05:19:07 +03:00

Owner

Originally created by @SuperL89 on GitHub (Sep 30, 2021).
Original GitHub issue: https://github.com/hibiken/asynq/issues/332

Originally created by @SuperL89 on GitHub (Sep 30, 2021). Original GitHub issue: https://github.com/hibiken/asynq/issues/332

kerem

2026-03-02 05:19:07 +03:00

closed this issue
added the
question
label

kerem commented

2026-03-02 05:19:08 +03:00

Author

Owner

@hibiken commented on GitHub (Oct 1, 2021):

@zhaitailang thank you for the question.
Yes, you can run multiple server processes to read from the same queues. For the most part, no locks are needed since we execute Lua script on Redis which is atomic in terms of execution of the script (see https://redis.io/commands/eval#atomicity-of-scripts).

@hibiken commented on GitHub (Oct 1, 2021): @zhaitailang thank you for the question. Yes, you can run multiple server processes to read from the same queues. For the most part, no locks are needed since we execute Lua script on Redis which is atomic in terms of execution of the script (see https://redis.io/commands/eval#atomicity-of-scripts).

kerem commented

2026-03-02 05:19:08 +03:00

Author

Owner

@gschurck commented on GitHub (Sep 10, 2023):

@hibiken I am wondering, in that case how Redis decides which server to choose to send the task to ?

@gschurck commented on GitHub (Sep 10, 2023): @hibiken I am wondering, in that case how Redis decides which server to choose to send the task to ?

kerem commented

2026-03-02 05:19:09 +03:00

Author

Owner

@kenshin579 commented on GitHub (May 24, 2024):

I'm developing a server program to allow distributed scheduling across multiple servers, and currently, I'm considering the asynq library. I'm a bit unclear on whether I can run multiple periodic schedulers as multiple instances.

In my program, I need to guaranteed at least one execution of a task even if there is more than one server instance is running

In your answer to this issue, you said that it's okay to run on multiple servers without locking. But, in the wiki doc mentioned below, it seems like I have to run only one periodic scheduler which means I cannot run my program in multiple servers.

Should I just make sure that only one instance is running the periodic scheduler? Is my understanding correct?

"You have to ensure only a single scheduler is running for a schedule at a time, otherwise you’d end up with duplicate tasks. Using a centralized approach means the schedule doesn’t have to be synchronized, and the service can operate without using locks."

Reference: https://github.com/hibiken/asynq/wiki/Periodic-Tasks

@kenshin579 commented on GitHub (May 24, 2024): I'm developing a server program to allow distributed scheduling across multiple servers, and currently, I'm considering the asynq library. I'm a bit unclear on whether I can run multiple periodic schedulers as multiple instances. - In my program, I need to guaranteed at least one execution of a task even if there is more than one server instance is running In your answer to this issue, you said that it's okay to run on multiple servers without locking. But, in the wiki doc mentioned below, it seems like I have to run only one periodic scheduler which means I cannot run my program in multiple servers. Should I just make sure that only one instance is running the periodic scheduler? Is my understanding correct? > "You have to ensure only a single scheduler is running for a schedule at a time, otherwise you’d end up with duplicate tasks. Using a centralized approach means the schedule doesn’t have to be synchronized, and the service can operate without using locks." Reference: https://github.com/hibiken/asynq/wiki/Periodic-Tasks

kerem referenced this issue

2026-03-02 05:54:22 +03:00

[PR #148] [MERGED] Simplify Logger interface #634

kerem referenced this issue

2026-03-07 22:04:57 +03:00

[GH-ISSUE #147] [FEATURE REQUEST] Change Logger interface #1058

kerem referenced this issue

2026-03-07 22:11:42 +03:00

[PR #148] [MERGED] Simplify Logger interface #1644

kerem referenced this issue

2026-03-15 19:01:03 +03:00

[GH-ISSUE #147] [FEATURE REQUEST] Change Logger interface #2068