starred/asynq

Fork 0

mirror of https://github.com/hibiken/asynq.git synced 2026-04-26 07:25:56 +03:00

[GH-ISSUE #271] [QUESTION] How does priority queue work? #104

New issue

Closed

opened 2026-03-02 05:18:42 +03:00 by kerem · 4 comments

kerem commented

2026-03-02 05:18:42 +03:00

Owner

Originally created by @sujit-baniya on GitHub (May 6, 2021).
Original GitHub issue: https://github.com/hibiken/asynq/issues/271

Originally assigned to: @hibiken on GitHub.

Please suggest how the priority queue works in following scenario:

srv := asynq.NewServer(redis, asynq.Config{
    Concurrency: 10,
    Queues: map[string]int{
        "critical": 6,
        "default":  3,
        "low":      1,
    },
})

There are already 1K messages on low, When I send a message on critical while other messages are still enqueued, will this message consume at first or wait for first 1K to get completed?

I'm trying to understand how priority queue works

Originally created by @sujit-baniya on GitHub (May 6, 2021). Original GitHub issue: https://github.com/hibiken/asynq/issues/271 Originally assigned to: @hibiken on GitHub. Please suggest how the priority queue works in following scenario: ```go srv := asynq.NewServer(redis, asynq.Config{ Concurrency: 10, Queues: map[string]int{ "critical": 6, "default": 3, "low": 1, }, }) ``` There are already 1K messages on `low`, When I send a message on `critical` while other messages are still enqueued, will this message consume at first or wait for first 1K to get completed? I'm trying to understand how priority queue works

kerem

2026-03-02 05:18:42 +03:00

closed this issue
added the
enhancement
label

kerem commented

2026-03-02 05:18:43 +03:00

Author

Owner

@hibiken commented on GitHub (May 7, 2021):

@sujit-baniya thank you for asking!

It's documented here in godoc (https://pkg.go.dev/github.com/hibiken/asynq#Config). Please take a look at Queues and StrictPriority fields. Also, there's a wiki page explaining how queue priorities work: https://github.com/hibiken/asynq/wiki/Queue-Priority

Let me know if it's still unclear! That means the documentation can be improved.

@hibiken commented on GitHub (May 7, 2021): @sujit-baniya thank you for asking! It's documented here in godoc (https://pkg.go.dev/github.com/hibiken/asynq#Config). Please take a look at `Queues` and `StrictPriority` fields. Also, there's a wiki page explaining how queue priorities work: https://github.com/hibiken/asynq/wiki/Queue-Priority Let me know if it's still unclear! That means the documentation can be improved.

kerem commented

2026-03-02 05:18:43 +03:00

Author

Owner

@sujit-baniya commented on GitHub (May 8, 2021):

@hibiken Thank you for your response. I understand the Queue Priority. I am trying to understand this..

When a task in enqueued, is the task sorted on queue at the time of queuing?
When is actual sorting of tasks is done?

@sujit-baniya commented on GitHub (May 8, 2021): @hibiken Thank you for your response. I understand the Queue Priority. I am trying to understand this.. 1) When a task in enqueued, is the task sorted on queue at the time of queuing? 2) When is actual sorting of tasks is done?

kerem commented

2026-03-02 05:18:43 +03:00

Author

Owner

@hibiken commented on GitHub (May 8, 2021):

Once a task is in pending state, it's appending to a list in redis. This list has FIFO semantics.
If a task is scheduled to be processed in the future (via ProcessIn or ProcessAt option), the task is stored in a sorted-set in redis (score being the unix time of the process_at time).
Asynq internally manages these tasks state and move tasks from sorted-set to the FIFO list.

If you are interested in the redis key design, good place to start is base.go and rdb.go file.

@hibiken commented on GitHub (May 8, 2021): Once a task is in pending state, it's appending to a list in redis. This list has FIFO semantics. If a task is scheduled to be processed in the future (via `ProcessIn` or `ProcessAt` option), the task is stored in a sorted-set in redis (score being the unix time of the process_at time). Asynq internally manages these tasks state and move tasks from sorted-set to the FIFO list. If you are interested in the redis key design, good place to start is base.go and rdb.go file.

kerem commented

2026-03-02 05:18:43 +03:00

Author

Owner

@hibiken commented on GitHub (May 12, 2021):

Closing this issue.

@hibiken commented on GitHub (May 12, 2021): Closing this issue.

kerem referenced this issue

2026-03-02 05:54:16 +03:00

[PR #111] [MERGED] Add Use method to better support middlewares with ServeMux #610

kerem referenced this issue

2026-03-07 22:04:33 +03:00