[GH-ISSUE #1087] [FEATURE REQUEST] GroupMaxSize per queue #2547

New issue

Open

opened 2026-03-15 20:49:57 +03:00 by kerem · 1 comment

kerem commented

2026-03-15 20:49:57 +03:00

Owner

Originally created by @liron-navon on GitHub (Dec 2, 2025).
Original GitHub issue: https://github.com/hibiken/asynq/issues/1087

Originally assigned to: @hibiken, @kamikazechaser on GitHub.

Is your feature request related to a problem? Please describe.
I work with multiple queues with one asynq server, I have multiple queues and multiple groups, we can pass "GroupMaxSize" to the asynq server, but this applies to ALL the groups, I'd like to be able to set a max size per group or per queue, since some groups might make sense to have 500 max size and others 100 max size.

Describe the solution you'd like
I'd suggest an interface like so where we can follow the "Queues" options style by setting a "Groups"
and each group can have its own options, or if it is too complicated just pass an int for the max size?

srv := asynq.NewServerFromRedisClient(
	tdb,
	asynq.Config{
		Concurrency:      10,
		GroupAggregator:  asynq.GroupAggregatorFunc(MyAggregate),
		GroupMaxDelay:    2 * time.Minute, // used as default
		GroupGracePeriod: 1 * time.Minute,  // used as default
		GroupMaxSize:     100,  // used as default
		Queues: map[string]int{
			"queue_A":         6,
			"queue_B":         2,
			"queue_C":         2
		},
		// OPTION 1
		Groups: map[string]GroupOptions{
			"queue_A": GroupOptions{
				GroupMaxDelay:    10 * time.Minute, 
				GroupGracePeriod: 2 * time.Minute,
				GroupMaxSize:     1000,
			},
			"queue_B": GroupOptions{
				GroupMaxSize:     300,
			},
		}
		// OPTION 2
		Groups: map[string]int{
			"queue_A": 1000,
			"queue_B": 300,
		}
	},
)

type GroupOptions struct {
	GroupMaxDelay      time.Duration
	GroupGracePeriod  time.Duration
	GroupMaxSize         int
}

Describe alternatives you've considered
alternative solutions are to create multiple instances of the asynq server, but that have performance panelties and makes the code more complex.

Using a single MaxSize for all the groups, but that reduces my options to optimize bulk insertions and bulk processing.

Originally created by @liron-navon on GitHub (Dec 2, 2025). Original GitHub issue: https://github.com/hibiken/asynq/issues/1087 Originally assigned to: @hibiken, @kamikazechaser on GitHub. **Is your feature request related to a problem? Please describe.** I work with multiple queues with one asynq server, I have multiple queues and multiple groups, we can pass "GroupMaxSize" to the asynq server, but this applies to ALL the groups, I'd like to be able to set a max size per group or per queue, since some groups might make sense to have 500 max size and others 100 max size. **Describe the solution you'd like** I'd suggest an interface like so where we can follow the "Queues" options style by setting a "Groups" and each group can have its own options, or if it is too complicated just pass an int for the max size? ```go srv := asynq.NewServerFromRedisClient( tdb, asynq.Config{ Concurrency: 10, GroupAggregator: asynq.GroupAggregatorFunc(MyAggregate), GroupMaxDelay: 2 * time.Minute, // used as default GroupGracePeriod: 1 * time.Minute, // used as default GroupMaxSize: 100, // used as default Queues: map[string]int{ "queue_A": 6, "queue_B": 2, "queue_C": 2 }, // OPTION 1 Groups: map[string]GroupOptions{ "queue_A": GroupOptions{ GroupMaxDelay: 10 * time.Minute, GroupGracePeriod: 2 * time.Minute, GroupMaxSize: 1000, }, "queue_B": GroupOptions{ GroupMaxSize: 300, }, } // OPTION 2 Groups: map[string]int{ "queue_A": 1000, "queue_B": 300, } }, ) type GroupOptions struct { GroupMaxDelay time.Duration GroupGracePeriod time.Duration GroupMaxSize int } ``` **Describe alternatives you've considered** alternative solutions are to create multiple instances of the asynq server, but that have performance panelties and makes the code more complex. Using a single MaxSize for all the groups, but that reduces my options to optimize bulk insertions and bulk processing.

kerem added the

enhancement

label

2026-03-15 20:49:57 +03:00

kerem commented

2026-03-15 20:50:08 +03:00

Author

Owner

@faleksic commented on GitHub (Jan 29, 2026):

I'd like to take a stab at this if this issue is open and something you want to support.
Basically I would follow the suggested config structure in this ticket and made groups optional in order to keep it backward compatible. I am open to other suggestions if you have some

@faleksic commented on GitHub (Jan 29, 2026): I'd like to take a stab at this if this issue is open and something you want to support. Basically I would follow the suggested config structure in this ticket and made groups optional in order to keep it backward compatible. I am open to other suggestions if you have some

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/asynq#2547

No description provided.

Rows
Columns