[GH-ISSUE #869] [FEATURE REQUEST] Support for Fixed-Length Queues in Asynq #1443

New issue

Open

opened 2026-03-07 22:09:36 +03:00 by kerem · 1 comment

kerem commented

2026-03-07 22:09:36 +03:00

Owner

Originally created by @FogDong on GitHub (Apr 23, 2024).
Original GitHub issue: https://github.com/hibiken/asynq/issues/869

Originally assigned to: @hibiken on GitHub.

Is your feature request related to a problem? Please describe.
In many applications, particularly those involving resource-constrained environments or systems where old tasks need to be automatically discarded, having the ability to maintain fixed-length queues is crucial. This feature would allow developers to set a maximum size on queues, ensuring that the queue does not exceed a certain number of tasks. This is particularly useful for maintaining the latest N tasks and automatically discarding older tasks when new ones arrive.

Describe the solution you'd like
I propose that Asynq supports an option to limit the size of task queues. When a queue reaches its maximum size, the system should automatically remove the oldest task(s) upon the addition of new tasks. This would ensure that the queue remains at a fixed length.

The implementation could involve:

Extending the QueueConfig struct to include a new field, MaxSize, which defines the maximum number of tasks the queue can hold.
Modifying the task enqueue logic to check the current queue size against MaxSize. If the queue size is at its maximum, the oldest task(s) would be dequeued or dropped before the new task is enqueued.
Providing configuration options in the Asynq server settings to enable and configure this feature per queue.

Describe alternatives you've considered

Additional context
If this feature is accepted by the Asynq community, I would be very interested in contributing to its development. I believe this feature would not only benefit my use case but also be a valuable addition for many other users facing similar requirements.

Originally created by @FogDong on GitHub (Apr 23, 2024). Original GitHub issue: https://github.com/hibiken/asynq/issues/869 Originally assigned to: @hibiken on GitHub. **Is your feature request related to a problem? Please describe.** In many applications, particularly those involving resource-constrained environments or systems where old tasks need to be automatically discarded, having the ability to maintain fixed-length queues is crucial. This feature would allow developers to set a maximum size on queues, ensuring that the queue does not exceed a certain number of tasks. This is particularly useful for maintaining the latest N tasks and automatically discarding older tasks when new ones arrive. **Describe the solution you'd like** I propose that Asynq supports an option to limit the size of task queues. When a queue reaches its maximum size, the system should automatically remove the oldest task(s) upon the addition of new tasks. This would ensure that the queue remains at a fixed length. The implementation could involve: 1. Extending the QueueConfig struct to include a new field, MaxSize, which defines the maximum number of tasks the queue can hold. 2. Modifying the task enqueue logic to check the current queue size against MaxSize. If the queue size is at its maximum, the oldest task(s) would be dequeued or dropped before the new task is enqueued. 3. Providing configuration options in the Asynq server settings to enable and configure this feature per queue. **Describe alternatives you've considered** **Additional context** If this feature is accepted by the Asynq community, I would be very interested in contributing to its development. I believe this feature would not only benefit my use case but also be a valuable addition for many other users facing similar requirements.

kerem added the

enhancement

label

2026-03-07 22:09:36 +03:00

kerem commented

2026-03-07 22:09:37 +03:00

Author

Owner

@sanudatta11 commented on GitHub (Oct 25, 2024):

what happens to the old tasks in the queue, are u thinking of a DLQ approach where these tasks go to die or just removed completely with no trace?

@sanudatta11 commented on GitHub (Oct 25, 2024): what happens to the old tasks in the queue, are u thinking of a DLQ approach where these tasks go to die or just removed completely with no trace?

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/asynq#1443

No description provided.

Rows
Columns