[GH-ISSUE #373] [FEATURE REQUEST] Optional WAIT flag for Enqueue #2186

Open
opened 2026-03-15 19:35:50 +03:00 by kerem · 5 comments
Owner

Originally created by @danthegoodman1 on GitHub (Dec 31, 2021).
Original GitHub issue: https://github.com/hibiken/asynq/issues/373

Originally assigned to: @hibiken on GitHub.

I think it would be useful to have the option to use or not use WAIT {some number} with Enqueue such that we can have assurance that we durably wrote to Redis (when not using AOF).

Since Redis can lose writes before syncing to a replica, this is still a conern present without AOF.

Maybe an asynq.Wait(int) would be a parameter that would be passed to the client.Enqueue() function.

Originally created by @danthegoodman1 on GitHub (Dec 31, 2021). Original GitHub issue: https://github.com/hibiken/asynq/issues/373 Originally assigned to: @hibiken on GitHub. I think it would be useful to have the option to use or not use `WAIT {some number}` with Enqueue such that we can have assurance that we durably wrote to Redis (when not using AOF). Since Redis can lose writes before syncing to a replica, this is still a conern present without AOF. Maybe an `asynq.Wait(int)` would be a parameter that would be passed to the `client.Enqueue()` function.
Author
Owner

@hibiken commented on GitHub (Jan 1, 2022):

@danthegoodman1 Great feature request.

I believe synchronous replication is needed in some use cases and may be a common need.
Let me do a little bit more research around this to come up with a suggestion.

Any suggestion on the API is welcome!

<!-- gh-comment-id:1003569708 --> @hibiken commented on GitHub (Jan 1, 2022): @danthegoodman1 Great feature request. I believe synchronous replication is needed in some use cases and may be a common need. Let me do a little bit more research around this to come up with a suggestion. Any suggestion on the API is welcome!
Author
Owner

@vishjain commented on GitHub (Jul 8, 2022):

@hibiken Can I submit a PR for this? Thanks!

<!-- gh-comment-id:1178390880 --> @vishjain commented on GitHub (Jul 8, 2022): @hibiken Can I submit a PR for this? Thanks!
Author
Owner

@danthegoodman1 commented on GitHub (Oct 4, 2023):

bumping, this would be really helpful as temporal overhead is inappropriate for some use cases

<!-- gh-comment-id:1747224108 --> @danthegoodman1 commented on GitHub (Oct 4, 2023): bumping, this would be really helpful as temporal overhead is inappropriate for some use cases
Author
Owner

@kamikazechaser commented on GitHub (Oct 5, 2023):

Possibly needs a refactor for https://github.com/hibiken/asynq/blob/master/internal/rdb/rdb.go#L152..

numreplicas and timeout metadata need to be added to base.TaskMessage.

<!-- gh-comment-id:1748391388 --> @kamikazechaser commented on GitHub (Oct 5, 2023): Possibly needs a refactor for https://github.com/hibiken/asynq/blob/master/internal/rdb/rdb.go#L152.. `numreplicas` and `timeout` metadata need to be added to `base.TaskMessage`.
Author
Owner

@danthegoodman1 commented on GitHub (Oct 5, 2023):

For those wondering, in the meantime you can just either add AOF (for single host deployments) or you can use a service like upstash redis/aws memorydb that natively has both persistence and HA.

Edit: AOF might work if only a single host in a cluster gets it too, but I have never tested or seen talk of that failover pattern (but I assume based on async replication it would sync it across the cluster)

<!-- gh-comment-id:1748710365 --> @danthegoodman1 commented on GitHub (Oct 5, 2023): For those wondering, in the meantime you can just either add AOF (for single host deployments) or you can use a service like upstash redis/aws memorydb that natively has both persistence and HA. Edit: AOF might work if only a single host in a cluster gets it too, but I have never tested or seen talk of that failover pattern (but I assume based on async replication it would sync it across the cluster)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/asynq#2186
No description provided.