starred/asynq

Fork 0

mirror of https://github.com/hibiken/asynq.git synced 2026-04-25 23:15:51 +03:00

[GH-ISSUE #363] [FEATURE REQUEST] retry policy enhancement #1173

New issue

Closed

opened 2026-03-07 22:06:49 +03:00 by kerem · 3 comments

kerem commented

2026-03-07 22:06:49 +03:00

Owner

Originally created by @sersh88 on GitHub (Dec 8, 2021).
Original GitHub issue: https://github.com/hibiken/asynq/issues/363

Originally assigned to: @hibiken on GitHub.

Feature request about retry policy.
Sometimes, when doing a task, it is clear that it will not be possible to complete it even on subsequent attempts.
So, it will be useful for a task handler to return special error type after which the task will be completely failed and no retries will be processed.
Also another enhancement is to be able to modify task payload in task handler. It's useful in some batch tasks, if I need to process i.e. 100 files and succeffully processed 99 of them, 1 was failed due to temporary network problem, so will be great if on next retry attempt task handler could use modifed payload with only 1 item to process.

Originally created by @sersh88 on GitHub (Dec 8, 2021). Original GitHub issue: https://github.com/hibiken/asynq/issues/363 Originally assigned to: @hibiken on GitHub. Feature request about retry policy. Sometimes, when doing a task, it is clear that it will not be possible to complete it even on subsequent attempts. So, it will be useful for a task handler to return special error type after which the task will be completely failed and no retries will be processed. Also another enhancement is to be able to modify task payload in task handler. It's useful in some batch tasks, if I need to process i.e. 100 files and succeffully processed 99 of them, 1 was failed due to temporary network problem, so will be great if on next retry attempt task handler could use modifed payload with only 1 item to process.

kerem

2026-03-07 22:06:49 +03:00

closed this issue
added the
enhancement
label

kerem commented

2026-03-07 22:06:50 +03:00

Author

Owner

@crossworth commented on GitHub (Dec 8, 2021):

Hello, you can already handle the error in different ways, see:
https://github.com/hibiken/asynq/wiki/Task-Retry#skip-retry

About changing the task payload, I don't is possible right now, but you can get the same result by enqueuing a new task with the payload changed and returning a skip retry on the task.

@crossworth commented on GitHub (Dec 8, 2021): Hello, you can already handle the error in different ways, see: https://github.com/hibiken/asynq/wiki/Task-Retry#skip-retry About changing the task payload, I don't is possible right now, but you can get the same result by enqueuing a new task with the payload changed and returning a skip retry on the task.

kerem commented

2026-03-07 22:06:50 +03:00

Author

Owner

@sersh88 commented on GitHub (Dec 8, 2021):

Thanks, somehow I missed this SkipRetry error.

@sersh88 commented on GitHub (Dec 8, 2021): Thanks, somehow I missed this SkipRetry error.

kerem commented

2026-03-07 22:06:50 +03:00

Author

Owner

@hibiken commented on GitHub (Dec 8, 2021):

@sersh88 thank you for creating this issue and thank you @crossworth for jumping in!

For the first question, please see @crossworth's comment above.

For the second question, payload should be immutable IMO. We introduced task Result in v0.19. you could probably write your partial result and read the result in the subsequent attempts.

Example:

func MyHandler(ctx context.Context, task *asynq.Task) error {
    // Read previous result: GetResult function is not available right now, but we can implement if needed.
    prevRes := asynq.GetResult(ctx) 
    res, err := DoStuff(ctx, task, prevResult)
    if err != nil {
        // Write partial result
        if _, err := task.ResultWriter().Write(res); err != nil {
           return fmt.Errorf("failed to write partial result: %v", err)
        }
        return err
    }
    if _, err := task.ResultWriter().Write(res); err != nil {
        return fmt.Errorf("failed to write result: %v", err)
    }
    return nil
}

@hibiken commented on GitHub (Dec 8, 2021): @sersh88 thank you for creating this issue and thank you @crossworth for jumping in! For the first question, please see @crossworth's comment above. For the second question, payload should be immutable IMO. We introduced task `Result` in v0.19. you could probably write your partial result and read the result in the subsequent attempts. Example: ```go func MyHandler(ctx context.Context, task *asynq.Task) error { // Read previous result: GetResult function is not available right now, but we can implement if needed. prevRes := asynq.GetResult(ctx) res, err := DoStuff(ctx, task, prevResult) if err != nil { // Write partial result if _, err := task.ResultWriter().Write(res); err != nil { return fmt.Errorf("failed to write partial result: %v", err) } return err } if _, err := task.ResultWriter().Write(res); err != nil { return fmt.Errorf("failed to write result: %v", err) } return nil } ```

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/asynq#1173

No description provided.

Rows
Columns