starred/asynq

Fork 0

mirror of https://github.com/hibiken/asynq.git synced 2026-04-25 23:15:51 +03:00

[GH-ISSUE #773] How to synchronize task information between asynq and database? #2403

New issue

Open

opened 2026-03-15 20:22:07 +03:00 by kerem · 5 comments

kerem commented

2026-03-15 20:22:07 +03:00

Owner

Originally created by @Milk-Tang on GitHub (Oct 31, 2023).
Original GitHub issue: https://github.com/hibiken/asynq/issues/773

Goal

I want to sync the task info to my database from asynq, so that I can do more complex indexing.

Attempts has made

I have thought about using asyncq.MiddlewareFunc, but the task obtained in my handler is not the final, such as the task state and retry count

Problem

Is there any way for me to achieve my goal?

Originally created by @Milk-Tang on GitHub (Oct 31, 2023). Original GitHub issue: https://github.com/hibiken/asynq/issues/773 ## Goal I want to **sync** the task info to my database from **asynq**, so that I can do more complex indexing. ## Attempts has made 1. I have thought about using **asyncq.MiddlewareFunc**, but the task obtained in my handler is **not the final**, such as the `task state` and `retry count` ## Problem Is there any way for me to achieve my goal?

kerem added the

question

label

2026-03-15 20:22:07 +03:00

kerem commented

2026-03-15 20:22:19 +03:00

Author

Owner

@kamikazechaser commented on GitHub (Oct 31, 2023):

You can use Inspector.GetTaskInfo if ypu know the ID. The TaskInfo returned has verbose info on the task.

@kamikazechaser commented on GitHub (Oct 31, 2023): You can use [Inspector.GetTaskInfo](https://pkg.go.dev/github.com/hibiken/asynq#Inspector.GetTaskInfo) if ypu know the ID. The TaskInfo returned has verbose info on the task.

kerem commented

2026-03-15 20:22:24 +03:00

Author

Owner

@Milk-Tang commented on GitHub (Oct 31, 2023):

You can use Inspector.GetTaskInfo if ypu know the ID. The TaskInfo returned has verbose info on the task.

Thank you for your reply, but using this cannot avoid only obtaining task that have not yet changed their state in the Handler func .
What I want is the task state after this worker consumption, as well as the retry count and other information

@Milk-Tang commented on GitHub (Oct 31, 2023): > You can use [Inspector.GetTaskInfo](https://pkg.go.dev/github.com/hibiken/asynq#Inspector.GetTaskInfo) if ypu know the ID. The TaskInfo returned has verbose info on the task. Thank you for your reply, but using this cannot avoid only obtaining task that have not yet changed their state in the **Handler func** . What I want is the task state after this worker consumption, as well as the `retry count` and other information

kerem commented

2026-03-15 20:22:29 +03:00

Author

Owner

@kamikazechaser commented on GitHub (Nov 1, 2023):

What I want is the task state after this worker consumption, as well as the retry count and other information

From what I can see TaskInfo contains the state and retried metadata.

How you determine when to collect this info is up to you. You can keep polling in a separate routine (Much easier if you pass your own ID when queueing). You could defer queue another Task which then fetches the TaskInfo within your main handler.

@kamikazechaser commented on GitHub (Nov 1, 2023): > What I want is the task state after this worker consumption, as well as the retry count and other information From what I can see [`TaskInfo`](https://pkg.go.dev/github.com/hibiken/asynq#TaskInfo) contains the state and retried metadata. How you determine when to collect this info is up to you. You can keep polling in a separate routine (Much easier if you pass your own ID when queueing). You could defer queue another Task which then fetches the TaskInfo within your main handler.

kerem commented

2026-03-15 20:22:34 +03:00

Author

Owner

@Milk-Tang commented on GitHub (Nov 2, 2023):

What I want is the task state after this worker consumption, as well as the retry count and other information

From what I can see TaskInfo contains the state and retried metadata.

How you determine when to collect this info is up to you. You can keep polling in a separate routine (Much easier if you pass your own ID when queueing). You could defer queue another Task which then fetches the TaskInfo within your main handler.

Okay, if there's no better way, I'll add the task ID to the queue after each task processing, and starting a new worker to synchronize the task information in this queue.

But there is also a problem with this: the task will be deleted after completion, and I must set the task retention time to reduce the probability of the task being deleted

@Milk-Tang commented on GitHub (Nov 2, 2023): > > What I want is the task state after this worker consumption, as well as the retry count and other information > > From what I can see [`TaskInfo`](https://pkg.go.dev/github.com/hibiken/asynq#TaskInfo) contains the state and retried metadata. > > How you determine when to collect this info is up to you. You can keep polling in a separate routine (Much easier if you pass your own ID when queueing). You could defer queue another Task which then fetches the TaskInfo within your main handler. Okay, if there's no better way, I'll add the task ID to the queue after each task processing, and starting a new worker to synchronize the task information in this queue. But there is also a problem with this: the task will **be deleted** after completion, and I **must** set the task retention time to reduce the probability of the task being deleted

kerem commented

2026-03-15 20:22:39 +03:00

Author

Owner

@kamikazechaser commented on GitHub (Nov 3, 2023):

and I must set the task retention time to reduce the probability of the task being deleted

Right. I don't think there is a way around this.

@kamikazechaser commented on GitHub (Nov 3, 2023): > and I must set the task retention time to reduce the probability of the task being deleted Right. I don't think there is a way around this.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/asynq#2403

No description provided.

Rows
Columns