[GH-ISSUE #501] [QUESTION] - Looking for some suggestions on data aggregation #1245

New issue

Closed

opened 2026-03-07 22:07:56 +03:00 by kerem · 1 comment

kerem commented

2026-03-07 22:07:56 +03:00

Owner

Originally created by @sujit-baniya on GitHub (Jun 22, 2022).
Original GitHub issue: https://github.com/hibiken/asynq/issues/501

Discussed in https://github.com/hibiken/asynq/discussions/500

^{Originally posted by sujit-baniya June 22, 2022}
Hello,
I've been trying to implement Workflow using this library.

https://github.com/sujit-baniya/asynq/blob/master/flow.go

For the purpose, I've implemented flow and this example works for simple Edge and Node implementation

https://github.com/sujit-baniya/asynq/blob/master/examples/asynq_worker.go

The example works for

An edge with single input and single output
A loop with single input and single output

I'm trying find solution where a edge/loop has multiple output. Since there would be multiple output, at some point it is required to combine such data.

For e.g.
NodeA with key A
NodeB with key B (generates phone_valid field)
NodeC with key C (generates email_valid field)
NodeD with key D

Edge is created from A -> B
Edge is created from A -> C
Edge is created from B -> D
Edge is created from C -> D

I'm trying to get aggregated data received from above flow.

If I pass data like {"phone": "123456789", "email": "abc@example"} to Node A, I would be expecting {"phone": "123456789", "email": "abc@example", "phone_valid": true, "email_valid": true} on Node D

Please suggest some ideas.

Originally created by @sujit-baniya on GitHub (Jun 22, 2022). Original GitHub issue: https://github.com/hibiken/asynq/issues/501 ### Discussed in https://github.com/hibiken/asynq/discussions/500 <div type='discussions-op-text'> <sup>Originally posted by **sujit-baniya** June 22, 2022</sup> Hello, I've been trying to implement Workflow using this library. https://github.com/sujit-baniya/asynq/blob/master/flow.go For the purpose, I've implemented flow and this example works for simple Edge and Node implementation https://github.com/sujit-baniya/asynq/blob/master/examples/asynq_worker.go The example works for 1) An edge with single input and single output 2) A loop with single input and single output I'm trying find solution where a edge/loop has multiple output. Since there would be multiple output, at some point it is required to combine such data. For e.g. NodeA with key A NodeB with key B (generates phone_valid field) NodeC with key C (generates email_valid field) NodeD with key D Edge is created from A -> B Edge is created from A -> C Edge is created from B -> D Edge is created from C -> D I'm trying to get aggregated data received from above flow. If I pass data like `{"phone": "123456789", "email": "abc@example"}` to Node A, I would be expecting `{"phone": "123456789", "email": "abc@example", "phone_valid": true, "email_valid": true}` on Node D Please suggest some ideas.</div>

kerem closed this issue

2026-03-07 22:07:57 +03:00

kerem commented

2026-03-07 22:07:58 +03:00

Author

Owner

@hibiken commented on GitHub (Jun 26, 2022):

Duplicate of #500

@hibiken commented on GitHub (Jun 26, 2022): Duplicate of #500

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/asynq#1245

No description provided.

Rows
Columns