[PR #1] [MERGED] Rescue dead "In Progress" tasks #529

New issue

Closed

opened 2026-03-02 05:38:10 +03:00 by kerem · 0 comments

kerem commented

2026-03-02 05:38:10 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/hibiken/asynq/pull/1
Author: @hibiken
Created: 11/23/2019
Status: ✅ Merged
Merged: 11/23/2019
Merged by: @hibiken

Base: master ← Head: feature/simple_rescue

📝 Commits (4)

3fa867b Remove per task heartbeat
76ceb28 Change the direction of list push/pop operations
4a32793 Use command BRPUSHLPOP to move from queue to in_progress in redis
fd80126 Add logic to restore unfinished tasks back into the default queue if

📊 Changes

3 files changed (+95 additions, -80 deletions)

View changed files

📝 processor.go (+16 -24)
📝 rdb.go (+30 -41)
📝 rdb_test.go (+49 -15)

📄 Description

Changes made:

Use BRPOPLPUSH to move a task from a queue to In-Progress list
Remove a task from In-Progress list once a worker has finished with the task
On start, pushes the In-Progress tasks (if any) back to the queue

This change makes the system more resilient. It prevents data loss in case there's a server crash while workers are still working on a task or a task is waiting for an available worker.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/hibiken/asynq/pull/1 **Author:** [@hibiken](https://github.com/hibiken) **Created:** 11/23/2019 **Status:** ✅ Merged **Merged:** 11/23/2019 **Merged by:** [@hibiken](https://github.com/hibiken) **Base:** `master` ← **Head:** `feature/simple_rescue` --- ### 📝 Commits (4) - [`3fa867b`](https://github.com/hibiken/asynq/commit/3fa867b190bc713efe316a0afb0a72f6fc4ef031) Remove per task heartbeat - [`76ceb28`](https://github.com/hibiken/asynq/commit/76ceb282a906a40e23e9c05a3e8852584fc5bcfe) Change the direction of list push/pop operations - [`4a32793`](https://github.com/hibiken/asynq/commit/4a327933bdd17725bdfd4703df7ec6bdcf472954) Use command BRPUSHLPOP to move from queue to in_progress in redis - [`fd80126`](https://github.com/hibiken/asynq/commit/fd80126a6756bd6cefa56557ba4a8875ee9d3784) Add logic to restore unfinished tasks back into the default queue if ### 📊 Changes **3 files changed** (+95 additions, -80 deletions) <details> <summary>View changed files</summary> 📝 `processor.go` (+16 -24) 📝 `rdb.go` (+30 -41) 📝 `rdb_test.go` (+49 -15) </details> ### 📄 Description Changes made: - Use BRPOPLPUSH to move a task from a queue to **In-Progress** list - Remove a task from **In-Progress** list once a worker has finished with the task - On start, pushes the **In-Progress** tasks (if any) back to the queue This change makes the system more resilient. It prevents data loss in case there's a server crash while workers are still working on a task or a task is waiting for an available worker. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>