[GH-ISSUE #205] Triggering dynamic layout change with a widget may lead to a deadlock #116

Closed
opened 2026-03-03 16:22:30 +03:00 by kerem · 1 comment
Owner

Originally created by @mum4k on GitHub (May 14, 2019).
Original GitHub issue: https://github.com/mum4k/termdash/issues/205

Originally assigned to: @mum4k on GitHub.

Thanks to @keithknott26 for reporting this.

The deadlock can occur when a button widget is configured so that its callback calls the Update method on the container API:
github.com/mum4k/termdash@0e5242ca6f/container/container.go (L228)

If there is a concurrent Redraw happening at the same time, the container is mutex locked. The rest is a race, the deadlock occurs in this sequence of events:

  1. Mouse event is delivered to the button widget, button's mutex is acquired, event is processed.
  2. Periodic or triggered redraw starts, the container's lock is acquired.
  3. Button calls the CallbackFn, which tries to call container's Update, container's Update needs to acquire container's lock so is blocked.
  4. The redraw reaches the button widget, its lock is held, the draw is locked.

Proof in the traceback below.

Button attempting to call Update:*

goroutine 74 [semacquire, 8 minutes]:
sync.runtime_SemacquireMutex(0xc0001c0014, 0x0)
        /usr/local/opt/go/libexec/src/runtime/sema.go:71 +0x3d
sync.(*Mutex).Lock(0xc0001c0010)
        /usr/local/opt/go/libexec/src/sync/mutex.go:134 +0x109
github.com/mum4k/termdash/container.(*Container).Update(0xc0001fc000, 0x9c9be5, 0x4, 0xc00968ff70, 0x1, 0x1, 0x0, 0x0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/container.go:229 +0x3b
main.setLayout(0xc0001fc000, 0xc000502700, 0x5, 0x78ec51, 0xc003467e18)
        /Users/keithknott/go/src/github.com/keithknott26/adlDashboard/cmd/dashboard.go:1850 +0x9d
main.newLayoutButtons.func5(0xc0001fe9c0, 0xc009698e80)
        /Users/keithknott/go/src/github.com/keithknott26/adlDashboard/cmd/dashboard.go:1928 +0x3c
github.com/mum4k/termdash/widgets/button.(*Button).Mouse(0xc000145cc0, 0xc009698e80, 0x0, 0x0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/widgets/button/button.go:187 +0xde
github.com/mum4k/termdash/container.(*Container).prepareEvTargets.func1(0xc0001c0010, 0xaa32c0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/container.go:297 +0x72
github.com/mum4k/termdash/container.(*Container).processEvent(0xc0001fc000, 0xaa32c0, 0xc009698c00, 0xc001c38d50, 0xc001c38d50)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/container.go:279 +0xb2
github.com/mum4k/termdash/container.(*Container).Subscribe.func1(0xaa32c0, 0xc009698c00)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/container.go:442 +0x4e
github.com/mum4k/termdash/internal/event.(*subscriber).callback(0xc001c38dc0, 0xaa32c0, 0xc009698c00)
        /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:95 +0x3d
github.com/mum4k/termdash/internal/event.(*subscriber).run(0xc001c38dc0, 0xaae6a0, 0xc001c38d00)
        /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:110 +0xa5
created by github.com/mum4k/termdash/internal/event.newSubscriber
        /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:89 +0x23a

Termdash attempting to redraw the button:

goroutine 70 [semacquire, 8 minutes]:
sync.runtime_SemacquireMutex(0xc000145cf4, 0xc001ebe900)
        /usr/local/opt/go/libexec/src/runtime/sema.go:71 +0x3d
sync.(*Mutex).Lock(0xc000145cf0)
        /usr/local/opt/go/libexec/src/sync/mutex.go:134 +0x109
github.com/mum4k/termdash/widgets/button.(*Button).Draw(0xc000145cc0, 0xc0042b6180, 0xc00234c60c, 0x0, 0x0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/widgets/button/button.go:116 +0x5c
github.com/mum4k/termdash/container.drawWidget(0xc002027730, 0x0, 0x0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/draw.go:138 +0x2b0
github.com/mum4k/termdash/container.drawCont(0xc002027730, 0x3e, 0x0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/draw.go:171 +0x1b3
github.com/mum4k/termdash/container.drawTree.func1(0xc002027730, 0x0, 0x51)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/draw.go:63 +0x83
github.com/mum4k/termdash/container.preOrder(0xc002027730, 0xc001ebede0, 0x9f2668)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:43 +0x4f
github.com/mum4k/termdash/container.preOrder(0xc0020276c0, 0xc001ebede0, 0x9f2668)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:47 +0xbe
github.com/mum4k/termdash/container.preOrder(0xc0020275e0, 0xc001ebede0, 0x9f2668)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:48 +0xe4
github.com/mum4k/termdash/container.preOrder(0xc002027500, 0xc001ebede0, 0x9f2668)
github.com/mum4k/termdash/container.preOrder(0xc002027420, 0xc001ebede0, 0x9f2668)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:48 +0xe4
github.com/mum4k/termdash/container.preOrder(0xc002027340, 0xc001ebede0, 0x9f2668)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:48 +0xe4
github.com/mum4k/termdash/container.preOrder(0xc0001fc000, 0xc001ebede0, 0x9f2668)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:47 +0xbe
github.com/mum4k/termdash/container.drawTree(0xc0001fc000, 0x33, 0x0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/draw.go:43 +0x124
github.com/mum4k/termdash/container.(*Container).Draw(0xc0001fc000, 0x0, 0x0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/container/container.go:219 +0x10a
github.com/mum4k/termdash.(*termdash).redraw(0xc000206ea0, 0x9f3480, 0xc000206ed4)
        /Users/keithknott/go/src/github.com/mum4k/termdash/termdash.go:274 +0x4e
github.com/mum4k/termdash.(*termdash).evRedraw(0xc000206ea0, 0x0, 0x0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/termdash.go:294 +0x83
github.com/mum4k/termdash.(*termdash).subscribers.func3(0xaa32c0, 0xc009698be0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/termdash.go:230 +0x2a
github.com/mum4k/termdash/internal/event.(*subscriber).callback(0xc001c38b80, 0xaa32c0, 0xc009698be0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:95 +0x3d
github.com/mum4k/termdash/internal/event.(*subscriber).run(0xc001c38b80, 0xaae6a0, 0xc001c38ac0)
        /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:110 +0xa5
created by github.com/mum4k/termdash/internal/event.newSubscriber
        /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:89 +0x23a
Originally created by @mum4k on GitHub (May 14, 2019). Original GitHub issue: https://github.com/mum4k/termdash/issues/205 Originally assigned to: @mum4k on GitHub. Thanks to @keithknott26 for reporting this. The deadlock can occur when a button widget is configured so that its callback calls the Update method on the container API: https://github.com/mum4k/termdash/blob/0e5242ca6f8386889a303c67b58a25f98aed75b6/container/container.go#L228 If there is a concurrent Redraw happening at the same time, the container is mutex locked. The rest is a race, the deadlock occurs in this sequence of events: 1) Mouse event is delivered to the button widget, button's mutex is acquired, event is processed. 2) Periodic or triggered redraw starts, the container's lock is acquired. 3) Button calls the CallbackFn, which tries to call container's Update, container's Update needs to acquire container's lock so is blocked. 4) The redraw reaches the button widget, its lock is held, the draw is locked. Proof in the traceback below. **Button attempting to call Update:*** ``` goroutine 74 [semacquire, 8 minutes]: sync.runtime_SemacquireMutex(0xc0001c0014, 0x0) /usr/local/opt/go/libexec/src/runtime/sema.go:71 +0x3d sync.(*Mutex).Lock(0xc0001c0010) /usr/local/opt/go/libexec/src/sync/mutex.go:134 +0x109 github.com/mum4k/termdash/container.(*Container).Update(0xc0001fc000, 0x9c9be5, 0x4, 0xc00968ff70, 0x1, 0x1, 0x0, 0x0) /Users/keithknott/go/src/github.com/mum4k/termdash/container/container.go:229 +0x3b main.setLayout(0xc0001fc000, 0xc000502700, 0x5, 0x78ec51, 0xc003467e18) /Users/keithknott/go/src/github.com/keithknott26/adlDashboard/cmd/dashboard.go:1850 +0x9d main.newLayoutButtons.func5(0xc0001fe9c0, 0xc009698e80) /Users/keithknott/go/src/github.com/keithknott26/adlDashboard/cmd/dashboard.go:1928 +0x3c github.com/mum4k/termdash/widgets/button.(*Button).Mouse(0xc000145cc0, 0xc009698e80, 0x0, 0x0) /Users/keithknott/go/src/github.com/mum4k/termdash/widgets/button/button.go:187 +0xde github.com/mum4k/termdash/container.(*Container).prepareEvTargets.func1(0xc0001c0010, 0xaa32c0) /Users/keithknott/go/src/github.com/mum4k/termdash/container/container.go:297 +0x72 github.com/mum4k/termdash/container.(*Container).processEvent(0xc0001fc000, 0xaa32c0, 0xc009698c00, 0xc001c38d50, 0xc001c38d50) /Users/keithknott/go/src/github.com/mum4k/termdash/container/container.go:279 +0xb2 github.com/mum4k/termdash/container.(*Container).Subscribe.func1(0xaa32c0, 0xc009698c00) /Users/keithknott/go/src/github.com/mum4k/termdash/container/container.go:442 +0x4e github.com/mum4k/termdash/internal/event.(*subscriber).callback(0xc001c38dc0, 0xaa32c0, 0xc009698c00) /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:95 +0x3d github.com/mum4k/termdash/internal/event.(*subscriber).run(0xc001c38dc0, 0xaae6a0, 0xc001c38d00) /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:110 +0xa5 created by github.com/mum4k/termdash/internal/event.newSubscriber /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:89 +0x23a ``` **Termdash attempting to redraw the button:** ``` goroutine 70 [semacquire, 8 minutes]: sync.runtime_SemacquireMutex(0xc000145cf4, 0xc001ebe900) /usr/local/opt/go/libexec/src/runtime/sema.go:71 +0x3d sync.(*Mutex).Lock(0xc000145cf0) /usr/local/opt/go/libexec/src/sync/mutex.go:134 +0x109 github.com/mum4k/termdash/widgets/button.(*Button).Draw(0xc000145cc0, 0xc0042b6180, 0xc00234c60c, 0x0, 0x0) /Users/keithknott/go/src/github.com/mum4k/termdash/widgets/button/button.go:116 +0x5c github.com/mum4k/termdash/container.drawWidget(0xc002027730, 0x0, 0x0) /Users/keithknott/go/src/github.com/mum4k/termdash/container/draw.go:138 +0x2b0 github.com/mum4k/termdash/container.drawCont(0xc002027730, 0x3e, 0x0) /Users/keithknott/go/src/github.com/mum4k/termdash/container/draw.go:171 +0x1b3 github.com/mum4k/termdash/container.drawTree.func1(0xc002027730, 0x0, 0x51) /Users/keithknott/go/src/github.com/mum4k/termdash/container/draw.go:63 +0x83 github.com/mum4k/termdash/container.preOrder(0xc002027730, 0xc001ebede0, 0x9f2668) /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:43 +0x4f github.com/mum4k/termdash/container.preOrder(0xc0020276c0, 0xc001ebede0, 0x9f2668) /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:47 +0xbe github.com/mum4k/termdash/container.preOrder(0xc0020275e0, 0xc001ebede0, 0x9f2668) /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:48 +0xe4 github.com/mum4k/termdash/container.preOrder(0xc002027500, 0xc001ebede0, 0x9f2668) github.com/mum4k/termdash/container.preOrder(0xc002027420, 0xc001ebede0, 0x9f2668) /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:48 +0xe4 github.com/mum4k/termdash/container.preOrder(0xc002027340, 0xc001ebede0, 0x9f2668) /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:48 +0xe4 github.com/mum4k/termdash/container.preOrder(0xc0001fc000, 0xc001ebede0, 0x9f2668) /Users/keithknott/go/src/github.com/mum4k/termdash/container/traversal.go:47 +0xbe github.com/mum4k/termdash/container.drawTree(0xc0001fc000, 0x33, 0x0) /Users/keithknott/go/src/github.com/mum4k/termdash/container/draw.go:43 +0x124 github.com/mum4k/termdash/container.(*Container).Draw(0xc0001fc000, 0x0, 0x0) /Users/keithknott/go/src/github.com/mum4k/termdash/container/container.go:219 +0x10a github.com/mum4k/termdash.(*termdash).redraw(0xc000206ea0, 0x9f3480, 0xc000206ed4) /Users/keithknott/go/src/github.com/mum4k/termdash/termdash.go:274 +0x4e github.com/mum4k/termdash.(*termdash).evRedraw(0xc000206ea0, 0x0, 0x0) /Users/keithknott/go/src/github.com/mum4k/termdash/termdash.go:294 +0x83 github.com/mum4k/termdash.(*termdash).subscribers.func3(0xaa32c0, 0xc009698be0) /Users/keithknott/go/src/github.com/mum4k/termdash/termdash.go:230 +0x2a github.com/mum4k/termdash/internal/event.(*subscriber).callback(0xc001c38b80, 0xaa32c0, 0xc009698be0) /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:95 +0x3d github.com/mum4k/termdash/internal/event.(*subscriber).run(0xc001c38b80, 0xaae6a0, 0xc001c38ac0) /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:110 +0xa5 created by github.com/mum4k/termdash/internal/event.newSubscriber /Users/keithknott/go/src/github.com/mum4k/termdash/internal/event/event.go:89 +0x23a ```
kerem 2026-03-03 16:22:30 +03:00
  • closed this issue
  • added the
    bug
    label
Author
Owner

@mum4k commented on GitHub (May 14, 2019):

@keithknott26, when you get a chance, please build from the 205-deadlock branch and verify that this indeed fixes the problem.

<!-- gh-comment-id:492055224 --> @mum4k commented on GitHub (May 14, 2019): @keithknott26, when you get a chance, please build from the [205-deadlock](https://github.com/mum4k/termdash/tree/205-deadlock) branch and verify that this indeed fixes the problem.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/termdash#116
No description provided.