mirror of
https://github.com/mum4k/termdash.git
synced 2026-04-27 03:15:55 +03:00
[GH-ISSUE #174] Bug in the linechart #104
Labels
No labels
bug
cleanup
enhancement
enhancement
enhancement
good first issue
help wanted
help wanted
pull-request
question
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/termdash#104
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @keithknott26 on GitHub (Mar 19, 2019).
Original GitHub issue: https://github.com/mum4k/termdash/issues/174
Hello,
I have been experiencing an intermittent problem where the linechart panics but have been unable to reproduce it with any certainty. I found today I was lucky enough to recreate the issue over and over again by cycling through the same dataset.
I have not tried this on its own because I suspect the bug has to do with the widths of the linecharts and perhaps that empty cell at the end of the chart?
Here's what I have so far:
Initialize the Linechart:
Then a periodic loop expires and this code gets run again, it's on the 3rd loop that the panic occurs:
I found I'm able to avoid the panic by populating a smaller amount of entries in inputsOne. It's a bug that happens maybe every hour or so and crashes the app, it seems to happen more frequently when I have graphs with multiple series (3 or more) but has happened infrequently on single series linecharts.
I'm wondering if there might be a way I can recover from the panic as a workaround and have the app still function? I will keep trying to find ways to reproduce this issue in the meantime.
@keithknott26 commented on GitHub (Mar 19, 2019):
Also wanted to know if you knew of anything that might cause this issue aside from a bug, I’ve been over the code a number of times and the labels and values are clean. Perhaps the next step is for me to reproduce this outside of the app in a single chart using the linechart demo
@keithknott26 commented on GitHub (Mar 20, 2019):
Hello,
A bit more info on this, I found that it doesn't matter whether or not XUnscaled() is enabled or not. I was able to capture a panic from both on/off scenarios where the failure was reported in the same position 36 out of bounds 0 <= pos <= 35 with different data. The panic appears to be caused when I set the labelMap.
Hopefully this helps, if there's anything specific you wanted me to try and dump to the log let me know.
@mum4k commented on GitHub (Mar 20, 2019):
Thank you @keithknott26, this is a great amount of debugging data and I have a theory of where this bug lives. I will try to reproduce this locally.
I think you also asked if there is a way to avoid the panic and continue instead. I am assuming that you are running using the periodic redraws. If so, you can provide your own error handler which will avoid the panic. Documented here:
https://github.com/mum4k/termdash/wiki/Termdash-API#termdasherrorhandler
Alternatively Termdash shouldn't panic if you're using the triggered redraws.
@mum4k commented on GitHub (Mar 20, 2019):
One thing that could help - if at all possible could you share the full code on your end that you are using to reproduce this panic?
It would help me if I had something ready to reproduce this locally.
@keithknott26 commented on GitHub (Mar 20, 2019):
I'm happy to share the code, it's the data that might be the problem. The dashboard effectively tails a log file and then picks out certain log lines and uses the metrics to build the line chart(s). The thing is, it uses time.Now() when looking at the log line so unless you're tailing the log live you wont be able to reproduce the same scenario. There are something like 14 charts in total but a couple charts in particular expose the problem (the ones which aren't very wide).
I was thinking I could probably come up with a sanitized version of the log file to drive the data stream, but that doesn't solve the time.Now() problem I mentioned above.
I'll continue trying to collect something that will allow us to reproduce this issue. In the meantime I've sent the (largely unfinished) dashboard code to termdash@termdash.com - it expects to tail a file - most of the code you'll care about will be in chart.go starting around like 210
@mum4k commented on GitHub (Mar 21, 2019):
Thank you @keithknott26, that helped a lot. I have a theory and a proof now, so I believe this one is almost tackled.
Interestingly enough this is again a data race, this time one that is between both datadash and termdash.
Background: as you probably know, a slice in Go is implemented as a pointer to the underlying array that holds the data. That array can sometimes be changed when appending to slice, but that only happens when it runs out of capacity. This article describes it well. Datadash passes a slice to Termdash (via LineChart.Series) and from the above explanation - both Datadash and Termdash then share a pointer to the same memory - the underlying array. To complete the picture, this is where that array gets initiated, it is actually owned by the float64RingBuffer:
github.com/keithknott26/datadash@9ed568bfad/ring.go (L4)Here's what's happening:
github.com/keithknott26/datadash@9ed568bfad/ring.go (L32)Let's say that it added a new value 3, so the array now is []float64{3,1,2}.
I have been discussing with myself whether this should be fixed in Datadash or Termdash. I feel that this behavior makes the API of the widgets brittle, since it is easy to make this mistake.
So I will push a fix inside Termdash that will ensure the widgets store a deep copy of the data passed in so they never share a pointer to the same memory with the caller. This might one day bite us if the performance of those copies becomes noticeable, but I think we are fine for now.
@mum4k commented on GitHub (Mar 21, 2019):
@keithknott26, the fix has been merged into the devel branch. I would appreciate if you could confirm that this works or reopen this issue if it reoccurs.
I have also improved the error messages with additional data, so if it happens again, please do paste the new (more complete) errors.
@keithknott26 commented on GitHub (Mar 21, 2019):
Thank you for looking into this! I've confirmed this hasn't come up again after 3 hours of running about 20 or so charts. I'm planning to convert the app to use the controller based redraws this weekend, I really appreciate your help in getting this sorted
@mum4k commented on GitHub (Mar 21, 2019):
Of course, I am glad to hear that we have successfully tackled this one.