mirror of
https://github.com/mum4k/termdash.git
synced 2026-04-27 03:15:55 +03:00
[GH-ISSUE #168] Question: Problem with Linechart #100
Labels
No labels
bug
cleanup
enhancement
enhancement
enhancement
good first issue
help wanted
help wanted
pull-request
question
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/termdash#100
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @keithknott26 on GitHub (Mar 7, 2019).
Original GitHub issue: https://github.com/mum4k/termdash/issues/168
@mum4k,
I noticed recently that the linechart is displaying the full dataset but only half the labels, the labels should go to 23:59 but stop for some reason.
I've been trying to debug things on my end but haven't had much luck in figuring out why its happening. Would you be able to run a dataset through on your side and see if its happening for you as well? I'm using a dataset with 1440 records in it and it displays labels up to about half.
It happens with the XAxisUnscaled option disabled.
@mum4k commented on GitHub (Mar 7, 2019):
This reminds me of #117. Could you share how you are creating the linechart? I.e. The set of options you're using when creating it and when writing data values to it.
I will try to recreate this issue locally once I have this.
@keithknott26 commented on GitHub (Mar 7, 2019):
I agree , it's very odd. I'm not sure its a bug though because I haven't seen you mess with the line chart much since you last fixed it. I'll keep looking on my end and let you know what I find
@mum4k commented on GitHub (Mar 7, 2019):
Agreed I am also unsure if this is a bug, but it is good that you have reported it. I have seen myself bring bugs back to life in the past.
I will test it and let you know.
@mum4k commented on GitHub (Mar 7, 2019):
I have tested this code:
And I got the following linechart:
The same with vertical labels where space plays less of a role:
In other words I failed to reproduce this :( Of course it is still possible that I am doing something slightly differently, maybe you can identify it in my code above?
Alternatively I will be happy to help you debug on your end. If you would like that - can you possibly share more of the code that generates the values and their labels. If there is an issue, it will be in that code. If you would prefer to do this over a more private channel, feel free to contact me directly at termdash@termdash.dev.
Another alternative - can you save the complete content of the var inputs []float64 and var labelMap = map[int]string{} variables just before the call to Series?
@keithknott26 commented on GitHub (Mar 8, 2019):
Thank you I really appreciate you taking the time to test, I worked on this more today and found that the data is being read in faster than its being picked up by the line chart. This was one of my first projects made to learn Go so i'm still trying to figure out how to sync those two things up.
Edit: The problem is there are twice as many records as there are labels, so that would explain why I was seeing the linechart populate with all the data but the linechart only showed half the labels. By the time we reach the end of the file there's no longer a 1:1 mapping between label and record its more like 1:2.
I've posted my code on github: https://github.com/keithknott26/datadash it still needs work particularly in cleaning up unused code and supporting infinite charts (today it assumes the number of charts will only ever range from 1 to 5), and refactoring the way it reads data vs displays it.
Thanks again, I'm actually curious which method you use to debug? I remember you mentioning you were using a Mac - are you using VSCode and Delve ? I found I had to disable drawing of the charts while debugging (with well placed print statements) otherwise it's impossible to read what's on the screen.
@keithknott26 commented on GitHub (Mar 8, 2019):
One other question I had was, is there already support for triggered updates? Instead of doing periodic redraws I was hoping to update the content of each widget only if it needed it. My fan runs on my Mac whenever I run the program so I know its probably doing way too many unnecessary redraws.
@mum4k commented on GitHub (Mar 8, 2019):
More than happy to help @keithknott26. Thank you very much for sharing the code with me among other things I am really glad to see this used. It is a cool idea to build an application like datadash, great work! I will try to answer all three of your questions below, please let me know if I missed anything.
Data races
The problem
Let's tackle the problem you're observing first, i.e. the 1:2 mapping between label and data as I think this has the most complexity.
TLDR; The code as it stands now has multiple data races. From database theory a data race happens when there are two memory accesses in a program where:
The important thing to note is that when a program has a data race, the behavior is undefined. As you probably know "undefined" in software engineering means "very very bad". I strongly recommend to read this article, Dmitry works on data race detection and goes into factual and partly entertaining details on what really happens when we write software with data races.
Since this if fairly abstract, let me go to the specifics. Note this is only describing one of the data races in datadash, there are a few more. The readDataChannel function spawns a separate goroutine in which it runs the parsePlotData function.
github.com/keithknott26/datadash@9ed568bfad/cmd/datadash.go (L345-L356)The parsePlotData function the calls The row.Update method which writes to multiple memory locations, specifically it writes into the Data and Label ring buffers. This is our access number one which Writes.
The row.createLineChart method starts a goroutine that reads from multiple memory locations, among others it reads from the Data and Labels ring buffers which happens to be the same memory location the access number one wrote. This is our access number two which Reads.
github.com/keithknott26/datadash@9ed568bfad/row.go (L469)A single race condition like this can result in any behavior including data loss. The simplest logical explanation of data races is when we imagine two bank accounts, say my checking account and my landlords account and their initial balances:
Initial
me: $1000
landlord: $2000
Now say that we have two transactions:
Each transaction needs to read the balance of both accounts, do the appropriate addition / subtraction and write the new balances. Now obviously with data races the behavior is undefined so any ordering of the operations can happen, but imagine this one:
Final
me: $1100
landlord: $3000
This example is easy to understand because there are two writes. Why is it also bad when there is only one write is better explained by Dmitry in the article listed above.
It is important to understand that Go's runtime doesn't give any guarantees about the ordering of operation between two goroutines. The Go's advice is share memory by communicating.
The solution
Honestly the one advice I give and find the most effective sounds counter-productive at first:
Specifically for datadash I would recommend switching to triggered redraws (see below) and remove all goroutines unless they are really needed. It makes life much easier.
If we wanted to just quickly fix this in place, we could add Mutexes to protect all occasions when data are read or written by separate goroutines.
Debugging
Sadly my answer to this will probably be disappointing. I am a bit old-school I don't use any of the debugging tools. I only needed something advanced once when I added too much concurrency into my code and lost my mind.
Instead I follow the "test driven development" school. This also includes designing software that is easy to test. Very specifically this involves strict separation of business logic from the UI which allows complete test coverage of the business logic in separation. Thus on any bugs or errors I first write a test that repeats the problem and then debug as per the usual (log / print statements).
When I had a bug in the UI portion of Termdash and I couldn't use the approach mentioned above, I typically use the Text widget to print out debugging data, or use ioutil.WriteFile to write them to a file. I.e. I accumulate what I want to see into a bytes.Buffer and then on a specific trigger like a condition or a middle mouse button a dump them into a file for inspection.
Triggered updates
Yes they are already implemented, instead of using periodic, which are fairly wasteful the user can trigger updates as needed. I have recently started documenting the API. It isn't finished yet, but this part has already been written:
https://github.com/mum4k/termdash/wiki/Termdash-API
Look for the termdash.NewController function.
Hope this helps.
@keithknott26 commented on GitHub (Mar 10, 2019):
Thank you for your very detailed response! I must admit I wasn't expecting you to provide so much helpful information I truly appreciate it. I wasn't aware of the race condition and it certainly explains the behavior I'm seeing when changing the timing values around. I've decided to re-write the app to include the features I mentioned, as well as remove the periodic redraws. Thank you for sharing the tip about debugging using a textbox or file, I've been using the textbox method and it's working great so far!
One thing that I'm confused about is the ability to place sets of widgets or Rows on the screen programatically. For example today I'm doing something like this to build the container options:
Then I build the container.
When there's 3 rows:
Or where there's 5 rows
But when you get to 3 or 5 rows you have different Horizontal Split percentages. Is there a good way to add "Rows" to the container programmatically? I wanted to get to a point where I could add "rows" one by one based on the number of columns in the input file. Today I'm using the manual assignment but would like to see if I can switch that over so that it could display an infinite amount of rows.
I must admit I haven't tried this yet but it seems like maybe i'd want to add more containers? Is it possible to run multiple containers and have them all subscribe to keyboard and mouse events? In this case each container would have 1 chart and 1 text box. Then I could add rows one by one:
Any thoughts on how you've been able to do this? Hope my update makes sense, if not feel free to let me know and I'll revise and provide additional detail and code examples.
@mum4k commented on GitHub (Mar 10, 2019):
Happy to help @keithknott26, feel free to let me know if you will have any further questions. Do note that keyboard and mouse events are also coming from a different goroutine all the way form termbox. I also noticed that you were setting some booleans from keyboard events:
github.com/keithknott26/datadash@9ed568bfad/cmd/datadash.go (L485)This too is a data race, the booleans would need to be protected with a Mutex. A simplest possible implementation will be something like this (bar any typos):
When all reads and writes come through the get() and set() methods, there is no data race.
Setting layout programatically
As for the ability to create layout programatically - the current API is a bit awkward for that. I have been pondering a grid-like API (similar to what termui has) and I think it is time to implement it.
Let me give it a try (tracking in #171) and I will let you know once it is ready on that bug.
Going to close this issue for now, but if you have further questions - please feel free to ask on this or other issues.
@mum4k commented on GitHub (Mar 11, 2019):
@keithknott26, fyi #171 is now resolved and contains details about the new grid builder (available in the devel branch). As always I would appreciate your feedback and bug reports.