starred/log.io-NarrativeScience-old

Fork 0

mirror of https://github.com/NarrativeScience-old/log.io.git synced 2026-04-26 09:35:53 +03:00

[GH-ISSUE #52] List of nodes and streams in web client becomes corrupted (0.3.0) #37

New issue

Closed

opened 2026-02-26 05:31:08 +03:00 by kerem · 9 comments

kerem commented

2026-02-26 05:31:08 +03:00

Owner

Originally created by @jochenonline on GitHub (Feb 22, 2013).
Original GitHub issue: https://github.com/NarrativeScience-old/log.io/issues/52

I have five machines running. One with the log.io-server and a log.io-harvester, four only with a log.io-harvester.

When I start the server and afterwards the five harvesters everything looks fine. But after a while (some minutes). The names of the streams and nodes start becoming corrupted:

empty list elements appear
names are cut (i. e. "ar0037002" becomes "ar0", "mongodb" becomes "mo")
some nodes or streams disappear completely

With time going on things get worse.

Refreshing the browser window does not help. I have to restart the server and sometimes also the harvesters.

Bug, feature or me?

Originally created by @jochenonline on GitHub (Feb 22, 2013). Original GitHub issue: https://github.com/NarrativeScience-old/log.io/issues/52 I have five machines running. One with the `log.io-server` and a `log.io-harvester`, four only with a `log.io-harvester`. When I start the server and afterwards the five harvesters everything looks fine. But after a while (some minutes). The names of the streams and nodes start becoming corrupted: - empty list elements appear - names are cut (i. e. "ar0037002" becomes "ar0", "mongodb" becomes "mo") - some nodes or streams disappear completely With time going on things get worse. Refreshing the browser window does not help. I have to restart the server and sometimes also the harvesters. Bug, feature or me?

kerem closed this issue

2026-02-26 05:31:08 +03:00

kerem commented

2026-02-26 05:31:08 +03:00

Author

Owner

@msmathers commented on GitHub (Feb 22, 2013):

Yikes, could you send me the harvester configuration files that you're using?

@msmathers commented on GitHub (Feb 22, 2013): Yikes, could you send me the harvester configuration files that you're using?

kerem commented

2026-02-26 05:31:08 +03:00

Author

Owner

@jochenonline commented on GitHub (Feb 22, 2013):

Sure. I have two kinds. The first one is on the server plus one of the clients (#hostname# is replaced by the individual hostname):

exports.config = {
  nodeName: "#hostname#",
  logStreams: {
    release: [
        "/home/green/log/release.log"
    ]
  },
  server: {
    host: 'ar0037001',
    port: 28777
  }
}

This one is on the other three clients:

exports.config = {
        nodeName: "#hostname#",
        logStreams: {
                mongodb: [
                        "/var/log/mongodb/mongodb.log"
                ],
                redis: [
                        "/home/green/log/redis.log"
                ],
                sentinel: [
                        "/home/green/log/sentinel.log"
                ],
                mms: [
                        "/home/green/log/mms.log"
                ],
                solr: [
                        "/home/green/log/solr.log"
                ]
        },
        server: {
                host: 'ar0037001',
                port: 28777
        }
}

@jochenonline commented on GitHub (Feb 22, 2013): Sure. I have two kinds. The first one is on the server plus one of the clients (#hostname# is replaced by the individual hostname): ``` exports.config = { nodeName: "#hostname#", logStreams: { release: [ "/home/green/log/release.log" ] }, server: { host: 'ar0037001', port: 28777 } } ``` This one is on the other three clients: ``` exports.config = { nodeName: "#hostname#", logStreams: { mongodb: [ "/var/log/mongodb/mongodb.log" ], redis: [ "/home/green/log/redis.log" ], sentinel: [ "/home/green/log/sentinel.log" ], mms: [ "/home/green/log/mms.log" ], solr: [ "/home/green/log/solr.log" ] }, server: { host: 'ar0037001', port: 28777 } } ```

kerem commented

2026-02-26 05:31:08 +03:00

Author

Owner

@CodeMichael commented on GitHub (Mar 1, 2013):

I'm getting something similar, but I noticed the server spitting error messages:

error: Invalid TCP message: layouts/content/two-column.vm HTTP/1.1" 200 309 - 3080 244
error: Invalid TCP message: s/templates/core/_layouts/content/two-column.vm HTTP/1.1" 200 309 - 3080 156
error: Invalid TCP message: /1.1" 200 3439 - 3080 225

These are from apache logs

@CodeMichael commented on GitHub (Mar 1, 2013): I'm getting something similar, but I noticed the server spitting error messages: ``` log error: Invalid TCP message: layouts/content/two-column.vm HTTP/1.1" 200 309 - 3080 244 error: Invalid TCP message: s/templates/core/_layouts/content/two-column.vm HTTP/1.1" 200 309 - 3080 156 error: Invalid TCP message: /1.1" 200 3439 - 3080 225 ``` These are from apache logs

kerem commented

2026-02-26 05:31:08 +03:00

Author

Owner

@msmathers commented on GitHub (Mar 1, 2013):

I've been unable to reproduce this problem, however I think I see what's going on. When the server receives data from the TCP socket, it casts the payload to a string and immediately processes it. If an inbound message gets chunked into multiple parts, the server will only process the first part with the valid prefix (i.e. +log, +node, etc...) and drop the rest. That would explain the invalid TCP message errors as well as the truncated node names, assuming the harvesters were reconnecting...

I think what I need to do is have the server buffer inbound messages until it hits the delimiter, and THEN process the entire buffer as a single message. I'll work on a patch this weekend, hopefully that will fix this behavior.

@msmathers commented on GitHub (Mar 1, 2013): I've been unable to reproduce this problem, however I think I see what's going on. When the server receives data from the TCP socket, it casts the payload to a string and immediately processes it. If an inbound message gets chunked into multiple parts, the server will only process the first part with the valid prefix (i.e. +log, +node, etc...) and drop the rest. That would explain the invalid TCP message errors as well as the truncated node names, assuming the harvesters were reconnecting... I think what I need to do is have the server buffer inbound messages until it hits the delimiter, and THEN process the entire buffer as a single message. I'll work on a patch this weekend, hopefully that will fix this behavior.

kerem commented

2026-02-26 05:31:08 +03:00

Author

Owner

@msmathers commented on GitHub (Mar 4, 2013):

I just pushed v0.3.1 to npm, see if that fixes it. Feel free to reopen if behavior continues.

@msmathers commented on GitHub (Mar 4, 2013): I just pushed v0.3.1 to npm, see if that fixes it. Feel free to reopen if behavior continues.

kerem commented

2026-02-26 05:31:08 +03:00

Author

Owner

@jochenonline commented on GitHub (Mar 31, 2013):

Sorry for late answer. It works!!!

@jochenonline commented on GitHub (Mar 31, 2013): Sorry for late answer. It works!!!

kerem commented

2026-02-26 05:31:08 +03:00

Author

Owner

@Hyvi commented on GitHub (Apr 11, 2013):

0.3.3 the problem also exists.

@Hyvi commented on GitHub (Apr 11, 2013): 0.3.3 the problem also exists.

kerem commented

2026-02-26 05:31:08 +03:00

Author

Owner

@jochenonline commented on GitHub (Apr 11, 2013):

Not on my side...

@jochenonline commented on GitHub (Apr 11, 2013): Not on my side...

kerem commented

2026-02-26 05:31:08 +03:00

Author

Owner

@geek-kb commented on GitHub (Dec 6, 2016):

I'm on 0.3.4 and the problem exists.. any idea how to fix it?
harvester:

exports.config = {
  nodeName: "application_server",
  logStreams: {
    apache: [
      "/var/log/apache2/access.log",
      "/var/log/apache2/error.log"
    ]
  },
  server: {
    host: '0.0.0.0',
    port: 28777
  }
}

logserver:

exports.config = {
  host: '0.0.0.0',
  port: 28777
}

web_server:

exports.config = {
  host: '0.0.0.0',
  port: 28778,

  /*
  // Enable HTTP Basic Authentication
  auth: {
    user: "admin",
    pass: "1234"
  },
  */

  /*
  // Enable HTTPS/SSL
  ssl: {
    key: '/path/to/privatekey.pem',
    cert: '/path/to/certificate.pem'
  },
  */

  /*
  // Restrict access to websocket (socket.io)
  // Uses socket.io 'origins' syntax
  restrictSocket: '*:*',
  */

  /*
  // Restrict access to http server (express)
  restrictHTTP: [
    "10.13.0.*",
    "10.0.*"
  ]
  */

}

Thanks in advance

@geek-kb commented on GitHub (Dec 6, 2016): I'm on 0.3.4 and the problem exists.. any idea how to fix it? harvester: ``` exports.config = { nodeName: "application_server", logStreams: { apache: [ "/var/log/apache2/access.log", "/var/log/apache2/error.log" ] }, server: { host: '0.0.0.0', port: 28777 } } ``` logserver: ``` exports.config = { host: '0.0.0.0', port: 28777 } ``` web_server: ``` exports.config = { host: '0.0.0.0', port: 28778, /* // Enable HTTP Basic Authentication auth: { user: "admin", pass: "1234" }, */ /* // Enable HTTPS/SSL ssl: { key: '/path/to/privatekey.pem', cert: '/path/to/certificate.pem' }, */ /* // Restrict access to websocket (socket.io) // Uses socket.io 'origins' syntax restrictSocket: '*:*', */ /* // Restrict access to http server (express) restrictHTTP: [ "10.13.0.*", "10.0.*" ] */ } ``` Thanks in advance

kerem referenced this issue

2026-02-26 05:31:41 +03:00

[PR #37] [CLOSED] The log lines are now appended to the bottom of the screen, like as any terminal window. #231