starred/monolog
1
0
Fork 0
mirror of https://github.com/Seldaek/monolog.git synced 2026-04-25 23:55:56 +03:00
Code Issues 31 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #298] Logstash throws error with json codec due time format #98

New issue
Closed
opened 2026-03-04 02:12:09 +03:00 by kerem · 23 comments
kerem commented 2026-03-04 02:12:09 +03:00
Owner
Copy link

Originally created by @saimaz on GitHub (Dec 29, 2013).
Original GitHub issue: https://github.com/Seldaek/monolog/issues/298

Here's an error that logstash 1.3.2 throws using V1 LogstashFormatter:

A plugin had an unrecoverable error. Will restart this plugin.
  Plugin: <LogStash::Inputs::File path=>["/Users/simas/Sites/kibana/logger/logs/app.log"], type=>"app", start_position=>"end">
  Error: Invalid format: "2013-12-29T22:11:23.429748+02:00" is malformed at "748+02:00" {:level=>:error}

Seems that logstash expects UTC timestamp without timezone, somethning like this: 2013-12-29T22:11:23.429Z

Originally created by @saimaz on GitHub (Dec 29, 2013). Original GitHub issue: https://github.com/Seldaek/monolog/issues/298 Here's an error that logstash 1.3.2 throws using V1 LogstashFormatter: ``` A plugin had an unrecoverable error. Will restart this plugin. Plugin: <LogStash::Inputs::File path=>["/Users/simas/Sites/kibana/logger/logs/app.log"], type=>"app", start_position=>"end"> Error: Invalid format: "2013-12-29T22:11:23.429748+02:00" is malformed at "748+02:00" {:level=>:error} ``` Seems that logstash expects UTC timestamp without timezone, somethning like this: `2013-12-29T22:11:23.429Z`
kerem closed this issue 2026-03-04 02:12:10 +03:00
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@saimaz commented on GitHub (Dec 29, 2013):

The problem is because of joda parser format set in logstash.

github.com/logstash/logstash@d5ba312238/lib/logstash/time_addon.rb (L20)

So logstash expects miliseconds with 3 digits.

<!-- gh-comment-id:31325685 --> @saimaz commented on GitHub (Dec 29, 2013): The problem is because of joda parser format set in logstash. https://github.com/logstash/logstash/blob/d5ba31223835e214acd63c268d744a1aa6331e05/lib/logstash/time_addon.rb#L20 So logstash expects miliseconds with 3 digits.
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@Seldaek commented on GitHub (Dec 30, 2013):

Could you report this on logstash's side too and link this issue? Because I'm thinking they probably want to be more liberal in what format they accept after the decimal if possible. More precision shouldn't hurt, and php's datetime only supports microseconds and not milliseconds so we would have to artificially trim the output.

<!-- gh-comment-id:31340731 --> @Seldaek commented on GitHub (Dec 30, 2013): Could you report this on logstash's side too and link this issue? Because I'm thinking they probably want to be more liberal in what format they accept after the decimal if possible. More precision shouldn't hurt, and php's datetime only supports microseconds and not milliseconds so we would have to artificially trim the output.
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@saimaz commented on GitHub (Dec 30, 2013):

Yes i already told them in IRC that on php there is no milliseconds, and i'm in trouble. And that format is probably for better performance.

Here's the ticket i have created in Jira: https://logstash.jira.com/browse/LOGSTASH-1738

I already have fixed this in my fork, but I don't know is that worth to open a pull request. IMO lets wait what logstash crew will say for this.

<!-- gh-comment-id:31341498 --> @saimaz commented on GitHub (Dec 30, 2013): Yes i already told them in IRC that on php there is no milliseconds, and i'm in trouble. And that format is probably for better performance. Here's the ticket i have created in Jira: https://logstash.jira.com/browse/LOGSTASH-1738 I already have fixed this in my fork, but I don't know is that worth to open a pull request. IMO lets wait what logstash crew will say for this.
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@Seldaek commented on GitHub (Dec 30, 2013):

OK cool then please keep this updated once you get news from them and we
will see what we do about it.

<!-- gh-comment-id:31341547 --> @Seldaek commented on GitHub (Dec 30, 2013): OK cool then please keep this updated once you get news from them and we will see what we do about it.
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@saimaz commented on GitHub (Dec 30, 2013):

In Beaver (python): https://github.com/josegonzalez/beaver, they also got this problem.

https://github.com/josegonzalez/beaver/pull/230

<!-- gh-comment-id:31341577 --> @saimaz commented on GitHub (Dec 30, 2013): In Beaver (python): https://github.com/josegonzalez/beaver, they also got this problem. https://github.com/josegonzalez/beaver/pull/230
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@guillaumepotier commented on GitHub (Jan 2, 2014):

Very interested by this issue outcome, I also have problems now with the microsecond format :( Best

<!-- gh-comment-id:31462927 --> @guillaumepotier commented on GitHub (Jan 2, 2014): Very interested by this issue outcome, I also have problems now with the microsecond format :( Best
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@aledelgo commented on GitHub (Jan 17, 2014):

same problem here, using the Monolog LogstashFormatter...

<!-- gh-comment-id:32615507 --> @aledelgo commented on GitHub (Jan 17, 2014): same problem here, using the Monolog LogstashFormatter...
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@saimaz commented on GitHub (Jan 24, 2014):

Logstash fixed time format issue in v1.3.3

<!-- gh-comment-id:33201267 --> @saimaz commented on GitHub (Jan 24, 2014): Logstash fixed time format issue in v1.3.3
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@aledelgo commented on GitHub (Jan 24, 2014):

I don't think it's fixed. I still have the same problem:
Error: Invalid format: "2014-01-21T22:12:13+02:00" is malformed at "+02:00"

I have Logstash 1.3.3-1+debian installed via apt-get.

<!-- gh-comment-id:33204350 --> @aledelgo commented on GitHub (Jan 24, 2014): I don't think it's fixed. I still have the same problem: Error: Invalid format: "2014-01-21T22:12:13+02:00" is malformed at "+02:00" I have Logstash 1.3.3-1+debian installed via apt-get.
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@Seldaek commented on GitHub (Feb 6, 2014):

@aledelgo maybe because you're missing the microseconds there?

<!-- gh-comment-id:34278791 --> @Seldaek commented on GitHub (Feb 6, 2014): @aledelgo maybe because you're missing the microseconds there?
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@aledelgo commented on GitHub (Feb 6, 2014):

@Seldaek yes, it maybe, but this is a problem as on apache i have no way to set subseconds without faking it with a fixed .000 - actually i do so but then i have this error:

{:timestamp=>"2014-02-05T14:49:16.338000+0100", :message=>"Failed parsing date from field", :field=>"@timestamp", :value=>"2014-02-05T14:49:16.000+01:00", :exception=>#<TypeError: cannot convert instance of class org.jruby.RubyTime to class java.lang.String>, :level=>:warn}

<!-- gh-comment-id:34301888 --> @aledelgo commented on GitHub (Feb 6, 2014): @Seldaek yes, it maybe, but this is a problem as on apache i have no way to set subseconds without faking it with a fixed .000 - actually i do so but then i have this error: {:timestamp=>"2014-02-05T14:49:16.338000+0100", :message=>"Failed parsing date from field", :field=>"@timestamp", :value=>"2014-02-05T14:49:16.000+01:00", :exception=>#<TypeError: cannot convert instance of class org.jruby.RubyTime to class java.lang.String>, :level=>:warn}
kerem commented 2026-03-04 02:12:11 +03:00
Author
Owner
Copy link

@saimaz commented on GitHub (Feb 6, 2014):

you missing a colon in the time zone

<!-- gh-comment-id:34302070 --> @saimaz commented on GitHub (Feb 6, 2014): you missing a colon in the time zone
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@aledelgo commented on GitHub (Feb 6, 2014):

@saimaz
this is the malformed field: "2014-02-05T14:49:16.000+01:00" . where should i put a colon?

<!-- gh-comment-id:34302311 --> @aledelgo commented on GitHub (Feb 6, 2014): @saimaz this is the malformed field: "2014-02-05T14:49:16.000+01:00" . where should i put a colon?
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@saimaz commented on GitHub (Feb 6, 2014):

in the beginning of message i saw ":timestamp=>"2014-02-05T14:49:16.338000+0100", but this probably is created from java.

<!-- gh-comment-id:34302643 --> @saimaz commented on GitHub (Feb 6, 2014): in the beginning of message i saw ":timestamp=>"2014-02-05T14:49:16.338000+0100", but this probably is created from java.
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@aledelgo commented on GitHub (Feb 6, 2014):

yes that is the timestamp of the logstash log.

<!-- gh-comment-id:34302820 --> @aledelgo commented on GitHub (Feb 6, 2014): yes that is the timestamp of the logstash log.
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@saimaz commented on GitHub (Feb 6, 2014):

I have tested with both "2014-02-06T12:27:33.271+02:00" and "2014-02-06T12:29:46.134421+02:00", and it's working.

What configuration you are using?

<!-- gh-comment-id:34310222 --> @saimaz commented on GitHub (Feb 6, 2014): I have tested with both "2014-02-06T12:27:33.271+02:00" and "2014-02-06T12:29:46.134421+02:00", and it's working. What configuration you are using?
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@guillaumepotier commented on GitHub (Feb 6, 2014):

I confirm that with 1.3.3 Logstash last version everything is working fine for me.

<!-- gh-comment-id:34310756 --> @guillaumepotier commented on GitHub (Feb 6, 2014): I confirm that with 1.3.3 Logstash last version everything is working fine for me.
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@aledelgo commented on GitHub (Feb 6, 2014):

@saimaz
in my configuration i have

filter {

date {
    locale => "en"
    match => ["@timestamp", "YYYY-MM-dd'T'H:mm:ssZZ", "ISO8601"]
}

}

<!-- gh-comment-id:34322869 --> @aledelgo commented on GitHub (Feb 6, 2014): @saimaz in my configuration i have filter { ``` date { locale => "en" match => ["@timestamp", "YYYY-MM-dd'T'H:mm:ssZZ", "ISO8601"] } ``` }
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@aledelgo commented on GitHub (Feb 12, 2014):

It seems to be solved after i added this to the date filter:
"YYYY-MM-dd'T'H:mm:ss.SZZ"
but this is the "ISO8601" format that was already added to my filter...

ISO8601 details here:
http://www.w3.org/TR/NOTE-datetime
YYYY-MM-DDThh:mm:ss.sTZD (eg 1997-07-16T19:20:30.45+01:00)

<!-- gh-comment-id:34852425 --> @aledelgo commented on GitHub (Feb 12, 2014): It seems to be solved after i added this to the date filter: "YYYY-MM-dd'T'H:mm:ss.SZZ" but this is the "ISO8601" format that was already added to my filter... ISO8601 details here: http://www.w3.org/TR/NOTE-datetime YYYY-MM-DDThh:mm:ss.sTZD (eg 1997-07-16T19:20:30.45+01:00)
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@momok2 commented on GitHub (Oct 1, 2014):

I have this Problem can some one help me ?
Failed parsing date from field {:field=>"message", :value=>"06 Sep 2014 00:03:35 is malformed at " Sep 2014 00:03:35,440 INFO
and that is my server time what i want to parse

<!-- gh-comment-id:57456168 --> @momok2 commented on GitHub (Oct 1, 2014): I have this Problem can some one help me ? Failed parsing date from field {:field=>"message", :value=>"06 Sep 2014 00:03:35 is malformed at " Sep 2014 00:03:35,440 INFO and that is my server time what i want to parse
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@saimaz commented on GitHub (Oct 1, 2014):

This was due logstash, and they already fixed it. Please check if you have latest logstash version.

<!-- gh-comment-id:57456296 --> @saimaz commented on GitHub (Oct 1, 2014): This was due logstash, and they already fixed it. Please check if you have latest logstash version.
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@momok2 commented on GitHub (Oct 1, 2014):

ok i have the latest version !!
but is not work so !!

<!-- gh-comment-id:57456400 --> @momok2 commented on GitHub (Oct 1, 2014): ok i have the latest version !! but is not work so !!
kerem commented 2026-03-04 02:12:12 +03:00
Author
Owner
Copy link

@momok2 commented on GitHub (Oct 1, 2014):

can some one help me in this Problem
i have the latest version of Logstash !!

<!-- gh-comment-id:57458013 --> @momok2 commented on GitHub (Oct 1, 2014): can some one help me in this Problem i have the latest version of Logstash !!
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
esfix
2.x
1.x
3.10.0
2.11.0
3.9.0
3.8.1
3.8.0
2.10.0
3.7.0
3.6.0
2.9.3
3.5.0
2.9.2
3.4.0
3.3.1
2.9.1
3.3.0
2.9.0
3.2.0
2.8.0
3.1.0
2.7.0
1.27.1
3.0.0
2.6.0
3.0.0-RC1
2.5.0
2.4.0
1.27.0
2.3.5
2.3.4
2.3.3
2.3.2
2.3.1
2.3.0
1.26.1
2.2.0
1.26.0
2.1.1
1.25.5
2.1.0
1.25.4
2.0.2
1.25.3
2.0.1
1.25.2
1.25.1
1.25.0
2.0.0
2.0.0-beta2
2.0.0-beta1
1.24.0
1.23.0
1.22.1
1.22.0
1.21.0
1.20.0
1.19.0
1.18.2
1.18.1
1.18.0
1.17.2
1.17.1
1.17.0
1.16.0
1.15.0
1.14.0
1.13.1
1.13.0
1.12.0
1.11.0
1.10.0
1.9.1
1.9.0
1.8.0
1.7.0
1.6.0
1.5.0
1.4.1
1.4.0
1.3.1
1.3.0
1.2.1
1.2.0
1.1.0
1.0.2
1.0.1
1.0.0
1.0.0-RC1
Labels
Clear labels   
Bug

   
Documentation

   
Feature

   
Needs Work

   
Support

   
pull-request

Mirrored from GitHub Pull Request

No labels
Bug
Documentation
Feature
Needs Work
Support
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/monolog#98
No description provided.