[GH-ISSUE #3186] Mail stats not working in fresh install of 2.2.4 #1790

Closed
opened 2026-02-27 11:19:08 +03:00 by kerem · 40 comments
Owner

Originally created by @pappastech on GitHub (Feb 9, 2024).
Original GitHub issue: https://github.com/modoboa/modoboa/issues/3186

  • OS Type: Debian
  • OS Version: 12.4
  • Database Type: MySQL
  • Modoboa: 2.2.4
  • installer used: Yes
  • Webserver: Nginx

Steps to reproduce

Install modoboa with installer on Debian 12.4, configure domain(s), identities, get e-mail flowing, look at stats on new-admin and they're empty. A review of the /srv/modoboa/rrdfiles shows that only the "new_accounts.rrd" file is being created.

Current behavior

Only the new_accounts.rrd file is being created as part of the statistics cron job.

rrdfile directory:
image

Manually trying to run the logparser results in the following:
image

Expected behavior

More stats rrd files should be created including an rrd file for postfix log stats from the configured postfix log file (currently /var/log/mail.log)

Originally created by @pappastech on GitHub (Feb 9, 2024). Original GitHub issue: https://github.com/modoboa/modoboa/issues/3186 * OS Type: Debian * OS Version: 12.4 * Database Type: MySQL * Modoboa: 2.2.4 * installer used: Yes * Webserver: Nginx # Steps to reproduce Install modoboa with installer on Debian 12.4, configure domain(s), identities, get e-mail flowing, look at stats on new-admin and they're empty. A review of the /srv/modoboa/rrdfiles shows that only the "new_accounts.rrd" file is being created. # Current behavior Only the new_accounts.rrd file is being created as part of the statistics cron job. rrdfile directory: ![image](https://github.com/modoboa/modoboa/assets/532780/1c59e4d0-cc4d-40fe-9693-6b0eb8257b6d) Manually trying to run the logparser results in the following: ![image](https://github.com/modoboa/modoboa/assets/532780/c902d157-10dc-4cbd-95b7-c7e7ef26221c) # Expected behavior More stats rrd files should be created including an rrd file for postfix log stats from the configured postfix log file (currently /var/log/mail.log)
kerem 2026-02-27 11:19:08 +03:00
  • closed this issue
  • added the
    stale
    label
Author
Owner

@pappastech commented on GitHub (Feb 18, 2024):

Does anyone have any advice on this issue?

<!-- gh-comment-id:1951032870 --> @pappastech commented on GitHub (Feb 18, 2024): Does anyone have any advice on this issue?
Author
Owner

@tonioo commented on GitHub (Feb 21, 2024):

@pappastech Do you have a lot of traffic on this server?

<!-- gh-comment-id:1956257480 --> @tonioo commented on GitHub (Feb 21, 2024): @pappastech Do you have a lot of traffic on this server?
Author
Owner

@pappastech commented on GitHub (Feb 21, 2024):

@pappastech Do you have a lot of traffic on this server?

Not sure on your definition of "a lot of traffic". The average is usually 20-100 e-mails per hour.

<!-- gh-comment-id:1958174045 --> @pappastech commented on GitHub (Feb 21, 2024): > @pappastech Do you have a lot of traffic on this server? Not sure on your definition of "a lot of traffic". The average is usually 20-100 e-mails per hour.
Author
Owner

@tonioo commented on GitHub (Feb 22, 2024):

@pappastech The log parser is not very efficient since it analyses the complete file every time it runs (so every 5 mins by default).
You should check how much time is required for it to complete and then adjust the frequency.

<!-- gh-comment-id:1958906905 --> @tonioo commented on GitHub (Feb 22, 2024): @pappastech The log parser is not very efficient since it analyses the complete file every time it runs (so every 5 mins by default). You should check how much time is required for it to complete and then adjust the frequency.
Author
Owner

@pappastech commented on GitHub (Feb 22, 2024):

@pappastech The log parser is not very efficient since it analyses the complete file every time it runs (so every 5 mins by default). You should check how much time is required for it to complete and then adjust the frequency.

I've run the log parser manually and it doesn't take more than 15-20 seconds. The problem is that it's not generating the rrd file at all. My guess is that it's running into an error somewhere in the processing, but doesn't log any errors itself, so I'm not sure where it's going sideways.

As a separate issue, none of the MX, DKIM, DMARC, etc. checks are working correctly at the moment either. In other words, I have all those records created in DNS, but the process to check them somehow doesn't see those records. There's a general lack of things working and it's difficult for me to troubleshoot since the cron tasks aren't logging output anywhere.

<!-- gh-comment-id:1959151209 --> @pappastech commented on GitHub (Feb 22, 2024): > @pappastech The log parser is not very efficient since it analyses the complete file every time it runs (so every 5 mins by default). You should check how much time is required for it to complete and then adjust the frequency. I've run the log parser manually and it doesn't take more than 15-20 seconds. The problem is that it's not generating the rrd file at all. My guess is that it's running into an error somewhere in the processing, but doesn't log any errors itself, so I'm not sure where it's going sideways. As a separate issue, none of the MX, DKIM, DMARC, etc. checks are working correctly at the moment either. In other words, I have all those records created in DNS, but the process to check them somehow doesn't see those records. There's a general lack of things working and it's difficult for me to troubleshoot since the cron tasks aren't logging output anywhere.
Author
Owner

@pappastech commented on GitHub (Feb 27, 2024):

I did figure out the MX, DKIM record checks. SEPM running on the Hyper-V host was causing that problem. I'm still not seeing log parsing working though; any ideas?

<!-- gh-comment-id:1965549911 --> @pappastech commented on GitHub (Feb 27, 2024): I did figure out the MX, DKIM record checks. SEPM running on the Hyper-V host was causing that problem. I'm still not seeing log parsing working though; any ideas?
Author
Owner

@layer3-fl commented on GitHub (Mar 20, 2024):

I have something similar happening on a fresh install of 2.2.4 on top of fresh install of Debian 12. This is something I happened to notice, I don't think the graphs are something I would look at frequently.

root@mail:/home/matto# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser
/srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/logparser.py:97: FutureWarning: Possible nested set at position 29
self._regex = {k: re.compile(v) for k, v in self._regex.items()}

drwxrwx--- 2 modoboa modoboa 4096 Mar 17 08:00 .
drwxr-xr-x 7 modoboa modoboa 4096 Mar 17 07:06 ..
-rw-r--r-- 1 root root 2448 Mar 20 18:00 new_accounts.rrd

My rrdfiles directory is owned by modoboa:modoboa but the present new_accounts.rrd is owned by root where as pappastech's new_accounts.rrd is owned by modoboa. The new accounts graph is working on my setup.

Does python version play any relevance?
root@mail:/home/matto# /srv/modoboa/env/bin/python -V
Python 3.11.2

<!-- gh-comment-id:2010755952 --> @layer3-fl commented on GitHub (Mar 20, 2024): I have something similar happening on a fresh install of 2.2.4 on top of fresh install of Debian 12. This is something I happened to notice, I don't think the graphs are something I would look at frequently. root@mail:/home/matto# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser /srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/logparser.py:97: FutureWarning: Possible nested set at position 29 self._regex = {k: re.compile(v) for k, v in self._regex.items()} drwxrwx--- 2 modoboa modoboa 4096 Mar 17 08:00 . drwxr-xr-x 7 modoboa modoboa 4096 Mar 17 07:06 .. -rw-r--r-- 1 root root 2448 Mar 20 18:00 new_accounts.rrd My rrdfiles directory is owned by modoboa:modoboa but the present new_accounts.rrd is owned by root where as pappastech's new_accounts.rrd is owned by modoboa. The new accounts graph is working on my setup. Does python version play any relevance? root@mail:/home/matto# /srv/modoboa/env/bin/python -V Python 3.11.2
Author
Owner

@tonioo commented on GitHub (Mar 21, 2024):

@layer3-fl And you confirm there is mail traffic on your server?

<!-- gh-comment-id:2012195203 --> @tonioo commented on GitHub (Mar 21, 2024): @layer3-fl And you confirm there is mail traffic on your server?
Author
Owner

@layer3-fl commented on GitHub (Mar 21, 2024):

@tonioo That is correct. I can also confirm my log file is /var/log/mail.log and I have not made any changes to the statistics parameters in the GUI.

<!-- gh-comment-id:2012229966 --> @layer3-fl commented on GitHub (Mar 21, 2024): @tonioo That is correct. I can also confirm my log file is /var/log/mail.log and I have not made any changes to the statistics parameters in the GUI.
Author
Owner

@mas1701 commented on GitHub (Jul 9, 2024):

Adding to this, I have the following cronjobs on Modoboa 2.2.4 on deb12 (upgraded from 1.17 on deb10, completely gotten rid of Python2, now Python 3.11):

# Logs parsing
*/5     *       *       *       *       root    $PYTHON $INSTANCE/manage.py logparser &> /dev/null
0       *       *       *       *       root    $PYTHON $INSTANCE/manage.py update_statistics

Running the first one manually:

root@mail:/srv/modoboa/rrdfiles# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser
/srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/logparser.py:97: FutureWarning: Possible nested set at position 29
  self._regex = {k: re.compile(v) for k, v in self._regex.items()}

And the second one:

root@mail:/srv/modoboa/rrdfiles# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py update_statistics
Traceback (most recent call last):
  File "/srv/modoboa/instance/manage.py", line 22, in <module>
    main()
  File "/srv/modoboa/instance/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/srv/modoboa/env/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/srv/modoboa/env/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/modoboa/env/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/modoboa/env/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/update_statistics.py", line 84, in handle
    self.update_account_creation_stats(options["rebuild"])
  File "/srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/update_statistics.py", line 79, in update_account_creation_stats
    rrdtool.update(str(db_path), *data)
rrdtool.OperationalError: /srv/modoboa/rrdfiles/new_accounts.rrd: illegal attempt to update using time 1720555200 when last update time is 1720555200 (minimum one second step)

rrd files:

root@mail:/srv/modoboa/rrdfiles# ls -la
total 524
drwxr-x--- 2 modoboa modoboa   4096 Jul  9 17:35 .
drwxr-xr-x 8 modoboa modoboa   4096 Jul  9 17:34 ..
-rw-r--r-- 1 root    root    123064 Jul  9 17:34 global.rrd
-rw-r--r-- 1 root    root    123064 Jul  9 17:34 domain1.rrd
-rw-r--r-- 1 root    root    123064 Jul  9 17:34 domain2.rrd
-rw-r--r-- 1 root    root      2448 Jul 10 00:00 new_accounts.rrd
-rw-r--r-- 1 root    root    123064 Jul  9 17:34 domain3.rrd

After the system and modoboa upgrades, there were no more mail stats. And yes, there is some mail traffic on the system.

<!-- gh-comment-id:2218837186 --> @mas1701 commented on GitHub (Jul 9, 2024): Adding to this, I have the following cronjobs on Modoboa 2.2.4 on deb12 (upgraded from 1.17 on deb10, completely gotten rid of Python2, now Python 3.11): ``` # Logs parsing */5 * * * * root $PYTHON $INSTANCE/manage.py logparser &> /dev/null 0 * * * * root $PYTHON $INSTANCE/manage.py update_statistics ``` Running the first one manually: ``` root@mail:/srv/modoboa/rrdfiles# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser /srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/logparser.py:97: FutureWarning: Possible nested set at position 29 self._regex = {k: re.compile(v) for k, v in self._regex.items()} ``` And the second one: ``` root@mail:/srv/modoboa/rrdfiles# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py update_statistics Traceback (most recent call last): File "/srv/modoboa/instance/manage.py", line 22, in <module> main() File "/srv/modoboa/instance/manage.py", line 18, in main execute_from_command_line(sys.argv) File "/srv/modoboa/env/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line utility.execute() File "/srv/modoboa/env/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/srv/modoboa/env/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv self.execute(*args, **cmd_options) File "/srv/modoboa/env/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute output = self.handle(*args, **options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/update_statistics.py", line 84, in handle self.update_account_creation_stats(options["rebuild"]) File "/srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/update_statistics.py", line 79, in update_account_creation_stats rrdtool.update(str(db_path), *data) rrdtool.OperationalError: /srv/modoboa/rrdfiles/new_accounts.rrd: illegal attempt to update using time 1720555200 when last update time is 1720555200 (minimum one second step) ``` rrd files: ``` root@mail:/srv/modoboa/rrdfiles# ls -la total 524 drwxr-x--- 2 modoboa modoboa 4096 Jul 9 17:35 . drwxr-xr-x 8 modoboa modoboa 4096 Jul 9 17:34 .. -rw-r--r-- 1 root root 123064 Jul 9 17:34 global.rrd -rw-r--r-- 1 root root 123064 Jul 9 17:34 domain1.rrd -rw-r--r-- 1 root root 123064 Jul 9 17:34 domain2.rrd -rw-r--r-- 1 root root 2448 Jul 10 00:00 new_accounts.rrd -rw-r--r-- 1 root root 123064 Jul 9 17:34 domain3.rrd ``` After the system and modoboa upgrades, there were no more mail stats. And yes, there is some mail traffic on the system.
Author
Owner

@tonioo commented on GitHub (Jul 10, 2024):

@mas1701 Have you checked both old and new UIs ?

<!-- gh-comment-id:2219754748 --> @tonioo commented on GitHub (Jul 10, 2024): @mas1701 Have you checked both old and new UIs ?
Author
Owner

@mas1701 commented on GitHub (Jul 10, 2024):

Hi tonioo,

since it does not seem production-ready yet (incomplete translations), I'm using the old UI only. Don't tell me it's deactivated there now?

I just checked the rrdfiles directory again. new_accounts.rrd gets updated, other timestamps unchanged. And it does not surprise me at all looking at the output of "update_statistics". It is also unchanged, many hours later. It's like a cat chasing it's own tail, it doesn't get anywhere.

I also still have the files of the old install in /srv/modoboa.old/rrdfiles. All the same user/group/rights settings, updated frequently when still in use.

<!-- gh-comment-id:2220261785 --> @mas1701 commented on GitHub (Jul 10, 2024): Hi tonioo, since it does not seem production-ready yet (incomplete translations), I'm using the old UI only. Don't tell me it's deactivated there now? I just checked the rrdfiles directory again. new_accounts.rrd gets updated, other timestamps unchanged. And it does not surprise me at all looking at the output of "update_statistics". It is also unchanged, many hours later. It's like a cat chasing it's own tail, it doesn't get anywhere. I also still have the files of the old install in /srv/modoboa.old/rrdfiles. All the same user/group/rights settings, updated frequently when still in use.
Author
Owner

@tonioo commented on GitHub (Jul 10, 2024):

No it should work on both versions but it could give hints. Could you please check the new UI ?

<!-- gh-comment-id:2220669738 --> @tonioo commented on GitHub (Jul 10, 2024): No it should work on both versions but it could give hints. Could you please check the new UI ?
Author
Owner

@mas1701 commented on GitHub (Jul 11, 2024):

I just re-enabled the new UI and had a look at it. Makes no difference, stats end at the same moment as with the old UI.

Modoboa-Stats-OldUI-2024-07-11-04-53-31
Modoboa-Stats-NewUI-2024-07-11-04-47-44

<!-- gh-comment-id:2221896336 --> @mas1701 commented on GitHub (Jul 11, 2024): I just re-enabled the new UI and had a look at it. Makes no difference, stats end at the same moment as with the old UI. ![Modoboa-Stats-OldUI-2024-07-11-04-53-31](https://github.com/modoboa/modoboa/assets/30816569/c61e5797-2fc3-46a9-a43b-380e5e3d260a) ![Modoboa-Stats-NewUI-2024-07-11-04-47-44](https://github.com/modoboa/modoboa/assets/30816569/ac32ce50-617f-4468-a4a7-9692e7ea403e)
Author
Owner

@tonioo commented on GitHub (Jul 11, 2024):

And they stopped since the upgrade to 2.2.4 ?

<!-- gh-comment-id:2222183174 --> @tonioo commented on GitHub (Jul 11, 2024): And they stopped since the upgrade to 2.2.4 ?
Author
Owner

@mas1701 commented on GitHub (Jul 11, 2024):

Yes. I upgraded from Debian 10 to Debian 12 and Modoboa 1.17 to 2.2.4, following instructions on how to re-build the Python virtual environment (for Python 3.11, Python 2 remains from Debian 10 completely removed) and worked through the version-specific upgrade instructions as well. After all updates, the "migrate" job was run.

There is a time difference between the actual update (I had the system shielded from newly arriving mail during the update) and the first, newly processed e-mail, but that was caused by a Postfix/Amavis problem I encountered in Debian 12. It is fixed. It would explain a gap of some hours in the stats, but not a permanent gap.

I mostly wonder about that timestamping (?) message "rrdtool.OperationalError: /srv/modoboa/rrdfiles/new_accounts.rrd: illegal attempt to update using time 1720555200 when last update time is 1720555200 (minimum one second step)".

That is from the evening right after the upgrades, when I was doing some checks (after fixing the Postfix/Amavis problem): Tue Jul 09 2024 22:00:00 GMT+0200

I just copied rrdfiles to rrdfiles.bak and emptied the rrdfiles directory with the exception of new_accounts.rrd, because that was fresh. I ran the logparser and generate_statistics jobs again and got the same error, but with a newer timestamp. No change inside of Modoboa's UI though, just more empty space, as the old stats are moving fuirther to the left.

New timestamp from error: 1720724400 (Thu Jul 11 2024 21:00:00 GMT+0200)

That matches new_accounts.rrd with an offset of exactly 2 hrs. - maybe the GMT+0200 offset applied twice.

For now, I reverted to the old rrdfiles, but also kept the new ones.

<!-- gh-comment-id:2224001476 --> @mas1701 commented on GitHub (Jul 11, 2024): Yes. I upgraded from Debian 10 to Debian 12 and Modoboa 1.17 to 2.2.4, following instructions on how to re-build the Python virtual environment (for Python 3.11, Python 2 remains from Debian 10 completely removed) and worked through the version-specific upgrade instructions as well. After all updates, the "migrate" job was run. There is a time difference between the actual update (I had the system shielded from newly arriving mail during the update) and the first, newly processed e-mail, but that was caused by a Postfix/Amavis problem I encountered in Debian 12. It is fixed. It would explain a gap of some hours in the stats, but not a permanent gap. I mostly wonder about that timestamping (?) message "rrdtool.OperationalError: /srv/modoboa/rrdfiles/new_accounts.rrd: illegal attempt to update using time 1720555200 when last update time is 1720555200 (minimum one second step)". That is from the evening right after the upgrades, when I was doing some checks (after fixing the Postfix/Amavis problem): Tue Jul 09 2024 22:00:00 GMT+0200 I just copied rrdfiles to rrdfiles.bak and emptied the rrdfiles directory with the exception of new_accounts.rrd, because that was fresh. I ran the logparser and generate_statistics jobs again and got the same error, but with a newer timestamp. No change inside of Modoboa's UI though, just more empty space, as the old stats are moving fuirther to the left. New timestamp from error: 1720724400 (Thu Jul 11 2024 21:00:00 GMT+0200) That matches new_accounts.rrd with an offset of exactly 2 hrs. - maybe the GMT+0200 offset applied twice. For now, I reverted to the old rrdfiles, but also kept the new ones.
Author
Owner

@mas1701 commented on GitHub (Jul 17, 2024):

tonioo,

is there anything new regarding above issue? Do you need additional information?

<!-- gh-comment-id:2232968508 --> @mas1701 commented on GitHub (Jul 17, 2024): tonioo, is there anything new regarding above issue? Do you need additional information?
Author
Owner

@mas1701 commented on GitHub (Jul 31, 2024):

Antoine,

I don't mean to push you since you're probably just busy, but I'd like to hear about any progress or additional information you might have on this issue.

If you need help, for example a test system that is experiencing this issue for your debugging purposes, please let me know. I might be able to help you there by providing a running copy of that system (with sensitive data removed), so you could see for yourself what happens and what doesn't.

I suppose that might be the easiest way to get this solved, unless you already have a mostly identical system at your disposal for that purpose. But I'd imagine that to be quite hard considering the sheer number of possible OS/software variations.

Thanks in advance for all your efforts.

With kind regards
Mark

<!-- gh-comment-id:2261265659 --> @mas1701 commented on GitHub (Jul 31, 2024): Antoine, I don't mean to push you since you're probably just busy, but I'd like to hear about any progress or additional information you might have on this issue. If you need help, for example a test system that is experiencing this issue for your debugging purposes, please let me know. I might be able to help you there by providing a running copy of that system (with sensitive data removed), so you could see for yourself what happens and what doesn't. I suppose that might be the easiest way to get this solved, unless you already have a mostly identical system at your disposal for that purpose. But I'd imagine that to be quite hard considering the sheer number of possible OS/software variations. Thanks in advance for all your efforts. With kind regards Mark
Author
Owner

@tonioo commented on GitHub (Aug 2, 2024):

Now that I'm thinking about it, have you checked that rsyslog is installed and running? And also, have you checked if the mail.log file has any new content?

<!-- gh-comment-id:2265023746 --> @tonioo commented on GitHub (Aug 2, 2024): Now that I'm thinking about it, have you checked that rsyslog is installed and running? And also, have you checked if the mail.log file has any new content?
Author
Owner

@mas1701 commented on GitHub (Aug 5, 2024):

Yes, rsyslogd is present and active,from "ps" command:

568 ? Ssl 0:09 /usr/sbin/rsyslogd -n -iNONE

What do you expect me to find in mail.log other than normal mail flow?

(Yes, normal mail flow is working. Although this is a test run before upgrading a larger server, this smaller one does have active mail flow and is used regularly. Any problems would be noticed very quickly. The system is also being actively monitored by an Icinga2 monitoring system, including an active client-side monitoring agent.)

Edit:
As an in-between solution for the larger server, is it possible to upgrade the plugins to the newest versions w/o upgrading the base Modoboa package or will this cause problems? Thinking about dependencies and auto-reply module.

<!-- gh-comment-id:2268586756 --> @mas1701 commented on GitHub (Aug 5, 2024): Yes, rsyslogd is present and active,from "ps" command: ` 568 ? Ssl 0:09 /usr/sbin/rsyslogd -n -iNONE` What do you expect me to find in mail.log other than normal mail flow? (Yes, normal mail flow is working. Although this is a test run before upgrading a larger server, this smaller one does have active mail flow and is used regularly. Any problems would be noticed very quickly. The system is also being actively monitored by an Icinga2 monitoring system, including an active client-side monitoring agent.) Edit: As an in-between solution for the larger server, is it possible to upgrade the plugins to the newest versions w/o upgrading the base Modoboa package or will this cause problems? Thinking about dependencies and auto-reply module.
Author
Owner

@tonioo commented on GitHub (Aug 9, 2024):

I was asking because Debian 12 does not install rsyslogd by default anymore.
If you see trafic traces in mail.log file, the parser should work. I don't know what could be the problem yet and without a way to reproduce your environment, I won't be able to understand what is going on...

<!-- gh-comment-id:2277447782 --> @tonioo commented on GitHub (Aug 9, 2024): I was asking because Debian 12 does not install rsyslogd by default anymore. If you see trafic traces in mail.log file, the parser should work. I don't know what could be the problem yet and without a way to reproduce your environment, I won't be able to understand what is going on...
Author
Owner

@mas1701 commented on GitHub (Aug 10, 2024):

I was asking because Debian 12 does not install rsyslogd by default anymore. If you see trafic traces in mail.log file, the parser should work. I don't know what could be the problem yet and without a way to reproduce your environment, I won't be able to understand what is going on...

I see. Again, wouldn't it help to provide you with access to a running copy of that system?

<!-- gh-comment-id:2282279780 --> @mas1701 commented on GitHub (Aug 10, 2024): > I was asking because Debian 12 does not install rsyslogd by default anymore. If you see trafic traces in mail.log file, the parser should work. I don't know what could be the problem yet and without a way to reproduce your environment, I won't be able to understand what is going on... I see. Again, wouldn't it help to provide you with access to a running copy of that system?
Author
Owner

@pappastech commented on GitHub (Oct 27, 2024):

After an upgrade to Modoboa 2.3.2, the mail stats on my system still are not working.

root@mail01:# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser --debug
/srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/logparser.py:103: FutureWarning: Possible nested set at position 29
  self._regex = {k: re.compile(v) for k, v in self._regex.items()}
[settings] greylisting enabled
[rrd] dealing with domain global
[rrd] dealing with domain domain.com
root@mail01:# ll /srv/modoboa/rrdfiles/
total 4
-rw-r--r-- 1 modoboa modoboa 2448 Oct 27 06:00 new_accounts.rrd

The log parser is still not generating any rrd files for global or the single domain hosted by this server. There is mail traffic. Anyone have any ideas?

Thanks.

<!-- gh-comment-id:2439984509 --> @pappastech commented on GitHub (Oct 27, 2024): After an upgrade to Modoboa 2.3.2, the mail stats on my system still are not working. ``` root@mail01:# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser --debug /srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/logparser.py:103: FutureWarning: Possible nested set at position 29 self._regex = {k: re.compile(v) for k, v in self._regex.items()} [settings] greylisting enabled [rrd] dealing with domain global [rrd] dealing with domain domain.com root@mail01:# ll /srv/modoboa/rrdfiles/ total 4 -rw-r--r-- 1 modoboa modoboa 2448 Oct 27 06:00 new_accounts.rrd ``` The log parser is still not generating any rrd files for global or the single domain hosted by this server. There is mail traffic. Anyone have any ideas? Thanks.
Author
Owner

@tonioo commented on GitHub (Oct 28, 2024):

@pappastech Have you checked in the mail.log file has content?

<!-- gh-comment-id:2441232921 --> @tonioo commented on GitHub (Oct 28, 2024): @pappastech Have you checked in the mail.log file has content?
Author
Owner

@pappastech commented on GitHub (Oct 28, 2024):

Yes. I 'tail -f' the mail.log file frequently and there's constant activity in there.

<!-- gh-comment-id:2441334104 --> @pappastech commented on GitHub (Oct 28, 2024): Yes. I 'tail -f' the mail.log file frequently and there's constant activity in there.
Author
Owner

@pappastech commented on GitHub (Oct 29, 2024):

Any thoughts here @tonioo? Is there other information I can provide that would help to understand what's going on? I've dug in a couple areas but I'm not a django expert, so not always sure where to look next.

<!-- gh-comment-id:2445422537 --> @pappastech commented on GitHub (Oct 29, 2024): Any thoughts here @tonioo? Is there other information I can provide that would help to understand what's going on? I've dug in a couple areas but I'm not a django expert, so not always sure where to look next.
Author
Owner

@tonioo commented on GitHub (Oct 31, 2024):

@pappastech Can you try to manually run the log parser using --verbose or --debug options?

<!-- gh-comment-id:2449453219 --> @tonioo commented on GitHub (Oct 31, 2024): @pappastech Can you try to manually run the log parser using --verbose or --debug options?
Author
Owner

@pappastech commented on GitHub (Oct 31, 2024):

I've tried that numerous times, but here's a fresh try and the output. Note that I've changed the actual domain hosted on this server to 'domain.com'; I'm not actually hosting 'domain.com'.

root@mail01:/# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser --verbose
/srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/logparser.py:103: FutureWarning: Possible nested set at position 29
  self._regex = {k: re.compile(v) for k, v in self._regex.items()}
root@spemail01:/# ll /srv/modoboa/rrdfiles/
total 4
-rw-r--r-- 1 root root 2448 Oct 31 16:00 new_accounts.rrd
root@mail01:/# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser --debug
/srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/logparser.py:103: FutureWarning: Possible nested set at position 29
  self._regex = {k: re.compile(v) for k, v in self._regex.items()}
[settings] greylisting enabled
[rrd] dealing with domain global
[rrd] dealing with domain domain.com
root@mail01:/# ll /srv/modoboa/rrdfiles/
total 4
-rw-r--r-- 1 root root 2448 Oct 31 16:00 new_accounts.rrd
<!-- gh-comment-id:2450821689 --> @pappastech commented on GitHub (Oct 31, 2024): I've tried that numerous times, but here's a fresh try and the output. Note that I've changed the actual domain hosted on this server to 'domain.com'; I'm not actually hosting 'domain.com'. ``` root@mail01:/# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser --verbose /srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/logparser.py:103: FutureWarning: Possible nested set at position 29 self._regex = {k: re.compile(v) for k, v in self._regex.items()} root@spemail01:/# ll /srv/modoboa/rrdfiles/ total 4 -rw-r--r-- 1 root root 2448 Oct 31 16:00 new_accounts.rrd root@mail01:/# /srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py logparser --debug /srv/modoboa/env/lib/python3.11/site-packages/modoboa/maillog/management/commands/logparser.py:103: FutureWarning: Possible nested set at position 29 self._regex = {k: re.compile(v) for k, v in self._regex.items()} [settings] greylisting enabled [rrd] dealing with domain global [rrd] dealing with domain domain.com root@mail01:/# ll /srv/modoboa/rrdfiles/ total 4 -rw-r--r-- 1 root root 2448 Oct 31 16:00 new_accounts.rrd ```
Author
Owner

@tonioo commented on GitHub (Nov 1, 2024):

Can you show a sample of your log file? Maybe that the format is different from what the parser is expecting...

<!-- gh-comment-id:2451488562 --> @tonioo commented on GitHub (Nov 1, 2024): Can you show a sample of your log file? Maybe that the format is different from what the parser is expecting...
Author
Owner

@pappastech commented on GitHub (Nov 1, 2024):

Here you go...

2024-10-27T06:43:32.765720-05:00 mail01 postfix/anvil[1529392]: statistics: max connection rate 1/60s for (submission:160.155.229.28) at Oct 27 06:39:35
2024-10-27T06:43:32.765997-05:00 mail01 postfix/anvil[1529392]: statistics: max connection count 1 for (submission:160.155.229.28) at Oct 27 06:39:35
2024-10-27T06:43:32.766029-05:00 mail01 postfix/anvil[1529392]: statistics: max cache size 4 at Oct 27 06:40:00
2024-10-27T06:49:28.958322-05:00 mail01 postfix/postscreen[1530135]: CONNECT from [208.70.128.86]:59027 to [123.123.123.108]:25
2024-10-27T06:49:28.958684-05:00 mail01 postfix/postscreen[1530135]: WHITELISTED [208.70.128.86]:59027
2024-10-27T06:49:28.958784-05:00 mail01 postfix/postscreen[1530135]: using backwards-compatible default setting respectful_logging=no for client [208.70.128.86]:59027
2024-10-27T06:49:29.251294-05:00 mail01 postfix/smtpd[1530136]: connect from smtp-in3.electric.net[208.70.128.86]
2024-10-27T06:49:29.603274-05:00 mail01 postfix/smtpd[1530136]: Anonymous TLS connection established from smtp-in3.electric.net[208.70.128.86]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
2024-10-27T06:49:29.949693-05:00 mail01 postfix/smtpd[1530136]: NOQUEUE: client=smtp-in3.electric.net[208.70.128.86]
2024-10-27T06:49:31.507979-05:00 mail01 postfix/smtpd[1530163]: connect from localhost[127.0.0.1]
2024-10-27T06:49:31.523465-05:00 mail01 postfix/smtpd[1530163]: 7FC48803CB: client=localhost[127.0.0.1], orig_client=smtp-in3.electric.net[208.70.128.86]
2024-10-27T06:49:31.526525-05:00 mail01 postfix/cleanup[1530164]: 7FC48803CB: message-id=<zGIkJ3wbT02PI7s79QOFzQ@geopod-ismtpd-33>
2024-10-27T06:49:31.571646-05:00 mail01 opendkim[1165868]: 7FC48803CB: no signing table match for 'energy@mail.beehiiv.com'
2024-10-27T06:49:31.873751-05:00 mail01 opendkim[1165868]: 7FC48803CB: signature=g0nkGUni domain=mail.beehiiv.com selector=s1 result="no signature error"
2024-10-27T06:49:31.873958-05:00 mail01 opendkim[1165868]: 7FC48803CB: DKIM verification successful
2024-10-27T06:49:31.874045-05:00 mail01 opendkim[1165868]: 7FC48803CB: s=s1 d=mail.beehiiv.com a=rsa-sha256 SSL
2024-10-27T06:49:31.918614-05:00 mail01 postfix/qmgr[1517973]: 7FC48803CB: from=<bounces+45474923-b1c9-user=domain.com@em2003.mail.beehiiv.com>, size=45137, nrcpt=1 (queue active)
2024-10-27T06:49:31.920802-05:00 mail01 postfix/smtpd[1530163]: disconnect from localhost[127.0.0.1] ehlo=1 xforward=1 mail=1 rcpt=1 data=1 quit=1 commands=6
2024-10-27T06:49:31.926577-05:00 mail01 amavis[1163800]: (1163800-10) Passed CLEAN {RelayedInbound}, [208.70.128.86]:59027 [159.183.141.16] <bounces+45474923-b1c9-user=domain.com@em2003.mail.beehiiv.com> -> <user@domain.com>, Message-ID: <zGIkJ3wbT02PI7s79QOFzQ@geopod-ismtpd-33>, mail_id: tdyazewrZlp9, Hits: -2.732, size: 44686, queued_as: 7FC48803CB, 1725 ms
2024-10-27T06:49:31.931294-05:00 mail01 postfix/smtpd[1530136]: proxy-accept: END-OF-MESSAGE: 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7FC48803CB; from=<bounces+45474923-b1c9-user=domain.com@em2003.mail.beehiiv.com> to=<user@domain.com> proto=ESMTP helo=<smtp-in3.electric.net>
2024-10-27T06:49:31.931701-05:00 mail01 postfix/smtpd[1530136]: disconnect from smtp-in3.electric.net[208.70.128.86] ehlo=2 starttls=1 mail=1 rcpt=1 bdat=2 quit=1 commands=8
2024-10-27T06:49:31.963069-05:00 mail01 dovecot: lmtp(1530166): Connect from local
2024-10-27T06:49:31.978496-05:00 mail01 dovecot: lmtp(user@domain.com)<1530166><Pa1eOcsoHmc2WRcAw3Pntw>: msgid=<zGIkJ3wbT02PI7s79QOFzQ@geopod-ismtpd-33>: saved mail to INBOX
2024-10-27T06:49:31.979848-05:00 mail01 postfix/lmtp[1530165]: 7FC48803CB: to=<user@domain.com>, relay=mail.domain.com[private/dovecot-lmtp], delay=0.46, delays=0.4/0.03/0.02/0.02, dsn=2.0.0, status=sent (250 2.0.0 <user@domain.com> Pa1eOcsoHmc2WRcAw3Pntw Saved)
2024-10-27T06:49:31.980531-05:00 mail01 postfix/qmgr[1517973]: 7FC48803CB: removed
2024-10-27T06:49:31.980684-05:00 mail01 dovecot: lmtp(1530166): Disconnect from local: Logged out (state=READY)
2024-10-27T06:52:52.112876-05:00 mail01 postfix/anvil[1530138]: statistics: max connection rate 1/60s for (smtpd:208.70.128.86) at Oct 27 06:49:29
2024-10-27T06:52:52.113149-05:00 mail01 postfix/anvil[1530138]: statistics: max connection count 1 for (smtpd:208.70.128.86) at Oct 27 06:49:29
2024-10-27T06:52:52.113187-05:00 mail01 postfix/anvil[1530138]: statistics: max cache size 1 at Oct 27 06:49:29
<!-- gh-comment-id:2451599832 --> @pappastech commented on GitHub (Nov 1, 2024): Here you go... ``` 2024-10-27T06:43:32.765720-05:00 mail01 postfix/anvil[1529392]: statistics: max connection rate 1/60s for (submission:160.155.229.28) at Oct 27 06:39:35 2024-10-27T06:43:32.765997-05:00 mail01 postfix/anvil[1529392]: statistics: max connection count 1 for (submission:160.155.229.28) at Oct 27 06:39:35 2024-10-27T06:43:32.766029-05:00 mail01 postfix/anvil[1529392]: statistics: max cache size 4 at Oct 27 06:40:00 2024-10-27T06:49:28.958322-05:00 mail01 postfix/postscreen[1530135]: CONNECT from [208.70.128.86]:59027 to [123.123.123.108]:25 2024-10-27T06:49:28.958684-05:00 mail01 postfix/postscreen[1530135]: WHITELISTED [208.70.128.86]:59027 2024-10-27T06:49:28.958784-05:00 mail01 postfix/postscreen[1530135]: using backwards-compatible default setting respectful_logging=no for client [208.70.128.86]:59027 2024-10-27T06:49:29.251294-05:00 mail01 postfix/smtpd[1530136]: connect from smtp-in3.electric.net[208.70.128.86] 2024-10-27T06:49:29.603274-05:00 mail01 postfix/smtpd[1530136]: Anonymous TLS connection established from smtp-in3.electric.net[208.70.128.86]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits) 2024-10-27T06:49:29.949693-05:00 mail01 postfix/smtpd[1530136]: NOQUEUE: client=smtp-in3.electric.net[208.70.128.86] 2024-10-27T06:49:31.507979-05:00 mail01 postfix/smtpd[1530163]: connect from localhost[127.0.0.1] 2024-10-27T06:49:31.523465-05:00 mail01 postfix/smtpd[1530163]: 7FC48803CB: client=localhost[127.0.0.1], orig_client=smtp-in3.electric.net[208.70.128.86] 2024-10-27T06:49:31.526525-05:00 mail01 postfix/cleanup[1530164]: 7FC48803CB: message-id=<zGIkJ3wbT02PI7s79QOFzQ@geopod-ismtpd-33> 2024-10-27T06:49:31.571646-05:00 mail01 opendkim[1165868]: 7FC48803CB: no signing table match for 'energy@mail.beehiiv.com' 2024-10-27T06:49:31.873751-05:00 mail01 opendkim[1165868]: 7FC48803CB: signature=g0nkGUni domain=mail.beehiiv.com selector=s1 result="no signature error" 2024-10-27T06:49:31.873958-05:00 mail01 opendkim[1165868]: 7FC48803CB: DKIM verification successful 2024-10-27T06:49:31.874045-05:00 mail01 opendkim[1165868]: 7FC48803CB: s=s1 d=mail.beehiiv.com a=rsa-sha256 SSL 2024-10-27T06:49:31.918614-05:00 mail01 postfix/qmgr[1517973]: 7FC48803CB: from=<bounces+45474923-b1c9-user=domain.com@em2003.mail.beehiiv.com>, size=45137, nrcpt=1 (queue active) 2024-10-27T06:49:31.920802-05:00 mail01 postfix/smtpd[1530163]: disconnect from localhost[127.0.0.1] ehlo=1 xforward=1 mail=1 rcpt=1 data=1 quit=1 commands=6 2024-10-27T06:49:31.926577-05:00 mail01 amavis[1163800]: (1163800-10) Passed CLEAN {RelayedInbound}, [208.70.128.86]:59027 [159.183.141.16] <bounces+45474923-b1c9-user=domain.com@em2003.mail.beehiiv.com> -> <user@domain.com>, Message-ID: <zGIkJ3wbT02PI7s79QOFzQ@geopod-ismtpd-33>, mail_id: tdyazewrZlp9, Hits: -2.732, size: 44686, queued_as: 7FC48803CB, 1725 ms 2024-10-27T06:49:31.931294-05:00 mail01 postfix/smtpd[1530136]: proxy-accept: END-OF-MESSAGE: 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7FC48803CB; from=<bounces+45474923-b1c9-user=domain.com@em2003.mail.beehiiv.com> to=<user@domain.com> proto=ESMTP helo=<smtp-in3.electric.net> 2024-10-27T06:49:31.931701-05:00 mail01 postfix/smtpd[1530136]: disconnect from smtp-in3.electric.net[208.70.128.86] ehlo=2 starttls=1 mail=1 rcpt=1 bdat=2 quit=1 commands=8 2024-10-27T06:49:31.963069-05:00 mail01 dovecot: lmtp(1530166): Connect from local 2024-10-27T06:49:31.978496-05:00 mail01 dovecot: lmtp(user@domain.com)<1530166><Pa1eOcsoHmc2WRcAw3Pntw>: msgid=<zGIkJ3wbT02PI7s79QOFzQ@geopod-ismtpd-33>: saved mail to INBOX 2024-10-27T06:49:31.979848-05:00 mail01 postfix/lmtp[1530165]: 7FC48803CB: to=<user@domain.com>, relay=mail.domain.com[private/dovecot-lmtp], delay=0.46, delays=0.4/0.03/0.02/0.02, dsn=2.0.0, status=sent (250 2.0.0 <user@domain.com> Pa1eOcsoHmc2WRcAw3Pntw Saved) 2024-10-27T06:49:31.980531-05:00 mail01 postfix/qmgr[1517973]: 7FC48803CB: removed 2024-10-27T06:49:31.980684-05:00 mail01 dovecot: lmtp(1530166): Disconnect from local: Logged out (state=READY) 2024-10-27T06:52:52.112876-05:00 mail01 postfix/anvil[1530138]: statistics: max connection rate 1/60s for (smtpd:208.70.128.86) at Oct 27 06:49:29 2024-10-27T06:52:52.113149-05:00 mail01 postfix/anvil[1530138]: statistics: max connection count 1 for (smtpd:208.70.128.86) at Oct 27 06:49:29 2024-10-27T06:52:52.113187-05:00 mail01 postfix/anvil[1530138]: statistics: max cache size 1 at Oct 27 06:49:29 ```
Author
Owner

@pappastech commented on GitHub (Nov 1, 2024):

I guess comparing that format to a different instance of Modoboa I have running for some other domains, the date format does look different. I'm not sure why that would have happened since this was a brand new install of Modoboa earlier this year.

Here's an excerpt from another instance where the date format is different and log parsing works:

Nov  1 05:47:16 ptmail01 postfix/dnsblog[32009]: addr 185.196.11.195 listed by domain zen.spamhaus.org as 127.0.0.2
Nov  1 05:47:16 ptmail01 postfix/dnsblog[32009]: addr 185.196.11.195 listed by domain zen.spamhaus.org as 127.0.0.9
Nov  1 05:47:16 ptmail01 postfix/dnsblog[32009]: addr 185.196.11.195 listed by domain zen.spamhaus.org as 127.0.0.4
Nov  1 05:47:22 ptmail01 postfix/postscreen[31181]: DNSBL rank 3 for [185.196.11.195]:54455
Nov  1 05:47:23 ptmail01 postfix/postscreen[31181]: HANGUP after 0.42 from [185.196.11.195]:54455 in tests after SMTP handshake
Nov  1 05:47:23 ptmail01 postfix/postscreen[31181]: DISCONNECT [185.196.11.195]:54455

Does the log parser handle the difference in date formats?

<!-- gh-comment-id:2451611322 --> @pappastech commented on GitHub (Nov 1, 2024): I guess comparing that format to a different instance of Modoboa I have running for some other domains, the date format does look different. I'm not sure why that would have happened since this was a brand new install of Modoboa earlier this year. Here's an excerpt from another instance where the date format is different and log parsing works: ``` Nov 1 05:47:16 ptmail01 postfix/dnsblog[32009]: addr 185.196.11.195 listed by domain zen.spamhaus.org as 127.0.0.2 Nov 1 05:47:16 ptmail01 postfix/dnsblog[32009]: addr 185.196.11.195 listed by domain zen.spamhaus.org as 127.0.0.9 Nov 1 05:47:16 ptmail01 postfix/dnsblog[32009]: addr 185.196.11.195 listed by domain zen.spamhaus.org as 127.0.0.4 Nov 1 05:47:22 ptmail01 postfix/postscreen[31181]: DNSBL rank 3 for [185.196.11.195]:54455 Nov 1 05:47:23 ptmail01 postfix/postscreen[31181]: HANGUP after 0.42 from [185.196.11.195]:54455 in tests after SMTP handshake Nov 1 05:47:23 ptmail01 postfix/postscreen[31181]: DISCONNECT [185.196.11.195]:54455 ``` Does the log parser handle the difference in date formats?
Author
Owner

@pappastech commented on GitHub (Nov 1, 2024):

One difference from a fresh install point of view is that the server where log parsing works is running CentOS and the server where log parsing doesn't work is running Debian. It occurs to me that the default logging behavior is possibly different just because of different Linux distros. Looking into how to change this on the Debian server...open to suggestions if people have them.

I'd prefer to not change default behavior in the OS though...I think this should be a change in the logparser function of Modoboa. So looking at that as well.

<!-- gh-comment-id:2451631235 --> @pappastech commented on GitHub (Nov 1, 2024): One difference from a fresh install point of view is that the server where log parsing works is running CentOS and the server where log parsing doesn't work is running Debian. It occurs to me that the default logging behavior is possibly different just because of different Linux distros. Looking into how to change this on the Debian server...open to suggestions if people have them. I'd prefer to not change default behavior in the OS though...I think this should be a change in the logparser function of Modoboa. So looking at that as well.
Author
Owner

@tonioo commented on GitHub (Nov 1, 2024):

And you configured the right file path in Modoboa?

<!-- gh-comment-id:2451668271 --> @tonioo commented on GitHub (Nov 1, 2024): And you configured the right file path in Modoboa?
Author
Owner

@pappastech commented on GitHub (Nov 1, 2024):

Alright, well I think the issue is that the regex in logparser.py only allowed for UTC+ time zones. I added a single line to the list of possible regex structures and now I see rrd files after running the logparser and I see stats in the web interface. There's probably a more elegant way to handle the regex option of a + versus a - in the timezone, but this seems to work.

From:

self._date_expressions = [
            r"(?P<month>\w+)\s+(?P<day>\d+)\s+(?P<hour>\d+):(?P<min>\d+):(?P<sec>\d+)(?P<eol>.*)",  # noqa
            r"(?P<year>\d+)-(?P<month>\d+)-(?P<day>\d+)T(?P<hour>\d+):(?P<min>\d+):(?P<sec>\d+)\.\d+\+\d+:\d+(?P<eol>.*)",  # noqa
        ]

To:

self._date_expressions = [
            r"(?P<month>\w+)\s+(?P<day>\d+)\s+(?P<hour>\d+):(?P<min>\d+):(?P<sec>\d+)(?P<eol>.*)",  # noqa
            r"(?P<year>\d+)-(?P<month>\d+)-(?P<day>\d+)T(?P<hour>\d+):(?P<min>\d+):(?P<sec>\d+)\.\d+\+\d+:\d+(?P<eol>.*)",  # noqa
            r"(?P<year>\d+)-(?P<month>\d+)-(?P<day>\d+)T(?P<hour>\d+):(?P<min>\d+):(?P<sec>\d+)\.\d+\-\d+:\d+(?P<eol>.*)",  # noqa
        ]
<!-- gh-comment-id:2451668793 --> @pappastech commented on GitHub (Nov 1, 2024): Alright, well I think the issue is that the regex in logparser.py only allowed for UTC+ time zones. I added a single line to the list of possible regex structures and now I see rrd files after running the logparser and I see stats in the web interface. There's probably a more elegant way to handle the regex option of a + versus a - in the timezone, but this seems to work. From: ``` self._date_expressions = [ r"(?P<month>\w+)\s+(?P<day>\d+)\s+(?P<hour>\d+):(?P<min>\d+):(?P<sec>\d+)(?P<eol>.*)", # noqa r"(?P<year>\d+)-(?P<month>\d+)-(?P<day>\d+)T(?P<hour>\d+):(?P<min>\d+):(?P<sec>\d+)\.\d+\+\d+:\d+(?P<eol>.*)", # noqa ] ``` To: ``` self._date_expressions = [ r"(?P<month>\w+)\s+(?P<day>\d+)\s+(?P<hour>\d+):(?P<min>\d+):(?P<sec>\d+)(?P<eol>.*)", # noqa r"(?P<year>\d+)-(?P<month>\d+)-(?P<day>\d+)T(?P<hour>\d+):(?P<min>\d+):(?P<sec>\d+)\.\d+\+\d+:\d+(?P<eol>.*)", # noqa r"(?P<year>\d+)-(?P<month>\d+)-(?P<day>\d+)T(?P<hour>\d+):(?P<min>\d+):(?P<sec>\d+)\.\d+\-\d+:\d+(?P<eol>.*)", # noqa ] ```
Author
Owner

@tonioo commented on GitHub (Nov 1, 2024):

Good catch!
Do you think you could create a PR with your fix?

<!-- gh-comment-id:2451672758 --> @tonioo commented on GitHub (Nov 1, 2024): Good catch! Do you think you could create a PR with your fix?
Author
Owner

@layer3-fl commented on GitHub (Nov 1, 2024):

Great find and fix!

Having had other projects taking priority I have had this on the back burner and haven’t looked at it further.

Your fix worked for my Debian based system as well.

Warm regards,
Mattco Incorporated dba, Layer3
Matthew Strowbridge
(o) 321.541.4159
(m) 321.722.7571
(e) @.***
Federal Cage Code: 9QPB1
NAICS Codes: 541512, 541519

On Nov 1, 2024, at 6:41 AM, Antoine Nguyen @.***> wrote:

Good catch!
Do you think you could create a PR with your fix?


Reply to this email directly, view it on GitHub https://github.com/modoboa/modoboa/issues/3186#issuecomment-2451672758, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHCJ6OFBQK5MNRYZGXZXJNDZ6NLHLAVCNFSM6AAAAABDCDTNPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJRGY3TENZVHA.
You are receiving this because you were mentioned.

<!-- gh-comment-id:2452093044 --> @layer3-fl commented on GitHub (Nov 1, 2024): Great find and fix! Having had other projects taking priority I have had this on the back burner and haven’t looked at it further. Your fix worked for my Debian based system as well. Warm regards, Mattco Incorporated dba, Layer3 Matthew Strowbridge (o) 321.541.4159 (m) 321.722.7571 (e) ***@***.*** Federal Cage Code: 9QPB1 NAICS Codes: 541512, 541519 On Nov 1, 2024, at 6:41 AM, Antoine Nguyen ***@***.***> wrote: Good catch! Do you think you could create a PR with your fix? — Reply to this email directly, view it on GitHub <https://github.com/modoboa/modoboa/issues/3186#issuecomment-2451672758>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BHCJ6OFBQK5MNRYZGXZXJNDZ6NLHLAVCNFSM6AAAAABDCDTNPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJRGY3TENZVHA>. You are receiving this because you were mentioned.
Author
Owner

@pappastech commented on GitHub (Nov 1, 2024):

Good catch! Do you think you could create a PR with your fix?

It's not obvious to me how to do this on Github.

<!-- gh-comment-id:2452166720 --> @pappastech commented on GitHub (Nov 1, 2024): > Good catch! Do you think you could create a PR with your fix? It's not obvious to me how to do this on Github.
Author
Owner

@tonioo commented on GitHub (Nov 1, 2024):

You first need fork this repo and clone it locally, then commit your changes to your fork and go back to Github. There, you'll be asked if you want to create a PR

<!-- gh-comment-id:2452170582 --> @tonioo commented on GitHub (Nov 1, 2024): You first need fork this repo and clone it locally, then commit your changes to your fork and go back to Github. There, you'll be asked if you want to create a PR
Author
Owner

@pappastech commented on GitHub (Nov 1, 2024):

Alright...I think it's done.

<!-- gh-comment-id:2452366995 --> @pappastech commented on GitHub (Nov 1, 2024): Alright...I think it's done.
Author
Owner

@stale[bot] commented on GitHub (Jan 31, 2025):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

<!-- gh-comment-id:2628548710 --> @stale[bot] commented on GitHub (Jan 31, 2025): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/modoboa-modoboa#1790
No description provided.