[GH-ISSUE #13 ] [Bug] Crash after some time in the list view, with no user action #12

Author

Owner

@d99kris commented on GitHub (Jan 16, 2020):

Sorry if you are hearing so much from me lately. :O

Haha, no worries. :)

Ok that sounds like a bad bug. I haven't come across anything similar before. In the log file I see some low-level libetpan (the library nmail uses for receiving and sending mail) calls returning MAILIMAP_ERROR_STREAM before the crash. I suspect these could be caused by network connectivity issues, but there are probably other possibilities as well.

If you open nmail again and try to access this folder, does it crash immediately again? Or is it somewhat intermittent?

@d99kris commented on GitHub (Jan 16, 2020): > Sorry if you are hearing so much from me lately. :O Haha, no worries. :) Ok that sounds like a bad bug. I haven't come across anything similar before. In the log file I see some low-level `libetpan` (the library `nmail` uses for receiving and sending mail) calls returning MAILIMAP_ERROR_STREAM before the crash. I suspect these could be caused by network connectivity issues, but there are probably other possibilities as well. If you open `nmail` again and try to access this folder, does it crash immediately again? Or is it somewhat intermittent?

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jan 16, 2020):

It is not intermittent, it is crashing all the time, but sometimes in one second and sometimes in just a fraction of a second. It is the same from different networks and different machines (I tried the now usual Xubuntu chroot from SFOS, SFOS, and Solus). Networks I tried are work, home, and with 3G/4G. It crashes just in the deleted folders that contains all those messages, not in other folders that either contain no messages, or just a few.

I don't know if it really is related to the number of messages being fetched, because one other mail account I set up (disroot.org) had more than 15k messages in its folder and it fetched correctly). The crashes only happen with the Exchange account.

prefetch_level is set to =1 for all my accounts.

@Kabouik commented on GitHub (Jan 16, 2020): It is not intermittent, it is crashing all the time, but sometimes in one second and sometimes in just a fraction of a second. It is the same from different networks and different machines (I tried the now usual Xubuntu chroot from SFOS, SFOS, and Solus). Networks I tried are work, home, and with 3G/4G. It crashes just in the deleted folders that contains all those messages, not in other folders that either contain no messages, or just a few. I don't know if it really is related to the number of messages being fetched, because one other mail account I set up (disroot.org) had more than 15k messages in its folder and it fetched correctly). The crashes only happen with the Exchange account. `prefetch_level` is set to `=1` for all my accounts.

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 20, 2020):

Thanks for sharing the additional details. I think I'll need to add more debug logging to nmail in order to troubleshoot this bug. I'll update here once I've added such logging.

@d99kris commented on GitHub (Jan 20, 2020): Thanks for sharing the additional details. I think I'll need to add more debug logging to `nmail` in order to troubleshoot this bug. I'll update here once I've added such logging.

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 21, 2020):

Improved debug logging has been implemented as of 2ecf580 - if possible, try latest nmail and share the new log file. The new verbose logging may log email message content, so please review the log file before sharing it and remove any email message data that you don't want to share.

@d99kris commented on GitHub (Jan 21, 2020): Improved debug logging has been implemented as of 2ecf580 - if possible, try latest `nmail` and share the new log file. The new verbose logging may log email message content, so please review the log file before sharing it and remove any email message data that you don't want to share.

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 21, 2020):

As an alternative to posting the log-file publicly in this issue, you may send it directly to me - d99kris at gmail dot com - and I will delete the email directly after analysing the log file, and not read/use any email message content.

@d99kris commented on GitHub (Jan 21, 2020): As an alternative to posting the log-file publicly in this issue, you may send it directly to me - d99kris at gmail dot com - and I will delete the email directly after analysing the log file, and not read/use any email message content.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jan 21, 2020):

Great. I'll send you that in a minute. Regarding the new debug logging feature, maybe it would be good to have two options for comprehensive logging or just the previous level of details, since the new comprehensive logging can cause unnecessary privacy issues in other cases?

@Kabouik commented on GitHub (Jan 21, 2020): Great. I'll send you that in a minute. Regarding the new debug logging feature, maybe it would be good to have two options for comprehensive logging or just the previous level of details, since the new comprehensive logging can cause unnecessary privacy issues in other cases?

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 28, 2020):

Thanks for sharing the log file. Somehow gmail's aggressive spam-filtering had marked your email as spam, so I only saw it now. I'll take a look now.

@d99kris commented on GitHub (Jan 28, 2020): Thanks for sharing the log file. Somehow gmail's aggressive spam-filtering had marked your email as spam, so I only saw it now. I'll take a look now.

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 28, 2020):

Hi, I suspect the issue occurs because nmail fetches message flags (read/unread, etc) for too many messages at a time. It looks like Outlook prefers imap requests to be max 10kb. So I've implemented some restriction, to only request flags for 1000 messages at a time. Hopefully this fixes the issue you've been seeing.

Please help to close the issue if it does solve the problem, or let me know if there are still issues.

@d99kris commented on GitHub (Jan 28, 2020): Hi, I suspect the issue occurs because nmail fetches message flags (read/unread, etc) for too many messages at a time. It looks like Outlook prefers imap requests to be max 10kb. So I've implemented some restriction, to only request flags for 1000 messages at a time. Hopefully this fixes the issue you've been seeing. Please help to close the issue if it does solve the problem, or let me know if there are still issues.

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 28, 2020):

Regarding the new debug logging feature, maybe it would be good to have two options for comprehensive logging or just the previous level of details, since the new comprehensive logging can cause unnecessary privacy issues in other cases?

Thanks for the feedback, I'll see what I can do to have a logging level that is still useful for debugging without causing privacy issues. I'll probably report a separate Github issue for this.

Edit: I've logged #19 for this.

@d99kris commented on GitHub (Jan 28, 2020): > Regarding the new debug logging feature, maybe it would be good to have two options for comprehensive logging or just the previous level of details, since the new comprehensive logging can cause unnecessary privacy issues in other cases? Thanks for the feedback, I'll see what I can do to have a logging level that is still useful for debugging without causing privacy issues. I'll probably report a separate Github issue for this. Edit: I've logged #19 for this.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jan 28, 2020):

Unfortunately I am still having the issue with nmail crashing when fetching the big folder on the Exchange account. I tried again after deleting the cache but the issue persisted.

@Kabouik commented on GitHub (Jan 28, 2020): Unfortunately I am still having the issue with nmail crashing when fetching the big folder on the Exchange account. I tried again after deleting the cache but the issue persisted.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jan 28, 2020):

Wait, dumb myself forgot to git pull before building (…).

@Kabouik commented on GitHub (Jan 28, 2020): Wait, dumb myself forgot to `git pull` before building (…).

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jan 28, 2020):

Yup, seems to work great now, thanks!

@Kabouik commented on GitHub (Jan 28, 2020): Yup, seems to work great now, thanks!

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jan 28, 2020):

However, this account still seems to have issue for sending emails. Receiving works, saving drafts works (but see #17), but sending always fails, with or without attachments, from drafts or newly composed message, while similar messages sent from other accounts work fine. Did you see something in the logs that may explain it? Else let me know and I can send you other logs when attempting to send an email.

@Kabouik commented on GitHub (Jan 28, 2020): However, this account still seems to have issue for sending emails. Receiving works, saving drafts works (but see #17), but sending always fails, with or without attachments, from drafts or newly composed message, while similar messages sent from other accounts work fine. Did you see something in the logs that may explain it? Else let me know and I can send you other logs when attempting to send an email.

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 29, 2020):

Yup, seems to work great now, thanks!

Thanks for confirming!

However, this account still seems to have issue for sending emails.

May I propose we track that as a separate Github issue? I've reported #22 to track it.

Did you see something in the logs that may explain it? Else let me know and I can send you other logs when attempting to send an email.

I did not really look for any such things, and I have already deleted the log-file and your email. Could you please try reproduce the sending problem and share the log file with me?

I will proceed to close this issue.

@d99kris commented on GitHub (Jan 29, 2020): > Yup, seems to work great now, thanks! Thanks for confirming! > However, this account still seems to have issue for sending emails. May I propose we track that as a separate Github issue? I've reported #22 to track it. > Did you see something in the logs that may explain it? Else let me know and I can send you other logs when attempting to send an email. I did not really look for any such things, and I have already deleted the log-file and your email. Could you please try reproduce the sending problem and share the log file with me? I will proceed to close this issue.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jan 30, 2020):

There seems to be remnants of this bug. The nmail session running for this Exchange account successfully fetches message, but it eventually crashes from the message list (without me doing anything).

unexpected termination: 13

0   0x00000000004782af  Util::SignalHandler(int)
1   0x00007fd0d12fb850  
2   0x00007fd0d1a5d7ef  __write
3   0x00007fd0d1906775  
4   0x00007fd0d1904674  BIO_write
5   0x00007fd0d1b5aba0  ssl3_write_pending
6   0x00007fd0d1b5b6ee  ssl3_write_bytes
7   0x00007fd0d1a94cdc  
8   0x00007fd0d1a93267  mailstream_low_write
9   0x00007fd0d1a93cc8  mailstream_flush
a   0x00007fd0d1a9af99  mailimap_select_condstore_optional
b   0x00007fd0d1a9be8e  mailimap_select
c   0x000000000042feb3  Imap::SelectFolder(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)
d   0x00000000004318b2  Imap::GetUids(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, std::set<unsigned int, std::less<unsigned int>, std::allocator<unsigned int> >&)
e   0x000000000043d678  ImapManager::PerformRequest(ImapManager::Request const&, bool, bool)
f   0x000000000043f186  ImapManager::Process()
10  0x0000000000440b44  std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (ImapManager::*)(), ImapManager*> > >::_M_run()
11  0x00007fd0d16d84a0  
12  0x00007fd0d1a5410e  
13  0x00007fd0d13d1fff  clone

@Kabouik commented on GitHub (Jan 30, 2020): There seems to be remnants of this bug. The nmail session running for this Exchange account successfully fetches message, but it eventually crashes from the message list (without me doing anything). ``` unexpected termination: 13 0 0x00000000004782af Util::SignalHandler(int) 1 0x00007fd0d12fb850 2 0x00007fd0d1a5d7ef __write 3 0x00007fd0d1906775 4 0x00007fd0d1904674 BIO_write 5 0x00007fd0d1b5aba0 ssl3_write_pending 6 0x00007fd0d1b5b6ee ssl3_write_bytes 7 0x00007fd0d1a94cdc 8 0x00007fd0d1a93267 mailstream_low_write 9 0x00007fd0d1a93cc8 mailstream_flush a 0x00007fd0d1a9af99 mailimap_select_condstore_optional b 0x00007fd0d1a9be8e mailimap_select c 0x000000000042feb3 Imap::SelectFolder(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) d 0x00000000004318b2 Imap::GetUids(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, std::set<unsigned int, std::less<unsigned int>, std::allocator<unsigned int> >&) e 0x000000000043d678 ImapManager::PerformRequest(ImapManager::Request const&, bool, bool) f 0x000000000043f186 ImapManager::Process() 10 0x0000000000440b44 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (ImapManager::*)(), ImapManager*> > >::_M_run() 11 0x00007fd0d16d84a0 12 0x00007fd0d1a5410e 13 0x00007fd0d13d1fff clone ```

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 30, 2020):

Hm.. That is odd.. Could you please share the log-file as well, if you're able to reproduce the crash?

@d99kris commented on GitHub (Jan 30, 2020): Hm.. That is odd.. Could you please share the log-file as well, if you're able to reproduce the crash?

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jan 30, 2020):

Will do. I do not know how to reproduce it on demand, but it eventually occurs after minutes or perhaps hours; I just need to enable logging and be patient.

How can I record logs the old way, without saving the content of all messages? If I have to let nmail log data for minutes or hours, the file will be huge and will contain a lot of private information.

@Kabouik commented on GitHub (Jan 30, 2020): Will do. I do not know how to reproduce it on demand, but it eventually occurs after minutes or perhaps hours; I just need to enable logging and be patient. How can I record logs the old way, without saving the content of all messages? If I have to let nmail log data for minutes or hours, the file will be huge and will contain a lot of private information.

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 31, 2020):

Just run nmail without the -e flag, and make sure verbose_logging=0 in the config file ~/.nmail/main.conf.

@d99kris commented on GitHub (Jan 31, 2020): Just run `nmail` without the `-e` flag, and make sure `verbose_logging=0` in the config file `~/.nmail/main.conf`.

kerem commented

Author

Owner

@d99kris commented on GitHub (Feb 5, 2020):

With the latest changes for #19 it should be fine to use verbose_logging=1 (or -e flag). If the resulting log file is very large you can try zip it, I think the compression ratio should be good.

@d99kris commented on GitHub (Feb 5, 2020): With the latest changes for #19 it should be fine to use `verbose_logging=1` (or `-e` flag). If the resulting log file is very large you can try zip it, I think the compression ratio should be good.

kerem commented

Author

Owner

@d99kris commented on GitHub (Feb 10, 2020):

It would also be useful if you could try reproduce the problem with the latest version of nmail as it adds some improved logging.

Also, for for this latest crash, which OS did you observe it on, and what type of Internet connection was in use?

Thanks!

@d99kris commented on GitHub (Feb 10, 2020): It would also be useful if you could try reproduce the problem with the latest version of `nmail` as it adds some improved logging. Also, for for this latest crash, which OS did you observe it on, and what type of Internet connection was in use? Thanks!

kerem commented

Author

Owner

@Kabouik commented on GitHub (Feb 18, 2020):

Before I collect some logs and send them to you, just a short update to say that I noticed the issue can happen with non-Exchange accounts too. I observed it with my Disroot accounts today too.

I observed it on Solus (with libetpan missing some feature, mentioned it to the Solus dev team) with both Ethernet and Wifi. I'm not sure if I saw it on Xubuntu from chroot on my phone, it seems to be more stable, but I'll keep an eye on it.

@Kabouik commented on GitHub (Feb 18, 2020): Before I collect some logs and send them to you, just a short update to say that I noticed the issue can happen with non-Exchange accounts too. I observed it with my Disroot accounts today too. I observed it on Solus (with `libetpan` missing some feature, mentioned it to the Solus dev team) with both Ethernet and Wifi. I'm not sure if I saw it on Xubuntu from chroot on my phone, it seems to be more stable, but I'll keep an eye on it.

kerem commented

Author

Owner

@d99kris commented on GitHub (Feb 19, 2020):

Thanks, this is useful information! When I get a chance I will try to reproduce it using my Solus installation with disroot.

@d99kris commented on GitHub (Feb 19, 2020): Thanks, this is useful information! When I get a chance I will try to reproduce it using my Solus installation with disroot.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Feb 19, 2020):

I think I only get this issue on Solus, so it might be related to libetpan. It would be interesting to see if you can reproduce it with libetpan from repository and compare with the version you built yourself.

@Kabouik commented on GitHub (Feb 19, 2020): I think I only get this issue on Solus, so it might be related to `libetpan`. It would be interesting to see if you can reproduce it with `libetpan` from repository and compare with the version you built yourself.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Feb 21, 2020):

I built libetpan on Solus and still observe the issue. Let me know in case you fail to reproduce it with your installation and I'll try to collect some logs.

@Kabouik commented on GitHub (Feb 21, 2020): I built `libetpan` on Solus and still observe the issue. Let me know in case you fail to reproduce it with your installation and I'll try to collect some logs.

kerem commented

Author

Owner

@d99kris commented on GitHub (Feb 22, 2020):

I am able to see issues (such as "Get headers failed" and other "Get ... failed") with the combination Solus Linux and Disroot. It is possible the root cause to your crash is the same as the errors I see, but it would be good to confirm, for example by sharing the log. Do you also get "Get ... failed" errors btw?

Interestingly I don't see issues with Disroot on other OS'es (macOS or Ubuntu).
Nor do I see issues with Gmail or Hotmail under Solus Linux.

@d99kris commented on GitHub (Feb 22, 2020): I am able to see issues (such as "Get headers failed" and other "Get ... failed") with the combination Solus Linux and Disroot. It is possible the root cause to your crash is the same as the errors I see, but it would be good to confirm, for example by sharing the log. Do you also get "Get ... failed" errors btw? Interestingly I don't see issues with Disroot on other OS'es (macOS or Ubuntu). Nor do I see issues with Gmail or Hotmail under Solus Linux.

kerem commented

Author

Owner

@d99kris commented on GitHub (Mar 8, 2020):

I've worked around the issue that I saw on my Solus Linux installation using disroot. Hopefully it fixes the issue for you too, if not please re-open the issue.

Dev notes: Solus Linux uses an end-of-life version of OpenSSL (1.0.2) which does not have the built-in thread-safety that OpenSSL >= 1.1.0 provides. When nmail AES decrypt operations fail (in this case due to trying to decrypt an empty file buffer) the global SSL state is affected, which may affect SSL communication with an email server. During troubleshooting an attempt was made to support OpenSSL 1.0.2 by registering necessary thread safety routines, only issue is that libetpan (which nmail uses for email server communication) overrides such registered routines without checking if already set or not. As OpenSSL 1.0.2 is EOL I decided to not to officially support this version, but still implement a workaround for 1.0.2 users which avoids a typical AES decrypt error for empty buffers.

@d99kris commented on GitHub (Mar 8, 2020): I've worked around the issue that I saw on my Solus Linux installation using disroot. Hopefully it fixes the issue for you too, if not please re-open the issue. **Dev notes:** Solus Linux uses an end-of-life version of OpenSSL (1.0.2) which does not have the built-in thread-safety that OpenSSL >= 1.1.0 provides. When nmail AES decrypt operations fail (in this case due to trying to decrypt an empty file buffer) the global SSL state is affected, which may affect SSL communication with an email server. During troubleshooting an attempt was made to support OpenSSL 1.0.2 by registering necessary thread safety routines, only issue is that libetpan (which nmail uses for email server communication) overrides such registered routines without checking if already set or not. As OpenSSL 1.0.2 is EOL I decided to not to officially support this version, but still implement a workaround for 1.0.2 users which avoids a typical AES decrypt error for empty buffers.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Mar 11, 2020):

I'll need to open a new task on the Solus issue tracker to warn then about this issue, hopefully they will fix it.

Thanks a lot for taking the time to investigate yourself in another distribution, and implementing a workaround. I just pulled from git, I'll see if I still have the random crash issues.

@Kabouik commented on GitHub (Mar 11, 2020): I'll need to open a new task on the Solus issue tracker to warn then about this issue, hopefully they will fix it. Thanks a lot for taking the time to investigate yourself in another distribution, and implementing a workaround. I just pulled from git, I'll see if I still have the random crash issues.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Mar 14, 2020):

I still get crashes with the latest commit on Solus, including with Disroot. That was without doing any action, just nmail setting iddle in the list view. Can you reopen the issue (you told me to do so but I have no rights)?

@Kabouik commented on GitHub (Mar 14, 2020): I still get crashes with the latest commit on Solus, including with Disroot. That was without doing any action, just `nmail` setting iddle in the list view. Can you reopen the issue (you told me to do so but I have no rights)?

kerem commented

Author

Owner

@d99kris commented on GitHub (Mar 14, 2020):

Oh no.. Ok, could you please help share the latest log?

@d99kris commented on GitHub (Mar 14, 2020): Oh no.. Ok, could you please help share the latest log?

kerem commented

Author

Owner

@Kabouik commented on GitHub (Mar 14, 2020):

Wait, I'm stupid. I had two different nmail sessions running concurrently on two different workspaces and got confused: the one that crashed is the Exchange one. I'll wait a bit to see if the other one (Disroot) crashes too.

@Kabouik commented on GitHub (Mar 14, 2020): Wait, I'm stupid. I had two different `nmail` sessions running concurrently on two different workspaces and got confused: the one that crashed is the Exchange one. I'll wait a bit to see if the other one (Disroot) crashes too.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Mar 14, 2020):

I finally got a crash with Disroot as well, I'm sending the logs.

@Kabouik commented on GitHub (Mar 14, 2020): I finally got a crash with Disroot as well, I'm sending the logs.

kerem commented

Author

Owner

@d99kris commented on GitHub (Mar 14, 2020):

Hi @Kabouik - thanks for sharing the log files!

I found one issue with nmail (it shouldn't "crash" on SIGPIPE), so I fixed that and also added some improved crash logging (in case there is still some crash..). It's added in v1.53 / f83ac26

Please let me know if you still see crashes after this fix. Thanks!

@d99kris commented on GitHub (Mar 14, 2020): Hi @Kabouik - thanks for sharing the log files! I found one issue with nmail (it shouldn't "crash" on SIGPIPE), so I fixed that and also added some improved crash logging (in case there is still some crash..). It's added in v1.53 / f83ac26 Please let me know if you still see crashes after this fix. Thanks!

kerem commented

Author

Owner

@Kabouik commented on GitHub (Mar 14, 2020):

Thanks for the quick fix, I'll let you know.

@Kabouik commented on GitHub (Mar 14, 2020): Thanks for the quick fix, I'll let you know.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Mar 17, 2020):

Still getting some crashes with both Exchange and Disroot, but more often with Exchange. I'm sending you the -e logs.

@Kabouik commented on GitHub (Mar 17, 2020): Still getting some crashes with both Exchange and Disroot, but more often with Exchange. I'm sending you the `-e` logs.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Mar 31, 2020):

After one day of use, I didn't get any crash with Disroot, but multiple crashes with Exchange. The crash is silent, no "unexpected termination" errors in the terminal after nmail closes (which was not the case with the crashes reported above). Changing inbox= to a smaller folder in main.conf didn't solve it.

@Kabouik commented on GitHub (Mar 31, 2020): After one day of use, I didn't get any crash with Disroot, but multiple crashes with Exchange. The crash is silent, no "unexpected termination" errors in the terminal after nmail closes (which was not the case with the crashes reported above). Changing `inbox=` to a smaller folder in `main.conf` didn't solve it.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jun 24, 2020):

I have been using nmail for several days consistently, I can confirm crashes are still frequent (< 1 hour) with Exchange, and rare with other servers (perhaps related to network changes and sleep/wake changes when moving places with the laptop).

Not sexy, but useable to mitigate the issue, I use this alias for the Exchange account to relaunch nmail whenever it crashes:

alias nmail-work='while true; do nmail -d ${HOME}/.nmail/nmail-work && break; done' # Work Exchange server

Now I'll need to enable logging and collect some data for you.

@Kabouik commented on GitHub (Jun 24, 2020): I have been using `nmail` for several days consistently, I can confirm crashes are still frequent (< 1 hour) with Exchange, and rare with other servers (perhaps related to network changes and sleep/wake changes when moving places with the laptop). Not sexy, but useable to mitigate the issue, I use this alias for the Exchange account to relaunch `nmail` whenever it crashes: ``` alias nmail-work='while true; do nmail -d ${HOME}/.nmail/nmail-work && break; done' # Work Exchange server ``` Now I'll need to enable logging and collect some data for you.

kerem commented

Author

Owner

@d99kris commented on GitHub (Jun 24, 2020):

Now I'll need to enable logging and collect some data for you.

Okay that would be great. I'll reopen this issue so I don't loose track of it.

I personally have not encountered this type of stability issues, and I generally keep nmail running for weeks/months at a time. I don't generally use Exchange though, but I will try start a long Exchange session and see if I can reproduce any issue.

@d99kris commented on GitHub (Jun 24, 2020): > Now I'll need to enable logging and collect some data for you. Okay that would be great. I'll reopen this issue so I don't loose track of it. I personally have not encountered this type of stability issues, and I generally keep nmail running for weeks/months at a time. I don't generally use Exchange though, but I will try start a long Exchange session and see if I can reproduce any issue.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jun 25, 2020):

I'm sending you some logs. Note that this time, nmail crashed and left the config directory locked:

error: unable to acquire lock for /home/mathieu/.nmail/nmail-inrae/
       only one nmail session per account/confdir is supported.

This does not happen all the time, but I observed that on a regular basis. I haven't found any way to unlock the config directory yet, there's no nmail zombie process to kill, only logging out seemed to work so far. But both issues are probably different, since I often observe crashes that do not cause this.

@Kabouik commented on GitHub (Jun 25, 2020): I'm sending you some logs. Note that this time, `nmail` crashed and left the config directory locked: ``` error: unable to acquire lock for /home/mathieu/.nmail/nmail-inrae/ only one nmail session per account/confdir is supported. ``` This does not happen all the time, but I observed that on a regular basis. I haven't found any way to unlock the config directory yet, there's no `nmail` zombie process to kill, only logging out seemed to work so far. But both issues are probably different, since I often observe crashes that do not cause this.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jun 26, 2020):

Turns out the locked state happens when nmail crashes when using $EDITOR in it because it leaves the process running:

$ lsof ~/.nmail/nmail-work/
COMMAND    PID    USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
kak     109935 kabouik    3r   DIR  259,5     4096 8291264 /home/kabouik/.nmail/nmail-work
$ kill -9 109935

On the one hand, it could be handy because that means there might be ways to recover the $EDITOR buffer after a crash (I lost messages I was composing because of that, didn't investigate if the editor was still running in the background and could be recovered), but on the other hand the locked state is an issue.

I assume this could happen too with external viewers if nmail crashes while viewing attachments or message parts externally, unless it depends on whether the program is running inside the nmail window or not? Perhaps opening a different issue here would be relevant to implement a way for nmail to kill child processes when it crashes (or it may be irrelevant if crashes are fixed anyway).

@Kabouik commented on GitHub (Jun 26, 2020): Turns out the locked state happens when `nmail` crashes when using `$EDITOR` in it because it leaves the process running: ``` $ lsof ~/.nmail/nmail-work/ COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME kak 109935 kabouik 3r DIR 259,5 4096 8291264 /home/kabouik/.nmail/nmail-work $ kill -9 109935 ``` On the one hand, it could be handy because that means there **might** be ways to recover the `$EDITOR` buffer after a crash (I lost messages I was composing because of that, didn't investigate if the editor was still running in the background and could be recovered), but on the other hand the locked state is an issue. I assume this could happen too with external viewers if `nmail` crashes while viewing attachments or message parts externally, unless it depends on whether the program is running inside the `nmail` window or not? Perhaps opening a different issue here would be relevant to implement a way for `nmail` to kill child processes when it crashes (or it may be irrelevant if crashes are fixed anyway).

kerem commented

Author

Owner

@d99kris commented on GitHub (Nov 24, 2020):

Thanks @Kabouik for sharing your extra verbose log-file via email. As mentioned in my email reply it's unfortunately still unclear what is happening, but I have some idea of additional logging I can add to nmail. I will update this issue again once I've implemented the appropriate logging.

@d99kris commented on GitHub (Nov 24, 2020): Thanks @Kabouik for sharing your extra verbose log-file via email. As mentioned in my email reply it's unfortunately still unclear what is happening, but I have some idea of additional logging I can add to `nmail`. I will update this issue again once I've implemented the appropriate logging.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jan 12, 2021):

It may be worth noting that, while I get crashes at seemingly random times when nmail is idle in the message list, message view or compose view, sending an email seems to increase the probability of a crash.

As an example, it is not unusual to send an email with nmail and this Exchange account and get a crash right after hitting Ctrl+x and y, somewhere between the "Sending" and the "Saving" statuses. Sometimes, sending went through and my recipient got the message. Sometimes not. Saving never goes through (crash happens before saving is over).

@Kabouik commented on GitHub (Jan 12, 2021): It may be worth noting that, while I get crashes at seemingly random times when nmail is idle in the message list, message view or compose view, sending an email seems to increase the probability of a crash. As an example, it is not unusual to send an email with nmail and this Exchange account and get a crash right after hitting `Ctrl+x` and `y`, somewhere between the "Sending" and the "Saving" statuses. Sometimes, sending went through and my recipient got the message. Sometimes not. Saving never goes through (crash happens before saving is over).

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 17, 2021):

Hi @Kabouik - I have created a simple script that allows you to run nmail through a debugger (gdb) and capture the backtraces of all threads when crashing. Could you help to try running nmail using it and see if you can reproduce any crash?

The script is located in the source package under ./util/debug-nmail.sh. If you use a custom nmail directory you need to adjust the beginning of the script and set ARGS accordingly, for example:

ARGS="-d ~/.nmail-gm"

Once you have encountered a crash, please review the generated log file and do a quick search either for pass or your actual plain text password, and make sure it is not there, before sharing the file. There is a possibility actual plain text password will be present, mainly if the crash happens during (IMAP or SMTP) authentication.

@d99kris commented on GitHub (Jan 17, 2021): Hi @Kabouik - I have created a simple script that allows you to run `nmail` through a debugger (`gdb`) and capture the backtraces of all threads when crashing. Could you help to try running `nmail` using it and see if you can reproduce any crash? The script is located in the source package under `./util/debug-nmail.sh`. If you use a custom nmail directory you need to adjust the beginning of the script and set `ARGS` accordingly, for example: ARGS="-d ~/.nmail-gm" Once you have encountered a crash, please review the generated log file and do a quick search either for `pass` or your actual plain text password, and make sure it is not there, before sharing the file. There is a possibility actual plain text password will be present, mainly if the crash happens during (IMAP or SMTP) authentication.

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 18, 2021):

FYI - An improvement to Linux crash logging was added in a2b3bb9 so it might not be necessary to reproduce the crash with ./util/debug-nmail.sh (although it would probably simplify debugging). The nmail log file itself should contain most relevant information going forward.

@d99kris commented on GitHub (Jan 18, 2021): FYI - An improvement to Linux crash logging was added in a2b3bb9 so it might not be necessary to reproduce the crash with `./util/debug-nmail.sh` (although it would probably simplify debugging). The nmail log file itself should contain most relevant information going forward.

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 18, 2021):

Do take note though that I simplified the logging options in e51b9a5 so now there's normal logging and verbose logging. The config param verbose_logging now only supports 0 or 1.

@d99kris commented on GitHub (Jan 18, 2021): Do take note though that I simplified the logging options in e51b9a5 so now there's normal logging and verbose logging. The config param `verbose_logging` now only supports `0` or `1`.

kerem commented

Author

Owner

@Kabouik commented on GitHub (Jan 22, 2021):

I just sent you some logs using the new logging feature, only to realize now that you said using the debug script would make it easier to debug. Sorry. I can do that too, let me know.

Note that nmail --help still shows -ee as an option, but it has no effect (just shows the help message again).

@Kabouik commented on GitHub (Jan 22, 2021): I just sent you some logs using the new logging feature, only to realize now that you said using the debug script would make it easier to debug. Sorry. I can do that too, let me know. Note that `nmail --help` still shows `-ee` as an option, but it has no effect (just shows the help message again).

kerem commented

Author

Owner

@d99kris commented on GitHub (Jan 23, 2021):

I just sent you some logs using the new logging feature, only to realize now that you said using the debug script would make it easier to debug. Sorry. I can do that too, let me know.

Thank you! I've looked through the log and unfortunately the new logging feature does not help. Yes, if you could try the ./util/debug-nmail.sh script it would be great.

Note that nmail --help still shows -ee as an option, but it has no effect (just shows the help message again).

Thanks for catching - I've removed it from the --help in b2d9301.

@d99kris commented on GitHub (Jan 23, 2021): > I just sent you some logs using the new logging feature, only to realize now that you said using the debug script would make it easier to debug. Sorry. I can do that too, let me know. Thank you! I've looked through the log and unfortunately the new logging feature does not help. Yes, if you could try the `./util/debug-nmail.sh` script it would be great. > Note that nmail --help still shows -ee as an option, but it has no effect (just shows the help message again). Thanks for catching - I've removed it from the --help in b2d9301.

kerem commented