[GH-ISSUE #10] Formatting issue in subject with UTF-8 #9

Closed
opened 2026-03-03 01:18:51 +03:00 by kerem · 11 comments
Owner

Originally created by @Kabouik on GitHub (Jan 13, 2020).
Original GitHub issue: https://github.com/d99kris/nmail/issues/10

An email with subject "Test nmail" send to a friend (Gmail in that case) arrived with "=?UTF-8?Q?Test nmail?=" as subject. The body text was alright.

Ubuntu 18.04 chrooted from SailfishOS, smtp: disroot.org (no such UTF-8 issue usually), nmail just installed from git.

Originally created by @Kabouik on GitHub (Jan 13, 2020). Original GitHub issue: https://github.com/d99kris/nmail/issues/10 An email with subject "Test nmail" send to a friend (Gmail in that case) arrived with "=?UTF-8?Q?Test nmail?=" as subject. The body text was alright. Ubuntu 18.04 chrooted from SailfishOS, smtp: disroot.org (no such UTF-8 issue usually), nmail just installed from git.
kerem closed this issue 2026-03-03 01:18:52 +03:00
Author
Owner

@d99kris commented on GitHub (Jan 13, 2020):

Do you know in which email reader the string "=?UTF-8?Q?Test nmail?=" appeared? Gmail web interface?

At a low level (for sending) nmail encodes every email subject in UTF-8. It's not really necessary for plain-ASCII subjects, but I chose to encode all to keep the code simple. Generally the receiving email client would decode it before displaying - I've tested it in a few popular ones (Gmail web, Outlook web, macOS Mail, etc) and haven't seen any failing to decode it, so it would be interesting to know which client fails. If you could send a simple test email to me (d99kris at gmail dot com) from disroot it would be great, otherwise I can try sign up for disroot to test. Just in case it's something with the SMTP service.

In any case - one could argue nmail shouldn't do UTF-8 encoding unless needed, so I'll look into fixing that.

<!-- gh-comment-id:573669881 --> @d99kris commented on GitHub (Jan 13, 2020): Do you know in which email reader the string `"=?UTF-8?Q?Test nmail?="` appeared? Gmail web interface? At a low level (for sending) nmail encodes every email subject in UTF-8. It's not really necessary for plain-ASCII subjects, but I chose to encode all to keep the code simple. Generally the receiving email client would decode it before displaying - I've tested it in a few popular ones (Gmail web, Outlook web, macOS Mail, etc) and haven't seen any failing to decode it, so it would be interesting to know which client fails. If you could send a simple test email to me (d99kris at gmail dot com) from disroot it would be great, otherwise I can try sign up for disroot to test. Just in case it's something with the SMTP service. In any case - one could argue `nmail` shouldn't do UTF-8 encoding unless needed, so I'll look into fixing that.
Author
Owner

@Kabouik commented on GitHub (Jan 13, 2020):

Good point, I wanted to check on the webmail but actually postponed and, since it was late, just ended up forgetting. The wrong formatting appeared in SFOS e-mail application, but when checking Gmail's webmail, no issue. I guess this is something with the SFOS application then, it might be a corner case. Note however that, usually, e-mails sent to Gmail and checked in the SFOS application do not result in this formatting issue.

I will be trying with other e-mail accounts including yours soon (maybe later today) and see if the same issue appears when sending something with nmail.

<!-- gh-comment-id:573671712 --> @Kabouik commented on GitHub (Jan 13, 2020): Good point, I wanted to check on the webmail but actually postponed and, since it was late, just ended up forgetting. The wrong formatting appeared in SFOS e-mail application, but when checking Gmail's webmail, no issue. I guess this is something with the SFOS application then, it might be a corner case. Note however that, usually, e-mails sent to Gmail and checked in the SFOS application do not result in this formatting issue. I will be trying with other e-mail accounts including yours soon (maybe later today) and see if the same issue appears when sending something with nmail.
Author
Owner

@d99kris commented on GitHub (Jan 13, 2020):

Cool thanks. Yeah, I think I can change nmail to not do the UTF-8 for plain ASCII (english) subjects.

<!-- gh-comment-id:573697237 --> @d99kris commented on GitHub (Jan 13, 2020): Cool thanks. Yeah, I think I can change nmail to not do the UTF-8 for plain ASCII (english) subjects.
Author
Owner

@Kabouik commented on GitHub (Jan 14, 2020):

Hold on, I tested again with the disroot.org (Rainloop) and Gmail webmails, and no issue this time. Not sure what caused it the first time.

Side question: do you have an idea why backspace works on one device but not the other? No issue on my phone with hardware keyboard and Ubuntu chroot, but it just prints "^?" on my real Linux computer running Solus. What is the built-in editor? Also, I can send emails from the phone but not the computer. I guess I might just be missing a couple dependencies on the computer.

<!-- gh-comment-id:574333306 --> @Kabouik commented on GitHub (Jan 14, 2020): Hold on, I tested again with the disroot.org (Rainloop) and Gmail webmails, and no issue this time. Not sure what caused it the first time. Side question: do you have an idea why `backspace` works on one device but not the other? No issue on my phone with hardware keyboard and Ubuntu chroot, but it just prints "^?" on my real Linux computer running Solus. What is the built-in editor? Also, I can send emails from the phone but not the computer. I guess I might just be missing a couple dependencies on the computer.
Author
Owner

@d99kris commented on GitHub (Jan 16, 2020):

Ok let me take a look at backspace on Linux. I mainly test/use nmail on Linux over SSH, and perhaps there's a different backspace mapping if using a native Linux terminal.

The built-in editor is a fairly simple text editor I implemented just for nmail. It may still have some bugs.

<!-- gh-comment-id:575088038 --> @d99kris commented on GitHub (Jan 16, 2020): Ok let me take a look at backspace on Linux. I mainly test/use `nmail` on Linux over SSH, and perhaps there's a different backspace mapping if using a native Linux terminal. The built-in editor is a fairly simple text editor I implemented just for `nmail`. It may still have some bugs.
Author
Owner

@Kabouik commented on GitHub (Jan 16, 2020):

Note that I do not encounter it in chrooted Xubuntu from my phone. However, chroot and ssh are likely fairly similar, and I realize now that my nmail instance built natively on SFOS also shows this backspace issue.

So to sum up: no issue in chrooted Xubuntu from the phone, but issue occurring on desktop computer running Solus, as well as on phone running SFOS. I do not have ways to test directly in a native Ubuntu terminal unfortunately.

<!-- gh-comment-id:575089956 --> @Kabouik commented on GitHub (Jan 16, 2020): Note that I do not encounter it in chrooted Xubuntu from my phone. However, chroot and ssh are likely fairly similar, and I realize now that my nmail instance built natively on SFOS also shows this backspace issue. So to sum up: no issue in chrooted Xubuntu from the phone, but issue occurring on desktop computer running Solus, as well as on phone running SFOS. I do not have ways to test directly in a native Ubuntu terminal unfortunately.
Author
Owner

@d99kris commented on GitHub (Jan 16, 2020):

I tried Ubuntu 18.04 desktop install now, and did not see the issue. This was in a VM though. Will see if I can try Solus - btw, do you use the Budgie, GNOME or MATE version? (sorry if you already posted this elsewhere)

<!-- gh-comment-id:575099032 --> @d99kris commented on GitHub (Jan 16, 2020): I tried Ubuntu 18.04 desktop install now, and did not see the issue. This was in a VM though. Will see if I can try Solus - btw, do you use the Budgie, GNOME or MATE version? (sorry if you already posted this elsewhere)
Author
Owner

@Kabouik commented on GitHub (Jan 16, 2020):

I use Solus Budgie, but I don't think I got the subject formatting issue when using nmail from Solus (actually, I did not send any mail from Solus, I must be missing a dependency like sendmail or similar).

I got it only from Xubuntu chrooted from SFOS, when sending from disroot.org to Gmail. However I tried again later the same parameters and used similar characters in the subject, and did not observe it anymore, so I am confused.

<!-- gh-comment-id:575100348 --> @Kabouik commented on GitHub (Jan 16, 2020): I use Solus Budgie, but I don't think I got the subject formatting issue when using nmail from Solus (actually, I did not send any mail from Solus, I must be missing a dependency like `sendmail` or similar). I got it only from Xubuntu chrooted from SFOS, when sending from disroot.org to Gmail. However I tried again later the same parameters and used similar characters in the subject, and did not observe it anymore, so I am confused.
Author
Owner

@d99kris commented on GitHub (Jan 16, 2020):

Thanks! Ok, sorry I meant the backspace issue - you saw it in Solus, right?

Regarding the UTF-8 subject, I don't need a system to reproduce it, I think there's a simple way it can be avoided (for plain ASCII text at least), so I'll look at fixing that as well.

<!-- gh-comment-id:575102021 --> @d99kris commented on GitHub (Jan 16, 2020): Thanks! Ok, sorry I meant the backspace issue - you saw it in Solus, right? Regarding the UTF-8 subject, I don't need a system to reproduce it, I think there's a simple way it can be avoided (for plain ASCII text at least), so I'll look at fixing that as well.
Author
Owner

@Kabouik commented on GitHub (Jan 16, 2020):

Yes, sorry, it's my fault actually: I opened an issue about UTF-8 and then mentioned another issue, which I forgot about when I replied to what I thought was just the formatting issue!

I observed the backspace issue in both Solus native terminal emulator (using Tilix) and SFOS native terminal (Fingerterm), not in Xubuntu chrooted from SFOS (Tilix within XFCE4). If I chroot the same Xubuntu directly from the SFOS terminal (Fingerterm) without launching XFCE4 and opening a terminal there, I don't see the issue either, meaning it is not specific to either Tilix or Fingerterm, but perhaps relates to whether there is chroot/ssh being used or not.

<!-- gh-comment-id:575130294 --> @Kabouik commented on GitHub (Jan 16, 2020): Yes, sorry, it's my fault actually: I opened an issue about UTF-8 and then mentioned another issue, which I forgot about when I replied to what I thought was just the formatting issue! I observed the backspace issue in both Solus native terminal emulator (using Tilix) and SFOS native terminal (Fingerterm), not in Xubuntu chrooted from SFOS (Tilix within XFCE4). If I chroot the same Xubuntu directly from the SFOS terminal (Fingerterm) without launching XFCE4 and opening a terminal there, I don't see the issue either, meaning it is not specific to either Tilix or Fingerterm, but perhaps relates to whether there is chroot/ssh being used or not.
Author
Owner

@d99kris commented on GitHub (Jan 17, 2020):

I reported a separate issue #16 to track the backspace issue.

<!-- gh-comment-id:575636717 --> @d99kris commented on GitHub (Jan 17, 2020): I reported a separate issue #16 to track the backspace issue.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/nmail#9
No description provided.