[GH-ISSUE #2] Accented chararacters converted to '?' after DKIM Transport Agent is enabled #3

Closed
opened 2026-02-26 10:35:10 +03:00 by kerem · 10 comments
Owner

Originally created by @RoFz on GitHub (Apr 10, 2013).
Original GitHub issue: https://github.com/Pro/dkim-exchange/issues/2

Hello,

Firstly i would like to tell that this is a great initiative and a very promising project, so thank you for bringing DKIM to Exchange environments. And for FREE.

We may have found a problem with the DKIM module that is related to messages containing accented characters when they are received by SMTP through an internal Receive Connector. DKIM works flawlessly when sending messages from Outlook and OWA, even with accented characters. The problem is related only to internal relaying via a Receive Connector and only with HTML code.

Accented chars, such as áéíóú are getting translated to ????? (question marks).

And the funniest thing: even accented characters written in HTML code, such as &aacute ; &eacute ; and so on are also being translated to question marks. (!)

This problem happens only when DKIM dll is activated as a Transport Agent.

Is there anything i could adapt on my e-mail system to make accented characters work with Receive Connectors and DKIM dll?

Any clues?

Originally created by @RoFz on GitHub (Apr 10, 2013). Original GitHub issue: https://github.com/Pro/dkim-exchange/issues/2 Hello, Firstly i would like to tell that this is a great initiative and a very promising project, so thank you for bringing DKIM to Exchange environments. And for FREE. We may have found a problem with the DKIM module that is related to messages containing accented characters when they are received by SMTP through an internal Receive Connector. DKIM works flawlessly when sending messages from Outlook and OWA, even with accented characters. The problem is related only to internal relaying via a Receive Connector and only with HTML code. Accented chars, such as áéíóú are getting translated to ????? (question marks). And the funniest thing: even accented characters written in HTML code, such as &aacute ; &eacute ; and so on are also being translated to question marks. (!) This problem happens only when DKIM dll is activated as a Transport Agent. Is there anything i could adapt on my e-mail system to make accented characters work with Receive Connectors and DKIM dll? Any clues?
kerem 2026-02-26 10:35:10 +03:00
  • closed this issue
  • added the
    bug
    label
Author
Owner

@Pro commented on GitHub (Apr 12, 2013):

Hi!
Sorry for the problems...
We don't use interal Receive Connectors, therefore I can't test it.
Maybe I find some time to setup a test environment where I can test it...

Or hopefully there is someone other who can test it.

This looks like a unicode problem somewhere!

<!-- gh-comment-id:16288441 --> @Pro commented on GitHub (Apr 12, 2013): Hi! Sorry for the problems... We don't use interal Receive Connectors, therefore I can't test it. Maybe I find some time to setup a test environment where I can test it... Or hopefully there is someone other who can test it. This looks like a unicode problem somewhere!
Author
Owner

@Pro commented on GitHub (Apr 12, 2013):

Appear those broken chars only in message body or also in the message subject?
Can you please test this and tell me, which message field are affected by this bug?

<!-- gh-comment-id:16289320 --> @Pro commented on GitHub (Apr 12, 2013): Appear those broken chars only in message body or also in the message subject? Can you please test this and tell me, which message field are affected by this bug?
Author
Owner

@RoFz commented on GitHub (Apr 12, 2013):

Hello Pro. Thanks for your response.

I've been testing a bit further to get a better understanding of what is really happening. Looks like it's not a straightforward issue, although 100% reproducible, it happens on only a few specific situations. Down below i'm listing a few of those situations envolving messages received with Outlook and GMail:

Situation #1:

  • DKIM transport agent enabled on Exchange 2010
  • From internal relay (using Receive Connector)
  • Received on Outlook 2010
  • Subject: wrong (question marks)
  • Body: wrong (question marks)
  • Message Headers (sensitive info, such as domain name, supressed):

DKIM-Signature: v=1; a=rsa-sha1; s=1365559627.XXXXXX; d=XXXXXX.com; c=simple/simple; q=dns/txt; h=Date : From : Message-ID : Subject : To; bh=k6wibIHMz9CHxRoGOaHh6G+4znE=; b=HcLXr3gwj+NQ57mhAUUZXIlhBk4zv7q5LIa8boNr9ess7Lbr6AbUQ3wOD12XujbWpP4eaw6uY4bQaRww/5JLfJB8J15sEnQrX0bLCGyIUpqhs8wKM/4CjAN47PphIkwwRdmMcJs7gdkWmrndpdQC3Sc0ndg+FZMDh8Sz3qFiQjU=;
Date: Fri, 12 Apr 2013 13:10:18 -0300
Subject: ?????
X-PHP-Originating-Script: 0:testmail.php
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Situation #2:

  • DKIM transport disabled on Exchange 2010
  • From internal relay (using Receive Connector)
  • Received on Outlook 2010
  • Subject: correct (áéíóú)
  • Body: correct (áéíóú)
  • Message Headers (sensitive info, such as domain name, supressed):

Date: Fri, 12 Apr 2013 13:17:39 -0300
Subject: ??????????
X-PHP-Originating-Script: 0:testmail.php
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Situation #3:

  • DKIM transport disabled on Exchange 2010
  • From internal relay (using Receive Connector)
  • Received on GMail Web Interface
  • Subject: correct (áéíóú)
  • Body: correct (áéíóú)
  • Message Headers (sensitive info, such as domain name, supressed):

Received-SPF: pass (google.com: domain of support@XXXXXX.com designates XXX.XXX.XXX.XXX as permitted sender) client-ip=XXX.XXX.XXX.XXX;
Authentication-Results: mx.google.com;
spf=pass (google.com: domain of support@XXXXXX.com designates XXX.XXX.XXX.XXX as permitted sender) smtp.mail=support@XXXXXX.com;
dmarc=pass (p=REJECT dis=none) d=XXXXXX.com
Date: Fri, 12 Apr 2013 13:20:51 -0300
Message-ID: 201304121620.r3CGKpS6018756@srv02.XXXXXX.local
To: XXXXXX@gmail.com
Subject: áéíóú
X-PHP-Originating-Script: 0:testmail.php
From: support@XXXXXX.com
Reply-To: support@XXXXXX.com
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Return-Path: support@XXXXXX.com

< html>
< head>
< meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8">
< /head>
< body>
Direct: =C3=A1=C3=A9=C3=AD=C3=B3=C3=BA

HTML: =C3=A1=C3=A9=C3=AD=C3=B3=C3=BA

< br>
...

Situation #4:

  • DKIM transport enabled on Exchange 2010
  • From internal relay (using Receive Connector)
  • Received on GMail Web Interface
  • Subject: wrong (question marks)
  • Body: correct (áéíóú)
  • Message Headers (sensitive info, such as domain name, supressed):

Received-SPF: pass (google.com: domain of support@XXXXXX.com designates XXX.XXX.XXX.XXX as permitted sender) client-ip=XXX.XXX.XXX.XXX;
Authentication-Results: mx.google.com;
spf=pass (google.com: domain of support@XXXXXX.com designates XXX.XXX.XXX.XXX as permitted sender) smtp.mail=support@XXXXXX.com;
dkim=fail header.i=@XXXXXX.com;
dmarc=pass (p=REJECT dis=none) d=XXXXXX.com
DKIM-Signature: v=1; a=rsa-sha1; s=1365559627.XXXXXX; d=XXXXXX.com; c=simple/simple; q=dns/txt; h=Date : From : Message-ID : Subject : To; bh=5RZWyJqarrW+sjRKoe1nC0ifLOs=; b=cIIHjvUO2ZER8+NGhATkLjyGL1ZiO/kKvgbyKbQAIeDutDVuxVA/GqYFKimotDzh9DdRD4wrOL+LM3bCR2q9C9mpBRCf3nLU6L8/DtMOfUqLti/YI7WwTocW65l4tlq8/PfkI2j0708F3ehJUeHfgT51R9zeMGTe4qaMg4FhB+8=;
Date: Fri, 12 Apr 2013 13:21:34 -0300
Message-ID: 201304121621.r3CGLYNF018763@srv02.XXXXXX.local
To: XXXXXX@gmail.com
Subject: ?????
X-PHP-Originating-Script: 0:testmail.php
From: support@XXXXXX.com
Reply-To: support@XXXXXX.com
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Return-Path: support@XXXXXX.com

< html>
< head>
< meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8">
< /head>
< body>
Direct: =C3=A1=C3=A9=C3=AD=C3=B3=C3=BA

HTML: =C3=A1=C3=A9=C3=AD=C3=B3=C3=BA

< br>

I'm still researching on how to dump a full mime source for a message received on a MAPI account on Exchange. Looks like it's possible with POP3 or IMAP accounts, but not with 'internal' Exchange messages. I guess a full source dump of those message would provide greater insight on what is really happening.

Meanwhile, please let me know how could i provide more info on the issue.

<!-- gh-comment-id:16303232 --> @RoFz commented on GitHub (Apr 12, 2013): Hello Pro. Thanks for your response. I've been testing a bit further to get a better understanding of what is really happening. Looks like it's not a straightforward issue, although 100% reproducible, it happens on only a few specific situations. Down below i'm listing a few of those situations envolving messages received with Outlook and GMail: Situation #1: - DKIM transport agent enabled on Exchange 2010 - From internal relay (using Receive Connector) - Received on Outlook 2010 - Subject: wrong (question marks) - Body: wrong (question marks) - Message Headers (sensitive info, such as domain name, supressed): DKIM-Signature: v=1; a=rsa-sha1; s=1365559627.XXXXXX; d=XXXXXX.com; c=simple/simple; q=dns/txt; h=Date : From : Message-ID : Subject : To; bh=k6wibIHMz9CHxRoGOaHh6G+4znE=; b=HcLXr3gwj+NQ57mhAUUZXIlhBk4zv7q5LIa8boNr9ess7Lbr6AbUQ3wOD12XujbWpP4eaw6uY4bQaRww/5JLfJB8J15sEnQrX0bLCGyIUpqhs8wKM/4CjAN47PphIkwwRdmMcJs7gdkWmrndpdQC3Sc0ndg+FZMDh8Sz3qFiQjU=; Date: Fri, 12 Apr 2013 13:10:18 -0300 Subject: ????? X-PHP-Originating-Script: 0:testmail.php Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Situation #2: - DKIM transport disabled on Exchange 2010 - From internal relay (using Receive Connector) - Received on Outlook 2010 - Subject: correct (áéíóú) - Body: correct (áéíóú) - Message Headers (sensitive info, such as domain name, supressed): Date: Fri, 12 Apr 2013 13:17:39 -0300 Subject: ?????????? X-PHP-Originating-Script: 0:testmail.php Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Situation #3: - DKIM transport disabled on Exchange 2010 - From internal relay (using Receive Connector) - Received on GMail Web Interface - Subject: correct (áéíóú) - Body: correct (áéíóú) - Message Headers (sensitive info, such as domain name, supressed): Received-SPF: pass (google.com: domain of support@XXXXXX.com designates XXX.XXX.XXX.XXX as permitted sender) client-ip=XXX.XXX.XXX.XXX; Authentication-Results: mx.google.com; spf=pass (google.com: domain of support@XXXXXX.com designates XXX.XXX.XXX.XXX as permitted sender) smtp.mail=support@XXXXXX.com; dmarc=pass (p=REJECT dis=none) d=XXXXXX.com Date: Fri, 12 Apr 2013 13:20:51 -0300 Message-ID: 201304121620.r3CGKpS6018756@srv02.XXXXXX.local To: XXXXXX@gmail.com Subject: áéíóú X-PHP-Originating-Script: 0:testmail.php From: support@XXXXXX.com Reply-To: support@XXXXXX.com Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Return-Path: support@XXXXXX.com < html> < head> < meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8"> < /head> < body> Direct: =C3=A1=C3=A9=C3=AD=C3=B3=C3=BA <br> HTML: =C3=A1=C3=A9=C3=AD=C3=B3=C3=BA <br> < br> ... Situation #4: - DKIM transport enabled on Exchange 2010 - From internal relay (using Receive Connector) - Received on GMail Web Interface - Subject: wrong (question marks) - Body: correct (áéíóú) - Message Headers (sensitive info, such as domain name, supressed): Received-SPF: pass (google.com: domain of support@XXXXXX.com designates XXX.XXX.XXX.XXX as permitted sender) client-ip=XXX.XXX.XXX.XXX; Authentication-Results: mx.google.com; spf=pass (google.com: domain of support@XXXXXX.com designates XXX.XXX.XXX.XXX as permitted sender) smtp.mail=support@XXXXXX.com; dkim=fail header.i=@XXXXXX.com; dmarc=pass (p=REJECT dis=none) d=XXXXXX.com DKIM-Signature: v=1; a=rsa-sha1; s=1365559627.XXXXXX; d=XXXXXX.com; c=simple/simple; q=dns/txt; h=Date : From : Message-ID : Subject : To; bh=5RZWyJqarrW+sjRKoe1nC0ifLOs=; b=cIIHjvUO2ZER8+NGhATkLjyGL1ZiO/kKvgbyKbQAIeDutDVuxVA/GqYFKimotDzh9DdRD4wrOL+LM3bCR2q9C9mpBRCf3nLU6L8/DtMOfUqLti/YI7WwTocW65l4tlq8/PfkI2j0708F3ehJUeHfgT51R9zeMGTe4qaMg4FhB+8=; Date: Fri, 12 Apr 2013 13:21:34 -0300 Message-ID: 201304121621.r3CGLYNF018763@srv02.XXXXXX.local To: XXXXXX@gmail.com Subject: ????? X-PHP-Originating-Script: 0:testmail.php From: support@XXXXXX.com Reply-To: support@XXXXXX.com Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Return-Path: support@XXXXXX.com < html> < head> < meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8"> < /head> < body> Direct: =C3=A1=C3=A9=C3=AD=C3=B3=C3=BA <br> HTML: =C3=A1=C3=A9=C3=AD=C3=B3=C3=BA <br> < br> I'm still researching on how to dump a full mime source for a message received on a MAPI account on Exchange. Looks like it's possible with POP3 or IMAP accounts, but not with 'internal' Exchange messages. I guess a full source dump of those message would provide greater insight on what is really happening. Meanwhile, please let me know how could i provide more info on the issue.
Author
Owner

@RoFz commented on GitHub (Apr 12, 2013):

I would like to change my initial statement that the problem affects only HTML e-mail. That is not true.

After testing a bit further, using linux 'mail' command (Heirloom mailx 12.4 7/29/08) and a text-only message with accented characters, i can confirm that is also affects messages with Content-Type = text/plain and not only html.

So the issue happens with both HTML and text messages.

The subject was not affected during these tests. It stayed correctly encoded as:
Subject: =?iso-8859-1?B?4ent8/o=?=

<!-- gh-comment-id:16305706 --> @RoFz commented on GitHub (Apr 12, 2013): I would like to change my initial statement that the problem affects only HTML e-mail. That is not true. After testing a bit further, using linux 'mail' command (Heirloom mailx 12.4 7/29/08) and a text-only message with accented characters, i can confirm that is also affects messages with Content-Type = text/plain and not only html. So the issue happens with both HTML and text messages. The subject was not affected during these tests. It stayed correctly encoded as: Subject: =?iso-8859-1?B?4ent8/o=?=
Author
Owner

@ashum81 commented on GitHub (Jan 10, 2014):

What about this issue? Is it resolved? I have it too.

<!-- gh-comment-id:32012258 --> @ashum81 commented on GitHub (Jan 10, 2014): What about this issue? Is it resolved? I have it too.
Author
Owner

@Pro commented on GitHub (Jan 10, 2014):

@ashum81 No, unfortunately it isn't solved yet and currently I don't have much spare time. Hopefully I'll get to it in the next days/weeks.

<!-- gh-comment-id:32033112 --> @Pro commented on GitHub (Jan 10, 2014): @ashum81 No, unfortunately it isn't solved yet and currently I don't have much spare time. Hopefully I'll get to it in the next days/weeks.
Author
Owner

@Pro commented on GitHub (Jan 17, 2014):

@ashum81 and @RoFz I tried to reproduce the error but no success.

My setup:

  • Virtual Box VM with Windows Server 2008 R2
  • Exchange 2010 SP3
  • Exchange DKIM dll Version 1.5.1.0 (current version from repo, check it by right click -> details)
  • Exchange 2010 default receive connector set to accept anonymous logins

My email contained the following data:
Subject: test áéíóú ←↑→↓
Body: Hello! test áéíóú ←↑→↓ END

Then I tried three different methods for sending mails through this server:

  1. Using C# code directly connecting to the SMTP service and sending the mail
  2. Using Thunderbird with IMAP+SMTP connecting to the server
  3. Using Outlook with IMAP+SMTP connecting to the server

I did sent the mail to my gmail account and also to my private address which is received in Outlook.
In none of the above cases I ran into this bug. The subject and body was always correct.

Which Exchange version are you using exactly? Which Windows Server version?

<!-- gh-comment-id:32615560 --> @Pro commented on GitHub (Jan 17, 2014): @ashum81 and @RoFz I tried to reproduce the error but no success. My setup: - Virtual Box VM with Windows Server 2008 R2 - Exchange 2010 SP3 - Exchange DKIM dll Version 1.5.1.0 (current version from repo, check it by right click -> details) - Exchange 2010 default receive connector set to accept anonymous logins My email contained the following data: Subject: test áéíóú ←↑→↓ Body: Hello! test áéíóú ←↑→↓ END Then I tried three different methods for sending mails through this server: 1. Using C# code directly connecting to the SMTP service and sending the mail 2. Using Thunderbird with IMAP+SMTP connecting to the server 3. Using Outlook with IMAP+SMTP connecting to the server I did sent the mail to my gmail account and also to my private address which is received in Outlook. In none of the above cases I ran into this bug. The subject and body was always correct. Which Exchange version are you using exactly? Which Windows Server version?
Author
Owner

@ashum81 commented on GitHub (Jan 18, 2014):

My setup:

  • Windows Server 2008 R2 SP1
  • Exchange 2010 SP2 RU8

I sent a letter from outside server to exchange with From: header "mail@mydomain.com" and with body in Russian: "Новости".
Exchange is configured to support domain mydomain.com.
DKIM is configured to sign letters for domain mydomain.com.
I received in Outlook a letter with ??? in body.

<!-- gh-comment-id:32678506 --> @ashum81 commented on GitHub (Jan 18, 2014): My setup: - Windows Server 2008 R2 SP1 - Exchange 2010 SP2 RU8 I sent a letter from outside server to exchange with From: header "mail@mydomain.com" and with body in Russian: "Новости". Exchange is configured to support domain mydomain.com. DKIM is configured to sign letters for domain mydomain.com. I received in Outlook a letter with ??? in body.
Author
Owner

@Pro commented on GitHub (Jan 18, 2014):

I think I found the problem in the code. This should be fixed now.
@RoFz & @ashum81 can you please try the new version (1.5.2).

<!-- gh-comment-id:32679621 --> @Pro commented on GitHub (Jan 18, 2014): I think I found the problem in the code. This should be fixed now. @RoFz & @ashum81 can you please try the new version (1.5.2).
Author
Owner

@ashum81 commented on GitHub (Jan 18, 2014):

It is ok now. Thank you.

<!-- gh-comment-id:32685130 --> @ashum81 commented on GitHub (Jan 18, 2014): It is ok now. Thank you.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/dkim-exchange-Pro#3
No description provided.