[GH-ISSUE #3318] Unicode strings with encoding declaration are not supported (some mails are not properly displayed) #1833

Closed
opened 2026-02-27 11:19:22 +03:00 by kerem · 1 comment
Owner

Originally created by @mirtouf on GitHub (Sep 13, 2024).
Original GitHub issue: https://github.com/modoboa/modoboa/issues/3318

Hi,

my setup is

  • OS Type: Debian
  • OS Version: 11
  • Database Type: MySQL
  • Database version: 10.5.23
  • Modoboa: 2.3.2
  • installer used: No
  • Webserver: Nginx

Steps to reproduce

Some mails in the webmail cannot be displayed

Current behavior

Web interface is throwing an error (see #410 for another example)

ValueError at /webmail/getmailcontent Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.Request Method: GET Request URL: https://mail-admin.mirtouf.fr/webmail/getmailcontent?mbox=INBOX&mailid=4216&links=1 Django Version: 4.2.8 Exception Type: ValueError Exception Value: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration. Exception Location: src/lxml/parser.pxi, line 1911, in lxml.etree._parseMemoryDocument Raised during: modoboa_webmail.views.getmailcontent Python Executable: /usr/bin/uwsgi-core Python Version: 3.9.2 Python Path: ['.', '', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/modoboa/env/lib/python3.9/site-packages'] Server time: Fri, 13 Sep 2024 18:50:51 +0000

Traceback
`Environment:

Request Method: GET
Request URL: https://mail-admin.mirtouf.fr/webmail/getmailcontent?mbox=INBOX&mailid=4216&links=1

Django Version: 4.2.8
Python Version: 3.9.2
Installed Applications:
('django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.sites',
'django.contrib.staticfiles',
'reversion',
'ckeditor',
'ckeditor_uploader',
'oauth2_provider',
'corsheaders',
'rest_framework',
'rest_framework.authtoken',
'drf_spectacular',
'phonenumber_field',
'django_otp',
'django_otp.plugins.otp_totp',
'django_otp.plugins.otp_static',
'django_rename_app',
'django_rq',
'modoboa',
'modoboa.core',
'modoboa.lib',
'modoboa.admin',
'modoboa.transport',
'modoboa.relaydomains',
'modoboa.limits',
'modoboa.parameters',
'modoboa.dnstools',
'modoboa.policyd',
'modoboa.maillog',
'modoboa.dmarc',
'modoboa.pdfcredentials',
'modoboa.imap_migration',
'modoboa.postfix_autoreply',
'modoboa.sievefilters',
'modoboa_radicale',
'modoboa_webmail',
'modoboa_contacts',
'webpack_loader')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.locale.LocaleMiddleware',
'x_forwarded_for.middleware.XForwardedForMiddleware',
'corsheaders.middleware.CorsMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'modoboa.core.middleware.LocalConfigMiddleware',
'modoboa.lib.middleware.AjaxLoginRedirect',
'modoboa.lib.middleware.CommonExceptionCatcher',
'modoboa.lib.middleware.RequestCatcherMiddleware')

Traceback (most recent call last):
File "/home/modoboa/env/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/home/modoboa/env/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/modoboa/env/lib/python3.9/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapper_view
return view_func(request, *args, **kwargs)
File "/home/modoboa/env/lib/python3.9/site-packages/modoboa/admin/lib.py", line 46, in wrapped_f
return f(request, *args, **kwargs)
File "/home/modoboa/env/lib/python3.9/site-packages/modoboa_webmail/views.py", line 553, in getmailcontent
"mailbody": email.body if email.body else ""
File "/home/modoboa/env/lib/python3.9/site-packages/modoboa_webmail/lib/imapemail.py", line 168, in body
bodyc = getattr(self, "post_process%s" % self.mformat)(bodyc)
File "/home/modoboa/env/lib/python3.9/site-packages/modoboa/lib/email_utils.py", line 229, in _post_process_html
html = lxml.html.fromstring(content)
File "/home/modoboa/env/lib/python3.9/site-packages/lxml/html/init.py", line 873, in fromstring
doc = document_fromstring(html, parser=parser, base_url=base_url, **kw)
File "/home/modoboa/env/lib/python3.9/site-packages/lxml/html/init.py", line 759, in document_fromstring
value = etree.fromstring(html, parser, **kw)
File "src/lxml/etree.pyx", line 3264, in lxml.etree.fromstring

File "src/lxml/parser.pxi", line 1911, in lxml.etree._parseMemoryDocument

Exception Type: ValueError at /webmail/getmailcontent
Exception Value: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.`

Expected behavior

Mail is displayed properly or at least without such big errors

Video/Screenshot link (optional)

Originally created by @mirtouf on GitHub (Sep 13, 2024). Original GitHub issue: https://github.com/modoboa/modoboa/issues/3318 Hi, my setup is * OS Type: Debian * OS Version: 11 * Database Type: MySQL * Database version: 10.5.23 * Modoboa: 2.3.2 * installer used: No * Webserver: Nginx # Steps to reproduce Some mails in the webmail cannot be displayed # Current behavior Web interface is throwing an error (see #410 for another example) `ValueError at /webmail/getmailcontent Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.Request Method: GET Request URL: https://mail-admin.mirtouf.fr/webmail/getmailcontent?mbox=INBOX&mailid=4216&links=1 Django Version: 4.2.8 Exception Type: ValueError Exception Value: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration. Exception Location: src/lxml/parser.pxi, line 1911, in lxml.etree._parseMemoryDocument Raised during: modoboa_webmail.views.getmailcontent Python Executable: /usr/bin/uwsgi-core Python Version: 3.9.2 Python Path: ['.', '', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/modoboa/env/lib/python3.9/site-packages'] Server time: Fri, 13 Sep 2024 18:50:51 +0000 ` Traceback `Environment: Request Method: GET Request URL: https://mail-admin.mirtouf.fr/webmail/getmailcontent?mbox=INBOX&mailid=4216&links=1 Django Version: 4.2.8 Python Version: 3.9.2 Installed Applications: ('django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.sites', 'django.contrib.staticfiles', 'reversion', 'ckeditor', 'ckeditor_uploader', 'oauth2_provider', 'corsheaders', 'rest_framework', 'rest_framework.authtoken', 'drf_spectacular', 'phonenumber_field', 'django_otp', 'django_otp.plugins.otp_totp', 'django_otp.plugins.otp_static', 'django_rename_app', 'django_rq', 'modoboa', 'modoboa.core', 'modoboa.lib', 'modoboa.admin', 'modoboa.transport', 'modoboa.relaydomains', 'modoboa.limits', 'modoboa.parameters', 'modoboa.dnstools', 'modoboa.policyd', 'modoboa.maillog', 'modoboa.dmarc', 'modoboa.pdfcredentials', 'modoboa.imap_migration', 'modoboa.postfix_autoreply', 'modoboa.sievefilters', 'modoboa_radicale', 'modoboa_webmail', 'modoboa_contacts', 'webpack_loader') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.locale.LocaleMiddleware', 'x_forwarded_for.middleware.XForwardedForMiddleware', 'corsheaders.middleware.CorsMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'modoboa.core.middleware.LocalConfigMiddleware', 'modoboa.lib.middleware.AjaxLoginRedirect', 'modoboa.lib.middleware.CommonExceptionCatcher', 'modoboa.lib.middleware.RequestCatcherMiddleware') Traceback (most recent call last): File "/home/modoboa/env/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner response = get_response(request) File "/home/modoboa/env/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3.9/contextlib.py", line 79, in inner return func(*args, **kwds) File "/home/modoboa/env/lib/python3.9/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapper_view return view_func(request, *args, **kwargs) File "/home/modoboa/env/lib/python3.9/site-packages/modoboa/admin/lib.py", line 46, in wrapped_f return f(request, *args, **kwargs) File "/home/modoboa/env/lib/python3.9/site-packages/modoboa_webmail/views.py", line 553, in getmailcontent "mailbody": email.body if email.body else "" File "/home/modoboa/env/lib/python3.9/site-packages/modoboa_webmail/lib/imapemail.py", line 168, in body bodyc = getattr(self, "_post_process_%s" % self.mformat)(bodyc) File "/home/modoboa/env/lib/python3.9/site-packages/modoboa/lib/email_utils.py", line 229, in _post_process_html html = lxml.html.fromstring(content) File "/home/modoboa/env/lib/python3.9/site-packages/lxml/html/__init__.py", line 873, in fromstring doc = document_fromstring(html, parser=parser, base_url=base_url, **kw) File "/home/modoboa/env/lib/python3.9/site-packages/lxml/html/__init__.py", line 759, in document_fromstring value = etree.fromstring(html, parser, **kw) File "src/lxml/etree.pyx", line 3264, in lxml.etree.fromstring <source code not available> File "src/lxml/parser.pxi", line 1911, in lxml.etree._parseMemoryDocument <source code not available> Exception Type: ValueError at /webmail/getmailcontent Exception Value: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.` # Expected behavior Mail is displayed properly or at least without such big errors # Video/Screenshot link (optional)
kerem closed this issue 2026-02-27 11:19:22 +03:00
Author
Owner

@mirtouf commented on GitHub (Sep 13, 2024):

Closed, will be open in modoboa-webmail.

<!-- gh-comment-id:2349955569 --> @mirtouf commented on GitHub (Sep 13, 2024): Closed, will be open in modoboa-webmail.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/modoboa-modoboa#1833
No description provided.