starred/modoboa-modoboa
1
0
Fork 0
mirror of https://github.com/modoboa/modoboa.git synced 2026-04-26 01:16:01 +03:00
Code Issues 60 Projects Releases 10 Packages Wiki Activity

[GH-ISSUE #1430] CSV import broken #1133

New issue
Closed
opened 2026-02-27 11:15:28 +03:00 by kerem · 7 comments
kerem commented 2026-02-27 11:15:28 +03:00
Owner
Copy link

Originally created by @almereyda on GitHub (Mar 8, 2018).
Original GitHub issue: https://github.com/modoboa/modoboa/issues/1430

Impacted versions

  • Modoboa: 10.10.2
  • installer used: Yes
  • Webserver: Nginx

Steps to reproduce

Run an email import from a CSV formed after http://modoboa.readthedocs.io/en/latest/moving.html#using-csv-files, i.e. one that worked before.

Current behavior

  • In the web frontend, the spinner will just rotate indifinitely.
  • On the command line, an error is thrown:
(env)email# python /srv/modoboa/instance/manage.py modo import accounts.csv
/srv/modoboa/env/local/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
Traceback (most recent call last):                                              |ETA:  --:--:--
  File "/srv/modoboa/instance/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/subcommand/base.py", line 53, in run_from_argv
    return super(SubcommandCommand, self).run_from_argv(argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/subcommand/base.py", line 84, in handle
    return command.run_from_argv(argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa/admin/management/commands/subcommands/_import.py", line 85, in handle
    self._import(filename, options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa/admin/management/commands/subcommands/_import.py", line 57, in _import
    reader = csv.reader(f, delimiter=options["sepchar"])
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/backports/csv.py", line 232, in __init__
    raise TypeError('Invalid dialect parameters')
TypeError: Invalid dialect parameters

It feels #1342 was a bit of a too quick shot at #1312 and may need further iterations to constrain the import a little more.

  • SMTPUTF8 for mailbox names is only supported with Postfix 3
  • Domain names should be converted to Punycode

Manually passing dialect='excel' by editing _import.py did not help, either.

github.com/modoboa/modoboa@83d0564c27/modoboa/admin/management/commands/subcommands/_import.py (L57)

The original error may reside around https://github.com/ryanhiebert/backports.csv/blob/master/src/backports/csv.py#L230
Unfortunately the upstream package does not provide a test suite.

Expected behavior

The import completes successfully.

Originally created by @almereyda on GitHub (Mar 8, 2018). Original GitHub issue: https://github.com/modoboa/modoboa/issues/1430 # Impacted versions * Modoboa: 10.10.2 * installer used: Yes * Webserver: Nginx # Steps to reproduce Run an email import from a CSV formed after http://modoboa.readthedocs.io/en/latest/moving.html#using-csv-files, i.e. one that worked before. # Current behavior - In the web frontend, the spinner will just rotate indifinitely. - On the command line, an error is thrown: ``` (env)email# python /srv/modoboa/instance/manage.py modo import accounts.csv /srv/modoboa/env/local/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>. """) Traceback (most recent call last): |ETA: --:--:-- File "/srv/modoboa/instance/manage.py", line 22, in <module> execute_from_command_line(sys.argv) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line utility.execute() File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/srv/modoboa/env/local/lib/python2.7/site-packages/subcommand/base.py", line 53, in run_from_argv return super(SubcommandCommand, self).run_from_argv(argv) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv self.execute(*args, **cmd_options) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute output = self.handle(*args, **options) File "/srv/modoboa/env/local/lib/python2.7/site-packages/subcommand/base.py", line 84, in handle return command.run_from_argv(argv) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv self.execute(*args, **cmd_options) File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute output = self.handle(*args, **options) File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa/admin/management/commands/subcommands/_import.py", line 85, in handle self._import(filename, options) File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa/admin/management/commands/subcommands/_import.py", line 57, in _import reader = csv.reader(f, delimiter=options["sepchar"]) File "/srv/modoboa/env/local/lib/python2.7/site-packages/backports/csv.py", line 232, in __init__ raise TypeError('Invalid dialect parameters') TypeError: Invalid dialect parameters ``` It feels #1342 was a bit of a too quick shot at #1312 and may need further iterations to constrain the import a little more. - SMTPUTF8 for mailbox names is only supported with Postfix 3 - Domain names should be converted to Punycode Manually passing `dialect='excel'` by editing `_import.py` did not help, either. https://github.com/modoboa/modoboa/blob/83d0564c27fa1e016f011b50f94a07278532cd6c/modoboa/admin/management/commands/subcommands/_import.py#L57 The original error may reside around https://github.com/ryanhiebert/backports.csv/blob/master/src/backports/csv.py#L230 Unfortunately the upstream package does not provide a test suite. # Expected behavior The import completes successfully.
kerem 2026-02-27 11:15:28 +03:00
kerem commented 2026-02-27 11:15:28 +03:00
Author
Owner
Copy link

@ghost commented on GitHub (Mar 9, 2018):

Well Line 232 of csv.py destroys any useful error message and replaces it with Invalid dialect parameters.

I don't think this is a utf8 problem, I think options["sepchar"] is either None or an instance of str instead of unicode, I'll need to do some testing in the morning.

Can you try running

python /srv/modoboa/instance/manage.py modo import --sepchar \; accounts.csv
<!-- gh-comment-id:371671398 --> @ghost commented on GitHub (Mar 9, 2018): Well Line 232 of csv.py destroys any useful error message and replaces it with `Invalid dialect parameters`. I don't think this is a utf8 problem, I think `options["sepchar"]` is either None or an instance of `str` instead of `unicode`, I'll need to do some testing in the morning. Can you try running ``` python /srv/modoboa/instance/manage.py modo import --sepchar \; accounts.csv ```
kerem commented 2026-02-27 11:15:28 +03:00
Author
Owner
Copy link

@almereyda commented on GitHub (Mar 9, 2018):

Thanks for the introspection.

The escaped SEPCHAR yields the very same error.

On 9 March 2018 at 01:22, Andrew Fyfe notifications@github.com wrote:

Well Line 232 of csv.py destroys any useful error message and replaces it
with Invalid dialect parameters.

I don't think this is a utf8 problem, I think options["sepchar"] is
either None or an instance of str instead of unicode, I'll need to do
some testing in the morning.

Can you try running

python /srv/modoboa/instance/manage.py modo import --sepchar ; accounts.csv

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/modoboa/modoboa/issues/1430#issuecomment-371671398,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABka_Fc7ZKE26ZnGMSaejz0yPmJygUrmks5tcctegaJpZM4Sjhmq
.

<!-- gh-comment-id:371798333 --> @almereyda commented on GitHub (Mar 9, 2018): Thanks for the introspection. The escaped SEPCHAR yields the very same error. On 9 March 2018 at 01:22, Andrew Fyfe <notifications@github.com> wrote: > Well Line 232 of csv.py destroys any useful error message and replaces it > with Invalid dialect parameters. > > I don't think this is a utf8 problem, I think options["sepchar"] is > either None or an instance of str instead of unicode, I'll need to do > some testing in the morning. > > Can you try running > > python /srv/modoboa/instance/manage.py modo import --sepchar \; accounts.csv > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <https://github.com/modoboa/modoboa/issues/1430#issuecomment-371671398>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ABka_Fc7ZKE26ZnGMSaejz0yPmJygUrmks5tcctegaJpZM4Sjhmq> > . >
kerem commented 2026-02-27 11:15:29 +03:00
Author
Owner
Copy link

@tonioo commented on GitHub (Oct 1, 2018):

@almereyda Is it still happening?

<!-- gh-comment-id:425870732 --> @tonioo commented on GitHub (Oct 1, 2018): @almereyda Is it still happening?
kerem commented 2026-02-27 11:15:29 +03:00
Author
Owner
Copy link

@almereyda commented on GitHub (Oct 2, 2018):

We didn't have a chance to test it again, but will have next week, Thu 11th
of October.

On Mon, 1 Oct 2018 at 13:12, Antoine Nguyen notifications@github.com
wrote:

@almereyda https://github.com/almereyda Is it still happening?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/modoboa/modoboa/issues/1430#issuecomment-425870732,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABka_DBniQlzhI5axVIoucOxOH6BXXA8ks5ugfiXgaJpZM4Sjhmq
.

<!-- gh-comment-id:426237769 --> @almereyda commented on GitHub (Oct 2, 2018): We didn't have a chance to test it again, but will have next week, Thu 11th of October. On Mon, 1 Oct 2018 at 13:12, Antoine Nguyen <notifications@github.com> wrote: > @almereyda <https://github.com/almereyda> Is it still happening? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/modoboa/modoboa/issues/1430#issuecomment-425870732>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ABka_DBniQlzhI5axVIoucOxOH6BXXA8ks5ugfiXgaJpZM4Sjhmq> > . >
kerem commented 2026-02-27 11:15:29 +03:00
Author
Owner
Copy link

@tonioo commented on GitHub (Oct 18, 2018):

@almereyda Any news?

<!-- gh-comment-id:430924449 --> @tonioo commented on GitHub (Oct 18, 2018): @almereyda Any news?
kerem commented 2026-02-27 11:15:29 +03:00
Author
Owner
Copy link

@augustodv commented on GitHub (Jun 24, 2019):

hello, i have de same error and test all but the error persist.

<!-- gh-comment-id:505103067 --> @augustodv commented on GitHub (Jun 24, 2019): hello, i have de same error and test all but the error persist.
kerem commented 2026-02-27 11:15:29 +03:00
Author
Owner
Copy link

@Spitfireap commented on GitHub (Jul 15, 2024):

Should be fixed by #3039. Please open a new ticket if it still shows up.

<!-- gh-comment-id:2228372343 --> @Spitfireap commented on GitHub (Jul 15, 2024): Should be fixed by #3039. Please open a new ticket if it still shows up.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
fix/sievefilters-allow-email-in-rule-name
dependabot/npm_and_yarn/frontend/axios-1.15.0
translations_04c0b653215cb5fa48ab101eae1a1917_ja_JP
translations_5d93982e9fd6dab991db4faf93ce9ded_ja_JP
fix/login-issue-3995
fix/sievefilters-issues
add-vitepress-openapi
2974-dmarc-empty-email
2.3.x
updated-doc-2.3.0
feature/OIDC
delete-all-alarms
translations_modoboa-admin-locale-en-lc-messages-djangojs-po--master_af_ZA
translations_modoboa-admin-locale-en-lc-messages-djangojs-po--master_ar
translations_modoboa-admin-locale-en-lc-messages-djangojs-po--master_ca
1.x
translations_modoboa-admin-locale-en-lc-messages-djangojs-po--master_sk
translations_modoboa-admin-locale-en-lc-messages-djangojs-po--master_tr
translations_modoboa-admin-locale-en-lc-messages-djangojs-po--master_es_ES
translations_modoboa-admin-locale-en-lc-messages-djangojs-po--master_uk
translations_modoboa-admin-locale-en-lc-messages-djangojs-po--master_zh_CN
translations_modoboa-admin-locale-en-lc-messages-djangojs-po--master_de_DE
translations_modoboa-admin-locale-en-lc-messages-djangojs-po--master_da_DK
1.8.x
1.7.x
1.6.x
1.4.x
1.3.x
1.2.x
2.8.2
2.8.1
2.8.0
2.7.2
2.7.1
2.7.0
2.6.5
2.6.4
2.6.3
2.6.2
2.6.1
2.6.0
2.5.1
2.5.0
2.4.11
2.4.10
2.4.9
2.4.8
2.4.7
2.4.6
2.4.5
2.4.4
2.4.3
2.4.2
2.4.1
2.4.0
2.3.6
2.3.5
2.3.4
2.3.3
2.3.2
2.3.1
2.3.0
2.3.0-beta.4
2.3.0-beta.3
2.3.0-beta.2
2.3.0-beta.1
2.2.4
2.2.3
2.2.2
2.2.1
2.2.0
2.1.2
2.1.1
2.1.0
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
2.0.0-beta.3
2.0.0-beta.2
2.0.0-beta.1
1.17.0
1.16.1
1.16.0
1.15.0
1.14.0
1.13.1
1.13.0
1.12.2
1.12.1
1.12.0
1.11.1
1.11.0
1.10.7
1.10.6
1.10.5
1.10.4
1.10.3
1.10.2
1.10.1
1.10.0
1.9.1
1.9.0
1.8.3
1.8.2
1.8.1
1.8.0
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.3
1.6.2
1.6.1
1.6.0
1.5.3
1.5.2
1.5.1
1.5.0
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.5
1.3.4
1.2.2
1.3.3
1.3.2
1.3.1
1.3.0
1.2.1
1.2.0
1.2.0-rc2
1.2.0-rc1
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.1
1.0.0
0.9.5
0.9.4
0.9.3
0.9.2
0.9.1
0.9
0.8.8
0.8.7
0.8.6.1
0.8.6
0.8.5
0.8.4
0.8.3
0.8.3-rc1
0.8.2
0.8.1
0.8
0.8-rc2
0.8-rc1
0.7.2
0.7.1
0.7
0.6.1
0.6
0.5
0.3
0.2
0.1
Labels
Clear labels   
bug

   
bug

   
dependencies

   
design

   
documentation

   
duplicate

   
enhancement

   
enhancement

   
enhancement

   
feedback-needed

   
help-needed

   
help-needed

   
installer

   
invalid

   
looking-for-sponsors

   
modoboa-contacts

   
new-ui

   
new-ui

   
pr

   
pull-request

Mirrored from GitHub Pull Request

  
pyconfr

   
python

   
question

   
security

   
stale

   
webmail

   
wontfix
No labels
bug
bug
dependencies
design
documentation
duplicate
enhancement
enhancement
enhancement
feedback-needed
help-needed
help-needed
installer
invalid
looking-for-sponsors
modoboa-contacts
new-ui
new-ui
pr
pull-request
pyconfr
python
question
security
stale
webmail
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/modoboa-modoboa#1133
No description provided.