mirror of
https://github.com/modoboa/modoboa.git
synced 2026-04-27 18:05:58 +03:00
[GH-ISSUE #2087] Problem with "manage.py communicate_with_public_api" #1523
Labels
No labels
bug
bug
dependencies
design
documentation
duplicate
enhancement
enhancement
enhancement
feedback-needed
help-needed
help-needed
installer
invalid
looking-for-sponsors
modoboa-contacts
new-ui
new-ui
pr
pull-request
pyconfr
python
question
security
stale
webmail
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/modoboa-modoboa#1523
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @nsabatelli on GitHub (Nov 24, 2020).
Original GitHub issue: https://github.com/modoboa/modoboa/issues/2087
Hello,
in the execution of the procedure "communicate_with_public_api" for some days I get these errors.
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked = chunked,
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 426, in _make_request
six.raise_from (e, None)
File "", line 3, in raise_from
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 421, in _make_request
httplib_response = conn.getresponse ()
File "/usr/lib64/python3.6/http/client.py", line 1346, in getresponse
response.begin ()
File "/usr/lib64/python3.6/http/client.py", line 307, in begin
version, status, reason = self._read_status ()
File "/usr/lib64/python3.6/http/client.py", line 276, in _read_status
raise RemoteDisconnected ("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout = timeout
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 725, in urlopen
method, url, error = e, _pool = self, _stacktrace = sys.exc_info () [2]
File "/usr/lib/python3.6/site-packages/urllib3/util/retry.py", line 403, in increment
raise six.reraise (type (error), error, _stacktrace)
File "/usr/lib/python3.6/site-packages/urllib3/packages/six.py", line 734, in reraise
raise value.with_traceback (tb)
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked = chunked,
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 426, in _make_request
six.raise_from (e, None)
File "", line 3, in raise_from
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 421, in _make_request
httplib_response = conn.getresponse ()
File "/usr/lib64/python3.6/http/client.py", line 1346, in getresponse
response.begin ()
File "/usr/lib64/python3.6/http/client.py", line 307, in begin
version, status, reason = self._read_status ()
File "/usr/lib64/python3.6/http/client.py", line 276, in _read_status
raise RemoteDisconnected ("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected ('Remote end closed connection without response',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/www/html/modoboa_ct/manage.py", line 21, in
main ()
File "/var/www/html/modoboa_ct/manage.py", line 17, in main
execute_from_command_line (sys.argv)
File "/usr/lib/python3.6/site-packages/django/core/management/init.py", line 381, in execute_from_command_line
utility.execute ()
File "/usr/lib/python3.6/site-packages/django/core/management/init.py", line 375, in execute
self.fetch_command (subcommand) .run_from_argv (self.argv)
File "/usr/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
self.execute (* args, ** cmd_options)
File "/usr/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
output = self.handle (* args, ** options)
File "/usr/lib/python3.6/site-packages/modoboa/core/management/commands/communicate_with_public_api.py", line 93, in handle
self.client.update_instance (local_config.api_pk, data)
File "/usr/lib/python3.6/site-packages/modoboa/lib/api_client.py", line 66, in update_instance
response = requests.put (url, data = data)
File "/usr/lib/python3.6/site-packages/requests/api.py", line 134, in put
return request ('put', url, data = date, ** kwargs)
File "/usr/lib/python3.6/site-packages/requests/api.py", line 61, in request
return session.request (method = method, url = url, ** kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
resp = self.send (prep, ** send_kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
r = adapter.send (request, ** kwargs)
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError (err, request = request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected ('Remote end closed connection without response',))
The modoboa version I use is 1.17.0 on CentOS 7 operating system.
How can I solve the problem?
Thanks.
Nicola
@floydwilde commented on GitHub (Nov 25, 2020):
I also get a similar error followed by:
CommandError: Failed to retrieve versions from the API.I think there is a problem w/ the public api here:
There is a cron job which triggers this command:
@zeroepix commented on GitHub (Jan 8, 2021):
Are you guys using IPv6? I started getting this same issue after I switched to that and I'm wondering if that was the cause. I also switch servers and to a newer version so...could be other things too ><
@nsabatelli commented on GitHub (Jan 8, 2021):
Hi everyone,
no I don't use IPv6, I just updated to the latest release available.
@tonioo commented on GitHub (Jan 18, 2021):
It might happen if the API is unreachable for some reason, or it the server hosting it is overloaded. But according to the monitoring, it is not...
@tonioo commented on GitHub (Jan 19, 2021):
@nsabatelli Could you give me the hostname of your server?
@nsabatelli commented on GitHub (Jan 19, 2021):
The hostname of my server is modoboa.rupar.puglia.it
@jaggeddragon commented on GitHub (Feb 11, 2021):
I continue to get the same error on a brand new installation. It seems to be intermittent, so I assume it is related to load on the API end. Have you considered cloudlflare for this resource endpoint? Is it possible for server admins to make our own api backend, and point the DNS for api.modoboa.org there to prevent this annoyance?
@jaggeddragon commented on GitHub (Feb 11, 2021):
I performed the following analysis, and my conclusion is that CloudFlare would be a good option for improving uptime and reliability of this endpoint, furthermore it seems that removing the 'communicate_with_public_api' cronjob entirely would have no downside, as the endpoint does not perform any function what-so-ever when it is responding. I would further urge the current admin to no longer host the api endpoint for this on their mailserver 'mail.koalabs.org' while performance and accessibility issues cause headaches for mailserver administrators.
# curl https://api.modoboa.org/1/curl: (6) Could not resolve host: api.modoboa.org# dig +short api.modoboa.orgmodoboa.org.94.23.45.181# sudo echo '94.23.45.181 api.modoboa.org modoboa.org' >> /etc/hosts# dig +short api.modoboa.org94.23.45.181# curl https://api.modoboa.org/1/Not Found The requested resource was not found on this server.What is this endpoint intended to do? What is the function of the cronjob which hits this endpoint? What was the original intent of this portion of the code?
@tonioo commented on GitHub (Feb 11, 2021):
@jaggeddragon The public API is used to check for updates and to collect statistics about used plugins and domain/mailbox/alias counters. That's something you can disable from the admin panel.
This issue is indeed caused by a server load problem and unfortunately, you can't duplicate the API on your own server...
@jaggeddragon commented on GitHub (Feb 11, 2021):
When will the server load problem be addressed? What efforts have you taken to address the server load problem already? Have you reviewed the free option of CloudFlare I presented previously?
@tonioo commented on GitHub (Feb 12, 2021):
@jaggeddragon It will be addressed when I'll have enough time to work on it. Don't forget Modoboa is a free software, no company behind, no incomes.
Can you explain how using Cloudflare would prevent this performance issue?
@jaggeddragon commented on GitHub (Feb 14, 2021):
If the root cause is server performance and load as you mentioned, leveraging CloudFlares DDOS mitigation to handle repeat requests for this web content would decrease load on the end web-server. Similar to how a content delivery network would help for static resources. CloudFlare also has a free option, any only involves a DNS update, so the onboarding is cheap and simple. Don't take my word for it, check out the CloudFlare website https://www.cloudflare.com/
@floydwilde commented on GitHub (Feb 14, 2021):
Another option you can consider is using something like Github pages to host the API, since you are already hosting the code base here and Github have a robust infrastructure with similar CDN qualities to Cloud Flare. Here is a blog post I found which describes how one person dealt with it:
Something to consider if you did not want to add another vendor in the mix, even though Cloud Flare is free and recommended, their terms could always change.
@kisst commented on GitHub (Feb 15, 2021):
A non-infra solution can be to have the manage.py to gracefully handle the exception and not consider it as error, hence it would not spam the postmaster with something which actually not an error, at least I would not call it an error.
@brucemartin1979 commented on GitHub (Mar 25, 2021):
This would be a good idea. I also have a new install that randomly generates this error.
@skandragon commented on GitHub (Mar 28, 2021):
So, two questions.
The reason I ask is because I am getting spammed by these errors when running from cron, but if I run it from the command line, no error.