[GH-ISSUE #842] requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) #517

Open
opened 2026-02-27 23:23:05 +03:00 by kerem · 2 comments
Owner

Originally created by @dkbarn on GitHub (Aug 3, 2022).
Original GitHub issue: https://github.com/spotipy-dev/spotipy/issues/842

Describe the bug
In my application I use spotipy pretty heavily, and frequently I see the error:
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
It happens once every few hours. There are already some retry options exposed in the spotipy.Spotify.__init__, which I am making use of, but this retry functionality seems to only apply to HTTP errors, not low-level ConnectionErrors. Please could we consider adding retry support for any network-related error?

Your code
The error can happen on any one of the public API methods that ultimately results in communicating with Spotify's REST API server.

Expected behavior
The spotipy library doesn't not throw an error during regular use.

Output

  File "/opt/venv/lib/python3.10/site-packages/spotipy/client.py", line 376, in artists
    return self._get("artists/?ids=" + ",".join(tlist))
  File "/opt/venv/lib/python3.10/site-packages/spotipy/client.py", line 297, in _get
    return self._internal_call("GET", url, payload, kwargs)
  File "/opt/venv/lib/python3.10/site-packages/spotipy/client.py", line 221, in _internal_call
    headers = self._auth_headers()
  File "/opt/venv/lib/python3.10/site-packages/spotipy/client.py", line 212, in _auth_headers
    token = self.auth_manager.get_access_token(as_dict=False)
  File "/opt/venv/lib/python3.10/site-packages/spotipy/oauth2.py", line 238, in get_access_token
    token_info = self._request_access_token()
  File "/opt/venv/lib/python3.10/site-packages/spotipy/oauth2.py", line 257, in _request_access_token
    response = self._session.post(
  File "/opt/venv/lib/python3.10/site-packages/requests/sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "/opt/venv/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/venv/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/opt/venv/lib/python3.10/site-packages/requests/adapters.py", line 547, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Environment:

  • OS: Linux
  • Python version: 3.10.4
  • spotipy version: 2.20.0
Originally created by @dkbarn on GitHub (Aug 3, 2022). Original GitHub issue: https://github.com/spotipy-dev/spotipy/issues/842 **Describe the bug** In my application I use spotipy pretty heavily, and frequently I see the error: ```requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))``` It happens once every few hours. There are already some retry options exposed in the `spotipy.Spotify.__init__`, which I am making use of, but this retry functionality seems to only apply to HTTP errors, not low-level ConnectionErrors. Please could we consider adding retry support for any network-related error? **Your code** The error can happen on any one of the public API methods that ultimately results in communicating with Spotify's REST API server. **Expected behavior** The spotipy library doesn't not throw an error during regular use. **Output** ``` File "/opt/venv/lib/python3.10/site-packages/spotipy/client.py", line 376, in artists return self._get("artists/?ids=" + ",".join(tlist)) File "/opt/venv/lib/python3.10/site-packages/spotipy/client.py", line 297, in _get return self._internal_call("GET", url, payload, kwargs) File "/opt/venv/lib/python3.10/site-packages/spotipy/client.py", line 221, in _internal_call headers = self._auth_headers() File "/opt/venv/lib/python3.10/site-packages/spotipy/client.py", line 212, in _auth_headers token = self.auth_manager.get_access_token(as_dict=False) File "/opt/venv/lib/python3.10/site-packages/spotipy/oauth2.py", line 238, in get_access_token token_info = self._request_access_token() File "/opt/venv/lib/python3.10/site-packages/spotipy/oauth2.py", line 257, in _request_access_token response = self._session.post( File "/opt/venv/lib/python3.10/site-packages/requests/sessions.py", line 635, in post return self.request("POST", url, data=data, json=json, **kwargs) File "/opt/venv/lib/python3.10/site-packages/requests/sessions.py", line 587, in request resp = self.send(prep, **send_kwargs) File "/opt/venv/lib/python3.10/site-packages/requests/sessions.py", line 701, in send r = adapter.send(request, **kwargs) File "/opt/venv/lib/python3.10/site-packages/requests/adapters.py", line 547, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) ``` **Environment:** - OS: Linux - Python version: 3.10.4 - spotipy version: 2.20.0
Author
Owner

@FrederikP commented on GitHub (Oct 12, 2022):

👍

Looking at the code in client.py the retry stuff is initialized like this:

        retry = urllib3.Retry(
            total=self.retries,
            connect=None,
            read=False,
            allowed_methods=frozenset(['GET', 'POST', 'PUT', 'DELETE']),
            status=self.status_retries,
            backoff_factor=self.backoff_factor,
            status_forcelist=self.status_forcelist)

The docs say connect and read should be ints:

:param int connect:
    How many connection-related errors to retry on.

    These are errors raised before the request is sent to the remote server, which we assume has not triggered the server to process the request.

    Set to 0 to fail on the first retry of this type.

:param int read:
    How many times to retry on read errors.

    These errors are raised after the request was sent to the server, so the request may have side-effects.

    Set to 0 to fail on the first retry of this type.

I would really like to see the ability to provide a custom retry object. In my use case I'm seeing connection errors every time I have a longer running process. Probably connections are dropped one sided and we only find out when we try to do the next request.

<!-- gh-comment-id:1276214322 --> @FrederikP commented on GitHub (Oct 12, 2022): :+1: Looking at the code in `client.py` the retry stuff is initialized like this: ``` retry = urllib3.Retry( total=self.retries, connect=None, read=False, allowed_methods=frozenset(['GET', 'POST', 'PUT', 'DELETE']), status=self.status_retries, backoff_factor=self.backoff_factor, status_forcelist=self.status_forcelist) ``` The docs say connect and read should be ints: ``` :param int connect: How many connection-related errors to retry on. These are errors raised before the request is sent to the remote server, which we assume has not triggered the server to process the request. Set to 0 to fail on the first retry of this type. :param int read: How many times to retry on read errors. These errors are raised after the request was sent to the server, so the request may have side-effects. Set to 0 to fail on the first retry of this type. ``` I would really like to see the ability to provide a custom retry object. In my use case I'm seeing connection errors every time I have a longer running process. Probably connections are dropped one sided and we only find out when we try to do the next request.
Author
Owner

@jbkze commented on GitHub (Jul 17, 2023):

This is my workaround:

    def add_tracks_to_playlist(self, tracks: list, playlist_id: str):
        MAX_RETRIES = 10
        retry_count = 0

        for chunk in tqdm(chunk_list(tracks, 100)):
            while retry_count < MAX_RETRIES:
                try:
                    self.spotify.playlist_add_items(playlist_id, chunk)
                    break
                except ConnectionError as e:
                    print(f"Error encountered: {e}")
                    print(f"Retrying... (Attempt {retry_count + 1} of {MAX_RETRIES})")
                    retry_count += 1
                    time.sleep(1)
            else:
                print(f"Failed to add chunk to playlist after {MAX_RETRIES} attempts. Skipping...")
            retry_count = 0
<!-- gh-comment-id:1637817652 --> @jbkze commented on GitHub (Jul 17, 2023): This is my workaround: ```python def add_tracks_to_playlist(self, tracks: list, playlist_id: str): MAX_RETRIES = 10 retry_count = 0 for chunk in tqdm(chunk_list(tracks, 100)): while retry_count < MAX_RETRIES: try: self.spotify.playlist_add_items(playlist_id, chunk) break except ConnectionError as e: print(f"Error encountered: {e}") print(f"Retrying... (Attempt {retry_count + 1} of {MAX_RETRIES})") retry_count += 1 time.sleep(1) else: print(f"Failed to add chunk to playlist after {MAX_RETRIES} attempts. Skipping...") retry_count = 0
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/spotipy#517
No description provided.