[GH-ISSUE #221] Session hangs indefinitely #174

Closed
opened 2026-02-27 22:08:26 +03:00 by kerem · 5 comments
Owner

Originally created by @colbeyhair on GitHub (Aug 20, 2021).
Original GitHub issue: https://github.com/sigma67/ytmusicapi/issues/221

I have found that while using this API in programs that run for a long time, the program will sometimes hang up indefinitely (for several days at least).

When the program hangs up, sending a ctrl+c sigkill results in a traceback showing that the program was in the middle of a session request, specifically in the get_playlists() method. (Unfortunately I lost the traceback and cannot copy it here).

The requests documentation page recommends passing a timeout parameter to all session instances to avoid such an indefinite hangup: https://docs.python-requests.org/en/master/user/quickstart/#timeouts

Such a parameter is not passed when the requests.Session() is initialized here: https://github.com/sigma67/ytmusicapi/blob/master/ytmusicapi/ytmusic.py#L62

I think that adding a timeout of a few seconds should allow the program to timeout when appropriate and avoid hangups. Perhaps the timeout can be a parameter passed when instantiating ytmusic if adjustability is desired.

Originally created by @colbeyhair on GitHub (Aug 20, 2021). Original GitHub issue: https://github.com/sigma67/ytmusicapi/issues/221 I have found that while using this API in programs that run for a long time, the program will sometimes hang up indefinitely (for several days at least). When the program hangs up, sending a ctrl+c sigkill results in a traceback showing that the program was in the middle of a session request, specifically in the `get_playlists()` method. (Unfortunately I lost the traceback and cannot copy it here). The requests documentation page recommends passing a timeout parameter to all session instances to avoid such an indefinite hangup: https://docs.python-requests.org/en/master/user/quickstart/#timeouts Such a parameter is not passed when the `requests.Session()` is initialized here: https://github.com/sigma67/ytmusicapi/blob/master/ytmusicapi/ytmusic.py#L62 I think that adding a timeout of a few seconds should allow the program to timeout when appropriate and avoid hangups. Perhaps the timeout can be a parameter passed when instantiating ytmusic if adjustability is desired.
kerem closed this issue 2026-02-27 22:08:26 +03:00
Author
Owner

@sigma67 commented on GitHub (Aug 20, 2021):

Good point, I didn't have this issue yet so I never noticed. The primary concern that could be affected negatively by a timeout would be upload_song, but that currently doesn't use the session anyway.

Since the primary concern is to avoid rare indefinite hangups, we could set a conservative value. Do you think 30 seconds is appropriate? I don't think we need to make it a parameter if it only happens rarely. Besides, it is already possible to pass a requests Session object as the requests_session parameter, which is then used by ytmusicapi.

Additionally, requests doesn't really support timeouts on a session level, you need to modify the session object like this: https://github.com/psf/requests/issues/2011#issuecomment-490050252

s = requests.Session()
s.request = functools.partial(s.request, timeout=3)
# this should now timeout
s.get('https://httpbin.org/delay/6')
<!-- gh-comment-id:902549490 --> @sigma67 commented on GitHub (Aug 20, 2021): Good point, I didn't have this issue yet so I never noticed. The primary concern that could be affected negatively by a timeout would be `upload_song`, but that currently doesn't use the session anyway. Since the primary concern is to avoid rare indefinite hangups, we could set a conservative value. Do you think 30 seconds is appropriate? I don't think we need to make it a parameter if it only happens rarely. Besides, it is already possible to pass a requests Session object as the `requests_session` parameter, which is then used by ytmusicapi. Additionally, requests doesn't really support timeouts on a session level, you need to modify the session object like this: https://github.com/psf/requests/issues/2011#issuecomment-490050252 ```python s = requests.Session() s.request = functools.partial(s.request, timeout=3) # this should now timeout s.get('https://httpbin.org/delay/6')
Author
Owner

@colbeyhair commented on GitHub (Aug 20, 2021):

I agree! A conservative value makes sense to avoid the hangups, and since it is only to keep a program moving in rare cases, a long timeout seems reasonable as a default value.

However, I think it would be very useful to have a timeout argument in the ytmusic constructor. If it happens that I have a bad internet connection and I get timeouts frequently, it could be very useful to pass a parameter from my user code so I can set the timeout as short as possible without interfering with normal request operations. That way, my program wouldn't have to wait a full 30s before raising exception for me to try again.

Having any timeout, even hard-coded, would fix the issue I'm observing. Having a variable timeout would just be "nice to have"

<!-- gh-comment-id:902796384 --> @colbeyhair commented on GitHub (Aug 20, 2021): I agree! A conservative value makes sense to avoid the hangups, and since it is only to keep a program moving in rare cases, a long timeout seems reasonable as a default value. However, I think it would be very useful to have a timeout argument in the ytmusic constructor. If it happens that I have a bad internet connection and I get timeouts frequently, it could be very useful to pass a parameter from my user code so I can set the timeout as short as possible without interfering with normal request operations. That way, my program wouldn't have to wait a full 30s before raising exception for me to try again. Having any timeout, even hard-coded, would fix the issue I'm observing. Having a variable timeout would just be "nice to have"
Author
Owner

@sigma67 commented on GitHub (Aug 20, 2021):

Setting a timeout is possible right now, it's a two-liner (which is what I was trying to say above). Here's an example for a 3 second timeout, feel free to try it out:

s = requests.Session()
s.request = functools.partial(s.request, timeout=3)

ytm = YTMusic(session=s)

See also the docs

I think that should cover the variable timeout part.

<!-- gh-comment-id:902967079 --> @sigma67 commented on GitHub (Aug 20, 2021): Setting a timeout is possible right now, it's a two-liner (which is what I was trying to say above). Here's an example for a 3 second timeout, feel free to try it out: ```python s = requests.Session() s.request = functools.partial(s.request, timeout=3) ytm = YTMusic(session=s) ``` [See also the docs](https://ytmusicapi.readthedocs.io/en/latest/reference.html?highlight=session#ytmusicapi.YTMusic.__init__) I think that should cover the variable timeout part.
Author
Owner

@colbeyhair commented on GitHub (Aug 22, 2021):

Thanks for the suggestion, sigma! I have tried the above recommendation (with a change to the YTMusic argument to avoid a kwarg typo):

s = requests.Session()
s.request = partial(s.request, timeout=10)
ytmusic = YTMusic(auth="youtube_auth_1.json", requests_session=s)

It took me a while to get back to you because I wanted to try your solution pretty thoroughly. I was able to capture the following exception and traceback when the session inevitably timed out:

Traceback (most recent call last):
  File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.7/http/client.py", line 1352, in getresponse
    response.begin()
  File "/usr/lib/python3.7/http/client.py", line 310, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.7/http/client.py", line 271, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.7/ssl.py", line 1052, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.7/ssl.py", line 911, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<redacted>/venv/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "<redacted>/venv/lib/python3.7/site-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "<redacted>/venv/lib/python3.7/site-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 337, in _raise_timeout
    self, url, "Read timed out. (read timeout=%s)" % timeout_value
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='music.youtube.com', port=443): Read timed out. (read timeout=10)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "create_new_playlists.py", line 192, in <module>
    create_new_music_playlists(date_to_generate = args.d)
  File "create_new_playlists.py", line 81, in create_new_music_playlists
    song_log_file_name=songs_log_fname_youtube,
  File "<redacted>/youtube_music_playlist.py", line 263, in create_youtube_music_playlist
    add_album_to_playlist(playlist_id, album_object, ytmusic)
  File "<redacted>/youtube_music_playlist.py", line 108, in add_album_to_playlist
    ytmusic.get_playlist(playlist_id, limit=PLAYLIST_LENGTH_TARGET), song_ids
  File "<redacted>/venv/lib/python3.7/site-packages/ytmusicapi/mixins/playlists.py", line 114, in get_playlist
    parse_func))
  File "<redacted>/venv/lib/python3.7/site-packages/ytmusicapi/parsers/utils.py", line 59, in get_continuations
    response = request_func(additionalParams)
  File "<redacted>/venv/lib/python3.7/site-packages/ytmusicapi/mixins/playlists.py", line 109, in <lambda>
    endpoint, body, additionalParams)
  File "<redacted>/venv/lib/python3.7/site-packages/ytmusicapi/ytmusic.py", line 128, in _send_request
    proxies=self.proxies)
  File "<redacted>/venv/lib/python3.7/site-packages/requests/sessions.py", line 590, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "<redacted>/venv/lib/python3.7/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "<redacted>/venv/lib/python3.7/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "<redacted>/venv/lib/python3.7/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='music.youtube.com', port=443): Read timed out. (read timeout=10)

I am able to successfully handle this exception:


for item in my_list:
    item_complete = False
    while not item_complete:
        try:
            function_that_uses_ytm()
            item_complete = True

        except requests.exceptions.ReadTimeout as e:
            # log the timeout occurrences so I can see how often they happen
            with open("timeout_log.txt", 'a') as fout:
                fout.write(str(datetime.now()))
<!-- gh-comment-id:903330389 --> @colbeyhair commented on GitHub (Aug 22, 2021): Thanks for the suggestion, sigma! I have tried the above recommendation (with a change to the YTMusic argument to avoid a kwarg typo): ```python s = requests.Session() s.request = partial(s.request, timeout=10) ytmusic = YTMusic(auth="youtube_auth_1.json", requests_session=s) ``` It took me a while to get back to you because I wanted to try your solution pretty thoroughly. I was able to capture the following exception and traceback when the session inevitably timed out: ``` Traceback (most recent call last): File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.7/http/client.py", line 1352, in getresponse response.begin() File "/usr/lib/python3.7/http/client.py", line 310, in begin version, status, reason = self._read_status() File "/usr/lib/python3.7/http/client.py", line 271, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.7/socket.py", line 589, in readinto return self._sock.recv_into(b) File "/usr/lib/python3.7/ssl.py", line 1052, in recv_into return self.read(nbytes, buffer) File "/usr/lib/python3.7/ssl.py", line 911, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<redacted>/venv/lib/python3.7/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 756, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "<redacted>/venv/lib/python3.7/site-packages/urllib3/util/retry.py", line 532, in increment raise six.reraise(type(error), error, _stacktrace) File "<redacted>/venv/lib/python3.7/site-packages/urllib3/packages/six.py", line 770, in reraise raise value File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 706, in urlopen chunked=chunked, File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 447, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "<redacted>/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 337, in _raise_timeout self, url, "Read timed out. (read timeout=%s)" % timeout_value urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='music.youtube.com', port=443): Read timed out. (read timeout=10) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "create_new_playlists.py", line 192, in <module> create_new_music_playlists(date_to_generate = args.d) File "create_new_playlists.py", line 81, in create_new_music_playlists song_log_file_name=songs_log_fname_youtube, File "<redacted>/youtube_music_playlist.py", line 263, in create_youtube_music_playlist add_album_to_playlist(playlist_id, album_object, ytmusic) File "<redacted>/youtube_music_playlist.py", line 108, in add_album_to_playlist ytmusic.get_playlist(playlist_id, limit=PLAYLIST_LENGTH_TARGET), song_ids File "<redacted>/venv/lib/python3.7/site-packages/ytmusicapi/mixins/playlists.py", line 114, in get_playlist parse_func)) File "<redacted>/venv/lib/python3.7/site-packages/ytmusicapi/parsers/utils.py", line 59, in get_continuations response = request_func(additionalParams) File "<redacted>/venv/lib/python3.7/site-packages/ytmusicapi/mixins/playlists.py", line 109, in <lambda> endpoint, body, additionalParams) File "<redacted>/venv/lib/python3.7/site-packages/ytmusicapi/ytmusic.py", line 128, in _send_request proxies=self.proxies) File "<redacted>/venv/lib/python3.7/site-packages/requests/sessions.py", line 590, in post return self.request('POST', url, data=data, json=json, **kwargs) File "<redacted>/venv/lib/python3.7/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "<redacted>/venv/lib/python3.7/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "<redacted>/venv/lib/python3.7/site-packages/requests/adapters.py", line 529, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='music.youtube.com', port=443): Read timed out. (read timeout=10) ``` I am able to successfully handle this exception: ```python for item in my_list: item_complete = False while not item_complete: try: function_that_uses_ytm() item_complete = True except requests.exceptions.ReadTimeout as e: # log the timeout occurrences so I can see how often they happen with open("timeout_log.txt", 'a') as fout: fout.write(str(datetime.now())) ```
Author
Owner

@sigma67 commented on GitHub (Aug 26, 2021):

That's great, thanks for the detailed feedback. Will add some documentation and the default timeout.

<!-- gh-comment-id:906592378 --> @sigma67 commented on GitHub (Aug 26, 2021): That's great, thanks for the detailed feedback. Will add some documentation and the default timeout.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/ytmusicapi#174
No description provided.