mirror of
https://github.com/spotipy-dev/spotipy.git
synced 2026-04-27 08:35:49 +03:00
[GH-ISSUE #790] API requests suddenly stopped working and hang forever #489
Labels
No labels
api-bug
bug
dependencies
documentation
duplicate
enhancement
external-ide
headless-mode
implicit-grant-flow
invalid
missing-endpoint
pr-welcome
private-api
pull-request
question
spotipy3
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/spotipy#489
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @julienbeisel on GitHub (Mar 11, 2022).
Original GitHub issue: https://github.com/spotipy-dev/spotipy/issues/790
Hi,
I recently started using Spotipy to get information about artists for a dataviz project.
As you can see, one of my apps makes a lot of requests, but I make sure to wait between requests so I don't get limited by the 30 second time-window limit.
Sometimes, the API just stops working for 1 to 24 hours. I don't get any error message, it just keeps running without responding.
Here is how I initialize the client
Do you know why I have this error ? I try every 12h to run my code, sometimes it works and sometimes it doesn't.
I can't find any information about the rate limite of Spotify, but I think I'm not reaching any limit otherwise I would get an error as a response?
Thanks for your help!
@ulysses82 commented on GitHub (Mar 28, 2022):
Same issue here. The API stopped working for me too (with no error messages). Pretty sure it's rate limiting.
@justingrisanti commented on GitHub (May 3, 2022):
I am also having this issue now when I use the .search() or .track() methods. Was this ever resolved for you?
@Peter-Schorn commented on GitHub (May 4, 2022):
If your requests are getting rate-limited, then you will receive a very specific error message indicating that you have been rate-limited.
Whenever your code freezes, you should step through it in the debugger to see exactly which function is blocking. Have you done this?
@julienbeisel commented on GitHub (May 4, 2022):
Hi @Peter-Schorn, I managed to debug by doing this :
When I tried it, the API sent an error message because I was rate-limited and asked me to try after x minutes. The issue was the rate-limitation but the behaviour Spotipy was not what I was expected, it just hanged forever without returning an error.
I think the Spotipy package should expose this information so we could automate the process of retrying after x seconds.
After looking at the code, I think it should work, so I don't know what the issue was: https://github.com/plamere/spotipy/blob/master/spotipy/exceptions.py#L8
@dasbts commented on GitHub (May 4, 2022):
I'm getting this issue when trying to do anything, and it works fine in the Spotify Dev API console, thus confirming the issue is somewhere else. Started getting the issue yesterday, then after some time it worked again and today it's just nothing at all. No changes to the code, but I tried multiple auth flows such as:
Or:
I don't know if I'm using debugger incorrectly, but as soon as I run any of the sp.XX functions from the Spotipy library, it just freezes and doesn't add anything to the output.
I'm using Anaconda JupyterLab (the newest version) by the way, and for some reason, this doesn't just freeze the code, it freezes the whole Python, so I have to close it manually in Windows Task Manager before starting a new one, otherwise, it's just gonna keep spinning even tho Anaconda doesn't realize it's still open, this as the port is still available for the Jupyter.
Also to add, similarly to the OP, I run a lot of requests in a loop until I'm satisfied, but again, it works on the console so it makes no sense. Removed the cache as well to try because initially I got a refused refresh token error.
EDIT: My first calls are sp.current_user_playlists() and sp.playlist(), haven't tested any other ones yet but it fails on these. Sometimes it almost seems like restarting the computer works.
@dasbts commented on GitHub (May 4, 2022):
Ok I noticed that a different access token is generated from Spotipy with the code examples posted above, than when generating one in the console. The Spotipy doesn't work but rather just loads in their console too.
So, it seems like after using the token generated on the console that this works fine when added to the Spotipy code, while Spotipy is no longer able to generate whatever they have changed the access token requests into.
@Peter-Schorn commented on GitHub (May 4, 2022):
You need to either use a different IDE or figure out how to use the debugger on Anaconda JupyterLab. With a proper debugger, you can step through each line of code to figure out exactly where the code is blocking. Merely saying that the code freezes at
sp.current_user_playlists()is not specific enough because this function calls through to another function, which calls through to another function, and so on.@dasbts commented on GitHub (May 6, 2022):
Alright, but as I told you in my next post, I already found where the issue is without debugger, in my case it's not in any of the features, but rather in the authentication.
I run the code successfully by going to the console and requesting a new token manually for the scopes I need, then copying that into my python code and running.
However, when I run the authentication code (any of the above, even the other Python library Tekore, gonna try manual request too later) then it freezes, and even if I paste this token in the console at Spotify, it freezes in their API too. Which means something has happened in the way you request tokens, and something has happened in the how their token standard looks like.
Please look into this ASAP if you can, I'm dependent on it for multiple products.
@Peter-Schorn commented on GitHub (May 6, 2022):
No you didn't find where the issue is. The fact that the authentication code freezes is not nearly specific enough. There are a million places where the authentication code can freeze. You can find the specific place where the code freezes by using the debugger. That's why it exists.
@Peter-Schorn commented on GitHub (May 6, 2022):
A different and unique access token is generated every time you go through the authorization process, no matter how you do it.
@dasbts commented on GitHub (May 6, 2022):
I know. But the ones with any Python library does not work any longer. It even freezes the Spotify Web console. If you try it out you will see what I mean. They changed something in the standard format of access tokens the recent days it seems like. When generated on the console and then manually entered into the other Spotipy functions it works fine as I explained.
@dasbts commented on GitHub (May 6, 2022):
The issue is in the generation of the access token. Since I can reproduce the issue without a library in the https://developer.spotify.com/console/ website when using an access token it's clear that the issue is in generating the access token, and not in any other functions of Spotipy. Yes, generating an access token works, but anything that token is used in, including the console at https://developer.spotify.com/console/ is going to freeze the output, while a token generated in https://developer.spotify.com/console/ is working in all cases.
@dasbts commented on GitHub (May 21, 2022):
Still getting this after running the code for about a day, then this happens for about 24h where I have to manually enter the token generated in https://developer.spotify.com/console/
The weird thing is, it works for some functions even with the token updated from Spotipy (which seems to freeze no matter where it's used in some functions). So to clarify:
When I generate a token with:
The code works when running:
sp.available_markets()But freezes (likely infinitely looping) when running for example:
sp.search('Test', limit=50, offset=0, type='playlist', market=None)Or as previously mentioned for example:
sp.current_user_playlists()This makes me think there's some try: except: somewhere that's depressing whatever error this is about.
Why? I don't get any error, the token from the console is working (thus, my account does not rate limited), and seeing that OAuth is required for Get Markets as described here, yet works while some other OAuth functionality does not work:
https://developer.spotify.com/console/get-available-markets/
Could it maybe be some functionality built-in for automatically avoiding rate limits? I have implemented time.sleep(0.3), but it seems like that's not helping prevent this from happening.
I have yet to get a debugger going that actually shows me valuable information. From what I've achieved so far, the debugger hasn't shown me anything of value, but I will look into this in more detail in the next weeks. Also, if I can't figure out the debugger to identify exactly where, and if it's not already found by someone else by then, I will just try all the functionalities to see what works or not and go through the Spotipy library code manually to see what causes this.
@Peter-Schorn commented on GitHub (May 21, 2022):
Never user
util.prompt_for_user_token! It's officially deprecated; the token will not get automatically refreshed.Look at the examples in the README:
This is how you authorize your app.
@dasbts commented on GitHub (May 21, 2022):
While I appreciate your tips for authentication, Peter, it does not help this issue. I already have a code to automatically refresh my token whenever I get an error that it expired, and I get a token fine, it's just whenever running certain functions (even with your code above) it's not working (which I have described in detail in the previous posts, how some OAuth functions work, and some don't whenever this happens), so util works fine for me. I tried the following and still get an infinitely frozen code, thus, indicating it is in some form of an infinite loop. I can look into the Spotify source code sometime likely next week to try to reproduce it manually.
My bet is either there's a try: except: somewhere in the defined functions talking to the Spotify API, or there's something in the handling of cache that contains some bug.
@BenAM63 commented on GitHub (Jun 16, 2022):
I was having this issue occur when trying to call
sp.search()orsp.playlist_items(). Using a debugger I found that eventually in the retry.py file in urllib3, a function calledsleep_for_retry()was being called:It seems that
time.sleep(retry_after)is used to wait instead of sending more requests and being rate limited. Since no message is printed, it seems like it just freezes. I couldn't find a consistent fix, as using different credentials when initializing the client didn't fix it. Waiting out the retry_after timer appears to work, but that could take many hours.@Peter-Schorn commented on GitHub (Jun 16, 2022):
If you use the library correctly, as I have mentioned before, the token will get automatically refreshed before it expires. This saves a lot of time, if nothing else.
@Terima commented on GitHub (Jul 10, 2022):
Apparently this happened to me too. It would be better if at least we know how long we need to wait before the next retry.
@TomBer0-0 commented on GitHub (Mar 28, 2023):
This happened to me today too. It's the third time, and it's more annoying than ever because my app isn't working after waiting for over 2 hours already. It almost seems like the cooldown time is random.
@stephanebruckert commented on GitHub (Mar 30, 2023):
That's not spotipy related. Well, the only thing we could do is add a debug line to let the users know how long they can expect to wait. See this thread where some element of answers were provided in https://github.com/spotipy-dev/spotipy/issues/937 and https://github.com/spotipy-dev/spotipy/issues/913#issuecomment-1474314058
I imagine that switching to a new Spotipfy app would solve the issue, for some time at least, depending on how much the Spotify API rate limits your app.
Let's continue this discussion in https://github.com/spotipy-dev/spotipy/issues/913