mirror of
https://github.com/spotipy-dev/spotipy.git
synced 2026-04-26 16:15:51 +03:00
[GH-ISSUE #726] Spotipy Search Method fails to find results containing a Right Single Quotation Mark #437
Labels
No labels
api-bug
bug
dependencies
documentation
duplicate
enhancement
external-ide
headless-mode
implicit-grant-flow
invalid
missing-endpoint
pr-welcome
private-api
pull-request
question
spotipy3
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/spotipy#437
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @jospkelly on GitHub (Sep 8, 2021).
Original GitHub issue: https://github.com/spotipy-dev/spotipy/issues/726
Bug Description
I am using Spotipy to build code that fetches Spotify Track IDs and generates a playlist. My input is a list of track information extracted from my Plex server.
I discovered that the Spotipy Search Method fails to return a track if the Track Name contains a Right Single Quotation Mark (i.e., a punctuation mark commonly used in contractions).
My Code (example)
Expected Behavior
The code should return a dictionary with one track item:
Actual Behavior
The code does not locate the track, and returns a dictionary with zero track items:
Environment:
OS = MacOS Big Sur 11.3
Python = version 3.8
Spotipy = version 2.19.0
IDE = Pycharm
Additional Context
If I remove the Right Single Quotation Mark from the track name:
The Search Method returns the expected response:
It appears that when Spotipy encodes the URL for use by the API, the Right Single Quotation Mark is converted to %E2%80%99:
@Peter-Schorn commented on GitHub (Sep 8, 2021):
The track you are searching for does not contain a RIGHT SINGLE QUOTATION MARK. It contains an apostrophe. That's why including the former in your search query does not return the track. Furthermore, I created a playlist with a RIGHT SINGLE QUOTATION MARK and was able to successfully retrieve it using the
searchfunction. This character is percent-encoded correctly.@jospkelly commented on GitHub (Sep 8, 2021):
@Peter-Schorn - %E2%80%99 is described as a RIGHT SINGLE QUOTATION MARK HERE and HERE. I did find a second reference HERE that describes the character as an apostrophe and as a Right Single Quote. This is the character stored in my Plex database for this track.
Please let me know the encoding of the character that you used successfully for your search.
If I change my search to this (change the character to an apostrophe, %27, as defined HERE):
artist = 'Broods'
track = "Couldn't Believe"
album = 'Conscious'
track_info_sp = sp.search(q='artist:' + artist + ' track:' + track + ' album:' + album, type='track', limit=25)
``
The search returns no track results:
and the returned dictionary shows this as the href :
@Peter-Schorn commented on GitHub (Sep 8, 2021):
Yes, as I said, this is the correct percent-encoding for this character.
It doesn't matter what you have stored in your Plex database. What matters is that the name of the track in the Spotify content catalog (and everywhere else I can find it) is
Couldn't Believe(with an apostrophe, not a RIGHT SINGLE QUOTATION MARK).I also didn't get any results. This is a bug, but not with spotipy. This is a bug with the Spotify web API.
Interestingly, this does return the track you're looking for. I can't explain why.
@jospkelly commented on GitHub (Sep 8, 2021):
@Peter-Schorn - Thanks for the response. I noticed that for the successful search, Spotipy does not create and send the same href message to the Spotify API – it does not the strings as field filters:
If I construct my command like you did (but instead using a Right Single Quotation Mark instead on an apostrophe), I also get the expected result:
If I omit the field filters and just pass the
artist,track, andalbumas strings in theq, either form of the contractionCouldn’twill fetch the expected result.@Earthscott commented on GitHub (Nov 14, 2021):
I agree there's some sort of bug on Spotify's end.
I ran a ~40,000 song dataset with song title and artist through sp.search() two nights ago. About 10,000 failed, a third of which have an apostrophe (my dataset doesn't have any right single quotation marks). Also worth noting, over 3,000 of my successful matches did have an apostrophe, so it's not a global problem.
This is my query string:
query = f"track:{song} artist:{artist}"I just did some testing with Don't Be Cruel by Elvis Presley, which fails. (using the right single quotation mark also fails). If I search for just Elvis Presley,
Don't Be Cruelshows up first as the 18th item (apostrophe). Interestingly, if I search for just Don't be cruel (either punctuation mark), I get back 20 results (all covers), 19 of which have the right single quotation mark and 1 of which has the left single quotation mark.Removing the quotation mark does work as noted by @jospkelly (I used Dont Be Cruel). This also works in testing with a few other songs. I'll rerun the failed items without the punctuation mark within the next few days and let y'all know how it goes.
@bghill commented on GitHub (Aug 15, 2022):
Punctuation encoding aside, in the answers listed here, many are ditching Spotify's query field system. It seems like
requestas library isn't a great match for how Spotify wants to do queries. The API docs seem to be clear about encoding of query field filters.Spotify wants a
:between a field and the value, a+between field/value pairs, and standard URL encoding for the song titles and artist names (e.g.%20for spaces). Spotify is trying to do its best when Spotipy queries come in mangled. If you add the:,requestencodes it. There doesn't seem to be a way to encode things yourself instead. If you put%20into the song titles with something likeurllib.parse.quote(), thenrequestwill just encode your encoding. So people are just dropping the field names and getting much messier results. Or they leave them in but I think Spotify is just adding another word to the overall search.When send the queries by hand via Spotify requests using Curl, I get tight, accurate results.
Anyone know how to get
requestto stop trying to be so smart on our behalf? Does it make sense to detect more elaborate queries and bypassrequest's encoding via one of the suggestions here?