[GH-ISSUE #582] Songs mistaken for Albums when .search() with language is not "en" #405

Closed
opened 2026-02-27 23:00:37 +03:00 by kerem · 6 comments
Owner

Originally created by @shibotto on GitHub (May 12, 2024).
Original GitHub issue: https://github.com/sigma67/ytmusicapi/issues/582

Describe the bug
When doing .search(), songs in the "Songs" category are mistaken for albums if language is not English. The problem started with version 1.7.1.

To Reproduce

(曲 = Songs)

from ytmusicapi import YTMusic

ytm = YTMusic(language="ja")

for x in ytm.search("beatles don't let me down"):
    if x["category"] == "曲":
        print(x)

with 1.7.0:

{'category': '曲', 'resultType': 'song', 'title': 'Don’t Let Me Down (First Rooftop Performance)', 'album': {'name': 'Let It Be (Super Deluxe)', 'id': 'MPREb_zBKX8qwlKte'}, 'inLibrary': False, 'feedbackTokens': {'add': None, 'remove': None}, 'videoId': 'CXlCLDP2Vr0', 'videoType': 'MUSIC_VIDEO_TYPE_ATV', 'duration': '3:30', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'duration_seconds': 210, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/0uSK3j19kosq8SmrnZZ_mlw3kL6ZWFcLRgt0cqhACJcA6cEfLgCscIllVfF-LjkuV3zhuYG6MSFih6PdMw=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/0uSK3j19kosq8SmrnZZ_mlw3kL6ZWFcLRgt0cqhACJcA6cEfLgCscIllVfF-LjkuV3zhuYG6MSFih6PdMw=w120-h120-l90-rj', 'width': 120, 'height': 120}]}
{'category': '曲', 'resultType': 'song', 'title': 'ドント・レット・ミー・ダウン (Remastered 2009)', 'album': {'name': 'Past Masters, Vols. 1 & 2 (Remastered)', 'id': 'MPREb_BcKRiCMN4nj'}, 'inLibrary': False, 'feedbackTokens': {'add': None, 'remove': None}, 'videoId': 'EyHKuyGsu4U', 'videoType': 'MUSIC_VIDEO_TYPE_ATV', 'duration': '3:36', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'duration_seconds': 216, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/8IP3yQUxIfSPUpDlkprDlHWaDxuddqYIRHsd9IGUu0BqoKrJn2vMCk--zhTQuokvszbIfnor23hoqNY=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/8IP3yQUxIfSPUpDlkprDlHWaDxuddqYIRHsd9IGUu0BqoKrJn2vMCk--zhTQuokvszbIfnor23hoqNY=w120-h120-l90-rj', 'width': 120, 'height': 120}]}
{'category': '曲', 'resultType': 'song', 'title': 'Don’t Let Me Down (2021 Mix)', 'album': {'name': 'The Beatles 1967 – 1970 (2023 Edition)', 'id': 'MPREb_eavRH79d25s'}, 'inLibrary': False, 'feedbackTokens': {'add': None, 'remove': None}, 'videoId': 'Xz1g63DlVAU', 'videoType': 'MUSIC_VIDEO_TYPE_ATV', 'duration': '3:35', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'duration_seconds': 215, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/MuYA5gq9UJYpV1w4VaWO_HSE-vqnAlMghcHoIC2Z44yESTkbSa7iTN3p7EVPrRa0UyeO8FqMxjTtStb6=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/MuYA5gq9UJYpV1w4VaWO_HSE-vqnAlMghcHoIC2Z44yESTkbSa7iTN3p7EVPrRa0UyeO8FqMxjTtStb6=w120-h120-l90-rj', 'width': 120, 'height': 120}]}

with 1.7.1:

{'category': '曲', 'resultType': 'album', 'title': 'Don’t Let Me Down (First Rooftop Performance)', 'type': 'ザ・ビートルズ', 'duration': '3:30', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'album': {'name': 'Let It Be (Super Deluxe)', 'id': 'MPREb_zBKX8qwlKte'}, 'duration_seconds': 210, 'browseId': None, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/0uSK3j19kosq8SmrnZZ_mlw3kL6ZWFcLRgt0cqhACJcA6cEfLgCscIllVfF-LjkuV3zhuYG6MSFih6PdMw=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/0uSK3j19kosq8SmrnZZ_mlw3kL6ZWFcLRgt0cqhACJcA6cEfLgCscIllVfF-LjkuV3zhuYG6MSFih6PdMw=w120-h120-l90-rj', 'width': 120, 'height': 120}]}
{'category': '曲', 'resultType': 'album', 'title': 'ドント・レット・ミー・ダウン (Remastered 2009)', 'type': 'ザ・ビートルズ', 'duration': '3:36', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'album': {'name': 'Past Masters, Vols. 1 & 2 (Remastered)', 'id': 'MPREb_BcKRiCMN4nj'}, 'duration_seconds': 216, 'browseId': None, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/8IP3yQUxIfSPUpDlkprDlHWaDxuddqYIRHsd9IGUu0BqoKrJn2vMCk--zhTQuokvszbIfnor23hoqNY=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/8IP3yQUxIfSPUpDlkprDlHWaDxuddqYIRHsd9IGUu0BqoKrJn2vMCk--zhTQuokvszbIfnor23hoqNY=w120-h120-l90-rj', 'width': 120, 'height': 120}]}
{'category': '曲', 'resultType': 'album', 'title': 'Don’t Let Me Down (2021 Mix)', 'type': 'ザ・ビートルズ', 'duration': '3:35', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'album': {'name': 'The Beatles 1967 – 1970 (2023 Edition)', 'id': 'MPREb_eavRH79d25s'}, 'duration_seconds': 215, 'browseId': None, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/MuYA5gq9UJYpV1w4VaWO_HSE-vqnAlMghcHoIC2Z44yESTkbSa7iTN3p7EVPrRa0UyeO8FqMxjTtStb6=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/MuYA5gq9UJYpV1w4VaWO_HSE-vqnAlMghcHoIC2Z44yESTkbSa7iTN3p7EVPrRa0UyeO8FqMxjTtStb6=w120-h120-l90-rj', 'width': 120, 'height': 120}]}
Originally created by @shibotto on GitHub (May 12, 2024). Original GitHub issue: https://github.com/sigma67/ytmusicapi/issues/582 **Describe the bug** When doing `.search()`, songs in the "Songs" category are mistaken for albums if language is not English. The problem started with version 1.7.1. **To Reproduce** (曲 = Songs) ```python from ytmusicapi import YTMusic ytm = YTMusic(language="ja") for x in ytm.search("beatles don't let me down"): if x["category"] == "曲": print(x) ``` with 1.7.0: ```python {'category': '曲', 'resultType': 'song', 'title': 'Don’t Let Me Down (First Rooftop Performance)', 'album': {'name': 'Let It Be (Super Deluxe)', 'id': 'MPREb_zBKX8qwlKte'}, 'inLibrary': False, 'feedbackTokens': {'add': None, 'remove': None}, 'videoId': 'CXlCLDP2Vr0', 'videoType': 'MUSIC_VIDEO_TYPE_ATV', 'duration': '3:30', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'duration_seconds': 210, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/0uSK3j19kosq8SmrnZZ_mlw3kL6ZWFcLRgt0cqhACJcA6cEfLgCscIllVfF-LjkuV3zhuYG6MSFih6PdMw=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/0uSK3j19kosq8SmrnZZ_mlw3kL6ZWFcLRgt0cqhACJcA6cEfLgCscIllVfF-LjkuV3zhuYG6MSFih6PdMw=w120-h120-l90-rj', 'width': 120, 'height': 120}]} {'category': '曲', 'resultType': 'song', 'title': 'ドント・レット・ミー・ダウン (Remastered 2009)', 'album': {'name': 'Past Masters, Vols. 1 & 2 (Remastered)', 'id': 'MPREb_BcKRiCMN4nj'}, 'inLibrary': False, 'feedbackTokens': {'add': None, 'remove': None}, 'videoId': 'EyHKuyGsu4U', 'videoType': 'MUSIC_VIDEO_TYPE_ATV', 'duration': '3:36', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'duration_seconds': 216, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/8IP3yQUxIfSPUpDlkprDlHWaDxuddqYIRHsd9IGUu0BqoKrJn2vMCk--zhTQuokvszbIfnor23hoqNY=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/8IP3yQUxIfSPUpDlkprDlHWaDxuddqYIRHsd9IGUu0BqoKrJn2vMCk--zhTQuokvszbIfnor23hoqNY=w120-h120-l90-rj', 'width': 120, 'height': 120}]} {'category': '曲', 'resultType': 'song', 'title': 'Don’t Let Me Down (2021 Mix)', 'album': {'name': 'The Beatles 1967 – 1970 (2023 Edition)', 'id': 'MPREb_eavRH79d25s'}, 'inLibrary': False, 'feedbackTokens': {'add': None, 'remove': None}, 'videoId': 'Xz1g63DlVAU', 'videoType': 'MUSIC_VIDEO_TYPE_ATV', 'duration': '3:35', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'duration_seconds': 215, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/MuYA5gq9UJYpV1w4VaWO_HSE-vqnAlMghcHoIC2Z44yESTkbSa7iTN3p7EVPrRa0UyeO8FqMxjTtStb6=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/MuYA5gq9UJYpV1w4VaWO_HSE-vqnAlMghcHoIC2Z44yESTkbSa7iTN3p7EVPrRa0UyeO8FqMxjTtStb6=w120-h120-l90-rj', 'width': 120, 'height': 120}]} ``` with 1.7.1: ```python {'category': '曲', 'resultType': 'album', 'title': 'Don’t Let Me Down (First Rooftop Performance)', 'type': 'ザ・ビートルズ', 'duration': '3:30', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'album': {'name': 'Let It Be (Super Deluxe)', 'id': 'MPREb_zBKX8qwlKte'}, 'duration_seconds': 210, 'browseId': None, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/0uSK3j19kosq8SmrnZZ_mlw3kL6ZWFcLRgt0cqhACJcA6cEfLgCscIllVfF-LjkuV3zhuYG6MSFih6PdMw=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/0uSK3j19kosq8SmrnZZ_mlw3kL6ZWFcLRgt0cqhACJcA6cEfLgCscIllVfF-LjkuV3zhuYG6MSFih6PdMw=w120-h120-l90-rj', 'width': 120, 'height': 120}]} {'category': '曲', 'resultType': 'album', 'title': 'ドント・レット・ミー・ダウン (Remastered 2009)', 'type': 'ザ・ビートルズ', 'duration': '3:36', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'album': {'name': 'Past Masters, Vols. 1 & 2 (Remastered)', 'id': 'MPREb_BcKRiCMN4nj'}, 'duration_seconds': 216, 'browseId': None, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/8IP3yQUxIfSPUpDlkprDlHWaDxuddqYIRHsd9IGUu0BqoKrJn2vMCk--zhTQuokvszbIfnor23hoqNY=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/8IP3yQUxIfSPUpDlkprDlHWaDxuddqYIRHsd9IGUu0BqoKrJn2vMCk--zhTQuokvszbIfnor23hoqNY=w120-h120-l90-rj', 'width': 120, 'height': 120}]} {'category': '曲', 'resultType': 'album', 'title': 'Don’t Let Me Down (2021 Mix)', 'type': 'ザ・ビートルズ', 'duration': '3:35', 'year': None, 'artists': [{'name': 'ザ・ビートルズ', 'id': 'UC2XdaAVUannpujzv32jcouQ'}], 'album': {'name': 'The Beatles 1967 – 1970 (2023 Edition)', 'id': 'MPREb_eavRH79d25s'}, 'duration_seconds': 215, 'browseId': None, 'isExplicit': False, 'thumbnails': [{'url': 'https://lh3.googleusercontent.com/MuYA5gq9UJYpV1w4VaWO_HSE-vqnAlMghcHoIC2Z44yESTkbSa7iTN3p7EVPrRa0UyeO8FqMxjTtStb6=w60-h60-l90-rj', 'width': 60, 'height': 60}, {'url': 'https://lh3.googleusercontent.com/MuYA5gq9UJYpV1w4VaWO_HSE-vqnAlMghcHoIC2Z44yESTkbSa7iTN3p7EVPrRa0UyeO8FqMxjTtStb6=w120-h120-l90-rj', 'width': 120, 'height': 120}]} ```
kerem 2026-02-27 23:00:37 +03:00
  • closed this issue
  • added the
    bug
    label
Author
Owner

@shibotto commented on GitHub (May 13, 2024):

I'm exploring this issue a bit because I have time. Before 614999c77d in parse_search_result() if an element had video_type it was assumed to be either a song or a video, else get_search_result_type() would be called to try and determine what we are dealing with.
Now this behavior has been (somewhat) reversed. The problem is that get_search_result_type() is unable to identify songs/videos and evaluates everything it doesn't recognize as an album, and since it's now an album it never gets a chance to become a song or a video further down the code.

I thought about reversing that part back, but I also noticed that now parse_search_result() is always called with some dirty value (category[:-1]) for result_type:
Schermata da 2024-05-13 15-35-30
due to changes in here:
github.com/sigma67/ytmusicapi@ae966bb96a/ytmusicapi/mixins/search.py (L220-L225)

<!-- gh-comment-id:2107679029 --> @shibotto commented on GitHub (May 13, 2024): I'm exploring this issue a bit because I have time. Before 614999c77d1c17acc5162a27bf98e1cf18cae73f in `parse_search_result()` if an element had `video_type` it was assumed to be either a song or a video, else `get_search_result_type()` would be called to try and determine what we are dealing with. Now this behavior has been (somewhat) reversed. The problem is that `get_search_result_type()` is unable to identify songs/videos and evaluates everything it doesn't recognize as an album, and since it's now an album it never gets a chance to become a song or a video further down the code. I thought about reversing that part back, but I also noticed that now `parse_search_result()` is always called with some dirty value (`category[:-1]`) for `result_type`: ![Schermata da 2024-05-13 15-35-30](https://github.com/sigma67/ytmusicapi/assets/3816409/4c430ffa-d281-45b6-b940-257d7b3076ab) due to changes in here: https://github.com/sigma67/ytmusicapi/blob/ae966bb96ae018e31f5d5e785d77d9514152a069/ytmusicapi/mixins/search.py#L220-L225
Author
Owner

@sigma67 commented on GitHub (May 13, 2024):

Thanks!

Sorry the code is so obscure, it's just hard to handle all the different cases. I've been trying to add more tests and make it more readable. I think I got a better grasp on some of the edge cases now, please check #583

<!-- gh-comment-id:2108723187 --> @sigma67 commented on GitHub (May 13, 2024): Thanks! Sorry the code is so obscure, it's just hard to handle all the different cases. I've been trying to add more tests and make it more readable. I think I got a better grasp on some of the edge cases now, please check #583
Author
Owner

@shibotto commented on GitHub (May 13, 2024):

I'm sorry to inform you that now English is broken as well :D

The original behavior, as I understood it, was to [:-1].lower() the category as a first step, which works well for English (Songs -> song, Albums -> album, etc). Then if nothing matched, which in most cases meant a language different than English, try to guess from the available data. Now this doesn't happen as category is never taken into account before calling parse_search_results(), and honestly I think it's a good thing as at least everything fails equally.

The fundamental problem however remains, maybe the last part I added later overshadowed it a little: get_search_result_type() is the first one to be fed, but get_item_text() feeds it the wrong data. As an example:
Schermata da 2024-05-14 00-31-19
And as a consequence the next step, result_type = 'album'. I took just a quick look, but nowhere inside the data structure of a song element I saw something that could actually be used by parse_search_results().
We have MUSIC_VIDEO_TYPE_ATV though, which as far as I can observe always identifies songs.

Oh, don't worry about the code, I once tried to do something similar for generic Youtube. After few months I trashed everything because the many edge cases and my poor skill made the thing incomprehensible. Kudos for hanging on for so many years.

<!-- gh-comment-id:2109002917 --> @shibotto commented on GitHub (May 13, 2024): I'm sorry to inform you that now English is broken as well :D The original behavior, as I understood it, was to `[:-1].lower()` the category as a first step, which works well for English (Songs -> song, Albums -> album, etc). Then if nothing matched, which in most cases meant a language different than English, try to guess from the available data. Now this doesn't happen as `category` is never taken into account before calling `parse_search_results()`, and honestly I think it's a good thing as at least everything fails equally. The fundamental problem however remains, maybe the last part I added later overshadowed it a little: `get_search_result_type()` is the first one to be fed, but `get_item_text()` feeds it the wrong data. As an example: ![Schermata da 2024-05-14 00-31-19](https://github.com/sigma67/ytmusicapi/assets/3816409/f407942c-f1a1-48fa-b015-287f0cd5faa1) And as a consequence the next step, `result_type = 'album'`. I took just a quick look, but nowhere inside the data structure of a song element I saw something that could actually be used by `parse_search_results()`. We have MUSIC_VIDEO_TYPE_ATV though, which as far as I can observe always identifies songs. Oh, don't worry about the code, I once tried to do something similar for generic Youtube. After few months I trashed everything because the many edge cases and my poor skill made the thing incomprehensible. Kudos for hanging on for so many years.
Author
Owner

@sigma67 commented on GitHub (May 15, 2024):

Thanks for your diligence.

The heuristic of using the first sub-run as an indicator is inherently broken because it's so inconsistent server-side. Depending on where a Song appears the Song indicator is sometimes there and sometimes not - it's just guesswork that's bound to change at their whim.

The problem was that that bad heuristic was actually covering up the better one that came after.

So I removed that part entirely and am only using the browseId (and MUSIC_VIDEO_TYPE_ATV) now, which seems to work pretty well. It uses the mapping in your screenshot (from the FAQ and my experience basically).

<!-- gh-comment-id:2113354605 --> @sigma67 commented on GitHub (May 15, 2024): Thanks for your diligence. The heuristic of using the first sub-run as an indicator is inherently broken because it's so inconsistent server-side. Depending on where a Song appears the Song indicator is sometimes there and sometimes not - it's just guesswork that's bound to change at their whim. The problem was that that bad heuristic was actually covering up the better one that came after. So I removed that part entirely and am only using the browseId (and MUSIC_VIDEO_TYPE_ATV) now, which seems to work pretty well. It uses the mapping in your screenshot (from the FAQ and my experience basically).
Author
Owner

@sigma67 commented on GitHub (May 15, 2024):

Please check again if the results are good now. There might be some browseId edge cases I've missed.

<!-- gh-comment-id:2113356641 --> @sigma67 commented on GitHub (May 15, 2024): Please check again if the results are good now. There might be some `browseId` edge cases I've missed.
Author
Owner

@shibotto commented on GitHub (May 16, 2024):

I tried trowing a bunch of different queries at it, observing each element returned and yep, everything matches perfectly now! If I was simply lucky enough to miss any obscure browseId I'll get back to you.
Thank you very much! 😀

<!-- gh-comment-id:2115977585 --> @shibotto commented on GitHub (May 16, 2024): I tried trowing a bunch of different queries at it, observing each element returned and yep, everything matches perfectly now! If I was simply lucky enough to miss any obscure `browseId` I'll get back to you. Thank you very much! 😀
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/ytmusicapi#405
No description provided.