mirror of
https://github.com/koel/koel.git
synced 2026-04-25 08:46:00 +03:00
[GH-ISSUE #77] Error importing songs #57
Labels
No labels
Authentication
Dependencies
Documentation
Feature Request
Flac
Help Wanted
Installation/Setup
Integration
Mobile
PR Welcome
Pending Release
Performance
Playlist
S3
Search
Sync
[Pri] Low
[Pri] Normal
[Status] Keep Open
[Status] Needs Author Reply
[Status] Needs Review
[Status] Stale
[Status] Will Implement
[Type] Blessed
[Type] Bug
[Type] Duplicate
[Type] Enhancement
[Type] Help Request
[Type] Question
[Type] Task
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/koel-koel#57
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @etopian on GitHub (Dec 16, 2015).
Original GitHub issue: https://github.com/koel/koel/issues/77
@etopian commented on GitHub (Dec 17, 2015):
Seems to be a problem with default artist being empty. Like the Laravel ORM is not correctly assigning ID 1 for artists that are empty. Instead it tries to create a new Artist with an empty name value, and that works for the first empty value, but it fails after that.
@funcoding commented on GitHub (Dec 17, 2015):
@etopian Any suggestions on how the error can be intentionally carried out?
@phanan commented on GitHub (Dec 17, 2015):
Can you try setting artist name to
preg_replace('/[^\p{L}\s]/u', '', $artist_name) and let me know if it works? See https://3v4l.org/Tj2M6. If it works for you, I'll introduce it into the code.@phanan commented on GitHub (Dec 17, 2015):
Ok, what's your table encoding?
@etopian commented on GitHub (Dec 17, 2015):
@phanan yes your fix works. pull request.
https://github.com/phanan/koel/pull/94
@phanan commented on GitHub (Dec 17, 2015):
@etopian commented on GitHub (Dec 17, 2015):
utf8...
@phanan commented on GitHub (Dec 17, 2015):
Ok.. anyway the fix above doesn't work for all cases, so I closed the PR. Sorry.
@etopian commented on GitHub (Dec 17, 2015):
i have a parser that i can use to fix all the album and artist names because I do think the tag is corrupt somehow. but in the wild you can receive any input. so it should be robust enough to handle this or any other input.
@phanan commented on GitHub (Dec 17, 2015):
Actually the code to save and check for unique record is 100% Laravel, so if this is a bug, it's their bug. I doubt it's a bug though.
@etopian commented on GitHub (Dec 17, 2015):
@phanan what library is used for parsing idv2 tags? where is the code for that. perhaps figure out what encoding it's reading the tags, and then use that information to make sure it comes out fine on the other side. ultimately the problem is with how the string is represented.
if i do a utf8_encode on the album and the artist strings everything comes out fine, like it parses everything. but i get weird characters that show up, so the initial encoding is not ISO-8859-1.
also might also be a problem with the way php is compiled in Alpine Linux. that could also be a problem.
@etopian commented on GitHub (Dec 17, 2015):
after some more investigation, if i utf-8 encode the strings... I get ÿþ in front of some of the titles. According to the article below that's the delimiter for UTF-16. So that's what's happening the string is not a UTF-8 string or a ISO string or an ascii string, it's a UTF-16 string. The only way to deal with this is to figure out ahead of time what type of string it is and then decode it properly before sending it to the database.
http://stackoverflow.com/questions/26493053/remove-%C3%BF%C3%BE-from-string
@etopian commented on GitHub (Dec 17, 2015):
A way to try to deal with this is to use this and probably use strict, but it's hit or miss here, the ideal way is to know ahead of time:
http://php.net/manual/en/function.mb-detect-encoding.php
@etopian commented on GitHub (Dec 17, 2015):
https://github.com/neitanod/forceutf8
this however does not support utf-16. it does seem however that utf-16 should be supported as some idv2 tags contain it.
@etopian commented on GitHub (Dec 17, 2015):
idv2 tags are marked with the encoding, https://en.wikipedia.org/wiki/ID3
the library which is reading the tag should be reading it. i can no longer afford to spend more time on this. hopefully enough information has been provided to find a solution to the problem.
$00 – ISO-8859-1 (LATIN-1, Identical to ASCII for values smaller than 0x80).
$01 – UCS-2 (UTF-16 encoded Unicode with BOM), in ID3v2.2 and ID3v2.3.
$02 – UTF-16BE encoded Unicode without BOM, in ID3v2.4.
$03 – UTF-8 encoded Unicode, in ID3v2.4.
@phanan commented on GitHub (Dec 17, 2015):
Thanks for the findings @etopian. I'll look into it.
@phanan commented on GitHub (Dec 17, 2015):
I've pushed an attempt to fix, please let me know if it works.
On Thu, Dec 17, 2015 at 10:15 PM, Etopian Inc. notifications@github.com
wrote:
@etopian commented on GitHub (Dec 17, 2015):
iconv fails to convert from UTF16L.. use mb_convert_encoding. also apply this to album names in case.
@etopian commented on GitHub (Dec 17, 2015):
Probably a good idea to apply it to all fields that you get out of getid3 in Media.php
@etopian commented on GitHub (Dec 17, 2015):
Not 100% sure but might be chopping off character on certain strings. It works, but not perfectly. It might be the case that the encoding detection is not 100%.
@phanan commented on GitHub (Dec 18, 2015):
This issue is only critical for artist, as it's required to be unique. I'd
leave the other tags alone to gain some speed.
On Fri, Dec 18, 2015 at 6:39 AM, Etopian Inc. notifications@github.com
wrote:
@etopian commented on GitHub (Dec 18, 2015):
same issue: https://github.com/phanan/koel/issues/113