[GH-ISSUE #68] On Windows we get GBK, not UTF8

kerem commented

2026-02-26 02:31:55 +03:00

Owner

Originally created by @baozijianke on GitHub (Dec 16, 2015).
Original GitHub issue: https://github.com/koel/koel/issues/68

I use a macbook and a thinkpad, so I tried Koel both on Mac OS and Windows. When I use my Windows platform I met a problem similar to @Merccy https://github.com/phanan/koel/issues/2.

I have a lot of Chinese songs and Korean songs. What I found is that after excuteing koel:sync I can's save the Chinese song's name into the database, and can't get the name of Korean songs.

the numbers and english characters in the name of Korean songs can be displayed correctly in the CMD, each Korean character is replaced with "?".
and it tells me that the name is valid, I can find nothing about these songs in the database.
for the Chinese songs, the CMD can display any chararcters (because the CMD is default characterized by GBK), but in the database I see that only the text before the Chinese character is saved.
eg: the real path is " C:/MusicBase/xxx一首纯中文歌曲.mp3 " , in the databse we can only see "C:/MusicBase/xxx" , if the path changes to " C:/歌曲库/xxx一首纯中文歌曲.mp3 ", we can only get "C:/" in the database.

I'm not familiar with PHP and Laravel or Symfony, I just read some of the codes and do some tests.
It seems that u use Symfony's component Finder to get the path of songs, so before the path being saved to database I add some codes to convert the path from GBK to UTF8, and the chinese songs' correct paths are shown in the database.

So I GUESS it's the problem of the default character setting. Windows use GBK. But GBK only supports most Chinese characters, Korean is not supported. So I can't get the Korean songs' path and I can get the Chinese song's path but can's save it to the database correctly.

I'm not skilled with English, if there is anything not clear, pls notice me. Thanks.

Originally created by @baozijianke on GitHub (Dec 16, 2015). Original GitHub issue: https://github.com/koel/koel/issues/68 I use a macbook and a thinkpad, so I tried Koel both on Mac OS and Windows. When I use my Windows platform I met a problem similar to @Merccy https://github.com/phanan/koel/issues/2. I have a lot of Chinese songs and Korean songs. What I found is that after excuteing koel:sync I can's save the Chinese song's name into the database, and can't get the name of Korean songs. 1. the numbers and english characters in the name of Korean songs can be displayed correctly in the CMD, each Korean character is replaced with "?". and it tells me that the name is valid, I can find nothing about these songs in the database. 2. for the Chinese songs, the CMD can display any chararcters (because the CMD is default characterized by GBK), but in the database I see that only the text before the Chinese character is saved. eg: the real path is " C:/MusicBase/xxx一首纯中文歌曲.mp3 " , in the databse we can only see "C:/MusicBase/xxx" , if the path changes to " C:/歌曲库/xxx一首纯中文歌曲.mp3 ", we can only get "C:/" in the database. I'm not familiar with PHP and Laravel or Symfony, I just read some of the codes and do some tests. It seems that u use Symfony's component Finder to get the path of songs, so before the path being saved to database I add some codes to convert the path from GBK to UTF8, and the chinese songs' correct paths are shown in the database. So I GUESS it's the problem of the default character setting. Windows use GBK. But GBK only supports most Chinese characters, Korean is not supported. So I can't get the Korean songs' path and I can get the Chinese song's path but can's save it to the database correctly. I'm not skilled with English, if there is anything not clear, pls notice me. Thanks.

kerem closed this issue

2026-02-26 02:31:55 +03:00

kerem commented

2026-02-26 02:31:55 +03:00

Author

Owner

@phanan commented on GitHub (Dec 16, 2015):

This is to my knowledge a limitation of PHP 5.x on Windows. Can you try upgrading to PHP 7 as a workaround, and see if it works out?

@phanan commented on GitHub (Dec 16, 2015): This is to my knowledge a limitation of PHP 5.x on Windows. Can you try upgrading to PHP 7 as a workaround, and see if it works out?

kerem commented

2026-02-26 02:31:55 +03:00

Author

Owner

@baozijianke commented on GitHub (Dec 16, 2015):

Oh I'm so sorry I didn't attach my system information.
Yes, I followed your wiki to install Koel, so I'm using PHP7.0

Windows10 Edu
Apache/2.4.18 (Win32)
Laravel 5.1
PHP/7.0.0
5.7.10-log - MySQL Community Server (GPL)

@baozijianke commented on GitHub (Dec 16, 2015): Oh I'm so sorry I didn't attach my system information. Yes, I followed your wiki to install Koel, so I'm using PHP7.0 Windows10 Edu Apache/2.4.18 (Win32) Laravel 5.1 PHP/7.0.0 5.7.10-log - MySQL Community Server (GPL)

kerem commented

2026-02-26 02:31:55 +03:00

Author

Owner

@phanan commented on GitHub (Dec 16, 2015):

Hmm. You mentioned to have added some code to convert from GBK to UTF8, can you share with me the code?

@phanan commented on GitHub (Dec 16, 2015): Hmm. You mentioned to have added some code to convert from GBK to UTF8, can you share with me the code?

kerem commented

2026-02-26 02:31:55 +03:00

Author

Owner

@baozijianke commented on GitHub (Dec 16, 2015):

OK.
In fact, in my test I use iconv("GBK","UTF-8//IGNORE",$string) .
I just searched this in Google and had a try.

@baozijianke commented on GitHub (Dec 16, 2015): OK. In fact, in my test I use iconv("GBK","UTF-8//IGNORE",$string) . I just searched this in Google and had a try.

kerem commented

2026-02-26 02:31:55 +03:00

Author

Owner

@baozijianke commented on GitHub (Dec 16, 2015):

http://www.php.net/manual/en/function.iconv.php
here is the document of iconv.

@baozijianke commented on GitHub (Dec 16, 2015): http://www.php.net/manual/en/function.iconv.php here is the document of iconv.

kerem commented

2026-02-26 02:31:55 +03:00

Author

Owner

@phanan commented on GitHub (Dec 16, 2015):

Strange. I've just tested in a Windows VM, the function works fine for me:

$file = new SplFileInfo("C:\Users\an\Desktop\中文好歌曲.mp3");
var_dump($file->getPathname()); // string(39) "C:\Users\an\Desktop\中文好歌曲.mp3"

@phanan commented on GitHub (Dec 16, 2015): Strange. I've just tested in a Windows VM, the function works fine for me: ``` php $file = new SplFileInfo("C:\Users\an\Desktop\中文好歌曲.mp3"); var_dump($file->getPathname()); // string(39) "C:\Users\an\Desktop\中文好歌曲.mp3" ```

kerem commented

2026-02-26 02:31:55 +03:00

Author

Owner

@baozijianke commented on GitHub (Dec 16, 2015):

em......My test is not the same as yours.
maybe it's the Symfony/Finder 's problem? I'm not sure.
I add 3 songs, one english song, one korean song ,one chinese song.

require('vendor/autoload.php');
use Symfony\Component\Finder\Finder;

$finder = new Finder();
$finder->files()->in("D:\MusicBase");

foreach ($finder as $file) {
    // Dump the absolute path
    var_dump(iconv('GBK', 'UTF-8', $file->getRealpath())); // convert
    var_dump($file->getRealpath()); //not convert
}

Below is what I see in chrome. chrome is using UTF-8 by default.

Forgive my ugly writings.

@baozijianke commented on GitHub (Dec 16, 2015): em......My test is not the same as yours. maybe it's the Symfony/Finder 's problem? I'm not sure. I add 3 songs, one english song, one korean song ,one chinese song. ![11](https://cloud.githubusercontent.com/assets/1576324/11836054/fbbe5156-a412-11e5-9ccc-aef24d57f1a6.PNG) ``` php require('vendor/autoload.php'); use Symfony\Component\Finder\Finder; $finder = new Finder(); $finder->files()->in("D:\MusicBase"); foreach ($finder as $file) { // Dump the absolute path var_dump(iconv('GBK', 'UTF-8', $file->getRealpath())); // convert var_dump($file->getRealpath()); //not convert } ``` Below is what I see in chrome. chrome is using UTF-8 by default. ![12](https://cloud.githubusercontent.com/assets/1576324/11836055/fbc40088-a412-11e5-9a2b-78ecf4453212.PNG) Forgive my ugly writings.

kerem commented

2026-02-26 02:31:55 +03:00

Author

Owner

@phanan commented on GitHub (Dec 16, 2015):

Your sample doesn't work for me though. This appears to be a long-known issue with PHP – Unicode settings on Windows, which unfortunately I don't have enough time to look for a workaround now. I'd suggest you keep the current local fix if it works for you.

@phanan commented on GitHub (Dec 16, 2015): Your sample doesn't work for me though. This appears to be a long-known issue with PHP – Unicode settings on Windows, which unfortunately I don't have enough time to look for a workaround now. I'd suggest you keep the current local fix if it works for you.

kerem referenced this issue

2026-02-26 02:33:03 +03:00

[GH-ISSUE #544] 'Illegal mix of collations' when scanning a path #387

kerem referenced this issue

2026-02-26 02:34:37 +03:00

[GH-ISSUE #1666] Failed to load home screen data GUI error #912

kerem referenced this issue

2026-02-26 02:34:38 +03:00

[GH-ISSUE #1669] Unable to create smart playlist #914

kerem referenced this issue

2026-02-26 02:34:54 +03:00

[GH-ISSUE #1808] [Bug]: Does not work after docker compose restart #1002