starred/git-ai

Fork 0

mirror of https://github.com/git-ai-project/git-ai.git synced 2026-04-25 06:15:51 +03:00

[GH-ISSUE #426] BUG Reprot: Git Commit Fails with Non-UTF-8 Files (e.g., GBK) #159

New issue

Closed

opened 2026-03-02 04:12:23 +03:00 by kerem · 6 comments

kerem commented

2026-03-02 04:12:23 +03:00

Owner

Originally created by @ShuboLiang on GitHub (Jan 30, 2026).
Original GitHub issue: https://github.com/git-ai-project/git-ai/issues/426

Originally assigned to: @svarlamov on GitHub.

If the code file is in a format other than UTF-8 (such as GBK—briefly, GBK is a character encoding for simplified Chinese), executing git commit will result in an error.

Originally created by @ShuboLiang on GitHub (Jan 30, 2026). Original GitHub issue: https://github.com/git-ai-project/git-ai/issues/426 Originally assigned to: @svarlamov on GitHub. If the code file is in a format other than UTF-8 (such as GBK—briefly, GBK is a character encoding for simplified Chinese), executing git commit will result in an error. <img width="881" height="786" alt="Image" src="https://github.com/user-attachments/assets/f84c444d-da0c-4c4e-b365-7f72648410f9" />

kerem

2026-03-02 04:12:23 +03:00

closed this issue
added the
bug

good first issue

help wanted
labels

kerem commented

2026-03-02 04:12:25 +03:00

Author

Owner

@ShuboLiang commented on GitHub (Feb 2, 2026):

@acunniffe Is this in progress?

@ShuboLiang commented on GitHub (Feb 2, 2026): @acunniffe Is this in progress?

kerem commented

2026-03-02 04:12:25 +03:00

Author

Owner

@acunniffe commented on GitHub (Feb 2, 2026):

Yes working on testing it x-platform

@acunniffe commented on GitHub (Feb 2, 2026): Yes working on testing it x-platform

kerem commented

2026-03-02 04:12:25 +03:00

Author

Owner

@svarlamov commented on GitHub (Feb 2, 2026):

Just catching up on this -- what's the expected behavior? I don't think Git, nor Git AI for that matter, can mix encodings rationally. File names for sure have to be UTF-8, and I'm not sure that Git treats GBK contents as text either (for the purposes of diffs, etc.).

I think we would have to detect non-UTF-8 and treat them as binary files (essentially bypassing Git AI). It would fix the error (this shouldn't error), but you will probably lose AI attribution for any non-UTF-8 file contents

@svarlamov commented on GitHub (Feb 2, 2026): Just catching up on this -- what's the expected behavior? I don't think Git, nor Git AI for that matter, can mix encodings rationally. File names for sure have to be UTF-8, and I'm not sure that Git treats GBK contents as text either (for the purposes of diffs, etc.). I think we would have to detect non-UTF-8 and treat them as binary files (essentially bypassing Git AI). It would fix the error (this shouldn't error), but you will probably lose AI attribution for any non-UTF-8 file contents

kerem commented

2026-03-02 04:12:26 +03:00

Author

Owner

@ShuboLiang commented on GitHub (Feb 2, 2026):

@svarlamov I hope that git-ai can properly track contribution history for code files encoded in GBK. Our company maintains many legacy projects that use GBK encoding. Git itself can handle these files as plain text without issues. However, when I use git-ai, it fails to work correctly during git commit. Specifically, after committing, the expected git-ai statistics UI (showing how much was written by humans vs. AI) doesn't appear.

@ShuboLiang commented on GitHub (Feb 2, 2026): @svarlamov I hope that git-ai can properly track contribution history for code files encoded in GBK. Our company maintains many legacy projects that use GBK encoding. Git itself can handle these files as plain text without issues. However, when I use git-ai, it fails to work correctly during git commit. Specifically, after committing, the expected git-ai statistics UI (showing how much was written by humans vs. AI) doesn't appear.

kerem commented

2026-03-02 04:12:26 +03:00

Author

Owner

@svarlamov commented on GitHub (Feb 2, 2026):

I see, we'd like to get support for this then! Can you share the settings you use with Git to support GBK and how it handles mixed encoding within a repo? I'd also like to confirm that the filenames are UTF-8 (I was under the impression that UTF-8 is a hard requirement for file names).

We use the byte type internally for most of our logic, so technically this should be possible. I don't think Rust Strings are GBK-compatible though, so this will be interesting to implement/might require a bit of refactoring in areas where we read files into String before converting to byte for processing

@svarlamov commented on GitHub (Feb 2, 2026): I see, we'd like to get support for this then! Can you share the settings you use with Git to support GBK and how it handles mixed encoding within a repo? I'd also like to confirm that the filenames are UTF-8 (I was under the impression that UTF-8 is a hard requirement for file names). We use the `byte` type internally for most of our logic, so technically this should be possible. I don't think Rust Strings are GBK-compatible though, so this will be interesting to implement/might require a bit of refactoring in areas where we read files into String before converting to `byte` for processing

kerem commented

2026-03-02 04:12:26 +03:00

Author

Owner

@ShuboLiang commented on GitHub (Feb 2, 2026):

Git works without any additional configuration. All our filenames are in English, so they are UTF-8 encoded. Thank you for your help! Since git-ai will be heavily used by our users, we’ll likely have many more related questions for you in the future. Thanks again!

@ShuboLiang commented on GitHub (Feb 2, 2026): Git works without any additional configuration. All our filenames are in English, so they are UTF-8 encoded. Thank you for your help! Since git-ai will be heavily used by our users, we’ll likely have many more related questions for you in the future. Thanks again!

kerem referenced this issue

2026-03-02 04:13:13 +03:00

[PR #159] fix history for single commit ranges #294