starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #839] git clone premature end of pack file #487

New issue
Closed
opened 2026-03-04 01:46:02 +03:00 by kerem · 10 comments
kerem commented 2026-03-04 01:46:02 +03:00
Owner
Copy link

Originally created by @quancore on GitHub (Oct 12, 2018).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/839

Version of s3fs being used: V1.84
Kernel information : 4.15.0-1023-aws
GNU/Linux Distribution:
image
/etc/fstab:
s3fs#csgo-ranking-prediction /home/ubuntu/s3 fuse _netdev,exec,allow_other,passwd_file=/etc/passwd-s3fs,cipher_suites=AESGCM,kernel_cache,max_background=1000,max_stat_cache_size=100000,multipart_size=52,parallel_count=30,multireq_max=30,umask=0002,uid=1000,gid=1000,dbglevel=warn 0 0
system logs:
log.txt

error details:
image

Originally created by @quancore on GitHub (Oct 12, 2018). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/839 **Version of s3fs being used:** V1.84 **Kernel information :** 4.15.0-1023-aws **GNU/Linux Distribution:** ![image](https://user-images.githubusercontent.com/15036825/46879261-d36cfa80-ce45-11e8-911d-547d1c7df098.png) **/etc/fstab:** `s3fs#csgo-ranking-prediction /home/ubuntu/s3 fuse _netdev,exec,allow_other,passwd_file=/etc/passwd-s3fs,cipher_suites=AESGCM,kernel_cache,max_background=1000,max_stat_cache_size=100000,multipart_size=52,parallel_count=30,multireq_max=30,umask=0002,uid=1000,gid=1000,dbglevel=warn 0 0` **system logs:** [log.txt](https://github.com/s3fs-fuse/s3fs-fuse/files/2473790/log.txt) **error details:** ![image](https://user-images.githubusercontent.com/15036825/46880605-6a878180-ce49-11e8-9eb2-8e111d253b18.png)
kerem closed this issue 2026-03-04 01:46:03 +03:00
kerem commented 2026-03-04 01:46:04 +03:00
Author
Owner
Copy link

@cortesea commented on GitHub (Nov 5, 2018):

Hi,
I have the same issue.
Have you succeeded in solving it?

<!-- gh-comment-id:435867033 --> @cortesea commented on GitHub (Nov 5, 2018): Hi, I have the same issue. Have you succeeded in solving it?
kerem commented 2026-03-04 01:46:04 +03:00
Author
Owner
Copy link

@quancore commented on GitHub (Nov 7, 2018):

I have switched to goofys and decided to handle all kind of git operations on my local

<!-- gh-comment-id:436555562 --> @quancore commented on GitHub (Nov 7, 2018): I have switched to goofys and decided to handle all kind of git operations on my local
kerem commented 2026-03-04 01:46:04 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Nov 11, 2018):

@quancore
I can not reproduce this problem.
I looked for similar problems, and they seemed to be occurring when repository capacity is large, git command is old, etc.
We would like to confirm the following for this Issue solution.
Is this problem also occurring in other repositories?
Is the git command occurred even at the latest?

<!-- gh-comment-id:437643715 --> @ggtakec commented on GitHub (Nov 11, 2018): @quancore I can not reproduce this problem. I looked for similar problems, and they seemed to be occurring when repository capacity is large, git command is old, etc. We would like to confirm the following for this Issue solution. Is this problem also occurring in other repositories? Is the git command occurred even at the latest?
kerem commented 2026-03-04 01:46:04 +03:00
Author
Owner
Copy link

@tkang007 commented on GitHub (Dec 5, 2018):

Hi,
It still happened, please reference below.

Error message when clone to s3 mounted directory:

Cloning into '/public/reference/jupyter-tutorial'...
remote: Enumerating objects: 142, done.
remote: Total 142 (delta 0), reused 0 (delta 0), pack-reused 142
Receiving objects: 100% (142/142), 223.22 KiB | 38.00 KiB/s, done.
fatal: premature end of pack file, 63 bytes missing
fatal: index-pack failed

[root@ip-10-94-89-236 ]# cat /etc/fstab
#
UUID=e9c3e7c2-e039-4b2b-9991-955a8c7cd8c0     /           xfs    defaults,noatime  1   1
/dev/xvdb  swap  swap defaults 0 0
/dev/xvdc /dat1  ext4  defaults 0 2
s3fs#jupyterhub-dev /s3f1 fuse _netdev,allow_other,iam_role=auto,use_cache=/tmp 0 0

[root@ip-10-94-89-236 ]# s3fs --version
Amazon Simple Storage Service File System V1.84(commit:unknown) with OpenSSL
Copyright (C) 2010 Randy Rizun <rrizun@gmail.com>
License GPL2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

[root@ip-10-94-89-236 ]# git --version
git version 2.14.5

[root@ip-10-94-89-236 ]# cat /etc/os-release
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"

[root@ip-10-94-89-236 ]# cat /etc/system-release
Amazon Linux release 2 (Karoo)
<!-- gh-comment-id:444677682 --> @tkang007 commented on GitHub (Dec 5, 2018): Hi, It still happened, please reference below. Error message when clone to s3 mounted directory: ``` Cloning into '/public/reference/jupyter-tutorial'... remote: Enumerating objects: 142, done. remote: Total 142 (delta 0), reused 0 (delta 0), pack-reused 142 Receiving objects: 100% (142/142), 223.22 KiB | 38.00 KiB/s, done. fatal: premature end of pack file, 63 bytes missing fatal: index-pack failed [root@ip-10-94-89-236 ]# cat /etc/fstab # UUID=e9c3e7c2-e039-4b2b-9991-955a8c7cd8c0 / xfs defaults,noatime 1 1 /dev/xvdb swap swap defaults 0 0 /dev/xvdc /dat1 ext4 defaults 0 2 s3fs#jupyterhub-dev /s3f1 fuse _netdev,allow_other,iam_role=auto,use_cache=/tmp 0 0 [root@ip-10-94-89-236 ]# s3fs --version Amazon Simple Storage Service File System V1.84(commit:unknown) with OpenSSL Copyright (C) 2010 Randy Rizun <rrizun@gmail.com> License GPL2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. [root@ip-10-94-89-236 ]# git --version git version 2.14.5 [root@ip-10-94-89-236 ]# cat /etc/os-release NAME="Amazon Linux" VERSION="2" ID="amzn" ID_LIKE="centos rhel fedora" VERSION_ID="2" PRETTY_NAME="Amazon Linux 2" ANSI_COLOR="0;33" CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2" HOME_URL="https://amazonlinux.com/" [root@ip-10-94-89-236 ]# cat /etc/system-release Amazon Linux release 2 (Karoo) ```
kerem commented 2026-03-04 01:46:04 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Jan 24, 2019):

I tested this again against master including 5f5da4b2cb but still experience these symptoms:

$ git clone https://github.com/s3fs-fuse/s3fs-fuse.git
Cloning into 's3fs-fuse'...
remote: Enumerating objects: 36, done.
remote: Counting objects: 100% (36/36), done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 4534 (delta 16), reused 20 (delta 10), pack-reused 4498
Receiving objects: 100% (4534/4534), 2.57 MiB | 4.00 KiB/s, done.
error: inflate: data stream error (unknown compression method)
error: inflate: data stream error (unknown compression method)
fatal: serious inflate inconsistency
fatal: index-pack failed
<!-- gh-comment-id:457028470 --> @gaul commented on GitHub (Jan 24, 2019): I tested this again against master including 5f5da4b2cbf2753024ff58b4b3a75b686879a7c5 but still experience these symptoms: ``` $ git clone https://github.com/s3fs-fuse/s3fs-fuse.git Cloning into 's3fs-fuse'... remote: Enumerating objects: 36, done. remote: Counting objects: 100% (36/36), done. remote: Compressing objects: 100% (26/26), done. remote: Total 4534 (delta 16), reused 20 (delta 10), pack-reused 4498 Receiving objects: 100% (4534/4534), 2.57 MiB | 4.00 KiB/s, done. error: inflate: data stream error (unknown compression method) error: inflate: data stream error (unknown compression method) fatal: serious inflate inconsistency fatal: index-pack failed ```
kerem commented 2026-03-04 01:46:04 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Mar 15, 2019):

Problem persists with 1.85 but with slightly different symptoms:

$ git clone https://github.com/s3fs-fuse/s3fs-fuse.git
Cloning into 's3fs-fuse'...
remote: Enumerating objects: 55, done.
remote: Counting objects: 100% (55/55), done.
remote: Compressing objects: 100% (32/32), done.
remote: Total 4898 (delta 22), reused 41 (delta 22), pack-reused 4843
Receiving objects: 100% (4898/4898), 2.74 MiB | 26.00 KiB/s, done.
warning: die() called many times. Recursion error or racy threaded death!
fatal: premature end of pack file, 48 bytes missing
fatal: premature end of pack file, 48 bytes missing
fatal: index-pack failed
<!-- gh-comment-id:473195859 --> @gaul commented on GitHub (Mar 15, 2019): Problem persists with 1.85 but with slightly different symptoms: ``` $ git clone https://github.com/s3fs-fuse/s3fs-fuse.git Cloning into 's3fs-fuse'... remote: Enumerating objects: 55, done. remote: Counting objects: 100% (55/55), done. remote: Compressing objects: 100% (32/32), done. remote: Total 4898 (delta 22), reused 41 (delta 22), pack-reused 4843 Receiving objects: 100% (4898/4898), 2.74 MiB | 26.00 KiB/s, done. warning: die() called many times. Recursion error or racy threaded death! fatal: premature end of pack file, 48 bytes missing fatal: premature end of pack file, 48 bytes missing fatal: index-pack failed ```
kerem commented 2026-03-04 01:46:04 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (May 4, 2019):

I made some progress on diagnosing this issue via mirrorfs. s3fs unexpectedly truncates a temporary file:

$ ls -l blobstore/s3proxy-bucket/java-httpbin/.git/objects/pack/tmp_pack_Zzimzc
-rw-rw----. 1 gaul gaul 0 May  2 18:50 blobstore/s3proxy-bucket/java-httpbin/.git/objects/pack/tmp_pack_Zzimzc
<!-- gh-comment-id:489290244 --> @gaul commented on GitHub (May 4, 2019): I made some progress on diagnosing this issue via [mirrorfs](https://github.com/gaul/mirrorfs). s3fs unexpectedly truncates a temporary file: ``` $ ls -l blobstore/s3proxy-bucket/java-httpbin/.git/objects/pack/tmp_pack_Zzimzc -rw-rw----. 1 gaul gaul 0 May 2 18:50 blobstore/s3proxy-bucket/java-httpbin/.git/objects/pack/tmp_pack_Zzimzc ```
kerem commented 2026-03-04 01:46:04 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (May 4, 2019):

I found that s3fs truncates unflushed open files when opening a second fd to them. The following example returns 4 with ext4 and 0 with s3fs:

(echo bar ; wc -c < foo >&2) > foo
<!-- gh-comment-id:489329125 --> @gaul commented on GitHub (May 4, 2019): I found that s3fs truncates unflushed open files when opening a second fd to them. The following example returns 4 with ext4 and 0 with s3fs: ``` (echo bar ; wc -c < foo >&2) > foo ```
kerem commented 2026-03-04 01:46:04 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (May 5, 2019):

@gaul I was able to reproduce this problem.

If I try on the local disk, it will look like

$ echo test > foo
$ (echo bar ; wc -c < foo >&2) > foo
  4
$ cat foo
  bar

The file system mounted with s3fs is as follows.

$ echo test > foo
$ (echo bar ; wc -c < foo >&2) > foo
  0
$ cat foo

I think that the cause is the problem of time lag until file upload, and tested it with the following command.

(echo bar ; sleep 10 ; wc -c < foo >&2) > foo

As a result, both got the same result.

$ echo test > foo
$ (echo bar ; sleep 10 ; wc -c < foo >&2) > foo
  4
$ cat foo
  bar

From this result, it seems that the following command is executed before the file output(upload) is completed.
The time to upload the object to S3 and complete will take much more time to complete writing to the local disk.
I think this is the reason.

<!-- gh-comment-id:489400721 --> @ggtakec commented on GitHub (May 5, 2019): @gaul I was able to reproduce this problem. If I try on the local disk, it will look like ``` $ echo test > foo $ (echo bar ; wc -c < foo >&2) > foo 4 $ cat foo bar ``` The file system mounted with s3fs is as follows. ``` $ echo test > foo $ (echo bar ; wc -c < foo >&2) > foo 0 $ cat foo ``` I think that the cause is the problem of time lag until file upload, and tested it with the following command. ``` (echo bar ; sleep 10 ; wc -c < foo >&2) > foo ``` As a result, both got the same result. ``` $ echo test > foo $ (echo bar ; sleep 10 ; wc -c < foo >&2) > foo 4 $ cat foo bar ``` From this result, it seems that the following command is executed before the file output(upload) is completed. The time to upload the object to S3 and complete will take much more time to complete writing to the local disk. I think this is the reason.
kerem commented 2026-03-04 01:46:04 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Jul 3, 2019):

I successfully cloned the s3fs-fuse repository into an s3fs mount. Fixed by c596441f58.

<!-- gh-comment-id:508287169 --> @gaul commented on GitHub (Jul 3, 2019): I successfully cloned the s3fs-fuse repository into an s3fs mount. Fixed by c596441f5872c5309908739de66397182229123c.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#487
No description provided.