mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-26 05:45:57 +03:00
[GH-ISSUE #654] Segfault in libstdc++.so #372
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#372
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @adrianh-cogapp on GitHub (Oct 4, 2017).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/654
Version of s3fs being used (s3fs --version)
Version of fuse being used (pkg-config --modversion fuse)
System information (uname -a)
Distro (cat /etc/issue)
s3fs command line used (if applicable)
The bucket is mounted using fstab but here is the output if I try to mount it manually:
s3fs: unable to access MOUNTPOINT /home/ubuntu/s3mount: Transport endpoint is not connected/etc/fstab entry (if applicable):
eap-images-converted-jp2 /home/ubuntu/s3mount fuse.s3fs _netdev,allow_other,umask=000,uid=1000,gid=1000,use_cache=/home/ubuntu/cache 0 0Details about issue
The machine was running for a few days without any problems till today when it lost the mount point and couldn't/won't remount it.
In Syslogs:
kernel: [82848.442271] s3fs[2149]: segfault at 10 ip 00007f1884a43c6a sp 00007f18761cd9d8 error 4 in libstdc++.so.6.0.21[7f188499f000+172000]Then when I try to remount the bucket I get:
Unable to access MOUNTPOINT /home/ubuntu/s3mount: Transport endpoint is not connectedsudo mount -agives the same output as the s3fs command)How can I be sure that this segfault won't happen again?
I've kept the machine running if you need anymore information.
Thanks
@gaul commented on GitHub (Oct 4, 2017):
Do you have a core file from this crash? If so please inspect it with
gdband provide the backtrace viabt.@adrianh-cogapp commented on GitHub (Oct 4, 2017):
Sadly the core file was not generated.
If there anything else I can provide you with?
@adrianh-cogapp commented on GitHub (Oct 5, 2017):
It has happened again today. Except this time I can't find any log message about a segfault error.
Symptoms are the same:
s3fs: unable to access MOUNTPOINT /home/ubuntu/s3mount: Transport endpoint is not connectedsudo umount /home/ubuntu/s3mount) before been able to successfully mount (sudo mount -a)Could a heavy load cause this?
@gaul commented on GitHub (Oct 5, 2017):
Please enable core files on your system and provide the backtrace. You can also try enabling debug logging via
s3fs -d -d -f -o f2but the core file is likely required to make progress.@ggtakec commented on GitHub (Oct 8, 2017):
@adrianh-cogapp
The reason for becoming "Unable to access MOUNTPOINT /home/ubuntu/s3mount: Transport endpoint is not connected" is probably because s3fs is not doing postprocessing by crush.
You can execute the following command before remounting.
I think that you can remount after this.
The reason for the crash, I want debug output as @gaul says.
Thanks in advance for your assistance.
@adrianh-cogapp commented on GitHub (Oct 9, 2017):
@gaul and @ggtakec thanks.
Remounting is not a problem because if the machine looses the endpoint my AWS setup gets rid of the machine and creates a new one. So I am more interested in stopping this from happening compared to fixing the "broken" machine.
I have enabled core dumps. Waiting for it to happen again and I'll update this issue asap.
@adrianh-cogapp commented on GitHub (Oct 23, 2017):
@gaul and @ggtakec
Sorry it has been a while.
This happened again on the 20th.
Oct 20 08:23:35 ip-172-31-4-80 kernel: [938535.268434] s3fs[2232]: segfault at 10 ip 00007f288a5d2c6a sp 00007f2857ffe9d8 error 4 in libstdc++.so.6.0.21[7f288a52e000+172000]This time I got a crash file. I've unpacked it and extracted a backtrace using gdb:
Let me know if you need anything else.
Thanks
@gaul commented on GitHub (Oct 23, 2017):
@ggtakec This looks like the map got corrupted, perhaps due to lack of locking? I notice
FdEntityprotects some of its members withfdent_lockbut notorgmeta.@gaul commented on GitHub (Oct 24, 2017):
@adrianh-cogapp Please test #663. This is a shallow fix and we should audit
FdEntitylock usage.@adrianh-cogapp commented on GitHub (Oct 24, 2017):
@gaul I've provisioned a new machine with this update. I'll let you know the outcome asap.
Thanks
@adrianh-cogapp commented on GitHub (Oct 31, 2017):
@gaul @ggtakec Quick update: this has now been running for 7 days without loosing the S3FS endpoint.
@ggtakec commented on GitHub (Nov 5, 2017):
@adrianh-cogapp @gaul I'm very sorry for my late reply.
I checked @gaul patches, that fix was correct.
I merged #663, and please use the merged code.
I'll close this issue, but if you have a problem about this, please reopen this.
Regards,