[GH-ISSUE #430] After success of s3 bucket mount, fail to write file into mounted folder. #230

Closed
opened 2026-03-04 01:43:27 +03:00 by kerem · 7 comments
Owner

Originally created by @choruspapa on GitHub (Jun 7, 2016).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/430

I installed s3fs on ubuntu 14.
In foreground mode, s3fs mounts s3 bucket to some path.

sudo s3fs -f -d my-bucket -o url=http://s3-website.ap-northeast-2.amazonaws.com -o use_cache=/tmp -o allow_other mount path

    set_moutpoint_attribute(3291): PROC(uid=0, gid=0) - MountPoint(uid=1000, gid=1000, mode=40775)
s3fs_init(2595): init
s3fs_check_service(2894): check services.
    CheckBucket(2228): check a bucket.
    RequestPerform(1467): connecting to URL http://my-fs-objects.s3-website.ap-northeast-2.amazonaws.com/
    RequestPerform(1483): HTTP response code 200

It seems to be mounted successfully.
After that, I wrote a file into mounted folder.

echo "test" > ./mountpath/test.txt
-bash: ./mountpath/test.txt: Input/output error

I got error message.
And console output messages of s3fs are pasted below.

s3fs_getattr(691): [path=/test.txt]
    HeadRequest(1920): [tpath=/test.txt]
    RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt
    RequestPerform(1483): HTTP response code 404
    RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(1920): [tpath=/test.txt/]
    RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt/
    RequestPerform(1483): HTTP response code 404
    RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(1920): [tpath=/test.txt_$folder$]
    RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt_%24folder%24
    RequestPerform(1483): HTTP response code 404
    RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2225): [path=/test.txt]
    ListBucketRequest(2270): [tpath=/test.txt]
    RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com?delimiter=/&prefix=test.txt/&max-keys=1000
    RequestPerform(1483): HTTP response code 200
    get_base_exp(2469): marker_xp->nodesetval is empty.
    append_objects_from_xml_ex(2299): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2299): contents_xp->nodesetval is empty.
    get_base_exp(2469): marker_xp->nodesetval is empty.
s3fs_create(798): [path=/test.txt][mode=100664][flags=33345]
    HeadRequest(1920): [tpath=/test.txt]
    RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt
    RequestPerform(1483): HTTP response code 404
    RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(1920): [tpath=/test.txt/]
    RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt/
    RequestPerform(1483): HTTP response code 404
    RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(1920): [tpath=/test.txt_$folder$]
    RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt_%24folder%24
    RequestPerform(1483): HTTP response code 404
    RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2225): [path=/test.txt]
    ListBucketRequest(2270): [tpath=/test.txt]
    RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com?delimiter=/&prefix=test.txt/&max-keys=1000
    RequestPerform(1483): HTTP response code 200
    get_base_exp(2469): marker_xp->nodesetval is empty.
    append_objects_from_xml_ex(2299): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2299): contents_xp->nodesetval is empty.
    get_base_exp(2469): marker_xp->nodesetval is empty.
    create_file_object(757): [path=/test.txt][mode=100664]
    PutRequest(2042): [tpath=/test.txt]
    PutRequest(2056): create zero byte file object.
    PutRequest(2135): uploading... [path=/test.txt][fd=-1][size=0]
    RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt
    RequestPerform(1483): HTTP response code 405
    RequestPerform(1512): HTTP response code = 405, returning EIO
    DelStat(375): delete stat cache entry[path=/test.txt]

let me know how to fix it, plz.
thanks in advance.

Originally created by @choruspapa on GitHub (Jun 7, 2016). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/430 I installed s3fs on ubuntu 14. In foreground mode, s3fs mounts s3 bucket to some path. ``` sudo s3fs -f -d my-bucket -o url=http://s3-website.ap-northeast-2.amazonaws.com -o use_cache=/tmp -o allow_other mount path set_moutpoint_attribute(3291): PROC(uid=0, gid=0) - MountPoint(uid=1000, gid=1000, mode=40775) s3fs_init(2595): init s3fs_check_service(2894): check services. CheckBucket(2228): check a bucket. RequestPerform(1467): connecting to URL http://my-fs-objects.s3-website.ap-northeast-2.amazonaws.com/ RequestPerform(1483): HTTP response code 200 ``` It seems to be mounted successfully. After that, I wrote a file into mounted folder. ``` echo "test" > ./mountpath/test.txt -bash: ./mountpath/test.txt: Input/output error ``` I got error message. And console output messages of s3fs are pasted below. ``` s3fs_getattr(691): [path=/test.txt] HeadRequest(1920): [tpath=/test.txt] RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt RequestPerform(1483): HTTP response code 404 RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT HeadRequest(1920): [tpath=/test.txt/] RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt/ RequestPerform(1483): HTTP response code 404 RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT HeadRequest(1920): [tpath=/test.txt_$folder$] RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt_%24folder%24 RequestPerform(1483): HTTP response code 404 RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT list_bucket(2225): [path=/test.txt] ListBucketRequest(2270): [tpath=/test.txt] RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com?delimiter=/&prefix=test.txt/&max-keys=1000 RequestPerform(1483): HTTP response code 200 get_base_exp(2469): marker_xp->nodesetval is empty. append_objects_from_xml_ex(2299): contents_xp->nodesetval is empty. append_objects_from_xml_ex(2299): contents_xp->nodesetval is empty. get_base_exp(2469): marker_xp->nodesetval is empty. s3fs_create(798): [path=/test.txt][mode=100664][flags=33345] HeadRequest(1920): [tpath=/test.txt] RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt RequestPerform(1483): HTTP response code 404 RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT HeadRequest(1920): [tpath=/test.txt/] RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt/ RequestPerform(1483): HTTP response code 404 RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT HeadRequest(1920): [tpath=/test.txt_$folder$] RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt_%24folder%24 RequestPerform(1483): HTTP response code 404 RequestPerform(1507): HTTP response code 404 was returned, returning ENOENT list_bucket(2225): [path=/test.txt] ListBucketRequest(2270): [tpath=/test.txt] RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com?delimiter=/&prefix=test.txt/&max-keys=1000 RequestPerform(1483): HTTP response code 200 get_base_exp(2469): marker_xp->nodesetval is empty. append_objects_from_xml_ex(2299): contents_xp->nodesetval is empty. append_objects_from_xml_ex(2299): contents_xp->nodesetval is empty. get_base_exp(2469): marker_xp->nodesetval is empty. create_file_object(757): [path=/test.txt][mode=100664] PutRequest(2042): [tpath=/test.txt] PutRequest(2056): create zero byte file object. PutRequest(2135): uploading... [path=/test.txt][fd=-1][size=0] RequestPerform(1467): connecting to URL http://my-bucket.s3-website.ap-northeast-2.amazonaws.com/test.txt RequestPerform(1483): HTTP response code 405 RequestPerform(1512): HTTP response code = 405, returning EIO DelStat(375): delete stat cache entry[path=/test.txt] ``` let me know how to fix it, plz. thanks in advance.
kerem closed this issue 2026-03-04 01:43:28 +03:00
Author
Owner

@choruspapa commented on GitHub (Jun 8, 2016):

Additionally, attach more detail log messages.

> PUT /test.txt HTTP/1.1
Host: my-bucket.s3-website.ap-northeast-2.amazonaws.com
Accept: */*
Authorization: AWS AKIAII2252DICSSTJTZA:ICqR9JHe1mv3pJQMof+brfMye5Y=
Content-Type:text/plain
Date: Wed, 08 Jun 2016 05:17:19 GMT
x-amz-acl:private
x-amz-meta-gid:1000
x-amz-meta-mode:33204
x-amz-meta-mtime:1465363039
x-amz-meta-uid:1000
Content-Length: 0
Expect: 100-continue

< HTTP/1.1 405 Method Not Allowed
< x-amz-request-id: 9F70721EF30119B5
< x-amz-id-2: 9dgdcGeKtZxLN1rJWTmP9uWYDy623b9MSDoKzsJI7Exe7syx3dj1Mfct4SXRM2+COtJUx2O4d3c=
< Allow: GET, HEAD, OPTIONS
< Content-Type: text/html; charset=utf-8
< Content-Length: 421
< Date: Wed, 08 Jun 2016 05:17:19 GMT
< Connection: close
* Server AmazonS3 is not blacklisted
< Server: AmazonS3
<
* Closing connection 13
    RequestPerform(1483): HTTP response code 405
    RequestPerform(1512): HTTP response code = 405, returning EIO
    DelStat(375): delete stat cache entry[path=/test.txt]
   unique: 5, error: -5 (Input/output error), outsize: 16
<!-- gh-comment-id:224491438 --> @choruspapa commented on GitHub (Jun 8, 2016): Additionally, attach more detail log messages. ``` > PUT /test.txt HTTP/1.1 Host: my-bucket.s3-website.ap-northeast-2.amazonaws.com Accept: */* Authorization: AWS AKIAII2252DICSSTJTZA:ICqR9JHe1mv3pJQMof+brfMye5Y= Content-Type:text/plain Date: Wed, 08 Jun 2016 05:17:19 GMT x-amz-acl:private x-amz-meta-gid:1000 x-amz-meta-mode:33204 x-amz-meta-mtime:1465363039 x-amz-meta-uid:1000 Content-Length: 0 Expect: 100-continue < HTTP/1.1 405 Method Not Allowed < x-amz-request-id: 9F70721EF30119B5 < x-amz-id-2: 9dgdcGeKtZxLN1rJWTmP9uWYDy623b9MSDoKzsJI7Exe7syx3dj1Mfct4SXRM2+COtJUx2O4d3c= < Allow: GET, HEAD, OPTIONS < Content-Type: text/html; charset=utf-8 < Content-Length: 421 < Date: Wed, 08 Jun 2016 05:17:19 GMT < Connection: close * Server AmazonS3 is not blacklisted < Server: AmazonS3 < * Closing connection 13 RequestPerform(1483): HTTP response code 405 RequestPerform(1512): HTTP response code = 405, returning EIO DelStat(375): delete stat cache entry[path=/test.txt] unique: 5, error: -5 (Input/output error), outsize: 16 ```
Author
Owner

@sqlbot commented on GitHub (Jun 8, 2016):

-o url=http://s3- http://s3-website.ap-northeast-2.amazonaws.com/
website.ap http://s3-website.ap-northeast-2.amazonaws.com/-northeast-2.
http://s3-website.ap-northeast-2.amazonaws.com/amazonaws.com
http://s3-website.ap-northeast-2.amazonaws.com/ is not correct. This is
a web site endpoint, rather than a REST endpoint, which only supports GET
requests, so 405 Method Not Allowed is actually the expected response.
Remove "s3-website." from the beginning.
On Jun 8, 2016 1:27 AM, "Yongho Hwang" notifications@github.com wrote:

Additionally, attach more detail log messages.

PUT /test.txt HTTP/1.1
Host: my-fs-objects.s3-website.ap-northeast-2.amazonaws.com
Accept: /
Authorization: AWS AKIAII2252DICSSTJTZA:ICqR9JHe1mv3pJQMof+brfMye5Y=
Content-Type:text/plain
Date: Wed, 08 Jun 2016 05:17:19 GMT
x-amz-acl:private
x-amz-meta-gid:1000
x-amz-meta-mode:33204
x-amz-meta-mtime:1465363039
x-amz-meta-uid:1000
Content-Length: 0
Expect: 100-continue

< HTTP/1.1 405 Method Not Allowed
< x-amz-request-id: 9F70721EF30119B5
< x-amz-id-2: 9dgdcGeKtZxLN1rJWTmP9uWYDy623b9MSDoKzsJI7Exe7syx3dj1Mfct4SXRM2+COtJUx2O4d3c=
< Allow: GET, HEAD, OPTIONS
< Content-Type: text/html; charset=utf-8
< Content-Length: 421
< Date: Wed, 08 Jun 2016 05:17:19 GMT
< Connection: close

  • Server AmazonS3 is not blacklisted
    < Server: AmazonS3
    <
  • Closing connection 13
    RequestPerform(1483): HTTP response code 405
    RequestPerform(1512): HTTP response code = 405, returning EIO
    DelStat(375): delete stat cache entry[path=/test.txt]
    unique: 5, error: -5 (Input/output error), outsize: 16


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/s3fs-fuse/s3fs-fuse/issues/430#issuecomment-224491438,
or mute the thread
https://github.com/notifications/unsubscribe/ACUxPExyuIDCV5wT1iSMcb-ZbM3hb3KDks5qJlLCgaJpZM4IvoY5
.

<!-- gh-comment-id:224563618 --> @sqlbot commented on GitHub (Jun 8, 2016): -o url=http://s3- http://s3-website.ap-northeast-2.amazonaws.com/ website.ap http://s3-website.ap-northeast-2.amazonaws.com/-northeast-2. http://s3-website.ap-northeast-2.amazonaws.com/amazonaws.com http://s3-website.ap-northeast-2.amazonaws.com/ is not correct. This is a web site endpoint, rather than a REST endpoint, which only supports GET requests, so 405 Method Not Allowed is actually the expected response. Remove "s3-website." from the beginning. On Jun 8, 2016 1:27 AM, "Yongho Hwang" notifications@github.com wrote: > Additionally, attach more detail log messages. > > > PUT /test.txt HTTP/1.1 > > Host: my-fs-objects.s3-website.ap-northeast-2.amazonaws.com > > Accept: _/_ > > Authorization: AWS AKIAII2252DICSSTJTZA:ICqR9JHe1mv3pJQMof+brfMye5Y= > > Content-Type:text/plain > > Date: Wed, 08 Jun 2016 05:17:19 GMT > > x-amz-acl:private > > x-amz-meta-gid:1000 > > x-amz-meta-mode:33204 > > x-amz-meta-mtime:1465363039 > > x-amz-meta-uid:1000 > > Content-Length: 0 > > Expect: 100-continue > > < HTTP/1.1 405 Method Not Allowed > < x-amz-request-id: 9F70721EF30119B5 > < x-amz-id-2: 9dgdcGeKtZxLN1rJWTmP9uWYDy623b9MSDoKzsJI7Exe7syx3dj1Mfct4SXRM2+COtJUx2O4d3c= > < Allow: GET, HEAD, OPTIONS > < Content-Type: text/html; charset=utf-8 > < Content-Length: 421 > < Date: Wed, 08 Jun 2016 05:17:19 GMT > < Connection: close > - Server AmazonS3 is not blacklisted > < Server: AmazonS3 > < > - Closing connection 13 > RequestPerform(1483): HTTP response code 405 > RequestPerform(1512): HTTP response code = 405, returning EIO > DelStat(375): delete stat cache entry[path=/test.txt] > unique: 5, error: -5 (Input/output error), outsize: 16 > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > https://github.com/s3fs-fuse/s3fs-fuse/issues/430#issuecomment-224491438, > or mute the thread > https://github.com/notifications/unsubscribe/ACUxPExyuIDCV5wT1iSMcb-ZbM3hb3KDks5qJlLCgaJpZM4IvoY5 > .
Author
Owner

@choruspapa commented on GitHub (Jun 13, 2016):

I'll share the result of @sqlbot 's recommand.

  1. without s3-website
    RequestPerform(1467): connecting to URL http://my-fs-objects.ap-northeast-2.amazonaws.com/
* Hostname was NOT found in DNS cache
* Could not resolve host: my-fs-objects.ap-northeast-2.amazonaws.com
* Closing connection 0
RequestPerform(1529): ### CURLE_COULDNT_RESOLVE_HOST
    RequestPerform(1617): ### retrying...
    RemakeHandle(1288): Retry request. [type=5][url=http://my-fs-objects.ap-northeast-2.amazonaws.com/][path=/]
* Hostname was NOT found in DNS cache
* Could not resolve host: my-fs-objects.ap-northeast-2.amazonaws.com
* Closing connection 1
  1. replace s3-website with s3
    set_moutpoint_attribute(3291): PROC(uid=0, gid=0) - MountPoint(uid=1000, gid=1000, mode=40775)
FUSE library version: 2.9.2
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.22
flags=0x0000f7fb
max_readahead=0x00020000
s3fs_init(2595): init
s3fs_check_service(2894): check services.
    CheckBucket(2228): check a bucket.
    RequestPerform(1467): connecting to URL http://my-bucket.s3.ap-northeast-2.amazonaws.com/my-fs-objects/
* Hostname was NOT found in DNS cache
*   Trying 52.92.8.7...
* Connected to my-bucket.s3.ap-northeast-2.amazonaws.com (52.92.8.7) port 80 (#0)
> GET /my-fs-objects/ HTTP/1.1
Host: my-bucket.s3.ap-northeast-2.amazonaws.com
Accept: */*
Authorization: AWS A###############A:C###################8=
Date: Mon, 13 Jun 2016 02:20:39 GMT

* The requested URL returned error: 400 Bad Request
* Closing connection 0
RequestPerform(1595): ### CURLE_HTTP_RETURNED_ERROR
RequestPerform(1600): HTTP response code =400
s3fs: Failed to access bucket.

How can I get a rest api endpoint for s3 bucket?

<!-- gh-comment-id:225478485 --> @choruspapa commented on GitHub (Jun 13, 2016): I'll share the result of @sqlbot 's recommand. 1) without s3-website ``` RequestPerform(1467): connecting to URL http://my-fs-objects.ap-northeast-2.amazonaws.com/ * Hostname was NOT found in DNS cache * Could not resolve host: my-fs-objects.ap-northeast-2.amazonaws.com * Closing connection 0 RequestPerform(1529): ### CURLE_COULDNT_RESOLVE_HOST RequestPerform(1617): ### retrying... RemakeHandle(1288): Retry request. [type=5][url=http://my-fs-objects.ap-northeast-2.amazonaws.com/][path=/] * Hostname was NOT found in DNS cache * Could not resolve host: my-fs-objects.ap-northeast-2.amazonaws.com * Closing connection 1 ``` 2) replace s3-website with s3 ``` set_moutpoint_attribute(3291): PROC(uid=0, gid=0) - MountPoint(uid=1000, gid=1000, mode=40775) FUSE library version: 2.9.2 nullpath_ok: 0 nopath: 0 utime_omit_ok: 0 unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0 INIT: 7.22 flags=0x0000f7fb max_readahead=0x00020000 s3fs_init(2595): init s3fs_check_service(2894): check services. CheckBucket(2228): check a bucket. RequestPerform(1467): connecting to URL http://my-bucket.s3.ap-northeast-2.amazonaws.com/my-fs-objects/ * Hostname was NOT found in DNS cache * Trying 52.92.8.7... * Connected to my-bucket.s3.ap-northeast-2.amazonaws.com (52.92.8.7) port 80 (#0) > GET /my-fs-objects/ HTTP/1.1 Host: my-bucket.s3.ap-northeast-2.amazonaws.com Accept: */* Authorization: AWS A###############A:C###################8= Date: Mon, 13 Jun 2016 02:20:39 GMT * The requested URL returned error: 400 Bad Request * Closing connection 0 RequestPerform(1595): ### CURLE_HTTP_RETURNED_ERROR RequestPerform(1600): HTTP response code =400 s3fs: Failed to access bucket. ``` How can I get a rest api endpoint for s3 bucket?
Author
Owner

@gaul commented on GitHub (Jun 13, 2016):

Which version of s3fs do you use? ap-northeast-2 requires use of signature v4 which only newer s3fs versions support. I successfully mounted such a bucket via:

s3fs $MY_BUCKET $MY_PATH -o url=http://s3.ap-northeast-2.amazonaws.com/ -o endpoint=ap-northeast-2
<!-- gh-comment-id:225502211 --> @gaul commented on GitHub (Jun 13, 2016): Which version of s3fs do you use? ap-northeast-2 requires use of signature v4 which only newer s3fs versions support. I successfully mounted such a bucket via: ``` s3fs $MY_BUCKET $MY_PATH -o url=http://s3.ap-northeast-2.amazonaws.com/ -o endpoint=ap-northeast-2 ```
Author
Owner

@choruspapa commented on GitHub (Jun 13, 2016):

Would you tell me which version of s3fs support v4?

$ s3fs --version
Amazon Simple Storage Service File System 1.74
<!-- gh-comment-id:225509212 --> @choruspapa commented on GitHub (Jun 13, 2016): Would you tell me which version of s3fs support v4? ``` $ s3fs --version Amazon Simple Storage Service File System 1.74 ```
Author
Owner

@gaul commented on GitHub (Jun 13, 2016):

Signature v4 support merged in 1.79 but but I recommend using the latest 1.80.

<!-- gh-comment-id:225510333 --> @gaul commented on GitHub (Jun 13, 2016): Signature v4 support merged in 1.79 but but I recommend using the latest 1.80.
Author
Owner

@choruspapa commented on GitHub (Jun 13, 2016):

It works. nice!!
Thank you. @andrewgaul.

<!-- gh-comment-id:225518241 --> @choruspapa commented on GitHub (Jun 13, 2016): It works. nice!! Thank you. @andrewgaul.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#230
No description provided.