starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 05:16:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #2401] mount fails when sse kmsid specified #1182

New issue
Closed
opened 2026-03-04 01:51:58 +03:00 by kerem · 2 comments
kerem commented 2026-03-04 01:51:58 +03:00
Owner
Copy link

Originally created by @aczire on GitHub (Jan 25, 2024).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2401

s3fs version = 1.93 (commit:54aa278) with OpenSSL

OS: .10.205-195.807.amzn2.x86_64 #1 SMP Tue Jan 16 18:28:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Fuse: 2.9.2

When running mount, if sse:kmsid is provided it fails.

Not Working: s3fs bucket-name /hps/dev/training -ouse_cache=/tmp -oallow_other -ouse_sse=kmsid:2670049f-xxxx-key-d0f8ff8c7ba8 -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg

Working (but write fails): s3fs bucket-name /hps/dev/training -ouse_cache=/tmp -oallow_other -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg

See the log below.

root@ip-100-XXX-XXX-182 ~]# s3fs bucket-name /hps/dev/training -ouse_cache=/tmp -oallow_other -ouse_sse=kmsid:2670049f-xxxx-key-d0f8ff8c7ba8 -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg
2024-01-25T22:53:28.168Z [INF] s3fs version 1.93(54aa278) : s3fs -ouse_cache=/tmp -oallow_other -ouse_sse=kmsid:2670049f-xxxx-key-d0f8ff8c7ba8 -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg bucket-name /hps/dev/training
2024-01-25T22:53:28.169Z [CRT] s3fs_logger.cpp:LowSetLogLevel(232): change debug level from [CRT] to [DBG]
2024-01-25T22:53:28.169Z [INF]     s3fs.cpp:set_mountpoint_attribute(4544): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
2024-01-25T22:53:28.170Z [INF] s3fs_util.cpp:compare_sysname(411): system name is Linux
2024-01-25T22:53:28.170Z [WAN] curl.cpp:InitMimeType(407): Could not find mime.types files, you have to create file(/etc/mime.types) or specify mime option for existing mime.types file.
2024-01-25T22:53:28.170Z [WAN] s3fs.cpp:main(5674): Missing MIME types prevents setting Content-Type on uploaded objects.
2024-01-25T22:53:28.170Z [INF] s3fs.cpp:main(5762): Free space ratio default to 10 %, ensure the available disk space is greater than 817.998 MB
2024-01-25T22:53:28.172Z [INF] s3fs.cpp:s3fs_init(4267): init v1.93(commit:54aa278) with OpenSSL, credential-library(built-in)
2024-01-25T22:53:28.172Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:53:28.172Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/api/token
2024-01-25T22:53:28.173Z [INF]       curl.cpp:RequestPerform(2488): HTTP response code 200
2024-01-25T22:53:28.173Z [INF]       s3fs_cred.cpp:SetIAMv2APIToken(444): Setting AWS IMDSv2 API token to AQAEAJ_rIhS2uBxxxxxxxxxxxxxHUlVw==
2024-01-25T22:53:28.173Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:53:28.173Z [INF]       curl.cpp:GetIAMRoleFromMetaData(3109): Get IAM Role name
2024-01-25T22:53:28.173Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:53:28.173Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/
2024-01-25T22:53:28.174Z [INF]       curl.cpp:RequestPerform(2488): HTTP response code 200
2024-01-25T22:53:28.174Z [INF]       s3fs_cred.cpp:SetIAMRoleFromMetaData(579): IAM role name response = "SC-196235955080-pp-hlltnp3bjy47e-Ec2InstanceRole-Kk4pmmHAF5V7"
2024-01-25T22:53:28.174Z [INF] s3fs_cred.cpp:LoadIAMRoleFromMetaData(540): loaded IAM role name = SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx
2024-01-25T22:53:28.174Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:53:28.174Z [INF] s3fs.cpp:s3fs_check_service(4411): check services.
2024-01-25T22:53:28.174Z [INF] s3fs_cred.cpp:CheckIAMCredentialUpdate(1149): IAM Access Token refreshing...
2024-01-25T22:53:28.174Z [INF]       s3fs_cred.cpp:GetIAMCredentialsURL(371): [IAM role=SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx]
2024-01-25T22:53:28.174Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:53:28.174Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/api/token
2024-01-25T22:53:28.174Z [INF]       curl.cpp:RequestPerform(2488): HTTP response code 200
2024-01-25T22:53:28.174Z [INF]       s3fs_cred.cpp:SetIAMv2APIToken(444): Setting AWS IMDSv2 API token to AQAEAJ_rIhxxxxxxxxxxxxxxxxxxCxY7REfg==
2024-01-25T22:53:28.174Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:53:28.175Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:53:28.175Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx
2024-01-25T22:53:28.175Z [INF]       curl.cpp:RequestPerform(2488): HTTP response code 200
2024-01-25T22:53:28.175Z [INF]       s3fs_cred.cpp:SetIAMCredentials(547): IAM credential response = "{
  "Code" : "Success",
  "LastUpdated" : "2024-01-25T22:35:39Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIAXXXXXXXXXXXI74",
  "SecretAccessKey" : "NSpcxwwY/jcFE+dqxr4xxxxxxxxxxxxxxANG",
  "Token" : "IQXXXXXVjEN///////////wEaDGV1LWNlxxxxxxxxxxxxxxxxxxxxxxxxxYXwazI9BS1z7/H+N",
  "Expiration" : "2024-01-26T05:11:03Z"
}"
2024-01-25T22:53:28.175Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:53:28.175Z [INF] s3fs_cred.cpp:CheckIAMCredentialUpdate(1163): IAM Access Token refreshed
2024-01-25T22:53:28.175Z [INF]       curl.cpp:CheckBucket(3616): check a bucket path(/).
2024-01-25T22:53:28.175Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:53:28.175Z [INF]       curl_util.cpp:prepare_url(210): URL is https://s3-eu-central-1.amazonaws.com/bucket-name/
2024-01-25T22:53:28.176Z [INF]       curl_util.cpp:prepare_url(243): URL changed is https://bucket-name.s3-eu-central-1.amazonaws.com/
2024-01-25T22:53:28.176Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL https://bucket-name.s3-eu-central-1.amazonaws.com/
2024-01-25T22:53:28.176Z [INF]       curl.cpp:insertV4Headers(2840): computing signature [GET] [/] [] []
2024-01-25T22:53:28.176Z [INF]       curl_util.cpp:url_to_host(265): url is https://s3-eu-central-1.amazonaws.com
2024-01-25T22:53:28.201Z [ERR] curl.cpp:RequestPerform(2529): HTTP response code 400, returning EIO. Body Text: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidArgument</Code><Message>x-amz-server-side-encryption header is not supported for this operation.</Message><ArgumentName>x-amz-server-side-encryption</ArgumentName><ArgumentValue>aws:kms</ArgumentValue><RequestId>HZ8XXXXXXXXXWSQHK</RequestId><HostId>MkhSNNIxxxxxxx6AloOYzJbkXA=</HostId></Error>
2024-01-25T22:53:28.201Z [ERR] curl.cpp:CheckBucket(3693): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidArgument</Code><Message>x-amz-server-side-encryption header is not supported for this operation.</Message><ArgumentName>x-amz-server-side-encryption</ArgumentName><ArgumentValue>aws:kms</ArgumentValue><RequestId>HZ8XXXXXXXXXWSQHK</RequestId><HostId>MkXXXXXXXXXXXXXXXXXXXX6AloOYzJbkXA=</HostId></Error>
2024-01-25T22:53:28.201Z [CRT] s3fs.cpp:s3fs_check_service(4488): Failed to connect by sigv4, so retry to connect by signature version 2. But you should to review url and endpoint option.
2024-01-25T22:53:28.201Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:53:28.201Z [INF]       curl.cpp:CheckBucket(3616): check a bucket path(/).
2024-01-25T22:53:28.201Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:53:28.201Z [INF]       curl_util.cpp:prepare_url(210): URL is https://s3-eu-central-1.amazonaws.com/bucket-name/
2024-01-25T22:53:28.201Z [INF]       curl_util.cpp:prepare_url(243): URL changed is https://bucket-name.s3-eu-central-1.amazonaws.com/
2024-01-25T22:53:28.201Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL https://bucket-name.s3-eu-central-1.amazonaws.com/
2024-01-25T22:53:28.212Z [ERR] curl.cpp:RequestPerform(2529): HTTP response code 400, returning EIO. Body Text: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>HZ86BASBQEK1F82Q</RequestId><HostId>orHSQVWBd3KQHMTOXXXXXXXXXXXXXXXXXXKgsVyPV8/k7c=</HostId></Error>
2024-01-25T22:53:28.212Z [ERR] curl.cpp:CheckBucket(3693): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>HZ86BASBQEK1F82Q</RequestId><HostId>orHSQVoBL9QHMTOXXXXXXXXXXXXXXXXXXKgsVyPV8/k7c=</HostId></Error>
2024-01-25T22:53:28.212Z [CRT] s3fs.cpp:s3fs_check_service(4500): Failed to check bucket and directory for mount point : Bad Request(host=https://s3-eu-central-1.amazonaws.com)
2024-01-25T22:53:28.212Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:53:28.212Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(4257): Exiting FUSE event loop due to errors

2024-01-25T22:53:28.212Z [INF] s3fs.cpp:s3fs_destroy(4316): destroy

If we dont specify the kms key id, then all is good, but write fails since it needs kms key id.

[root@ip-100-XXX-XXX-182 ~]# s3fs bucket-name /hps/dev/training -ouse_cache=/tmp -oallow_other -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg
2024-01-25T22:54:51.584Z [INF] s3fs version 1.93(54aa278) : s3fs -ouse_cache=/tmp -oallow_other -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg bucket-name /hps/dev/training
2024-01-25T22:54:51.586Z [CRT] s3fs_logger.cpp:LowSetLogLevel(232): change debug level from [CRT] to [DBG]
2024-01-25T22:54:51.586Z [INF]     s3fs.cpp:set_mountpoint_attribute(4544): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
2024-01-25T22:54:51.586Z [INF] s3fs_util.cpp:compare_sysname(411): system name is Linux
2024-01-25T22:54:51.586Z [WAN] curl.cpp:InitMimeType(407): Could not find mime.types files, you have to create file(/etc/mime.types) or specify mime option for existing mime.types file.
2024-01-25T22:54:51.586Z [WAN] s3fs.cpp:main(5674): Missing MIME types prevents setting Content-Type on uploaded objects.
2024-01-25T22:54:51.586Z [INF] s3fs.cpp:main(5762): Free space ratio default to 10 %, ensure the available disk space is greater than 817.998 MB
2024-01-25T22:54:51.587Z [INF] s3fs.cpp:s3fs_init(4267): init v1.93(commit:54aa278) with OpenSSL, credential-library(built-in)
2024-01-25T22:54:51.587Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:54:51.587Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/api/token
2024-01-25T22:54:51.588Z [INF]       curl.cpp:RequestPerform(2488): HTTP response code 200
2024-01-25T22:54:51.588Z [INF]       s3fs_cred.cpp:SetIAMv2APIToken(444): Setting AWS IMDSv2 API token to AQAEAJ_rIhRu4lxxxxxxxxxxxxxxx67vU9fTKeA==
2024-01-25T22:54:51.588Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:54:51.588Z [INF]       curl.cpp:GetIAMRoleFromMetaData(3109): Get IAM Role name
2024-01-25T22:54:51.588Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:54:51.588Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/
2024-01-25T22:54:51.589Z [INF]       curl.cpp:RequestPerform(2488): HTTP response code 200
2024-01-25T22:54:51.589Z [INF]       s3fs_cred.cpp:SetIAMRoleFromMetaData(579): IAM role name response = "SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx"
2024-01-25T22:54:51.589Z [INF] s3fs_cred.cpp:LoadIAMRoleFromMetaData(540): loaded IAM role name = SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx
2024-01-25T22:54:51.589Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:54:51.589Z [INF] s3fs.cpp:s3fs_check_service(4411): check services.
2024-01-25T22:54:51.589Z [INF] s3fs_cred.cpp:CheckIAMCredentialUpdate(1149): IAM Access Token refreshing...
2024-01-25T22:54:51.589Z [INF]       s3fs_cred.cpp:GetIAMCredentialsURL(371): [IAM role=SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx]
2024-01-25T22:54:51.589Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:54:51.589Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/api/token
2024-01-25T22:54:51.590Z [INF]       curl.cpp:RequestPerform(2488): HTTP response code 200
2024-01-25T22:54:51.590Z [INF]       s3fs_cred.cpp:SetIAMv2APIToken(444): Setting AWS IMDSv2 API token to AQAEAJ_rIhTSKjNxs7tDb7i6OjNIqf8-hVoCUF7wPOw0i7Wj34kxRw==
2024-01-25T22:54:51.590Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:54:51.590Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:54:51.590Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx
2024-01-25T22:54:51.591Z [INF]       curl.cpp:RequestPerform(2488): HTTP response code 200
2024-01-25T22:54:51.591Z [INF]       s3fs_cred.cpp:SetIAMCredentials(547): IAM credential response = "{
  "Code" : "Success",
  "LastUpdated" : "2024-01-25T22:35:39Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIXXXXXXXXIE2BRI74",
  "SecretAccessKey" : "NSpcxxxxxxxxxxxxxxxxenANG",
  "Token" : "IQoJb3JpZ2luX2VjEN///////////wEaDGV1Lxxxxxxxxxxxxt3lbDN9NUG78h5lglNwazI9BS1z7/H+N",
  "Expiration" : "2024-01-26T05:11:03Z"
}"
2024-01-25T22:54:51.591Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:54:51.591Z [INF] s3fs_cred.cpp:CheckIAMCredentialUpdate(1163): IAM Access Token refreshed
2024-01-25T22:54:51.591Z [INF]       curl.cpp:CheckBucket(3616): check a bucket path(/).
2024-01-25T22:54:51.591Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31
2024-01-25T22:54:51.591Z [INF]       curl_util.cpp:prepare_url(210): URL is https://s3-eu-central-1.amazonaws.com/bucket-name/
2024-01-25T22:54:51.591Z [INF]       curl_util.cpp:prepare_url(243): URL changed is https://bucket-name.s3-eu-central-1.amazonaws.com/
2024-01-25T22:54:51.591Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL https://bucket-name.s3-eu-central-1.amazonaws.com/
2024-01-25T22:54:51.591Z [INF]       curl.cpp:insertV4Headers(2840): computing signature [GET] [/] [] []
2024-01-25T22:54:51.591Z [INF]       curl_util.cpp:url_to_host(265): url is https://s3-eu-central-1.amazonaws.com
2024-01-25T22:54:51.647Z [INF]       curl.cpp:RequestPerform(2488): HTTP response code 200
2024-01-25T22:54:51.647Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2024-01-25T22:54:51.647Z [INF] threadpoolman.cpp:StopThreads(178): Any threads are running now, then nothing to do.
2024-01-25T22:54:51.647Z [INF]       threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan.
2024-01-25T22:54:51.647Z [INF]       threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan.
2024-01-25T22:54:51.647Z [INF]       threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan.
2024-01-25T22:54:51.647Z [INF]       threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan.
2024-01-25T22:54:51.647Z [INF]       threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan.



^C2024-01-25T22:55:40.794Z [INF] s3fs.cpp:s3fs_destroy(4316): destroy
2024-01-25T22:55:40.794Z [DBG] threadpoolman.cpp:StopThreads(195): succeed pthread_join - return code(0)
2024-01-25T22:55:40.794Z [DBG] threadpoolman.cpp:StopThreads(195): succeed pthread_join - return code(0)
2024-01-25T22:55:40.794Z [DBG] threadpoolman.cpp:StopThreads(195): succeed pthread_join - return code(0)
2024-01-25T22:55:40.794Z [DBG] threadpoolman.cpp:StopThreads(195): succeed pthread_join - return code(0)
2024-01-25T22:55:40.794Z [DBG] threadpoolman.cpp:StopThreads(195): succeed pthread_join - return code(0)
[root@ip-100-XXX-XXX-182 ~]# 4ls -la /hps/dev/training/
Originally created by @aczire on GitHub (Jan 25, 2024). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2401 s3fs version = 1.93 (commit:54aa278) with OpenSSL OS: .10.205-195.807.amzn2.x86_64 #1 SMP Tue Jan 16 18:28:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Fuse: 2.9.2 When running mount, if sse:kmsid is provided it fails. Not Working: s3fs bucket-name /hps/dev/training -ouse_cache=/tmp -oallow_other -ouse_sse=kmsid:2670049f-xxxx-key-d0f8ff8c7ba8 -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg Working (but write fails): s3fs bucket-name /hps/dev/training -ouse_cache=/tmp -oallow_other -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg See the log below. ``` root@ip-100-XXX-XXX-182 ~]# s3fs bucket-name /hps/dev/training -ouse_cache=/tmp -oallow_other -ouse_sse=kmsid:2670049f-xxxx-key-d0f8ff8c7ba8 -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg 2024-01-25T22:53:28.168Z [INF] s3fs version 1.93(54aa278) : s3fs -ouse_cache=/tmp -oallow_other -ouse_sse=kmsid:2670049f-xxxx-key-d0f8ff8c7ba8 -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg bucket-name /hps/dev/training 2024-01-25T22:53:28.169Z [CRT] s3fs_logger.cpp:LowSetLogLevel(232): change debug level from [CRT] to [DBG] 2024-01-25T22:53:28.169Z [INF] s3fs.cpp:set_mountpoint_attribute(4544): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) 2024-01-25T22:53:28.170Z [INF] s3fs_util.cpp:compare_sysname(411): system name is Linux 2024-01-25T22:53:28.170Z [WAN] curl.cpp:InitMimeType(407): Could not find mime.types files, you have to create file(/etc/mime.types) or specify mime option for existing mime.types file. 2024-01-25T22:53:28.170Z [WAN] s3fs.cpp:main(5674): Missing MIME types prevents setting Content-Type on uploaded objects. 2024-01-25T22:53:28.170Z [INF] s3fs.cpp:main(5762): Free space ratio default to 10 %, ensure the available disk space is greater than 817.998 MB 2024-01-25T22:53:28.172Z [INF] s3fs.cpp:s3fs_init(4267): init v1.93(commit:54aa278) with OpenSSL, credential-library(built-in) 2024-01-25T22:53:28.172Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:53:28.172Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/api/token 2024-01-25T22:53:28.173Z [INF] curl.cpp:RequestPerform(2488): HTTP response code 200 2024-01-25T22:53:28.173Z [INF] s3fs_cred.cpp:SetIAMv2APIToken(444): Setting AWS IMDSv2 API token to AQAEAJ_rIhS2uBxxxxxxxxxxxxxHUlVw== 2024-01-25T22:53:28.173Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:53:28.173Z [INF] curl.cpp:GetIAMRoleFromMetaData(3109): Get IAM Role name 2024-01-25T22:53:28.173Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:53:28.173Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/ 2024-01-25T22:53:28.174Z [INF] curl.cpp:RequestPerform(2488): HTTP response code 200 2024-01-25T22:53:28.174Z [INF] s3fs_cred.cpp:SetIAMRoleFromMetaData(579): IAM role name response = "SC-196235955080-pp-hlltnp3bjy47e-Ec2InstanceRole-Kk4pmmHAF5V7" 2024-01-25T22:53:28.174Z [INF] s3fs_cred.cpp:LoadIAMRoleFromMetaData(540): loaded IAM role name = SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx 2024-01-25T22:53:28.174Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:53:28.174Z [INF] s3fs.cpp:s3fs_check_service(4411): check services. 2024-01-25T22:53:28.174Z [INF] s3fs_cred.cpp:CheckIAMCredentialUpdate(1149): IAM Access Token refreshing... 2024-01-25T22:53:28.174Z [INF] s3fs_cred.cpp:GetIAMCredentialsURL(371): [IAM role=SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx] 2024-01-25T22:53:28.174Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:53:28.174Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/api/token 2024-01-25T22:53:28.174Z [INF] curl.cpp:RequestPerform(2488): HTTP response code 200 2024-01-25T22:53:28.174Z [INF] s3fs_cred.cpp:SetIAMv2APIToken(444): Setting AWS IMDSv2 API token to AQAEAJ_rIhxxxxxxxxxxxxxxxxxxCxY7REfg== 2024-01-25T22:53:28.174Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:53:28.175Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:53:28.175Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx 2024-01-25T22:53:28.175Z [INF] curl.cpp:RequestPerform(2488): HTTP response code 200 2024-01-25T22:53:28.175Z [INF] s3fs_cred.cpp:SetIAMCredentials(547): IAM credential response = "{ "Code" : "Success", "LastUpdated" : "2024-01-25T22:35:39Z", "Type" : "AWS-HMAC", "AccessKeyId" : "ASIAXXXXXXXXXXXI74", "SecretAccessKey" : "NSpcxwwY/jcFE+dqxr4xxxxxxxxxxxxxxANG", "Token" : "IQXXXXXVjEN///////////wEaDGV1LWNlxxxxxxxxxxxxxxxxxxxxxxxxxYXwazI9BS1z7/H+N", "Expiration" : "2024-01-26T05:11:03Z" }" 2024-01-25T22:53:28.175Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:53:28.175Z [INF] s3fs_cred.cpp:CheckIAMCredentialUpdate(1163): IAM Access Token refreshed 2024-01-25T22:53:28.175Z [INF] curl.cpp:CheckBucket(3616): check a bucket path(/). 2024-01-25T22:53:28.175Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:53:28.175Z [INF] curl_util.cpp:prepare_url(210): URL is https://s3-eu-central-1.amazonaws.com/bucket-name/ 2024-01-25T22:53:28.176Z [INF] curl_util.cpp:prepare_url(243): URL changed is https://bucket-name.s3-eu-central-1.amazonaws.com/ 2024-01-25T22:53:28.176Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL https://bucket-name.s3-eu-central-1.amazonaws.com/ 2024-01-25T22:53:28.176Z [INF] curl.cpp:insertV4Headers(2840): computing signature [GET] [/] [] [] 2024-01-25T22:53:28.176Z [INF] curl_util.cpp:url_to_host(265): url is https://s3-eu-central-1.amazonaws.com 2024-01-25T22:53:28.201Z [ERR] curl.cpp:RequestPerform(2529): HTTP response code 400, returning EIO. Body Text: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>InvalidArgument</Code><Message>x-amz-server-side-encryption header is not supported for this operation.</Message><ArgumentName>x-amz-server-side-encryption</ArgumentName><ArgumentValue>aws:kms</ArgumentValue><RequestId>HZ8XXXXXXXXXWSQHK</RequestId><HostId>MkhSNNIxxxxxxx6AloOYzJbkXA=</HostId></Error> 2024-01-25T22:53:28.201Z [ERR] curl.cpp:CheckBucket(3693): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>InvalidArgument</Code><Message>x-amz-server-side-encryption header is not supported for this operation.</Message><ArgumentName>x-amz-server-side-encryption</ArgumentName><ArgumentValue>aws:kms</ArgumentValue><RequestId>HZ8XXXXXXXXXWSQHK</RequestId><HostId>MkXXXXXXXXXXXXXXXXXXXX6AloOYzJbkXA=</HostId></Error> 2024-01-25T22:53:28.201Z [CRT] s3fs.cpp:s3fs_check_service(4488): Failed to connect by sigv4, so retry to connect by signature version 2. But you should to review url and endpoint option. 2024-01-25T22:53:28.201Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:53:28.201Z [INF] curl.cpp:CheckBucket(3616): check a bucket path(/). 2024-01-25T22:53:28.201Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:53:28.201Z [INF] curl_util.cpp:prepare_url(210): URL is https://s3-eu-central-1.amazonaws.com/bucket-name/ 2024-01-25T22:53:28.201Z [INF] curl_util.cpp:prepare_url(243): URL changed is https://bucket-name.s3-eu-central-1.amazonaws.com/ 2024-01-25T22:53:28.201Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL https://bucket-name.s3-eu-central-1.amazonaws.com/ 2024-01-25T22:53:28.212Z [ERR] curl.cpp:RequestPerform(2529): HTTP response code 400, returning EIO. Body Text: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>HZ86BASBQEK1F82Q</RequestId><HostId>orHSQVWBd3KQHMTOXXXXXXXXXXXXXXXXXXKgsVyPV8/k7c=</HostId></Error> 2024-01-25T22:53:28.212Z [ERR] curl.cpp:CheckBucket(3693): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><RequestId>HZ86BASBQEK1F82Q</RequestId><HostId>orHSQVoBL9QHMTOXXXXXXXXXXXXXXXXXXKgsVyPV8/k7c=</HostId></Error> 2024-01-25T22:53:28.212Z [CRT] s3fs.cpp:s3fs_check_service(4500): Failed to check bucket and directory for mount point : Bad Request(host=https://s3-eu-central-1.amazonaws.com) 2024-01-25T22:53:28.212Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:53:28.212Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(4257): Exiting FUSE event loop due to errors 2024-01-25T22:53:28.212Z [INF] s3fs.cpp:s3fs_destroy(4316): destroy ``` If we dont specify the kms key id, then all is good, but write fails since it needs kms key id. ``` [root@ip-100-XXX-XXX-182 ~]# s3fs bucket-name /hps/dev/training -ouse_cache=/tmp -oallow_other -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg 2024-01-25T22:54:51.584Z [INF] s3fs version 1.93(54aa278) : s3fs -ouse_cache=/tmp -oallow_other -oiam_role=auto -f -ourl=https://s3-eu-central-1.amazonaws.com -oendpoint=eu-central-1 -odbglevel=dbg bucket-name /hps/dev/training 2024-01-25T22:54:51.586Z [CRT] s3fs_logger.cpp:LowSetLogLevel(232): change debug level from [CRT] to [DBG] 2024-01-25T22:54:51.586Z [INF] s3fs.cpp:set_mountpoint_attribute(4544): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) 2024-01-25T22:54:51.586Z [INF] s3fs_util.cpp:compare_sysname(411): system name is Linux 2024-01-25T22:54:51.586Z [WAN] curl.cpp:InitMimeType(407): Could not find mime.types files, you have to create file(/etc/mime.types) or specify mime option for existing mime.types file. 2024-01-25T22:54:51.586Z [WAN] s3fs.cpp:main(5674): Missing MIME types prevents setting Content-Type on uploaded objects. 2024-01-25T22:54:51.586Z [INF] s3fs.cpp:main(5762): Free space ratio default to 10 %, ensure the available disk space is greater than 817.998 MB 2024-01-25T22:54:51.587Z [INF] s3fs.cpp:s3fs_init(4267): init v1.93(commit:54aa278) with OpenSSL, credential-library(built-in) 2024-01-25T22:54:51.587Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:54:51.587Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/api/token 2024-01-25T22:54:51.588Z [INF] curl.cpp:RequestPerform(2488): HTTP response code 200 2024-01-25T22:54:51.588Z [INF] s3fs_cred.cpp:SetIAMv2APIToken(444): Setting AWS IMDSv2 API token to AQAEAJ_rIhRu4lxxxxxxxxxxxxxxx67vU9fTKeA== 2024-01-25T22:54:51.588Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:54:51.588Z [INF] curl.cpp:GetIAMRoleFromMetaData(3109): Get IAM Role name 2024-01-25T22:54:51.588Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:54:51.588Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/ 2024-01-25T22:54:51.589Z [INF] curl.cpp:RequestPerform(2488): HTTP response code 200 2024-01-25T22:54:51.589Z [INF] s3fs_cred.cpp:SetIAMRoleFromMetaData(579): IAM role name response = "SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx" 2024-01-25T22:54:51.589Z [INF] s3fs_cred.cpp:LoadIAMRoleFromMetaData(540): loaded IAM role name = SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx 2024-01-25T22:54:51.589Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:54:51.589Z [INF] s3fs.cpp:s3fs_check_service(4411): check services. 2024-01-25T22:54:51.589Z [INF] s3fs_cred.cpp:CheckIAMCredentialUpdate(1149): IAM Access Token refreshing... 2024-01-25T22:54:51.589Z [INF] s3fs_cred.cpp:GetIAMCredentialsURL(371): [IAM role=SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx] 2024-01-25T22:54:51.589Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:54:51.589Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/api/token 2024-01-25T22:54:51.590Z [INF] curl.cpp:RequestPerform(2488): HTTP response code 200 2024-01-25T22:54:51.590Z [INF] s3fs_cred.cpp:SetIAMv2APIToken(444): Setting AWS IMDSv2 API token to AQAEAJ_rIhTSKjNxs7tDb7i6OjNIqf8-hVoCUF7wPOw0i7Wj34kxRw== 2024-01-25T22:54:51.590Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:54:51.590Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:54:51.590Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL http://169.254.169.254/latest/meta-data/iam/security-credentials/SC-19xxxxxxxxx0-pp-xxx-Ec2InstanceRole-xxxx 2024-01-25T22:54:51.591Z [INF] curl.cpp:RequestPerform(2488): HTTP response code 200 2024-01-25T22:54:51.591Z [INF] s3fs_cred.cpp:SetIAMCredentials(547): IAM credential response = "{ "Code" : "Success", "LastUpdated" : "2024-01-25T22:35:39Z", "Type" : "AWS-HMAC", "AccessKeyId" : "ASIXXXXXXXXIE2BRI74", "SecretAccessKey" : "NSpcxxxxxxxxxxxxxxxxenANG", "Token" : "IQoJb3JpZ2luX2VjEN///////////wEaDGV1Lxxxxxxxxxxxxt3lbDN9NUG78h5lglNwazI9BS1z7/H+N", "Expiration" : "2024-01-26T05:11:03Z" }" 2024-01-25T22:54:51.591Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:54:51.591Z [INF] s3fs_cred.cpp:CheckIAMCredentialUpdate(1163): IAM Access Token refreshed 2024-01-25T22:54:51.591Z [INF] curl.cpp:CheckBucket(3616): check a bucket path(/). 2024-01-25T22:54:51.591Z [DBG] curl_handlerpool.cpp:GetHandler(83): Get handler from pool: rest = 31 2024-01-25T22:54:51.591Z [INF] curl_util.cpp:prepare_url(210): URL is https://s3-eu-central-1.amazonaws.com/bucket-name/ 2024-01-25T22:54:51.591Z [INF] curl_util.cpp:prepare_url(243): URL changed is https://bucket-name.s3-eu-central-1.amazonaws.com/ 2024-01-25T22:54:51.591Z [DBG] curl.cpp:RequestPerform(2453): connecting to URL https://bucket-name.s3-eu-central-1.amazonaws.com/ 2024-01-25T22:54:51.591Z [INF] curl.cpp:insertV4Headers(2840): computing signature [GET] [/] [] [] 2024-01-25T22:54:51.591Z [INF] curl_util.cpp:url_to_host(265): url is https://s3-eu-central-1.amazonaws.com 2024-01-25T22:54:51.647Z [INF] curl.cpp:RequestPerform(2488): HTTP response code 200 2024-01-25T22:54:51.647Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2024-01-25T22:54:51.647Z [INF] threadpoolman.cpp:StopThreads(178): Any threads are running now, then nothing to do. 2024-01-25T22:54:51.647Z [INF] threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan. 2024-01-25T22:54:51.647Z [INF] threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan. 2024-01-25T22:54:51.647Z [INF] threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan. 2024-01-25T22:54:51.647Z [INF] threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan. 2024-01-25T22:54:51.647Z [INF] threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan. ^C2024-01-25T22:55:40.794Z [INF] s3fs.cpp:s3fs_destroy(4316): destroy 2024-01-25T22:55:40.794Z [DBG] threadpoolman.cpp:StopThreads(195): succeed pthread_join - return code(0) 2024-01-25T22:55:40.794Z [DBG] threadpoolman.cpp:StopThreads(195): succeed pthread_join - return code(0) 2024-01-25T22:55:40.794Z [DBG] threadpoolman.cpp:StopThreads(195): succeed pthread_join - return code(0) 2024-01-25T22:55:40.794Z [DBG] threadpoolman.cpp:StopThreads(195): succeed pthread_join - return code(0) 2024-01-25T22:55:40.794Z [DBG] threadpoolman.cpp:StopThreads(195): succeed pthread_join - return code(0) [root@ip-100-XXX-XXX-182 ~]# 4ls -la /hps/dev/training/ ```
kerem closed this issue 2026-03-04 01:51:59 +03:00
kerem commented 2026-03-04 01:51:59 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Feb 12, 2024):

@aczire Sorry for the late reply.
I have reviewed your report and created #2409.
If you can try it, check it out.
Thanks in advance for your assistance.

<!-- gh-comment-id:1938008765 --> @ggtakec commented on GitHub (Feb 12, 2024): @aczire Sorry for the late reply. I have reviewed your report and created #2409. If you can try it, check it out. Thanks in advance for your assistance.
kerem commented 2026-03-04 01:51:59 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Feb 19, 2024):

@aczire The code that would resolve this issue has been merged.
If possible, try with the latest master branch code.
I will close this issue, but if the problem persists, please reopen it.
Thanks for your help.

<!-- gh-comment-id:1952564114 --> @ggtakec commented on GitHub (Feb 19, 2024): @aczire The code that would resolve this issue has been merged. If possible, try with the latest master branch code. I will close this issue, but if the problem persists, please reopen it. Thanks for your help.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#1182
No description provided.