starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 05:16:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #507] Does not check ECS task role location #280

New issue
Closed
opened 2026-03-04 01:44:00 +03:00 by kerem · 6 comments
kerem commented 2026-03-04 01:44:00 +03:00
Owner
Copy link

Originally created by @countergram on GitHub (Nov 21, 2016).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/507

Is it possible to use s3fs from within ECS with an IAM role? I can't get it to work.

s3fs has a hardcoded URL for roles: http://169.254.169.254/latest/meta-data/iam/security-credentials/

ECS container specific roles (Task roles) are at http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI

Originally created by @countergram on GitHub (Nov 21, 2016). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/507 Is it possible to use s3fs from within ECS with an IAM role? I can't get it to work. s3fs has a hardcoded URL for roles: `http://169.254.169.254/latest/meta-data/iam/security-credentials/` ECS container specific roles (Task roles) are at `http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI`
kerem closed this issue 2026-03-04 01:44:00 +03:00
kerem commented 2026-03-04 01:44:01 +03:00
Author
Owner
Copy link

@psyvision commented on GitHub (Nov 1, 2017):

Hi @ggtakec - We've stumbled across this issue today and I'd like to get a pull request in to use the alternate URL to use the ECS task role rather than the EC2 instance role.

How do you think would be best to implement this? I.e. an additional command line switch to change the URL used or an alternative command line option e.g. task_iam_role?

<!-- gh-comment-id:341111770 --> @psyvision commented on GitHub (Nov 1, 2017): Hi @ggtakec - We've stumbled across this issue today and I'd like to get a pull request in to use the alternate URL to use the ECS task role rather than the EC2 instance role. How do you think would be best to implement this? I.e. an additional command line switch to change the URL used or an alternative command line option e.g. task_iam_role?
kerem commented 2026-03-04 01:44:01 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Nov 5, 2017):

@psyvision @countergram
I'm sorry for not care about ECS.

If I implement this function, consider the following:

  1. Add an argument indicating that it is ECS as the startup option (ex. "-ecs" etc)
  2. Use the value of the environment variable $ AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
  3. Required for (1) and (2), if an exclusion option such as iamrole is specified, an error

It maybe fixed S3fsCurl::CheckIAMCredentialUpdate, I think that it will be a way to add a review per function.
This function will be called from s3fs_check_service.

It seems that we need to aupport ECS on s3fs.

<!-- gh-comment-id:341967518 --> @ggtakec commented on GitHub (Nov 5, 2017): @psyvision @countergram I'm sorry for not care about ECS. If I implement this function, consider the following: 1) Add an argument indicating that it is ECS as the startup option (ex. "-ecs" etc) 2) Use the value of the environment variable $ AWS_CONTAINER_CREDENTIALS_RELATIVE_URI 3) Required for (1) and (2), if an exclusion option such as iamrole is specified, an error It maybe fixed S3fsCurl::CheckIAMCredentialUpdate, I think that it will be a way to add a review per function. This function will be called from s3fs_check_service. It seems that we need to aupport ECS on s3fs.
kerem commented 2026-03-04 01:44:01 +03:00
Author
Owner
Copy link

@psyvision commented on GitHub (Nov 5, 2017):

Hi @ggtakec no worries! I'm happy to work on implementing that, I had already had a good look over the code base but anted to check best practices.

I'll submit a pull request as I start getting things implemented.

<!-- gh-comment-id:341970547 --> @psyvision commented on GitHub (Nov 5, 2017): Hi @ggtakec no worries! I'm happy to work on implementing that, I had already had a good look over the code base but anted to check best practices. I'll submit a pull request as I start getting things implemented.
kerem commented 2026-03-04 01:44:01 +03:00
Author
Owner
Copy link

@psyvision commented on GitHub (Nov 6, 2017):

Nearly got this working... :/

<!-- gh-comment-id:342301094 --> @psyvision commented on GitHub (Nov 6, 2017): Nearly got this working... :/
kerem commented 2026-03-04 01:44:01 +03:00
Author
Owner
Copy link

@psyvision commented on GitHub (Nov 8, 2017):

It works! I need to do a code tidy up to remove some of my debug logging.

I have a IAM role created with S3FullAccess. My ECS task is assigned this role. I can then mount a bucket with the following command:

s3fs -o allow_other -o ecs -o use_sse=1 my-bucket-name /mnt

Note
I've used jsoncpp to parse the JSON metadata returned by AWS because the previous parser wouldn't handle the ECS specific endpoint. At the moment this appears to be dynamically linked so I have to run apt-get install libjsoncpp-dev before I can use s3fs.

My knowledge of the build setup here isn't great (I'm not really a C++ programmer) so I haven't worked out how to static link it yet. If I do I'll commit that change.

<!-- gh-comment-id:342858867 --> @psyvision commented on GitHub (Nov 8, 2017): It works! I need to do a code tidy up to remove some of my debug logging. I have a IAM role created with S3FullAccess. My ECS task is assigned this role. I can then mount a bucket with the following command: `s3fs -o allow_other -o ecs -o use_sse=1 my-bucket-name /mnt` **Note** I've used jsoncpp to parse the JSON metadata returned by AWS because the previous parser wouldn't handle the ECS specific endpoint. At the moment this appears to be dynamically linked so I have to run `apt-get install libjsoncpp-dev` before I can use s3fs. My knowledge of the build setup here isn't great (I'm not really a C++ programmer) so I haven't worked out how to static link it yet. If I do I'll commit that change.
kerem commented 2026-03-04 01:44:02 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Nov 19, 2017):

@psyvision I merged your PR #671
If you found something wrong about it, please let me know.
Thanks all.

<!-- gh-comment-id:345494062 --> @ggtakec commented on GitHub (Nov 19, 2017): @psyvision I merged your PR #671 If you found something wrong about it, please let me know. Thanks all.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#280
No description provided.