[GH-ISSUE #273] Issue when using iam_role in fstab #139

Closed
opened 2026-03-04 01:42:33 +03:00 by kerem · 17 comments
Owner

Originally created by @JorisLimousin on GitHub (Sep 23, 2015).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/273

Hi everyone, I have an issue when trying to use the iam_role in the fstab to mount my bucket at boot time.

It works perfectly when doing the following command:

sudo /usr/local/bin/s3fs -d -o iam_role=my-role -o url=http://s3-eu-west-1.amazonaws.com -o endpoint=eu-west-1 my-bucket /mnt/my-bucket

And it even works when I have this in my fstab and I manually issue the "mount -a" command:

my-bucket /mnt/my-bucket fuse.s3fs rw,nosuid,-d,nodev,allow_other,umask=0022,iam_role=my-role,endpoint=eu-west-1,url=http://s3-eu-west-1.amazonaws.com 0 0

But no way to make it work when rebooting...

Did some of you already encountered this kind of problem ?

Thank you in advance and have a nice day !

BTW: Thank you very much for this powerful tool !

Regards.

Originally created by @JorisLimousin on GitHub (Sep 23, 2015). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/273 Hi everyone, I have an issue when trying to use the iam_role in the fstab to mount my bucket at boot time. It works perfectly when doing the following command: ``` sudo /usr/local/bin/s3fs -d -o iam_role=my-role -o url=http://s3-eu-west-1.amazonaws.com -o endpoint=eu-west-1 my-bucket /mnt/my-bucket ``` And it even works when I have this in my fstab and I manually issue the "mount -a" command: ``` my-bucket /mnt/my-bucket fuse.s3fs rw,nosuid,-d,nodev,allow_other,umask=0022,iam_role=my-role,endpoint=eu-west-1,url=http://s3-eu-west-1.amazonaws.com 0 0 ``` But no way to make it work when rebooting... Did some of you already encountered this kind of problem ? Thank you in advance and have a nice day ! BTW: Thank you very much for this powerful tool ! Regards.
kerem closed this issue 2026-03-04 01:42:33 +03:00
Author
Owner

@RobbKistler commented on GitHub (Sep 23, 2015):

Its possible the network isn't up at boot time. Take a look at PR #267 and recent discussion on Issue #57. You can add the _netdev option, which might help.

<!-- gh-comment-id:142704017 --> @RobbKistler commented on GitHub (Sep 23, 2015): Its possible the network isn't up at boot time. Take a look at PR #267 and recent discussion on Issue #57. You can add the _netdev option, which might help.
Author
Owner

@JorisLimousin commented on GitHub (Sep 24, 2015):

Awesome!

This was the only thing that was missing. I worked quite a lot of time to resolve this problem but could not figure this out. I saw the _netdev option a few times but as I did not know what it was doing I did not want to use it.

But this is exactly what I needed. Thank you very much!

<!-- gh-comment-id:142894046 --> @JorisLimousin commented on GitHub (Sep 24, 2015): Awesome! This was the only thing that was missing. I worked quite a lot of time to resolve this problem but could not figure this out. I saw the _netdev option a few times but as I did not know what it was doing I did not want to use it. But this is exactly what I needed. Thank you very much!
Author
Owner

@JorisLimousin commented on GitHub (Sep 28, 2015):

Hi again everyone,

I have added the _netdev option to the entry in fstab, and it seemed to work pretty fine.

But it does not work anymore.

The only thing I changed on the system was the time zone. As I am in England, I wanted to have the right time settings for my instance, so I changed the time zone in the "/etc/sysconfig/clock" file to "Europe/London".

But now, (I do not know if it is because of this) it does not mount my S3 bucket at boot time anymore.

The weird thing as well is that there are no logs. Not a single line in the "/var/log/messages" file.

However, when I do a "mount -a", the bucket is mounted properly and a log line is generated:

init v1.79(commit:489f9ed) with OpenSSL

Did you already encounter this type of problem before?

Thank you in advance!

Kind regards.

<!-- gh-comment-id:143704699 --> @JorisLimousin commented on GitHub (Sep 28, 2015): Hi again everyone, I have added the _netdev option to the entry in fstab, and it seemed to work pretty fine. But it does not work anymore. The only thing I changed on the system was the time zone. As I am in England, I wanted to have the right time settings for my instance, so I changed the time zone in the "/etc/sysconfig/clock" file to "Europe/London". But now, (I do not know if it is because of this) it does not mount my S3 bucket at boot time anymore. The weird thing as well is that there are no logs. Not a single line in the "/var/log/messages" file. However, when I do a "mount -a", the bucket is mounted properly and a log line is generated: ``` init v1.79(commit:489f9ed) with OpenSSL ``` Did you already encounter this type of problem before? Thank you in advance! Kind regards.
Author
Owner

@ggtakec commented on GitHub (Sep 28, 2015):

I think, if local time is different significantly from the API server time, s3fs will probably fail to access AWS API.
If you does not check about ntp, please see http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html.
(If you already set ntp correctly, we need to get details about this problem.)
Thanks in advance for your help.

<!-- gh-comment-id:143776088 --> @ggtakec commented on GitHub (Sep 28, 2015): I think, if local time is different significantly from the API server time, s3fs will probably fail to access AWS API. If you does not check about ntp, please see http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html. (If you already set ntp correctly, we need to get details about this problem.) Thanks in advance for your help.
Author
Owner

@JorisLimousin commented on GitHub (Sep 28, 2015):

Hi Takeshi,

I followed the tutorial of the link you just provided, when I was setting the time and the ntp service.

So it should be configured correctly.

If you have any idea of how to resolve this problem, or if you require any additional information, please do not hesitate to contact me.

Thanks again for your tool and your help!

Kind regards.

<!-- gh-comment-id:143778956 --> @JorisLimousin commented on GitHub (Sep 28, 2015): Hi Takeshi, I followed the tutorial of the link you just provided, when I was setting the time and the ntp service. So it should be configured correctly. If you have any idea of how to resolve this problem, or if you require any additional information, please do not hesitate to contact me. Thanks again for your tool and your help! Kind regards.
Author
Owner

@ggtakec commented on GitHub (Sep 30, 2015):

I will modify the codes for printing error message(to syslog) at booting soon.
After that, we will get error message why s3fs fails to run.
Please wait a while.
Thanks in advance.

<!-- gh-comment-id:144263968 --> @ggtakec commented on GitHub (Sep 30, 2015): I will modify the codes for printing error message(to syslog) at booting soon. After that, we will get error message why s3fs fails to run. Please wait a while. Thanks in advance.
Author
Owner

@JorisLimousin commented on GitHub (Sep 30, 2015):

Yep no problem!

I'll provide feedback when your update is here.

Thank you.

<!-- gh-comment-id:144336328 --> @JorisLimousin commented on GitHub (Sep 30, 2015): Yep no problem! I'll provide feedback when your update is here. Thank you.
Author
Owner

@ggtakec commented on GitHub (Sep 30, 2015):

@xXZaryelXx I merged new codes to master branch, it is able to put debug message when s3fs is run on background.
Probably you can find the reason in syslog message, when you run s3fs with "dbglevel" option.
I think it should be set "info" or "dbg"(dbglevel=info).
If the problem is due to the communication to S3, you can set "curldbg" option. It puts more message about libcurl.
I hope this change helps us to solve this issue.

Regards,

<!-- gh-comment-id:144525768 --> @ggtakec commented on GitHub (Sep 30, 2015): @xXZaryelXx I merged new codes to master branch, it is able to put debug message when s3fs is run on background. Probably you can find the reason in syslog message, when you run s3fs with "dbglevel" option. I think it should be set "info" or "dbg"(dbglevel=info). If the problem is due to the communication to S3, you can set "curldbg" option. It puts more message about libcurl. I hope this change helps us to solve this issue. Regards,
Author
Owner

@JorisLimousin commented on GitHub (Oct 1, 2015):

Hi Takeshi,

Thank you for the update.

I have been adding these 2 options to the fstab entry. However, there is still no log generated corresponding to s3fs. :/

When trying to remount it manually with "mount -a", it displays "fuse: unknown option 'dbglevel'".

I am working on an Amazon Linux EC2 instance, so is there anything particular I should do to get logs about this issue ?

Thank you again for your time and investment in this.

Regards.

<!-- gh-comment-id:144683248 --> @JorisLimousin commented on GitHub (Oct 1, 2015): Hi Takeshi, Thank you for the update. I have been adding these 2 options to the fstab entry. However, there is still no log generated corresponding to s3fs. :/ When trying to remount it manually with "mount -a", it displays "fuse: unknown option 'dbglevel'". I am working on an Amazon Linux EC2 instance, so is there anything particular I should do to get logs about this issue ? Thank you again for your time and investment in this. Regards.
Author
Owner

@ggtakec commented on GitHub (Oct 1, 2015):

Did you pull master branch from s3fs-fuse and build & install?
Your log by command line manually said "unknown option dbglevel", it means s3fs is not latest build.
(If there are many s3fs installed some directories, you need to check it. ex: /usr/bin and /usr/local/bin)

Following line is my test fstab entry, it works good and puts log message into /var/log/messages.

fstab:

mybucket   /mnt/s3 fuse.s3fs       _netdev,allow_other,nonempty,enable_noobj_cache,use_cache=/tmp/.s3_cache,url=https://s3.amazonaws.com,dbglevel=dbg   0 0

/var/log/messages:(following is version line)

Oct  1 13:41:44 ********* s3fs[1102]: s3fs_init(3282): init v1.79(commit:cfdfecb) with OpenSSL

Please check and try.
Thanks in advance for your help.

<!-- gh-comment-id:144732899 --> @ggtakec commented on GitHub (Oct 1, 2015): Did you pull master branch from s3fs-fuse and build & install? Your log by command line manually said "unknown option dbglevel", it means s3fs is not latest build. (If there are many s3fs installed some directories, you need to check it. ex: /usr/bin and /usr/local/bin) Following line is my test fstab entry, it works good and puts log message into /var/log/messages. fstab: ``` mybucket /mnt/s3 fuse.s3fs _netdev,allow_other,nonempty,enable_noobj_cache,use_cache=/tmp/.s3_cache,url=https://s3.amazonaws.com,dbglevel=dbg 0 0 ``` /var/log/messages:(following is version line) ``` Oct 1 13:41:44 ********* s3fs[1102]: s3fs_init(3282): init v1.79(commit:cfdfecb) with OpenSSL ``` Please check and try. Thanks in advance for your help.
Author
Owner

@JorisLimousin commented on GitHub (Oct 1, 2015):

Aaah ok! I have just put "dbglevel" in my fstab entry, instead of "dbglevel=dbg" or "dbglevel=info", that's why it did not work when I tried to mount. ^^ But anyway, this still does not log anything when I reboot. :/

And the version of s3fs is the right one:

Amazon Simple Storage Service File System V1.79(commit:e29069b) with OpenSSL

There is not a single line related to S3FS for boot time in my logs. However, I noticed that the ntp daemon seems to start at the end, when everything else has been loaded. The finals rows of the log file are about this ntpd daemon.

BTW, here is my fstab entry if it can be interesting for you:

my-bucket /mnt/my-bucket fuse.s3fs rw,nosuid,-d,nodev,allow_other,_netdev,umask=0022,iam_role=my-role,endpoint=eu-west-1,url=http://s3-eu-west-1.amazonaws.com,retries=5,multireq_max=5,curldbg,dbglevel=info 0 0

Thank you and do not hesitate if you require anything.

<!-- gh-comment-id:144741522 --> @JorisLimousin commented on GitHub (Oct 1, 2015): Aaah ok! I have just put "dbglevel" in my fstab entry, instead of "dbglevel=dbg" or "dbglevel=info", that's why it did not work when I tried to mount. ^^ But anyway, this still does not log anything when I reboot. :/ And the version of s3fs is the right one: ``` Amazon Simple Storage Service File System V1.79(commit:e29069b) with OpenSSL ``` There is not a single line related to S3FS for boot time in my logs. However, I noticed that the ntp daemon seems to start at the end, when everything else has been loaded. The finals rows of the log file are about this ntpd daemon. BTW, here is my fstab entry if it can be interesting for you: ``` my-bucket /mnt/my-bucket fuse.s3fs rw,nosuid,-d,nodev,allow_other,_netdev,umask=0022,iam_role=my-role,endpoint=eu-west-1,url=http://s3-eu-west-1.amazonaws.com,retries=5,multireq_max=5,curldbg,dbglevel=info 0 0 ``` Thank you and do not hesitate if you require anything.
Author
Owner

@ggtakec commented on GitHub (Oct 1, 2015):

At first, I think you should check syslog.conf because we want to check s3fs debug log, if you can.
And I propose, how wonder whether it will try to reboot after running the ntpdate manually if possible.
In my case, ntpdate run after running s3fs too.
Last, there is no effect but we do not need to specify "-d" option in fstab, because it was replaced by dbglevel option.

Regards,

<!-- gh-comment-id:144763186 --> @ggtakec commented on GitHub (Oct 1, 2015): At first, I think you should check syslog.conf because we want to check s3fs debug log, if you can. And I propose, how wonder whether it will try to reboot after running the ntpdate manually if possible. In my case, ntpdate run after running s3fs too. Last, there is no effect but we do not need to specify "-d" option in fstab, because it was replaced by dbglevel option. Regards,
Author
Owner

@JorisLimousin commented on GitHub (Oct 1, 2015):

My /etc/rsyslog.conf looks like this:

...
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
...

I did not modify this file.

OK, so it may not be linked to the ntp daemon. Maybe only the date configuration that I changed, because actually I did not change anything related to the ntp. In fact, I just changed the time zone.

And yep I'll remove the -d option from the fstab !

Thank you.

Regards.

<!-- gh-comment-id:144765717 --> @JorisLimousin commented on GitHub (Oct 1, 2015): My `/etc/rsyslog.conf` looks like this: ``` ... # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none /var/log/messages ... ``` I did not modify this file. OK, so it may not be linked to the ntp daemon. Maybe only the date configuration that I changed, because actually I did not change anything related to the ntp. In fact, I just changed the time zone. And yep I'll remove the -d option from the fstab ! Thank you. Regards.
Author
Owner

@ggtakec commented on GitHub (Oct 2, 2015):

I tried to change timezone on my instance and run s3fs at booting.
But my result is success without problem, so ntp and s3fs and logging did not have any problem.
(my s3fs line in fstab has Iam_role option.)

I think about solving this issue, we will be able to find where is the problem after s3fs will be bale to put the logs normally.
Regards,

<!-- gh-comment-id:144895620 --> @ggtakec commented on GitHub (Oct 2, 2015): I tried to change timezone on my instance and run s3fs at booting. But my result is success without problem, so ntp and s3fs and logging did not have any problem. (my s3fs line in fstab has Iam_role option.) I think about solving this issue, we will be able to find where is the problem after s3fs will be bale to put the logs normally. Regards,
Author
Owner

@JorisLimousin commented on GitHub (Oct 2, 2015):

Hmmm yeah I don't know if it is related actually because even when I put bad values for url or endpoint for example, it still does not log anything... I will investigate and tell you ;) Just hope I did not waste your time because of some stupid thing I could have done...

<!-- gh-comment-id:144982852 --> @JorisLimousin commented on GitHub (Oct 2, 2015): Hmmm yeah I don't know if it is related actually because even when I put bad values for url or endpoint for example, it still does not log anything... I will investigate and tell you ;) Just hope I did not waste your time because of some stupid thing I could have done...
Author
Owner

@JorisLimousin commented on GitHub (Oct 2, 2015):

OK I found my mistake... So it was when I wanted to "clean" the instance to only keep the daemons at boot that I require, and consequently because I thought that the "netfs" daemon was not useful to me I turned it off.

I just turned it on again and everything works perfectly.

Should you update the documentation to specify that this daemon is required in order for s3fs to work at boot time?

<!-- gh-comment-id:144987252 --> @JorisLimousin commented on GitHub (Oct 2, 2015): OK I found my mistake... So it was when I wanted to "clean" the instance to only keep the daemons at boot that I require, and consequently because I thought that the "netfs" daemon was not useful to me I turned it off. I just turned it on again and everything works perfectly. Should you update the documentation to specify that this daemon is required in order for s3fs to work at boot time?
Author
Owner

@ggtakec commented on GitHub (Oct 3, 2015):

@xXZaryelXx I'm glad taht we know the cause of your problem.
And I updated FAQ about netfs in Fuse Over Amazon.
Thanks all.
(I closed this issue.)

<!-- gh-comment-id:145213486 --> @ggtakec commented on GitHub (Oct 3, 2015): @xXZaryelXx I'm glad taht we know the cause of your problem. And I updated FAQ about netfs in [Fuse Over Amazon](https://github.com/s3fs-fuse/s3fs-fuse/wiki/Fuse-Over-Amazon). Thanks all. (I closed this issue.)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#139
No description provided.