[GH-ISSUE #1456] Input/output error with GCS: An unexpected TLS packet was received. #763

Closed
opened 2026-03-04 01:48:35 +03:00 by kerem · 3 comments
Owner

Originally created by @afeller-quotient on GitHub (Oct 19, 2020).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1456

Version of s3fs being used (s3fs --version)

Amazon Simple Storage Service File System V1.87 (commit:cf86fa5) with OpenSSL

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

Version: 2.9.7-1ubuntu1

Kernel information (uname -r)

5.4.0-1025-gcp

GNU/Linux Distribution, if applicable (cat /etc/os-release)

NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.5 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

s3fs command line used, if applicable

s3fs <bucket-name> /mnt/gs -f -d -d -o nomultipart -o passwd_file=/etc/gcs-auth.txt -o url=https://storage.googleapis.com -o curldbg

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

[INF] s3fs.cpp:s3fs_getattr(777): [path=/airflow]
[INF]       curl.cpp:HeadRequest(2881): [tpath=/airflow]
[INF]       curl.cpp:PreHeadRequest(2841): [tpath=/airflow][bpath=][save=][sseckeypos=-1]
[INF]       curl_util.cpp:prepare_url(243): URL is https://storage.googleapis.com/bucket-name/airflow
[INF]       curl_util.cpp:prepare_url(276): URL changed is https://bucket-name.storage.googleapis.com/airflow
[INF]       curl.cpp:insertV4Headers(2572): computing signature [HEAD] [/airflow] [] []
[INF]       curl_util.cpp:url_to_host(320): url is https://storage.googleapis.com
[CURL DBG] * Connection 0 seems to be dead!
[CURL DBG] * Closing connection 0
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#1)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 1
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow][path=/airflow]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#2)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 2
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow][path=/airflow]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#3)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 3
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow][path=/airflow]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#4)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 4
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow][path=/airflow]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#5)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 5
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow][path=/airflow]
[ERR] curl.cpp:RequestPerform(2429): ### giving up
[INF]       curl.cpp:HeadRequest(2881): [tpath=/airflow/]
[INF]       curl.cpp:PreHeadRequest(2841): [tpath=/airflow/][bpath=][save=][sseckeypos=-1]
[INF]       curl_util.cpp:prepare_url(243): URL is https://storage.googleapis.com/bucket-name/airflow/
[INF]       curl_util.cpp:prepare_url(276): URL changed is https://bucket-name.storage.googleapis.com/airflow/
[INF]       curl.cpp:insertV4Headers(2572): computing signature [HEAD] [/airflow/] [] []
[INF]       curl_util.cpp:url_to_host(320): url is https://storage.googleapis.com
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#6)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 6
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow/][path=/airflow/]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#7)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 7
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow/][path=/airflow/]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#8)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 8
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow/][path=/airflow/]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#9)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 9
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow/][path=/airflow/]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#10)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 10
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow/][path=/airflow/]
[ERR] curl.cpp:RequestPerform(2429): ### giving up
[INF]       curl.cpp:HeadRequest(2881): [tpath=/airflow_$folder$]
[INF]       curl.cpp:PreHeadRequest(2841): [tpath=/airflow_$folder$][bpath=][save=][sseckeypos=-1]
[INF]       curl_util.cpp:prepare_url(243): URL is https://storage.googleapis.com/bucket-name/airflow_%24folder%24
[INF]       curl_util.cpp:prepare_url(276): URL changed is https://bucket-name.storage.googleapis.com/airflow_%24folder%24
[INF]       curl.cpp:insertV4Headers(2572): computing signature [HEAD] [/airflow_$folder$] [] []
[INF]       curl_util.cpp:url_to_host(320): url is https://storage.googleapis.com
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#11)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 11
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow_%24folder%24][path=/airflow_$folder$]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#12)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 12
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow_%24folder%24][path=/airflow_$folder$]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#13)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 13
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow_%24folder%24][path=/airflow_$folder$]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#14)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 14
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow_%24folder%24][path=/airflow_$folder$]
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#15)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 15
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow_%24folder%24][path=/airflow_$folder$]
[ERR] curl.cpp:RequestPerform(2429): ### giving up
[INF]   s3fs.cpp:list_bucket(2610): [path=/airflow]
[INF]       curl.cpp:ListBucketRequest(3268): [tpath=/airflow]
[INF]       curl_util.cpp:prepare_url(243): URL is https://storage.googleapis.com/bucket-name?delimiter=/&max-keys=2&prefix=airflow/
[INF]       curl_util.cpp:prepare_url(276): URL changed is https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/
[INF]       curl.cpp:insertV4Headers(2572): computing signature [GET] [/] [delimiter=/&max-keys=2&prefix=airflow/] []
[INF]       curl_util.cpp:url_to_host(320): url is https://storage.googleapis.com
[CURL DBG] * Rebuilt URL to: https://bucket-name.storage.googleapis.com/?delimiter=/&max-keys=2&prefix=airflow/
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#16)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 16
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=6][url=https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/][path=/airflow]
[CURL DBG] * Rebuilt URL to: https://bucket-name.storage.googleapis.com/?delimiter=/&max-keys=2&prefix=airflow/
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#17)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 17
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=6][url=https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/][path=/airflow]
[CURL DBG] * Rebuilt URL to: https://bucket-name.storage.googleapis.com/?delimiter=/&max-keys=2&prefix=airflow/
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#18)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 18
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=6][url=https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/][path=/airflow]
[CURL DBG] * Rebuilt URL to: https://bucket-name.storage.googleapis.com/?delimiter=/&max-keys=2&prefix=airflow/
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#19)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 19
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=6][url=https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/][path=/airflow]
[CURL DBG] * Rebuilt URL to: https://bucket-name.storage.googleapis.com/?delimiter=/&max-keys=2&prefix=airflow/
[CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache
[CURL DBG] *   Trying 172.217.204.128...
[CURL DBG] * TCP_NODELAY set
[CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#20)
[CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt
[CURL DBG] * found 383 certificates in /etc/ssl/certs
[CURL DBG] * SSL re-using session ID
[CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received.
[CURL DBG] * stopped the pause stream!
[CURL DBG] * Closing connection 20
[ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR
[INF] curl.cpp:RequestPerform(2411): ### retrying...
[INF]       curl.cpp:RemakeHandle(2034): Retry request. [type=6][url=https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/][path=/airflow]
[ERR] curl.cpp:RequestPerform(2429): ### giving up
[ERR] s3fs.cpp:list_bucket(2646): ListBucketRequest returns with error.
[ERR] s3fs.cpp:directory_empty(1081): list_bucket returns error.
   unique: 4, error: -5 (Input/output error), outsize: 16

Details about issue

About an hour after mounting, when trying to list files or directories, I get this error

Originally created by @afeller-quotient on GitHub (Oct 19, 2020). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1456 #### Version of s3fs being used (s3fs --version) `Amazon Simple Storage Service File System V1.87 (commit:cf86fa5) with OpenSSL` #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) `Version: 2.9.7-1ubuntu1` #### Kernel information (uname -r) `5.4.0-1025-gcp` #### GNU/Linux Distribution, if applicable (cat /etc/os-release) ``` NAME="Ubuntu" VERSION="18.04.5 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.5 LTS" VERSION_ID="18.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=bionic UBUNTU_CODENAME=bionic ``` #### s3fs command line used, if applicable ``` s3fs <bucket-name> /mnt/gs -f -d -d -o nomultipart -o passwd_file=/etc/gcs-auth.txt -o url=https://storage.googleapis.com -o curldbg ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) ``` [INF] s3fs.cpp:s3fs_getattr(777): [path=/airflow] [INF] curl.cpp:HeadRequest(2881): [tpath=/airflow] [INF] curl.cpp:PreHeadRequest(2841): [tpath=/airflow][bpath=][save=][sseckeypos=-1] [INF] curl_util.cpp:prepare_url(243): URL is https://storage.googleapis.com/bucket-name/airflow [INF] curl_util.cpp:prepare_url(276): URL changed is https://bucket-name.storage.googleapis.com/airflow [INF] curl.cpp:insertV4Headers(2572): computing signature [HEAD] [/airflow] [] [] [INF] curl_util.cpp:url_to_host(320): url is https://storage.googleapis.com [CURL DBG] * Connection 0 seems to be dead! [CURL DBG] * Closing connection 0 [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#1) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 1 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow][path=/airflow] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#2) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 2 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow][path=/airflow] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#3) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 3 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow][path=/airflow] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#4) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 4 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow][path=/airflow] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#5) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 5 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow][path=/airflow] [ERR] curl.cpp:RequestPerform(2429): ### giving up [INF] curl.cpp:HeadRequest(2881): [tpath=/airflow/] [INF] curl.cpp:PreHeadRequest(2841): [tpath=/airflow/][bpath=][save=][sseckeypos=-1] [INF] curl_util.cpp:prepare_url(243): URL is https://storage.googleapis.com/bucket-name/airflow/ [INF] curl_util.cpp:prepare_url(276): URL changed is https://bucket-name.storage.googleapis.com/airflow/ [INF] curl.cpp:insertV4Headers(2572): computing signature [HEAD] [/airflow/] [] [] [INF] curl_util.cpp:url_to_host(320): url is https://storage.googleapis.com [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#6) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 6 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow/][path=/airflow/] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#7) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 7 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow/][path=/airflow/] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#8) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 8 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow/][path=/airflow/] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#9) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 9 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow/][path=/airflow/] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#10) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 10 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow/][path=/airflow/] [ERR] curl.cpp:RequestPerform(2429): ### giving up [INF] curl.cpp:HeadRequest(2881): [tpath=/airflow_$folder$] [INF] curl.cpp:PreHeadRequest(2841): [tpath=/airflow_$folder$][bpath=][save=][sseckeypos=-1] [INF] curl_util.cpp:prepare_url(243): URL is https://storage.googleapis.com/bucket-name/airflow_%24folder%24 [INF] curl_util.cpp:prepare_url(276): URL changed is https://bucket-name.storage.googleapis.com/airflow_%24folder%24 [INF] curl.cpp:insertV4Headers(2572): computing signature [HEAD] [/airflow_$folder$] [] [] [INF] curl_util.cpp:url_to_host(320): url is https://storage.googleapis.com [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#11) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 11 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow_%24folder%24][path=/airflow_$folder$] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#12) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 12 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow_%24folder%24][path=/airflow_$folder$] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#13) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 13 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow_%24folder%24][path=/airflow_$folder$] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#14) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 14 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow_%24folder%24][path=/airflow_$folder$] [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#15) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 15 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=1][url=https://bucket-name.storage.googleapis.com/airflow_%24folder%24][path=/airflow_$folder$] [ERR] curl.cpp:RequestPerform(2429): ### giving up [INF] s3fs.cpp:list_bucket(2610): [path=/airflow] [INF] curl.cpp:ListBucketRequest(3268): [tpath=/airflow] [INF] curl_util.cpp:prepare_url(243): URL is https://storage.googleapis.com/bucket-name?delimiter=/&max-keys=2&prefix=airflow/ [INF] curl_util.cpp:prepare_url(276): URL changed is https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/ [INF] curl.cpp:insertV4Headers(2572): computing signature [GET] [/] [delimiter=/&max-keys=2&prefix=airflow/] [] [INF] curl_util.cpp:url_to_host(320): url is https://storage.googleapis.com [CURL DBG] * Rebuilt URL to: https://bucket-name.storage.googleapis.com/?delimiter=/&max-keys=2&prefix=airflow/ [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#16) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 16 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=6][url=https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/][path=/airflow] [CURL DBG] * Rebuilt URL to: https://bucket-name.storage.googleapis.com/?delimiter=/&max-keys=2&prefix=airflow/ [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#17) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 17 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=6][url=https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/][path=/airflow] [CURL DBG] * Rebuilt URL to: https://bucket-name.storage.googleapis.com/?delimiter=/&max-keys=2&prefix=airflow/ [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#18) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 18 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=6][url=https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/][path=/airflow] [CURL DBG] * Rebuilt URL to: https://bucket-name.storage.googleapis.com/?delimiter=/&max-keys=2&prefix=airflow/ [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#19) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 19 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=6][url=https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/][path=/airflow] [CURL DBG] * Rebuilt URL to: https://bucket-name.storage.googleapis.com/?delimiter=/&max-keys=2&prefix=airflow/ [CURL DBG] * Hostname bucket-name.storage.googleapis.com was found in DNS cache [CURL DBG] * Trying 172.217.204.128... [CURL DBG] * TCP_NODELAY set [CURL DBG] * Connected to bucket-name.storage.googleapis.com (172.217.204.128) port 443 (#20) [CURL DBG] * found 127 certificates in /etc/ssl/certs/ca-certificates.crt [CURL DBG] * found 383 certificates in /etc/ssl/certs [CURL DBG] * SSL re-using session ID [CURL DBG] * gnutls_handshake() failed: An unexpected TLS packet was received. [CURL DBG] * stopped the pause stream! [CURL DBG] * Closing connection 20 [ERR] curl.cpp:RequestPerform(2346): ### CURLE_SSL_CONNECT_ERROR [INF] curl.cpp:RequestPerform(2411): ### retrying... [INF] curl.cpp:RemakeHandle(2034): Retry request. [type=6][url=https://bucket-name.storage.googleapis.com?delimiter=/&max-keys=2&prefix=airflow/][path=/airflow] [ERR] curl.cpp:RequestPerform(2429): ### giving up [ERR] s3fs.cpp:list_bucket(2646): ListBucketRequest returns with error. [ERR] s3fs.cpp:directory_empty(1081): list_bucket returns error. unique: 4, error: -5 (Input/output error), outsize: 16 ``` ### Details about issue About an hour after mounting, when trying to list files or directories, I get this error
kerem 2026-03-04 01:48:35 +03:00
  • closed this issue
  • added the
    need info
    label
Author
Owner

@gaul commented on GitHub (Oct 19, 2020):

I left a GCS bucket mounted overnight and could not reproduce these symptoms:

[CURL DBG] * Too old connection (39755 seconds), disconnect it
[CURL DBG] * Connection 0 seems to be dead!
[CURL DBG] * Closing connection 0
[CURL DBG] * Hostname in DNS cache was stale, zapped
[CURL DBG] *   Trying 172.217.27.80:443...
[CURL DBG] * Connected to gaultest.storage.googleapis.com (172.217.27.80) port 443 (#1)
[CURL DBG] * successfully set certificate verify locations:
[CURL DBG] *   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
[CURL DBG] * SSL re-using session ID
[CURL DBG] * TLSv1.3 (OUT), TLS handshake, Client hello (1):
[CURL DBG] * TLSv1.3 (IN), TLS handshake, Server hello (2):
[CURL DBG] * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
[CURL DBG] * TLSv1.3 (IN), TLS handshake, Finished (20):
[CURL DBG] * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
[CURL DBG] * TLSv1.3 (OUT), TLS handshake, Finished (20):
[CURL DBG] * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

Can you try the flag -o nosscache as a workaround? This may help narrow down the issue. Also can you make sure that your system clock is accurate, e.g., ntpd is running? Did you suspend your laptop or some similar operation?

<!-- gh-comment-id:712503949 --> @gaul commented on GitHub (Oct 19, 2020): I left a GCS bucket mounted overnight and could not reproduce these symptoms: ``` [CURL DBG] * Too old connection (39755 seconds), disconnect it [CURL DBG] * Connection 0 seems to be dead! [CURL DBG] * Closing connection 0 [CURL DBG] * Hostname in DNS cache was stale, zapped [CURL DBG] * Trying 172.217.27.80:443... [CURL DBG] * Connected to gaultest.storage.googleapis.com (172.217.27.80) port 443 (#1) [CURL DBG] * successfully set certificate verify locations: [CURL DBG] * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none [CURL DBG] * SSL re-using session ID [CURL DBG] * TLSv1.3 (OUT), TLS handshake, Client hello (1): [CURL DBG] * TLSv1.3 (IN), TLS handshake, Server hello (2): [CURL DBG] * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): [CURL DBG] * TLSv1.3 (IN), TLS handshake, Finished (20): [CURL DBG] * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): [CURL DBG] * TLSv1.3 (OUT), TLS handshake, Finished (20): [CURL DBG] * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 ``` Can you try the flag `-o nosscache` as a workaround? This may help narrow down the issue. Also can you make sure that your system clock is accurate, e.g., `ntpd` is running? Did you suspend your laptop or some similar operation?
Author
Owner

@afeller-quotient commented on GitHub (Oct 20, 2020):

ntpd is running. It's not laptop, it's an instance in GCE
Will check with -o nosscache flag, thanks!

<!-- gh-comment-id:712591648 --> @afeller-quotient commented on GitHub (Oct 20, 2020): `ntpd` is running. It's not laptop, it's an instance in GCE Will check with `-o nosscache` flag, thanks!
Author
Owner

@afeller-quotient commented on GitHub (Oct 20, 2020):

@gaul looks like it was my fail during build :( works now. Thanks!

<!-- gh-comment-id:712683175 --> @afeller-quotient commented on GitHub (Oct 20, 2020): @gaul looks like it was my fail during build :( works now. Thanks!
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#763
No description provided.