[GH-ISSUE #2036] Object metadata update with POST and X-Http-Method-Override: PATCH is not processed as a PATCH #243

New issue

Closed

opened 2026-03-03 12:09:23 +03:00 by kerem · 0 comments

kerem commented

2026-03-03 12:09:23 +03:00

Owner

Originally created by @BenWhitehead on GitHub (Sep 30, 2025).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/2036

Issuing a PATCH object update as a POST with X-Http-Method-Override: PATCH is not in fact processed as a patch. In the following reproducer if I change to using the PATCH verb everything works as expected.

This issue was brought to my attention when I received a bug for my client https://github.com/googleapis/java-storage/issues/3301 claiming a regression even tough all tests pass against GCS itself. And later it being noted this server is being used in their tests https://github.com/googleapis/java-storage/issues/3301#issuecomment-3336967509.

Unfortunately it is not possible for me to change my client to use the PATCH verb, as we are calling code that is generated by a separate process which can't change their generator due to historical compatibility needs.

I see that github.com/fsouza/fake-gcs-server@1e417de4a4/fakestorage/server.go (L275) has a route that should be handling this already, but I'm not sure why it isn't -- and I don't know enough go to dig deeper at this time.

Reproducer

Start server

Start server by running (I wasn't able to test v1.50.3 due to some docker pull errors)

docker run -it --rm --net=host fsouza/fake-gcs-server:1.50.2 -scheme http -port 8443

Run script

Then run the following reproducer script that uses xh to issue requests

#!/bin/bash
set -o errexit -o nounset -o pipefail

function rand_name() {
  dd ibs=10 count=1 if=/dev/urandom 2>/dev/null | base32 | awk '{print tolower($0)}'
}

_BASE_URL="http://localhost:8443"
_BUCKET="b$(rand_name)"

function assert() {
  if [ "$1" != "$2" ]; then
    err "assertion '$1' == '$2' did not hold"
  fi
}

function main() {
  msg "create a bucket"
  xh POST --print=Hh ${_BASE_URL}/storage/v1/b name="$_BUCKET"

  msg "create the object without any metadata"
  local o_name="o_$(rand_name)"
  { upload_id=$(xh --print=Hh POST "${_BASE_URL}/upload/storage/v1/b/${_BUCKET}/o" name=="${o_name}" uploadType==resumable | tee /dev/fd/3 | grep -i 'Location:' | cut -b11-) ; } 3>&1
  dd ibs=10 count=1 if=/dev/urandom 2>/dev/null | xh --print=b PUT "$upload_id" Content-Range:'bytes 0-9/10'

  msg "update the objects metadata to include two new keys"
  echo "{\"metadata\":{\"k1\":\"v1\",\"k2\":\"v2\"},\"bucket\":\"${_BUCKET}\",\"name\":\"${o_name}\"}" \
    | xh POST --print=HhBb "${_BASE_URL}/storage/v1/b/${_BUCKET}/o/${o_name}" \
      x-http-method-override:PATCH

  msg "update the objects metadata to include one new key and change the value of one existing key"
  echo "{\"metadata\":{\"k1\":\"v1_1\",\"k3\":\"v3\"},\"bucket\":\"${_BUCKET}\",\"name\":\"${o_name}\"}" \
    | xh POST --print=HhBb "${_BASE_URL}/storage/v1/b/${_BUCKET}/o/${o_name}" \
      x-http-method-override:PATCH

  local current=$(xh GET --print=b "${_BASE_URL}/storage/v1/b/${_BUCKET}/o/${o_name}")

  assert "v1_1" "$(echo $current | jq -r '.metadata.k1')"
  assert "v3" "$(echo $current | jq -r '.metadata.k3')"
  assert "v2" "$(echo $current | jq -r '.metadata.k2')"

}

function now { date +"%Y-%m-%d %H:%M:%S" | tr -d '\n' ;}
function msg { println "$(now) $*" >&2 ;}
function err { local x=$? ; msg "$*" ; return $(( $x == 0 ? 1 : $x )) ;}
function println { printf '%s\n' "$*" ;}
function print { printf '%s ' "$(now) $*" ;}

main

Results

2025-09-30 18:53:04 assertion 'v2' == 'null' did not hold

full script output

2025-09-30 18:53:04 create a bucket
POST /storage/v1/b HTTP/1.1
Accept-Encoding: gzip, deflate, br
User-Agent: xh/0.17.0
Connection: keep-alive
Accept: application/json, */*;q=0.5
Content-Type: application/json
Content-Length: 28
Host: localhost:8443

HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 30 Sep 2025 18:53:04 GMT
Content-Length: 357

2025-09-30 18:53:04 create the object without any metadata
POST /upload/storage/v1/b/byqj37d6dexarzo7x/o?name=o_2v6urcp54xbflwuf&uploadType=resumable HTTP/1.1
Accept-Encoding: gzip, deflate, br
User-Agent: xh/0.17.0
Connection: keep-alive
Accept: */*
Host: localhost:8443

HTTP/1.1 200 OK
Content-Type: application/json
Location: http://0.0.0.0:8443/upload/storage/v1/b/byqj37d6dexarzo7x/o?uploadType=resumable&name=o_2v6urcp54xbflwuf&upload_id=9094348117931a8544ae1e82d9a1da2c
Date: Tue, 30 Sep 2025 18:53:04 GMT
Content-Length: 585

{"kind":"storage#object","name":"o_2v6urcp54xbflwuf","id":"byqj37d6dexarzo7x/o_2v6urcp54xbflwuf","bucket":"byqj37d6dexarzo7x","size":"10","contentType":"application/json","crc32c":"bynMdA==","acl":[{"bucket":"byqj37d6dexarzo7x","entity":"projectOwner-test-project","etag":"RVRhZw==","kind":"storage#objectAccessControl","object":"o_2v6urcp54xbflwuf","projectTeam":{},"role":"OWNER"}],"md5Hash":"k5bVYTzQQ6ffYPda5d+lqw==","etag":"k5bVYTzQQ6ffYPda5d+lqw==","storageClass":"STANDARD","timeCreated":"2025-09-30T18:53:04.648684Z","timeStorageClassUpdated":"2025-09-30T18:53:04.64869Z","updated":"2025-09-30T18:53:04.64869Z","generation":"1759258384648692","selfLink":"http://0.0.0.0:8443/storage/v1/b/byqj37d6dexarzo7x/o/o_2v6urcp54xbflwuf","mediaLink":"http://0.0.0.0:8443/download/storage/v1/b/byqj37d6dexarzo7x/o/o_2v6urcp54xbflwuf?alt=media","metageneration":"1"}
2025-09-30 18:53:04 update the objects metadata to include two new keys
POST /storage/v1/b/byqj37d6dexarzo7x/o/o_2v6urcp54xbflwuf HTTP/1.1
Accept-Encoding: gzip, deflate, br
User-Agent: xh/0.17.0
Connection: keep-alive
Accept: application/json, */*;q=0.5
Content-Type: application/json
X-Http-Method-Override: PATCH
Content-Length: 92
Host: localhost:8443

{"metadata":{"k1":"v1","k2":"v2"},"bucket":"byqj37d6dexarzo7x","name":"o_2v6urcp54xbflwuf"}


HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 30 Sep 2025 18:53:04 GMT
Content-Length: 571

{"bucket":"byqj37d6dexarzo7x","name":"o_2v6urcp54xbflwuf","size":"10","contentType":"application/json","contentEncoding":"","contentDisposition":"","crc32c":"bynMdA==","md5Hash":"k5bVYTzQQ6ffYPda5d+lqw==","etag":"k5bVYTzQQ6ffYPda5d+lqw==","acl":[{"entity":"projectOwner-test-project","entityId":"","role":"OWNER","domain":"","email":"","projectTeam":null}],"created":"2025-09-30T18:53:04.648684Z","updated":"2025-09-30T18:53:04.64869Z","deleted":"0001-01-01T00:00:00Z","customTime":"0001-01-01T00:00:00Z","generation":"1759258384675628","metadata":{"k1":"v1","k2":"v2"}}
2025-09-30 18:53:04 update the objects metadata to include one new key and change the value of one existing key
POST /storage/v1/b/byqj37d6dexarzo7x/o/o_2v6urcp54xbflwuf HTTP/1.1
Accept-Encoding: gzip, deflate, br
User-Agent: xh/0.17.0
Connection: keep-alive
Accept: application/json, */*;q=0.5
Content-Type: application/json
X-Http-Method-Override: PATCH
Content-Length: 94
Host: localhost:8443

{"metadata":{"k1":"v1_1","k3":"v3"},"bucket":"byqj37d6dexarzo7x","name":"o_2v6urcp54xbflwuf"}


HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 30 Sep 2025 18:53:04 GMT
Content-Length: 573

{"bucket":"byqj37d6dexarzo7x","name":"o_2v6urcp54xbflwuf","size":"10","contentType":"application/json","contentEncoding":"","contentDisposition":"","crc32c":"bynMdA==","md5Hash":"k5bVYTzQQ6ffYPda5d+lqw==","etag":"k5bVYTzQQ6ffYPda5d+lqw==","acl":[{"entity":"projectOwner-test-project","entityId":"","role":"OWNER","domain":"","email":"","projectTeam":null}],"created":"2025-09-30T18:53:04.648684Z","updated":"2025-09-30T18:53:04.64869Z","deleted":"0001-01-01T00:00:00Z","customTime":"0001-01-01T00:00:00Z","generation":"1759258384701414","metadata":{"k1":"v1_1","k3":"v3"}}
2025-09-30 18:53:04 assertion 'v2' == 'null' did not hold

Originally created by @BenWhitehead on GitHub (Sep 30, 2025). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/2036 Issuing a PATCH object update as a `POST` with `X-Http-Method-Override: PATCH` is not in fact processed as a patch. In the following reproducer if I change to using the `PATCH` verb everything works as expected. This issue was brought to my attention when I received a bug for my client https://github.com/googleapis/java-storage/issues/3301 claiming a regression even tough all tests pass against GCS itself. And later it being noted this server is being used in their tests https://github.com/googleapis/java-storage/issues/3301#issuecomment-3336967509. Unfortunately it is not possible for me to change my client to use the PATCH verb, as we are calling code that is generated by a separate process which can't change their generator due to historical compatibility needs. I see that https://github.com/fsouza/fake-gcs-server/blob/1e417de4a4c45a78692c6637e349c135d33dcc22/fakestorage/server.go#L275 has a route that should be handling this already, but I'm not sure why it isn't -- and I don't know enough go to dig deeper at this time. #### Reproducer ##### Start server Start server by running (I wasn't able to test v1.50.3 due to some docker pull errors) ```bash docker run -it --rm --net=host fsouza/fake-gcs-server:1.50.2 -scheme http -port 8443 ``` ##### Run script Then run the following reproducer script that uses [`xh`](https://github.com/ducaale/xh) to issue requests ```bash #!/bin/bash set -o errexit -o nounset -o pipefail function rand_name() { dd ibs=10 count=1 if=/dev/urandom 2>/dev/null | base32 | awk '{print tolower($0)}' } _BASE_URL="http://localhost:8443" _BUCKET="b$(rand_name)" function assert() { if [ "$1" != "$2" ]; then err "assertion '$1' == '$2' did not hold" fi } function main() { msg "create a bucket" xh POST --print=Hh ${_BASE_URL}/storage/v1/b name="$_BUCKET" msg "create the object without any metadata" local o_name="o_$(rand_name)" { upload_id=$(xh --print=Hh POST "${_BASE_URL}/upload/storage/v1/b/${_BUCKET}/o" name=="${o_name}" uploadType==resumable | tee /dev/fd/3 | grep -i 'Location:' | cut -b11-) ; } 3>&1 dd ibs=10 count=1 if=/dev/urandom 2>/dev/null | xh --print=b PUT "$upload_id" Content-Range:'bytes 0-9/10' msg "update the objects metadata to include two new keys" echo "{\"metadata\":{\"k1\":\"v1\",\"k2\":\"v2\"},\"bucket\":\"${_BUCKET}\",\"name\":\"${o_name}\"}" \ | xh POST --print=HhBb "${_BASE_URL}/storage/v1/b/${_BUCKET}/o/${o_name}" \ x-http-method-override:PATCH msg "update the objects metadata to include one new key and change the value of one existing key" echo "{\"metadata\":{\"k1\":\"v1_1\",\"k3\":\"v3\"},\"bucket\":\"${_BUCKET}\",\"name\":\"${o_name}\"}" \ | xh POST --print=HhBb "${_BASE_URL}/storage/v1/b/${_BUCKET}/o/${o_name}" \ x-http-method-override:PATCH local current=$(xh GET --print=b "${_BASE_URL}/storage/v1/b/${_BUCKET}/o/${o_name}") assert "v1_1" "$(echo $current | jq -r '.metadata.k1')" assert "v3" "$(echo $current | jq -r '.metadata.k3')" assert "v2" "$(echo $current | jq -r '.metadata.k2')" } function now { date +"%Y-%m-%d %H:%M:%S" | tr -d '\n' ;} function msg { println "$(now) $*" >&2 ;} function err { local x=$? ; msg "$*" ; return $(( $x == 0 ? 1 : $x )) ;} function println { printf '%s\n' "$*" ;} function print { printf '%s ' "$(now) $*" ;} main ``` ##### Results ``` 2025-09-30 18:53:04 assertion 'v2' == 'null' did not hold ``` <details> <summary>full script output</summary> ``` 2025-09-30 18:53:04 create a bucket POST /storage/v1/b HTTP/1.1 Accept-Encoding: gzip, deflate, br User-Agent: xh/0.17.0 Connection: keep-alive Accept: application/json, */*;q=0.5 Content-Type: application/json Content-Length: 28 Host: localhost:8443 HTTP/1.1 200 OK Content-Type: application/json Date: Tue, 30 Sep 2025 18:53:04 GMT Content-Length: 357 2025-09-30 18:53:04 create the object without any metadata POST /upload/storage/v1/b/byqj37d6dexarzo7x/o?name=o_2v6urcp54xbflwuf&uploadType=resumable HTTP/1.1 Accept-Encoding: gzip, deflate, br User-Agent: xh/0.17.0 Connection: keep-alive Accept: */* Host: localhost:8443 HTTP/1.1 200 OK Content-Type: application/json Location: http://0.0.0.0:8443/upload/storage/v1/b/byqj37d6dexarzo7x/o?uploadType=resumable&name=o_2v6urcp54xbflwuf&upload_id=9094348117931a8544ae1e82d9a1da2c Date: Tue, 30 Sep 2025 18:53:04 GMT Content-Length: 585 {"kind":"storage#object","name":"o_2v6urcp54xbflwuf","id":"byqj37d6dexarzo7x/o_2v6urcp54xbflwuf","bucket":"byqj37d6dexarzo7x","size":"10","contentType":"application/json","crc32c":"bynMdA==","acl":[{"bucket":"byqj37d6dexarzo7x","entity":"projectOwner-test-project","etag":"RVRhZw==","kind":"storage#objectAccessControl","object":"o_2v6urcp54xbflwuf","projectTeam":{},"role":"OWNER"}],"md5Hash":"k5bVYTzQQ6ffYPda5d+lqw==","etag":"k5bVYTzQQ6ffYPda5d+lqw==","storageClass":"STANDARD","timeCreated":"2025-09-30T18:53:04.648684Z","timeStorageClassUpdated":"2025-09-30T18:53:04.64869Z","updated":"2025-09-30T18:53:04.64869Z","generation":"1759258384648692","selfLink":"http://0.0.0.0:8443/storage/v1/b/byqj37d6dexarzo7x/o/o_2v6urcp54xbflwuf","mediaLink":"http://0.0.0.0:8443/download/storage/v1/b/byqj37d6dexarzo7x/o/o_2v6urcp54xbflwuf?alt=media","metageneration":"1"} 2025-09-30 18:53:04 update the objects metadata to include two new keys POST /storage/v1/b/byqj37d6dexarzo7x/o/o_2v6urcp54xbflwuf HTTP/1.1 Accept-Encoding: gzip, deflate, br User-Agent: xh/0.17.0 Connection: keep-alive Accept: application/json, */*;q=0.5 Content-Type: application/json X-Http-Method-Override: PATCH Content-Length: 92 Host: localhost:8443 {"metadata":{"k1":"v1","k2":"v2"},"bucket":"byqj37d6dexarzo7x","name":"o_2v6urcp54xbflwuf"} HTTP/1.1 200 OK Content-Type: application/json Date: Tue, 30 Sep 2025 18:53:04 GMT Content-Length: 571 {"bucket":"byqj37d6dexarzo7x","name":"o_2v6urcp54xbflwuf","size":"10","contentType":"application/json","contentEncoding":"","contentDisposition":"","crc32c":"bynMdA==","md5Hash":"k5bVYTzQQ6ffYPda5d+lqw==","etag":"k5bVYTzQQ6ffYPda5d+lqw==","acl":[{"entity":"projectOwner-test-project","entityId":"","role":"OWNER","domain":"","email":"","projectTeam":null}],"created":"2025-09-30T18:53:04.648684Z","updated":"2025-09-30T18:53:04.64869Z","deleted":"0001-01-01T00:00:00Z","customTime":"0001-01-01T00:00:00Z","generation":"1759258384675628","metadata":{"k1":"v1","k2":"v2"}} 2025-09-30 18:53:04 update the objects metadata to include one new key and change the value of one existing key POST /storage/v1/b/byqj37d6dexarzo7x/o/o_2v6urcp54xbflwuf HTTP/1.1 Accept-Encoding: gzip, deflate, br User-Agent: xh/0.17.0 Connection: keep-alive Accept: application/json, */*;q=0.5 Content-Type: application/json X-Http-Method-Override: PATCH Content-Length: 94 Host: localhost:8443 {"metadata":{"k1":"v1_1","k3":"v3"},"bucket":"byqj37d6dexarzo7x","name":"o_2v6urcp54xbflwuf"} HTTP/1.1 200 OK Content-Type: application/json Date: Tue, 30 Sep 2025 18:53:04 GMT Content-Length: 573 {"bucket":"byqj37d6dexarzo7x","name":"o_2v6urcp54xbflwuf","size":"10","contentType":"application/json","contentEncoding":"","contentDisposition":"","crc32c":"bynMdA==","md5Hash":"k5bVYTzQQ6ffYPda5d+lqw==","etag":"k5bVYTzQQ6ffYPda5d+lqw==","acl":[{"entity":"projectOwner-test-project","entityId":"","role":"OWNER","domain":"","email":"","projectTeam":null}],"created":"2025-09-30T18:53:04.648684Z","updated":"2025-09-30T18:53:04.64869Z","deleted":"0001-01-01T00:00:00Z","customTime":"0001-01-01T00:00:00Z","generation":"1759258384701414","metadata":{"k1":"v1_1","k3":"v3"}} 2025-09-30 18:53:04 assertion 'v2' == 'null' did not hold ``` </details>