[GH-ISSUE #537] Signed URLs: Cannot sign data with client_email

kerem commented

2026-03-03 12:08:18 +03:00

Owner

Originally created by @psteinroe on GitHub (Jul 16, 2021).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/537

Hi,

I am having troubles with signed urls using node. When using the following config,

new Storage({
    apiEndpoint: `https://${process.env.GCS_EMULATOR_HOST}:${process.env.GCS_EMULATOR_PORT}`,
    projectId: process.env.GCLOUD_PROJECT_ID,
});

generating a signed url with

const [url] = await file.getSignedUrl({
      version: 'v4',
      action: 'write',
      expires: expiresAt,
});

fails with Cannot sign data without 'client_email'.

When I add a fake service account with

new Storage({
    apiEndpoint: `https://${process.env.GCS_EMULATOR_HOST}:${process.env.GCS_EMULATOR_PORT}`,
    projectId: process.env.GCLOUD_PROJECT_ID,
    credentials: {
              type: 'service_account',
              private_key:
                '-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDY3E8o1NEFcjMM\nHW/5ZfFJw29/8NEqpViNjQIx95Xx5KDtJ+nWn9+OW0uqsSqKlKGhAdAo+Q6bjx2c\nuXVsXTu7XrZUY5Kltvj94DvUa1wjNXs606r/RxWTJ58bfdC+gLLxBfGnB6CwK0YQ\nxnfpjNbkUfVVzO0MQD7UP0Hl5ZcY0Puvxd/yHuONQn/rIAieTHH1pqgW+zrH/y3c\n59IGThC9PPtugI9ea8RSnVj3PWz1bX2UkCDpy9IRh9LzJLaYYX9RUd7++dULUlat\nAaXBh1U6emUDzhrIsgApjDVtimOPbmQWmX1S60mqQikRpVYZ8u+NDD+LNw+/Eovn\nxCj2Y3z1AgMBAAECggEAWDBzoqO1IvVXjBA2lqId10T6hXmN3j1ifyH+aAqK+FVl\nGjyWjDj0xWQcJ9ync7bQ6fSeTeNGzP0M6kzDU1+w6FgyZqwdmXWI2VmEizRjwk+/\n/uLQUcL7I55Dxn7KUoZs/rZPmQDxmGLoue60Gg6z3yLzVcKiDc7cnhzhdBgDc8vd\nQorNAlqGPRnm3EqKQ6VQp6fyQmCAxrr45kspRXNLddat3AMsuqImDkqGKBmF3Q1y\nxWGe81LphUiRqvqbyUlh6cdSZ8pLBpc9m0c3qWPKs9paqBIvgUPlvOZMqec6x4S6\nChbdkkTRLnbsRr0Yg/nDeEPlkhRBhasXpxpMUBgPywKBgQDs2axNkFjbU94uXvd5\nznUhDVxPFBuxyUHtsJNqW4p/ujLNimGet5E/YthCnQeC2P3Ym7c3fiz68amM6hiA\nOnW7HYPZ+jKFnefpAtjyOOs46AkftEg07T9XjwWNPt8+8l0DYawPoJgbM5iE0L2O\nx8TU1Vs4mXc+ql9F90GzI0x3VwKBgQDqZOOqWw3hTnNT07Ixqnmd3dugV9S7eW6o\nU9OoUgJB4rYTpG+yFqNqbRT8bkx37iKBMEReppqonOqGm4wtuRR6LSLlgcIU9Iwx\nyfH12UWqVmFSHsgZFqM/cK3wGev38h1WBIOx3/djKn7BdlKVh8kWyx6uC8bmV+E6\nOoK0vJD6kwKBgHAySOnROBZlqzkiKW8c+uU2VATtzJSydrWm0J4wUPJifNBa/hVW\ndcqmAzXC9xznt5AVa3wxHBOfyKaE+ig8CSsjNyNZ3vbmr0X04FoV1m91k2TeXNod\njMTobkPThaNm4eLJMN2SQJuaHGTGERWC0l3T18t+/zrDMDCPiSLX1NAvAoGBAN1T\nVLJYdjvIMxf1bm59VYcepbK7HLHFkRq6xMJMZbtG0ryraZjUzYvB4q4VjHk2UDiC\nlhx13tXWDZH7MJtABzjyg+AI7XWSEQs2cBXACos0M4Myc6lU+eL+iA+OuoUOhmrh\nqmT8YYGu76/IBWUSqWuvcpHPpwl7871i4Ga/I3qnAoGBANNkKAcMoeAbJQK7a/Rn\nwPEJB+dPgNDIaboAsh1nZhVhN5cvdvCWuEYgOGCPQLYQF0zmTLcM+sVxOYgfy8mV\nfbNgPgsP5xmu6dw2COBKdtozw0HrWSRjACd1N4yGu75+wPCcX/gQarcjRcXXZeEa\nNtBLSfcqPULqD+h7br9lEJio\n-----END PRIVATE KEY-----\n',
              client_email: '123-abc@developer.gserviceaccount.com',
            },
});

it returns an upload url looking something like https://storage.googleapis.com/network-test/File/4ce53b09-648c-4d95-8408-db1acd967b37/4ce53b09-648c-4d95-8408-db1acd967b37.jpg?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=123-abc%40developer.gserviceaccount.com%2F20210716%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20210716T140253Z&X-Goog-Expires=901&X-Goog-SignedHeaders=host&X-Goog-Signature=2f0287af9ed53efbd95d4090471ce8702a06b0ecfd8a39b2c771a32c3c865be7053e919fa0f1e8e8ac7026ed729b0c871b09cfc08c4337c39fdefbb300df517e46eeec6828ec23a24b27fd1e9ada30604aca2ce067b83ff5ebfbd1a7365efdfb7d2c3c428980f94a40121853493343216fa82e6f6b512897cb9475d5743fb45a5f278c135cd0301adc912d4c6ac6b4bfae3834628dc8705e38e3032f738f3ddc61ad564c7e3614914d2f260a2cb84b63230347265b86d193d40000590802823236c1ba2b269fa64da1452a1ceae56f74089421815da7c3ced08df73df41ec5821e9e83a9ee1a33aa0015cde4b1656e0ade8716b4b3a9d603ad933cebc03b5ba6

and then the upload fails with Request failed with status code 403.

Does anyone has an example for uploading to fake-gcs using a signed url?

Originally created by @psteinroe on GitHub (Jul 16, 2021). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/537 Hi, I am having troubles with signed urls using node. When using the following config, ``` new Storage({ apiEndpoint: `https://${process.env.GCS_EMULATOR_HOST}:${process.env.GCS_EMULATOR_PORT}`, projectId: process.env.GCLOUD_PROJECT_ID, }); ``` generating a signed url with ``` const [url] = await file.getSignedUrl({ version: 'v4', action: 'write', expires: expiresAt, }); ``` fails with `Cannot sign data without 'client_email'`. When I add a fake service account with ``` new Storage({ apiEndpoint: `https://${process.env.GCS_EMULATOR_HOST}:${process.env.GCS_EMULATOR_PORT}`, projectId: process.env.GCLOUD_PROJECT_ID, credentials: { type: 'service_account', private_key: '-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDY3E8o1NEFcjMM\nHW/5ZfFJw29/8NEqpViNjQIx95Xx5KDtJ+nWn9+OW0uqsSqKlKGhAdAo+Q6bjx2c\nuXVsXTu7XrZUY5Kltvj94DvUa1wjNXs606r/RxWTJ58bfdC+gLLxBfGnB6CwK0YQ\nxnfpjNbkUfVVzO0MQD7UP0Hl5ZcY0Puvxd/yHuONQn/rIAieTHH1pqgW+zrH/y3c\n59IGThC9PPtugI9ea8RSnVj3PWz1bX2UkCDpy9IRh9LzJLaYYX9RUd7++dULUlat\nAaXBh1U6emUDzhrIsgApjDVtimOPbmQWmX1S60mqQikRpVYZ8u+NDD+LNw+/Eovn\nxCj2Y3z1AgMBAAECggEAWDBzoqO1IvVXjBA2lqId10T6hXmN3j1ifyH+aAqK+FVl\nGjyWjDj0xWQcJ9ync7bQ6fSeTeNGzP0M6kzDU1+w6FgyZqwdmXWI2VmEizRjwk+/\n/uLQUcL7I55Dxn7KUoZs/rZPmQDxmGLoue60Gg6z3yLzVcKiDc7cnhzhdBgDc8vd\nQorNAlqGPRnm3EqKQ6VQp6fyQmCAxrr45kspRXNLddat3AMsuqImDkqGKBmF3Q1y\nxWGe81LphUiRqvqbyUlh6cdSZ8pLBpc9m0c3qWPKs9paqBIvgUPlvOZMqec6x4S6\nChbdkkTRLnbsRr0Yg/nDeEPlkhRBhasXpxpMUBgPywKBgQDs2axNkFjbU94uXvd5\nznUhDVxPFBuxyUHtsJNqW4p/ujLNimGet5E/YthCnQeC2P3Ym7c3fiz68amM6hiA\nOnW7HYPZ+jKFnefpAtjyOOs46AkftEg07T9XjwWNPt8+8l0DYawPoJgbM5iE0L2O\nx8TU1Vs4mXc+ql9F90GzI0x3VwKBgQDqZOOqWw3hTnNT07Ixqnmd3dugV9S7eW6o\nU9OoUgJB4rYTpG+yFqNqbRT8bkx37iKBMEReppqonOqGm4wtuRR6LSLlgcIU9Iwx\nyfH12UWqVmFSHsgZFqM/cK3wGev38h1WBIOx3/djKn7BdlKVh8kWyx6uC8bmV+E6\nOoK0vJD6kwKBgHAySOnROBZlqzkiKW8c+uU2VATtzJSydrWm0J4wUPJifNBa/hVW\ndcqmAzXC9xznt5AVa3wxHBOfyKaE+ig8CSsjNyNZ3vbmr0X04FoV1m91k2TeXNod\njMTobkPThaNm4eLJMN2SQJuaHGTGERWC0l3T18t+/zrDMDCPiSLX1NAvAoGBAN1T\nVLJYdjvIMxf1bm59VYcepbK7HLHFkRq6xMJMZbtG0ryraZjUzYvB4q4VjHk2UDiC\nlhx13tXWDZH7MJtABzjyg+AI7XWSEQs2cBXACos0M4Myc6lU+eL+iA+OuoUOhmrh\nqmT8YYGu76/IBWUSqWuvcpHPpwl7871i4Ga/I3qnAoGBANNkKAcMoeAbJQK7a/Rn\nwPEJB+dPgNDIaboAsh1nZhVhN5cvdvCWuEYgOGCPQLYQF0zmTLcM+sVxOYgfy8mV\nfbNgPgsP5xmu6dw2COBKdtozw0HrWSRjACd1N4yGu75+wPCcX/gQarcjRcXXZeEa\nNtBLSfcqPULqD+h7br9lEJio\n-----END PRIVATE KEY-----\n', client_email: '123-abc@developer.gserviceaccount.com', }, }); ``` it returns an upload url looking something like `https://storage.googleapis.com/network-test/File/4ce53b09-648c-4d95-8408-db1acd967b37/4ce53b09-648c-4d95-8408-db1acd967b37.jpg?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=123-abc%40developer.gserviceaccount.com%2F20210716%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20210716T140253Z&X-Goog-Expires=901&X-Goog-SignedHeaders=host&X-Goog-Signature=2f0287af9ed53efbd95d4090471ce8702a06b0ecfd8a39b2c771a32c3c865be7053e919fa0f1e8e8ac7026ed729b0c871b09cfc08c4337c39fdefbb300df517e46eeec6828ec23a24b27fd1e9ada30604aca2ce067b83ff5ebfbd1a7365efdfb7d2c3c428980f94a40121853493343216fa82e6f6b512897cb9475d5743fb45a5f278c135cd0301adc912d4c6ac6b4bfae3834628dc8705e38e3032f738f3ddc61ad564c7e3614914d2f260a2cb84b63230347265b86d193d40000590802823236c1ba2b269fa64da1452a1ceae56f74089421815da7c3ced08df73df41ec5821e9e83a9ee1a33aa0015cde4b1656e0ade8716b4b3a9d603ad933cebc03b5ba6` and then the upload fails with `Request failed with status code 403`. Does anyone has an example for uploading to fake-gcs using a signed url?

kerem

2026-03-03 12:08:18 +03:00

closed this issue
added the
question
label

kerem commented

2026-03-03 12:08:19 +03:00

Author

Owner

@BruAndreo commented on GitHub (Sep 16, 2021):

Hi @steinroe ,
How did you solve this? I have the same problem.

@BruAndreo commented on GitHub (Sep 16, 2021): Hi @steinroe , How did you solve this? I have the same problem.

kerem commented

2026-03-03 12:08:19 +03:00

Author

Owner

@psteinroe commented on GitHub (Sep 16, 2021):

Unfortunately I did not solve it yet, but I am still highly interested in doing so.

@psteinroe commented on GitHub (Sep 16, 2021): Unfortunately I did not solve it yet, but I am still highly interested in doing so.

kerem commented

2026-03-03 12:08:19 +03:00

Author

Owner

@fsouza commented on GitHub (Oct 7, 2021):

Hey @steinroe @BruAndreo, signing is a client-side only change. You'd need to manually override the host.

fake-gcs-server doesn't really care about signed URLs though, it should take and process them, but there's no way for it to do any validation of the params.

@fsouza commented on GitHub (Oct 7, 2021): Hey @steinroe @BruAndreo, signing is a client-side only change. You'd need to manually override the host. fake-gcs-server doesn't really care about signed URLs though, it should take and process them, but there's no way for it to do any validation of the params.

kerem commented

2026-03-03 12:08:19 +03:00

Author

Owner

@psteinroe commented on GitHub (Oct 7, 2021):

Thanks for the clarification @fsouza! Would you mind providing an example (preferably with a node client) to upload files to fake-gcs-server? I can't get it to work...

@psteinroe commented on GitHub (Oct 7, 2021): Thanks for the clarification @fsouza! Would you mind providing an example (preferably with a node client) to upload files to fake-gcs-server? I can't get it to work...

kerem commented

2026-03-03 12:08:19 +03:00

Author

Owner

@fsouza commented on GitHub (Oct 14, 2021):

I haven't used the nodejs much, but are you running into the same issue as #346? Or is it something different?

@fsouza commented on GitHub (Oct 14, 2021): I haven't used the nodejs much, but are you running into the same issue as #346? Or is it something different?

kerem commented

2026-03-03 12:08:19 +03:00

Author

Owner

@josmardias commented on GitHub (Oct 22, 2021):

@steinroe

Create a docker-compose.yml like this:

  services:
    app:
        build: node:16-alpine
        user: node
        ports:
            - "8080:8080"
        environment:
            CUSTOM_STORAGE_API: http://fake-storage:9199
            GCLOUD_PROJECT: test-project
        depends_on:
            - fake-storage
    fake-storage:
        image: fsouza/fake-gcs-server
        command: ["-scheme", "http", "-port", "9199", "-backend", "memory"]

then this is how you upload a file:

import { Storage } from '@google-cloud/storage'

const storage = new Storage({
  apiEndpoint: process.env.CUSTOM_STORAGE_API,
})

const bucket = storage.bucket('my-bucket')
await bucket.create()

await bucket.upload('./example.wav')

if instead you are trying to use createWriteStream remember to disable resumable, like this

const stream = bucket.file('file.mp3').createWriteStream({ resumable: false })
// write to stream

@josmardias commented on GitHub (Oct 22, 2021): @steinroe Create a docker-compose.yml like this: ```yml services: app: build: node:16-alpine user: node ports: - "8080:8080" environment: CUSTOM_STORAGE_API: http://fake-storage:9199 GCLOUD_PROJECT: test-project depends_on: - fake-storage fake-storage: image: fsouza/fake-gcs-server command: ["-scheme", "http", "-port", "9199", "-backend", "memory"] ``` then this is how you upload a file: ```js import { Storage } from '@google-cloud/storage' const storage = new Storage({ apiEndpoint: process.env.CUSTOM_STORAGE_API, }) const bucket = storage.bucket('my-bucket') await bucket.create() await bucket.upload('./example.wav') ``` if instead you are trying to use `createWriteStream` remember to disable resumable, like this ```js const stream = bucket.file('file.mp3').createWriteStream({ resumable: false }) // write to stream ```

kerem commented

2026-03-03 12:08:19 +03:00

Author

Owner

@psteinroe commented on GitHub (Oct 22, 2021):

Thanks for your inputs @josmardias @fsouza! I got it to work now. The fixes were:

setting resumable
create the bucket on app startup (without, it led to an 404 error when trying to upload)
problems with apple silicon (solved by #580)

@psteinroe commented on GitHub (Oct 22, 2021): Thanks for your inputs @josmardias @fsouza! I got it to work now. The fixes were: - setting resumable - create the bucket on app startup (without, it led to an 404 error when trying to upload) - problems with apple silicon (solved by #580)

kerem referenced this issue

2026-03-03 12:29:33 +03:00

[PR #102] [MERGED] build(deps): bump golang from 1.13.1-alpine to 1.13.3-alpine #340

Rows
Columns

[GH-ISSUE #537] Signed URLs: Cannot sign data with client_email #102