starred/fake-gcs-server

Fork 0

mirror of https://github.com/fsouza/fake-gcs-server.git synced 2026-04-25 13:45:52 +03:00

[GH-ISSUE #24] How to use the server? #2159

New issue

Closed

opened 2026-03-15 17:48:16 +03:00 by kerem · 6 comments

kerem commented

2026-03-15 17:48:16 +03:00

Owner

Originally created by @rogierlommers on GitHub (Mar 28, 2019).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/24

Hey! Nice project, appreciate it!

But I'm wondering: how can I use the server in my go apps? For example, I can use the pubsub emulator by setting the PUBSUB_EMULATOR_HOST to the address where the pubsub emulator is running. This wat the client directly connects to the one running on that port. Is there also some kind of mechanism for the Go storage SDK?

In other words: how can I point the client to the emulated server?

I tried to set the options while creating the storage client, but I can only override the endpoint. Not the hostname + port.

Originally created by @rogierlommers on GitHub (Mar 28, 2019). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/24 Hey! Nice project, appreciate it! But I'm wondering: how can I use the server in my go apps? For example, I can use the pubsub emulator by setting the `PUBSUB_EMULATOR_HOST` to the address where the pubsub emulator is running. This wat the client directly connects to the one running on that port. Is there also some kind of mechanism for the Go storage SDK? In other words: how can I point the client to the emulated server? I tried to set the options while creating the storage client, but I can only override the endpoint. Not the hostname + port. <img width="681" alt="Screenshot 2019-03-28 at 21 34 12" src="https://user-images.githubusercontent.com/3907131/55191049-6f46b480-51a1-11e9-8c8a-36d1701bbced.png">

kerem closed this issue

2026-03-15 17:48:21 +03:00

kerem commented

2026-03-15 17:48:27 +03:00

Author

Owner

@fsouza commented on GitHub (Mar 28, 2019):

Hey, you should call the Client() method on the fake server instance, that will give you a storage.Client instance pre-configured to talk to the server.

Example from the docs: https://godoc.org/github.com/fsouza/fake-gcs-server/fakestorage#example-Server-Client

Some projects that use it:

gcs-helper

production code, expects a BucketHandle: github.com/nytimes/gcs-helper@784d637554/vodmodule/mapper.go (L22)
test code, passes a BucketHandle derived from the fake server: github.com/nytimes/gcs-helper@784d637554/vodmodule/mapper_test.go (L109)

porteiro

production code, takes a storage.Client: github.com/fsouza/porteiro@57052b847a/gcs/gcs.go (L47)
test code, passes storage.Client derived from the fake server: github.com/fsouza/porteiro@57052b847a/gcs/gcs_test.go (L30)

@fsouza commented on GitHub (Mar 28, 2019): Hey, you should call the [Client()](https://godoc.org/github.com/fsouza/fake-gcs-server/fakestorage#Server.Client) method on the fake server instance, that will give you a storage.Client instance pre-configured to talk to the server. Example from the docs: https://godoc.org/github.com/fsouza/fake-gcs-server/fakestorage#example-Server-Client Some projects that use it: ### gcs-helper - production code, expects a BucketHandle: https://github.com/nytimes/gcs-helper/blob/784d63755494bff1f7a7390aef610f18267383c1/vodmodule/mapper.go#L22 - test code, passes a BucketHandle derived from the fake server: https://github.com/nytimes/gcs-helper/blob/784d63755494bff1f7a7390aef610f18267383c1/vodmodule/mapper_test.go#L109 ### porteiro - production code, takes a storage.Client: https://github.com/fsouza/porteiro/blob/57052b847ab25055a95013299776a71af230bab5/gcs/gcs.go#L47 - test code, passes storage.Client derived from the fake server: https://github.com/fsouza/porteiro/blob/57052b847ab25055a95013299776a71af230bab5/gcs/gcs_test.go#L30

kerem commented

2026-03-15 17:48:32 +03:00

Author

Owner

@rogierlommers commented on GitHub (Mar 28, 2019):

Ok, clear. This would mean that my Go code should contain something like:

"if running locally, then use this client" (coming from the client()-method. The nice part of the pubsub emulator stuff is that it's just an environment var which sets emulation ON/OFF.

@rogierlommers commented on GitHub (Mar 28, 2019): Ok, clear. This would mean that my Go code should contain something like: "if running locally, then use this client" (coming from the `client()`-method. The nice part of the pubsub emulator stuff is that it's just an environment var which sets emulation ON/OFF.

kerem commented

2026-03-15 17:48:37 +03:00

Author

Owner

@fsouza commented on GitHub (Mar 28, 2019):

Oh so this is not for automated tests? In that case, you're probably looking for something like https://github.com/teone/gc-fake-storage, that gives you a standalone process that starts a fake server that you can point to.

@fsouza commented on GitHub (Mar 28, 2019): Oh so this is not for automated tests? In that case, you're probably looking for something like https://github.com/teone/gc-fake-storage, that gives you a standalone process that starts a fake server that you can point to.

kerem commented

2026-03-15 17:48:42 +03:00

Author

Owner

@rogierlommers commented on GitHub (Mar 29, 2019):

Yeah, that's exactly what I need. But unfortunately I can't get it to work. So I did the following:

First, I started the docker container (the fake server): docker run -d --name gc-fake-storage -p 4443:4443 -v /tmp/data:/data matteoscandolo/gc-fake-storage:0.1.0

Then I tried to create a bucket using the normal Go SDK. The code is like this:

package main

import (
	"context"
	"crypto/tls"
	"log"
	"net/http"

	"cloud.google.com/go/storage"
	"github.com/sirupsen/logrus"
	"google.golang.org/api/option"
)

const (
	projectID = "rogier"
	bucket    = "bucket"
)

func main() {

	ctx := context.Background()
	insecureHTTPClient := getInsecureClient()

	client, err := storage.NewClient(ctx,
		option.WithoutAuthentication(),
		option.WithEndpoint("https://127.0.0.1:4443"),
		option.WithHTTPClient(insecureHTTPClient))
	if err != nil {
		log.Fatalf("Failed to create client: %v", err)
	}

	bucket := client.Bucket(bucket)

	if err := bucket.Create(ctx, projectID, nil); err != nil {
		logrus.Fatalf("bucket creation error: %s", err)
	}

}

func getInsecureClient() *http.Client {
	tr := &http.Transport{
		TLSClientConfig: &tls.Config{
			InsecureSkipVerify: true},
	}

	return &http.Client{Transport: tr}
}

Unfortunately this does not work. I keep receiving this error: FATA[0000] bucket creation error: googleapi: got HTTP response code 404 with body: 404 page not found. Any ideas about what I'm doing wrong?

@rogierlommers commented on GitHub (Mar 29, 2019): Yeah, that's exactly what I need. But unfortunately I can't get it to work. So I did the following: First, I started the docker container (the fake server): `docker run -d --name gc-fake-storage -p 4443:4443 -v /tmp/data:/data matteoscandolo/gc-fake-storage:0.1.0` Then I tried to create a bucket using the normal Go SDK. The code is like this: ``` package main import ( "context" "crypto/tls" "log" "net/http" "cloud.google.com/go/storage" "github.com/sirupsen/logrus" "google.golang.org/api/option" ) const ( projectID = "rogier" bucket = "bucket" ) func main() { ctx := context.Background() insecureHTTPClient := getInsecureClient() client, err := storage.NewClient(ctx, option.WithoutAuthentication(), option.WithEndpoint("https://127.0.0.1:4443"), option.WithHTTPClient(insecureHTTPClient)) if err != nil { log.Fatalf("Failed to create client: %v", err) } bucket := client.Bucket(bucket) if err := bucket.Create(ctx, projectID, nil); err != nil { logrus.Fatalf("bucket creation error: %s", err) } } func getInsecureClient() *http.Client { tr := &http.Transport{ TLSClientConfig: &tls.Config{ InsecureSkipVerify: true}, } return &http.Client{Transport: tr} } ``` Unfortunately this does not work. I keep receiving this error: `FATA[0000] bucket creation error: googleapi: got HTTP response code 404 with body: 404 page not found`. Any ideas about what I'm doing wrong?

kerem commented

2026-03-15 17:48:47 +03:00

Author

Owner

@jwhitlock commented on GitHub (Jun 5, 2019):

@rogierlommers bucket.Create would not have worked before PR #34 was merged. Now the POST to /storage/v1/b to create a bucket should work.

In gc-fake-storage, I believe I continued to get 404s unless the directory /storage/ was created first. This can be created as a side effect of providing a /data mount. A single/data/first_bucket/first_object.txt would create the /storage folder, a bucket first_bucket, and a blob / object first_object.txt in that bucket. I've submitted https://github.com/teone/gc-fake-storage/pull/3 to fix that.

@jwhitlock commented on GitHub (Jun 5, 2019): @rogierlommers ``bucket.Create`` would not have worked before PR #34 was merged. Now the ``POST`` to ``/storage/v1/b`` to create a bucket should work. In ``gc-fake-storage``, I believe I continued to get 404s unless the directory ``/storage/`` was created first. This can be created as a side effect of providing a ``/data`` mount. A single``/data/first_bucket/first_object.txt`` would create the ``/storage`` folder, a bucket ``first_bucket``, and a blob / object ``first_object.txt`` in that bucket. I've submitted https://github.com/teone/gc-fake-storage/pull/3 to fix that.

kerem commented

2026-03-15 17:48:53 +03:00

Author

Owner

@fsouza commented on GitHub (Jul 18, 2019):

README now has instructions on how to use the emulator locally with Docker. Closing this issue, but feel free to open a new one in case of questions!

@fsouza commented on GitHub (Jul 18, 2019): README now has instructions on how to use the emulator locally with Docker. Closing this issue, but feel free to open a new one in case of questions!