[GH-ISSUE #64] potential use question #36

New issue

Closed

opened 2026-02-27 23:15:08 +03:00 by kerem · 4 comments

kerem commented 2026-02-27 23:15:08 +03:00

Owner

Copy link

Originally created by @Outstep on GitHub (Apr 27, 2018).
Original GitHub issue: https://github.com/snail007/goproxy/issues/64

Hello,

I am working on a small proxy-type project which I have been coding up in C++, but it is taking a long time and does not support the features that I need so I started to investigate Golang multiplexers/routers, and Proxy servers.

What I am looking for is to develop a type of "middleware" that receives a REST API call from a frontend server and then re-writes the request so that it can be sent out like a proxy server to a backend REST API data server that will return a JSON result.

That JSON result will be re-written into a different JSON format & structure which will be sent back to the original caller.

I am wondering if your "GoProxy" (Proxy Networks) could be made to work along these lines, so I wanted to ask and investigate the potential.

Any ideas, or suggestion would be greatly appreciated.
Thanks in advance.

Originally created by @Outstep on GitHub (Apr 27, 2018). Original GitHub issue: https://github.com/snail007/goproxy/issues/64 Hello, I am working on a small proxy-type project which I have been coding up in C++, but it is taking a long time and does not support the features that I need so I started to investigate Golang multiplexers/routers, and Proxy servers. What I am looking for is to develop a type of "middleware" that receives a REST API call from a frontend server and then re-writes the request so that it can be sent out like a proxy server to a backend REST API data server that will return a JSON result. That JSON result will be re-written into a different JSON format & structure which will be sent back to the original caller. I am wondering if your "GoProxy" (Proxy Networks) could be made to work along these lines, so I wanted to ask and investigate the potential. Any ideas, or suggestion would be greatly appreciated. Thanks in advance.

kerem 2026-02-27 23:15:08 +03:00

• closed this issue
• added the
  help wanted
  label

kerem commented 2026-02-27 23:15:09 +03:00

Author

Owner

Copy link

@Outstep commented on GitHub (Apr 27, 2018):

Sorry for the message, I just found your group on Gitter and Telegram

<!-- gh-comment-id:384993698 --> @Outstep commented on GitHub (Apr 27, 2018): Sorry for the message, I just found your group on Gitter and Telegram

kerem commented 2026-02-27 23:15:09 +03:00

Author

Owner

Copy link

@snail007 commented on GitHub (Apr 28, 2018):

your purpose is easily achieved by golang,the fllowing is an example for you,

package main

import (
    "fmt"
    "io"
    "net/http"
    "time"
)

var verbose = false;

var passthruRequestHeaderKeys = [...]string{
    "Accept",
    "Accept-Encoding",
    "Accept-Language",
    "Cache-Control",
    "Cookie",
    "Referer",
    "User-Agent",
}

var passthruResponseHeaderKeys = [...]string{
    "Content-Encoding",
    "Content-Language",
    "Content-Type",
    "Cache-Control", // TODO: Is this valid in a response?
    "Date",
    "Etag",
    "Expires",
    "Last-Modified",
    "Location",
    "Server",
    "Vary",
}

func main() {
    handler := http.DefaultServeMux
    
    handler.HandleFunc("/", handleFunc)
    
    s := &http.Server{
        Addr:           ":8080",
        Handler:        handler,
        ReadTimeout:    10 * time.Second,
        WriteTimeout:   10 * time.Second,
        MaxHeaderBytes: 1 << 20,
    }
    
    s.ListenAndServe()
}

func handleFunc(w http.ResponseWriter, r *http.Request) {
    fmt.Printf("--> %v %v\n", r.Method, r.URL)
    
    // Construct filtered header to send to origin server
    hh := http.Header{}
    for _, hk := range passthruRequestHeaderKeys {
        if hv, ok := r.Header[hk]; ok {
            hh[hk] = hv
        }
    }
    
    // Construct request to send to origin server
    rr := http.Request{
        Method: r.Method,
        URL: r.URL,
        Header: hh,
        Body: r.Body,
        // TODO: Is this correct for a 0 value?
        //       Perhaps a 0 may need to be reinterpreted as -1?
        ContentLength: r.ContentLength,
        Close: r.Close,
    }
    
    // Forward request to origin server
    resp, err := http.DefaultTransport.RoundTrip(&rr)
    if err != nil {
        // TODO: Passthru more error information
        http.Error(w, "Could not reach origin server", 500)
        return
    }
    defer resp.Body.Close()
    
    if (verbose) {
        fmt.Printf("<-- %v %+v\n", resp.Status, resp.Header)
    } else {
        fmt.Printf("<-- %v\n", resp.Status)
    }
    
    // Transfer filtered header from origin server -> client
    respH := w.Header()
    for _, hk := range passthruResponseHeaderKeys {
        if hv, ok := resp.Header[hk]; ok {
            respH[hk] = hv
        }
    }
    w.WriteHeader(resp.StatusCode)
    
    // Transfer response from origin server -> client
    if resp.ContentLength > 0 {
        // (Ignore I/O errors, since there's nothing we can do)
        io.CopyN(w, resp.Body, resp.ContentLength)
    } else if (resp.Close) { // TODO: Is this condition right?
        // Copy until EOF or some other error occurs
        for {
            if _, err := io.Copy(w, resp.Body); err != nil {
                break
            }
        }
    }
}

<!-- gh-comment-id:385139560 --> @snail007 commented on GitHub (Apr 28, 2018): your purpose is easily achieved by golang,the fllowing is an example for you, ```golang package main import ( "fmt" "io" "net/http" "time" ) var verbose = false; var passthruRequestHeaderKeys = [...]string{ "Accept", "Accept-Encoding", "Accept-Language", "Cache-Control", "Cookie", "Referer", "User-Agent", } var passthruResponseHeaderKeys = [...]string{ "Content-Encoding", "Content-Language", "Content-Type", "Cache-Control", // TODO: Is this valid in a response? "Date", "Etag", "Expires", "Last-Modified", "Location", "Server", "Vary", } func main() { handler := http.DefaultServeMux handler.HandleFunc("/", handleFunc) s := &http.Server{ Addr: ":8080", Handler: handler, ReadTimeout: 10 * time.Second, WriteTimeout: 10 * time.Second, MaxHeaderBytes: 1 << 20, } s.ListenAndServe() } func handleFunc(w http.ResponseWriter, r *http.Request) { fmt.Printf("--> %v %v\n", r.Method, r.URL) // Construct filtered header to send to origin server hh := http.Header{} for _, hk := range passthruRequestHeaderKeys { if hv, ok := r.Header[hk]; ok { hh[hk] = hv } } // Construct request to send to origin server rr := http.Request{ Method: r.Method, URL: r.URL, Header: hh, Body: r.Body, // TODO: Is this correct for a 0 value? // Perhaps a 0 may need to be reinterpreted as -1? ContentLength: r.ContentLength, Close: r.Close, } // Forward request to origin server resp, err := http.DefaultTransport.RoundTrip(&rr) if err != nil { // TODO: Passthru more error information http.Error(w, "Could not reach origin server", 500) return } defer resp.Body.Close() if (verbose) { fmt.Printf("<-- %v %+v\n", resp.Status, resp.Header) } else { fmt.Printf("<-- %v\n", resp.Status) } // Transfer filtered header from origin server -> client respH := w.Header() for _, hk := range passthruResponseHeaderKeys { if hv, ok := resp.Header[hk]; ok { respH[hk] = hv } } w.WriteHeader(resp.StatusCode) // Transfer response from origin server -> client if resp.ContentLength > 0 { // (Ignore I/O errors, since there's nothing we can do) io.CopyN(w, resp.Body, resp.ContentLength) } else if (resp.Close) { // TODO: Is this condition right? // Copy until EOF or some other error occurs for { if _, err := io.Copy(w, resp.Body); err != nil { break } } } } ```

kerem commented 2026-02-27 23:15:09 +03:00

Author

Owner

Copy link

@Outstep commented on GitHub (Apr 28, 2018):

Thank you so much for getting back to me and for sending over the code. It
looks like I may have to spend some time to get up to speed on the power of
Golang as it seems to have some advantages over my old experiences with
C/C++.

For the project that I was discussing I was thinking about trying to combine

Goproxy (https://github.com/snail007/goproxy)

with the routing/multiplexer abilities of

Goji (https://goji.io/) or a similar router library so that you could
establish channels in the re-writing JSON proxy so that it could talk to
different JSON (or possibly XML) backend servers to collect returns and
send them forward to the frontend web server.

This might have potential as a MetaSearch engine middleware that gets data
from different existing search engines and aggregates all of returns into a
single collection that returns a JSON to the frontend server.

That's the basic idea that I am shooting for, but with a simpler start of
just re-writing a single JSON source to test things out.

Cheers and have a great day :)

On Sat, Apr 28, 2018 at 12:32 AM, snail007 notifications@github.com wrote:

your purpose is easyly achieved by golang,the fllowing is an example for
you,

package main
import (
"fmt"
"io"
"net/http"
"time"
)
var verbose = false;
var passthruRequestHeaderKeys = [...]string{
"Accept",
"Accept-Encoding",
"Accept-Language",
"Cache-Control",
"Cookie",
"Referer",
"User-Agent",
}
var passthruResponseHeaderKeys = [...]string{
"Content-Encoding",
"Content-Language",
"Content-Type",
"Cache-Control", // TODO: Is this valid in a response?
"Date",
"Etag",
"Expires",
"Last-Modified",
"Location",
"Server",
"Vary",
}
func main() {
handler := http.DefaultServeMux

handler.HandleFunc("/", handleFunc)

s := &http.Server{
    Addr:           ":8080",
    Handler:        handler,
    ReadTimeout:    10 * time.Second,
    WriteTimeout:   10 * time.Second,
    MaxHeaderBytes: 1 << 20,
}

s.ListenAndServe()

}
func handleFunc(w http.ResponseWriter, r *http.Request) {
fmt.Printf("--> %v %v\n", r.Method, r.URL)

// Construct filtered header to send to origin server
hh := http.Header{}
for _, hk := range passthruRequestHeaderKeys {
    if hv, ok := r.Header[hk]; ok {
        hh[hk] = hv
    }
}

// Construct request to send to origin server
rr := http.Request{
    Method: r.Method,
    URL: r.URL,
    Header: hh,
    Body: r.Body,
    // TODO: Is this correct for a 0 value?
    //       Perhaps a 0 may need to be reinterpreted as -1?
    ContentLength: r.ContentLength,
    Close: r.Close,
}

// Forward request to origin server
resp, err := http.DefaultTransport.RoundTrip(&rr)
if err != nil {
    // TODO: Passthru more error information
    http.Error(w, "Could not reach origin server", 500)
    return
}
defer resp.Body.Close()

if (verbose) {
    fmt.Printf("<-- %v %+v\n", resp.Status, resp.Header)
} else {
    fmt.Printf("<-- %v\n", resp.Status)
}

// Transfer filtered header from origin server -> client
respH := w.Header()
for _, hk := range passthruResponseHeaderKeys {
    if hv, ok := resp.Header[hk]; ok {
        respH[hk] = hv
    }
}
w.WriteHeader(resp.StatusCode)

// Transfer response from origin server -> client
if resp.ContentLength > 0 {
    // (Ignore I/O errors, since there's nothing we can do)
    io.CopyN(w, resp.Body, resp.ContentLength)
} else if (resp.Close) { // TODO: Is this condition right?
    // Copy until EOF or some other error occurs
    for {
        if _, err := io.Copy(w, resp.Body); err != nil {
            break
        }
    }
}

}

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/snail007/goproxy/issues/64#issuecomment-385139560,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AXkxHv4_e6X183MZSa7EkB5hkoTiTTnDks5ts_DfgaJpZM4Tqevt
.

<!-- gh-comment-id:385174447 --> @Outstep commented on GitHub (Apr 28, 2018): Thank you so much for getting back to me and for sending over the code. It looks like I may have to spend some time to get up to speed on the power of Golang as it seems to have some advantages over my old experiences with C/C++. For the project that I was discussing I was thinking about trying to combine Goproxy (https://github.com/snail007/goproxy) with the routing/multiplexer abilities of Goji (https://goji.io/) or a similar router library so that you could establish channels in the re-writing JSON proxy so that it could talk to different JSON (or possibly XML) backend servers to collect returns and send them forward to the frontend web server. This might have potential as a MetaSearch engine middleware that gets data from different existing search engines and aggregates all of returns into a single collection that returns a JSON to the frontend server. That's the basic idea that I am shooting for, but with a simpler start of just re-writing a single JSON source to test things out. Cheers and have a great day :) On Sat, Apr 28, 2018 at 12:32 AM, snail007 <notifications@github.com> wrote: > your purpose is easyly achieved by golang,the fllowing is an example for > you, > > package main > import ( > "fmt" > "io" > "net/http" > "time" > ) > var verbose = false; > var passthruRequestHeaderKeys = [...]string{ > "Accept", > "Accept-Encoding", > "Accept-Language", > "Cache-Control", > "Cookie", > "Referer", > "User-Agent", > } > var passthruResponseHeaderKeys = [...]string{ > "Content-Encoding", > "Content-Language", > "Content-Type", > "Cache-Control", // TODO: Is this valid in a response? > "Date", > "Etag", > "Expires", > "Last-Modified", > "Location", > "Server", > "Vary", > } > func main() { > handler := http.DefaultServeMux > > handler.HandleFunc("/", handleFunc) > > s := &http.Server{ > Addr: ":8080", > Handler: handler, > ReadTimeout: 10 * time.Second, > WriteTimeout: 10 * time.Second, > MaxHeaderBytes: 1 << 20, > } > > s.ListenAndServe() > } > func handleFunc(w http.ResponseWriter, r *http.Request) { > fmt.Printf("--> %v %v\n", r.Method, r.URL) > > // Construct filtered header to send to origin server > hh := http.Header{} > for _, hk := range passthruRequestHeaderKeys { > if hv, ok := r.Header[hk]; ok { > hh[hk] = hv > } > } > > // Construct request to send to origin server > rr := http.Request{ > Method: r.Method, > URL: r.URL, > Header: hh, > Body: r.Body, > // TODO: Is this correct for a 0 value? > // Perhaps a 0 may need to be reinterpreted as -1? > ContentLength: r.ContentLength, > Close: r.Close, > } > > // Forward request to origin server > resp, err := http.DefaultTransport.RoundTrip(&rr) > if err != nil { > // TODO: Passthru more error information > http.Error(w, "Could not reach origin server", 500) > return > } > defer resp.Body.Close() > > if (verbose) { > fmt.Printf("<-- %v %+v\n", resp.Status, resp.Header) > } else { > fmt.Printf("<-- %v\n", resp.Status) > } > > // Transfer filtered header from origin server -> client > respH := w.Header() > for _, hk := range passthruResponseHeaderKeys { > if hv, ok := resp.Header[hk]; ok { > respH[hk] = hv > } > } > w.WriteHeader(resp.StatusCode) > > // Transfer response from origin server -> client > if resp.ContentLength > 0 { > // (Ignore I/O errors, since there's nothing we can do) > io.CopyN(w, resp.Body, resp.ContentLength) > } else if (resp.Close) { // TODO: Is this condition right? > // Copy until EOF or some other error occurs > for { > if _, err := io.Copy(w, resp.Body); err != nil { > break > } > } > } > } > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <https://github.com/snail007/goproxy/issues/64#issuecomment-385139560>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AXkxHv4_e6X183MZSa7EkB5hkoTiTTnDks5ts_DfgaJpZM4Tqevt> > . >

kerem commented 2026-02-27 23:15:09 +03:00

Author

Owner

Copy link

@snail007 commented on GitHub (Apr 28, 2018):

You are beginner of golang, and also http proxies; Through you say above, i have some suggestions for you with below:
1.a powerful http server library in golang, https://github.com/valyala/fasthttp
2.a powerful router of fasthttp https://github.com/buaazp/fasthttprouter
3.json encode/decode example in golang, https://golang.org/src/encoding/json/example_test.go
4.http request library,https://github.com/parnurzeal/gorequest

<!-- gh-comment-id:385186223 --> @snail007 commented on GitHub (Apr 28, 2018): You are beginner of golang, and also http proxies; Through you say above, i have some suggestions for you with below: 1.a powerful http server library in golang, https://github.com/valyala/fasthttp 2.a powerful router of fasthttp https://github.com/buaazp/fasthttprouter 3.json encode/decode example in golang, https://golang.org/src/encoding/json/example_test.go 4.http request library,https://github.com/parnurzeal/gorequest

kerem referenced this issue 2026-02-27 23:17:11 +03:00

[PR #36] [MERGED] Update README.md #491

kerem referenced this issue 2026-02-27 23:17:12 +03:00

[PR #37] [MERGED] update readme #493

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

master

v15.2

v15.1

v15.0

Labels

Clear labels   

TODO

  

bug

  

duplicate

  

enhancement

  

good first issue

  

help wanted

  

helpful

  

invalid

  

need-confirm

  

pull-request

Mirrored from GitHub Pull Request

  

question

  

wontfix

No labels

TODO

bug

duplicate

enhancement

good first issue

help wanted

helpful

invalid

need-confirm

pull-request

question

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/goproxy#36

No description provided.