Day 20 - A go http/2 server example

I was really excited when i read that support for http2 is now on golang.org! I was thinking about checking out SPDY in go for a long time, but now HTTP/2 support moved one important step further.

I think its one of the "Next big Things" on the interweb (the last change to http was 18! years ago)

So i skip SPDY and go directly for HTTP/2 - this post is about how to get a first grip and test it serverside as a go html server example.

Test preparations clientside:

1. Get a browser that has http/2 support enabled by default: Chrome Canary
2. Download the http2 and spdy indicator addon for chrome 
3. Test your browsers http2 support at: https://http2.akamai.com/demo

 Serverside::

1. Cd into your GOPATH/src
2. Run go get golang.org/x/net/http2
3. Check if the package has been downloaded

Create, build and run:

1. Create a new go project, name it eg. testHttp2Server
2. Create a ssl certificate (localhost.cert and localhost.key)
   You can ether generate a self signed cert: selfsignedcertificate
   or use mine: testHttp2Server
   This two files must be in the same directory as the server binary.
3. Code for http2Server.go:
   
   
   package main

   import (

       "fmt"

       "golang.org/x/net/http2"

       "html"

       "log"

       "net/http"

   )

   func main() {

       var srv http.Server

       http2.VerboseLogs = true

       srv.Addr = ":8080"

       // This enables http2 support

       http2.ConfigureServer(&srv, nil)

       // Plain text test handler

       // Open https://localhost:8080/randomtest

       // in your Chrome Canary browser

       http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {

           fmt.Fprintf(w, "Hi tester %q\n", html.EscapeString(r.URL.Path))

           ShowRequestInfoHandler(w, r)

       })

       // Listen as https ssl server

       // NOTE: WITHOUT SSL IT WONT WORK!!

       // To self generate a test ssl cert/key you could go to

       // http://www.selfsignedcertificate.com/

       // or read the openssl manual

       log.Fatal(srv.ListenAndServeTLS("localhost.cert", "localhost.key"))

   }

   func ShowRequestInfoHandler(w http.ResponseWriter, r *http.Request) {

       w.Header().Set("Content-Type", "text/plain")

       fmt.Fprintf(w, "Method: %s\n", r.Method)

       fmt.Fprintf(w, "Protocol: %s\n", r.Proto)

       fmt.Fprintf(w, "Host: %s\n", r.Host)

       fmt.Fprintf(w, "RemoteAddr: %s\n", r.RemoteAddr)

       fmt.Fprintf(w, "RequestURI: %q\n", r.RequestURI)

       fmt.Fprintf(w, "URL: %#v\n", r.URL)

       fmt.Fprintf(w, "Body.ContentLength: %d (-1 means unknown)\n", r.ContentLength)

       fmt.Fprintf(w, "Close: %v (relevant for HTTP/1 only)\n", r.Close)

       fmt.Fprintf(w, "TLS: %#v\n", r.TLS)

       fmt.Fprintf(w, "\nHeaders:\n")

       r.Header.Write(w)

   }

   
   

4. Compile:  go build http2Server.go
5. Run the http2Server binary

Test it:

1. Open in Chrome Canary: https://localhost:8080/kim
2. Accept the certificate security error, continue to localhost
3. Check if Protocol is HTTP/2.0, example output should be like:
   
   

   Hello tester "/kim"
   Method: GET
   Protocol: HTTP/2.0
   Host: localhost:8080

   ... 

NOTE: http/2 in go ONLY WORKS WITH SSL/TLS! (currently)
I not sure why, but i did not got the server to support http/2 without TLS!
This could be a clientside or serverside issue - i dont know currently.

Some discussions about this:
why-wouldnt-it-be-great-if-http-2-would-only-allow-communication-via-tls
daniel.haxx.se tls-in-http2
arstechnica http2-finished-coming-to-browsers-within-weeks

As far as i have understood it, the spec says it "should be" possible to use it without the need to buy SSL certificates:
https://http2.github.io/http2-spec/
rfc7540 - html2

Day 19 - Upgrading to Go 1.5.1 on Win7 64

I upgraded to Go 1.5.1 from 1.4.2 today

0:) Deinstall old version of Go (doc/install)

1:) Download https://storage.googleapis.com/golang/go1.5.1.windows-amd64.msi

2:) Get worried by this message from chrome:

It says that there is something fishy with the download and could be harmful.

3:) Go on and accept it - its from google, so what ...

4:) Go to the download folder and run your virus scanner over the msi:

5:) UhOh - WTF is an Archive Bomb?

6:) Google for Archive Bomb, read Wiki about Zip_bomb

7:) Looks like its detected because of special compression used by bzip2, i feel brave today and go on installing the msi.

8.) Checking again the c:\go folder for viruses - all ok

9:)
D:\>go version
go version go1.5.1 windows/amd64 

Yeah, it works

10:) Recompiling and running some of my most important projects - no problems whatsoever.

Day 18 - How to get a token from a JWT server

There is a wonderful middleware for ant0ine/go-json-rest to implement JWT for go: go-json-rest-middleware-jwt

The problem I had is that the docu seems not to be working for curl on windows, the go code is alright, only the curl command line is a problem:

So here is a windows example of how to make a test call with curl to get a JWT token:

-- snip
curl -X POST --data "{\"username\": \"admin\", \"password\": \"admin\"}" -H "Content-Type:application/json" http://localhost:8080/login
 -- snip

-- output should be something like this:
"token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE0NDA4NjAzMzgsImlkIjoiYWRtaW4iLCJvcmlnX2lhdCI6MTQ0MDg1NjczOH0.cIIACBWOFCMZI4vBgSdQXkUsm8f7WpRoSoFtmoOtceQ"
--output

Docu of jwt with json-rest
https://github.com/ant0ine/go-json-rest#jwt

Full source of my example app:
intogooglego/restPostServer


--- demo ---

    i.jwt_middleware = &jwt.JWTMiddleware{

        Key:        []byte("secret key"),

        Realm:      "jwt auth",

        Timeout:    time.Hour,

        MaxRefresh: time.Hour * 24,

        Authenticator: func(userId string, password string) bool {

            if debugLevel > 2 {

                fmt.Printf("Authenticator: '%s' - '%s'\n", userId, password)

            }

            return userId == "admin" && password == "admin"

        },

    }

    api := rest.NewApi()

    var MiddleWareStack = []rest.Middleware{

        &rest.AccessLogApacheMiddleware{},

        &rest.TimerMiddleware{},

        &rest.RecorderMiddleware{},

        //&rest.PoweredByMiddleware{},

        &rest.RecoverMiddleware{

            EnableResponseStackTrace: true,

        },

        &rest.JsonIndentMiddleware{},

        &rest.ContentTypeCheckerMiddleware{},

    }

    statusMw := &rest.StatusMiddleware{}

    api.Use(statusMw)

    api.Use(MiddleWareStack...)

    api.Use(&rest.IfMiddleware{

        Condition: func(request *rest.Request) bool {

            if debugLevel > 2 {

                fmt.Printf("AUTH Request.URL.Path: '%s' returning '%b'\n", request.URL.Path, request.URL.Path != "/login")

            }

            return request.URL.Path != "/login"

        },

        IfTrue: i.jwt_middleware,

    })

    router, err := rest.MakeRouter(

        // JSON

        rest.Get("/j/t/:postid", i.JsonGetPostThreadComments),

        rest.Get("/j/p/:orderby", i.JsonGetPosts),

        // HTML

        rest.Get("/", i.SendStaticMainHtml),

        rest.Get("/t/:postid", i.SendStaticCommentsHtml),

        // Auth JWT

        rest.Post("/login", i.jwt_middleware.LoginHandler),

        rest.Get("/jwttest", i.JwtTest),

        rest.Post("/jwtposttest", i.JwtPostTest),

        rest.Get("/refresh_token", i.jwt_middleware.RefreshHandler),

        // JSON Depricated

        rest.Get("/p/:orderby", i.JsonGetAllPosts),

        rest.Get("/p", i.JsonGetAllPosts),

        // HTML, Images, CSS and JS

        rest.Get("/img/#filename", i.SendStaticImage),

        rest.Get("/css", i.SendStaticCss),

        rest.Get("/css/#cssfile", i.SendStaticCss),

        rest.Get("/js/#jsfile", i.SendStaticJS),

        rest.Get("/js", i.SendStaticJS),

        rest.Get("/html/*filename", i.GetHtmlFile),

        rest.Get("/test/*filename", i.GetTestFile),

        rest.Get("/.status", func(w rest.ResponseWriter, r *rest.Request) {

            w.WriteJson(statusMw.GetStatus())

        }),

    )

    if err != nil {

        log.Fatal(err)

    }

    api.SetApp(router)

    //http.Handle("/static/", http.StripPrefix("/static", http.FileServer(http.Dir("."))))

    http.Handle("/api/", http.StripPrefix("/api", api.MakeHandler()))

    http.Handle("/", api.MakeHandler())

    if debugLevel > 2 {

        fmt.Println("Starting http.ListenAndServe :8080")

    }

    //log.Fatal(http.ListenAndServe(":8080", api.MakeHandler()))

    log.Fatal(http.ListenAndServe(":8080", nil))

}

--- demo --- 

Day 17 - Using reflection to write into an interface slice

How to use reflection to write into a slice of unknown items inside an unknown struct:

While working on Gorp with Indexes i had to solve the problem of how to write into a slice inside a struct passed as an interface{}. A new struct with the filled slice should be returned. Both the struct and the slice are not known at compile time, i only have the fieldname of the slice passed to me at runtime. After some serious headscratching and reading through the reflection code i luckily found a way to do it.


The basic method is:

1. Convert the incoming interface i to a reflect.Type t
2. Dereference until we have a t which is a Struct
3. Create a new reflect.Value v from the Type t
4. From v we now can ask for the FieldName of the slice, getting back another reflect.Value s
5. As we now have the slice s as a reflect.Value, we need to get the slice elements type
6. From the slice elements type we create a new instance of it (reflect.New)
7. Write to fields in this newItem using FieldByName (Hardcoded in the example)
8. Append the newItem to the slice (the set append was the hard part to find out for me)
9. Return the reflect.Value v as an interface
10. ???
11. Profit - Heureka!!

Output:
Input Type main.Post:

Slice Type []*main.Comment:

Slice Elem Type main.Comment:

Comment 0, Body XYZ 0, PostId 0

Comment 1, Body XYZ 1, PostId 2

Comment 2, Body XYZ 2, PostId 4

Comment 3, Body XYZ 3, PostId 6

Comment 4, Body XYZ 4, PostId 8

Erfolg: Prozess beendet mit Rückgabewert 0.

// This is a demo to show how to convert from a normal struct

// to a reflection type and back to a struct without knowing

// the original one. Input is passed as an Interface and the  

// output will be an interface, too.

// Bonus points for writing into an embedded slice

// (= the embedded Comment struct slice in Post)

package main

import (

    "errors"

    "fmt"

    "os"

    "reflect"

)

type Post struct {

    Id       uint64

    Title    string

    Comments []*Comment

}

// holds a single comment bound to a post

type Comment struct {

    Id     uint64

    PostId uint64

    Body   string

}

func CreateAndFillSlice(i interface{}, sliceName string) (interface{}, error) {

    // Convert the interface i to a reflect.Type t 

    t := reflect.TypeOf(i)

    // Check if the input is a pointer and dereference it if yes

    if t.Kind() == reflect.Ptr {

        t = t.Elem()

    }

    // Check if the input is a struct

    if t.Kind() != reflect.Struct {

        return nil, errors.New("Input param is not a struct")

    }

    fmt.Printf("Input Type %v:\n", t)

    // Create a new Value from the input type

    // this will be returned to the caller

    v := reflect.New(t).Elem()

    // Get the field named "sliceName" from the input struct, which should be a slice

    s := v.FieldByName(sliceName)

    if s.Kind() == reflect.Slice {

        st := s.Type()

        fmt.Printf("Slice Type %s:\n", st)

        // Get the type of a single slice element

        sliceType := st.Elem()

        // Pointer?

        if sliceType.Kind() == reflect.Ptr {

            // Then dereference it

            sliceType = sliceType.Elem()

        }

        fmt.Printf("Slice Elem Type %v:\n", sliceType)

        for i := 0; i < 5; i++ {

            // Create a new slice element

            newitem := reflect.New(sliceType)

            // Set some field in it

            newitem.Elem().FieldByName("Body").SetString(fmt.Sprintf("XYZ %d", i))

            newitem.Elem().FieldByName("PostId").SetUint(uint64(i * 2))

            // This is the important part here - append and set

            // Append the newitem to the slice in "v" which will be the output

            s.Set(reflect.Append(s, newitem))

        }

    } else {

        return nil, fmt.Errorf("Field %s is not a slice\n", sliceName)

    }

    // IMPORTANT

    // Cast back to the empty interface type

    // So the cast back to Post outside will work

    return v.Interface(), nil

}

func main() {

    var err error

    p := Post{Id: 1, Title: "Title 1"}

    result, err := CreateAndFillSlice(p, "Comments")

    if err != nil {

        fmt.Println(err.Error())

        os.Exit(1)

    }

    // Cast the returned interface to a Post

    post := result.(Post)

    for i, c := range post.Comments {

        fmt.Printf("Comment %d, Body %s, PostId %d\n", i, c.Body, c.PostId)

    }

}