One of the hardest things to unit test has always been HTTP endpoints. Over the years, I've thought up some creative ways to do so, some of which I'm not proud of.
When we add open-source RESTful libraries into the equation, things become even harder. They have their own internal methods to account for, their own interfaces that we need to work with, and even their own way of exposing endpoints for consumption.
In this post, I'm going to dive into how I test simple RESTful endpoints built using the go-restful library. By simple, I mean endpoints that don't rely on the library to parse the request before sending it to the handler. More complex requests will be addressed in my next post.
tl;dr: if you just want to see the code, I've pushed up a repo on GitHub. It has all of the code that will be covered in both blog posts.
And, one last thing before we get started, a big shout out to Kourtney Barnes, who helped me put this blog post together.
Demo Project
To keep things simple, I started by mostly copying and pasting some of the example code in the go-restful repo. This particular example sets up a series of endpoints for working with the /user resource:
GET /users ->
UserResource.FindAllUsersGET /users/{user-id} ->
UserResource.FindUserPOST /users ->
UserResource.CreateUserPUT /users/{user-id} ->
UserResource.UpsertUserDELETE /users/{user-id} ->
UserResource.RemoveUser
I won't go into the actual wiring-in of these endpoints into a server, because that part was just copied and pasted verbatim.
The goal, at this point, is to test all five of these handler functions with 100% coverage. In this post, I'll cover FindAllUsers and CreateUser, because they don't rely on path parameters. The next post will show how to cover the other three, which requires leveraging the library to interpret the user-id path parameter.
Here's the setup code that defines an interface for my UserResource struct, which is subsequently implemented and attached to a publicly accessible variable:
type userResource interface {
FindAllUsers(request *restful.Request, response *restful.Response)
FindUser(request *restful.Request, response *restful.Response)
UpsertUser(request *restful.Request, response *restful.Response)
CreateUser(request *restful.Request, response *restful.Response)
RemoveUser(request *restful.Request, response *restful.Response)
}
type userResourceImpl struct {}
var (
UserResource userResource = &userResourceImpl{}
)
As you can see, they don't just take an http.Request, they take an extended version defined by go-restful. That will be important later.
For storage, much like the example code I copied, I'm just using an in-memory map. I stored it in a package variable so that I could access it in my tests to ensure that it had all of the data that I expected:
var (
UserResource userResource = &userResourceImpl{}
// normally one would use DAO (data access object)
users map[string]models.User = map[string]models.User{
"1": {"1", "Mario", 35},
"2": {"2", "Luigi", 32},
"3": {"3", "Toad", 481},
"4": {"4", "Peach", 27},
}
)
Getting Started With Something Simple
The first enpoint that we'll test is the simplest one of all, FindAllUsers. The code is extremely straightforward - it loops over the users map and builds a models.User slice, and then writes it to the restful.Response as JSON (with a nice 200 response added behind the scenes).
// FindAllUsers GET http://localhost:8080/users
func (u *userResourceImpl) FindAllUsers(request *restful.Request, response *restful.Response) {
log.Println("findAllUsers")
var list []models.User
for _, each := range users {
list = append(list, each)
}
response.WriteAsJson(list)
}
What makes this one simple is that it works with the underlying http.Request and http.Response that the library passes it, albeit using some of the library's convenience methods. That means that we can test this function directly, without having to set up a server.
First, we start with mocking a request, then we mock a response, and we send them both to the FindAllUsers function. We'll utilize two methods from the built-in Golang httptest package:
NewRequest(method string, target string, body io.Reader) *http.Request- this will give us the mock HTTP request that we'll send to theFindAllUsershandler.NewRecorder() *ResponseRecorder- this will give us anhttp.ResponseWriterthat we can use to inspect what was sent as a response, which makes it a convenient tool for testing.
Once we've mocked those, we can pass them to the go-restful functions that encapsulate them in their library goodness:
func TestUserResourceImpl_FindAllUsers(t *testing.T) {
// mocking the request
httpReq := httptest.NewRequest(http.MethodGet, "/users", nil)
req := restful.NewRequest(httpReq)
// mocking the response
rec := httptest.NewRecorder()
res := restful.NewResponse(rec)
// calling the function under test
UserResource.FindAllUsers(req, res)
// we'll add our test assertions here
// ...
}
This being a simple endpoint, I just need one run-through, and I'll test two things - first that I got a 200 response, and second, that the amount of users listed in my response matches the amount of users in my package variable map:
func TestUserResourceImpl_FindAllUsers(t *testing.T) {
//...
// all the setup stuff shown above...
// checking the status code
if res.StatusCode() != http.StatusOK {
t.Fatalf("expected status code %d, got %d", http.StatusOK, res.StatusCode())
}
var list []models.User
err := json.Unmarshal(rec.Body.Bytes(), &list)
if err != nil {
t.Fatalf("error parsing response body: %s", err.Error())
}
// checking the response list against the test data map
if len(list) != len(users) {
t.Fatalf("expected %d users, got %d", len(users), len(list))
}
}
And that's it! That fully tests this first endpoint. Let's move on to something more complicated.
Testing Error Cases
The next handler that we'll test is the CreateUser function:
// CreateUser POST http://localhost:8080/users
// <models.User><Id>1</Id><Name>Melissa</Name></models.User>
func (u *userResourceImpl) CreateUser(request *restful.Request, response *restful.Response) {
log.Println("createUser")
usr := models.User{}
err := request.ReadEntity(&usr)
if err == nil {
usr.ID = fmt.Sprintf("%d", time.Now().Unix())
users[usr.ID] = usr
response.WriteHeaderAndJson(http.StatusCreated, usr, restful.MIME_JSON)
} else {
response.WriteError(http.StatusBadRequest, err)
return
}
}
To fully test this, we'll need to test the happy path and the error case. <rant>I can't tell you how many codebases I've seen that don't bother testing the error cases, just to have the error not behave as expected in production. Always test the error cases!</rant>
The happy path setup is very similar to what we have above, with one caveat - we have to pass it some data.
The httptest.NewRequest method takes a io.Reader as a body argument. That means that anything that implements the following interface will suffice:
type Reader interface {
Read(p []byte) (n int, err error)
}
I wrote a previous blog post about using interfaces for mocking in unit tests. This is super important, and you'll see why when we get to the error case.
For the happy path, we'll simply pass in a strings.NewReader that takes a JSON string as a mock model.User. We'll then also have to make it clear that the request is sending application/json as a content type so that our go-restful convenience methods know how to interpret the data that they're receiving.
func TestUserResourceImpl_CreateUser(t *testing.T) {
// setting up a mock request with a POST body
httpReq, _ := http.NewRequest(http.MethodPost, "/users", strings.NewReader(`{"Name": "Bowser", "Age": 13}`))
httpReq.Header.Set("Content-Type", restful.MIME_JSON)
req := restful.NewRequest(httpReq)
// more to come...
// ...
}
After that, it's just a matter of creating the mock response, the same way we did before, passing both to the CreateUser function, and testing assertions on the response. Here's the happy path test, in its entirety:
func TestUserResourceImpl_CreateUser(t *testing.T) {
// setting up a mock request with a POST body
httpReq, _ := http.NewRequest(http.MethodPost, "/users", strings.NewReader(`{"Name": "Bowser", "Age": 13}`))
httpReq.Header.Set("Content-Type", restful.MIME_JSON)
req := restful.NewRequest(httpReq)
// mocking the response
rec := httptest.NewRecorder()
res := restful.NewResponse(rec)
// calling the function under test
UserResource.CreateUser(req, res)
if res.StatusCode() != http.StatusCreated {
t.Fatalf("expected status code %d, got %d", http.StatusCreated, res.StatusCode())
}
if len(users) != 5 {
t.Fatalf("expected 5 users, found %d", len(users))
}
}
Now we need to test the error case. To do that, we'll create a struct that implements the io.Reader interface, and in doing so, returns an error every time Read(p []byte) is called. Let's add the following to the top of our test file:
var testError = "test error"
type errReader struct{}
func (er errReader) Read([]byte) (int, error) {
return 0, fmt.Errorf(testError)
}
Now, whenever we pass this as a POST body to the httptest.NewRequest method, the handler function will pick up the error returned by the Read and respond with an error status code.
Let's test it out. We'll mock a new request using the errReader struct, mock a response, and pass them to the CreateUser function. Then we'll run our assertions to ensure that we receive a 400 status code in our response, and that the error sent back is the one we mocked in our testError variable:
func TestUserResourceImpl_CreateUser_BadRequest(t *testing.T) {
// passing a struct that throws an error on Read(), error should propagate back to the response
httpReq, _ := http.NewRequest(http.MethodPost, "/users", errReader{})
httpReq.Header.Set("Content-Type", restful.MIME_JSON)
req := restful.NewRequest(httpReq)
rec := httptest.NewRecorder()
res := restful.NewResponse(rec)
UserResource.CreateUser(req, res)
if res.StatusCode() != http.StatusBadRequest {
t.Fatalf("expected status code %d, got %d", http.StatusBadRequest, res.StatusCode())
}
if rec.Body.String() != testError {
t.Fatalf("expected error as %s, got %s", testError, rec.Body.String())
}
}
And it passes! That fully covers the CreateUser function.
That's it for this first part. As stated before, I'll cover the more complex endpoints in my next post, which will demonstrate how to set up a test server so that our handlers can receive a request with a fully parsed request.