Blog

Build a JSON REST API in Go with the Standard Library

Written by

A task-list REST API in Go with no third-party packages. Build the store, routes, safe JSON decoding, validation, status codes, middleware and slog logging, one layer at a time.

A JSON REST API is a set of URLs that programs call to read and change data, with JSON in the request and response bodies. Go’s standard library has everything you need to build one: net/http for routing, encoding/json for the bodies and log/slog for the logs.

This post builds a small task-list API as a real module, one layer at a time. The code shown comes straight from the module’s files, which pass go vet and go test -race. Every program below was run on Go 1.26, and its output is pasted from the run.

What the API does

The API keeps a list of tasks, and each task has an ID, a title and a done flag. Five routes cover reading, creating, replacing and deleting them:

Request What it does Success Client errors
GET /tasks list every task 200
POST /tasks create a task 201 with Location 400, 413, 415, 422
GET /tasks/{id} fetch one task 200 404
PUT /tasks/{id} replace one task 200 400, 404, 413, 415, 422
DELETE /tasks/{id} delete one task 204 404

Any route can also answer 500. A known path with the wrong method gets 405, and an unknown path gets 404.

The module has two packages under internal/ and one command:

19-tasks-api/
├── go.mod
├── cmd/
│   └── tasksd/
│       └── main.go
└── internal/
    ├── task/
    │   ├── task.go
    │   └── memstore.go
    └── api/
        ├── api.go
        ├── handlers.go
        ├── json.go
        ├── validate.go
        └── middleware.go

task knows nothing about HTTP, and api knows nothing about where tasks are stored. cmd/tasksd picks a store and starts a server. This part leaves the tests out, because the next part, on testing and shipping, is about them.

The store: an interface in front of a map

A task store in this API is anything with five methods, so the handlers depend on an interface, not on a map. Here’s task.go, the first of the package’s two files:

// Package task defines a task and the storage behind it.
package task

import (
	"context"
	"errors"
)

// Task is one item on the list.
type Task struct {
	ID    int64  `json:"id"`
	Title string `json:"title"`
	Done  bool   `json:"done"`
}

// ErrNotFound is returned when no task has the requested ID.
var ErrNotFound = errors.New("task not found")

// Store is everything the API needs from storage.
type Store interface {
	List(ctx context.Context) ([]Task, error)
	Get(ctx context.Context, id int64) (Task, error)
	Create(ctx context.Context, t Task) (Task, error)
	Update(ctx context.Context, t Task) (Task, error)
	Delete(ctx context.Context, id int64) error
}

The JSON tags give the fields lower-case names on the wire. ErrNotFound is a sentinel error, like io.EOF in the part on errors. Every method takes a context.Context first. The in-memory store ignores it, but a database store would pass it to each query, so a cancelled request stops its query too.

MemStore is the implementation this post uses:

// The build fails here if MemStore stops satisfying Store.
var _ Store = (*MemStore)(nil)

// MemStore keeps tasks in memory. It's safe for concurrent use.
type MemStore struct {
	mu     sync.RWMutex
	lastID int64
	tasks  map[int64]Task
}

// NewMemStore returns an empty store. The first task gets ID 1.
func NewMemStore() *MemStore {
	return &MemStore{tasks: map[int64]Task{}}
}

// List returns every task, ordered by ID. It never returns nil.
func (s *MemStore) List(_ context.Context) ([]Task, error) {
	s.mu.RLock()
	defer s.mu.RUnlock()
	list := make([]Task, 0, len(s.tasks))
	for _, id := range slices.Sorted(maps.Keys(s.tasks)) {
		list = append(list, s.tasks[id])
	}
	return list, nil
}

// Get returns the task with the given ID, or ErrNotFound.
func (s *MemStore) Get(_ context.Context, id int64) (Task, error) {
	s.mu.RLock()
	defer s.mu.RUnlock()
	t, ok := s.tasks[id]
	if !ok {
		return Task{}, ErrNotFound
	}
	return t, nil
}

// Create stores t under the next ID and returns it with the ID set.
// Any ID already in t is ignored.
func (s *MemStore) Create(_ context.Context, t Task) (Task, error) {
	s.mu.Lock()
	defer s.mu.Unlock()
	s.lastID++
	t.ID = s.lastID
	s.tasks[t.ID] = t
	return t, nil
}

Update and Delete follow the same shape, with Lock. Every request runs in its own goroutine, so without the mutex two POST requests could both bump lastID and get the same ID. Reads take the read lock, so many requests can list tasks at once. The var _ Store line is the compile-time check from the part on interfaces.

List builds its result with make, not var list []Task, and the comment says it never returns nil. That matters once the slice becomes JSON:

package main

import (
	"encoding/json"
	"fmt"
)

type Task struct {
	ID    int64  `json:"id"`
	Title string `json:"title"`
	Done  bool   `json:"done"`
}

func main() {
	var none []Task
	empty := []Task{}
	one := []Task{{ID: 1, Title: "Buy milk"}}

	for _, tasks := range [][]Task{none, empty, one} {
		b, err := json.Marshal(tasks)
		fmt.Println(string(b), err)
	}
}

It prints:

null <nil>
[] <nil>
[{"id":1,"title":"Buy milk","done":false}] <nil>

A nil slice encodes as null. An empty one encodes as []. A JavaScript client that loops over null throws an error, so an empty task list must be [].

The constructor: routes and dependencies

The api package exposes one function, New, which builds the whole API as a single http.Handler. It registers the routes with the method-and-path patterns that arrived in Go 1.22, which the previous part on net/http servers covers in detail:

// server holds the dependencies every handler needs.
type server struct {
	store  task.Store
	logger *slog.Logger
}

// New returns the API as an http.Handler. It keeps no global state:
// every call builds a fresh handler, with its own request ID counter.
func New(store task.Store, logger *slog.Logger) http.Handler {
	s := &server{store: store, logger: logger}

	mux := http.NewServeMux()
	mux.HandleFunc("GET /tasks", s.listTasks)
	mux.HandleFunc("POST /tasks", s.createTask)
	mux.HandleFunc("GET /tasks/{id}", s.getTask)
	mux.HandleFunc("PUT /tasks/{id}", s.updateTask)
	mux.HandleFunc("DELETE /tasks/{id}", s.deleteTask)

	var h http.Handler = s.jsonRouteErrors(mux)
	h = s.recoverPanics(h)
	h = s.logRequests(h)
	h = withRequestID(h)
	return h
}

New takes the store and the logger as arguments and keeps them in a server struct. There are no package-level variables and no init function, so every call builds a separate handler. A test can build one with a fresh store and a logger that writes to a buffer. The four lines after the routes wrap the mux in middleware, which comes later in this post.

Handlers: read the request, call the store, write the answer

Every handler in the API reads what it needs from the request, calls the store, and turns the result into a status code and a body. createTask is the longest:

func (s *server) createTask(w http.ResponseWriter, r *http.Request) {
	var in taskInput
	if err := decodeJSON(w, r, &in); err != nil {
		s.clientError(w, err)
		return
	}
	if fields := in.validate(); len(fields) > 0 {
		s.writeError(w, http.StatusUnprocessableEntity, "validation failed", fields)
		return
	}
	t, err := s.store.Create(r.Context(), task.Task{Title: in.Title, Done: in.Done})
	if err != nil {
		s.internalError(w, r, err)
		return
	}
	w.Header().Set("Location", "/tasks/"+strconv.FormatInt(t.ID, 10))
	s.writeJSON(w, http.StatusCreated, t)
}

Each step that can fail writes its own response and returns, so the handler never writes twice. After a successful create, it sets Location to the new task’s URL and answers 201 Created with the task in the body, so the client learns the ID without a second request.

getTask shows how store errors become status codes:

func (s *server) getTask(w http.ResponseWriter, r *http.Request) {
	id, ok := parseID(r)
	if !ok {
		s.notFound(w)
		return
	}
	t, err := s.store.Get(r.Context(), id)
	if errors.Is(err, task.ErrNotFound) {
		s.notFound(w)
		return
	}
	if err != nil {
		s.internalError(w, r, err)
		return
	}
	s.writeJSON(w, http.StatusOK, t)
}

The check uses errors.Is, not ==, so it still works if a database store wraps ErrNotFound with %w. Any other error is the server’s fault, so it becomes a 500.

The ID comes out of the path through a small helper:

// parseID reads the {id} wildcard. Only positive integers name a task.
func parseID(r *http.Request) (int64, bool) {
	id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
	return id, err == nil && id > 0
}

/tasks/abc and /tasks/-3 answer 404, the same as /tasks/999, because no task lives at any of those URLs.

Why PUT and not PATCH

The API replaces tasks with PUT and doesn’t offer PATCH. With PUT, the client sends the whole task, and sending it twice gives the same result, so a client can safely retry after a timeout.

PATCH means “change only the fields I send”, so the handler would have to tell a missing done from "done": false. In Go that needs pointer fields such as *bool. A task has two fields, so sending both costs almost nothing. The price is that leaving out done sets it to false. With twenty fields, PATCH would earn its extra code.

Reading JSON without trusting it

A request body comes from whoever sent the request, so the API decodes it through one helper that sets limits before it reads a single byte. Here’s decodeJSON:

// decodeJSON reads exactly one JSON value from the body into dst.
// Every error it returns is a *requestError.
func decodeJSON(w http.ResponseWriter, r *http.Request, dst any) error {
	mediaType, _, _ := mime.ParseMediaType(r.Header.Get("Content-Type"))
	if mediaType != "application/json" {
		return &requestError{http.StatusUnsupportedMediaType, "Content-Type must be application/json"}
	}

	r.Body = http.MaxBytesReader(w, r.Body, maxBodyBytes)
	dec := json.NewDecoder(r.Body)
	dec.DisallowUnknownFields()

	if err := dec.Decode(dst); err != nil {
		var syntaxErr *json.SyntaxError
		var typeErr *json.UnmarshalTypeError
		var tooBig *http.MaxBytesError
		switch {
		case errors.As(err, &tooBig):
			msg := fmt.Sprintf("request body must not be larger than %d bytes", tooBig.Limit)
			return &requestError{http.StatusRequestEntityTooLarge, msg}
		case errors.Is(err, io.EOF):
			return &requestError{http.StatusBadRequest, "request body must not be empty"}
		case errors.Is(err, io.ErrUnexpectedEOF):
			return &requestError{http.StatusBadRequest, "request body ends in the middle of the JSON"}
		case errors.As(err, &syntaxErr):
			msg := fmt.Sprintf("malformed JSON at byte %d", syntaxErr.Offset)
			return &requestError{http.StatusBadRequest, msg}
		case errors.As(err, &typeErr) && typeErr.Field == "":
			return &requestError{http.StatusBadRequest, "request body must be a JSON object"}
		case errors.As(err, &typeErr):
			msg := fmt.Sprintf("field %q has the wrong type", typeErr.Field)
			return &requestError{http.StatusBadRequest, msg}
		case strings.HasPrefix(err.Error(), "json: unknown field "):
			field := strings.TrimPrefix(err.Error(), "json: unknown field ")
			return &requestError{http.StatusBadRequest, "unknown field " + field}
		default:
			return &requestError{http.StatusBadRequest, "malformed JSON"}
		}
	}

	if err := dec.Decode(&struct{}{}); !errors.Is(err, io.EOF) {
		return &requestError{http.StatusBadRequest, "request body must hold a single JSON value"}
	}
	return nil
}

It makes four decisions, and each failure gets a message the client can act on:

  • The Content-Type must be application/json, with or without ; charset=utf-8. Anything else gets 415 Unsupported Media Type, not 400, because the body might be fine and it’s the label that’s wrong. The status tells the client what to fix.
  • The body is capped at 1 MiB with http.MaxBytesReader. Past the limit, reads fail with *http.MaxBytesError, and the helper answers 413. Without a cap, one client could make the decoder hold gigabytes.
  • Unknown fields are rejected with DisallowUnknownFields, so a typo such as "titel" is an error instead of being silently ignored. encoding/json has no error type for this case, so the helper matches the message’s prefix.
  • Only one JSON value is allowed. Decode reads one value and stops, so a second Decode must hit io.EOF.

This program puts the same decodeJSON, copied unchanged, in front of eleven request bodies:

package main

import (
	"encoding/json"
	"errors"
	"fmt"
	"io"
	"mime"
	"net/http"
	"net/http/httptest"
	"strings"
)

// maxBodyBytes caps a request body at 1 MiB.
const maxBodyBytes = 1 << 20

// requestError is a problem with the request that the client can fix.
type requestError struct {
	status  int
	message string
}

func (e *requestError) Error() string { return e.message }

// decodeJSON reads exactly one JSON value from the body into dst.
// Every error it returns is a *requestError.
func decodeJSON(w http.ResponseWriter, r *http.Request, dst any) error {
	mediaType, _, _ := mime.ParseMediaType(r.Header.Get("Content-Type"))
	if mediaType != "application/json" {
		return &requestError{http.StatusUnsupportedMediaType, "Content-Type must be application/json"}
	}

	r.Body = http.MaxBytesReader(w, r.Body, maxBodyBytes)
	dec := json.NewDecoder(r.Body)
	dec.DisallowUnknownFields()

	if err := dec.Decode(dst); err != nil {
		var syntaxErr *json.SyntaxError
		var typeErr *json.UnmarshalTypeError
		var tooBig *http.MaxBytesError
		switch {
		case errors.As(err, &tooBig):
			msg := fmt.Sprintf("request body must not be larger than %d bytes", tooBig.Limit)
			return &requestError{http.StatusRequestEntityTooLarge, msg}
		case errors.Is(err, io.EOF):
			return &requestError{http.StatusBadRequest, "request body must not be empty"}
		case errors.Is(err, io.ErrUnexpectedEOF):
			return &requestError{http.StatusBadRequest, "request body ends in the middle of the JSON"}
		case errors.As(err, &syntaxErr):
			msg := fmt.Sprintf("malformed JSON at byte %d", syntaxErr.Offset)
			return &requestError{http.StatusBadRequest, msg}
		case errors.As(err, &typeErr) && typeErr.Field == "":
			return &requestError{http.StatusBadRequest, "request body must be a JSON object"}
		case errors.As(err, &typeErr):
			msg := fmt.Sprintf("field %q has the wrong type", typeErr.Field)
			return &requestError{http.StatusBadRequest, msg}
		case strings.HasPrefix(err.Error(), "json: unknown field "):
			field := strings.TrimPrefix(err.Error(), "json: unknown field ")
			return &requestError{http.StatusBadRequest, "unknown field " + field}
		default:
			return &requestError{http.StatusBadRequest, "malformed JSON"}
		}
	}

	if err := dec.Decode(&struct{}{}); !errors.Is(err, io.EOF) {
		return &requestError{http.StatusBadRequest, "request body must hold a single JSON value"}
	}
	return nil
}

type taskInput struct {
	Title string `json:"title"`
	Done  bool   `json:"done"`
}

func try(name, contentType, body string) {
	req := httptest.NewRequest("POST", "/tasks", strings.NewReader(body))
	req.Header.Set("Content-Type", contentType)

	var in taskInput
	err := decodeJSON(httptest.NewRecorder(), req, &in)

	var re *requestError
	if errors.As(err, &re) {
		fmt.Printf("%-13s %d %s\n", name, re.status, re.message)
		return
	}
	fmt.Printf("%-13s ok  %+v\n", name, in)
}

func main() {
	const ct = "application/json"
	try("valid", ct, `{"title":"Buy milk"}`)
	try("charset", ct+"; charset=utf-8", `{"title":"Buy milk","done":true}`)
	try("form", "application/x-www-form-urlencoded", "title=Buy+milk")
	try("empty", ct, "")
	try("broken", ct, `{"title" "Buy milk"}`)
	try("cut off", ct, `{"title":"Buy`)
	try("unknown", ct, `{"title":"Buy milk","id":7}`)
	try("wrong type", ct, `{"title":"Buy milk","done":"yes"}`)
	try("array", ct, `["Buy milk"]`)
	try("two values", ct, `{"title":"a"}{"title":"b"}`)
	try("too big", ct, `{"title":"`+strings.Repeat("a", 2<<20)+`"}`)
}

It prints:

valid         ok  {Title:Buy milk Done:false}
charset       ok  {Title:Buy milk Done:true}
form          415 Content-Type must be application/json
empty         400 request body must not be empty
broken        400 malformed JSON at byte 10
cut off       400 request body ends in the middle of the JSON
unknown       400 unknown field "id"
wrong type    400 field "done" has the wrong type
array         400 request body must be a JSON object
two values    400 request body must hold a single JSON value
too big       413 request body must not be larger than 1048576 bytes

Look at the unknown line. The input type has no ID field, so a client can’t choose its own ID by sending "id": 7. The store picks IDs, and the URL names them.

encoding/json/v2 exists in Go 1.26’s source tree, but only behind GOEXPERIMENT=jsonv2, so this module sticks to encoding/json.

Validation, and why it’s 422

Validation checks the values in a request that decoded cleanly, and reports problems per field. Here’s the whole of validate.go:

// maxTitleLen is the longest title allowed, counted in characters.
const maxTitleLen = 200

// taskInput is what a client sends for POST and PUT. It has no ID field:
// the ID comes from the store or the URL, never from the body.
type taskInput struct {
	Title string `json:"title"`
	Done  bool   `json:"done"`
}

// validate trims the title in place, then returns one message per
// invalid field, or nil if the input is fine.
func (in *taskInput) validate() map[string]string {
	fields := map[string]string{}
	in.Title = strings.TrimSpace(in.Title)
	switch {
	case in.Title == "":
		fields["title"] = "must not be empty"
	case utf8.RuneCountInString(in.Title) > maxTitleLen:
		fields["title"] = "must be at most 200 characters"
	}
	if len(fields) == 0 {
		return nil
	}
	return fields
}

The length is counted with utf8.RuneCountInString, not len, which counts bytes. A title in Chinese gets 200 characters too.

When validate returns fields, the handler answers 422 Unprocessable Content. A 400 from decodeJSON means “I couldn’t read your request”. A 422 means “I read it fine, and the values break a rule”. A client can show the field messages next to its input boxes, and treat a 400 as its own bug. Go’s constant is http.StatusUnprocessableEntity, and its status text still says “Unprocessable Entity”, the older name for the same code.

One JSON shape for every error

Every error the API sends, from 400 to 500, is a JSON object with an error message and, for validation, a fields object. A client needs one piece of code to read any of them:

// errorBody is the one shape every error response uses.
type errorBody struct {
	Error  string            `json:"error"`
	Fields map[string]string `json:"fields,omitempty"`
}

// writeJSON encodes v before writing anything, so an encoding failure
// can still become a clean 500.
func (s *server) writeJSON(w http.ResponseWriter, status int, v any) {
	body, err := json.Marshal(v)
	if err != nil {
		s.logger.Error("encode response", "err", err)
		w.Header().Set("Content-Type", "application/json")
		w.WriteHeader(http.StatusInternalServerError)
		io.WriteString(w, `{"error":"internal server error"}`+"\n")
		return
	}
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(status)
	w.Write(append(body, '\n'))
}

omitempty drops fields when there are none, so a 404 is just {"error":"task not found"}.

writeJSON calls json.Marshal before it writes anything. Once WriteHeader runs, the status can’t be taken back, so an encoder failing halfway would leave the client a 200 with half a body. Marshalling first means a failure can still become a clean 500.

A 500 never tells the client why:

// internalError logs the real error and tells the client nothing about it.
func (s *server) internalError(w http.ResponseWriter, r *http.Request, err error) {
	s.logger.ErrorContext(r.Context(), "internal error", "err", err, "request_id", requestIDFrom(r.Context()))
	s.writeError(w, http.StatusInternalServerError, "internal server error", nil)
}

The real error, which might name a database host or a file path, goes to the log. The client gets internal server error and a request ID in the X-Request-Id header, so a bug report can point to the log line.

Status codes, end to end

A status code is the first thing a client reads, so every response in the API picks one on purpose. A standalone program can’t import the module’s internal/ packages, so this program is a trimmed copy of the API: a map for a store and a short decode step, with the same statuses and error shape. It runs on httptest.NewServer, a real HTTP server on a random local port:

package main

import (
	"encoding/json"
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"
	"strconv"
	"strings"
	"sync"
)

type Task struct {
	ID    int64  `json:"id"`
	Title string `json:"title"`
	Done  bool   `json:"done"`
}

// A trimmed version of the module's API: a map for a store, and a short
// decode step in place of decodeJSON.
type api struct {
	mu     sync.Mutex
	lastID int64
	tasks  map[int64]Task
}

func writeJSON(w http.ResponseWriter, status int, v any) {
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(status)
	json.NewEncoder(w).Encode(v)
}

type errorBody struct {
	Error  string            `json:"error"`
	Fields map[string]string `json:"fields,omitempty"`
}

func writeError(w http.ResponseWriter, status int, msg string, fields map[string]string) {
	writeJSON(w, status, errorBody{Error: msg, Fields: fields})
}

func (a *api) decode(w http.ResponseWriter, r *http.Request) (Task, bool) {
	var in struct {
		Title string `json:"title"`
		Done  bool   `json:"done"`
	}
	dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<20))
	dec.DisallowUnknownFields()
	if err := dec.Decode(&in); err != nil {
		writeError(w, http.StatusBadRequest, "malformed JSON", nil)
		return Task{}, false
	}
	if strings.TrimSpace(in.Title) == "" {
		writeError(w, http.StatusUnprocessableEntity, "validation failed",
			map[string]string{"title": "must not be empty"})
		return Task{}, false
	}
	return Task{Title: in.Title, Done: in.Done}, true
}

func (a *api) create(w http.ResponseWriter, r *http.Request) {
	t, ok := a.decode(w, r)
	if !ok {
		return
	}
	a.mu.Lock()
	a.lastID++
	t.ID = a.lastID
	a.tasks[t.ID] = t
	a.mu.Unlock()
	w.Header().Set("Location", "/tasks/"+strconv.FormatInt(t.ID, 10))
	writeJSON(w, http.StatusCreated, t)
}

func (a *api) get(w http.ResponseWriter, r *http.Request) {
	id, _ := strconv.ParseInt(r.PathValue("id"), 10, 64)
	a.mu.Lock()
	t, ok := a.tasks[id]
	a.mu.Unlock()
	if !ok {
		writeError(w, http.StatusNotFound, "task not found", nil)
		return
	}
	writeJSON(w, http.StatusOK, t)
}

func (a *api) update(w http.ResponseWriter, r *http.Request) {
	id, _ := strconv.ParseInt(r.PathValue("id"), 10, 64)
	t, ok := a.decode(w, r)
	if !ok {
		return
	}
	t.ID = id
	a.mu.Lock()
	_, found := a.tasks[id]
	if found {
		a.tasks[id] = t
	}
	a.mu.Unlock()
	if !found {
		writeError(w, http.StatusNotFound, "task not found", nil)
		return
	}
	writeJSON(w, http.StatusOK, t)
}

func (a *api) remove(w http.ResponseWriter, r *http.Request) {
	id, _ := strconv.ParseInt(r.PathValue("id"), 10, 64)
	a.mu.Lock()
	_, found := a.tasks[id]
	delete(a.tasks, id)
	a.mu.Unlock()
	if !found {
		writeError(w, http.StatusNotFound, "task not found", nil)
		return
	}
	w.WriteHeader(http.StatusNoContent)
}

func main() {
	a := &api{tasks: map[int64]Task{}}
	mux := http.NewServeMux()
	mux.HandleFunc("POST /tasks", a.create)
	mux.HandleFunc("GET /tasks/{id}", a.get)
	mux.HandleFunc("PUT /tasks/{id}", a.update)
	mux.HandleFunc("DELETE /tasks/{id}", a.remove)

	srv := httptest.NewServer(mux)
	defer srv.Close()

	send := func(method, path, body string) {
		req, _ := http.NewRequest(method, srv.URL+path, strings.NewReader(body))
		req.Header.Set("Content-Type", "application/json")
		resp, err := http.DefaultClient.Do(req)
		if err != nil {
			fmt.Println(err)
			return
		}
		defer resp.Body.Close()
		out, _ := io.ReadAll(resp.Body)

		fmt.Println(method, path, "->", resp.Status)
		if loc := resp.Header.Get("Location"); loc != "" {
			fmt.Println("  Location:", loc)
		}
		if len(out) > 0 {
			fmt.Print("  ", string(out))
		}
	}

	send("POST", "/tasks", `{"title":"Buy milk"}`)
	send("GET", "/tasks/1", "")
	send("PUT", "/tasks/1", `{"title":"Buy milk","done":true}`)
	send("POST", "/tasks", `{"title":"  "}`)
	send("DELETE", "/tasks/1", "")
	send("GET", "/tasks/1", "")
	send("PATCH", "/tasks/1", `{"done":false}`)
}

It prints:

POST /tasks -> 201 Created
  Location: /tasks/1
  {"id":1,"title":"Buy milk","done":false}
GET /tasks/1 -> 200 OK
  {"id":1,"title":"Buy milk","done":false}
PUT /tasks/1 -> 200 OK
  {"id":1,"title":"Buy milk","done":true}
POST /tasks -> 422 Unprocessable Entity
  {"error":"validation failed","fields":{"title":"must not be empty"}}
DELETE /tasks/1 -> 204 No Content
GET /tasks/1 -> 404 Not Found
  {"error":"task not found"}
PATCH /tasks/1 -> 405 Method Not Allowed
  Method Not Allowed

That’s the life of one task. DELETE answers 204 No Content, which must have no body, so the handler calls WriteHeader and writes nothing. After that, the task is a 404.

The last response is plain text, Method Not Allowed, because the trimmed copy leaves 405 to the mux. The real module fixes that with a middleware in the next section.

Middleware: a handler that wraps a handler

Middleware in Go is a function of type func(http.Handler) http.Handler: it takes a handler and returns a new one that does work before or after calling it. This program wraps a handler in three layers that print as the request passes:

package main

import (
	"fmt"
	"net/http"
	"net/http/httptest"
)

// layer returns a middleware that prints when a request passes in and out.
func layer(name, indent string) func(http.Handler) http.Handler {
	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			fmt.Println(indent + name + ": in")
			next.ServeHTTP(w, r)
			fmt.Println(indent + name + ": out")
		})
	}
}

func main() {
	var h http.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		fmt.Println("      handler")
	})
	h = layer("recover", "    ")(h)
	h = layer("log", "  ")(h)
	h = layer("request id", "")(h)

	h.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest("GET", "/tasks", nil))
}

It prints:

request id: in
  log: in
    recover: in
      handler
    recover: out
  log: out
request id: out

The last layer applied, request id, is the outermost, so it runs first. Code after next.ServeHTTP runs on the way out, in reverse order.

Explain it like I’m ten

Imagine posting a letter to a big office. At the front door, someone stamps a number on the envelope. Next, a clerk notes the time it arrived. Then it passes a first-aid helper, who is there in case something goes wrong. Only then does it reach the person who writes the answer.

The answer goes back out the same way, in reverse. The first-aid helper waves it through. The clerk writes down “letter 7: answered, yes”. The front door sends it off.

Each person only knows who to hand the letter to next.

The precise version

A middleware returns an http.HandlerFunc that closes over next. a(b(c(h))) builds a chain in which a‘s handler calls b‘s, and so on down to h. Code before next.ServeHTTP runs outside-in. Code after it, including deferred functions, runs inside-out. So a deferred recover in one layer catches a panic from every layer inside it.

A layer can answer by itself and never call next, or change the request on the way in with r.WithContext. It can’t read the response on the way out unless it wraps the http.ResponseWriter.

Where the analogy breaks: the answer isn’t a second letter that travels back past everyone. The handler writes straight into the http.ResponseWriter, and the bytes can reach the client before the outer layers’ “after” code runs. That’s why the logging layer below wraps the writer to find out the status code.

withRequestID gives the request an ID, sets X-Request-Id logRequests logs one line after the inside returns recoverPanics turns a panic into a 500 jsonRouteErrors sends the mux's 404 and 405 as JSON ServeMux, then a handler such as createTask

The order New builds. A request passes each layer on the way in, and each layer’s code after next.ServeHTTP runs on the way out, innermost first.

The first middleware in the module gives each request an ID:

// requestIDKey is the context key for the request ID. It's unexported,
// so no other package can read or overwrite the value by accident.
type requestIDKey struct{}

// withRequestID gives every request an ID from a counter, stores it in the
// request's context and sends it back in the X-Request-Id header.
func withRequestID(next http.Handler) http.Handler {
	var counter atomic.Uint64
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		id := "req-" + strconv.FormatUint(counter.Add(1), 10)
		w.Header().Set("X-Request-Id", id)
		ctx := context.WithValue(r.Context(), requestIDKey{}, id)
		next.ServeHTTP(w, r.WithContext(ctx))
	})
}

// requestIDFrom returns the request ID stored by withRequestID, or "".
func requestIDFrom(ctx context.Context) string {
	id, _ := ctx.Value(requestIDKey{}).(string)
	return id
}

The counter is local to withRequestID, so each call to New starts again at req-1, which keeps IDs predictable in tests. It’s an atomic.Uint64 because requests run in parallel. The ID travels in the context under an unexported key type, the pattern from the part on context.

The innermost middleware makes the mux’s own errors match the API’s error shape:

// jsonRouteErrors lets the mux decide 404 and 405 for requests that match no
// pattern, including the Allow header, but sends the body as JSON.
func (s *server) jsonRouteErrors(mux *http.ServeMux) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		h, pattern := mux.Handler(r)
		if pattern != "" {
			mux.ServeHTTP(w, r)
			return
		}
		// No pattern matched. h is the mux's own 404 or 405 handler.
		// Run it against a writer that keeps the status and drops the text.
		probe := &statusOnly{header: w.Header()}
		h.ServeHTTP(probe, r)
		s.writeError(w, probe.status, strings.ToLower(http.StatusText(probe.status)), nil)
	})
}

// statusOnly is a ResponseWriter that shares the real headers but keeps
// only the status code, discarding the body.
type statusOnly struct {
	header http.Header
	status int
}

func (p *statusOnly) Header() http.Header         { return p.header }
func (p *statusOnly) WriteHeader(status int)      { p.status = status }
func (p *statusOnly) Write(b []byte) (int, error) { return len(b), nil }

mux.Handler(r) asks the mux which pattern would handle the request, without running it. An empty pattern means no route matched, and the handler returned is the mux’s own 404 or 405 responder. Running it against statusOnly keeps the status and the Allow header and throws away the text. The client gets {"error":"method not allowed"}.

Logging with log/slog

The log/slog package writes structured logs: a message plus key-value pairs a machine can search. Its JSON and text handlers add the current time to every line. That would change the output here on every run, so the programs in this post use ReplaceAttr to drop the time key:

package main

import (
	"errors"
	"log/slog"
	"os"
	"time"
)

func main() {
	opts := &slog.HandlerOptions{
		// Drop the time so the output is the same on every run.
		ReplaceAttr: func(groups []string, a slog.Attr) slog.Attr {
			if a.Key == slog.TimeKey && len(groups) == 0 {
				return slog.Attr{}
			}
			return a
		},
	}

	logger := slog.New(slog.NewJSONHandler(os.Stdout, opts))
	logger.Info("request", "method", "GET", "status", 200, "duration", 1500*time.Microsecond)

	reqLogger := logger.With("request_id", "req-7")
	reqLogger.Error("internal error", "err", errors.New("disk full"))

	text := slog.New(slog.NewTextHandler(os.Stdout, opts))
	text.Info("request", "method", "GET", "status", 200, "duration", 1500*time.Microsecond)
}

It prints:

{"level":"INFO","msg":"request","method":"GET","status":200,"duration":1500000}
{"level":"ERROR","msg":"internal error","request_id":"req-7","err":"disk full"}
level=INFO msg=request method=GET status=200 duration=1.5ms

ReplaceAttr sees every attribute before it’s written, and returning an empty slog.Attr removes it. The len(groups) == 0 check keeps a time field inside a group. logger.With returns a logger that adds request_id to every line.

Check the duration values. The JSON handler writes a time.Duration as a whole number of nanoseconds, 1500000, while the text handler writes 1.5ms. Anything that searches your logs needs to know which one it’s reading.

The logging middleware needs the status code, which the handler writes into the ResponseWriter. So it hands the handler a wrapper that remembers it:

// statusRecorder remembers the status code a handler wrote.
type statusRecorder struct {
	http.ResponseWriter
	status int
}

func (rec *statusRecorder) WriteHeader(status int) {
	if rec.status == 0 {
		rec.status = status
	}
	rec.ResponseWriter.WriteHeader(status)
}

func (rec *statusRecorder) Write(b []byte) (int, error) {
	if rec.status == 0 {
		rec.status = http.StatusOK
	}
	return rec.ResponseWriter.Write(b)
}

// Unwrap lets http.ResponseController reach the real ResponseWriter.
func (rec *statusRecorder) Unwrap() http.ResponseWriter {
	return rec.ResponseWriter
}

// logRequests writes one log line per request, after the handler returns.
func (s *server) logRequests(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		start := time.Now()
		rec := &statusRecorder{ResponseWriter: w}
		next.ServeHTTP(rec, r)
		s.logger.LogAttrs(r.Context(), slog.LevelInfo, "request",
			slog.String("method", r.Method),
			slog.String("path", r.URL.Path),
			slog.Int("status", rec.status),
			slog.Duration("duration", time.Since(start)),
			slog.String("request_id", requestIDFrom(r.Context())),
		)
	})
}

statusRecorder embeds http.ResponseWriter and overrides two methods. A Write without WriteHeader sends a 200, so Write records that too. Unwrap lets http.NewResponseController reach the original writer, for features such as flushing that the wrapper doesn’t have.

Recovering from a panic

A panic in a handler doesn’t crash a Go HTTP server, but the client gets no answer. net/http recovers it, logs a stack trace and closes the connection. When I tried it, the client’s http.Get returned an EOF error and no status code. The module’s middleware turns it into a proper 500:

// recoverPanics turns a panic in a handler into a 500 response. The panic
// value and stack go to the log, never to the client.
func (s *server) recoverPanics(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		defer func() {
			v := recover()
			if v == nil {
				return
			}
			if v == http.ErrAbortHandler {
				panic(v)
			}
			s.logger.LogAttrs(r.Context(), slog.LevelError, "panic",
				slog.Any("value", v),
				slog.String("stack", string(debug.Stack())),
				slog.String("request_id", requestIDFrom(r.Context())),
			)
			s.writeError(w, http.StatusInternalServerError, "internal server error", nil)
		}()
		next.ServeHTTP(w, r)
	})
}

The panic value and the stack trace go to the log, and the client gets the same generic 500 body as any other internal error. http.ErrAbortHandler is re-panicked on purpose: a handler panics with it to abort a response deliberately, and net/http handles it quietly.

This program copies the three middlewares unchanged and wraps them around a handler that indexes a slice without checking the bounds. The log handler also drops duration and stack, which change on every run:

package main

import (
	"context"
	"fmt"
	"log/slog"
	"net/http"
	"net/http/httptest"
	"os"
	"runtime/debug"
	"strconv"
	"sync/atomic"
	"time"
)

type server struct {
	logger *slog.Logger
}

func (s *server) writeError(w http.ResponseWriter, status int, msg string, fields map[string]string) {
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(status)
	fmt.Fprintf(w, "{\"error\":%q}\n", msg)
}

// The middleware below is copied unchanged from internal/api/middleware.go.

// requestIDKey is the context key for the request ID. It's unexported,
// so no other package can read or overwrite the value by accident.
type requestIDKey struct{}

// withRequestID gives every request an ID from a counter, stores it in the
// request's context and sends it back in the X-Request-Id header.
func withRequestID(next http.Handler) http.Handler {
	var counter atomic.Uint64
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		id := "req-" + strconv.FormatUint(counter.Add(1), 10)
		w.Header().Set("X-Request-Id", id)
		ctx := context.WithValue(r.Context(), requestIDKey{}, id)
		next.ServeHTTP(w, r.WithContext(ctx))
	})
}

// requestIDFrom returns the request ID stored by withRequestID, or "".
func requestIDFrom(ctx context.Context) string {
	id, _ := ctx.Value(requestIDKey{}).(string)
	return id
}

// statusRecorder remembers the status code a handler wrote.
type statusRecorder struct {
	http.ResponseWriter
	status int
}

func (rec *statusRecorder) WriteHeader(status int) {
	if rec.status == 0 {
		rec.status = status
	}
	rec.ResponseWriter.WriteHeader(status)
}

func (rec *statusRecorder) Write(b []byte) (int, error) {
	if rec.status == 0 {
		rec.status = http.StatusOK
	}
	return rec.ResponseWriter.Write(b)
}

// Unwrap lets http.ResponseController reach the real ResponseWriter.
func (rec *statusRecorder) Unwrap() http.ResponseWriter {
	return rec.ResponseWriter
}

// logRequests writes one log line per request, after the handler returns.
func (s *server) logRequests(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		start := time.Now()
		rec := &statusRecorder{ResponseWriter: w}
		next.ServeHTTP(rec, r)
		s.logger.LogAttrs(r.Context(), slog.LevelInfo, "request",
			slog.String("method", r.Method),
			slog.String("path", r.URL.Path),
			slog.Int("status", rec.status),
			slog.Duration("duration", time.Since(start)),
			slog.String("request_id", requestIDFrom(r.Context())),
		)
	})
}

// recoverPanics turns a panic in a handler into a 500 response. The panic
// value and stack go to the log, never to the client.
func (s *server) recoverPanics(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		defer func() {
			v := recover()
			if v == nil {
				return
			}
			if v == http.ErrAbortHandler {
				panic(v)
			}
			s.logger.LogAttrs(r.Context(), slog.LevelError, "panic",
				slog.Any("value", v),
				slog.String("stack", string(debug.Stack())),
				slog.String("request_id", requestIDFrom(r.Context())),
			)
			s.writeError(w, http.StatusInternalServerError, "internal server error", nil)
		}()
		next.ServeHTTP(w, r)
	})
}

func main() {
	// Drop the attributes that change on every run: time, duration, stack.
	opts := &slog.HandlerOptions{
		ReplaceAttr: func(groups []string, a slog.Attr) slog.Attr {
			switch a.Key {
			case slog.TimeKey, "duration", "stack":
				return slog.Attr{}
			}
			return a
		},
	}
	s := &server{logger: slog.New(slog.NewJSONHandler(os.Stdout, opts))}

	titles := []string{"Buy milk", "Walk the dog"}
	mux := http.NewServeMux()
	mux.HandleFunc("GET /tasks/{id}", func(w http.ResponseWriter, r *http.Request) {
		id, _ := strconv.Atoi(r.PathValue("id"))
		fmt.Fprintln(w, titles[id-1]) // a bug: no bounds check
	})

	var h http.Handler = s.recoverPanics(mux)
	h = s.logRequests(h)
	h = withRequestID(h)

	for _, path := range []string{"/tasks/2", "/tasks/5"} {
		rec := httptest.NewRecorder()
		h.ServeHTTP(rec, httptest.NewRequest("GET", path, nil))
		fmt.Printf("client got %d %s: %s", rec.Code, rec.Header().Get("X-Request-Id"), rec.Body)
	}
}

It prints:

{"level":"INFO","msg":"request","method":"GET","path":"/tasks/2","status":200,"request_id":"req-1"}
client got 200 req-1: Walk the dog
{"level":"ERROR","msg":"panic","value":"runtime error: index out of range [4] with length 2","request_id":"req-2"}
{"level":"INFO","msg":"request","method":"GET","path":"/tasks/5","status":500,"request_id":"req-2"}
client got 500 req-2: {"error":"internal server error"}

The second request panics. recoverPanics logs the panic with req-2, writes the 500 and returns normally, so logRequests still logs status: 500 with the same ID. The client’s body says nothing about slices. That depends on the order in New: with recoverPanics outside logRequests, the panic would skip the log line.

One limit: if a handler has already sent its status when it panics, it’s too late for a 500.

Running the server

The command in cmd/tasksd builds the pieces and starts an http.Server. main reads an -addr flag, creates a JSON logger on standard error, and calls run, which opens the listener and hands it to serve:

func run(addr string, logger *slog.Logger) error {
	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
	defer stop()

	ln, err := net.Listen("tcp", addr)
	if err != nil {
		return err
	}
	return serve(ctx, ln, api.New(task.NewMemStore(), logger), logger)
}

// serve answers requests on ln until ctx is cancelled, then shuts down
// gracefully: no new connections, and requests in flight get to finish.
func serve(ctx context.Context, ln net.Listener, handler http.Handler, logger *slog.Logger) error {
	srv := &http.Server{
		Handler:           handler,
		ReadHeaderTimeout: 5 * time.Second,
		ReadTimeout:       10 * time.Second,
		WriteTimeout:      10 * time.Second,
		IdleTimeout:       60 * time.Second,
		ErrorLog:          slog.NewLogLogger(logger.Handler(), slog.LevelError),
	}

	errc := make(chan error, 1)
	go func() {
		logger.Info("listening", "addr", ln.Addr().String())
		errc <- srv.Serve(ln)
	}()

	select {
	case err := <-errc:
		return err
	case <-ctx.Done():
	}

	logger.Info("shutting down")
	shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel()
	if err := srv.Shutdown(shutdownCtx); err != nil {
		return err
	}
	if err := <-errc; !errors.Is(err, http.ErrServerClosed) {
		return err
	}
	return nil
}

This is the only place that chooses MemStore. serve takes a listener and a handler instead of an address, so a test can run it on a free port with a handler of its own. The timeouts are the ones from the previous part, with ReadHeaderTimeout as the one to never skip. signal.NotifyContext cancels ctx on Ctrl+C, and srv.Shutdown then waits for requests in flight to finish. The next part explains the timeouts and the shutdown in detail.

With go run ./cmd/tasksd running in one terminal, I sent requests from another:

$ curl -s -X POST localhost:8080/tasks -H 'Content-Type: application/json' -d '{"title":"Buy milk"}'
{"id":1,"title":"Buy milk","done":false}
$ curl -s -X PUT localhost:8080/tasks/1 -H 'Content-Type: application/json' -d '{"title":"Buy milk","done":true}'
{"id":1,"title":"Buy milk","done":true}
$ curl -s localhost:8080/tasks
[{"id":1,"title":"Buy milk","done":true}]
$ curl -s -X PATCH localhost:8080/tasks/1 -w '%{http_code}\n'
{"error":"method not allowed"}
405
$ curl -s -X POST localhost:8080/tasks -d 'title=Walk the dog' -w '%{http_code}\n'
{"error":"Content-Type must be application/json"}
415

The 405 is JSON this time, thanks to jsonRouteErrors. The last request sends a form body, which is what curl -d sends by default, and gets 415. The server’s terminal showed one JSON log line per request. I haven’t pasted them, because the time and duration change on every run.

Where a database would go

A real task list has to survive a restart, and the Store interface is where a database plugs in without the handlers changing. database/sql is in the standard library, but every database driver is a third-party module, so this series stops at the shape. A SQL store’s Get would look like this:

// SQLStore keeps tasks in a SQL database. With the other four methods
// written the same way, it satisfies Store just as MemStore does.
type SQLStore struct {
	db *sql.DB
}

func (s *SQLStore) Get(ctx context.Context, id int64) (Task, error) {
	var t Task
	err := s.db.QueryRowContext(ctx,
		"SELECT id, title, done FROM tasks WHERE id = $1", id,
	).Scan(&t.ID, &t.Title, &t.Done)
	if errors.Is(err, sql.ErrNoRows) {
		return Task{}, ErrNotFound
	}
	if err != nil {
		return Task{}, fmt.Errorf("get task %d: %w", id, err)
	}
	return t, nil
}

QueryRowContext takes the request’s context, so a client that disconnects cancels its query. sql.ErrNoRows becomes ErrNotFound, so getTask still answers 404, and other errors are wrapped and become a 500. The $1 placeholder is PostgreSQL’s style; MySQL and SQLite drivers use ?. In run, the SQL store would replace task.NewMemStore(), and nothing in internal/api would change.

What to remember

  • Put storage behind a small interface, and return sentinel errors that handlers check with errors.Is.
  • Decode bodies through one helper: check Content-Type (415), cap the size with http.MaxBytesReader (413), call DisallowUnknownFields, and allow a single JSON value (400).
  • Answer 422 with per-field messages for well-formed input that breaks a rule, 201 with Location for a create, and 204 with no body for a delete.
  • Send every error in one JSON shape, marshal before you write the status, and log internal errors instead of returning them.
  • Middleware is func(http.Handler) http.Handler. The last one applied runs first, so put request IDs outside logging, and logging outside panic recovery. Log with slog through a wrapped ResponseWriter.
  • Build the handler with a constructor that takes its dependencies and uses no globals.

Choose every status code on purpose, and send every error in the same shape.

How useful was this post?

Click on a heart to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.