Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ a := api.New(
// {"level":"INFO","msg":"request","event_id":"…","method":"GET","uri":"/login?access_token=[REDACTED]&page=2","headers":{"Pin":["[REDACTED]"]}}
```

`api.ProcessRequest` writes a `request` record (`event_id`, `method`, `uri`, `headers`) and a `response` record (`event_id`, `method`, `uri`, `status`, `status_text`, `duration_ms`) at info level. A rejected body, a failed `Get*Value*`/`UnmarshalBody` helper, a recovered panic (`panic recovered`) and a response that cannot be encoded are error records with an `error` or `panic` attribute; a panic with `http.ErrAbortHandler`, plain or wrapped, is a `handler aborted` debug record and is re-panicked as the plain `http.ErrAbortHandler`, so `net/http` aborts the response without logging it; nothing is written and there is no `response` record (nor after any other recovered panic). A panic record always has `panic_type`, the Go type of the value (`string`, `*errors.errorString`, `runtime.boundsError`); its `panic` is the value only for a `runtime.Error` (nil map write, index out of range), which carries no request data, and `[REDACTED]` for any other value, which may. A failed `Get*Value*` helper adds the name as `key`; for a sensitive name its `error` is only the kind (`invalid syntax`, `value out of range`, `invalid value`), never the value. A response that cannot be encoded (for example a `NaN` in the content) is answered with the `Error500()` body instead; so is a nil formatted response (a custom formatter that returned nil), logged as `response cannot be written` with `error` `nil formatted response`; the writer gets no request, so its records carry no request context or `event_id`. Bodies and the parsed form are never logged (query values appear only in `uri`). Header values with a sensitive name, query values in `uri` and in other header values (after each `?`), and the user information of the `uri` and of URL-valued headers are masked (see [Security notes](#security-notes)); secrets in path segments are not masked. The sensitive keys are the default list (`authorization`, `token`, `password`, `secret`, `credential`, `apikey`, `api-key`, `key`, `cookie`, `session`) plus the keys of `api.WithSensitiveLogKeys`, which can only add. `ExampleWithLogger` is the compiled version.
`api.ProcessRequest` writes a `request` record (`event_id`, `method`, `uri`, `headers`) and a `response` record (`event_id`, `method`, `uri`, `status`, `status_text`, `duration_ms`) at info level. A rejected body, a failed `Get*Value*`/`UnmarshalBody` helper, a recovered panic (`panic recovered`) and a response that cannot be encoded are error records with an `error` or `panic` attribute; a panic with `http.ErrAbortHandler`, plain or wrapped, is a `handler aborted` debug record and is re-panicked as the plain `http.ErrAbortHandler`, so `net/http` aborts the response without logging it; nothing is written and there is no `response` record (nor after any other recovered panic). A panic record always has `panic_type`, the Go type of the value (`string`, `*errors.errorString`, `runtime.boundsError`); its `panic` is the value only for a `runtime.Error` (nil map write, index out of range), which carries no request data, and `[REDACTED]` for any other value, which may. A failed `Get*Value*` helper adds the name as `key`; for a sensitive name its `error` is only the kind (`invalid syntax`, `value out of range`, `invalid value`), never the value. A response that cannot be encoded (for example a `NaN` in the content) is answered with the `Error500()` body instead; so is a nil formatted response (a custom formatter that returned nil), logged as `response cannot be written` with `error` `nil formatted response`; the writer gets no request, so its records carry no request context or `event_id`. Bodies and the parsed form are never logged (query values appear only in `uri`). The `error` of a body that cannot be decoded (`request body rejected`, `request body not unmarshaled`) carries no body value from `encoding/json` either: a JSON type mismatch gives the kind of value, the field, the Go type and the offset (`json: cannot unmarshal number into Go struct field pin_code of type int16 (offset 21)`), and a JSON syntax error only the offset (`json: syntax error at offset 14`). Any other `UnmarshalBody` error, such as a bad `time.Time` value, is logged as `request body cannot be read or decoded`, except the body limit. Other errors of a request receiver keep their text, so a custom `APIRequestReceiver` must keep body values out of its errors. Header values with a sensitive name, query values in `uri` and in other header values (after each `?`), and the user information of the `uri` and of URL-valued headers are masked (see [Security notes](#security-notes)); secrets in path segments are not masked. The sensitive keys are the default list (`authorization`, `token`, `password`, `secret`, `credential`, `apikey`, `api-key`, `key`, `cookie`, `session`) plus the keys of `api.WithSensitiveLogKeys`, which can only add. `ExampleWithLogger` is the compiled version.

Since v1.0.0 the default instance writes the same records through `slog.Default()`, read at call time (route them with `slog.SetDefault`). Its sensitive keys are the default list plus `api.SensitiveLogKeys`, which can add keys but no longer remove a default one. Neither calls `api.Fatal`: a response that cannot be encoded is answered with 500. The package no longer calls `api.Print` or `api.PrintError` (only the deprecated `api.LogRequest`, `api.LogResponse` and `api.LogResponseWithDuration` call `api.Print`). `api.Print`, `api.PrintError`, `api.Fatal`, `api.SensitiveLogKeys` and the `api.Log*` functions are deprecated in favor of the options; `api.LogRequestBody`, `api.LogResponseBody`, `api.PrintFullEvent` and `api.MaxLoggedBodyBytes` are deprecated too: they affect only the deprecated `api.Log*` functions, since bodies are never logged.

Expand Down
2 changes: 1 addition & 1 deletion functions.go
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ func UnmarshalBody(v interface{}, r *http.Request) Response {
err := api.UnmarshalBody(v, r)
if err != nil {
a := instanceFrom(r)
a.logError(r, "request body not unmarshaled", err)
a.logBodyError(r, "request body not unmarshaled", err, true)
if bodyTooLarge(err) {
return bodyTooLargeError()
}
Expand Down
38 changes: 34 additions & 4 deletions instance_log.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,43 @@ func (a *API) logs(ctx context.Context, level slog.Level) bool {
return a.log().Enabled(ctx, level)
}

// logError writes an error record with message msg for err, met while
// serving r.
func (a *API) logError(r *http.Request, msg string, err error) {
// logBodyError reports err from reading or decoding the request body. A
// json.UnmarshalTypeError or json.SyntaxError is described without the body
// values it quotes (see jsonErrorText). Any other error is logged as it is,
// unless strict is true: then only the body limit keeps its text and the rest
// become a fixed text, because UnmarshalBody's other encoding/json errors
// (from a json.Unmarshaler such as time.Time, for one) may quote the value.
func (a *API) logBodyError(r *http.Request, msg string, err error, strict bool) {
if !a.logs(r.Context(), slog.LevelError) {
return
}
a.log().LogAttrs(r.Context(), slog.LevelError, msg, slog.Any("error", err))
errAttr := slog.Any("error", err)
if text, ok := jsonErrorText(err); ok {
errAttr = slog.String("error", text)
} else if strict && !bodyTooLarge(err) {
errAttr = slog.String("error", "request body cannot be read or decoded")
}
a.log().LogAttrs(r.Context(), slog.LevelError, msg, errAttr)
}

// jsonErrorText describes a json.UnmarshalTypeError by the kind of JSON value
// (never the value), the field, the Go type and the offset, and a
// json.SyntaxError by its offset only. ok is false for any other error.
func jsonErrorText(err error) (text string, ok bool) {
var typeErr *json.UnmarshalTypeError
if errors.As(err, &typeErr) {
kind, _, _ := strings.Cut(typeErr.Value, " ")
target := "Go value"
if typeErr.Field != "" {
target = "Go struct field " + typeErr.Field
}
return fmt.Sprintf("json: cannot unmarshal %s into %s of type %s (offset %d)", kind, target, typeErr.Type, typeErr.Offset), true
}
var syntaxErr *json.SyntaxError
if errors.As(err, &syntaxErr) {
return fmt.Sprintf("json: syntax error at offset %d", syntaxErr.Offset), true
}
return "", false
}

// logNamedError reports err from reading the value of the header, query
Expand Down
143 changes: 143 additions & 0 deletions instance_log_body_error_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
package api

import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)

// TestBodyDecodeErrorsAreLoggedWithoutValues checks that the error record of
// a body that cannot be decoded names the field, type and offset but never a
// value of the body.
func TestBodyDecodeErrorsAreLoggedWithoutValues(t *testing.T) {
tests := []struct {
name string
body string
wantError string
secret string
}{
{
name: "number overflows its field",
body: `{"pin_code":123456789}`,
wantError: `json: cannot unmarshal number into Go struct field pin_code of type int16 (offset 21)`,
secret: "123456789",
},
{
name: "negative number into an unsigned field",
body: `{"account":-4242}`,
wantError: `json: cannot unmarshal number into Go struct field account of type uint32 (offset 16)`,
secret: "4242",
},
{
name: "syntax error",
body: `{"pin_code":9x}`,
wantError: `json: syntax error at offset 14`,
secret: "'x'",
},
{
name: "bad value for a string-tagged number",
body: `{"amount":"98765x"}`,
wantError: "json: cannot unmarshal number into Go struct field amount of type int (offset 18)",
secret: "98765x",
},
{
name: "bad time",
body: `{"at":"secret-4711"}`,
wantError: "request body cannot be read or decoded",
secret: "secret-4711",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
logger, buf := newRecordingLogger()
handler := New(WithLogger(logger)).Handler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var in struct {
PIN int16 `json:"pin_code"`
Account uint32 `json:"account"`
Amount int `json:"amount,string"`
At time.Time `json:"at"`
}
if response := UnmarshalBody(&in, r); response != nil {
response.Write(w, r)
}
}))

handler.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest(http.MethodPost, "/pay", strings.NewReader(tt.body)))

record := recordNamed(t, decodeRecords(t, buf), "request body not unmarshaled")
if record["error"] != tt.wantError {
t.Fatalf("error = %q, want %q", record["error"], tt.wantError)
}
if strings.Contains(buf.String(), tt.secret) {
t.Fatalf("log %q contains body value %q", buf.String(), tt.secret)
}
})
}
}

// TestBodyTooLargeErrorIsLoggedAsIs checks that an error without body values,
// such as the body limit, keeps its text.
func TestBodyTooLargeErrorIsLoggedAsIs(t *testing.T) {
logger, buf := newRecordingLogger()
handler := New(WithLogger(logger), WithMaxBodyBytes(4)).Handler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var in struct{}
if response := UnmarshalBody(&in, r); response != nil {
response.Write(w, r)
}
}))

handler.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest(http.MethodPost, "/pay", strings.NewReader(`{"a":1}`)))

record := recordNamed(t, decodeRecords(t, buf), "request body not unmarshaled")
if record["error"] != "http: request body too large" {
t.Fatalf("error = %q, want the body limit error", record["error"])
}
}

// TestRejectedBodyIsLoggedWithoutValues checks the ProcessRequest and
// RequestBody records of a body that is not valid JSON.
func TestRejectedBodyIsLoggedWithoutValues(t *testing.T) {
for name, middleware := range map[string]func(http.HandlerFunc) http.HandlerFunc{
"ProcessRequest": ProcessRequest,
"RequestBody": RequestBody,
} {
t.Run(name, func(t *testing.T) {
logger, buf := newRecordingLogger()
handler := New(WithLogger(logger), WithRequestMode(RequestModeEnvelope)).Handler(middleware(okHandler))

handler.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest(http.MethodPost, "/pay", strings.NewReader(`{"content":7x}`)))

record := recordNamed(t, decodeRecords(t, buf), "request body rejected")
if record["error"] != "json: syntax error at offset 13" {
t.Fatalf("error = %q, want the offset only", record["error"])
}
if strings.Contains(buf.String(), "'x'") {
t.Fatalf("log %q contains the body byte", buf.String())
}
})
}
}

// TestTopLevelTypeErrorIsLoggedWithoutValue checks the description of a type
// error outside a struct field.
func TestTopLevelTypeErrorIsLoggedWithoutValue(t *testing.T) {
logger, buf := newRecordingLogger()
handler := New(WithLogger(logger)).Handler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var pin int16
if response := UnmarshalBody(&pin, r); response != nil {
response.Write(w, r)
}
}))

handler.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest(http.MethodPost, "/pay", strings.NewReader(`123456789`)))

record := recordNamed(t, decodeRecords(t, buf), "request body not unmarshaled")
if want := "json: cannot unmarshal number into Go value of type int16 (offset 9)"; record["error"] != want {
t.Fatalf("error = %q, want %q", record["error"], want)
}
if strings.Contains(buf.String(), "123456789") {
t.Fatalf("log %q contains the body value", buf.String())
}
}
4 changes: 2 additions & 2 deletions middlewares.go
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ func NewRequestBodyMiddleware(keyListMethods string) core.Middleware {
boundDefaultBody(w, r)
requestData, err := a.core.ProcessBody(r)
if err != nil {
a.logError(r, "request body rejected", err)
a.logBodyError(r, "request body rejected", err, false)
bodyRejection(err).Write(w, r)
return
}
Expand Down Expand Up @@ -244,7 +244,7 @@ func ProcessRequest(next http.HandlerFunc) http.HandlerFunc {
boundDefaultBody(w, r)
rctx, err := processBody(r)
if err != nil {
a.logError(r, "request body rejected", err)
a.logBodyError(r, "request body rejected", err, false)
bodyRejection(err).Write(w, r)
return
}
Expand Down
Loading