Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ a := api.New(
// {"level":"INFO","msg":"request","event_id":"…","method":"GET","uri":"/login?access_token=[REDACTED]&page=2","headers":{"Pin":["[REDACTED]"]}}
```

`api.ProcessRequest` writes a `request` record (`event_id`, `method`, `uri`, `headers`) and a `response` record (`event_id`, `method`, `uri`, `status`, `status_text`, `duration_ms`) at info level. A rejected body, a failed `Get*Value*`/`UnmarshalBody` helper, a recovered panic (`panic recovered`) and a response that cannot be encoded are error records with an `error` or `panic` attribute; a panic with `http.ErrAbortHandler`, plain or wrapped, is a `handler aborted` debug record and is re-panicked as the plain `http.ErrAbortHandler`, so `net/http` aborts the response without logging it; nothing is written and there is no `response` record (nor after any other recovered panic). A panic record always has `panic_type`, the Go type of the value (`string`, `*errors.errorString`, `runtime.boundsError`); its `panic` is the value only for a `runtime.Error` (nil map write, index out of range), which carries no request data, and `[REDACTED]` for any other value, which may. A failed `Get*Value*` helper adds the name as `key`; for a sensitive name its `error` is only the kind (`invalid syntax`, `value out of range`, `invalid value`), never the value. A response that cannot be encoded (for example a `NaN` in the content) is answered with the `Error500()` body instead; so is a nil formatted response (a custom formatter that returned nil), logged as `response cannot be written` with `error` `nil formatted response`; the writer gets no request, so its records carry no request context or `event_id`. Bodies and the parsed form are never logged (query values appear only in `uri`). Header values with a sensitive name, query values in `uri` and in other header values (after each `?`), and the user information of the `uri` and of URL-valued headers are masked (see [Security notes](#security-notes)); secrets in path segments are not masked. The sensitive keys are the default list (`authorization`, `token`, `password`, `secret`, `credential`, `apikey`, `api-key`, `key`, `cookie`, `session`) plus the keys of `api.WithSensitiveLogKeys`, which can only add. `ExampleWithLogger` is the compiled version.
`api.ProcessRequest` writes a `request` record (`event_id`, `method`, `uri`, `headers`) and a `response` record (`event_id`, `method`, `uri`, `status`, `status_text`, `duration_ms`) at info level. A rejected body, a failed `Get*Value*`/`UnmarshalBody` helper, a recovered panic (`panic recovered`) and a response that cannot be encoded are error records with an `error` or `panic` attribute; a panic with `http.ErrAbortHandler`, plain or wrapped, is a `handler aborted` debug record and is re-panicked as the plain `http.ErrAbortHandler`, so `net/http` aborts the response without logging it; nothing is written and there is no `response` record (nor after any other recovered panic). A panic record always has `panic_type`, the Go type of the value (`string`, `*errors.errorString`, `runtime.boundsError`); its `panic` is the value only for a `runtime.Error` (nil map write, index out of range), which carries no request data, and `[REDACTED]` for any other value, which may. A failed `Get*Value*` helper adds the name as `key`; for a sensitive name its `error` is only the kind (`invalid syntax`, `value out of range`, `invalid value`), never the value. A response that cannot be encoded (for example a `NaN` in the content) is answered with the `Error500()` body instead; so is a nil formatted response (a custom formatter that returned nil), logged as `response cannot be written` with `error` `nil formatted response`; the writer gets no request, so its records carry no request context or `event_id`. Bodies and the parsed form are never logged (query values appear only in `uri`). Header values with a sensitive name, query values in `uri` and in other header values (after each `?`), and the user information of the `uri` and of URL-valued headers are masked (see [Security notes](#security-notes)); secrets in path segments are not masked. The sensitive keys are the default list (`authorization`, `token`, `password`, `secret`, `credential`, `apikey`, `api-key`, `key`, `cookie`, `session`, and since v1.1.2 `code`, `signature`, `sig`, which mask an OAuth `code` and signed-URL signatures such as `X-Amz-Signature`) plus the keys of `api.WithSensitiveLogKeys`, which can only add. `ExampleWithLogger` is the compiled version.

Since v1.0.0 the default instance writes the same records through `slog.Default()`, read at call time (route them with `slog.SetDefault`). Its sensitive keys are the default list plus `api.SensitiveLogKeys`, which can add keys but no longer remove a default one. Neither calls `api.Fatal`: a response that cannot be encoded is answered with 500. The package no longer calls `api.Print` or `api.PrintError` (only the deprecated `api.LogRequest`, `api.LogResponse` and `api.LogResponseWithDuration` call `api.Print`). `api.Print`, `api.PrintError`, `api.Fatal`, `api.SensitiveLogKeys` and the `api.Log*` functions are deprecated in favor of the options; `api.LogRequestBody`, `api.LogResponseBody`, `api.PrintFullEvent` and `api.MaxLoggedBodyBytes` are deprecated too: they affect only the deprecated `api.Log*` functions, since bodies are never logged.

Expand Down Expand Up @@ -373,8 +373,8 @@ See [ADR 0002](docs/adr/0002-stdadapter-servemux.md).
- `doc.Security` and `docs.SetDefaultSecurity` only document authentication; they do not enforce it. Enforce it with a middleware: `api.CustomToken` (validated by `api.CustomTokenValidatorFunc`), `api.BasicToken` (validated by `api.ValidateBasicAuthCredentialsFunc`), or your own.
- `api.BasicToken` with the default validator answers 401 to every request while `api.Username` or `api.Password` is empty or `"default"` (since v0.11.0; before, it accepted `default:default`). Set both, or replace `api.ValidateBasicAuthCredentialsFunc`, before exposing a route. The default validator compares credentials in constant time; a custom one should too (`crypto/subtle`). After a successful check, `BasicToken` copies the user name into the `Basic-Client` request header. Since v1.0.0 it no longer copies the plain password into a `Basic-Secret` header, and it removes a `Basic-Secret` header the client sent; a handler that needs the password must take it from its own credential source.
- The request token is never echoed (since v1.0.0, on the default instance too): a response body carries a `token` only when the handler sets `SecurityToken` on the response, and `api.RequestHeaderSession` does not copy the `SecurityToken` request header into the response header; a handler that wants to return a token sets it itself.
- `api.ProcessRequest` never logs bodies or the parsed form. It masks, by its sensitive keys (a name matches when it contains a key, ignoring case; the defaults include `authorization`, `token`, `password`, `key`, `cookie` and `session`, plus `api.WithSensitiveLogKeys` on an instance or `api.SensitiveLogKeys` on the default instance): header values and query parameter values in `uri` (names are found by splitting on `&` and `;`, and a masked value runs to the next `&`; a name that cannot be unescaped is masked too). It also masks the user information of an absolute-form, `//host` or CONNECT authority-form `uri` (`http://[REDACTED]@host/...`), up to the last `@`, so an `@` in the path or query hides the host and path too. Path segments are not masked. In a header value not masked by its name, every `?` starts a query that runs to the next `?`, and its values are masked by key as in `uri` (`Referer: https://app.example/cb?token=[REDACTED]`; each URL of a `Link` list); a value that starts with `/` or contains `://` also has its user information masked (`Origin: https://[REDACTED]@app.example`). Other values (`User-Agent`, `Accept`, `From: ops@example.com`) are kept. This goes by the value's shape, not a list of header names, so custom proxy headers are covered; a non-URL value with `://` or `?` may be over-masked. Otherwise masking is by name: a secret under a non-sensitive name (such as a URL with credentials in `?next=`) is logged as sent. See [Logging](#logging).
- The deprecated `api.LogRequest`, `api.LogResponse` and `api.LogResponseWithDuration` log bodies only when `api.LogRequestBody` / `api.LogResponseBody` are true; they mask JSON body fields and `form` values by the keys in `api.SensitiveLogKeys` alone, cut bodies at `api.MaxLoggedBodyBytes`, and log a request body that is not JSON as `[non-JSON body, N bytes]` unless `api.PrintFullEvent` is true.
- `api.ProcessRequest` never logs bodies or the parsed form. It masks, by its sensitive keys (a name matches when it contains a key, ignoring case; the defaults include `authorization`, `token`, `password`, `key`, `cookie`, `session`, `code`, `signature` and `sig`, plus `api.WithSensitiveLogKeys` on an instance or `api.SensitiveLogKeys` on the default instance): header values and query parameter values in `uri` (names are found by splitting on `&` and `;`, and a masked value runs to the next `&`; a name that cannot be unescaped is masked too). It also masks the user information of an absolute-form, `//host` or CONNECT authority-form `uri` (`http://[REDACTED]@host/...`), up to the last `@`, so an `@` in the path or query hides the host and path too. Path segments are not masked. In a header value not masked by its name, every `?` starts a query that runs to the next `?`, and its values are masked by key as in `uri` (`Referer: https://app.example/cb?token=[REDACTED]`; each URL of a `Link` list); a value that starts with `/` or contains `://` also has its user information masked (`Origin: https://[REDACTED]@app.example`). Other values (`User-Agent`, `Accept`, `From: ops@example.com`) are kept. This goes by the value's shape, not a list of header names, so custom proxy headers are covered; a non-URL value with `://` or `?` may be over-masked. Otherwise masking is by name: a secret under a non-sensitive name (such as a URL with credentials in `?next=`) is logged as sent. Substring matching also masks names that merely contain a key (`country_code`, `design`). See [Logging](#logging).
- The deprecated `api.LogRequest`, `api.LogResponse` and `api.LogResponseWithDuration` log bodies only when `api.LogRequestBody` / `api.LogResponseBody` are true; they mask JSON body fields and `form` values by the keys in `api.SensitiveLogKeys` alone (since v1.1.2 this includes `code`, so the envelope's response `code` and fields such as `error_code` are logged as `[REDACTED]`), cut bodies at `api.MaxLoggedBodyBytes`, and log a request body that is not JSON as `[non-JSON body, N bytes]` unless `api.PrintFullEvent` is true.
- With `stdadapter`, a route variable can contain `/` and `..` (`/files/..%2F..%2Fetc` gives `name` = `../../etc`). Validate it before using it in a file path or another lookup that trusts it.
- `api.ProcessRequest` stores the value of the `X-Forwarded-For` header up to its first `", "` as the `proxiedIPAddress` info value (and, when the request has no device UUID, as the event ID input). Only the first `X-Forwarded-For` header line is read. That value is set by the client: trust it only behind a proxy that overwrites the header, and do not use it on its own for rate limits, allow lists or audit.
- An `api.New` instance bounds request bodies at 10 MiB: its `Handler` wraps the body in `http.MaxBytesReader`, and `api.ProcessRequest`, `api.RequestBody` and `api.UnmarshalBody` answer a larger body with 413 and code `E002` (title `Request content too large`). Set another limit with `api.WithMaxBodyBytes(n)`; `n <= 0` is ignored, so pass `math.MaxInt64` for no practical limit. Nested instance limits combine, so the smallest applies. A custom receiver or a handler that reads `r.Body` itself gets the `*http.MaxBytesError` and decides how to answer. `ExampleWithMaxBodyBytes` is the compiled version.
Expand Down
2 changes: 1 addition & 1 deletion docs/adr/0001-v1-instance-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ sequenceDiagram
- An instance without `WithLogger` writes no logs.
- With the built-in writer, an instance never calls `Fatal` and never exits: a response that cannot be encoded is logged at error level through the instance logger. Since v0.11.0 (#46) the default instance does not call `Fatal` either: it reports the failure through `PrintError` and answers 500. *(Superseded by Amendment 1: the default instance logs it through `slog.Default`, not `PrintError`.)*
- `WithSensitiveLogKeys(keys...)` adds keys to the default list; it cannot remove a default key (security defaults are not weakened).
- As landed in 5.5: `ProcessRequest` writes two info records, `request` (`event_id`, `method`, `uri`, `headers`) and `response` (`event_id`, `method`, `uri`, `status`, `status_text`, `duration_ms`). Errors met while serving a request (a rejected body in `ProcessRequest`/`RequestBody`, a failed `Get*Value*`/`UnmarshalBody` helper) are error records with an `error` attribute; a `Get*Value*` record adds the name as `key` and, for a sensitive name, gives only the kind of error (`invalid syntax`, `value out of range`, `invalid value`), never the value. A recovered panic is a `panic recovered` error record with exactly the panic value and `event_id`, never a dump of the request; a panic with `http.ErrAbortHandler` is a `handler aborted` debug record. *(Amended in v1.1.1: the record carries `panic_type`, and `panic` is the value only for a `runtime.Error`, `[REDACTED]` otherwise, since an opaque value cannot be masked by key; header values are masked after each `?` and, when they carry a URL, in their user information, as `uri` is; a panic with `http.ErrAbortHandler`, plain or wrapped, is re-panicked as the plain `http.ErrAbortHandler` after its `handler aborted` record (a wrapped value would be logged by `net/http` with its text and a stack), writing nothing, so `net/http` aborts the response as it does without `ProcessRequest`; the built-in writer answers a nil formatted response like an encode failure and logs `response cannot be written` with `error` `nil formatted response`.)* An instance's key list is the built-in default list (the shipped value of `SensitiveLogKeys`, which has `cookie` and `session` since v0.11.0) plus its added keys; changes to the `SensitiveLogKeys` variable never reach an instance; added keys are lowercased and blank ones ignored; matching stays v0's case-insensitive substring match. The `uri` is masked with the default instance's helpers from v0.11.0 (#60), so both agree: query values by key with the literal `[REDACTED]` (a masked value runs to the next `&`, a key that cannot be unescaped is masked) and the user information of an absolute-form, `//host` or CONNECT authority-form target up to the last `@`; path segments are not. Bodies and the parsed form are never logged (no option, per section 5; query values still appear in `uri`). The built-in writer of an instance encodes before writing: a body that cannot be encoded is logged as `response cannot be encoded` and answered with 500 and the instance formatter's `Error500()` body (bare 500, with the response headers, if the formatter returns nil or that fails too); `http.ErrBodyNotAllowed` is not logged; the `Content-Type` comes only from the formatted headers (`application/json` when they set none), so one a handler set on `w.Header()` never labels the body; the encode-failure 500 drops the failed response's body headers (`Content-Type`, `Content-Encoding`, `Content-Length`, `Content-Range`, `Content-Disposition`, `Content-Language`, `Content-Location`, `ETag`, `Last-Modified`) and uses the error body's `Content-Type`; since 5.6 the default instance's writer also takes a `Content-Type` set in `Headers`, else `application/json` (section 8). `core.APIResponseWriter.Write` receives no request, so the writer's `response cannot be encoded`/`response cannot be written` records carry no request context or `event_id`. A custom writer, including a type that embeds `ResponseWriter`, is used as given and keeps the default instance's behavior. An instance never reads `Print`, `PrintError`, `Fatal` or `SensitiveLogKeys`, now deprecated with `LogRequest`, `LogResponse` and `LogResponseWithDuration`. *(Superseded by Amendment 1: the default instance writes these same records through `slog.Default`, and a type that embeds `ResponseWriter` logs through `slog.Default` with the registered formatter.)*
- As landed in 5.5: `ProcessRequest` writes two info records, `request` (`event_id`, `method`, `uri`, `headers`) and `response` (`event_id`, `method`, `uri`, `status`, `status_text`, `duration_ms`). Errors met while serving a request (a rejected body in `ProcessRequest`/`RequestBody`, a failed `Get*Value*`/`UnmarshalBody` helper) are error records with an `error` attribute; a `Get*Value*` record adds the name as `key` and, for a sensitive name, gives only the kind of error (`invalid syntax`, `value out of range`, `invalid value`), never the value. A recovered panic is a `panic recovered` error record with exactly the panic value and `event_id`, never a dump of the request; a panic with `http.ErrAbortHandler` is a `handler aborted` debug record. *(Amended in v1.1.1: the record carries `panic_type`, and `panic` is the value only for a `runtime.Error`, `[REDACTED]` otherwise, since an opaque value cannot be masked by key; header values are masked after each `?` and, when they carry a URL, in their user information, as `uri` is; a panic with `http.ErrAbortHandler`, plain or wrapped, is re-panicked as the plain `http.ErrAbortHandler` after its `handler aborted` record (a wrapped value would be logged by `net/http` with its text and a stack), writing nothing, so `net/http` aborts the response as it does without `ProcessRequest`; the built-in writer answers a nil formatted response like an encode failure and logs `response cannot be written` with `error` `nil formatted response`.)* An instance's key list is the built-in default list (the shipped value of `SensitiveLogKeys`, which has `cookie` and `session` since v0.11.0, and `code`, `signature` and `sig` since v1.1.2) plus its added keys; changes to the `SensitiveLogKeys` variable never reach an instance; added keys are lowercased and blank ones ignored; matching stays v0's case-insensitive substring match. The `uri` is masked with the default instance's helpers from v0.11.0 (#60), so both agree: query values by key with the literal `[REDACTED]` (a masked value runs to the next `&`, a key that cannot be unescaped is masked) and the user information of an absolute-form, `//host` or CONNECT authority-form target up to the last `@`; path segments are not. Bodies and the parsed form are never logged (no option, per section 5; query values still appear in `uri`). The built-in writer of an instance encodes before writing: a body that cannot be encoded is logged as `response cannot be encoded` and answered with 500 and the instance formatter's `Error500()` body (bare 500, with the response headers, if the formatter returns nil or that fails too); `http.ErrBodyNotAllowed` is not logged; the `Content-Type` comes only from the formatted headers (`application/json` when they set none), so one a handler set on `w.Header()` never labels the body; the encode-failure 500 drops the failed response's body headers (`Content-Type`, `Content-Encoding`, `Content-Length`, `Content-Range`, `Content-Disposition`, `Content-Language`, `Content-Location`, `ETag`, `Last-Modified`) and uses the error body's `Content-Type`; since 5.6 the default instance's writer also takes a `Content-Type` set in `Headers`, else `application/json` (section 8). `core.APIResponseWriter.Write` receives no request, so the writer's `response cannot be encoded`/`response cannot be written` records carry no request context or `event_id`. A custom writer, including a type that embeds `ResponseWriter`, is used as given and keeps the default instance's behavior. An instance never reads `Print`, `PrintError`, `Fatal` or `SensitiveLogKeys`, now deprecated with `LogRequest`, `LogResponse` and `LogResponseWithDuration`. *(Superseded by Amendment 1: the default instance writes these same records through `slog.Default`, and a type that embeds `ResponseWriter` logs through `slog.Default` with the registered formatter.)*

### 7. Errors: kind-to-status override and error hook (API-01)

Expand Down
Loading
Loading