Skip to content

Structured logging and audit logs - #106

Merged
rlahfa-dinum merged 13 commits into
mainfrom
logs
Jun 18, 2026
Merged

rlahfa-dinum merged 13 commits into
mainfrom
logs

Conversation

@rlahfa-dinum

@rlahfa-dinum rlahfa-dinum commented Jun 10, 2026 •

Copy link
Copy Markdown
Collaborator

This bring a comprehensive facility to trace:

  • access logs
  • error logs
  • RPC audit logs

on a client or a server.

Trace IDs are added to allow correlation to occur once we present rejection pages and so on.

The output facilities are thought for:

  • development (stdout/stderr) with text
  • systemd environments (journald/stderr) with structured (JSON) or text for stderr.
  • container environments (stdout/stderr) with structured (JSON).

TODO:

  • don't use target for the log routing
  • log RPC stuff
  • add log-preset CLI param
  • update nixos module
  • ensure the policy for check ACL logging and the RPC command is clear

Fixes #65.

Signed-off-by: Ryan Lahfa ryan.lahfa.ext@numerique.gouv.fr

@rlahfa-dinum
rlahfa-dinum force-pushed the logs branch 2 times, most recently from 197c32b to 2e91720 Compare June 15, 2026 15:36
@rlahfa-dinum
rlahfa-dinum force-pushed the logs branch 2 times, most recently from d690479 to 9bd9358 Compare June 15, 2026 16:43
@rlahfa-dinum
rlahfa-dinum marked this pull request as ready for review June 15, 2026 16:43
@rlahfa-dinum rlahfa-dinum changed the title WIP: structured logging and audit logs Structured logging and audit logs Jun 15, 2026
Comment thread src/proxy/http_connect.rs
Comment thread src/main.rs
Comment thread src/logging/mod.rs
Comment thread src/logging/mod.rs Outdated
@RealHinome

Copy link
Copy Markdown

These changes are really great. It's an excellent idea. My reviews are just comments, not formal reviews.

@agrimault-dinum agrimault-dinum left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The subsystem routing is a great idea.

I assumed you used duration_ms for network call and duration_us otherwise where is makes sense

Comment thread src/proxy/socks5.rs Outdated
Comment thread src/proxy/socks5.rs Outdated
Comment thread src/proxy/socks5.rs Outdated
Comment thread src/main.rs
Comment thread src/logging/mod.rs
Comment thread src/proxy/socks5.rs Outdated
Comment thread src/proxy/http_connect.rs Outdated
Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
This way, it is possible to print it to users and be able to reconcile a
very specific request.

Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
As it can be used in final contexts as well after an async move, making
it non-initial.

Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
With timings

Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
Known to be broken: spans for connection-passing function are dropped. I
assume this is a macro problem with zlink.

Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
This adds a bunch of structures to configure log subsystems according to
targets and filter them adequately.

The logging mechanism supports routing to multiple outputs and different
formats such as JSON, compact, pretty or the normal one (called full).

Instead of letting the user configure everything, we offer 3 presets:

- development — everything on stdout & stderr in the pretty format
- systemd — some files from LogsDirectory=portail and stderr for system
  & errors, ideally, via journald later on, with json in some places
- container — stdout & stderr in the json format.

Let's not offer any customization for now as this complicates (for no
good reason) the user's job to configure the system.

Log shippers loves structured JSON, this is our default format. Our
traces are updated to be routed properly and enriched with more spans
now.

Logging has been made non-blocking using tracing_appender which uses a
thread pool to dispatch logs to a queue and let the worker depop them,
this should ensure that logging does not add any meaningful overhead
during proxying. This needs to be benchmarked and measured wrt to
latency (with runtime deactivation, compile-time deactivation, etc.).

Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
This makes full use of the logging work.

Signed-off-by: Ryan Lahfa <ryan.lahfa.ext@numerique.gouv.fr>
@rlahfa-dinum
rlahfa-dinum merged commit 9aa666b into main Jun 18, 2026
14 checks passed
@rlahfa-dinum
rlahfa-dinum deleted the logs branch June 18, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Audit logs

3 participants