Skip to content

Redact API keys and tokens from morpc.req logs and errors - #207

Merged
jinskeep-morpc merged 1 commit into
mainfrom
fix/redact-api-keys
Sep 24, 2026
Merged

jinskeep-morpc merged 1 commit into
mainfrom
fix/redact-api-keys

Conversation

@jinskeep-morpc

Copy link
Copy Markdown
Collaborator

Fixes #206.

Change

  • New morpc.req.redact(value): replaces the values of the key, token, and api_key query parameters (case-insensitive) with REDACTED in a URL string or a parameter dict. It's public so other packages (e.g. morpc-census) can reuse it.
  • Every log statement and exception message in morpc/req.py that includes url, r.url, or params now goes through redact(). That covers get_text_safely, get_json_safely, get_file_safely, post_safely, and delete_safely.
  • get_file_safely catches raise_for_status()'s HTTPError and re-raises it with a redacted message (from None, so the original message doesn't appear in the traceback), because requests puts the full URL in that message.
  • get_text_safely now raises HTTPError with a message (redacted URL) and response, instead of a bare HTTPError.
  • rest_api._max_record_count redacts the URL it logs.

Requests themselves are unchanged; only what gets logged or raised is redacted.

Testing

  • 8 new tests in tests/test_req.py. With a secret key/token in the URL or params, each checks that the secret is absent from captured logs at DEBUG level and from the exception message. Paths covered: get_json_safely (204, 400, 404, success, JSON decode error), get_text_safely, and get_file_safely. There's also a unit test for redact(). All 8 failed before the change.
  • test_req (11), test_frictionless, test_gpkg, test_morpc, test_release, and test_geocode all pass.
  • test_utils: the same 4 datetime tests fail on main.

Not covered here

🤖 Generated with Claude Code

Request URLs and parameter dicts were logged and put in exception messages
as-is, so the Census API key (key=) ended up in notebook outputs. Add
redact(), which replaces key, token, and api_key values with REDACTED,
and use it wherever req.py logs or raises with a URL or parameters, plus
the one rest_api log of a returned URL. get_file_safely re-raises
requests' HTTPError with a redacted message, since requests includes the
full URL.

Fixes #206.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Census API key leaks into logs and exception messages from morpc.req

1 participant