Skip to content

fix: report accurate UTF-8 hook line numbers - #1252

Open
sylvesterkaczmarek wants to merge 1 commit into
gchq:mainfrom
sylvesterkaczmarek:bugfix/utf8-line-numbers
Open

sylvesterkaczmarek wants to merge 1 commit into
gchq:mainfrom
sylvesterkaczmarek:bugfix/utf8-line-numbers

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown
Contributor

Fixes #1251.

Read files as binary lines and decode each line independently so invalid UTF-8 is reported against the correct physical line. The hook now also continues after a decode failure and reports multiple invalid lines in one run.

Adds regression coverage for invalid bytes on multiple lines and for valid UTF-8 input, and records the fix in the changelog.

Validation:

  • pytest tests/unit/test_require_utf8.py -q: 2 passed
  • pre-commit checks on all changed files: passed
  • git diff --check: passed

Read files as binary lines before decoding so invalid UTF-8 is reported against the physical source line and multiple invalid lines are surfaced in one run.

Refs: gchq#1251
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

coreax-metadata UTF-8 hook reports the wrong line number

1 participant