Skip to content

Output files get ordinary modes; CSV escapes line-feed cells; canary fails on a block - #11

Merged
jehrr merged 1 commit into
mainfrom
fix/audit-2026-10-14
Oct 15, 2026
Merged

jehrr merged 1 commit into
mainfrom
fix/audit-2026-10-14

Conversation

@jehrr

@jehrr jehrr commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator

Problems (audit of 2026-10-14, at 3fc2346 = current main)

  1. Output modes. _atomic_write used mkstemp (0600) then os.replace, so every JSON/CSV/sidecar was 0600 under umask 022, 002 and 077, and an existing 0664 file was narrowed to 0600.
  2. CSV guard. A cell starting with LF was not escaped, and the sidecar had no csv_cells_escaped.
  3. Credential scan. In a tree with no .git (e.g. git archive extracted), git ls-files failed silently and the scan printed ok 0 files scanned, rc 0, with a planted 32-hex key present.
  4. Weekly pip-audit red since 2026-10-05. urllib3 1.26.20 in the pyppeteer lock: PYSEC-2026-4175 / -4177.
  5. Private citations. 8 tracked files (incl. the released 0.2.1 CHANGELOG section) cited private maintainer notes by section number.
  6. Canary. Exit 3 from a runner with no proxy secret was a green ::notice::.

Resulting behaviour

  • New output files get 0o666 & ~umask (umask read from /proc/self/status, falling back to a locked os.umask round-trip); an existing file keeps its mode. The atomic write itself is unchanged.
  • \n added to the formula-lead set. write_csv returns the escaped count; finish_run records csv_cells_escaped in the sidecar (null when no CSV is written). JSON is unchanged, numbers are untouched.
  • ci_checks.py: when git cannot list the tree, it walks the disk (paths judged relative to the repo, caches and virtualenvs skipped) and says so. A scan that read 0 files now fails.
  • audit.yml: both IDs are ignored by ID for the pyppeteer lock only. On osv.dev both are fixed only in urllib3 2.8.0, and pyppeteer 2.0.0 (latest on PyPI) requires urllib3<2, so no compatible pin exists. README now says seven advisories.
  • Citations reworded or removed in code, the workflow, the CHANGELOG 0.2.1 line and the v0.2.1 release notes. Only that one phrase was changed; the rest of the body is byte-identical. A new suite check fails on any reference to private notes or a §N in shipped files. The tokens are assembled from pieces, so the check does not match itself.
  • Canary: exit 3 is now ::error:: + red, with or without SCREENER_PROXY. Exits 4/5/6 were already red, and the parser thresholds (≥20 rows, ranks, INR, market cap) are unchanged. A suite check asserts that no branch of the exit-code step ends green.
  • The proxy_pool.py docstring no longer names another repo as provenance.

Evidence

  • Offline suite, core-only venv (Python 3.13): 605 passed, 0 failed, 10 skipped. The skips are the three engine groups, which have no driver installed. ci_checks.py --all exits 0.
  • Controls (PYTHONDONTWRITEBYTECODE=1, __pycache__ removed per run). Each planted fault turned the suite red on the named check:
    • removing the chmod: write_* creates a file 0o644 under umask 0o22 and the keep-0664 checks;
    • a flat 0o644: … 0o664 under umask 0o2;
    • dropping \n from the lead set: a cell led by a line feed is neutralised;
    • removing the sidecar key: the sidecar records csv_cells_escaped;
    • disabling the disk fallback: outside a git checkout … catches a planted key;
    • removing the zero-scan failure: a scan that read 0 files fails;
    • planting a citation in proxy_pool.py: no shipped file cites private maintainer notes;
    • exit 0 in the canary's exit-3 branch: the canary turns no exit code but 0 into a green run.
  • Manual run in a git archive tree: clean gives 52 files scanned, rc 0; a planted key gives FAILED, rc 1.
  • pip-audit run locally on all five locks with the new ignore list: no known vulnerabilities.
  • Engine files changed in comments only. CI engine-smoke runs each engine's suite.

Not changed

  • --fingerprint / --concurrency warn rather than refuse in the Selenium and pyppeteer engines (documented). --locale / --mode are absent. The 3.13/3.14 matrix is unchanged. All of these are owner decisions.
  • Shared fetch loop refactor, schema/sinks, version bump, tag and release: out of scope.
  • Dependabot Bump actions/upload-artifact from 7.0.1 to 7.0.2 in the actions group across 1 directory #10 is left alone. The red pip-audit check it inherits should go green once this lands.

🤖 Generated with Claude Code

…fails on a block

- output_writer: new files get 0o666 under the umask, existing files keep
  their mode (they were always 0600). Atomic replace unchanged.
- CSV guard escapes a leading LF; the sidecar records csv_cells_escaped.
- ci_checks: a tree git cannot list is scanned from disk, and a scan that
  read 0 files fails instead of reporting ok.
- audit.yml: ignore two urllib3 1.x advisories fixed only in 2.8.0, which
  pyppeteer's urllib3<2 pin rules out; README counts seven.
- canary: exit 3 fails the job with or without the proxy secret.
- Comments and the 0.2.1 changelog line no longer cite private maintainer
  notes; a suite check guards it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jehrr
jehrr marked this pull request as ready for review October 15, 2026 13:03
@jehrr
jehrr merged commit 68f0260 into main Oct 15, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant