Feat/m5 corpus

.vedeiignore

@@ -72,6 +72,22 @@ a83871a7916f6b9c44907082aa289cfc  # +***********21 - a second phone in the same
 d554cea51a6155fd70c74664b448aff2  # +***********94 - a third
 9ee2bb2c7f2a591ab50bb3070faa1a6d  # a UUID v4 Pix key, in the overlap test
 
+# --- M5 corpus and test fixtures ---
+# The labeled corpus and the mask/AWS tests carry example values on purpose.
+# Fingerprinted, not path-ignored, so the principle holds: every value is named,
+# and a real secret committed into these files would still be reported.
+d0dead6eceff137cafd15c0029bf1214  # AKIA...LE example key in corpus
+a8cadd7b8a5caff89d90408d8a6c31bc  # AKIA...TQ assembled test key
+e58b0b5d6b990a005bb44e5d490a851e  # ASIA...TQ assembled test key
+2ffebaa6e81961e4cef8beeb61c5f8d1  # generated CNH vector
+648782aa9ca9e35c33bddea812bb255f  # generated CNH vector
+2040ad664a71a504a6ff863b0b78a4f0  # generated PIS vector, space-separated
+4d4eb30077a44f6a1e088b2c4cff6f1d  # generated PIS vector
+62949e15935851bd24d617292e7230f8  # generated titulo vector, space-separated
+f60e47e7fe914c3e5d10a9b2a99d8b96  # generated titulo vector
+ec2a7642f9821664294f09e54573ca66  # generated CNS vector
+83f7388d806b21f4c0c4ea6e6a651937  # Pix email key fixture
+
 # Nothing is ignored by path, on purpose. Silencing *_test.go wholesale would
 # also silence a credential genuinely committed into a test, which is a place
 # credentials genuinely end up. Every entry above is one value, named.

ARCHITECTURE.md

@@ -122,7 +122,7 @@ const (
      │              (ex.: node_modules, testdata, binario)
      │
  [2] extract        regex por detector, tolerante a mascara
-     │              123.456.789-09 / 12345678909 / 123.456.78909
+     │              123.456.789-09 / 12345678909 / 123 456 789 09
      │
  [3] validate       digito verificador | Luhn | ISPB | estrutura
      │              -> invalido morre aqui, nao vira achado
@@ -137,7 +137,7 @@ const (
 
 Etapa 3 é o que separa vedei de um scanner de regex. Um `grep` de CPF acha `000.000.000-00`; vedei não, porque o dígito não fecha.
 
-**Divergência conhecida na etapa 2.** "Tolerante a máscara" não é uniforme entre detectores. CNS, título e PAN aceitam espaço como separador; CPF, CNPJ e PIS aceitam só `.`, `-` e `/`. Então `529 982 247 25` **não** é detectado como CPF hoje. Isso foi descoberto escrevendo um teste que assumia o contrário, e é uma decisão de calibração, não um esquecimento: aceitar espaço no CPF casa com qualquer coluna de tabela de três dígitos, e medir esse custo é justamente o que o corpus do M5 existe para fazer. Até lá o comportamento está documentado aqui em vez de prometido no código.
+**Separadores de máscara, padronizados (M5).** Todo detector aceita o mesmo conjunto entre grupos de dígitos: a pontuação própria do documento, um espaço, um tab, ou nada — **nunca** uma quebra de linha. Antes disso havia duas divergências: CNS e título usavam `[\s.]`, e `\s` casa `\n`, então casavam um valor partido em duas linhas (o mesmo bug que o padrão de cartão já evitava); e CPF, CNPJ e PIS recusavam espaço, então `529 982 247 25` não era detectado. O medo de aceitar espaço era casar coluna de tabela de três dígitos — medido contra o corpus, o dígito verificador mantém essas colunas quietas, então a precision ficou em 1.000. A medição está em `corpus/cases/cpf.txt`.
 
 ---
 

Makefile

@@ -6,7 +6,7 @@ HOOKBIN := vedei-hook
 VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
 LDFLAGS := -s -w -X main.version=$(VERSION)
 
-.PHONY: build test lint bench cover clean install-tools all fuzz security vuln sast latency
+.PHONY: build test lint bench cover clean install-tools all fuzz security vuln sast latency corpus
 
 all: lint test build
 
@@ -50,6 +50,12 @@ FUZZ_TRANSCRIPT := FuzzScrubRecord_PreservesMeaning FuzzScrubRecord_NumbersSurvi
                    FuzzScrubRecord_RedactionKeepsItValid FuzzReadableText
 FUZZ_SCAN := FuzzParseIgnore FuzzIgnoreStack FuzzIsBinary
 
+# The labeled precision/recall corpus. Separate from the unit tests because it
+# is the number that answers "is this getting noisier", and worth running on its
+# own when calibrating.
+corpus:
+	go test -v -run TestCorpus ./corpus/
+
 fuzz:
 	@for t in $(FUZZ_DETECT); do \
 		echo "== $$t"; go test -run '^$$' -fuzz "^$$t$$" -fuzztime=$(FUZZTIME) ./detect/br/ || exit 1; \

README.md

@@ -133,7 +133,7 @@ Full records in [`docs/adr/`](docs/adr/).
 | M2 | Secret engine via betterleaks | ✅ |
 | M3 | **MVP — AI agent surface** | ✅ |
 | M4 | CLI, CI/CD, SARIF, transcript scrub | ✅ |
-| M5 | Multilingual false-positive corpus + pt-BR calibration | ⬜ |
+| M5 | Multilingual false-positive corpus + pt-BR calibration | 🚧 |
 | M6 | Runtime SDK | ⬜ |
 | M7 | AI layer | ⬜ |
 | M8 | Observability collector | ⬜ |
@@ -149,16 +149,16 @@ Features, implementation steps, test strategy and exit criteria per stage: [`MIL
 ```
 detect/br/        # the core: pure validators, zero dependencies
 detect/engine.go  # the Engine interface — insulation from betterleaks' v2 API
-engine/secrets/   # betterleaks wrapped behind that interface
+detect/aws/       # AWS access key ids — the one secret rule vedei owns
+engine/secrets/   # betterleaks wrapped behind that interface, plus detect/aws
 redact/           # format-preserving redaction
-report/           # JSON, JSONL, SARIF
-scan/             # directory walker, gitignore matcher, git history
 report/           # table, JSON, JSONL, SARIF
+scan/             # directory walker, gitignore matcher, git history
 stream/           # stdin -> stdout, the MVP path
 daemon/           # Unix socket server and client — the 8.4 ms path
 transcript/       # agent session log reader
 hooks/            # agent hooks, git hooks, launchd and systemd units
-corpus/           # multilingual false-positive corpus
+corpus/           # labeled precision/recall harness (make corpus)
 ```
 
 ## The name

README.pt-BR.md

@@ -132,7 +132,7 @@ Registros completos em [`docs/adr/`](docs/adr/).
 | M2 | Motor de segredos via betterleaks | ✅ |
 | M3 | **MVP — superfície de IA** | ✅ |
 | M4 | CLI, CI/CD, SARIF, scrub de transcript | ✅ |
-| M5 | Corpus multilíngue + calibração pt-BR | ⬜ |
+| M5 | Corpus multilíngue + calibração pt-BR | 🚧 |
 | M6 | SDK de runtime | ⬜ |
 | M7 | Camada de IA | ⬜ |
 | M8 | Coletor de observabilidade | ⬜ |
@@ -157,7 +157,7 @@ stream/           # stdin -> stdout, o caminho do MVP
 daemon/           # servidor e cliente do socket Unix — o caminho de 8,4 ms
 transcript/       # leitor de log de sessao de agente
 hooks/            # hooks de agente, hooks de git, units de launchd e systemd
-corpus/           # corpus multilingue de falso positivo
+corpus/           # harness rotulado de precision/recall (make corpus)
 ```
 
 ## O nome

SECURITY.md

@@ -71,11 +71,15 @@ Run the same checks locally with `make security` and `make fuzz`.
 check digits, never identity, and never will (ADR-002). A generated CPF in a
 test fixture is indistinguishable from a real one by arithmetic alone.
 
-**AWS access key IDs are not detected.** Neither the canonical example nor a
-random one: the upstream corpus has no rule for `AKIA…` alone, which is a
-defensible position — an access key ID is an identifier, and the secret access
-key is the credential. For keeping data out of an agent's context it is still a
-gap, and a real one. Stripe, GitHub PAT and Slack tokens are detected. M5.
+**AWS access key IDs are detected; the secret access key is not.** vedei owns one
+rule the upstream corpus lacks: an access key id (`AKIA…`, `ASIA…`) is matched
+structurally — a credential-bearing prefix and a 16-character base32 body. It is
+reported High confidence and Unknown validity, because the shape proves it is
+well-formed, not that it still works. The 40-character *secret* access key has no
+structure to key on and is left to context-based rules, which is a known gap. The
+account-id checksum an access key id also carries is not yet verified; doing so
+would raise confidence and surface the account. Stripe, GitHub PAT and Slack
+tokens are detected by the upstream corpus.
 
 **The daemon is a long-lived process holding a Unix socket.** If you run
 `vedei daemon`, note what it is and is not:

corpus/README.md

@@ -0,0 +1,48 @@
+# Corpus
+
+Labeled examples that measure what vedei catches and what it cries wolf over.
+
+A scanner's worth is not that it compiles. It is precision and recall on real
+input, and for Brazilian data the hard one is precision: the upstream secret
+rules score candidates against an English dictionary, so a Portuguese word like
+`senhapadrao` reads as random and gets reported. That cost is invisible until
+you measure it against labeled data. This directory is that data.
+
+## Format
+
+One directive per line in `cases/*.txt`:
+
+```
+expect cpf  529.982.247-25     # must be found, as a cpf
+reject      529.982.247-00     # must not be reported — wrong check digit
+```
+
+- **`expect <type> <value>`** — a true positive. The engine must report a
+  finding of `<type>` over `<value>`. A miss is a recall failure: a leak walked
+  past.
+- **`reject <value>  # reason`** — a false positive. The engine must report
+  nothing. A hit is a precision failure: on the agent hook, redacting something
+  the model needed. The reason is required — a reject nobody can explain is a
+  reject nobody can safely remove.
+
+Everything after `#` is a comment.
+
+## Running
+
+```bash
+make corpus          # or: go test ./corpus/
+```
+
+The gate is strict: every `expect` must be found and every `reject` must stay
+quiet. That is right for hand-labeled data, where each line is a deliberate
+claim — a threshold is for scraped corpora where some error is expected. The
+test still prints precision and recall, because the numbers are the point and a
+reviewer should see them move.
+
+## Adding a case
+
+When a scan produces a false positive, add the value here as a `reject` with the
+reason, then fix the detector until this test passes again. When a detector
+gains a document type, add `expect` cases for it. The file
+`cases/false-positives.txt` is the running record of what scanning the real
+world actually produced — every entry in it was once a wrong finding.

corpus/cases/cnpj.txt

@@ -0,0 +1,9 @@
+# CNPJ — numeric and the alphanumeric format in force from 2026-07-06.
+
+expect cnpj  11.222.333/0001-81
+expect cnpj  11222333000181
+
+expect cnpj  00.000.000/0001-91   # the Uniao's CNPJ: body all zeros, but check digits close
+
+reject  11.222.333/0001-82   # check digit wrong
+reject  12.345.678/0001-00   # check digits do not close

corpus/cases/cpf.txt

@@ -0,0 +1,23 @@
+# CPF — true positives and the near-misses that must stay quiet.
+# All valid values have correct check digits by construction; a fixture with a
+# wrong one would only prove the generator agrees with the validator.
+
+expect cpf  529.982.247-25
+expect cpf  52998224725
+expect cpf  111.444.777-35
+
+# The check digit is the whole point: these are one digit off and must not be
+# reported, or vedei is just a regex and claims otherwise.
+reject  529.982.247-00     # last digit wrong, fails mod 11
+reject  111.444.777-30     # check digit does not close
+reject  000.000.000-00     # repeated sequence, invalid by rule even though mod 11 passes
+
+# Shapes that look CPF-ish but are not.
+reject  1234567890         # ten digits, too short
+reject  app version 1.2.3  # dotted, but not eleven digits
+
+# Space as a separator, standardized across every document type. The feared
+# cost is a three-digit table column; the check digit is what keeps it quiet.
+expect cpf  529 982 247 25
+reject  linha 100 200 300 40     # three-digit columns, check digit does not close
+reject  valores 123 456 789 10   # same shape, fails mod 11

corpus/cases/false-positives.txt

@@ -0,0 +1,22 @@
+# The precision corpus proper: things that pass a checksum or match a shape by
+# accident. Every one of these was a real false positive, or the exact class of
+# one, that an earlier version reported. This file is the record of what running
+# vedei against the world actually produced.
+
+# A Go module pseudo-version. The timestamp YYYYMMDDHHMMSS passes Luhn about one
+# time in ten; this one did, and was reported as a credit card.
+reject  v0.0.0-20240915155400-7ee5256398cf   # timestamp read as card-pan
+reject  migration_20240915155400_add.sql     # same timestamp, in a filename
+
+# A UUID. A v4 is a valid Pix EVP key by shape and essentially never one in
+# fact; the sixteen digits straddling a hyphen also passed Luhn once.
+reject  session 11111111-2222-3333-4444-555555555555   # UUID read as card-pan
+
+# A Brazilian phone number in E.164 once read as a card-pan: it begins with 55,
+# inside the Mastercard issuer range, and its digits passed Luhn. That reading is
+# gone. It is still a Pix key by type, which is correct and lives in pix.txt — the
+# bare number here must simply never be a card.
+reject  boleto 4532 1111 1111 111    # 15 digits, grouped as a card but fails Luhn
+
+# A base64 module hash in go.sum: entropy by design, not a secret.
+reject  h1:o1uxfymjZ7jZ4MsgCErcwWGtVKSiNAXtS59Lhs6uI/g=   # go.sum hash

corpus/cases/other-documents.txt

@@ -0,0 +1,16 @@
+# CNH, PIS, título and CNS. Vectors generated by each published algorithm and
+# confirmed against the live extractor, never copied from a third party.
+
+expect cnh             73022314994
+expect cnh             20909550300
+expect pis             43912547832
+expect pis             22322316163
+expect titulo-eleitor  306636791040
+expect titulo-eleitor  862441572097
+expect cns             100000000000007
+
+reject  11111111111    # a repeated digit is not a valid document of any kind
+
+# Space-separated forms, now caught uniformly.
+expect pis             439 12547 83 2
+expect titulo-eleitor  3066 3679 1040

corpus/cases/pix.txt

@@ -0,0 +1,9 @@
+# Pix keys. A phone or email used as a Pix key is sensitive data, not noise, so
+# these are true positives. The phone is the one that used to be read as a card.
+
+expect pix-key  recebedor +5511980198775
+expect pix-key  chave pix fulano@exemplo.com.br
+
+# A random UUID with no Pix context is not a key: a v4 is a valid EVP key by
+# shape and almost never one in fact.
+reject  trace 11111111-2222-3333-4444-555555555555   # UUID, no Pix context

corpus/cases/secrets.txt

@@ -0,0 +1,12 @@
+# Credentials. The AWS access-key cases are the gap this milestone closes: the
+# upstream corpus reports none of them, measured, not even with an assignment
+# context. Values are assembled so the file carries no live-shaped secret.
+
+expect aws-access-key-id  AKIA4KJ7WQZR2LMNXBTQ
+expect aws-access-key-id  ASIA4KJ7WQZR2LMNXBTQ
+expect aws-access-key-id  export AWS_ACCESS_KEY_ID=AKIA4KJ7WQZR2LMNXBTQ
+
+# base32 has no 0, 1, 8 or 9, and the prefix must be a credential-bearing one.
+reject  ABCD4KJ7WQZR2LMNXBTQ   # 20 chars, base32 body, but not an AWS prefix
+reject  AKIA1KJ7WQZR2LMNXBTQ   # AKIA prefix but a 1 is not base32
+reject  the quick brown fox jumped over the lazy dog   # ordinary prose