Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 20 additions & 19 deletions .vedeiignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ aede264bea2e590f2e60467ab2ae74b8 # *************31 - 2x, detect/br/others_test.
cf4011fdac9970fb933927c3c2dc2e79 # *********80 - 1x, detect/br/others_test.go

# --- cnpj ---
# CNPJ test vectors, numeric and alphanumeric. The alphanumeric form takes effect 2026-07-06 and needs a fixture before then, not after.
# CNPJ test vectors, numeric and alphanumeric. The alphanumeric form was first issued on 2026-07-31.
e901dbbb99a1ba293aee61bf533bb180 # **.***.***/****-35 - 7x, detect/br/cnpj_test.go and 5 more
3e210b2f70e511a859c255d442cc346a # **.***.***/****-81 - 12x, detect/br/cnpj_test.go and 6 more
e186e56ca5d2b8fda5aa3be015e79407 # **.***.***/****-91 - 1x, detect/br/cnpj_test.go
Expand Down Expand Up @@ -72,22 +72,23 @@ a83871a7916f6b9c44907082aa289cfc # +***********21 - a second phone in the same
d554cea51a6155fd70c74664b448aff2 # +***********94 - a third
9ee2bb2c7f2a591ab50bb3070faa1a6d # a UUID v4 Pix key, in the overlap test

# --- M5 corpus and test fixtures ---
# The labeled corpus and the mask/AWS tests carry example values on purpose.
# Fingerprinted, not path-ignored, so the principle holds: every value is named,
# and a real secret committed into these files would still be reported.
d0dead6eceff137cafd15c0029bf1214 # AKIA...LE example key in corpus
a8cadd7b8a5caff89d90408d8a6c31bc # AKIA...TQ assembled test key
e58b0b5d6b990a005bb44e5d490a851e # ASIA...TQ assembled test key
2ffebaa6e81961e4cef8beeb61c5f8d1 # generated CNH vector
648782aa9ca9e35c33bddea812bb255f # generated CNH vector
2040ad664a71a504a6ff863b0b78a4f0 # generated PIS vector, space-separated
4d4eb30077a44f6a1e088b2c4cff6f1d # generated PIS vector
62949e15935851bd24d617292e7230f8 # generated titulo vector, space-separated
f60e47e7fe914c3e5d10a9b2a99d8b96 # generated titulo vector
ec2a7642f9821664294f09e54573ca66 # generated CNS vector
83f7388d806b21f4c0c4ea6e6a651937 # Pix email key fixture
# --- test fixtures outside the corpus ---
# Values the mask and hook tests use. The same values also sit in corpus/cases/,
# which the path rule below covers; these lines cover the test files.
d0dead6eceff137cafd15c0029bf1214 # AKIA...LE example key, cmd/vedei-hook/main_test.go
2040ad664a71a504a6ff863b0b78a4f0 # generated PIS vector, space-separated, detect/br/extract_test.go
62949e15935851bd24d617292e7230f8 # generated titulo vector, space-separated, detect/br/extract_test.go
ec2a7642f9821664294f09e54573ca66 # generated CNS vector, detect/br/extract_test.go
3d916d891e1aab79bc68592c2d7cd3a5 # the first alphanumeric CNPJ the Receita issued; an earlier version of this file quoted it in a comment, so git history still holds it

# Nothing is ignored by path, on purpose. Silencing *_test.go wholesale would
# also silence a credential genuinely committed into a test, which is a place
# credentials genuinely end up. Every entry above is one value, named.
# --- the one path rule: the labeled corpus ---
# Everything else in this file is a single value, named, and that is the rule:
# silencing *_test.go wholesale would also silence a credential genuinely
# committed into a test, which is a place credentials genuinely end up.
#
# corpus/cases/ is the one exception. It exists to hold labeled example values,
# and the corpus test fails on any line that is not an "expect" or a "reject",
# so a value there is a labeled case by construction. Naming each one here
# would mean a fingerprint per case — hundreds, at the size M5 aims for — each
# one easy to forget. History and diff scans honour this rule too.
corpus/cases/
49 changes: 29 additions & 20 deletions MILESTONES.md
Original file line number Diff line number Diff line change
Expand Up @@ -424,37 +424,46 @@ pendência.

**Objetivo.** Transformar falso positivo de opinião em métrica, e corrigir um viés medido.

### O problema, com número
### A hipótese original — e o que a medição mostrou

O filtro de token efficiency do betterleaks usa dicionário de 33.775 palavras **em inglês** e limiar fixo de 2,5 caracteres por token. Medido com o tokenizador que ele mesmo embute:
**A hipótese era de precision.** O filtro de token efficiency do betterleaks descarta um candidato como prosa quando ele contém uma palavra de um dicionário de 33.775 palavras **em inglês**; senão compara a razão caracteres/token com 2,5. Medida isolada, essa razão dava `senhapadrao` = 2,20 e `defaultpassword` = 7,50, e daí se concluiu que placeholder em português seria reportado como segredo.

| String | Razão | Resultado |
|---|---:|---|
| `defaultpassword` | 7,50 | descartado, correto |
| `senhapadrao` | **2,20** | **reportado como segredo** |
| `cobrancaboleto` | **2,33** | **reportado como segredo** |
| segredo real | 1,0–1,6 | reportado, correto |
**Medido de ponta a ponta (betterleaks v1.8.1), não é.** Só a regra `generic-api-key` usa esse filtro, e a cadeia completa rejeita placeholders em prosa nos três idiomas igualmente — 0 de 30 em inglês, português e espanhol. A conclusão vinha de olhar a razão fora da cadeia de filtros.

Em inglês a separação entre prosa e segredo é confortável. Em português quase some. Ninguém mediu isso.
**O viés real era de recall, e era maior.** As regras reconhecem nomes de chave em inglês. Com o mesmo segredo real sob nomes equivalentes:

### O que implementar
| idioma da chave | segredos reais achados |
|---|---:|
| inglês | 100% |
| espanhol | 57% |
| português | **29%** |

`senha`, `segredo` e `senha_banco` nunca eram reconhecidos. As chaves em português que funcionavam (`chave_api`, `token_acesso`) funcionavam só por conter `api` e `token`.

### O que foi implementado

1. **`corpus/`** com formato rotulado (`expect` / `reject`), gate estrito e métrica **por tipo** — ✅
2. **Passada de localização** em `engine/secrets/localize.go`: nomes de chave em português e espanhol que recebem atribuição são traduzidos (`senha`→`password`, `segredo`→`secret`, `chave`→`key`…) e o texto é escaneado de novo pelas **mesmas regras**. Valores nunca são reescritos, então o segredo é localizado no texto original. Os filtros do upstream — que também têm inglês embutido — se aplicam inteiros, o que dá paridade em vez de uma regra paralela com precision própria — ✅
3. **Reordenação head-final**: português e espanhol nomeiam `senha_banco`; inglês nomeia `db_password`, e a regra do upstream exige fronteira de palavra logo depois de `password`, então `password_db` falha até em inglês. A tradução move `password` para o fim — ✅

Resultado, mesmos valores sob chaves equivalentes: recall **100%** em português e espanhol (inglês 88%, por causa do `password_db` que o próprio upstream perde). Falsos positivos com placeholders são iguais por chave nos três idiomas.

**Não implementado, e por quê:** o dicionário pt-BR para o filtro de token efficiency (itens 1–2 do plano original). A medição mostrou que ele não corrigiria nada.

### Achado lateral — fora do escopo do M5

1. Dicionário pt-BR embutido, mesmo mecanismo `go:embed` + gzip
2. `--locale pt-BR`, com detecção automática do idioma predominante do repositório
3. **`corpus/`** — strings reais de código em pt, en, es, com rótulo esperado
4. `make fp-report` — imprime precisão e recall do conjunto; **falha o CI se regredir**
5. Publicar a medição como issue no betterleaks
A `generic-password` do upstream reporta placeholders instrucionais com confiança baixa **em qualquer idioma**: `"your-password-here"`, `"replace_me"`, `"changeme"` e seus equivalentes `"sua-senha-aqui"`, `"troque_isto"` — 5 de 10 em cada idioma. Referências (`${VAR}`, `os.getenv`, `<marcador>`) são suprimidas corretamente nos três. É fraqueza de precision do upstream, neutra de idioma; candidata a issue no betterleaks ou a um filtro do vedei num marco próprio.

### Como testar

O próprio `make fp-report` é o teste. Roda no CI, com limiar mínimo travado.
`make corpus` (também roda em `go test ./...`, portanto no CI). A tabela por tipo aparece no log.

### Critério de saída

- [ ] `corpus/` com ≥ 500 casos rotulados, em 3 idiomas
- [ ] `make fp-report` no CI, falhando em regressão
- [ ] Número de precisão publicado no README
- [ ] Issue aberta no betterleaks com a medição
- [ ] `corpus/` com ≥ 500 casos rotulados, em 3 idiomas — **119 hoje, nos 3 idiomas**
- [x] Corpus no CI, falhando em regressão — `corpus_test.go` roda dentro de `go test ./...`
- [x] Número publicado no README — recall por idioma e precision/recall por tipo
- [ ] Issue aberta no betterleaks com a medição (recall por idioma + `password_db`)

**Risco:** nenhum técnico. É curadoria — chato, e é por isso que ninguém fez.

Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Wire the hook into your agent: [`hooks/`](hooks/). Two binaries, because the spl

## Silencing what is deliberate

`.vedeiignore` takes a finding fingerprint, derived from the type and the normalized value and **never from the location** — so an entry survives the file being moved or renamed. This repository's own file is [`.vedeiignore`](.vedeiignore): 71 findings, 21 fingerprints, each with the reason written next to it. Nothing is silenced by path, on purpose. Silencing `*_test.go` wholesale would also silence a credential genuinely committed into a test, which is a place credentials genuinely end up.
`.vedeiignore` takes a finding fingerprint, derived from the type and the normalized value and **never from the location** — so an entry survives the file being moved or renamed. This repository's own file is [`.vedeiignore`](.vedeiignore): one fingerprint per deliberate test value, each with the reason written next to it. Paths are almost never silenced: ignoring `*_test.go` wholesale would also silence a credential genuinely committed into a test, which is a place credentials genuinely end up. The single exception is `corpus/cases/`, the labeled corpus — every line there is a labeled example by construction, and a fingerprint per case would not scale.

## What it detects

Expand All @@ -60,7 +60,7 @@ Wire the hook into your agent: [`hooks/`](hooks/). Two binaries, because the spl
| **CPF** | ✅ | Mod 11, two check digits; repeated-sequence rejection |
| **CIN** | ✅ | Uses the CPF as the national number — same validation |
| **CNPJ** numeric | ✅ | Mod 11, weights `5,4,3,2,9,8,7,6,5,4,3,2` |
| **CNPJ** alphanumeric | ✅ | Mod 11 with `ASCII − 48` for letters — **in force since 2026-07-06** |
| **CNPJ** alphanumeric | ✅ | Mod 11 with `ASCII − 48` for letters — **first issued on 2026-07-31** ([Receita Federal](https://www.gov.br/receitafederal/pt-br/assuntos/noticias/2026/julho/receita-federal-gera-o-primeiro-cnpj-em-formato-alfanumerico)) |
| **CNH** | ✅ | Mod 11, 11-digit variant |
| **PIS / NIS / NIT** | ✅ | Mod 11, weights `3,2,9,8,7,6,5,4,3,2` |
| **Título de eleitor** | ✅ | Two check digits, mod 11, embedded state code |
Expand Down Expand Up @@ -88,7 +88,7 @@ ISPB registry from [`guibranco/BancosBrasileiros`](https://github.com/guibranco/

## Why it exists

Secret scanning is solved and crowded — betterleaks ships 463 rules, kingfisher 1,051. Rebuilding that corpus loses. So vedei **imports betterleaks** (MIT) and spends everything on what nobody built: **Brazilian sensitive data**. No maintained open-source tool detects CPF, CNPJ, CNH, Pix keys, card PANs and E2EIDs. Not underserved — empty.
Secret scanning is solved and crowded — betterleaks ships 463 rules, kingfisher 1,051. Rebuilding that corpus loses. So vedei **imports betterleaks** (MIT) and spends its effort where global tools are weakest: **Brazilian sensitive data**, validated by check digit rather than matched by shape — CPF, CNPJ including the alphanumeric format, CNH, Pix keys and E2EIDs, card PANs with Brazilian brands. And it enforces at the boundary where data leaves: the agent's context, the CI run, the log line. One Go binary, no runtime, no network.

The second reason is architectural. betterleaks validates a secret by asking the provider over HTTP, which needs a rate limiter, a timeout budget, and care about where those requests go. Brazilian personal data doesn't work that way: a CPF is a mod-11 check digit, a card is Luhn, a Pix key is its type. **All offline.** No network, no rate limits, no SSRF surface. Simpler and safer, not just different.

Expand Down Expand Up @@ -142,7 +142,7 @@ Full records in [`docs/adr/`](docs/adr/).

Features, implementation steps, test strategy and exit criteria per stage: [`MILESTONES.md`](MILESTONES.md). Component design and performance budgets: [`ARCHITECTURE.md`](ARCHITECTURE.md).

**On M5.** betterleaks filters candidates with a BPE token-efficiency ratio against a 33,775-word **English** dictionary and a fixed 2.5 threshold. Measured with its own embedded tokenizer: `defaultpassword` scores 7.50 and is correctly discarded; `senhapadrao` scores 2.20 and is reported as a secret. Real secrets sit at 1.0–1.6. In English the gap is comfortable; in Portuguese it nearly vanishes. M5 turns that into a versioned corpus and a CI-enforced metric.
**On M5.** The original hypothesis was a precision bias: betterleaks discards a candidate as prose when it contains a word from a 33,775-word **English** dictionary, so `defaultpassword` is dropped and `senhapadrao` falls through to a ratio threshold. Measured end to end on v1.8.1, the full filter chain rejects prose placeholders equally in English, Portuguese and Spanish. The real bias was in **recall**: the rules key on English names, so a real credential was found 100% of the time under an English key, 57% under a Spanish one and **29% under a Portuguese one** — `senha` and `segredo` were never recognized. vedei now translates key names before a second pass, reaching 100% in all three. The labeled corpus behind these numbers is in [`corpus/`](corpus/): 119 cases, precision and recall 1.000 per type, enforced in CI.

## Project structure

Expand All @@ -163,9 +163,9 @@ corpus/ # labeled precision/recall harness (make corpus)

## The name

**Надзор** (*vedei*) is Russian for *oversight* — the word used for regulatory supervision. It is `dozor` (the watch) with the prefix `nad-` (over, above).
**Vedei** is Portuguese: the first person past tense of *vedar* — to seal, to block, to keep from passing. *Eu vedei*: "I sealed it."

A tool that inspects what passes a boundary and enforces a rule about it is not a guard. It is oversight.
It is what the person running the tool says once it has done its job. The CPF did not reach the model, the key did not reach the log, the card number did not reach the commit. Sealed.

## Contributing · Security · License

Expand Down
Loading
Loading