Please report security issues privately through GitHub Security Advisories rather than in a public issue.

Include what you did, what you expected, and what happened. If the issue is a detection gap, a short example of text that slips through is the most useful thing you can send. Use invented data. Do not send a real name, a real address or a real key to demonstrate a bug in a privacy tool.

Expect an acknowledgement within a week. mamori is a young project maintained in spare time; there is no paid response commitment and no bug bounty.

mamori reduces the chance that sensitive data reaches an external LLM when somebody pastes real text into a prompt. It is a safety net for a mistake that people make constantly and notice too late.

This section is longer than the previous one on purpose. A privacy tool that is trusted past its actual reach is worse than none at all, because it licenses riskier behaviour than the behaviour it replaced.

The detectors are regular expressions, surname lists, and a pass that propagates a value confirmed once to its other mentions. They are known to miss:

Language-independent:

Japanese (ja):

English (en):

Chinese (zh):

Not covered at all: Korean, and every language with no pack. Universal rules (email, credentials, card numbers, private addresses) still apply to text in those languages; nothing else does.

Two commands answer most of what this document asserts, against your own configuration rather than the defaults described here:

mamori privacy

It reports what is detected, where text goes, what is kept, and -- separately -- what is true however you configure it and what mamori cannot check for you. Settings that widen exposure are warnings with a non-zero exit status.

Those construction claims are backed by tests/test_promises.py, which replaces socket.connect with a function that raises and then runs the whole default path. If a dependency ever starts contacting something, that suite fails before a release does.

A leak rate is a rate against a label set, and this one is ours. Every figure below counts sensitive characters that a detection did not cover, where sensitive means marked in a dataset written by the people who wrote the rules, using the types those rules have. A value nobody here thought to give a type to is not a miss in these numbers. It is not in them at all.

That is not a hedge, it is the unit. 0.00% on ja-core means the rules cover everything mamori's own type system knows how to mark, and says nothing about what an owner of that document would have wanted hidden. The two questions have different answers, and the second one is open.

The type system reaches past personal data on purpose — COMPANY_NAME, EMPLOYEE_ID and PROJECT_NAME are business-confidential rather than personal — so "it is only a PII tool" is not available as an explanation for what it misses there. Where it stops is stated in the gap tables above rather than by category.

Detection is measured against bundled labelled datasets, at two scales. Run mamori eval yourself. Every figure below is what that command prints on the current version, at the default recall-first stance, and tests/test_security_figures.py fails the build when it stops being so — the table was hand-copied until 0.28 and three of its rows had drifted:

MamoriConfig(nlp="gliner"), same corpora, same command, on the same day as the table above. The rules are unchanged; this is what the model adds:

Twelve of twelve at zero leaked characters, bought with between zero and 2.9 further points of over-redaction. Read that as a floor and not as a probability. These corpora were written by mamori, they are small, and a model that scores perfectly on twelve small sets written by the project measuring it has demonstrated that it does not regress, not that it is safe. The unshipped thousand-document corpus is where the bugs come from, and this configuration has not been through it.

Two things this does not do, measured in the same run and stated in gliner.py with the numbers: it is wrong on Japanese sentences taken alone -- wrong spans, not merely low recall -- and it does not find an internal project codename at any threshold. The Japanese rows above improve because the Japanese rules claim those names first and this pass only takes what nothing anchored claimed.

It costs pip install "mamori[gliner]", several hundred megabytes of torch, and roughly 90ms a sentence against 4ms for the whole rule pipeline.

A third corpus of a thousand generated documents across twelve genres is not shipped -- it lives in the development repository, because the bundled sets are the regression floor and that one is for finding bugs. It found five in its first run, and four more in Chinese the run after that. See the 0.14.0 and 0.15.0 entries in CHANGELOG.md.

The -agent rows are payloads rather than prose: tool-call arguments and tool definitions, where the label is a key rather than a word in a sentence. A key is a stronger anchor than any prose, which is worth reading as "the rules read keys now" rather than as a claim about agent payloads being easy -- before 0.18 the same rows would have been most of the way to a total loss. See ADR 0030.

Three of those rows changed in 0.33, and one of them got worse. A wide-tier rule for an identifier written as a prefix, a separator and a number -- E-45033 -- took en-context from 6.31% leaked to 0.00%, zh-docs from 2.37% to 0.00%, and en-docs from 3.50% to 2.65%. It also took a support ticket number out of an agent payload, which is the whole of en-agent's new 2.07% over-redaction. That is the wide tier working as described rather than misfiring: it accepts false positives, the balanced stance does not run it, and a deployment that needs its ticket numbers intact says so with mamori correct or with stance = "balanced".

Two entity-precision figures fell without anything leaking, and the reason is worth stating because the numbers look like a regression. E-45033 is labelled EMPLOYEE_ID and is now found by a rule that reports IDENTIFIER. The characters are covered -- the leak rate is what says so -- and the type is coarser than the label. A value protected under a more general name is protected; a value with the right name and no placeholder is not.

The -context rows are assembled prompts -- what a retrieval layer or an agent framework renders, rather than what a person typed. They leak more than documents do at the balanced stance (46.85% against 20.02% in English) for a reason worth stating: selection strips anchors. A passage chosen out of a note arrives without the salutation, the signature block and the form label that made its values detectable, because the anchor stayed behind in the part that was not selected. See ADR 0029.

Read the document rows first. The -core sets are sentence fragments with a median length of 28 to 44 characters; the -docs sets are business documents at the length people actually send. Documents leak several times more, because a document is full of names with no anchor beside them. Every figure this project published before 0.9.0 came from the fragment sets alone, which described mamori at its easiest.

At --stance balanced, which runs only the anchored rules:

That 20.02% is not a typo. A fifth of the sensitive characters in an English document have nothing anchored near them -- a name in an attendee list, a name under a sign-off, a name after "Reported by:". It is the clearest reason the recall-first stance is the default, and the clearest argument against turning it off without measuring your own text first.

Leak rate is the share of labelled sensitive characters that no detection covered — the part that would have left the machine. Over-redaction is the share of ordinary characters replaced anyway.

Read these as regression floors, not as a claim about your data. The sets are small and synthetic. They were written to cover the cases the rules are meant to handle and the ones they are known to miss, which makes them good at catching a change that breaks something and poor at estimating recall on a corpus nobody has seen. A leak rate near zero on fifty invented sentences says nothing about a real inbox.

The residual leak is one documented gap: a trading name with no legal suffix.

Two things closed the rest, and both cost something. A value confirmed once is now protected everywhere it appears, which moved English from 7.4% to 2.0% at no cost in precision. The recall-first stance then added rules that match on shape alone, which took English to 0.67% and Japanese to zero and roughly doubled over-redaction. Read the two tables together. A tool that redacts everything has a perfect leak rate and destroys every answer.

The over-redaction figures fell in 0.5.0 without any rule changing. Some labelled spans are genuinely ambiguous -- ten bare digits are an order number to the anchored rules and a possible phone number to the wide ones -- and the datasets previously scored the wide reading as a mistake. Those spans are now marked tolerated: neither required nor wrong, excluded from both sides. The earlier numbers charged the recall-first stance for doing exactly its job, and overstated its cost by roughly half.

Added in 0.4.0, measured for the first time in 0.7.0, and re-measured in 0.10.0 after a bug was found in the harness that produced those numbers. Balanced stance, llama3.1:8b running locally, against the fragment sets:

At this size the model raises English recall -- it closes en-006, a name in running prose with nothing to anchor on -- and costs precision in both languages. It does not improve Japanese. At the recall-first default it does not move the leak rate at all, because the wide rules already reach those values, and costs roughly six times the over-redaction.

At 14B the answer is different, and 0.23 is where that was first measurable. qwen2.5:14b-instruct-q4_K_M against the document sets, which are documents rather than fragments. These four rows were taken on a GPU, which matters because the device is not always neutral -- see docs/choosing-a-model.md, where one smaller model redacts 44% more on a CPU:

Over-redaction holds in English and Japanese, which is what did not happen at 8B, and two of the three languages reach zero. It does move in Chinese, by +0.56 points, which this section claimed it did not until the row was re-measured: the earlier figure carried a stale baseline and an unchecked delta. Entity precision falls with it, 0.900 to 0.865. What closes in English is the anchorless name, the largest measured gap in this project since 0.9. The Chinese row read 2.37% -> 2.37% until 0.24, when this library started accepting the type names the model actually uses. The 345 seconds per document this line used to report has been withdrawn: an interrupted Ollama update had left no CUDA library on the machine, so every run was CPU inference with a 16 GB card idle. The accuracy above is unaffected for these numbers, and that is now measured rather than assumed: changing only the device leaves the leak rate and the entity recall byte-identical. It does not leave everything identical — one model redacts 44% more on CPU than on GPU — so "the device cannot change what a model returns" is not a claim this file makes. See docs/choosing-a-model.md. The tier is off by default for a reason the stopwatch was never carrying — it needs a model you have to run, on hardware that decides which one — and "measure it on your own hardware" is still not a formality.

A model can score well on text it has memorised, and here it has not. The bundled corpora were written on 2026-08-29 and 2026-08-30, in a repository whose first commit is 2026-08-29. Three of the models measured above were already on the machine before those files existed -- qwen2.5:14b and llama3.1:8b by twenty-six days, gemma4:12b by nineteen. Weights cannot memorise text that did not exist when they were trained.

The other two were pulled after, and their published weights certainly predate the pull, but this project has not verified that and does not claim it.

This is the kind of true statement that stops being true. The corpora are public now, under a licence that invites copying, so every model trained after this week may have read them. The date is the floor: a figure above is uncontaminated for a model released before 2026-08-29 and says nothing about one released after. Any corpus commissioned to replace these inherits the same clock from the day it is committed.

Why this took sixteen releases. Every attempt since 0.7 reported that the model timed out, and every write-up repeated it. LLMRequest.timeout defaulted to thirty seconds and the provider took the smaller of the request's and the endpoint's, so llm.timeout above thirty did nothing -- three attempts plus backoff is ninety-seven seconds, which looks exactly like hardware that is too slow. The model pass degrades to nothing by design, so the symptom was silence. If you ran a model tier before 0.23 with a timeout above thirty seconds, you were not getting the timeout you configured.

About the correction. The 0.7.0 figures were produced by a harness that rebuilt the detection pipeline by hand and left out the co-occurrence pass, so the model was being scored against a baseline that had a pass it lacked. The conclusions survived re-measurement -- the model is an English-recall tool at 8B and does nothing for Japanese -- but the over-redaction figures moved and the comparison was not sound when it was published. The harness now has one assembly path, and a test pins it.

Two things this measurement corrected in the library itself, worth knowing if you ran an earlier version: the model was asked for character offsets and got 0 of 52 right, so its findings were being discarded almost entirely (0.4.0 to 0.6.0); and OTHER_SENSITIVE, which the default policy blocks, was being proposed for weekdays and error codes. Both are fixed.

Run mamori eval --compare against your own data -- docs/measuring-your-own-data.md says how, and what to be careful about. These are two models on small synthetic sets and they are a floor for judgement, not a substitute. The thing they do establish is that "does a bigger model help" has a different answer at each size, which is an argument for measuring rather than for either of the two conclusions.

Nothing here has been assessed against GDPR, HIPAA, APPI or any other regime. Pseudonymized personal data remains personal data under most of them, and a placeholder that a mapping can reverse is pseudonymization, not anonymization.

Detection, the mapping table and restoration all run locally. Anything that can read your process memory, your files or your keystrokes has already won. The mapping table is the highest-value object in the system: it holds only the values you were trying to protect.

mamori protects the text you pass to it. It cannot intercept a call that does not go through it. The proxy planned for v0.5 narrows this gap; it does not close it.

Once the protected text is sent, it is out of scope. Placeholder structure itself carries information -- that there were three people and two companies in the message, and roughly where. If that fact is sensitive, do not send the message.

The long form, including what is in and out of scope for each threat, is in docs/threat-model.md.

• Mappings live in memory by default and are discarded when the session closes.
• Nothing is written to disk unless you ask for it. --save-mapping writes plaintext and says so on stderr every time.
• The library emits no log records, so there is nothing to configure away.
• No telemetry, no network calls, no model downloads.

The mapping table is the highest-value object here — it is the only thing that holds the values, and it is what makes restoration possible at all. So this section is the whole of its life, and not a summary.

Where it lives. In this process's memory, in a dict, in an InMemoryMappingStore created per session. Nothing else is written unless a caller asks. mamori protect --save-mapping PATH writes JSON in plain text and prints a warning to stderr every time; --encrypt-mapping PATH writes a Fernet envelope whose key comes from an environment variable and never from the file beside it.

How long it lasts, and what ends it:

The last row is the design, not an accident. There is no write-ahead log and no flush-on-exit, because a mapping table that survived a crash would be a mapping table on a disk somebody has to remember to delete. The cost is that a crash costs you the restoration, and a saved mapping file is the opt-in for anybody who would rather pay the other way.

Retention is a rule you can read, not a thread you cannot see. It is enforced on every read and write of the store rather than by a sweeper, so the contents never depend on how long the process has been running. mamori privacy prints the rule in force.

What is not in the table: nothing derived from the model's answer, and nothing about the request. It maps a placeholder to the exact original text and an identity key, in one scope, and holds no timestamps beyond what retention needs.

One sharp edge, measured and stated rather than smoothed over. Placeholder numbering restarts per scope, so two independent sessions both mint <EMAIL_001>. Feeding one session's response to another session's restore therefore returns the other value, with nothing reported — the token is perfectly well known, it just means somebody else. Restoration takes a scope for exactly this reason, and the rule is that the text you restore must come from the session that protected it. There is no fix from inside a scope: a token another scope also minted is indistinguishable.

The latest 0.x only, while the project is pre-1.0. Fixes land on main.