diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 6528fde..e8abca2 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -42,7 +42,7 @@ body: attributes: label: Toril version description: Help → About, or the release you installed. If you built from source, give the commit. - placeholder: "v1.1.1" + placeholder: "v1.1.2" validations: required: true diff --git a/CHANGELOG.md b/CHANGELOG.md index a58575f..bd23ada 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,43 @@ GitHub Release notes plus the commits that shipped in it. ## [Unreleased] +## [v1.1.2] — 2026-11-09 + +Saving a note in an Obsidian vault no longer breaks its links, embeds and callouts, and +notes containing ©, ® or ™ open again. + +### Fixed +- **Saving a note no longer breaks Obsidian links, embeds and callouts.** When you edited a + note in an Obsidian vault, saving it rewrote `[[Pneumothorax]]` as `\[\[Pneumothorax]]`, + `![[scan.png]]` as `!\[\[scan.png]]`, and `> [!tip]` as `> \[!tip]`. Obsidian shows those + as plain text, so every link, embedded image and callout in that note stopped working. + Notes you only opened were never affected; notes you edited were. They now save + exactly as written. + + If a note was saved by an earlier version, its links are still escaped on disk. Removing + the backslashes is safe: open the note in Toril, make any edit, and save. Toril writes + the links back without them. Version history also holds the copy from before Toril + first saved it. + +- **Toril no longer adds backslashes to ordinary text.** `figure [1]`, `~3 cm`, `5 * 3`, + `snake_case` and bare web addresses are left as you typed them. A backslash is now + written only where the text would otherwise change meaning, such as a line that would + turn into a heading or list. Every save checks this by re-reading what it is about to + write and comparing it with your note, so no backslash is removed where it is needed. + +- **Notes containing ©, ® or ™ open again, and those symbols are kept when you save.** + Any note with one of them failed to open with *"RangeError: Invalid array passed to + renderSpec"*. The emoji support treated the three symbols as emoji it could not draw, and + a save would also have deleted them: `Omnipaque® and Visipaque™, © 2026` would have been + written as `Omnipaque and Visipaque, 2026`. They are now ordinary text. The emoji + versions (©️ ®️ ™️, as an emoji keyboard or `:tm:` types them) are unchanged. + +- **A note that fails to open no longer leaves a broken tab behind.** After a failed open, + the new tab stayed selected while the editor still showed the previous note, and + pressing Save could write the wrong text into the file that had failed to open. Toril + now stays on the note you were in and reports the error, including when you open a note + from the sidebar, which previously failed without any message. + ## [v1.1.1] — 2026-10-15 ### Fixed diff --git a/CLAUDE.md b/CLAUDE.md index bd35e1e..5d01912 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -54,13 +54,32 @@ the feature-by-feature record. **Known trade-off:** Toril's canonical form (`src/editor/canonical.ts`) is `-` bullets and `---` thematic breaks, matching Obsidian — so most notes survive open→save untouched. Some constructs still reformat (setext headings, indented code blocks, `~~~` fences, two-space hard breaks, link -reference definitions, `*`/`+`-authored bullets, `1)` ordered markers, over-indented nesting, bare -URLs, and **CRLF line endings → LF**, which rewrites every line of a Windows-authored note); the +reference definitions, `*`/`+`-authored bullets, `1)` ordered markers, over-indented nesting, +`` autolinks → bare URLs, superfluous backslash escapes dropped, and **CRLF line +endings → LF**, which rewrites every line of a Windows-authored note); the `normalized` class in `tests/roundtrip.test.ts` is the authoritative list — each entry pinned to its exact output. The reformatting is idempotent and never drops *rendered* content; the one known exception is that an **unused link reference definition is deleted outright** (see the deferred list above). Relevant to Obsidian-vault diffs (§1). +**Escapes are minimal and proven — `fix/obsidian-syntax-escaping`.** remark-stringify escapes +defensively (every `[`, `*`, `_`, `~`), which was a §3 bug, not a style issue: saving an edited +note rewrote `[[Note]]` as `\[\[Note]]`, `![[x.png]]` as `!\[\[x.png]]` and `> [!tip]` as +`> \[!tip]` — literal text in Obsidian, so every wikilink, image embed and callout in that note +broke. It also rewrote ordinary prose (`figure [1]`, `~3 cm`, `snake_case`). `docToMarkdown` +(`serializer.ts`) now passes the escaped output through `src/editor/escapes.ts`, which removes +each escape **only if re-parsing proves the document unchanged** — never by re-implementing +CommonMark's escaping rules. Per-block bisection (memoized, so between keystrokes only the edited +block re-parses), then one whole-document parse as the proof; if that fails (a block whose meaning +depends on another, e.g. a reference definition), a bounded whole-document search. Cost: roughly +one extra full parse per serialization (~12 ms at 800 words, ~180 ms at 12k words in jsdom), paid +on the debounced change listener. Rules for whoever touches it: the property gate compares against +the **fully escaped** output, not the live doc, because two upstream quirks are lossy or unstable +with or without this module — remark moves a space out of a strikethrough boundary +(`~~a ~~b` shapes), and **a bare URL ending in `\` gains a backslash on every save** (pre-existing on +`main`, not yet fixed). Table column padding is computed before minimization, so a cell holding +`[[x\|y]]` may pad a character or two wide (pinned as `wikilinkAliasInTable`). + **HTML as a first-class editable format — shipped in v0.1.1-alpha.1.** Open/edit/save `.html` WYSIWYG alongside `.md`, motivated by AI assistants emitting rich HTML instead of Markdown. The editor engine is unchanged — HTML is the native shape of a ProseMirror doc, so it is a second canonical @@ -344,6 +363,8 @@ toril/ │ ├── editor/ │ │ ├── milkdown.ts # WYSIWYG setup + plugins (incl. clipboard-image paste, §6) │ │ ├── serializer.ts # the ONE markdown <-> doc converter (§3.2) +│ │ ├── escapes.ts # proof-checked escape minimization, used only by serializer.ts +│ │ ├── emoji.ts # the ONE emoji stack: upstream + ©/®/™ kept as text (§6) │ │ ├── html-serializer.ts # the ONE HTML <-> doc converter (§3.2/§3.3) │ │ └── html-constructs.ts # richer HTML-only schema: callout/details/dl/mark/sub/sup (§6) │ ├── ui/ @@ -615,7 +636,7 @@ frontend never touches the filesystem directly; it asks via `invoke()`. | WYSIWYG core | `@milkdown/core` + `@milkdown/preset-commonmark` | | GFM (tables, task lists, strikethrough) | `@milkdown/preset-gfm` | | Math (KaTeX) | **Deferred** — `@milkdown/plugin-math` is deprecated; omitted until a maintained option exists (§0/§8) | -| Emoji shortcodes | `@milkdown/plugin-emoji` | +| Emoji shortcodes | `@milkdown/plugin-emoji`, assembled **only** in `src/editor/emoji.ts` (app and every gate import `emoji` from there). That module adds one remark step: an emoji node twemoji did not draw as an `` (bare ©, ®, ™ — `emoji-regex` matches them, twemoji leaves them as text) goes back to plain text. Upstream made them nodes that crash the view on load (prosemirror-model ≥ 1.25.5 rejects the Text node `toDOM` returns) **and** serialize to nothing. **Do not upgrade to 7.23.0 thinking it fixes this**: it fixes the crash, not the deletion on save | | Inline / slash shortcuts | `@milkdown/plugin-slash` + keymap config | | Formatting toolbar | `toolbar.ts` → Milkdown commands via `callCommand` (the few command-less items use a plain ProseMirror transaction); **never inserts raw markdown text** (§3.2). Buttons reflect active state via `activeState()`. The pure command layer is exported separately from the DOM so the gate tests it headlessly. *Underline omitted by design* (no markdown form). *Front-matter button deferred* (not lossless yet). | | Front matter | **Not a Milkdown plugin, by design** — `src/editor/frontmatter.ts` splits the block off at the load/serialize boundary so it never enters the ProseMirror doc (§3). Edited in `src/ui/properties.ts`, a collapsible strip above the writing surface: typed rows for a block that provably re-serializes byte-exact, raw text for anything else | @@ -657,15 +678,24 @@ Phases 0–3 are complete and Phase 4 (polish) is in progress; the shipped detai **Gates (all green) — keep them green:** - **Atomic save:** `cargo test -p fsatomic` — interrupting a save leaves the original intact (§3.1). - **Round-trip:** `tests/roundtrip.test.ts` — real Milkdown in jsdom, built through - `canonical.ts` so the gate tests the canon that ships. Three classes: `fixtures` + `canonical.ts` so the gate tests the canon that ships. Classes: `fixtures` (canonical input is stable), `preserved` (human/Obsidian-authored input is **not** - rewritten), `normalized` (what we do rewrite, pinned to exact output), and + rewritten), `obsidianPreserved` (wikilinks, embeds, callouts, `==highlight==`, + `%%comments%%`, block IDs, nested tags, Anki cloze — Obsidian syntax Toril does not + render yet but must never break), `escapesKept` (the escapes that *are* needed — + task markers, heading/list/quote markers, raw HTML — stay), `normalized` (what we + do rewrite, pinned to exact output), and `frontMatter` (**whole-file** trip — split, round-trip the body, rejoin — the class that pins the §3 front-matter fix, including blocks no serializer could reproduce). Add math fixtures if math ever lands (§3.2). Plus `tests/frontmatter.test.ts`, the splitter's own gate: `join(split(x)) === x` over fixtures *and* 2000 randomized documents, detection per format, and the Rust-parity cases mirrored in - `mdhtml`/`mdrtf`. + `mdhtml`/`mdrtf`. Plus `tests/escapes.test.ts` — the minimizer's own gate: the + bisection (joint verification, so `\*a\*` never loses both escapes), the + whole-document fallback, the parse budget, a throwing parser read as *inequality*, + and a 200-case randomized property over the real parser (the saved file loads + exactly as the fully escaped file would, and is stable wherever the old output was). + Verified with teeth: forcing the verifier to accept everything fails 8 tests. - **Front-matter properties:** `tests/frontmatter-values.test.ts` — Obsidian's own writes come back as typed rows (the feature is useless otherwise), and everything we would *reformat* lands in raw mode instead: comments, anchors, flow sequences, @@ -731,6 +761,13 @@ Phases 0–3 are complete and Phase 4 (polish) is in progress; the shipped detai in the **future** fails *open* — a corrected clock must not disable update checks forever with no symptom. What it cannot cover is anything downstream of the network: see §D of `docs/ON-DEVICE-VERIFICATION.md`. +- **Emoji / symbols:** `tests/emoji.test.ts` + `trademarkSymbols`/`symbolsBesideEmoji` in the + round-trip `preserved` class — ©/®/™ parse as text, load into a *mounted* view, and save + byte-exact, while the emoji-presentation forms (©️ ®️ ™️) stay image emoji. Red before the fix. +- **Failed loads:** `tests/tabs.test.ts` ("an activation that throws") — a document the editor + cannot load never becomes the active tab. `onActivate` throws *after* the editor state was + replaced, so `setActive` restores and reloads the previous tab, never flushes the editor into + the failed one, and rethrows; `openPath` then drops the never-activated tab. - Plus `vaultscan`, `imgasset`, `theme`, `statusbar`, `search`, `security`, `tabs` suites. > **The browser harness.** `dev-harness.html` is `app.html` plus a fake Tauri IPC bridge diff --git a/README.md b/README.md index 4aa8624..3eeec94 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ It is built on Tauri 2, TypeScript, and Milkdown. The primary platform is > **Status: 1.0.** The editor and the workflow around it work, and your notes stay > plain files you can walk away with. Installers: -> **[v1.1.1](https://github.com/kovirlabs/toril/releases/latest)**. +> **[v1.1.2](https://github.com/kovirlabs/toril/releases/latest)**. > > Keep backups of anything important, as with any editor. See > [CHANGELOG.md](./CHANGELOG.md) for what each release changed. @@ -56,8 +56,10 @@ distinction matters, so they are listed separately. **Not yet built:** -- Math rendering (KaTeX), PDF export, and in-editor YAML front matter. -- Global search across a vault, wikilinks, and backlinks. +- Math rendering (KaTeX) and PDF export. +- Global search across a vault, and wikilink navigation and backlinks. + Obsidian's `[[links]]`, `![[embeds]]` and `> [!callouts]` are kept intact when + Toril saves a note, but are not yet clickable or rendered. - Signed installers, so Windows SmartScreen and macOS Gatekeeper warn on first run. That is expected for an unsigned build, not a sign of a problem. @@ -70,18 +72,18 @@ Installers are on the | Platform | Download | |---|---| -| **Windows** | [`Toril_1.1.1_x64-setup.exe`](https://github.com/kovirlabs/toril/releases/download/v1.1.1/Toril_1.1.1_x64-setup.exe) | -| **Windows** (MSI, for IT deployment) | [`Toril_1.1.1_x64_en-US.msi`](https://github.com/kovirlabs/toril/releases/download/v1.1.1/Toril_1.1.1_x64_en-US.msi) | -| **macOS** (Apple Silicon) | [`Toril_1.1.1_aarch64.dmg`](https://github.com/kovirlabs/toril/releases/download/v1.1.1/Toril_1.1.1_aarch64.dmg) | -| **macOS** (Intel) | [`Toril_1.1.1_x64.dmg`](https://github.com/kovirlabs/toril/releases/download/v1.1.1/Toril_1.1.1_x64.dmg) | -| **Linux** (AppImage) | [`Toril_1.1.1_amd64.AppImage`](https://github.com/kovirlabs/toril/releases/download/v1.1.1/Toril_1.1.1_amd64.AppImage) | -| **Linux** (Debian/Ubuntu) | [`Toril_1.1.1_amd64.deb`](https://github.com/kovirlabs/toril/releases/download/v1.1.1/Toril_1.1.1_amd64.deb) | -| **Linux** (Fedora/RHEL) | [`Toril-1.1.1-1.x86_64.rpm`](https://github.com/kovirlabs/toril/releases/download/v1.1.1/Toril-1.1.1-1.x86_64.rpm) | +| **Windows** | [`Toril_1.1.2_x64-setup.exe`](https://github.com/kovirlabs/toril/releases/download/v1.1.2/Toril_1.1.2_x64-setup.exe) | +| **Windows** (MSI, for IT deployment) | [`Toril_1.1.2_x64_en-US.msi`](https://github.com/kovirlabs/toril/releases/download/v1.1.2/Toril_1.1.2_x64_en-US.msi) | +| **macOS** (Apple Silicon) | [`Toril_1.1.2_aarch64.dmg`](https://github.com/kovirlabs/toril/releases/download/v1.1.2/Toril_1.1.2_aarch64.dmg) | +| **macOS** (Intel) | [`Toril_1.1.2_x64.dmg`](https://github.com/kovirlabs/toril/releases/download/v1.1.2/Toril_1.1.2_x64.dmg) | +| **Linux** (AppImage) | [`Toril_1.1.2_amd64.AppImage`](https://github.com/kovirlabs/toril/releases/download/v1.1.2/Toril_1.1.2_amd64.AppImage) | +| **Linux** (Debian/Ubuntu) | [`Toril_1.1.2_amd64.deb`](https://github.com/kovirlabs/toril/releases/download/v1.1.2/Toril_1.1.2_amd64.deb) | +| **Linux** (Fedora/RHEL) | [`Toril-1.1.2-1.x86_64.rpm`](https://github.com/kovirlabs/toril/releases/download/v1.1.2/Toril-1.1.2-1.x86_64.rpm) |
Installing on Windows -**1. Download** `Toril_1.1.1_x64-setup.exe` from the table above. +**1. Download** `Toril_1.1.2_x64-setup.exe` from the table above. **2. Unblock it if Windows marked it.** Files downloaded from the internet carry a "mark of the web". If the installer refuses to start, right-click it → @@ -138,9 +140,9 @@ Toril…"*. Open **System Settings → Privacy & Security**, scroll down, and cl > later. Nothing was wrong with the download itself. **Linux:** the AppImage is portable — -`chmod +x Toril_1.1.1_amd64.AppImage` and run it. Or install the `.deb` -(`sudo apt install ./Toril_1.1.1_amd64.deb`) or `.rpm` -(`sudo dnf install ./Toril-1.1.1-1.x86_64.rpm`). +`chmod +x Toril_1.1.2_amd64.AppImage` and run it. Or install the `.deb` +(`sudo apt install ./Toril_1.1.2_amd64.deb`) or `.rpm` +(`sudo dnf install ./Toril-1.1.2-1.x86_64.rpm`).
diff --git a/ROADMAP.md b/ROADMAP.md index 9ef1c6e..1c1d696 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -96,6 +96,39 @@ no AI. That gap is this roadmap. > gospel, and the tag has run ahead of it twice now; §3's trust-before-reach rule still > governs what may be *reached* from the UI, regardless of version. +> **Status update (2026-11-09) — who is actually using it, and a §3 fix it surfaced.** +> Toril is seeing heavy use by **radiology / medical students**. That audience changes +> priorities: their notes are image-heavy, structured around cases, studied rather than +> just written, often live in an Obsidian vault they also read on a phone, and sometimes +> brush against patient-identifying images. Two new tracks below — **Track H +> (hardening)** and **Track M (medical & radiology students)** — and §9 (value Toril +> does not offer today) come out of that review. +> +> Reviewing for that audience found a **§3 round-trip bug**: saving an *edited* note +> backslash-escaped Obsidian syntax — `[[link]]` → `\[\[link]]`, `![[img.png]]` → +> `!\[\[img.png]]`, `> [!tip]` → `> \[!tip]` — which Obsidian renders as literal text, +> so links, embedded images and callouts broke. It also rewrote ordinary prose +> (`figure [1]`, `~3 cm`, `snake_case`). **Fixed on `fix/obsidian-syntax-escaping`**: +> `src/editor/escapes.ts` removes every escape that re-parsing *proves* unnecessary. +> Branch 8 was framed as a feature; its *preservation* half was really a data-safety +> fix, the same way front matter (branch 10) was. Rendering and navigation remain branch 8. +> +> **Recommended order from here** (audience-weighted; supersedes the numbering for +> sequencing, not the numbering itself): +> 1. Ship `fix/obsidian-syntax-escaping` as a patch release — it is a §3 fix. +> 2. Merge **branch 6 (vault search)** — already built on `origin/feat/vault-search`. +> 3. **H1** vault-wide repair of notes damaged by earlier saves, and **H2** save +> compare-and-swap (closes the documented TOCTOU). +> 4. **H3 / H4** images in and out of notes the way Obsidian expects. +> 5. **H5** render what H0 now preserves: clickable wikilinks, image embeds, callouts. +> 6. **M1** templates → **M2** case log. +> 7. **M5** privacy kit, **M6** self-quiz, **M3** image viewing. +> 8. Branches 9 (tags) and 7 (palette); then **M7 / M8** study tools; then AI, local-first. +> +> Deprioritized for this audience: 13 (code highlighting), 16 (focus/typewriter), +> 17 (writing stats), and 28 (own mobile app — see §8: with faithful Obsidian syntax, +> Obsidian mobile on the same synced vault *is* the mobile story). + --- ## 2. How we work (recap — these don't change) @@ -286,6 +319,10 @@ editor something you keep using rather than something you tried. - [ ] **8. `feat/wikilinks-backlinks`** — `[[wikilink]]` with `[[`-autocomplete + a backlinks panel. The main thing that makes a folder of notes navigable, and it is the syntax Obsidian already uses, so vaults stay readable in both. + - *Preservation already done (H0, 2026-11-09):* `[[links]]` and `![[embeds]]` survive a + save byte-for-byte. Before that fix they were escaped on save, which made this a §3 + bug and not only a missing feature. What remains here is rendering, navigation, + autocomplete and backlinks (shared with H5). - *New crate:* `crates/linkgraph` — parse `[[links]]`, resolve to files, maintain a backlink index; updates on watcher events. - *Touches:* a Milkdown `$node`/`$inputRule` for the link + autocomplete (canonical, @@ -491,6 +528,226 @@ that has to land before a `v1.0` means anything. --- +### Track H — Hardening · *make "Obsidian-compatible" and "never loses a note" fully true* + +**Why:** each item below is a place where a claim in the README is weaker than it +reads. None is glamorous; all of them are what a heavy daily user hits first. + +- [x] **H0. `fix/obsidian-syntax-escaping`** — *(2026-11-09)* proof-checked escape + minimization in the canonical serializer, so wikilinks, embeds, callouts and plain + prose survive a save byte-for-byte. See CLAUDE.md §0. + - *Built:* `src/editor/escapes.ts`, wired in `serializer.ts`. + - *Gate:* `tests/escapes.test.ts` (search + 200-case property over the real parser) and + the `obsidianPreserved` / `escapesKept` classes in `roundtrip.test.ts`. + - *Open:* serialization now costs about one extra full parse (~180 ms at 12k words in + jsdom, paid on the debounced change listener). If very long notes feel sluggish, take + the whole-document proof off the dirty check, which only needs to know whether the text + differs from disk, and keep it on the write path. + - *Pre-existing quirks found while gating, not yet fixed:* a bare URL ending in `\` + gains a backslash on every save; remark moves a space out of a strikethrough + boundary. Both reproduce on `main` without H0. + +- [ ] **H0b. Findings from a real vault (2026-11-09).** H0 was checked against a + 179-note personal Obsidian vault: each note was split, round-tripped and rejoined + exactly as `main.ts` does, on a copy. Results: + - Before H0, saving broke links in 23 notes: 352 wikilinks/embeds/callout markers + came back as 296. + - After H0, no note lost a link, no note loads differently, and no note became + unstable. + - The only differences between `main` and H0 across the whole vault are 1,175 removed + backslashes and 81 removed autolink `<…>` pairs. + + The same run surfaced rewrites and a crash that predate H0. Each one is a §3 item, in + order of reach: + - **A note containing `®`, `©` or `™` fails to load** in the jsdom gate + (`RangeError: Invalid array passed to renderSpec`). It comes from `@milkdown/plugin-emoji` + treating those characters as emoji: the same input loads fine without the plugin. + **Confirm in WebView2/WKWebView first.** If it reproduces there, it is the most urgent + item on this list: medical notes are full of `®` drug and contrast-agent names. + - **An empty list item `- ` is saved as `-
`** in 56 of 178 notes. Obsidian users + routinely leave a trailing empty bullet, and Toril writes raw HTML into the file. + - Blank-line normalization, in about 130 notes. + - Trailing whitespace stripped, in 71 notes. This is not in CLAUDE.md's + normalization list. + - Table re-padding, in 31 notes (already pinned as `tablePadding`). + - Tab-indented nested lists rewritten to two spaces. + - Emphasis split around inline code: `_Source: \`path\`_` → `_Source:_ _\`path\`_`. + + Each needs a fixture pinning today's behavior before it is changed, as with every + `normalized` entry. + +- [ ] **H1. `feat/repair-escaped-syntax`** — a vault-wide, **explicit** action that finds + notes damaged by pre-H0 saves (`\[\[`, `!\[\[`, `> \[!`) and repairs them. It shows the + list and per-file diff first, snapshots every file before writing, and writes atomically. + Never automatic (§3: no unrequested rewrites). Opening and saving a note in Toril already + repairs it, which is pinned as `repairsEscapedObsidianSyntax`. This is for the ones + nobody reopens. + - *Touches:* reuse `vaultsearch` for the scan; `snapshots` + `fsatomic` for the write. + - *Gate:* repair is idempotent; a note with no damage is byte-untouched. + +- [ ] **H2. `fix/save-compare-and-swap`** — close the documented TOCTOU (CLAUDE.md §5, + "Known limit of the guarantee"): `save_file` takes the expected base hash and refuses + the write unless the bytes on disk still match, inside the same Rust call. + - *Gate:* `fsatomic`-level test with a write landing between check and rename. + +- [ ] **H3. `feat/image-insertion`** — image input that works the way people try it: + - Drag an image file onto the editor and it is inserted. Today a drop only *opens* + files, and images are not on that allowlist, so they are silently dropped. + - Paste into an **unsaved** note: stage the image, and move it into place on first save. + - Paths with spaces, written as `<…>` or `%20`. Obsidian accepts both; CommonMark rejects + a raw space. + - Large-image guard: warn above a size threshold, and optionally downscale a *copy*. + - *Gate:* `imgasset` tests + a drop-allowlist case in `tests/paths.test.ts`. + +- [ ] **H4. `feat/obsidian-attachment-settings`** — read `.obsidian/app.json` + (`attachmentFolderPath`, `newLinkFormat`, `useMarkdownLinks`), so a pasted image lands + where Obsidian would put it and is written as `![[…]]` or `![](…)` to match the vault. + Read-only on `.obsidian/`; never write it. With no `.obsidian/`, keep `./assets/`. + +- [ ] **H5. `feat/obsidian-rendering`** — the rendering half of branch 8 plus its siblings: + - Clickable `[[wikilinks]]` (navigation only, resolved by `linkgraph`). + - Rendered `![[image.png]]` and `![[image.png|400]]`. + - Callouts `> [!type] Title`, including foldable `+`/`-`. + - The `==highlight==` mark. + + Each must stay a canonical node whose markdown form is *exactly* the Obsidian spelling, + so the H0 fixtures double as this branch's round-trip gate. + +- [ ] **H6. `feat/trash-and-purge`** — a trash browser (`list_trash` already has a command + and an `ipc.ts` wrapper) plus **permanent delete**. Permanent delete removes the + `.trash/` entry, the note's `history/` directory, and any `recovery.json` entry, after an + explicit confirm. Today nothing Toril deletes is ever really gone, and §M5 explains why + that matters. + +- [ ] **H7. Small items, each a few lines but each a stated promise:** + - `spellcheck="true"` on the editable, verified in WebView2. Also check whether the DOM + context menu suppresses the native spelling suggestions. + - Decide `app.security.csp` (CLAUDE.md §0, still open #3). + - Show `.html`/`.htm` in the sidebar, since HTML is advertised as first-class. + - Pull **branch 24 (per-tab undo)** forward. Losing undo on every tab switch is a daily + irritation, not a Movement V concern. + +- [ ] **H8. Reconsider PDF export (§7).** Students print and share study sheets, and + HTML → browser → "Save as PDF" is friction they notice. Re-weigh the per-platform + print APIs against the demand that now exists. Alternatively, a one-click "Print" + that opens the exported HTML in the default browser's print dialog would reduce the + friction without new unsafe FFI. + +--- + +### Track M — Medical & radiology students · *the audience that showed up* + +**Why:** a radiology note is a **case**: images first, a fixed structure (history → +findings → impression → differential → pearls), studied repeatedly, and sometimes +derived from material containing patient identifiers. Obsidian can approximate all of +this with five community plugins; Toril can do it out of the box, and the +local-first, no-account stance is a real fit for sensitive material. Keep every item +plain-markdown and Obsidian-readable: these users also open the vault in Obsidian, +often on a phone. + +- [ ] **M1. `feat/daily-notes-templates`** (= **branch 18**, pulled forward) with + **starter templates** shipped as ordinary `.md` files the user can edit: + - *Case note:* front matter for `modality`, `region`, `diagnosis`, `source`, + `rotation`, `date` and `tags`, then History / Technique / Findings / Impression / + Differential / Teaching points. + - *Structured report:* a Technique / Comparison / Findings / Impression skeleton in + the familiar structured-reporting shape. + - *Topic note:* Definition / Imaging features by modality / Mimics / Pearls / Pitfalls. + - *Differential table:* a GFM table template. + - *Gate:* `tests/templates.test.ts`. Every shipped template is also a round-trip + fixture, the same discipline as the welcome note. + +- [ ] **M2. `feat/case-log`** — a table view of front-matter properties across the vault, + sortable and filterable (*every CT chest case on my IR rotation*), with **CSV + export** for logbooks and applications. The properties strip already parses all three + formats. + - *Shape:* Rust returns raw front-matter blocks and the TS side parses them with the + *existing* `frontmatter-values.ts`. Do **not** add a Rust YAML crate: `serde_yaml` is + archived, so it fails §2. + - *Touches:* a `vault_properties` command (sibling of `vaultscan`) and + `src/ui/caselog.ts` in the rail or a tab. + - *Gate:* the splitter parity cases (TS ⇄ Rust) + a pure table-model test. + +- [ ] **M3. `feat/image-viewing`** — radiology is read on black: + - Click an image to open a lightbox with zoom and pan on a black background. + - Consecutive images in one paragraph lay out **side by side** (prior vs current, + T1/T2/FLAIR). This needs no new syntax, so it stays Obsidian-portable. + - Alt text shown as a caption. + - A low-luminance "reading room" theme. + - **Do not build a DICOM viewer**: screenshots are the medium, and DICOM parsing pulls in + a large, risky surface. + +- [ ] **M4. `feat/image-annotation`** — arrows, circles and short labels drawn on a + **copy** (`scan.annotated.png`); the original is never modified, and the note links + the copy. Canvas in the frontend, written through `imgasset`. Larger than it looks; + scope to three tools and undo. + +- [ ] **M5. `feat/privacy-kit`** — for material that may carry patient identifiers: + - **Strip image metadata** on insert (JPEG APPn/EXIF, PNG text chunks), done losslessly + in `imgasset` by dropping segments rather than re-encoding. + - **Crop on insert**, to cut burned-in names or record numbers off a screenshot before + it is ever written. + - **Permanent delete** (H6). Today a deleted note survives in `.trash/`, which sits + *inside* the synced vault, and in version history. + - A short `docs/` page on using Toril with clinical images: de-identify first, follow + your institution's policy, and think twice before a consumer cloud sync. **Never + claim HIPAA or GDPR compliance**: Toril is a local editor, and compliance is about the + whole workflow. + - *Gate:* `imgasset` tests that stripped output decodes identically and carries no + metadata segments. + +- [ ] **M6. `feat/self-quiz`** — unknown-case practice: + - Callouts folded with `-` (`> [!question]- What is the sign?`) render **collapsed** + and open on click. + - A per-note "hide answers" toggle collapses every foldable callout and `
`. + - Optionally hide sections by heading (Impression, Diagnosis) until revealed. + + It is purely view state, and the file never changes. Depends on H5's callout rendering. + +- [ ] **M7. `feat/flashcard-export`** — export `{{c1::cloze}}` and `Q::A` lines (the + Obsidian_to_Anki syntax, both already pinned as preserved) to an Anki-importable + **CSV**. An `.apkg` file needs SQLite, which means C (`rusqlite` bundles it) and fails + §2's C-free rule, so do not build it. Media cannot ride a CSV, so say so in the dialog. + Building spaced repetition *into* Toril is out of scope: students already use Anki, and + the job is to feed it. + +- [ ] **M8. `feat/review-queue`** — lightweight review without Anki: `review_next:` / + `last_reviewed:` front-matter dates, a "due today" list in the rail, and a "Reviewed" + action that updates the property. Uses typed rows, so it is only offered when the + block re-serializes byte-exact. + +- [ ] **M9. `feat/text-snippets`** — user-defined expansions (`;nlcxr` → a normal chest + radiograph paragraph), stored as a plain `.md` file in the vault so they sync and + stay editable. Insertion is a ProseMirror transaction, never raw text (§3.2). + +- [ ] **M10. Medical spelling** — depends on H7's spellcheck. A user dictionary that + persists, and optionally a bundled medical word list, **after a licence check** (several + common medical dictionaries are GPL or unclear; §2 applies to data too). + +- [ ] **M11. Pull forward branch 15 (Mermaid)** — decision trees and imaging algorithms + are a staple of these notes, far more than code highlighting or math. + +- [ ] **M12. AI, local-first for clinical content** — when branch 20 lands, default to + **Ollama**, and require an explicit per-vault opt-in, with a plain warning about patient + data, before any note goes to a cloud provider. Useful and low-risk: generate flashcards + or quiz questions from a note, summarize, outline. **Do not** build "suggest a diagnosis" + or differential features: the accuracy and liability profile is wrong for an editor. The + MCP bridge (branch 23) needs the same opt-in. + +- [ ] **M13. `feat/web-clip`** — paste from a reference site keeps structure, downloads + remote images into the attachment folder (offline study, no link rot), and records + `source:` in front matter for attribution. Sanitize as always (§3.3). + +- [ ] **M14. `feat/case-presentation`** — present a note as slides (split on `---` or H2), + image-first on black, arrow keys to advance: case conferences and teaching rounds. A + view, not a format, so there is nothing new to round-trip. + +- [ ] **M15. Portable build** — a zip that runs without installing, for locked-down + hospital and library PCs (WebView2 is already present on Windows 11). Settings and + history stay beside the executable when a marker file says so. + +--- + ### Parallel track — Toril-TUI follow-ups · *parked 2026-07-08* Not part of the desktop Movement ladder. **Toril-TUI** is a second, webview-free @@ -566,4 +823,41 @@ floor supports. Everything else is execution. - **Beta graduation bar:** which exact branches must be green to drop `-alpha`? (Proposed: all of Movements I–II.) - **Mobile scope for v1.0:** full edit parity, or read + light-edit first? (Lean - light-edit first; the sandbox makes full parity expensive.) + light-edit first; the sandbox makes full parity expensive.) **Re-opened 2026-11-09:** + with Obsidian syntax now preserved (H0), Obsidian mobile reading the same synced + vault may simply *be* the mobile answer — a quarter of work (branch 28) against a + free, mature client. Decide after H5 makes the desktop side render what mobile + writes. +- **How far to specialize for medical users?** Track M is mostly general-purpose + features with medical *defaults* (templates, case log, quiz, privacy). Keep it that + way: a templates system with a radiology starter pack, not a radiology app. Revisit + if the audience broadens or narrows. +- **Learn the audience without telemetry.** Toril does not phone home, and that stays. + Use an opt-in feedback channel (a GitHub Discussions category, or a short survey + linked from Help) and talk to a handful of the students directly before committing + to the larger M items (M2, M4). + +--- + +## 9. Value Toril does not offer today (unplaced — pull from here) + +Ideas that came out of the 2026-11-09 review, beyond Tracks H and M. Unscheduled and +in scope, held to the same rules: plain files, one serializer per format, all disk +I/O in Rust, healthy dependencies only. + +| Idea | Why it matters | Notes / constraints | +|---|---|---| +| **Split view** — two notes side by side | Comparing cases, writing from a source note | Today one shared editor swaps content (branch 24 is the prerequisite) | +| **PDF reader pane** with page-linked notes | Lecture slides and papers are where study notes come from | `pdf.js` (Mozilla, healthy); a link format like `[[deck.pdf#page=12]]` matches Obsidian | +| **Study-packet export** — many notes → one HTML (later PDF) | Printable revision packs, sharing with a study group | Reuses `mdhtml` + the sanitize chokepoint; ordering by folder or a list note | +| **Vault health check** | Broken `[[links]]`, orphaned attachments, oversized images, H1 damage | Read-only report; any fix is a separate, explicit, snapshotted action | +| **Citations** — Pandoc `[@key]` + a Zotero (Better BibTeX) JSON file | Research-track students and residents | `[@key]` already survives a save (H0); render a reference list on export | +| **Searchable image text (OCR)** | Find a case by text inside a screenshot | Use the OS engines (Windows.Media.Ocr, macOS Vision) through Rust bindings; Tesseract is C and fails §2. Store results outside the vault, like history | +| **Scheduled backup** — vault + history to a second folder or zip | Version history lives in the app config dir; a disk loss takes both | Atomic, additive, opt-in; never into the vault itself | +| **Import cleanup** — Notion, OneNote, Evernote exports | Students arrive from OneNote and Notion | Notion's escape-heavy markdown is now normalized by H0 on first edit; an explicit batch import is the remaining piece | +| **Link previews** for reference URLs | Context on hover | Any network fetch is a privacy decision: opt-in, no telemetry, nothing sent but the URL the user wrote | +| **Shared study vault guidance** | Groups already share a synced folder | The conflict machinery (branch 4) is built for this; it needs documenting more than code | + +**Considered and deliberately left out:** a graph view (rarely used after the first +week; backlinks carry the value), a plugin system (README: "no plans"), and an in-app +spaced-repetition engine (feed Anki instead — M7). diff --git a/SECURITY.md b/SECURITY.md index af14efa..6fbb420 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -6,7 +6,7 @@ notes are plain files on your own disk. That shapes the threat model below. ## Supported Versions -Toril is at **`v1.1.1`**. Security fixes land on the latest release and `main` +Toril is at **`v1.1.2`**. Security fixes land on the latest release and `main` only; there are no maintained back-release branches. | Version | Supported | diff --git a/docs/index.html b/docs/index.html index 3f6a196..7757fd2 100644 --- a/docs/index.html +++ b/docs/index.html @@ -47,7 +47,7 @@ "downloadUrl": "https://github.com/kovirlabs/toril/releases/latest", "installUrl": "https://github.com/kovirlabs/toril/releases/latest", "releaseNotes": "https://github.com/kovirlabs/toril/blob/main/CHANGELOG.md", - "softwareVersion": "1.1.1", + "softwareVersion": "1.1.2", "datePublished": "2026-10-15", "isAccessibleForFree": true, "license": "https://github.com/kovirlabs/toril/blob/main/LICENSE", @@ -345,7 +345,7 @@

Toril

Download View on GitHub -
Free & open source · Apache-2.0 · v1.1.1 · Windows, macOS, Linux
+
Free & open source · Apache-2.0 · v1.1.2 · Windows, macOS, Linux
@@ -497,7 +497,7 @@

Data safety first, then features.

Download Toril

Installers for Windows (.exe, .msi), macOS (Apple Silicon and Intel) and Linux (AppImage, .deb, .rpm). Free, Apache-2.0, and nothing to sign up for.

Windows is the primary platform · Installed copies update themselves from v1.1.0 on · Changelog

diff --git a/package.json b/package.json index 72a2d1e..6cb3a54 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "toril", "private": true, - "version": "1.1.1", + "version": "1.1.2", "type": "module", "scripts": { "dev": "vite", diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock index 515136d..7ac6bae 100644 --- a/src-tauri/Cargo.lock +++ b/src-tauri/Cargo.lock @@ -4841,7 +4841,7 @@ checksum = "756daf9b1013ebe47a8776667b466417e2d4c5679d441c26230efd9ef78692db" [[package]] name = "toril-app" -version = "1.1.1" +version = "1.1.2" dependencies = [ "fileops", "fsatomic", diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index f221e5b..1af8a8e 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -20,7 +20,7 @@ exclude = ["third-party/glib"] [package] name = "toril-app" -version = "1.1.1" +version = "1.1.2" description = "Toril — a WYSIWYG markdown editor" authors = ["Toril contributors"] edition = "2024" diff --git a/src-tauri/tauri.conf.json b/src-tauri/tauri.conf.json index 214c6cf..6df9668 100644 --- a/src-tauri/tauri.conf.json +++ b/src-tauri/tauri.conf.json @@ -1,7 +1,7 @@ { "$schema": "https://schema.tauri.app/config/2", "productName": "Toril", - "version": "1.1.1", + "version": "1.1.2", "identifier": "com.toril.app", "build": { "beforeDevCommand": "pnpm dev", diff --git a/src/editor/emoji.ts b/src/editor/emoji.ts new file mode 100644 index 0000000..59aff03 --- /dev/null +++ b/src/editor/emoji.ts @@ -0,0 +1,68 @@ +// The ONE emoji stack (CLAUDE.md §6): upstream @milkdown/plugin-emoji, plus a +// correction that keeps text twemoji does not draw as an image out of the +// `emoji` node. The app (milkdown.ts) and every gate import `emoji` from here, +// so the stack under test is the stack that ships — like canonical.ts. +// +// The defect, in @milkdown/plugin-emoji 7.21.1 and still half-present in 7.23.0: +// its remark step splits text on `emoji-regex`, which matches ©, ® and ™ (they +// are emoji with a *text* default presentation), and wraps each match in an +// `emoji` node holding `twemoji.parse(match)`. twemoji only draws the +// emoji-presentation forms (©️ with U+FE0F), so for a bare © the node holds the +// plain character. Two things then go wrong with that node: +// +// 1. It cannot be drawn. Its `toDOM` hands ProseMirror a Text node, and +// prosemirror-model ≥ 1.25.5 accepts only Element nodes in a DOM spec, so +// the load throws "Invalid array passed to renderSpec" — any note with a ® +// in it fails to open. (7.23.0 fixes this half.) +// 2. It cannot be saved. Its `toMarkdown` writes the ``'s title, and +// there is no ``, so the character is DELETED: `A © B` saves as +// `A B` (§3.2). (7.23.0 does not fix this half — upgrading alone would +// turn a visible open failure into silent loss on save.) +// +// The fix is at the source: after twemoji runs, any `emoji` node whose html is +// not an image goes back to being ordinary text. A ™ in prose is prose — it +// takes the same search, word-count, escaping and serialization path as the +// letters around it, and no custom schema is involved (§11). +import type { MilkdownPlugin } from "@milkdown/kit/ctx"; +import type { Node as MdNode } from "@milkdown/kit/transformer"; +import { $remark } from "@milkdown/kit/utils"; +import { emoji as upstreamEmoji } from "@milkdown/plugin-emoji"; + +type Literal = MdNode & { value: string }; +type Parent = MdNode & { children: MdNode[] }; + +const isParent = (node: MdNode): node is Parent => + Array.isArray((node as Partial).children); + +/** True for an `emoji` mdast node that twemoji left as plain text (no image). */ +function isTextOnlyEmoji(node: MdNode): node is Literal { + if (node.type !== "emoji") return false; + const value = (node as Partial).value; + return typeof value === "string" && !value.includes(" () => demoteTextOnlyEmoji); + +export const emoji: MilkdownPlugin[] = [...upstreamEmoji, ...remarkSymbolsAsText].flat(); diff --git a/src/editor/escapes.ts b/src/editor/escapes.ts new file mode 100644 index 0000000..4c2afe5 --- /dev/null +++ b/src/editor/escapes.ts @@ -0,0 +1,235 @@ +// Verified escape minimization for the canonical serializer (CLAUDE.md §3.2). +// +// remark-stringify escapes defensively: every `[`, `*`, `_`, `~` in prose gets a +// backslash whether or not it could ever be read as syntax. That is safe but it +// rewrites the user's file — and for Obsidian vaults it is *destructive*: +// `[[Note]]` becomes `\[\[Note]]`, which Obsidian renders as literal text, so +// every wikilink, image embed (`![[x.png]]`) and callout (`> [!tip]`) in an +// edited note silently breaks. +// +// This module removes the escapes that are not needed, and it decides "not +// needed" by **proof, not by rule**: a removal is kept only if the candidate +// markdown parses to exactly the same document as the fully escaped output. The +// escaping rules of CommonMark + GFM (flanking delimiter runs, reference labels, +// task-list markers, list bullets at line start, autolink-literal boundaries…) +// are never re-implemented here, so they cannot be re-implemented wrongly. The +// worst a bug in candidate extraction can do is cost a parse. +// +// Pure — no DOM, no Milkdown — so the search is gated headlessly with a fake +// `sameDoc` (tests/escapes.test.ts); `serializer.ts` supplies the real parser. + +/** One removable piece of the serializer's output: the indices to delete. */ +interface Candidate { + readonly drop: readonly number[]; +} + +/** + * Upper bound on whole-document parses in the fallback search. It only bounds + * pathological input — anything not yet verified when it is hit keeps its + * escape, which is always safe. + */ +const MAX_PARSES = 64; + +const ASCII_PUNCT = /[!-/:-@[-`{-~]/; + +// ``, ``, `` and `` — the forms remark emits +// for a link whose text equals its URL. Removing both brackets yields the bare +// form Obsidian writes; GFM's autolink-literal parse decides if that holds. +const AUTOLINK = /<((?:https?:\/\/|mailto:|www\.)[^\s<>]+|[^\s@<>]+@[^\s@<>]+\.[^\s<>]+)>/g; + +// A fence opener/closer, allowing for blockquote and list-item prefixes. +const FENCE = /^[ \t>]*(?:(?:[-+*]|\d{1,9}[.)])[ \t]+)?(`{3,}|~{3,})/; + +/** + * Collect candidate removals outside code. Code is skipped only to save parses: + * a backslash inside a code span or fence is content, so removing it would be + * rejected by verification anyway. + */ +export function findCandidates(md: string): Candidate[] { + const out: Candidate[] = []; + let fence: string | null = null; + let lineStart = 0; + + for (const line of md.split("\n")) { + const m = FENCE.exec(line); + if (fence !== null) { + if (m && m[1][0] === fence[0] && m[1].length >= fence.length) fence = null; + } else if (m) { + fence = m[1]; + } else { + scanLine(line, lineStart, out); + } + lineStart += line.length + 1; + } + return out; +} + +function scanLine(line: string, offset: number, out: Candidate[]): void { + const code = codeSpans(line); + const inCode = (i: number): boolean => code.some(([s, e]) => i >= s && i < e); + + for (let i = 0; i < line.length; i++) { + if (line[i] !== "\\") continue; + const next = line[i + 1]; + if (next === undefined || !ASCII_PUNCT.test(next)) continue; + if (!inCode(i) && next !== "\\") out.push({ drop: [offset + i] }); + i++; // the escaped character is consumed either way (`\\` pairs too) + } + + AUTOLINK.lastIndex = 0; + for (let m = AUTOLINK.exec(line); m !== null; m = AUTOLINK.exec(line)) { + const start = m.index; + const end = start + m[0].length - 1; + if (inCode(start) || line[start - 1] === "\\") continue; + out.push({ drop: [offset + start, offset + end] }); + } +} + +/** [start, end) of each inline code span on a line (backtick runs of equal length). */ +function codeSpans(line: string): Array<[number, number]> { + const spans: Array<[number, number]> = []; + let i = 0; + while (i < line.length) { + if (line[i] !== "`" || (i > 0 && line[i - 1] === "\\")) { + i++; + continue; + } + let n = 0; + while (line[i + n] === "`") n++; + const run = "`".repeat(n); + let j = line.indexOf(run, i + n); + while (j !== -1 && (line[j + n] === "`" || line[j - 1] === "`")) { + j = line.indexOf(run, j + 1); + } + if (j === -1) { + i += n; + continue; + } + spans.push([i, j + n]); + i = j + n; + } + return spans; +} + +function apply(md: string, candidates: readonly Candidate[], chosen: Iterable): string { + const drop = new Set(); + for (const c of chosen) for (const i of candidates[c].drop) drop.add(i); + if (drop.size === 0) return md; + let s = ""; + for (let i = 0; i < md.length; i++) if (!drop.has(i)) s += md[i]; + return s; +} + +/** + * Remove every candidate whose removal `same` accepts, verifying each accepted + * group *jointly* with all previously accepted ones — so interacting removals + * (`\*a\*`: either star alone is literal, both together are emphasis) can never + * both slip through. Groups are bisected on failure; once `budget` parses are + * spent, everything still unverified keeps its escape, which is always safe. + */ +function search(md: string, same: (candidate: string) => boolean, budget: number): string { + const candidates = findCandidates(md); + if (candidates.length === 0) return md; + + const accepted: number[] = []; + let parses = 0; + const test = (group: readonly number[]): boolean => { + if (parses >= budget) return false; + parses++; + try { + return same(apply(md, candidates, [...accepted, ...group])); + } catch { + return false; // a parser failure is never evidence of equality + } + }; + const bisect = (group: readonly number[]): void => { + if (group.length === 0) return; + if (test(group)) { + accepted.push(...group); + return; + } + if (group.length === 1) return; + const mid = group.length >> 1; + bisect(group.slice(0, mid)); + bisect(group.slice(mid)); + }; + bisect(candidates.map((_, i) => i)); + return apply(md, candidates, accepted); +} + +/** + * Split markdown into top-level blocks at blank lines outside fences, keeping + * every separator so `join("")` reproduces the input exactly. A block is a + * *unit of cheap verification*, not a claim that blocks parse independently — + * they do not always (loose lists, reference definitions), which is why the + * whole document is verified again afterwards. + */ +export function splitBlocks(md: string): string[] { + const parts: string[] = []; + let fence: string | null = null; + let current = ""; + let blank = false; + for (const line of md.split(/(?<=\n)/)) { + const isBlank = fence === null && /^[ \t]*\n?$/.test(line); + if (isBlank !== blank && current !== "") { + parts.push(current); + current = ""; + } + blank = isBlank; + current += line; + const m = FENCE.exec(line); + if (fence !== null) { + if (m && m[1][0] === fence[0] && m[1].length >= fence.length) fence = null; + } else if (m) { + fence = m[1]; + } + } + if (current !== "") parts.push(current); + return parts; +} + +/** Budgets: per block (small parses), and for the whole-document fallback. */ +const BLOCK_PARSES = 24; + +// Block → minimized block. Minimizing a block depends only on its own text (it +// is verified standalone, against itself), so between keystrokes every block but +// the one being edited is a cache hit and costs no parse at all. +const blockCache = new Map(); +const BLOCK_CACHE_MAX = 2000; + +/** + * Return `md` with every escape (and autolink bracket pair) removed that can be + * removed without changing the parsed document. + * + * `same(candidate, reference)` must answer whether `candidate` parses to the + * same document as `reference`. Two passes: + * + * 1. **Per block**, each block verified standalone against its own escaped text + * — small parses, memoized. + * 2. **Whole document**, one parse: the joined result against `md`. This is the + * proof that matters; pass 1 only proposes. If it fails (a block whose + * meaning depends on another — a `[label]` that a definition elsewhere turns + * into a link, a loose list split across blocks), fall back to searching the + * whole document directly, within `MAX_PARSES`. + */ +export function minimizeEscapes(md: string, same: (candidate: string, reference: string) => boolean): string { + if (findCandidates(md).length === 0) return md; + + const blocks = splitBlocks(md).map((block) => { + const hit = blockCache.get(block); + if (hit !== undefined) return hit; + const minimized = search(block, (c) => same(c, block), BLOCK_PARSES); + if (blockCache.size >= BLOCK_CACHE_MAX) blockCache.clear(); + blockCache.set(block, minimized); + return minimized; + }); + const joined = blocks.join(""); + if (joined === md) return md; + + try { + if (same(joined, md)) return joined; + } catch { + // fall through to the whole-document search + } + return search(md, (c) => same(c, md), MAX_PARSES); +} diff --git a/src/editor/milkdown.ts b/src/editor/milkdown.ts index fa59e70..9d41c4b 100644 --- a/src/editor/milkdown.ts +++ b/src/editor/milkdown.ts @@ -12,7 +12,7 @@ import { listener, listenerCtx } from "@milkdown/kit/plugin/listener"; import { $prose } from "@milkdown/kit/utils"; import { EditorState, Plugin, PluginKey, TextSelection, type Selection } from "@milkdown/kit/prose/state"; import type { EditorView } from "@milkdown/kit/prose/view"; -import { emoji } from "@milkdown/plugin-emoji"; +import { emoji } from "./emoji"; import { nord } from "@milkdown/theme-nord"; import { searchPlugin } from "../ui/search"; import { useCanonical } from "./canonical"; diff --git a/src/editor/serializer.ts b/src/editor/serializer.ts index da9eb6a..ef3d01b 100644 --- a/src/editor/serializer.ts +++ b/src/editor/serializer.ts @@ -7,11 +7,49 @@ // ProseMirror), so we inherit its tested round-trip behavior rather than // hand-rolling a schema (§11). import type { Editor } from "@milkdown/kit/core"; -import { getMarkdown, replaceAll } from "@milkdown/kit/utils"; +import { editorViewCtx, parserCtx, serializerCtx } from "@milkdown/kit/core"; +import type { Node } from "@milkdown/kit/prose/model"; +import { replaceAll } from "@milkdown/kit/utils"; +import { minimizeEscapes } from "./escapes"; -/** Serialize the editor's current document to canonical markdown. */ +/** + * Serialize the editor's current document to canonical markdown. + * + * remark escapes defensively, which turns Obsidian's `[[wikilinks]]`, + * `![[embeds]]` and `> [!callouts]` into literal text. `minimizeEscapes` takes + * back every escape whose removal provably parses to the same document. + * + * For the whole document, a candidate is accepted if it loads as the **live** + * document (the ideal, and one parse) or, failing that, as the fully escaped + * output would — the exact pre-minimization behavior, which differs from the + * live doc only where remark's round trip is already lossy. + */ export function docToMarkdown(editor: Editor): string { - return editor.action(getMarkdown()); + return editor.action((ctx) => { + const serializer = ctx.get(serializerCtx); + const parser = ctx.get(parserCtx); + const live = ctx.get(editorViewCtx).state.doc; + const escaped = serializer(live); + + let escapedDoc: Node | null = null; + let lastRef: string | null = null; + let lastRefDoc: Node | null = null; + return minimizeEscapes(escaped, (candidate, reference) => { + const parsed = parser(candidate); + if (reference === escaped) { + if (parsed.eq(live)) return true; + escapedDoc ??= parser(escaped); + return parsed.eq(escapedDoc); + } + // A block, verified standalone. Bisection re-tests against the same + // reference many times, so keep its parse. + if (reference !== lastRef) { + lastRef = reference; + lastRefDoc = parser(reference); + } + return lastRefDoc !== null && parsed.eq(lastRefDoc); + }); + }); } /** Replace the editor's entire document with the result of parsing `markdown`. */ diff --git a/src/main.ts b/src/main.ts index 02cd2e3..6e9157b 100644 --- a/src/main.ts +++ b/src/main.ts @@ -5,7 +5,7 @@ import "@milkdown/theme-nord/style.css"; import "./styles.css"; import { editorViewCtx, type Editor } from "@milkdown/kit/core"; -import { caretOf, createEditor, resetHistory } from "./editor/milkdown"; +import { type CaretSnapshot, caretOf, createEditor, resetHistory } from "./editor/milkdown"; import { docToMarkdown, markdownToDoc } from "./editor/serializer"; import { docToHtml, htmlToDoc } from "./editor/html-serializer"; import type { FrontMatter, SplitFile } from "./editor/frontmatter"; @@ -255,6 +255,21 @@ function loadIntoEditor(content: string, format: DocFormat, caret: "keep" | "sta // Taken before the swap: afterwards every position has been mapped to the end. const before = caret === "keep" ? caretOf(editor) : "start"; loading = true; + try { + loadIntoEditorUnguarded(content, format, before); + } finally { + // A load that throws (a document the view cannot draw) must not leave every + // later keystroke ignored as "still loading" — TabManager reloads the + // previous tab after the failure, and that reload is a normal load. + loading = false; + } +} + +function loadIntoEditorUnguarded( + content: string, + format: DocFormat, + before: CaretSnapshot | "start", +): void { if (format === "html") { // Front matter is a markdown-file concept; an `.html` document has none, and // the strip stays hidden for those tabs. @@ -395,7 +410,17 @@ async function openPath(path: string): Promise { return; } const file = await openFile(path); - openDocument(file.path, basename(file.path), file.content, formatForPath(file.path)); + try { + openDocument(file.path, basename(file.path), file.content, formatForPath(file.path)); + } catch (e) { + // The editor could not load it. TabManager has already put the previous tab + // back; the new tab never became active, holds exactly what is on disk, and + // can never be opened — so drop it rather than leave a dead tab whose click + // only fails again. The caller reports the error. + const failed = tabs.byPath(file.path); + if (failed && failed.id !== tabs.active()?.id && !failed.dirty) tabs.close(failed.id); + throw e; + } // Recorded from the path the backend resolved, and only once the read // succeeded — a file that failed to open must not enter a list whose whole // purpose is offering it again. @@ -403,6 +428,21 @@ async function openPath(path: string): Promise { setStatus(`Opened ${basename(file.path)}`); } +/** + * `openPath` for the fire-and-forget callers (sidebar click, a second launch, a + * drop). Awaited there, a failure was an unhandled rejection: clicking a note + * the editor could not load did nothing at all, with no message. + */ +async function openPathOrReport(path: string): Promise { + try { + await openPath(path); + return true; + } catch (e) { + setStatus(`Open failed: ${String(e)}`); + return false; + } +} + async function doOpenFile(): Promise { try { const path = await pickFileToOpen(); @@ -1995,7 +2035,7 @@ window.addEventListener("DOMContentLoaded", async () => { if (!editorRoot || !tabbar || !sidebarEl || !formatBar) return; sidebar = new Sidebar(sidebarEl, { - onOpenFile: (p) => void openPath(p), + onOpenFile: (p) => void openPathOrReport(p), // Routed through the same action as the menu item, so the empty state's // button cannot drift from File → Open Folder. onOpenFolder: () => runAction("menu_open_folder"), @@ -2145,7 +2185,7 @@ window.addEventListener("DOMContentLoaded", async () => { // While Toril is already running, a second double-click is forwarded here by // the single-instance plugin rather than starting a new process (§5). - void onOpenFile((path) => void openPath(path)); + void onOpenFile((path) => void openPathOrReport(path)); // Dropping notes on the window opens them. A drop is the one open path with // no file-type filter in front of it, so `selectOpenable` is an allowlist — @@ -2160,9 +2200,19 @@ window.addEventListener("DOMContentLoaded", async () => { return; } void (async () => { - for (const path of openable) await openPath(path); + let opened = 0; + for (const path of openable) if (await openPathOrReport(path)) opened++; const skipped = paths.length - openable.length; - if (skipped > 0) setStatus(`Opened ${openable.length}; skipped ${skipped}`); + // A failed open already put its own message up; only overwrite it with a + // summary that still mentions it. + if (skipped > 0 || opened < openable.length) { + const failed = openable.length - opened; + setStatus( + `Opened ${opened}` + + (skipped > 0 ? `; skipped ${skipped}` : "") + + (failed > 0 ? `; ${failed} failed to open` : ""), + ); + } })(); }); diff --git a/src/ui/tabs.ts b/src/ui/tabs.ts index ac551f1..16d6a35 100644 --- a/src/ui/tabs.ts +++ b/src/ui/tabs.ts @@ -142,7 +142,26 @@ export class TabManager { if (prev) this.cb.onDeactivate(prev); this.activeId = id; const next = this.active(); - if (next) this.cb.onActivate(next); + try { + if (next) this.cb.onActivate(next); + } catch (e) { + // A document the editor cannot load must never become the active tab. + // onActivate throws after the editor's *state* was replaced but before the + // screen redrew, so leaving `next` active would show the previous note + // while Save — which needs no dirty flag — wrote the half-loaded text over + // the file that failed. Put `prev` back (re-loading it is what clears the + // half-loaded state) and let the caller report the failure. The failed + // tab stays in the list with its buffer untouched: it was never active, so + // nothing was ever flushed into it, and a recovered buffer is not dropped. + this.activeId = prev?.id ?? null; + try { + if (prev) this.cb.onActivate(prev); + } catch { + this.activeId = null; // nothing loadable to fall back to + } + this.render(); + throw e; + } this.render(); } diff --git a/src/welcome.ts b/src/welcome.ts index 4cd45d5..6d36df9 100644 --- a/src/welcome.ts +++ b/src/welcome.ts @@ -60,7 +60,7 @@ also records a version you can go back to, and if a file changes underneath you --- -Ctrl-click a link to open it in your browser: +Ctrl-click a link to open it in your browser: https://github.com/kovirlabs/toril `; export const EMPTY = `# Untitled diff --git a/tests/emoji.test.ts b/tests/emoji.test.ts new file mode 100644 index 0000000..8ea7d6e --- /dev/null +++ b/tests/emoji.test.ts @@ -0,0 +1,81 @@ +// Gate for src/editor/emoji.ts: a character twemoji does not draw as an image +// (©, ®, ™ — emoji-regex matches them, twemoji leaves them as text) must stay +// ordinary text. As an upstream `emoji` node it crashed the view on load and was +// deleted on save (§3.2). Real emoji must still become emoji nodes. +import { afterEach, describe, expect, it } from "vitest"; +import { Editor, defaultValueCtx, editorViewCtx, parserCtx, rootCtx } from "@milkdown/kit/core"; +import type { Node as ProseNode } from "@milkdown/kit/prose/model"; +import { commonmark } from "@milkdown/kit/preset/commonmark"; +import { gfm } from "@milkdown/kit/preset/gfm"; +import { emoji } from "../src/editor/emoji"; +import { useCanonical } from "../src/editor/canonical"; +import { docToMarkdown, markdownToDoc } from "../src/editor/serializer"; + +let live: Editor | null = null; +afterEach(async () => { + await live?.destroy(); + live = null; + document.body.innerHTML = ""; +}); + +async function make(md = ""): Promise { + const root = document.createElement("div"); + document.body.appendChild(root); + live = await useCanonical( + Editor.make() + .config((ctx) => { + ctx.set(rootCtx, root); + ctx.set(defaultValueCtx, md); + }) + .use(commonmark) + .use(gfm) + .use(emoji), + ).create(); + return live; +} + +function emojiNodes(doc: ProseNode): string[] { + const out: string[] = []; + doc.descendants((n) => { + if (n.type.name === "emoji") out.push(String(n.attrs.html)); + }); + return out; +} + +describe("emoji: symbols twemoji leaves as text", () => { + it("parses ©, ® and ™ as text, not emoji nodes", async () => { + const editor = await make(); + const doc = editor.action((ctx) => ctx.get(parserCtx)("A © B ® C ™ D\n")); + expect(emojiNodes(doc)).toEqual([]); + expect(doc.textContent).toBe("A © B ® C ™ D"); + }); + + it("still parses a real emoji as an image emoji node", async () => { + const editor = await make(); + const doc = editor.action((ctx) => ctx.get(parserCtx)("Ship 🚀 now ™\n")); + const nodes = emojiNodes(doc); + expect(nodes).toHaveLength(1); + expect(nodes[0]).toContain(" { + const editor = await make("# Before\n"); + expect(() => markdownToDoc(editor, "Omnipaque® and Visipaque™, © 2026.\n")).not.toThrow(); + const view = editor.action((ctx) => ctx.get(editorViewCtx)); + expect(view.dom.textContent).toBe("Omnipaque® and Visipaque™, © 2026."); + expect(docToMarkdown(editor)).toBe("Omnipaque® and Visipaque™, © 2026.\n"); + }); + + // The emoji-presentation forms (U+FE0F) are what the `:tm:` shortcode and an + // emoji keyboard produce. twemoji *does* draw those, so they stay emoji nodes — + // the fix must not demote them — and they must survive a save byte-exact. + it("keeps the emoji-presentation forms ©️ ®️ ™️ as image emoji that round-trip", async () => { + const md = "Mark ©️ and ®️ and ™️ here.\n"; + const editor = await make(md); + const nodes = editor.action((ctx) => emojiNodes(ctx.get(editorViewCtx).state.doc)); + expect(nodes).toHaveLength(3); + for (const html of nodes) expect(html).toContain(" number { + let s = seed >>> 0; + return () => { + s = (s * 1664525 + 1013904223) >>> 0; + return s / 2 ** 32; + }; +} + +describe("findCandidates", () => { + it("finds escapes and autolink bracket pairs in prose", () => { + expect(findCandidates("a \\[b] \\* \n")).toEqual([ + { drop: [2] }, + { drop: [7] }, + { drop: [10, 22] }, + ]); + }); + + it("never offers an escaped backslash", () => { + expect(findCandidates("x\\\\y and \\\\[z]")).toEqual([]); + }); + + it("skips inline code and fenced code", () => { + const md = "`a\\[b` ``c\\*`d``\n```\n\\[lit]\n```\n> ~~~\n> \\*q\n> ~~~\nafter \\_\n"; + const found = findCandidates(md); + expect(found).toHaveLength(1); + expect(md[found[0].drop[0] + 1]).toBe("_"); + }); + + it("ignores a backslash before a non-punctuation character", () => { + expect(findCandidates("C:\\Users\\me\n")).toEqual([]); + }); +}); + +describe("splitBlocks", () => { + it("reproduces its input exactly", () => { + const rand = rng(7); + const bits = ["a", "\n", "\n\n", " \n", "```\n", "~~~\n", "> x\n", "- y\n", " "]; + for (let n = 0; n < 500; n++) { + const md = Array.from({ length: Math.floor(rand() * 20) }, () => bits[Math.floor(rand() * bits.length)]).join(""); + expect(splitBlocks(md).join("")).toBe(md); + } + }); + + it("keeps a fence with blank lines inside as one block", () => { + expect(splitBlocks("p\n\n```\na\n\nb\n```\n\nq\n")).toEqual(["p\n", "\n", "```\na\n\nb\n```\n", "\n", "q\n"]); + }); +}); + +describe("minimizeEscapes (search)", () => { + // Every case uses text no other case uses: the block cache is keyed on text, + // so reusing a block would read another case's verdict. + it("returns the input untouched, without parsing, when there is nothing to try", () => { + let calls = 0; + expect(minimizeEscapes("plain text\n", () => (calls++, true))).toBe("plain text\n"); + expect(calls).toBe(0); + }); + + it("costs one block parse plus one document parse when every removal is valid", () => { + let calls = 0; + const out = minimizeEscapes("\\[\\[a]] \\_b\\_ \\~c\n", () => (calls++, true)); + expect(out).toBe("[[a]] _b_ ~c\n"); + expect(calls).toBe(2); + }); + + it("re-verifies only the document when every block is cached", () => { + const md = "\\[\\[cached]]\n\n\\_also\\_\n"; + minimizeEscapes(md, () => true); + let calls = 0; + expect(minimizeEscapes(md, () => (calls++, true))).toBe("[[cached]]\n\n_also_\n"); + expect(calls).toBe(1); + }); + + it("verifies removals jointly — interacting escapes never both go", () => { + // Either star alone is literal; both together would be emphasis. + const out = minimizeEscapes("\\*joint\\*\n", (c) => c.includes("\\*")); + expect(out.match(/\\\*/g)).toHaveLength(1); + }); + + it("bisects to keep only the escapes that are needed", () => { + const out = minimizeEscapes("\\[1] \\[2] \\- \\[3]\n", (c) => c.includes("\\-")); + expect(out).toBe("[1] [2] \\- [3]\n"); + }); + + it("falls back to a whole-document search when blocks pass but the document does not", () => { + // Models a definition in another block turning `[ref]` into a link: standalone + // the bracket is literal, in the document it is not. + const md = "\\[ref] and \\_x\\_\n\n[ref]: https://e.x\n"; + const same = (c: string, r: string): boolean => r !== md || c.includes("\\[ref]"); + expect(minimizeEscapes(md, same)).toBe("\\[ref] and _x_\n\n[ref]: https://e.x\n"); + }); + + it("treats a throwing parser as inequality", () => { + const boom = (): boolean => { + throw new Error("boom"); + }; + expect(minimizeEscapes("\\[thrown]\n", boom)).toBe("\\[thrown]\n"); + }); + + it("bounds the number of parses on pathological input", () => { + let calls = 0; + const md = Array.from({ length: 500 }, (_, i) => `\\[p${i}]`).join(" ") + "\n"; + const out = minimizeEscapes(md, () => (calls++, false)); + expect(calls).toBeLessThanOrEqual(24 + 64); + expect(out).toBe(md); // unverified ⇒ escape kept, which is always safe + }); +}); + +async function makeEditor(md: string): Promise { + const root = document.createElement("div"); + document.body.appendChild(root); + return useCanonical( + Editor.make() + .config((ctx) => { + ctx.set(rootCtx, root); + ctx.set(defaultValueCtx, md); + }) + .use(commonmark) + .use(gfm) + .use(emoji), + ).create(); +} + +const docJson = (e: Editor): string => + JSON.stringify(e.action((ctx) => ctx.get(editorViewCtx).state.doc.toJSON())); + +const TOKENS = [ + "a", "b", "word", " ", " ", " ", "*", "_", "~", "[", "]", "(", ")", "!", "#", + "-", "+", "1.", "<", ">", "`", "|", ":", "[[x]]", "![[i.png]]", "[!tip]", + "https://e.x/a_b", "\\", "&", "^", "=", "%%", +]; + +/** What the serializer emits before minimization — the pre-fix behavior. */ +const fullyEscaped = (e: Editor): string => + e.action((ctx) => ctx.get(serializerCtx)(ctx.get(editorViewCtx).state.doc)); + +describe("minimizeEscapes (property, real parser)", () => { + // The guarantee is relative, and deliberately so: the minimized file loads as + // EXACTLY the document the fully escaped file would. Random input also hits a + // few upstream quirks that are lossy or unstable with or without this module + // (remark trims a space out of a strikethrough boundary; a bare URL ending in + // `\` grows a backslash per save — CLAUDE.md §0). Comparing against the + // escaped baseline gates *this* change without being masked by those. + it("never loads differently from the fully escaped output, and is no less stable", async () => { + const rand = rng(20261109); + let minimizedSomething = 0; + for (let n = 0; n < 200; n++) { + const lines = Array.from({ length: 1 + Math.floor(rand() * 3) }, () => + Array.from({ length: 1 + Math.floor(rand() * 10) }, () => TOKENS[Math.floor(rand() * TOKENS.length)]).join(""), + ); + const input = lines.join("\n\n") + "\n"; + const label = `case ${n}: ${JSON.stringify(input)}`; + + const first = await makeEditor(input); + const escaped = fullyEscaped(first); + const saved = docToMarkdown(first); + if (saved !== escaped) minimizedSomething++; + + const fromSaved = await makeEditor(saved); + const fromEscaped = await makeEditor(escaped); + expect(docJson(fromSaved), label).toBe(docJson(fromEscaped)); + + // Stability: wherever the old serializer was idempotent, so is the new one. + if (fullyEscaped(fromEscaped) === escaped) { + expect(docToMarkdown(fromSaved), label).toBe(saved); + } + await Promise.all([first.destroy(), fromSaved.destroy(), fromEscaped.destroy()]); + } + // Guard against a vacuous pass (e.g. a generator that stops producing escapes). + expect(minimizedSomething).toBeGreaterThan(100); + }, 120_000); +}); diff --git a/tests/html-roundtrip.test.ts b/tests/html-roundtrip.test.ts index 428f96f..4890397 100644 --- a/tests/html-roundtrip.test.ts +++ b/tests/html-roundtrip.test.ts @@ -13,7 +13,7 @@ import { afterAll, beforeAll, describe, expect, it } from "vitest"; import { Editor, rootCtx } from "@milkdown/kit/core"; import { commonmark } from "@milkdown/kit/preset/commonmark"; import { gfm } from "@milkdown/kit/preset/gfm"; -import { emoji } from "@milkdown/plugin-emoji"; +import { emoji } from "../src/editor/emoji"; import { useCanonical } from "../src/editor/canonical"; import { docToHtml, htmlToDoc } from "../src/editor/html-serializer"; import { docToMarkdown } from "../src/editor/serializer"; diff --git a/tests/roundtrip.test.ts b/tests/roundtrip.test.ts index 7120a29..42286f6 100644 --- a/tests/roundtrip.test.ts +++ b/tests/roundtrip.test.ts @@ -22,7 +22,7 @@ import { describe, expect, it } from "vitest"; import { Editor, defaultValueCtx, rootCtx } from "@milkdown/kit/core"; import { commonmark } from "@milkdown/kit/preset/commonmark"; import { gfm } from "@milkdown/kit/preset/gfm"; -import { emoji } from "@milkdown/plugin-emoji"; +import { emoji } from "../src/editor/emoji"; import { useCanonical } from "../src/editor/canonical"; import { docToMarkdown } from "../src/editor/serializer"; import { joinFrontMatter, splitFrontMatter } from "../src/editor/frontmatter"; @@ -89,6 +89,59 @@ const preserved: Record = { thematicBreak: "Above.\n\n---\n\nBelow.\n", // deliberate overlap with `fixtures` asteriskEmphasis: "Some *italic* and **bold** text.\n", underscoreEmphasis: "Some _italic_ and __bold__ text.\n", + // Escapes are emitted only where the parse proves them necessary + // (src/editor/escapes.ts). Before that, every one of these gained backslashes. + literalAsterisk: "A literal asterisk * and an underscore_in_word here.\n", + bracketCitation: "See figure [1] and [a, b].\n", + approximateTilde: "50% stenosis, ~3 cm, 5 * 3 mm.\n", + bareAutolink: "Visit https://example.com today.\n", + // emoji-regex counts ©/®/™ as emoji but twemoji leaves them as text, so + // upstream made them `emoji` nodes that crashed the view on load ("Invalid + // array passed to renderSpec") and serialized to nothing — `A © B` saved as + // `A B` (src/editor/emoji.ts). Mounting the view is part of this gate. + trademarkSymbols: "Omnipaque® and Visipaque™, © 2026.\n", + symbolsBesideEmoji: "Ship it 🚀 under the Toril™ name.\n", +}; + +// Obsidian-specific syntax. Toril does not *render* most of it yet, but it must +// never break it: before src/editor/escapes.ts, saving an edited note rewrote +// `[[Note]]` as `\[\[Note]]`, which Obsidian shows as literal text — every +// wikilink, image embed and callout in that note silently stopped working. +const obsidianPreserved: Record = { + wikilink: "See [[Pneumothorax]] and [[Pneumothorax#Signs|signs]].\n", + wikilinkBlockRef: "As in [[Case 12^finding]].\n", + adjacentWikilinks: "[[a]][[b]]\n", + wikilinkInHeading: "# Heading with [[Link]]\n", + wikilinkInBold: "**bold [[Link]]** text\n", + wikilinkInList: "- item [[Link]]\n- ![[img.png]]\n", + imageEmbed: "![[Pasted image 20261101.png]]\n", + imageEmbedSized: "![[cxr.png|400]]\n", + noteEmbed: "![[Chest#Signs]]\n", + callout: "> [!tip] Pearl\n> Deep sulcus sign on supine film.\n", + calloutFoldOpen: "> [!note]+ Title\n> body\n", + calloutFoldClosed: "> [!question]- What is the sign?\n> Answer.\n", + nestedCallout: "> > [!tip] nested\n> > x\n", + highlight: "The ==silhouette sign== matters.\n", + comment: "Text %%hidden%% here.\n", + blockId: "A key finding. ^finding1\n", + nestedTag: "Case of #chest/pneumothorax here.\n", + sizedImage: "![cxr|300](assets/cxr.png)\n", + ankiCloze: "The {{c1::deep sulcus}} sign.\n", + ankiInline: "Q:: What is X?\nA:: Y\n", + inlineMath: "Inline $E=mc^2$ math.\n", + blockMath: "$$\nx^2\n$$\n", +}; + +// The other half of the escape rule: where an escape IS needed, it stays. Each +// of these would change meaning without its backslash (task item, heading, +// ordered list, bullet, blockquote, raw HTML), so the output must keep it. +const escapesKept: Record = { + taskMarkerInList: "- \\[ ] literal\n", + headingMarker: "\\# not heading\n", + orderedMarker: "1\\. not list\n", + bulletMarker: "\\- not bullet\n", + quoteMarker: "\\> not quote\n", + rawHtml: "\\
text\n", }; // Input that Toril legitimately rewrites. Each entry pins the EXACT output, so @@ -109,9 +162,17 @@ const normalized: Record = { asteriskRule: ["Above.\n\n***\n\nBelow.\n", "Above.\n\n---\n\nBelow.\n"], underscoreRule: ["Above.\n\n___\n\nBelow.\n", "Above.\n\n---\n\nBelow.\n"], tablePadding: ["| A | B |\n|---|---|\n| 1 | 2 |\n", "| A | B |\n| - | - |\n| 1 | 2 |\n"], - intrawordUnderscore: [ - "A literal asterisk \\* and an underscore_in_word here.\n", + // Superfluous escapes are dropped — the reverse of the old normalization, + // which *added* them. Rendering is identical; only unneeded backslashes go. + superfluousEscapes: [ "A literal asterisk \\* and an underscore\\_in\\_word here.\n", + "A literal asterisk * and an underscore_in_word here.\n", + ], + // Obsidian table cells keep their escaped pipe; the column padding is computed + // before minimization, so it may run a character wide. + wikilinkAliasInTable: [ + "| a | b |\n| - | - |\n| [[x\\|y]] | z |\n", + "| a | b |\n| ---------- | - |\n| [[x\\|y]] | z |\n", ], emojiShortcode: ["Hello :smile: world\n", "Hello 😄 world\n"], // An UNUSED link reference definition is DELETED — the one place the rewrite @@ -123,7 +184,13 @@ const normalized: Record = { // this single normalization dominates the conflict rate that feat/sync-coexistence // reasons about, far more than any construct-level reformat below. crlfLineEndings: ["- one\r\n- two\r\n", "- one\n- two\n"], - bareAutolink: ["Visit https://example.com today.\n", "Visit today.\n"], + // A note damaged by an earlier Toril save (CHANGELOG, Unreleased) is repaired by + // the next one — the claim the release notes make to affected users. + repairsEscapedObsidianSyntax: [ + "See \\[\\[Note]] and !\\[\\[scan.png]].\n\n> \\[!tip] Pearl\n", + "See [[Note]] and ![[scan.png]].\n\n> [!tip] Pearl\n", + ], + angleAutolink: ["Visit today.\n", "Visit https://example.com today.\n"], plusBullets: ["+ one\n+ two\n", "- one\n- two\n"], parenOrderedMarker: ["1) one\n2) two\n", "1. one\n2. two\n"], overIndentedNesting: ["- Parent\n - Child\n", "- Parent\n - Child\n"], @@ -192,6 +259,18 @@ describe("round-trip fidelity (Phase 1 gate)", () => { }); } + for (const [name, md] of Object.entries(obsidianPreserved)) { + it(`preserves Obsidian syntax: ${name}`, async () => { + expect(await roundtrip(md)).toBe(md); + }); + } + + for (const [name, md] of Object.entries(escapesKept)) { + it(`keeps a necessary escape: ${name}`, async () => { + expect(await roundtrip(md)).toBe(md); + }); + } + for (const [name, md] of Object.entries(preserved)) { it(`preserves human-authored input: ${name}`, async () => { const once = await roundtrip(md); diff --git a/tests/tabs.test.ts b/tests/tabs.test.ts index 567f552..de95780 100644 --- a/tests/tabs.test.ts +++ b/tests/tabs.test.ts @@ -138,3 +138,71 @@ describe("TabManager", () => { expect(tab.diverged?.theirs).toBe("y\n"); // clearing one leaves the other }); }); + +// A document the editor cannot load must never become the active tab. The +// controller's onActivate throws *after* the editor's state was replaced but +// before the screen redrew, so a tab left active there is the worst of both: +// the user sees the previous note, while Save (which does not need a dirty tab) +// writes the editor's half-loaded text over the file that failed to open. +describe("TabManager: an activation that throws", () => { + function failing(badName: string) { + const container = document.createElement("div"); + const events: string[] = []; + const tm = new TabManager(container, { + onActivate: (t) => { + events.push(`activate:${t.name}`); + if (t.name === badName) throw new RangeError("cannot load"); + }, + onDeactivate: (t) => events.push(`deactivate:${t.name}`), + onCloseRequest: () => {}, + }); + return { tm, container, events }; + } + + it("keeps the previous tab active, reloads it, and rethrows", () => { + const h = failing("bad.md"); + const a = h.tm.open({ path: "/a.md", name: "a.md", content: "A" }); + h.events.length = 0; + expect(() => h.tm.open({ path: "/bad.md", name: "bad.md", content: "B" })).toThrow("cannot load"); + expect(h.tm.active()).toBe(a); + // deactivate a (flush), try bad, then reload a — the editor holds bad's + // half-loaded state until a is put back. + expect(h.events).toEqual(["deactivate:a.md", "activate:bad.md", "activate:a.md"]); + expect(h.container.querySelector('.tab[data-active="true"]')?.textContent).toContain("a.md"); + }); + + it("never flushes the editor into the tab that failed to load", () => { + const h = failing("bad.md"); + h.tm.open({ path: "/a.md", name: "a.md", content: "A" }); + try { + h.tm.open({ path: "/bad.md", name: "bad.md", content: "B" }); + } catch { + /* expected */ + } + const bad = h.tm.byPath("/bad.md"); + h.events.length = 0; + h.tm.open({ path: "/c.md", name: "c.md", content: "C" }); + // Only the tab that was really active is flushed; bad.md's buffer is untouched. + expect(h.events).toEqual(["deactivate:a.md", "activate:c.md"]); + expect(bad?.content).toBe("B"); + }); + + it("leaves no tab active when there was none to fall back to", () => { + const h = failing("bad.md"); + expect(() => h.tm.open({ path: "/bad.md", name: "bad.md", content: "B" })).toThrow(); + expect(h.tm.active()).toBeUndefined(); + }); + + it("restores the active tab when switching to an existing tab fails", () => { + const h = failing("bad.md"); + const a = h.tm.open({ path: "/a.md", name: "a.md", content: "A" }); + try { + h.tm.open({ path: "/bad.md", name: "bad.md", content: "B" }); + } catch { + /* expected */ + } + const bad = h.tm.byPath("/bad.md")!; + expect(() => h.tm.setActive(bad.id)).toThrow(); + expect(h.tm.active()).toBe(a); + }); +}); diff --git a/tests/toolbar.test.ts b/tests/toolbar.test.ts index f460932..3d274c0 100644 --- a/tests/toolbar.test.ts +++ b/tests/toolbar.test.ts @@ -18,7 +18,7 @@ import { describe, expect, it } from "vitest"; import { Editor, defaultValueCtx, editorViewCtx, rootCtx } from "@milkdown/kit/core"; import { commonmark } from "@milkdown/kit/preset/commonmark"; import { gfm } from "@milkdown/kit/preset/gfm"; -import { emoji } from "@milkdown/plugin-emoji"; +import { emoji } from "../src/editor/emoji"; import { TextSelection } from "@milkdown/kit/prose/state"; import { useCanonical } from "../src/editor/canonical"; import { docToMarkdown } from "../src/editor/serializer";