new version with source references

.github/workflows/python-package.yml

@@ -82,7 +82,7 @@ jobs:
         os: [ ubuntu-latest, macos-latest ]
         python-version: [ "3.10", "3.13" ]
         # Oldest najaeda the package declares support for, and the newest.
-        najaeda: [ "==0.7.27", "" ]
+        najaeda: [ "==0.7.29", "" ]
     steps:
       - name: Checkout code
         uses: actions/checkout@v5

.gitmodules

@@ -7,3 +7,6 @@
 [submodule "thirdparty/googletest"]
 	path = thirdparty/googletest
 	url = https://github.com/google/googletest.git
+[submodule "thirdparty/ImGuiColorTextEdit"]
+	path = thirdparty/ImGuiColorTextEdit
+	url = https://github.com/goossens/ImGuiColorTextEdit

CLAUDE.md

@@ -56,7 +56,8 @@ remaining piece to actually produce the file this flag consumes.
 
 ## Building
 
-Clone with submodules (`thirdparty/imgui`, `thirdparty/naja`) — if already cloned
+Clone with submodules (`thirdparty/imgui`, `thirdparty/naja`,
+`thirdparty/ImGuiColorTextEdit`) — if already cloned
 without them: `git submodule update --init --recursive`.
 
 ### Native standalone target (`naja-schematic-standalone`)
@@ -91,12 +92,18 @@ Run directly, optionally with a netlist to load (dispatched by extension —
 directory), `--liberty <path>` (repeatable, once per file — defines the
 primitive cell library for a `.v` design; not supported for `.sv`, since
 `LocalSNLProvider::loadSystemVerilog()`/`SNLSVConstructor` has no liberty
-hook, and the CLI fails loudly rather than silently ignoring it), and/or a
-diagnosis JSON to pre-load (`--diagnosis <path>`, any order, all optional —
-see `loadDiagnosisFile()` in `AppLogic.cpp`):
+hook, and the CLI fails loudly rather than silently ignoring it), a
+diagnosis JSON to pre-load (`--diagnosis <path>` — see
+`loadDiagnosisFile()` in `AppLogic.cpp`), and/or directories to look up
+relative source file references in (`--source-root <dir>`, repeatable; see
+"Source references" under Wire protocol), path prefixes to rewrite for
+files recorded on another machine (`--source-map FROM=TO`, repeatable),
+and `--yosys-src` (`.v` only:
+decode Yosys `(* src *)` attributes into source origins; also a checkbox in
+File > Open Verilog...) — any order, all optional:
 
 ```bash
-../naja-schematic-build/native-debug/naja-schematic-standalone [path/to/netlist] [--liberty path/to/cells.lib]... [--diagnosis path/to/diagnosis.json]
+../naja-schematic-build/native-debug/naja-schematic-standalone [path/to/netlist] [--liberty path/to/cells.lib]... [--diagnosis path/to/diagnosis.json] [--source-root dir]... [--source-map FROM=TO]... [--yosys-src]
 
 # gate-level Verilog + Liberty example:
 ../naja-schematic-build/native-debug/naja-schematic-standalone design.v --liberty cells.lib
@@ -200,7 +207,11 @@ WASM viewer, so najaeda users get a viewer from `pip install` alone:
   the universe directly on the host's thread -- so a host holding its lock,
   or running on the thread `run` hops to, never deadlocks on itself.
 - `server.py` — the `naja-schematic` CLI (same flags as above, plus `--host`,
-  `--open`, `--diagnosis <json>`, `--stdio`) and its two transports: a
+  `--open`, `--diagnosis <json>`, `--stdio`, `--source-root <dir>`
+  (repeatable; `source_roots=` on `ViewerSession`/`ViewerServer`/`show()`),
+  `--source-map FROM=TO` (repeatable; `source_maps=` there too),
+  `--yosys-src` (with `--verilog`: `VerilogConfig.yosys_source_attributes`))
+  and its two transports: a
   WebSocket server that also serves the viewer page on the same port
   (`static/index.html` sets `Module.najaWsUrl` from `location`, query
   string included), and JSON
@@ -532,7 +543,7 @@ the same `id_path`/`path` pair as diagnosis items:
     // path/id_path: the resolved instance's names and ids (the request's, as given, if not found)
     "instance": {                        // absent for the top design ([])
       "path": [["u1", 3, "Mod"], ["u2", 7, "AND2"]],   // [name, child_id, model] per level
-      "design_ref": {...}, "primitive_type": "and", "has_instances": false, "source_loc": null,
+      "design_ref": {...}, "primitive_type": "and", "has_instances": false, "has_sources": true,
       "terms": [ {"name": "A", "child_id": 0, "direction": 0}, ... ] } }  // expanded_instance_terms shape
   ```
   The viewer then:
@@ -651,6 +662,48 @@ so unlike a term's properties there's no "Direction" entry, and nets don't
 offer a schematic-side "Show Properties" entry point (only terms/instances
 appear as boxes/pins there).
 
+**Source references** (full design: `docs/source_information.md`; needs
+najaeda >= 0.7.29). naja keeps, per object, an optional *declaration*
+(where it's written in the loaded input: a gate's netlist line, an
+instance's instantiation, a design's definition) and ordered *origins*
+(upstream RTL, e.g. Yosys `src`), each qualified by provider/language/
+representation (`""` = unknown -- shown as is, never inferred, same rule as
+gate types), plus the older unclassified legacy slot (`getSourceLoc()`).
+SV and gate-level Verilog fill declarations; Yosys `src` becomes origins
+only when asked (`--yosys-src`, `yosys_source_attributes`: naja never
+guesses provenance). Entries carry a sparse `"has_sources": true`
+(instances: own references -- `hasSourceReferences()`, allocation-free --
+or legacy, or the model's declaration; a bus: itself or any bit) -- this
+replaced the old inline `source_loc`. `get_sources`
+takes the `get_properties` address and answers `sources_response` with
+`subject`, the address echoed, and `sources`: `declaration`, `origin`s
+(with `index`), `model_declaration` (instances), `legacy` only when there's
+no typed reference; a bus bit without its own falls back to its bus's.
+`load_source` names one entry (address + `role` + `index`), **never a
+path**: the host reads only files the design references -- first through
+each source map whose FROM prefixes it at a path boundary (if a file is
+there), then an absolute path as is, a relative one under each source root
+then the current directory (`resolveSourceFile()`/`resolve_source_file()`,
+Python groups roots and maps as `protocol.SourcePaths`), whole up to 4 MiB, else
+the lines within 500 of the reference (`first_line`, `truncated`). Both
+backends: `requestSources()` in `LocalSNLProvider.cpp`,
+`request_sources()` in `protocol.py`; shared cases in
+`tests/data/source_protocol_cases.json`, on a Yosys-style gate netlist
+both runners load with naja's Verilog loader (add one when changing
+either). The raw Python API is read-only for sources, so Python tests get
+source data by loading files, never by setting it.
+The tree (flagged rows) and schematic boxes (and module frames, always)
+offer "Show Sources"; the reply fills the "Source" tab, opens the first
+entry's file, and grows the bottom panel to 45% of the height once
+(`AppState::growSourcePanel`; never shrinks it). A found `source_response`
+also says `"stale"`: the file was modified after the design was loaded
+(`LocalSNLProvider::loadedAt_`, stamped in `freshDB()`/`setDB()`;
+`SourcePaths.loaded_at`, stamped by `ViewerSession` at start and on each
+`design_changed()`), so naja's ranges may not match it -- the Source tab
+shows a warning. naja keeps no file hashes, so modification time is the
+check. A found `source_response` also carries `path`, the absolute
+path the host read, used by the native "Open in Editor".
+
 ### Core modules (`src/`)
 
 - **`NetlistTree`** — the design hierarchy tree (instances, terms, nets, bus
@@ -680,6 +733,31 @@ appear as boxes/pins there).
   `root_response`/`root_loaded`.
 - **`PropertiesView`** — renders the current `PropertiesStore` contents as a
   two-column name/value table into the "Properties" bottom-panel tab.
+- **`SourceStore`** — global static store (same pattern) for the "Source"
+  tab: the last `sources_response` (subject, the object's address, its
+  `SourceRef`s; `loadRequest(i)` builds the `load_source` for one) and the
+  shown file (`source_response`: lines, `first_line` offset, the reference
+  range; `highlightBytes()` gives the byte span to shade per line, by the
+  1-based inclusive columns, never inventing an unknown end). Cleared on
+  every fresh `root_response`/`root_loaded`.
+- **`SourceView`** — renders `SourceStore` into the "Source" tab: the
+  reference rows (click to load one) above the file, shown in a read-only
+  [ImGuiColorTextEdit](https://github.com/goossens/ImGuiColorTextEdit)
+  editor (submodule; `TextEditor.cpp` is in `CORE_SOURCES`) drawn with the
+  embedded monospace Cousine font (`CousineFont.h`, loaded in
+  `setupFonts()`): coloring by `VerilogLanguage`, find (Ctrl+F), copy, the
+  reference's range as a background squiggle (byte columns converted with
+  `SourceStore::glyphIndex()`), line markers for it and the object's other
+  references into the same file, line numbers offset by `first_line` for a
+  windowed file. Native only: "Open in Editor" runs
+  `$NAJA_SCHEMATIC_EDITOR`, else VS Code (`code` on PATH, else the CLI in
+  the app bundle), else `open`/`xdg-open` (`SourceStore::editorPattern()`,
+  `editorCommand()`, path shell-quoted) on a worker thread, and shows its
+  result or error next to the button.
+- **`VerilogLanguage`** — the editor's Verilog/SystemVerilog language
+  definition: keywords, types, `` ` `` directive lines, comments, strings,
+  `$system` tasks, escaped identifiers and sized/based numbers (its own
+  tokenizers, `tests/VerilogLanguageTest.cpp`). Coloring only, not a parser.
 - **`TraceStore`** — global static store (same pattern) for the driver
   traces in the schematic: id, label (the start pin), palette color,
   visibility. A trace is registered when its `trace_driver` request is sent,

CMakeLists.txt

@@ -97,13 +97,16 @@ set(CORE_SOURCES
   src/DiagnosisView.cpp
   src/SourceStore.cpp
   src/SourceView.cpp
+  src/VerilogLanguage.cpp
   src/PropertiesStore.cpp
   src/PropertiesView.cpp
   src/SelectionStore.cpp
   src/TraceStore.cpp
   src/TraceView.cpp
   src/AppLogic.cpp
   ${IMGUI_SOURCES}
+  # Read-only code view of the Source tab (SourceView).
+  ${CMAKE_SOURCE_DIR}/thirdparty/ImGuiColorTextEdit/TextEditor.cpp
 )
 
 # ============================================================
@@ -267,10 +270,12 @@ if(NOT EMSCRIPTEN_BUILD)
       src/EquipotentialView.cpp
       src/SchematicView.cpp
       src/SvgPainter.cpp
+      src/VerilogLanguage.cpp
       ${IMGUI_DIR}/imgui.cpp
       ${IMGUI_DIR}/imgui_draw.cpp
       ${IMGUI_DIR}/imgui_tables.cpp
       ${IMGUI_DIR}/imgui_widgets.cpp
+      ${CMAKE_SOURCE_DIR}/thirdparty/ImGuiColorTextEdit/TextEditor.cpp
       ${NAJA_SCHEMATIC_TEST_SOURCES}
     )
     target_link_libraries(naja-schematic-tests PRIVATE gtest_main)
@@ -294,7 +299,7 @@ if(NOT EMSCRIPTEN_BUILD)
       naja_nl_dump
     )
     target_compile_definitions(naja-schematic-provider-tests PRIVATE
-      NAJA_SCHEMATIC_PROTOCOL_CASES="${CMAKE_CURRENT_SOURCE_DIR}/tests/data/anonymous_protocol_cases.json"
+      NAJA_SCHEMATIC_TEST_DATA="${CMAKE_CURRENT_SOURCE_DIR}/tests/data"
     )
 
     enable_testing()

LICENSES/OFL-1.1.txt

@@ -0,0 +1,43 @@
+SIL OPEN FONT LICENSE
+
+Version 1.1 - 26 February 2007
+
+PREAMBLE
+
+The goals of the Open Font License (OFL) are to stimulate worldwide development of collaborative font projects, to support the font creation efforts of academic and linguistic communities, and to provide a free and open framework in which fonts may be shared and improved in partnership with others.
+
+The OFL allows the licensed fonts to be used, studied, modified and redistributed freely as long as they are not sold by themselves. The fonts, including any derivative works, can be bundled, embedded, redistributed and/or sold with any software provided that any reserved names are not used by derivative works. The fonts and derivatives, however, cannot be released under any other type of license. The requirement for fonts to remain under this license does not apply to any document created using the fonts or their derivatives.
+
+DEFINITIONS
+
+"Font Software" refers to the set of files released by the Copyright Holder(s) under this license and clearly marked as such. This may include source files, build scripts and documentation.
+
+"Reserved Font Name" refers to any names specified as such after the copyright statement(s).
+
+"Original Version" refers to the collection of Font Software components as distributed by the Copyright Holder(s).
+
+"Modified Version" refers to any derivative made by adding to, deleting, or substituting — in part or in whole — any of the components of the Original Version, by changing formats or by porting the Font Software to a new environment.
+
+"Author" refers to any designer, engineer, programmer, technical writer or other person who contributed to the Font Software.
+
+PERMISSION & CONDITIONS
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of the Font Software, to use, study, copy, merge, embed, modify, redistribute, and sell modified and unmodified copies of the Font Software, subject to the following conditions:
+
+1) Neither the Font Software nor any of its individual components, in Original or Modified Versions, may be sold by itself.
+
+2) Original or Modified Versions of the Font Software may be bundled, redistributed and/or sold with any software, provided that each copy contains the above copyright notice and this license. These can be included either as stand-alone text files, human-readable headers or in the appropriate machine-readable metadata fields within text or binary files as long as those fields can be easily viewed by the user.
+
+3) No Modified Version of the Font Software may use the Reserved Font Name(s) unless explicit written permission is granted by the corresponding Copyright Holder. This restriction only applies to the primary font name as presented to the users.
+
+4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font Software shall not be used to promote, endorse or advertise any Modified Version, except to acknowledge the contribution(s) of the Copyright Holder(s) and the Author(s) or with their explicit written permission.
+
+5) The Font Software, modified or unmodified, in part or in whole, must be distributed entirely under this license, and must not be distributed under any other license. The requirement for fonts to remain under this license does not apply to any document created using the Font Software.
+
+TERMINATION
+
+This license becomes null and void if any of the above conditions are not met.
+
+DISCLAIMER
+
+THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER DEALINGS IN THE FONT SOFTWARE.

README.md

@@ -23,6 +23,10 @@ native desktop app and a WASM app that runs in a browser or a VSCode webview.
 - Diagnosis overlay: color-codes and annotates flagged instances/pins/nets
   from a `diagnosis_response` JSON payload, with hover tooltips and a flat
   diagnostics list panel
+- Source cross-probing: **Show Sources** on an instance, term or net lists
+  where it is declared and, for synthesized logic, the RTL it came from
+  (naja source references, e.g. from Yosys `src` attributes), and opens
+  the file at the highlighted range
 - Loads Verilog, SystemVerilog, or pre-built SNL netlists natively; the
   browser build talks to a Python netlist server over WebSocket
 - Runs natively on macOS or Linux, or anywhere with a browser/VSCode via WASM
@@ -54,7 +58,23 @@ Cmd in the native macOS app (Ctrl in a browser, on any OS).
 | Click a box | Select the instance (also in the design tree) |
 | Click a wire | Select (highlight) its net |
 | Click a box's hierarchy glyph | Show / hide what is inside it |
-| Right-click | Context menu: trace to driver, properties, fit, clear |
+| Right-click | Context menu: trace to driver, sources, properties, fit, clear |
+
+| Keys (Source tab, after clicking in the text) | Action |
+|---|---|
+| Ctrl `F` | Find in the shown file |
+| Ctrl `G` | Find the next match |
+| Ctrl `C` | Copy the selected text |
+| Ctrl `A` | Select the whole file |
+
+The native app also has an **Open in Editor** button in the Source tab: it
+runs `$NAJA_SCHEMATIC_EDITOR` (`{file}`, `{line}` and `{col}` are filled in,
+e.g. `subl {file}:{line}:{col}`). Without it, VS Code is used at the line
+(`code -g` if on PATH, else the CLI inside `Visual Studio Code.app`), else
+the system's default app; the result (or the error) shows next to the
+button. Edits made there reach the viewer only when the design is loaded
+again: until then, the Source tab warns that the file changed after the
+design was loaded and its highlighted lines may be off.
 
 In a notebook or a VSCode webview, the viewer takes keys only while its
 canvas has focus (click it first), and the host may keep some Ctrl chords
@@ -104,15 +124,22 @@ Run (`../naja-schematic-build/native-debug/naja-schematic-standalone` on
 macOS, `build/naja-schematic-standalone` on Linux), optionally with a
 netlist to load (dispatched by extension — `.v` → Verilog, `.sv` →
 SystemVerilog, otherwise treated as an SNL directory), one or more
-`--liberty` files defining the primitive cell library for a `.v` design, and/or
-a diagnosis JSON to pre-load:
+`--liberty` files defining the primitive cell library for a `.v` design,
+a diagnosis JSON to pre-load, and/or directories to look up relative source
+file paths in (Show Sources):
 
 ```bash
-naja-schematic-standalone [path/to/netlist] [--liberty path/to/cells.lib]... [--diagnosis path/to/diagnosis.json]
+naja-schematic-standalone [path/to/netlist] [--liberty path/to/cells.lib]... [--diagnosis path/to/diagnosis.json] [--source-root dir]... [--source-map FROM=TO]... [--yosys-src]
 ```
 
-`<netlist>`, `--liberty <path>` and `--diagnosis <path>` are all optional and
-order-independent. `--liberty` is repeatable (once per file) and only
+`<netlist>`, `--liberty <path>`, `--diagnosis <path>` and `--source-root
+<dir>` are all optional and order-independent. `--source-root` is
+repeatable: a relative source path is looked up in each, in order, then in
+the current directory. `--source-map FROM=TO` (repeatable) rewrites paths
+recorded on another machine, e.g. Yosys `src` paths of a synthesis server:
+`--source-map /home/ci/flow=/work/flow`. `--yosys-src` (`.v` only; also a checkbox in File >
+Open Verilog...) decodes Yosys `(* src *)` attributes, so Show Sources on a
+synthesized gate also lists the RTL it came from. `--liberty` is repeatable (once per file) and only
 supported for `.v` designs — a gate-level Verilog netlist that instantiates
 Liberty std cells needs it to resolve them; it errors out for `.sv`, since the
 SystemVerilog frontend has no liberty hook. Without `--diagnosis`, use File >