Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -472,6 +472,24 @@ every input the tool ever sees, costs nothing in release, and tells the optimize
ordered container); and `unordered_dense` invalidates references on insert (values live in one
vector), so never hold a `T&` into it across an insert.

### Adding a language: common stays common

A new language is **data plus, at most, small per-language pieces for semantics that genuinely differ** —
never a copy of another language's mechanism. The step-by-step path is `prompts/add-a-language.md`; the
rules it enforces:

- **Data, not branches.** A `kLangTable` row, a `queries/<lang>/tags.scm` in the shared capture
vocabulary, and a row in each exhaustive `switch( Lang )` table. No language-named capture kinds.
- **Reuse the shared mechanism.** Scope walks, shadowing (`VarDecl` bindings with spans) and imports
(`Binding`/`Include`) each exist once. Extend them; a second copy under a new language's name is a
review finding, and `--clones` shows it. No language-neutral qualified-call resolver exists yet
(`src/elixir_resolve.h` is Elixir-specific); raise generalising it in your PR.
- **Refuse, don't guess.** A call whose qualifier (`Mod.f`, `ns/f`) cannot be resolved is counted as
unresolved, never laddered to every definition of that short name.
- **Known gap.** Tags-pass predicates (`#eq?`/`#any-of?`) do not run yet, so keyword-headed definitions
(Elixir today; a Lisp-style `defn` if one is added) keep a small keyword table in C++. That table belongs in `tags.scm` once the
mechanism lands; it is not a pattern to extend.

### Interfaces

- **Structured-binding returns** over out-params: `auto [ nodes, edges ] = build( … );`.
Expand Down
46 changes: 40 additions & 6 deletions prompts/add-a-language.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,12 +60,43 @@ The build must still complete **with the network off**. That is the point of ven

## STEP 3 — extraction

- `src/ingest_LANG.h` — the extraction module. Read `src/ingest_elixir.h` first; it is the newest
and the most conventional.
- `src/ingest.cpp` — include it and dispatch to it.
- `src/ingest_sidecap.h` — sidecar capture, if LANG has doc comments worth carrying.
- `src/ingest_metrics.h`, `src/clones.h`, `src/lintcatalog.h`, `src/htmlexport.h`, `src/cli.h` — each
has a per-language arm. Grep for the newest language name to find every one.
**A language is data first, and C++ only where its semantics truly differ.** What is common must stay
common: twenty languages each carrying a private copy of the same idea is twenty places to fix one bug.

- `queries/LANG/tags.scm` — this is where most of a language lives. Use **only the shared capture
vocabulary**: `@name`, `@definition.{function,method,class,struct,interface,var,constant,macro,module,type,field}`,
`@reference.{call,import,implementation,type}`. Do not invent a language-named capture
(`@definition.LANG`, `@LANG.head`); every consumer downstream has to learn a new name for it.
- The exhaustive `switch( Lang )` tables (`langTag`, `isCodeLang`, `dependencyCapable`, …) — the compiler
lists every one you missed (`-Werror=switch`). A row each, no logic.
- Then the arms the compiler cannot see: `grep` for a recently added language (`Lang::Elixir`) and read
every hit. Lists and masks are not switches — e.g. `kAllCatalogLangs` in `src/lintcatalog.h`, the
`langBit` masks in `src/clones.h`, `wordLang` in `src/ingest_metrics.h`. Semantic hooks (arity,
params, attributes) live today in the per-language branches of `src/ingest_sidecap.h`.
- Per-language C++ is for semantics the grammar cannot express. **Before writing a helper, look for the
shared one:**

| You need | Use (do not copy) |
|---|---|
| the scope a definition sits in | `enclosingScopeOf` (`src/ingest_names.h`) |
| a local that shadows a global name (`let`, params) | a `VarDecl` binding with its block span — the generic shadow pass (`suppressShadowedReferences`) does the rest |
| imports / aliases / `require … :as` | `Binding` / `Include` facts (`src/model.h`), as JS imports and Elixir aliases already do |
| module-qualified calls (`Mod.f`, `ns/f`) | no language-neutral resolver exists yet; `src/elixir_resolve.h` is Elixir-specific and the only one. Raise generalising it in your PR rather than writing a `LANG_resolve.h` beside it |

Then run `ripwire --clones` on your tree: your new code should add no clone group.
- **A qualified call you cannot resolve is counted as unresolved — never sent to the global name ladder.**
`(str/join …)` laddered to every `join` in the corpus mints edges that are not real (the PR #81 decision:
the ladder gave one Elixir function 55 callers where 3 were real).

**A known gap, so you are not surprised by it:** the tags pass does not evaluate `#eq?`/`#any-of?`
predicates yet (see the notes in `queries/javascript/tags.scm`). A language whose definitions are *a call
with a keyword head* — Elixir `def`, or a Lisp-style grammar (e.g. Clojure's `defn`, if added) — therefore has to keep that keyword list in C++ for
now. That is one reason `src/ingest_elixir.h` exists (it also holds arity, params, attribute and alias logic); it is **not a template to copy**. If you hit this, keep
the list a small `constexpr` table in one place, and say in the PR that it belongs in `tags.scm` once
predicates run. Maintainers will move it; you are not asked to build the mechanism.

If the shared mechanism you need does not exist, say so in the PR. We would rather add it with you than
see the pattern forked.

Extract **definitions with spans** first. Call edges are a separate round.

Expand All @@ -82,6 +113,9 @@ Miss the mirror and `qextractionkeycheck` fails reporting a version mismatch tha
totally unrelated bug. Bump on **any** grammar, `.scm`, or extraction change — the cache is keyed on
it, and a stale cache serving old symbols for new code is the worst failure this tool has.

**Do not take the next integer blindly.** Parallel pull requests each need their own number, and the
maintainers assign you a parser version number in the PR.

Then regenerate `test/qschemetrip.hash` and update `test/printf_parity.manifest`.

---
Expand Down
Loading