RFC 040: Scoped DSL Surface Forms

[dannymeijer]

Area

Incan Language (syntax/semantics), Compiler (parser/typechecker/lowering), Tooling (formatter/LSP)

Summary

RFC 040 introduces scoped DSL surface forms for explicit DSL blocks and their registered subgrammars. A registered DSL may declare local syntax shapes that carry block-owned meaning only inside eligible positions of the owning DSL context, while preserving ordinary Incan semantics and targeted diagnostics elsewhere.

RFC document: 040_scoped_dsl_surface_forms.md

Motivation

Global operator overloading is the right model when a type genuinely supports an operator everywhere, but DSLs often need concise syntax that is meaningful only inside explicit blocks. Scoped surface forms let DSL authors register that local syntax without turning it into global operator behavior or ad hoc parser exceptions.

Planned design contract

• Scoped DSL surface forms are the parent model, covering operator-like glyphs (>>, |>, ->, +), binding-like glyphs (:=), and expression-form surfaces such as leading-dot paths (.column, .order.amount).
• Accepted scoped-surface occurrences must carry durable semantic metadata: descriptor identity, owning block identity, eligible-position identity, parsed payload, source span, diagnostics identity, and lowering/tooling handoff identity.
• Positive scope and misuse scope are separate descriptor concerns: a surface may be accepted narrowly while still producing targeted diagnostics elsewhere in the activating file or module.
• Resolution is deterministic: core grammar wins in core-only positions; in eligible DSL positions, the innermost eligible owning block wins; same-depth conflicts are ambiguity diagnostics.
• DSL authors may provide diagnostic templates, but the compiler gates when those messages can fire.
• Formatter and LSP consume scoped-surface metadata rather than rediscovering meaning from raw punctuation.

Canonical examples

pipeline user_sync:
    extract >> normalize >> validate >> store

query active_orders:
    FROM orders
    WHERE .status == "active" and .amount > 100
    SELECT .customer_id, .order.amount

api app:
    route get + delete "/users/:id" -> users.destroy

Layers affected

• Frontend recognition — distinguish scoped-surface occurrences in eligible DSL positions from ordinary expressions.
• Semantic analysis — preserve descriptor identity and deterministic ownership through typechecking and diagnostics.
• Lowering / execution handoff — preserve block-owned meaning and expand pairwise chains correctly.
• RFC 027 extension — add scoped-surface descriptor metadata to the vocab registration model.
• Formatter — preserve scoped surface forms and use semantic hints such as chain mode without hardcoding DSL syntax.
• LSP — expose block-local surface meaning in hover/highlighting/completions and diagnostics.

Status

RFC 040 is now Planned. Design decisions are settled in the RFC document; implementation can begin.

Related RFCs

• RFC 027 (incan-vocab block registration — the substrate this RFC builds on)
• RFC 028 (Global operator overloading — the complementary mechanism this RFC does not replace)
• RFC 045 (Scoped DSL symbol surfaces — the identifier-level companion mechanism)

Checklist

• I checked for an existing RFC/issue covering this.
• I can describe how this impacts existing code and how to migrate (if needed).

[dannymeijer]

RFC has moved to Planned status. Design is settled around scoped DSL surface forms as the parent model, including operator-like glyphs, binding-like glyphs, and expression-form surfaces such as leading-dot paths. Implementation can begin from this planned contract.

[dannymeijer]

RFC has moved to In Progress. Implementation plan added to the RFC document and pasted below for reference.

Implementation Plan

Phase 1: Descriptor contract and manifest transport

• Extend the public vocab API with scoped-surface descriptors for operator-like glyphs, binding-like glyphs, and expression-form surfaces.
• Serialize registered scoped surfaces through library manifests without losing family, eligibility, misuse scope, diagnostic, receiver, and formatter metadata.
• Validate descriptor conflicts early enough that a malformed vocab crate cannot silently publish ambiguous scoped-surface behavior.

Phase 2: Frontend recognition and semantic artifacts

• Extend syntax/AST support so accepted scoped-surface occurrences produce typed payloads instead of raw punctuation matches.
• Add leading-dot expression-form recognition for eligible DSL positions, including chained paths such as .order.amount.
• Preserve descriptor identity, owning block identity, eligible-position identity, payload, receiver derivation, and source span for later phases and tooling.

Phase 3: Activation, resolution, and diagnostics

• Carry active scoped-surface descriptors from imported vocab crates into the file/module surface context.
• Resolve scoped-surface occurrences using the deterministic lexical ownership rules in this RFC.
• Emit author-provided, compiler-gated diagnostics for outside-scope use, wrong operands, invalid binding targets, and ambiguous ownership.

Phase 4: Formatter, LSP, and lowering handoff

• Format scoped-surface expressions from semantic metadata rather than ad-hoc punctuation rewriting.
• Expose scoped-surface identity and payload metadata to LSP diagnostics, hover, highlighting, and completions.
• Hand accepted scoped-surface artifacts to desugarers/lowering without requiring later phases to rediscover meaning from source text.

Phase 5: User-facing docs, release notes, and versioning

• Update vocab-authoring documentation with descriptor examples and diagnostic-template guidance.
• Add release notes for RFC 040 and document current limitations.
• Bump the active development version for the implementation.