diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 512c823..d376119 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -1,6 +1,6 @@ # Contributing -Thanks for helping keep Pine Script highlighting accurate. This document covers the workflow; the README explains how the grammar is put together. +Thanks for helping keep Pine Script support accurate. This document covers the workflow; the README explains what the extension does and how the pieces fit together. ## Setup @@ -15,23 +15,30 @@ Node 20 or newer is required. ## Where things live -| Change you want to make | Edit this | -| --------------------------------------- | --------------------------------------------------------------------------------------------------------- | -| Add or remove a built-in function | `src/data/functions.json` | -| Add or remove a built-in variable | `src/data/variables.json` | -| Add or remove a built-in constant | `src/data/constants.json` | -| Add a compiler annotation (`//@...`) | `src/data/annotations.json` | -| Add a keyword, type or change a scope | `src/grammar.mjs` | -| Change folding, brackets or indentation | `language-configuration.json` | -| Add a snippet | `snippets/pinescript.code-snippets` | -| Completion, hover, commands, inference | `src/extension/core/*` (logic) and `src/extension/providers/*`, `src/extension/commands/*` (VS Code glue) | -| Change a theme color | `themes/*.json` | -| Never | `syntaxes/pinescript.tmLanguage.json`, `src/data/reference.json`, `dist/` (generated) | +| Change you want to make | Edit this | +| --------------------------------------- | ------------------------------------------------------------------------------------------------------------- | +| Add or remove a built-in function | `src/data/functions.json` | +| Add or remove a built-in variable | `src/data/variables.json` | +| Add or remove a built-in constant | `src/data/constants.json` | +| Add a compiler annotation (`//@...`) | `src/data/annotations.json` | +| Add a keyword, type or change a scope | `src/grammar.mjs` | +| Change folding, brackets or indentation | `language-configuration.json` | +| Add a snippet | `snippets/pinescript.code-snippets` | +| Completion, hover, commands, inference | `src/extension/core/*` (logic) and `src/extension/providers/*`, `src/extension/commands/*` (VS Code glue) | +| Change how a document is formatted | `src/extension/core/formatter.ts` | +| Add or change an offline rule | `src/extension/core/lint.ts`, and list the rule in the `pinescript.lint.disabledRules` enum in `package.json` | +| Change a quick fix | `src/extension/core/quick-fix.ts` | +| Definition, references, rename | `src/extension/core/symbols.ts` | +| Colour swatches, semantic tokens | `src/extension/core/colors.ts`, `src/extension/core/semantic.ts` | +| Change a theme color | `themes/*.json` (`tokenColors` for the grammar, `semanticTokenColors` for declared names) | +| Never | `syntaxes/pinescript.tmLanguage.json`, `src/data/reference.json`, `dist/` (generated) | The data files are grouped by namespace. A function `ta.sma` goes under the `"ta"` key as `"sma"`. Identifiers without a namespace go under the `""` key. Nested namespaces such as `strategy.closedtrades` are their own key. `src/extension/core` never imports `vscode`; that is what makes it testable with vitest. Providers and commands only translate between VS Code types and core types. +Everything a provider needs from a document goes through `analyze()` in `src/extension/vscode/document-cache.ts`, which parses each document once per version. Anything that walks every identifier of a file should go through `sourceIndex()` in `src/extension/core/symbols.ts` rather than rescanning the token stream: that is what keeps the offline checks linear instead of quadratic on long scripts. + ## Working on the extension ```sh @@ -68,18 +75,22 @@ Built-in identifiers come from the [Pine Script v6 reference](https://www.tradin ## Manual checklist before a release - Completion after `ta.`, inside `plot(`, after `//@`, after `import ` -- Hover on `close`, `ta.sma`, a user function, an import line +- Hover on `close`, `ta.sma`, a user function, a function parameter, an import line - Signature help through all parameters of `plot(` -- Outline shows functions, types, enums, variables +- Outline shows functions, types with fields, enums with members, top-level variables +- Go to Definition, Find All References and Rename on a user function, a variable and an enum member +- Format Document on a file with two space indentation and on one with wrapped calls +- Colour swatch and picker on `#FF9800`, `color.red` and `color.new(color.blue, 25)` +- Offline checks flag an old pragma and a bare `sma`; Convert to v6 fixes them - Generate Docstring on a function, a type and an enum -- Add Type Annotations on `tests/snapshots/strategy-v6.pine` -- Both themes on `tests/snapshots/strategy-v6.pine` -- With `pinescript.diagnostics.remote` on: an undeclared identifier is underlined +- Add Type Annotations on `tests/snapshots/strategy-v6.pine` and on `var a = 1` +- Both themes on `tests/snapshots/strategy-v6.pine`, with semantic highlighting on +- With `pinescript.diagnostics.remote` on: an undeclared identifier is underlined and the lightbulb offers a fix ## Releasing Maintainers only. 1. Update `CHANGELOG.md` and bump `version` in `package.json`. -2. Commit, then tag: `git tag v3.x.y && git push --tags`. +2. Commit, then tag: `git tag -a v3.x.y -m 3.x.y && git push origin v3.x.y`. 3. The release workflow builds the `.vsix`, attaches it to a GitHub release and publishes to the Marketplace when the `VSCE_PAT` secret is configured. diff --git a/CHANGELOG.md b/CHANGELOG.md index 148dd36..06f1351 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,29 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and ## [Unreleased] +## [3.4.0] - 2026-09-08 + +A review pass over the whole extension: three defects that produced wrong output, a large performance +fix, and documentation for every exported declaration. + +### Fixed + +- **Add Type Annotations wrote broken code.** A declaration's column was found by searching the raw line for its name, so `var a = 1` reported column 1 and the command produced `vint ar a = 1`. Columns now come from the tokens. The same column is used by Go to Definition and Rename, which were off by the same amount. +- **A `for` counter was recorded with `for` as its type**, so a value read from it was annotated `for x = i`. Counters are now typed `int`, and `for x in`, `for [index, element] in` declare their names too. All three are scoped to the loop block rather than the rest of the file. +- **The Outline pointed at the wrong lines** for the fields of a type or the members of an enum whenever a comment or a blank line sat between them; every entry was off by one per interleaved line. Fields and members now carry the line they are written on. +- Range formatting on a document with Windows line endings never recognised that nothing had changed, and returned an edit anyway. + +### Changed + +- The offline checks are 51 times faster on a long script: 160 ms to 3 ms on a 1,500 line document, and now linear rather than quadratic. Semantic highlighting on the same file went from 13 ms to 2 ms. Both used to rescan the token stream for every symbol they looked at; they share one index per document version now. +- Hovering a function parameter shows its declaration and its `//@param` text instead of nothing. +- The Outline lists the declarations at the top level of a script and no longer mixes in variables declared inside blocks. + +### Documentation + +- Every exported function, class, interface and type in `src/extension` now carries a comment saying what it is for and what its contract is, rather than only the ones with surprising behaviour. +- `CONTRIBUTING.md` covers the formatter, the offline rules, navigation, colours and semantic tokens, and explains where a change that walks every identifier belongs. + ## [3.3.0] - 2026-09-08 ### Added diff --git a/package.json b/package.json index 8de19b2..ecae288 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "pine-script-syntax-highlighter", "displayName": "Pine Script Syntax Highlighter", "description": "Pine Script v6 for Visual Studio Code: highlighting, completion, hover documentation, signature help, snippets, themes and compiler diagnostics.", - "version": "3.3.0", + "version": "3.4.0", "publisher": "ex-codes", "author": { "name": "Yankı Küçük", diff --git a/src/extension/commands/add-type-annotations.ts b/src/extension/commands/add-type-annotations.ts index 5eb0dc8..5a1f0af 100644 --- a/src/extension/commands/add-type-annotations.ts +++ b/src/extension/commands/add-type-annotations.ts @@ -3,8 +3,10 @@ import { loadReference } from '../core/reference'; import { planTypeAnnotations } from '../core/type-inference'; import { analyze } from '../vscode/document-cache'; +/** The types the compiler last reported for a document, when diagnostics are on. */ export type CompilerTypesLookup = (uri: vscode.Uri) => ReadonlyMap | undefined; +/** Registers the command that prefixes untyped declarations with their inferred type. */ export function registerAddTypeAnnotations(context: vscode.ExtensionContext, compilerTypes: CompilerTypesLookup): void { context.subscriptions.push( vscode.commands.registerCommand('pinescript.addTypeAnnotations', async () => { diff --git a/src/extension/commands/convert-to-v6.ts b/src/extension/commands/convert-to-v6.ts index 32a22d1..4383012 100644 --- a/src/extension/commands/convert-to-v6.ts +++ b/src/extension/commands/convert-to-v6.ts @@ -7,6 +7,7 @@ import { analyze } from '../vscode/document-cache'; /** The rules whose fixes together move a script from an older Pine version to the current one. */ const MIGRATION_RULES = new Set(['missing-version', 'old-version', 'legacy-name', 'unknown-argument']); +/** Registers the command that applies every migration fix in a document at once. */ export function registerConvertToV6(context: vscode.ExtensionContext, ref: ReferenceIndex): void { context.subscriptions.push( vscode.commands.registerCommand('pinescript.convertToV6', async () => { diff --git a/src/extension/commands/generate-docstring.ts b/src/extension/commands/generate-docstring.ts index 921c06f..78223d1 100644 --- a/src/extension/commands/generate-docstring.ts +++ b/src/extension/commands/generate-docstring.ts @@ -3,6 +3,7 @@ import { annotationBlockRange, docstringLines } from '../core/docstring'; import { declarationAt, type DeclSymbol } from '../core/document-model'; import { analyze } from '../vscode/document-cache'; +/** The edit that writes, or completes, the `//@` block for one declaration. */ export function docstringEdit(document: vscode.TextDocument, symbol: DeclSymbol): vscode.TextEdit { const { lines } = analyze(document); const indent = lines[symbol.line]!.match(/^\s*/)![0]; @@ -14,6 +15,7 @@ export function docstringEdit(document: vscode.TextDocument, symbol: DeclSymbol) : vscode.TextEdit.insert(new vscode.Position(symbol.line, 0), text); } +/** Registers the command and its lightbulb action for documenting a declaration. */ export function registerGenerateDocstring(context: vscode.ExtensionContext): void { context.subscriptions.push( vscode.commands.registerCommand('pinescript.generateDocstring', async (line?: number) => { diff --git a/src/extension/commands/new-file.ts b/src/extension/commands/new-file.ts index c0acce2..b4717d7 100644 --- a/src/extension/commands/new-file.ts +++ b/src/extension/commands/new-file.ts @@ -8,6 +8,7 @@ const TITLES: Record = { }; const LABELS: Record = { indicator: 'Indicator', strategy: 'Strategy', library: 'Library' }; +/** Registers the three commands that open a new script from a template. */ export function registerNewFileCommands(context: vscode.ExtensionContext): void { for (const kind of ['indicator', 'strategy', 'library'] as const) { context.subscriptions.push( diff --git a/src/extension/commands/open-reference.ts b/src/extension/commands/open-reference.ts index 31cbf09..433241d 100644 --- a/src/extension/commands/open-reference.ts +++ b/src/extension/commands/open-reference.ts @@ -2,6 +2,7 @@ import * as vscode from 'vscode'; import { REFERENCE_URL, loadReference } from '../core/reference'; import { wordAt } from '../core/tokenizer'; +/** Registers the command that opens the reference at the built-in under the cursor. */ export function registerOpenReference(context: vscode.ExtensionContext): void { context.subscriptions.push( vscode.commands.registerCommand('pinescript.openReference', async () => { diff --git a/src/extension/core/call-resolver.ts b/src/extension/core/call-resolver.ts index f0cdc6e..4442721 100644 --- a/src/extension/core/call-resolver.ts +++ b/src/extension/core/call-resolver.ts @@ -1,5 +1,6 @@ import type { Token, TokenizedLine } from './tokenizer'; +/** The call the cursor sits inside, and which argument it is on. */ export interface CallInfo { name: string; argIndex: number; @@ -28,6 +29,10 @@ function tokensBefore(tokenLines: TokenizedLine[], line: number, col: number): T return out; } +/** + * Finds the innermost call the cursor is inside, looking back over wrapped lines. Grouping + * parentheses and index brackets are stepped over, so their commas do not count as arguments. + */ export function enclosingCall(tokenLines: TokenizedLine[], line: number, col: number): CallInfo | null { const tokens = tokensBefore(tokenLines, line, col); let depth = 0; diff --git a/src/extension/core/colors.ts b/src/extension/core/colors.ts index a97746b..617667e 100644 --- a/src/extension/core/colors.ts +++ b/src/extension/core/colors.ts @@ -9,6 +9,7 @@ export interface Rgba { alpha: number; } +/** One colour found in the document, and the range the picker should replace. */ export interface ColorSpot { line: number; startCol: number; diff --git a/src/extension/core/context.ts b/src/extension/core/context.ts index 5280ca6..99e6ee0 100644 --- a/src/extension/core/context.ts +++ b/src/extension/core/context.ts @@ -1,6 +1,7 @@ import { enclosingCall, type CallInfo } from './call-resolver'; import { isInStringOrComment, type TokenizedLine } from './tokenizer'; +/** What the cursor is in the middle of typing, which decides what completion offers. */ export type CompletionContext = | { kind: 'none' } | { kind: 'annotation'; prefix: string } @@ -14,6 +15,7 @@ const RE_IMPORT = /^\s*import\s+([\w\-/.]*)$/; const RE_MEMBER = /([A-Za-z_][\w.]*)\.(\w*)$/; const RE_WORD = /(\w*)$/; +/** Works out what is being typed at a position. */ export function completionContext( tokenLines: TokenizedLine[], lineText: string, diff --git a/src/extension/core/diagnostics.ts b/src/extension/core/diagnostics.ts index 20d479c..e4dda41 100644 --- a/src/extension/core/diagnostics.ts +++ b/src/extension/core/diagnostics.ts @@ -1,5 +1,6 @@ import type { CompileResult, RawIssue } from './pine-facade'; +/** One compiler issue placed on the document, with its message resolved. */ export interface CompileDiagnostic { line: number; startCol: number; @@ -24,6 +25,7 @@ function one(issue: RawIssue, severity: 'error' | 'warning', lineCount: number): return { line, startCol, endCol, message: render(issue), severity, code: issue.code ?? null, ctx: issue.ctx }; } +/** Converts a compile result to diagnostics, clamping positions to the document. */ export function toDiagnostics(result: CompileResult, lineCount: number): CompileDiagnostic[] { const out = [ ...result.errors.map((e) => one(e, 'error', lineCount)), @@ -43,6 +45,7 @@ export function toDiagnostics(result: CompileResult, lineCount: number): Compile return out; } +/** The types the compiler inferred for each variable, which beats guessing them. */ export function compilerVariableTypes(result: CompileResult): Map { return new Map(result.variables.map((v) => [v.name, v.type])); } diff --git a/src/extension/core/docstring.ts b/src/extension/core/docstring.ts index 029a717..ef46ac0 100644 --- a/src/extension/core/docstring.ts +++ b/src/extension/core/docstring.ts @@ -3,6 +3,7 @@ import type { DeclSymbol, FunctionSymbol, LineRange } from './document-model'; const VOID_CALLS = /^(?:plot|plotshape|plotchar|plotarrow|plotcandle|plotbar|bgcolor|barcolor|fill|hline|alert|alertcondition|log\.\w+|runtime\.error|strategy\.(?:entry|exit|close|close_all|order|cancel|cancel_all|risk\.\w+)|label\.set_\w+|line\.set_\w+|box\.set_\w+|table\.(?:cell|set_\w+|clear|merge_cells)|array\.(?:push|set|unshift|clear|insert|fill|sort|reverse)|matrix\.(?:set|fill|add_row|add_col|remove_row|remove_col)|map\.(?:put|remove|clear))\s*\(/; +/** The `//@` block above a declaration, or null when the comments above it carry no annotation. */ export function annotationBlockRange(lines: string[], declLine: number): LineRange | null { let start = declLine; while (start > 0 && /^\s*\/\//.test(lines[start - 1]!)) start--; @@ -24,6 +25,7 @@ function bodyReturnsValue(fn: FunctionSymbol): boolean { return !fn.lastLine || !VOID_CALLS.test(fn.lastLine); } +/** Builds the documentation block for a declaration, keeping whatever the author already wrote. */ export function docstringLines(symbol: DeclSymbol, existing: string[], indent: string): string[] { const wanted: { tag: string; name: string | null }[] = []; if (symbol.kind === 'function') { @@ -57,6 +59,7 @@ export function docstringLines(symbol: DeclSymbol, existing: string[], indent: s return out; } +/** True when a declaration is missing part of its documentation, which is what offers the fix. */ export function needsDocstring(symbol: DeclSymbol): boolean { if (symbol.kind === 'function') { if (!symbol.docs.function && !symbol.docs.description) return true; diff --git a/src/extension/core/document-model.ts b/src/extension/core/document-model.ts index dbbde52..89b9dc7 100644 --- a/src/extension/core/document-model.ts +++ b/src/extension/core/document-model.ts @@ -1,10 +1,12 @@ import { tokenize, type TokenizedLine } from './tokenizer'; +/** An inclusive span of document lines. */ export interface LineRange { start: number; end: number; } +/** The `//@` documentation written above a declaration, split by tag. */ export interface Annotations { function: string | null; description: string | null; @@ -16,12 +18,14 @@ export interface Annotations { raw: string[]; } +/** One parameter of a user function, as written in its header. */ export interface ParamDecl { name: string; type: string | null; default: string | null; } +/** A function or method the document declares. `range` covers the header and the body. */ export interface FunctionSymbol { kind: 'function'; name: string; @@ -35,12 +39,15 @@ export interface FunctionSymbol { lastLine: string | null; } +/** One field of a user type, with the line it is written on. */ export interface FieldDecl { name: string; type: string; default: string | null; + line: number; } +/** A user-defined type and its fields. */ export interface TypeSymbol { kind: 'type'; name: string; @@ -51,11 +58,14 @@ export interface TypeSymbol { range: LineRange; } +/** One member of a user enum, with the line it is written on. */ export interface EnumMember { name: string; title: string | null; + line: number; } +/** A user-defined enum and its members. */ export interface EnumSymbol { kind: 'enum'; name: string; @@ -66,6 +76,10 @@ export interface EnumSymbol { range: LineRange; } +/** + * A variable the document declares. `scope` is where the name refers to this declaration: the rest + * of the file, the enclosing function body, or the loop block for a `for` counter. + */ export interface VariableSymbol { kind: 'variable'; name: string; @@ -77,6 +91,7 @@ export interface VariableSymbol { scope: LineRange; } +/** An `import owner/library/version as alias` line. */ export interface ImportDecl { owner: string; name: string; @@ -85,6 +100,7 @@ export interface ImportDecl { line: number; } +/** Everything read from one document without asking the compiler. */ export interface DocumentModel { version: number | null; scriptKind: 'indicator' | 'strategy' | 'library' | null; @@ -97,6 +113,7 @@ export interface DocumentModel { lineCount: number; } +/** The declarations that can carry a `//@` documentation block. */ export type DeclSymbol = FunctionSymbol | TypeSymbol | EnumSymbol; const TYPE = String.raw`[A-Za-z_][\w.]*(?:<[^<>]*(?:<[^<>]*>[^<>]*)*>)?(?:\[\])?`; @@ -110,6 +127,8 @@ const RE_ENUM = new RegExp(String.raw`^(export\s+)?enum\s+(${NAME})\s*$`); const RE_FIELD = new RegExp(String.raw`^(${TYPE})\s+(${NAME})(?:\s*=\s*(.+))?$`); const RE_ENUM_MEMBER = new RegExp(String.raw`^(${NAME})(?:\s*=\s*(.+))?$`); const RE_VARIABLE = new RegExp(String.raw`^(?:(var|varip)\s+)?(?:(${TYPE})\s+)?(${NAME})\s*=(?![=>])\s*(.*)$`); +const RE_FOR_COUNTER = new RegExp(String.raw`^for\s+(${NAME})\s*=(?![=>])`); +const RE_FOR_IN = new RegExp(String.raw`^for\s+(?:\[\s*(${NAME})\s*,\s*(${NAME})\s*\]|(${NAME}))\s+in\b`); const RE_TUPLE = /^\[\s*([\w\s,]+)\]\s*=(?![=>])/; const RE_PARAM = new RegExp(String.raw`^(?:(${TYPE})\s+)?(${NAME})(?:\s*=\s*(.+))?$`); @@ -132,10 +151,15 @@ const KEYWORDS = new Set([ 'varip', ]); +/** An annotation block with nothing filled in. */ export function emptyAnnotations(): Annotations { return { function: null, description: null, params: {}, returns: null, type: null, fields: {}, enum: null, raw: [] }; } +/** + * Reads a document into a model of what it declares. Wrapped lines and lines inside strings are + * skipped, so only real statements are considered. + */ export function buildModel(text: string): DocumentModel { const lines = text.split(/\r?\n/); const tokenLines = tokenize(text); @@ -182,7 +206,7 @@ export function buildModel(text: string): DocumentModel { const fields: FieldDecl[] = []; for (let j = i + 1; j <= range.end; j++) { const f = stripComment(lines[j]!, tokenLines[j]!).trim().match(RE_FIELD); - if (f) fields.push({ name: f[2]!, type: f[1]!, default: f[3]?.trim() ?? null }); + if (f) fields.push({ name: f[2]!, type: f[1]!, default: f[3]?.trim() ?? null, line: j }); } model.types.push({ kind: 'type', @@ -203,7 +227,7 @@ export function buildModel(text: string): DocumentModel { const members: EnumMember[] = []; for (let j = i + 1; j <= range.end; j++) { const m = stripComment(lines[j]!, tokenLines[j]!).trim().match(RE_ENUM_MEMBER); - if (m) members.push({ name: m[1]!, title: m[2]?.trim() ?? null }); + if (m) members.push({ name: m[1]!, title: m[2]?.trim() ?? null, line: j }); } model.enums.push({ kind: 'enum', @@ -243,6 +267,34 @@ export function buildModel(text: string): DocumentModel { } } + // A `for` header declares its counter, or its index and element, for the length of the loop. + const counter = code.match(RE_FOR_COUNTER); + const forIn = counter ? null : code.match(RE_FOR_IN); + if (counter || forIn) { + const range = blockRange(lines, i, indent); + const declared: { name: string; type: string | null }[] = counter + ? [{ name: counter[1]!, type: 'int' }] + : forIn![3] + ? [{ name: forIn![3], type: null }] + : [ + { name: forIn![1]!, type: 'int' }, + { name: forIn![2]!, type: null }, + ]; + for (const { name, type } of declared) { + model.variables.push({ + kind: 'variable', + name, + declaredType: type, + qualifier: null, + initializer: null, + line: i, + column: columnOf(tl, name, raw), + scope: range, + }); + } + continue; + } + const tuple = code.match(RE_TUPLE); if (tuple) { for (const name of tuple[1]! @@ -256,7 +308,7 @@ export function buildModel(text: string): DocumentModel { qualifier: null, initializer: null, line: i, - column: raw.indexOf(name), + column: columnOf(tl, name, raw), scope: { start: i, end: lines.length - 1 }, }); } @@ -272,20 +324,21 @@ export function buildModel(text: string): DocumentModel { qualifier: (v[1] as 'var' | 'varip' | undefined) ?? null, initializer: v[4]?.trim() || null, line: i, - column: raw.indexOf(v[3]!), + column: columnOf(tl, v[3]!, raw), scope: { start: i, end: lines.length - 1 }, }); } } - // Narrow variable scopes to the enclosing function body. + // Narrow variable scopes to the enclosing function body, keeping a narrower loop scope as it is. for (const variable of model.variables) { const owner = model.functions.find((f) => variable.line > f.line && variable.line <= f.range.end); - if (owner) variable.scope = { start: variable.line, end: owner.range.end }; + if (owner) variable.scope = { start: variable.line, end: Math.min(variable.scope.end, owner.range.end) }; } return model; } +/** The function, type or enum declared on a line, or null when the line declares none. */ export function declarationAt(model: DocumentModel, line: number): DeclSymbol | null { return ( model.functions.find((f) => f.line === line) ?? @@ -295,10 +348,20 @@ export function declarationAt(model: DocumentModel, line: number): DeclSymbol | ); } +/** The variables whose scope covers a line. */ export function visibleVariables(model: DocumentModel, line: number): VariableSymbol[] { return model.variables.filter((v) => v.scope.start <= line && line <= v.scope.end); } +/** + * The column a declared name sits at. Searching the raw text would find the letter inside a keyword, + * so `var a = 1` would report column 1 rather than 4. + */ +function columnOf(tl: TokenizedLine, name: string, raw: string): number { + const token = tl.tokens.find((t) => t.kind === 'ident' && t.text === name); + return token ? token.start : raw.indexOf(name); +} + function stripComment(raw: string, tl: TokenizedLine): string { const c = tl.tokens.find((t) => t.kind === 'comment'); return c ? raw.slice(0, c.start) : raw; @@ -332,6 +395,7 @@ function joinHeader( return null; } +/** Splits the parameter list of a function header, dropping the qualifiers Pine allows. */ export function parseParams(paramText: string): ParamDecl[] { const parts: string[] = []; let depth = 0; @@ -363,6 +427,7 @@ export function parseParams(paramText: string): ParamDecl[] { type DocKey = { kind: 'params' | 'fields'; name: string } | 'function' | 'description' | 'returns' | 'type' | 'enum'; +/** Reads the `//@` block written directly above a line. */ export function annotationsAbove(lines: string[], line: number): Annotations { const docs = emptyAnnotations(); const block: string[] = []; diff --git a/src/extension/core/formatter.ts b/src/extension/core/formatter.ts index 0497a47..5c99ae0 100644 --- a/src/extension/core/formatter.ts +++ b/src/extension/core/formatter.ts @@ -1,5 +1,6 @@ import { tokenize, type Token } from './tokenizer'; +/** How the editor wants blocks indented. Pine accepts four spaces or one tab per level. */ export interface FormatOptions { /** Indent with one tab per level instead of four spaces. */ useTabs?: boolean; diff --git a/src/extension/core/libraries.ts b/src/extension/core/libraries.ts index f941d41..dfb6b5b 100644 --- a/src/extension/core/libraries.ts +++ b/src/extension/core/libraries.ts @@ -1,5 +1,6 @@ import { buildModel, type EnumSymbol, type FunctionSymbol, type ImportDecl, type TypeSymbol } from './document-model'; +/** A Pine library and the symbols it exports, from the workspace or from TradingView. */ export interface LibraryInfo { id: string; title: string; @@ -12,18 +13,21 @@ export interface LibraryInfo { source: 'local' | 'remote'; } +/** How the providers reach libraries, so tests can supply their own. */ export interface LibraryLookup { forImport(imp: ImportDecl): Promise; local(): LibraryInfo[]; search(prefix: string): Promise; } +/** A lookup that finds nothing, for tests and for when library support is switched off. */ export const noLibraries: LibraryLookup = { forImport: async () => null, local: () => [], search: async () => [], }; +/** Reads a library script into its exported symbols, or null when the script is not a library. */ export function parseLibrary( text: string, id: string, diff --git a/src/extension/core/lint.ts b/src/extension/core/lint.ts index 11d546c..d9bbe5a 100644 --- a/src/extension/core/lint.ts +++ b/src/extension/core/lint.ts @@ -2,18 +2,20 @@ import { argumentSpan, renamedTo, suggestNames, type QuickFix } from './quick-fi import type { ReferenceIndex } from './reference'; import { calleeOf, - flattenTokens, isImportPath, isMemberAccess, isNamedArgument, occurrencesOf, + sourceIndex, type SymbolSource, } from './symbols'; import { nearest } from './text'; import type { Token, TokenizedLine } from './tokenizer'; +/** How loudly a rule speaks. `hint` renders faded rather than as a warning. */ export type LintSeverity = 'error' | 'warning' | 'information' | 'hint'; +/** One finding, with the fix that resolves it when there is an unambiguous one. */ export interface LintIssue { rule: string; line: number; @@ -26,10 +28,12 @@ export interface LintIssue { fix: QuickFix | null; } +/** A document analysed once, shared by every rule. */ export interface LintSource extends SymbolSource { lines: string[]; } +/** The Pine version this extension documents and checks against. */ export const TARGET_VERSION = 6; /** Calls the compiler rejects anywhere but the top level. */ @@ -53,6 +57,7 @@ const TOP_LEVEL_ONLY = new Set([ /** Names that are conventionally declared without being read. */ const IGNORED_PREFIX = '_'; +/** Runs every rule over a document. Nothing here touches the network. */ export function lint(source: LintSource, ref: ReferenceIndex): LintIssue[] { const issues: LintIssue[] = [...versionIssues(source), ...tokenIssues(source, ref), ...unusedIssues(source)]; return issues.sort((a, b) => a.line - b.line || a.startCol - b.startCol); @@ -109,7 +114,8 @@ function versionIssues(source: LintSource): LintIssue[] { /** One pass over the tokens, covering the rules that read a call or a name in place. */ function tokenIssues(source: LintSource, ref: ReferenceIndex): LintIssue[] { - const flat = flattenTokens(source.tokens); + const index = sourceIndex(source); + const { flat } = index; const bound = boundNames(source); const issues: LintIssue[] = []; const seenArguments = new Map>(); @@ -133,7 +139,7 @@ function tokenIssues(source: LintSource, ref: ReferenceIndex): LintIssue[] { } continue; } - if (isImportPath(flat, i) || isMemberAccess(flat, i)) continue; + if (isImportPath(index, i) || isMemberAccess(flat, i)) continue; if (TOP_LEVEL_ONLY.has(token.text) && flat[i + 1]?.kind === 'open') { const local = localScopeCall(source, token); diff --git a/src/extension/core/markdown.ts b/src/extension/core/markdown.ts index a234800..01eb7d4 100644 --- a/src/extension/core/markdown.ts +++ b/src/extension/core/markdown.ts @@ -3,12 +3,14 @@ import type { RefEntry, ReferenceIndex } from './reference'; const fence = (code: string) => '```pine\n' + code + '\n```'; +/** The one-line detail shown beside a completion item. */ export function entryDetail(entry: RefEntry): string { if (entry.kind === 'function') return entry.overloads[0]?.syntax ?? `${entry.name}()`; if (entry.type) return entry.type; return entry.kind; } +/** The hover card for a built-in: signature, description, parameters, remarks and a reference link. */ export function entryMarkdown(entry: RefEntry, ref: ReferenceIndex, overloadIndex = 0): string { const parts: string[] = []; if (entry.kind === 'function') { @@ -47,10 +49,12 @@ function paramLabel(p: ParamDecl): string { return `${p.type ? `${p.type} ` : ''}${p.name}${p.default !== null ? ` = ${p.default}` : ''}`; } +/** `name(type param = default, ...)` for a user function. */ export function functionSignatureLabel(fn: FunctionSymbol): string { return `${fn.name}(${fn.params.map(paramLabel).join(', ')})`; } +/** The hover card for a user function, using whatever `//@` documentation it carries. */ export function functionMarkdown(fn: FunctionSymbol, origin?: string): string { const parts = [fence(`${fn.isExport ? 'export ' : ''}${fn.isMethod ? 'method ' : ''}${functionSignatureLabel(fn)}`)]; if (origin) parts.push(`_${origin}_`); @@ -63,6 +67,7 @@ export function functionMarkdown(fn: FunctionSymbol, origin?: string): string { return parts.join('\n\n'); } +/** The hover card for a user type and its fields. */ export function typeMarkdown(t: TypeSymbol, origin?: string): string { const parts = [fence(`${t.isExport ? 'export ' : ''}type ${t.name}`)]; if (origin) parts.push(`_${origin}_`); @@ -79,6 +84,7 @@ export function typeMarkdown(t: TypeSymbol, origin?: string): string { return parts.join('\n\n'); } +/** The hover card for a user enum and its members. */ export function enumMarkdown(e: EnumSymbol, origin?: string): string { const parts = [fence(`${e.isExport ? 'export ' : ''}enum ${e.name}`)]; if (origin) parts.push(`_${origin}_`); diff --git a/src/extension/core/pine-facade.ts b/src/extension/core/pine-facade.ts index df6319c..e2b4b08 100644 --- a/src/extension/core/pine-facade.ts +++ b/src/extension/core/pine-facade.ts @@ -1,3 +1,4 @@ +/** One published library as the TradingView library list describes it. */ export interface RemoteLibrary { libId: string; user: string; @@ -7,6 +8,7 @@ export interface RemoteLibrary { docs: string; } +/** One error or warning as the compiler returns it, with its message placeholders unresolved. */ export interface RawIssue { code?: string; message: string; @@ -15,6 +17,7 @@ export interface RawIssue { end?: { line: number; column: number }; } +/** What the compiler reports about one script. */ export interface CompileResult { success: boolean; reason?: string; @@ -24,6 +27,7 @@ export interface CompileResult { functions: { name: string; syntax: string; desc?: string; args: { name: string; type: string; info?: string }[] }[]; } +/** Seams for tests: a fetch, a clock and a log. */ export interface FacadeOptions { fetch?: typeof fetch; now?: () => number; @@ -39,6 +43,10 @@ const COMPILE_CACHE_SIZE = 100; type RawResponse = { success?: boolean; reason?: string; result?: Record }; +/** + * The TradingView endpoints the extension can use. Every call is cached and de-duplicated, and + * repeated failures pause the client for a few minutes rather than retrying in a loop. + */ export class PineFacade { private readonly fetchFn: typeof fetch; private readonly now: () => number; diff --git a/src/extension/core/quick-fix.ts b/src/extension/core/quick-fix.ts index 1e79db2..0fd6ae7 100644 --- a/src/extension/core/quick-fix.ts +++ b/src/extension/core/quick-fix.ts @@ -4,6 +4,7 @@ import { ReferenceIndex } from './reference'; import { nearest } from './text'; import { tokenize, wordAt, type Token } from './tokenizer'; +/** One replacement, which may span lines when a fix renames throughout a document. */ export interface QuickFixEdit { startLine: number; startCol: number; @@ -12,6 +13,7 @@ export interface QuickFixEdit { newText: string; } +/** A named set of edits offered under the lightbulb. */ export interface QuickFix { title: string; edits: QuickFixEdit[]; diff --git a/src/extension/core/reference.ts b/src/extension/core/reference.ts index 34041cf..9544f59 100644 --- a/src/extension/core/reference.ts +++ b/src/extension/core/reference.ts @@ -1,7 +1,9 @@ import data from '../../data/reference.json'; +/** The kinds of entry the generated v6 reference holds. */ export type EntryKind = 'function' | 'variable' | 'constant' | 'keyword' | 'type' | 'annotation' | 'operator'; +/** One documented parameter of a built-in, or one field of a built-in type. */ export interface RefParam { name: string; type: string; @@ -10,12 +12,14 @@ export interface RefParam { default: string | null; } +/** One signature of a built-in function. Most have exactly one. */ export interface RefOverload { syntax: string; params: RefParam[]; returns: { type: string; description: string } | null; } +/** One entry of the v6 reference, generated by `scripts/scrape-reference.mjs`. */ export interface RefEntry { id: string; kind: EntryKind; @@ -30,17 +34,20 @@ export interface RefEntry { seeAlso: string[]; } +/** The shape of `src/data/reference.json`. */ export interface ReferenceData { version: string; generatedAt: string; entries: RefEntry[]; } +/** The published reference the hover links point into. */ export const REFERENCE_URL = 'https://www.tradingview.com/pine-script-reference/v6/'; const KIND_PRIORITY: EntryKind[] = ['function', 'variable', 'constant', 'type', 'keyword', 'annotation', 'operator']; const QUALIFIERS = /^(?:series|simple|const|input|literal)\s+/; +/** Lookups over the reference data, built once and shared by every provider. */ export class ReferenceIndex { private readonly byName = new Map(); private readonly byNamespace = new Map(); @@ -115,6 +122,7 @@ function push(map: Map, key: K, value: V): void { let singleton: ReferenceIndex | undefined; +/** The shared reference index. The data is bundled, so this never touches the network. */ export function loadReference(): ReferenceIndex { if (!singleton) singleton = new ReferenceIndex(data as ReferenceData); return singleton; diff --git a/src/extension/core/semantic.ts b/src/extension/core/semantic.ts index 9ceb074..f9f8a68 100644 --- a/src/extension/core/semantic.ts +++ b/src/extension/core/semantic.ts @@ -1,9 +1,9 @@ import { - flattenTokens, isImportPath, isMemberAccess, isNamedArgument, resolveSymbolAt, + sourceIndex, type SymbolSource, type SymbolTarget, } from './symbols'; @@ -12,6 +12,7 @@ import { export type SemanticKind = 'function' | 'method' | 'type' | 'enum' | 'enumMember' | 'parameter' | 'variable' | 'property' | 'namespace'; +/** One identifier the editor should colour for what it is rather than how it is spelled. */ export interface SemanticToken { line: number; startCol: number; @@ -36,14 +37,15 @@ const OF_TARGET: Partial> = { * already colours those, and only the document itself knows which names are the author's own. */ export function semanticTokens(source: SymbolSource): SemanticToken[] { - const flat = flattenTokens(source.tokens); + const index = sourceIndex(source); + const { flat } = index; const declared = declaredNames(source); const out: SemanticToken[] = []; for (let i = 0; i < flat.length; i++) { const token = flat[i]!; if (token.kind !== 'ident') continue; if (!declared.has(token.text.split('.')[0]!)) continue; - if (isImportPath(flat, i) || isNamedArgument(flat, i) || isMemberAccess(flat, i)) continue; + if (isImportPath(index, i) || isNamedArgument(flat, i) || isMemberAccess(flat, i)) continue; const field = memberDeclaration(source, token.line, token.start, token.text); if (field) { @@ -62,7 +64,7 @@ export function semanticTokens(source: SymbolSource): SemanticToken[] { startCol: token.start, length: member ? token.text.length : target.name.length, kind: member ? 'enumMember' : kind, - isDeclaration: target.declaration?.line === token.line && target.declaration.startCol === token.start, + isDeclaration: target.declaration?.line === token.line && target.declaration?.startCol === token.start, }); } return out; diff --git a/src/extension/core/symbols.ts b/src/extension/core/symbols.ts index cf3a4c2..ca8a820 100644 --- a/src/extension/core/symbols.ts +++ b/src/extension/core/symbols.ts @@ -1,9 +1,19 @@ -import type { DocumentModel, LineRange } from './document-model'; +import type { + DocumentModel, + EnumSymbol, + FunctionSymbol, + ImportDecl, + LineRange, + TypeSymbol, + VariableSymbol, +} from './document-model'; import { tokenAt, type Token, type TokenizedLine } from './tokenizer'; +/** What a name refers to. `builtin` covers everything the document did not declare. */ export type TargetKind = 'function' | 'method' | 'type' | 'enum' | 'enumMember' | 'variable' | 'parameter' | 'import' | 'builtin'; +/** One place a name is written. */ export interface Occurrence { line: number; startCol: number; @@ -11,6 +21,7 @@ export interface Occurrence { isDeclaration: boolean; } +/** The symbol a position refers to, and where its name means that symbol. */ export interface SymbolTarget { /** The name being referenced: the first segment of a dotted token, or the whole built-in name. */ name: string; @@ -22,11 +33,91 @@ export interface SymbolTarget { owner: string | null; } +/** The parts of a document analysis the symbol lookups need. */ export interface SymbolSource { model: DocumentModel; tokens: TokenizedLine[]; } +/** + * Lookups over one document, built once and reused. Without it every caller that walks the + * identifiers of a file would rescan the whole token stream for each symbol it asks about. + */ +export interface SourceIndex { + /** Every meaningful token in document order. */ + flat: Token[]; + /** Positions in `flat` of the identifiers sharing a first name segment. */ + byBaseName: Map; + importLines: Set; + variablesByName: Map; + functionsByName: Map; + typesByName: Map; + enumsByName: Map; + importsByAlias: Map; + /** For each function, the names its body binds, so a shadowed outer name can be skipped. */ + functionBindings: { range: LineRange; names: ReadonlySet }[]; +} + +const indexes = new WeakMap(); + +/** The index for a document, rebuilt only when the document changes. */ +export function sourceIndex(source: SymbolSource): SourceIndex { + const cached = indexes.get(source.tokens); + if (cached && cached.model === source.model) return cached.index; + const index = buildIndex(source); + indexes.set(source.tokens, { model: source.model, index }); + return index; +} + +function buildIndex(source: SymbolSource): SourceIndex { + const flat = flattenTokens(source.tokens); + const byBaseName = new Map(); + for (let i = 0; i < flat.length; i++) { + const token = flat[i]!; + if (token.kind !== 'ident') continue; + const dot = token.text.indexOf('.'); + const base = dot < 0 ? token.text : token.text.slice(0, dot); + const bucket = byBaseName.get(base); + if (bucket) bucket.push(i); + else byBaseName.set(base, [i]); + } + const { model } = source; + const variablesByName = new Map(); + for (const variable of model.variables) { + const bucket = variablesByName.get(variable.name); + if (bucket) bucket.push(variable); + else variablesByName.set(variable.name, [variable]); + } + return { + flat, + byBaseName, + importLines: new Set(model.imports.map((i) => i.line)), + variablesByName, + functionBindings: bindingsPerFunction(model), + // A name declared twice keeps its first declaration, which is the one in scope first. + functionsByName: firstByName(model.functions), + typesByName: firstByName(model.types), + enumsByName: firstByName(model.enums), + importsByAlias: new Map(model.imports.filter((i) => i.alias).map((i) => [i.alias!, i])), + }; +} + +/** Collects the parameters and local variables of every function in one pass over the model. */ +function bindingsPerFunction(model: DocumentModel): { range: LineRange; names: ReadonlySet }[] { + const bindings = model.functions.map((f) => ({ range: f.range, names: new Set(f.params.map((p) => p.name)) })); + for (const variable of model.variables) { + const owner = bindings.find((b) => variable.line > b.range.start && variable.line <= b.range.end); + if (owner) owner.names.add(variable.name); + } + return bindings; +} + +function firstByName(items: readonly T[]): Map { + const map = new Map(); + for (const item of items) if (!map.has(item.name)) map.set(item.name, item); + return map; +} + /** True for a symbol the user declared in this document, which is the only kind that can be renamed. */ export function isUserSymbol(target: SymbolTarget): boolean { return target.kind !== 'builtin' && target.declaration !== null; @@ -48,10 +139,10 @@ export function resolveSymbolAt(source: SymbolSource, line: number, column: numb /** Resolves `Enum.member`, and treats every other dotted name as a built-in. */ function resolveMember(source: SymbolSource, token: Token, segments: string[]): SymbolTarget | null { - const owner = source.model.enums.find((e) => e.name === segments[0]); + const owner = sourceIndex(source).enumsByName.get(segments[0]!); const member = owner?.members.find((m) => m.name === segments[1]); if (owner && member) { - const declaration = findMemberDeclaration(source, owner.range, member.name); + const declaration = findNameOnLine(source, member.line, member.name); return { name: `${owner.name}.${member.name}`, kind: 'enumMember', @@ -71,6 +162,7 @@ function resolveBase( lineTokens: Token[], ): SymbolTarget { const { model } = source; + const index = sourceIndex(source); const calling = nextSignificant(lineTokens, token)?.kind === 'open'; const enclosing = model.functions.find((f) => line >= f.line && line <= f.range.end); @@ -85,7 +177,7 @@ function resolveBase( }; } - const fn = model.functions.find((f) => f.name === name); + const fn = index.functionsByName.get(name); if (fn && calling) { return { name, @@ -96,10 +188,13 @@ function resolveBase( }; } - const visible = model.variables - .filter((v) => v.name === name && v.scope.start <= line && line <= v.scope.end) - .sort((a, b) => b.line - a.line); - const declared = visible[0] ?? model.variables.find((v) => v.name === name); + // The declaration in scope is the closest one above the cursor; failing that, any of that name. + const sameName = index.variablesByName.get(name); + let declared: VariableSymbol | undefined = sameName?.[0]; + for (const candidate of sameName ?? []) { + if (candidate.scope.start > line || line > candidate.scope.end) continue; + if (!declared || declared.scope.start > line || candidate.line > declared.line) declared = candidate; + } if (declared) { return { name, @@ -124,7 +219,7 @@ function resolveBase( owner: null, }; } - const type = model.types.find((t) => t.name === name); + const type = index.typesByName.get(name); if (type) { return { name, @@ -134,7 +229,7 @@ function resolveBase( owner: null, }; } - const enumeration = model.enums.find((e) => e.name === name); + const enumeration = index.enumsByName.get(name); if (enumeration) { return { name, @@ -144,7 +239,7 @@ function resolveBase( owner: null, }; } - const imported = model.imports.find((i) => i.alias === name); + const imported = index.importsByAlias.get(name); if (imported) { return { name, @@ -159,16 +254,17 @@ function resolveBase( /** Every place the target is written, in document order. */ export function occurrencesOf(source: SymbolSource, target: SymbolTarget): Occurrence[] { - const flat = flattenTokens(source.tokens); - const shadowed = shadowingRanges(source.model, target); + const index = sourceIndex(source); + const { flat } = index; + const shadowed = shadowingRanges(index, target); const found: Occurrence[] = []; - for (let i = 0; i < flat.length; i++) { + const base = target.name.split('.')[0]!; + for (const i of index.byBaseName.get(base) ?? []) { const token = flat[i]!; - if (token.kind !== 'ident') continue; if (token.line < target.scope.start || token.line > target.scope.end) continue; if (shadowed.some((r) => token.line >= r.start && token.line <= r.end)) continue; if (!matches(token.text, target.name)) continue; - if (isImportPath(flat, i) || isMemberAccess(flat, i)) continue; + if (isImportPath(index, i) || isMemberAccess(flat, i)) continue; if (isMemberDeclaration(source, token, flat, i) && target.kind !== 'enumMember') continue; if (isNamedArgument(flat, i) && !namesOwnParameter(flat, i, target)) continue; found.push({ @@ -191,17 +287,15 @@ export function occurrencesOf(source: SymbolSource, target: SymbolTarget): Occur * Function bodies that declare the same name again, where the name means something else. A global * `length` is not the `length` a function takes as a parameter. */ -function shadowingRanges(model: DocumentModel, target: SymbolTarget): LineRange[] { +function shadowingRanges(index: SourceIndex, target: SymbolTarget): LineRange[] { if (target.kind !== 'variable' && target.kind !== 'builtin') return []; const declaration = target.declaration?.line ?? -1; - return model.functions - .filter((f) => !(declaration > f.line && declaration <= f.range.end)) - .filter( - (f) => - f.params.some((p) => p.name === target.name) || - model.variables.some((v) => v.name === target.name && v.line > f.line && v.line <= f.range.end), - ) - .map((f) => f.range); + const ranges: LineRange[] = []; + for (const binding of index.functionBindings) { + if (declaration > binding.range.start && declaration <= binding.range.end) continue; + if (binding.names.has(target.name)) ranges.push(binding.range); + } + return ranges; } /** A token refers to `name` when it is the name itself or starts with it as a namespace. */ @@ -214,7 +308,7 @@ function isSamePosition(token: Token, declaration: Occurrence | null): boolean { } /** Every token that carries meaning, in document order, each knowing its line. */ -export function flattenTokens(lines: TokenizedLine[]): Token[] { +function flattenTokens(lines: TokenizedLine[]): Token[] { const flat: Token[] = []; for (const line of lines) for (const token of line.tokens) if (token.kind !== 'ws' && token.kind !== 'comment') flat.push(token); @@ -226,13 +320,13 @@ function nextSignificant(tokens: Token[], after: Token): Token | undefined { } /** The owner, library and version of an `import` are a path, not references to anything. */ -export function isImportPath(flat: Token[], index: number): boolean { - let i = index; - while (i >= 0 && flat[i]!.line === flat[index]!.line) i--; - const first = flat[i + 1]; - if (!first || first.text !== 'import') return false; +export function isImportPath(index: SourceIndex, position: number): boolean { + const token = index.flat[position]!; + if (!index.importLines.has(token.line)) return false; // Only the alias after `as` is a symbol. - for (let j = i + 1; j < index; j++) if (flat[j]!.text === 'as') return false; + for (let i = position - 1; i >= 0 && index.flat[i]!.line === token.line; i--) { + if (index.flat[i]!.text === 'as') return false; + } return true; } @@ -305,14 +399,6 @@ function findParameterDeclaration(source: SymbolSource, headerLine: number, name return null; } -function findMemberDeclaration(source: SymbolSource, range: LineRange, name: string): Occurrence | null { - for (let line = range.start + 1; line <= range.end; line++) { - const token = source.tokens[line]?.tokens.find((t) => t.kind === 'ident'); - if (token?.text === name) return { line, startCol: token.start, endCol: token.end, isDeclaration: true }; - } - return null; -} - function wholeFile(source: SymbolSource): LineRange { return { start: 0, end: Math.max(0, source.tokens.length - 1) }; } diff --git a/src/extension/core/templates.ts b/src/extension/core/templates.ts index 0039e53..56d99b3 100644 --- a/src/extension/core/templates.ts +++ b/src/extension/core/templates.ts @@ -1,5 +1,7 @@ +/** The three script kinds the New File commands can start from. */ export type TemplateKind = 'indicator' | 'strategy' | 'library'; +/** A ready-to-edit v6 script of the given kind. */ export function renderTemplate(kind: TemplateKind, opts: { title: string; date: string }): string { const title = opts.title.replace(/"/g, '\\"'); const header = `//@version=6\n// ${title}\n// Created ${opts.date}\n\n`; diff --git a/src/extension/core/tokenizer.ts b/src/extension/core/tokenizer.ts index 1378201..4f49d88 100644 --- a/src/extension/core/tokenizer.ts +++ b/src/extension/core/tokenizer.ts @@ -1,5 +1,7 @@ +/** The coarse categories the grammar-independent tokenizer distinguishes. */ export type TokenKind = 'comment' | 'string' | 'number' | 'ident' | 'op' | 'open' | 'close' | 'comma' | 'ws'; +/** One token, with the columns it spans on its line. `line` is a zero-based document line. */ export interface Token { kind: TokenKind; start: number; @@ -8,6 +10,10 @@ export interface Token { line: number; } +/** + * One tokenized line. `depthAtStart` is the bracket nesting the line opens with, which is what marks a + * wrapped line; `continuesString` says a triple-quoted string is still open at the end of it. + */ export interface TokenizedLine { tokens: Token[]; depthAtStart: number; @@ -41,6 +47,10 @@ const OPS = [ '!', ]; +/** + * Splits a document into tokens, line by line. It is deliberately not a parser: it tracks strings, + * comments and bracket depth well enough for every other module to avoid rescanning raw text. + */ export function tokenize(text: string): TokenizedLine[] { const lines = text.split(/\r?\n/); const result: TokenizedLine[] = []; @@ -151,6 +161,7 @@ export function tokenize(text: string): TokenizedLine[] { return result; } +/** The token at a column, preferring an identifier the cursor sits just after (`ta.sma|(`). */ export function tokenAt(tokens: Token[], col: number): Token | undefined { const at = tokens.find((t) => t.start <= col && col < t.end); if (at?.kind === 'ident') return at; @@ -158,6 +169,7 @@ export function tokenAt(tokens: Token[], col: number): Token | undefined { return tokens.find((t) => t.end === col && t.kind === 'ident') ?? at; } +/** True when a column falls inside a string or a comment, where completion and hover stay quiet. */ export function isInStringOrComment(line: TokenizedLine, col: number): boolean { const t = line.tokens.find((t) => t.start <= col && col < t.end) ?? line.tokens.find((t) => t.start < col && col <= t.end); @@ -165,6 +177,7 @@ export function isInStringOrComment(line: TokenizedLine, col: number): boolean { return t.kind === 'string' || t.kind === 'comment'; } +/** The dotted word around a column, such as `ta.sma`, or null when there is no word there. */ export function wordAt(lineText: string, col: number): { text: string; start: number; end: number } | null { const isWord = (c: string) => /[A-Za-z0-9_.]/.test(c); let start = col; diff --git a/src/extension/core/type-inference.ts b/src/extension/core/type-inference.ts index 77c802a..6d1d0d0 100644 --- a/src/extension/core/type-inference.ts +++ b/src/extension/core/type-inference.ts @@ -2,6 +2,7 @@ import { visibleVariables, type DocumentModel } from './document-model'; import { ReferenceIndex } from './reference'; import { tokenize, type Token } from './tokenizer'; +/** What inference may consult: the reference, the document and, when available, the compiler. */ export interface InferenceScope { ref: ReferenceIndex; model: DocumentModel; @@ -9,6 +10,7 @@ export interface InferenceScope { compilerTypes?: ReadonlyMap; } +/** An insertion that puts a type keyword in front of a declared name. */ export interface AnnotationEdit { line: number; column: number; @@ -34,6 +36,10 @@ const INPUT_TYPES: Record = { const COMPARISON = new Set(['==', '!=', '<', '<=', '>', '>=']); const NOT_A_VALUE = new Set(['plot', 'hline', 'void']); +/** + * The type of an initialiser expression, or null when it cannot be known without the compiler. + * Returning null is always safe: the caller simply leaves the declaration alone. + */ export function inferType(expr: string, scope: InferenceScope): string | null { const tokens = tokenize(expr)[0]?.tokens.filter((t) => t.kind !== 'ws' && t.kind !== 'comment') ?? []; if (!tokens.length) return null; @@ -217,6 +223,7 @@ function numeric(a: string | null, b: string | null): string | null { return null; } +/** Plans a type keyword for every untyped declaration it can, and names the ones it cannot. */ export function planTypeAnnotations( model: DocumentModel, ref: ReferenceIndex, diff --git a/src/extension/extension.ts b/src/extension/extension.ts index 3d78b90..bacb2a9 100644 --- a/src/extension/extension.ts +++ b/src/extension/extension.ts @@ -30,6 +30,7 @@ import { getSettings } from './vscode/settings'; const SELECTOR: vscode.DocumentSelector = { language: 'pinescript' }; let providerDisposables: vscode.Disposable[] = []; +/** Starts the controllers and registers everything the extension contributes. */ export function activate(context: vscode.ExtensionContext): void { context.subscriptions.push(output()); const facade = new PineFacade({ log }); @@ -76,6 +77,7 @@ export function activate(context: vscode.ExtensionContext): void { log('Pine Script extension activated'); } +/** Registers the providers that a setting can switch off, replacing any previous registration. */ export function registerProviders(context: vscode.ExtensionContext, libraries: LibraryLookup): void { for (const d of providerDisposables) d.dispose(); providerDisposables = []; @@ -110,4 +112,5 @@ export function registerProviders(context: vscode.ExtensionContext, libraries: L context.subscriptions.push(...providerDisposables); } +/** Nothing to tear down: every disposable is owned by the extension context. */ export function deactivate(): void {} diff --git a/src/extension/providers/code-action.ts b/src/extension/providers/code-action.ts index 565ca27..a746e7f 100644 --- a/src/extension/providers/code-action.ts +++ b/src/extension/providers/code-action.ts @@ -14,6 +14,7 @@ export type IssueLookup = (uri: vscode.Uri) => readonly CompileDiagnostic[] | un /** Looks up the offline lint issues last reported for a document. */ export type LintLookup = (uri: vscode.Uri) => readonly LintIssue[] | undefined; +/** Offers the fixes for compiler and offline findings, plus the docstring refactor. */ export class PineCodeActionProvider implements vscode.CodeActionProvider { static readonly metadata: vscode.CodeActionProviderMetadata = { providedCodeActionKinds: [vscode.CodeActionKind.QuickFix, vscode.CodeActionKind.Refactor], diff --git a/src/extension/providers/completion.ts b/src/extension/providers/completion.ts index 865db81..fca89ca 100644 --- a/src/extension/providers/completion.ts +++ b/src/extension/providers/completion.ts @@ -77,6 +77,7 @@ const TRIGGER_HINTS: vscode.Command = { title: 'Trigger parameter hints', }; +/** Suggests built-ins, keywords, user symbols, named arguments, annotations and import paths. */ export class PineCompletionProvider implements vscode.CompletionItemProvider { constructor( private readonly ref: ReferenceIndex, @@ -221,6 +222,7 @@ export class PineCompletionProvider implements vscode.CompletionItemProvider { } } +/** A completion item for a function the user or a library declares. */ export function userFunctionItem(f: FunctionSymbol, origin?: string): vscode.CompletionItem { const item = new vscode.CompletionItem( f.name, @@ -233,18 +235,21 @@ export function userFunctionItem(f: FunctionSymbol, origin?: string): vscode.Com return item; } +/** A completion item for a user or library type. */ export function userTypeItem(t: TypeSymbol, origin?: string): vscode.CompletionItem { const item = new vscode.CompletionItem(t.name, vscode.CompletionItemKind.Class); item.documentation = new vscode.MarkdownString(typeMarkdown(t, origin)); return item; } +/** A completion item for a user or library enum. */ export function userEnumItem(e: EnumSymbol, origin?: string): vscode.CompletionItem { const item = new vscode.CompletionItem(e.name, vscode.CompletionItemKind.Enum); item.documentation = new vscode.MarkdownString(enumMarkdown(e, origin)); return item; } +/** Completion items for everything a library exports. */ export function libraryExportItems(lib: LibraryInfo): vscode.CompletionItem[] { const origin = `from ${lib.id}`; return [ diff --git a/src/extension/providers/decorations.ts b/src/extension/providers/decorations.ts index 6fee04a..623301c 100644 --- a/src/extension/providers/decorations.ts +++ b/src/extension/providers/decorations.ts @@ -37,6 +37,7 @@ const TOKEN_TYPES: SemanticKind[] = [ ]; const TOKEN_MODIFIERS = ['declaration']; +/** The token types and modifiers this extension emits, in the order the builder indexes them. */ export const SEMANTIC_LEGEND = new vscode.SemanticTokensLegend(TOKEN_TYPES, TOKEN_MODIFIERS); /** diff --git a/src/extension/providers/diagnostics-controller.ts b/src/extension/providers/diagnostics-controller.ts index ac6fb82..ba5d76f 100644 --- a/src/extension/providers/diagnostics-controller.ts +++ b/src/extension/providers/diagnostics-controller.ts @@ -5,6 +5,10 @@ import type { Settings } from '../vscode/settings'; const DEBOUNCE_MS = 600; +/** + * Sends open documents to the TradingView compiler and shows what it reports. Off unless + * `pinescript.diagnostics.remote` is on, because the whole script leaves the machine. + */ export class DiagnosticsController { private readonly collection = vscode.languages.createDiagnosticCollection('pinescript'); private readonly timers = new Map>(); diff --git a/src/extension/providers/document-symbol.ts b/src/extension/providers/document-symbol.ts index a17070d..9f6d084 100644 --- a/src/extension/providers/document-symbol.ts +++ b/src/extension/providers/document-symbol.ts @@ -1,6 +1,7 @@ import * as vscode from 'vscode'; import { analyze } from '../vscode/document-cache'; +/** Fills the Outline view with the declarations at the top level of a script. */ export class PineDocumentSymbolProvider implements vscode.DocumentSymbolProvider { provideDocumentSymbols(document: vscode.TextDocument): vscode.DocumentSymbol[] { const { model, lines } = analyze(document); @@ -27,13 +28,13 @@ export class PineDocumentSymbolProvider implements vscode.DocumentSymbolProvider lineRange(t.line, t.line), ); s.children = t.fields.map( - (f, i) => + (f) => new vscode.DocumentSymbol( f.name, f.type, vscode.SymbolKind.Field, - lineRange(t.line + 1 + i, t.line + 1 + i), - lineRange(t.line + 1 + i, t.line + 1 + i), + lineRange(f.line, f.line), + lineRange(f.line, f.line), ), ); symbols.push(s); @@ -47,19 +48,20 @@ export class PineDocumentSymbolProvider implements vscode.DocumentSymbolProvider lineRange(e.line, e.line), ); s.children = e.members.map( - (m, i) => + (m) => new vscode.DocumentSymbol( m.name, m.title ?? '', vscode.SymbolKind.EnumMember, - lineRange(e.line + 1 + i, e.line + 1 + i), - lineRange(e.line + 1 + i, e.line + 1 + i), + lineRange(m.line, m.line), + lineRange(m.line, m.line), ), ); symbols.push(s); } for (const v of model.variables) { - if (model.functions.some((f) => v.line > f.line && v.line <= f.range.end)) continue; // locals stay out of the outline + // Only the declarations at the top level: locals, loop counters and block variables are noise here. + if (/^\s/.test(lines[v.line] ?? '')) continue; symbols.push( new vscode.DocumentSymbol( v.name, diff --git a/src/extension/providers/formatting.ts b/src/extension/providers/formatting.ts index cedd06b..6cc0b39 100644 --- a/src/extension/providers/formatting.ts +++ b/src/extension/providers/formatting.ts @@ -26,7 +26,8 @@ export class PineFormattingProvider const last = range.end.character === 0 && range.end.line > first ? range.end.line - 1 : range.end.line; const formatted = formatRange(document.getText(), first, last, { useTabs: !options.insertSpaces }); const target = new vscode.Range(first, 0, last, lineLength(document, last)); - if (formatted === textOf(document, first, last)) return []; + // The document may use CRLF while the comparison text is joined with LF. + if (formatted.replace(/\r\n/g, '\n') === textOf(document, first, last)) return []; return [vscode.TextEdit.replace(target, formatted)]; } } diff --git a/src/extension/providers/hover.ts b/src/extension/providers/hover.ts index c4d54ea..23ba938 100644 --- a/src/extension/providers/hover.ts +++ b/src/extension/providers/hover.ts @@ -6,6 +6,7 @@ import type { ReferenceIndex } from '../core/reference'; import { isInStringOrComment, wordAt } from '../core/tokenizer'; import { analyze } from '../vscode/document-cache'; +/** Documents whatever the cursor rests on: built-ins, user symbols, parameters and imports. */ export class PineHoverProvider implements vscode.HoverProvider { constructor( private readonly ref: ReferenceIndex, @@ -81,6 +82,15 @@ export class PineHoverProvider implements vscode.HoverProvider { if (type) return new vscode.Hover(new vscode.MarkdownString(typeMarkdown(type)), range); const en = model.enums.find((e) => e.name === word.text || word.text.startsWith(`${e.name}.`)); if (en) return new vscode.Hover(new vscode.MarkdownString(enumMarkdown(en)), range); + const enclosing = model.functions.find((f) => position.line >= f.line && position.line <= f.range.end); + const parameter = enclosing?.params.find((p) => p.name === word.text); + if (parameter) { + const decl = `${parameter.type ? `${parameter.type} ` : ''}${parameter.name}${parameter.default !== null ? ` = ${parameter.default}` : ''}`; + const described = enclosing!.docs.params[parameter.name]; + const md = '```pine\n' + decl + '\n```' + (described ? `\n\n${described}` : ''); + return new vscode.Hover(new vscode.MarkdownString(md), range); + } + const variable = visibleVariables(model, position.line).find((v) => v.name === word.text); if (variable) { const decl = `${variable.qualifier ? `${variable.qualifier} ` : ''}${variable.declaredType ? `${variable.declaredType} ` : ''}${variable.name}${variable.initializer ? ` = ${variable.initializer}` : ''}`; diff --git a/src/extension/providers/library-index.ts b/src/extension/providers/library-index.ts index 31bbb05..518f439 100644 --- a/src/extension/providers/library-index.ts +++ b/src/extension/providers/library-index.ts @@ -4,6 +4,7 @@ import { parseLibrary, type LibraryInfo, type LibraryLookup } from '../core/libr import type { PineFacade } from '../core/pine-facade'; import type { Settings } from '../vscode/settings'; +/** Finds libraries: the ones in the workspace, and the published ones when that is enabled. */ export class LibraryIndex implements LibraryLookup { private readonly localByUri = new Map(); private readonly remote = new Map>(); diff --git a/src/extension/providers/signature-help.ts b/src/extension/providers/signature-help.ts index 1ba3469..acd0252 100644 --- a/src/extension/providers/signature-help.ts +++ b/src/extension/providers/signature-help.ts @@ -6,6 +6,7 @@ import { functionSignatureLabel } from '../core/markdown'; import type { ReferenceIndex } from '../core/reference'; import { analyze } from '../vscode/document-cache'; +/** Shows the signature of the call being typed, including library and constructor calls. */ export class PineSignatureHelpProvider implements vscode.SignatureHelpProvider { constructor( private readonly ref: ReferenceIndex, diff --git a/src/extension/vscode/document-cache.ts b/src/extension/vscode/document-cache.ts index 3b9df1f..42b6df3 100644 --- a/src/extension/vscode/document-cache.ts +++ b/src/extension/vscode/document-cache.ts @@ -2,6 +2,7 @@ import * as vscode from 'vscode'; import { buildModel, type DocumentModel } from '../core/document-model'; import { tokenize, type TokenizedLine } from '../core/tokenizer'; +/** Everything read from a document once per version: its model, its tokens and its lines. */ export interface Analysis { model: DocumentModel; tokens: TokenizedLine[]; @@ -11,6 +12,7 @@ export interface Analysis { const MAX = 20; const cache = new Map(); +/** The analysis of a document, reusing the last one while the document is unchanged. */ export function analyze(document: vscode.TextDocument): Analysis { const key = document.uri.toString(); const hit = cache.get(key); @@ -23,6 +25,7 @@ export function analyze(document: vscode.TextDocument): Analysis { return analysis; } +/** Drops a closed document from the cache. */ export function forget(uri: vscode.Uri): void { cache.delete(uri.toString()); } diff --git a/src/extension/vscode/output.ts b/src/extension/vscode/output.ts index 4510b43..84d19a9 100644 --- a/src/extension/vscode/output.ts +++ b/src/extension/vscode/output.ts @@ -2,6 +2,7 @@ import * as vscode from 'vscode'; let channel: vscode.OutputChannel | undefined; +/** The "Pine Script" output channel, created on first use. */ export function output(): vscode.OutputChannel { if (!channel) { channel = vscode.window.createOutputChannel('Pine Script'); @@ -9,6 +10,7 @@ export function output(): vscode.OutputChannel { return channel; } +/** Writes a timestamped line to the output channel. */ export function log(message: string): void { const stamp = new Date().toISOString().slice(11, 19); output().appendLine(`[${stamp}] ${message}`); diff --git a/src/extension/vscode/settings.ts b/src/extension/vscode/settings.ts index 1cd0d05..b3ca0f5 100644 --- a/src/extension/vscode/settings.ts +++ b/src/extension/vscode/settings.ts @@ -1,5 +1,6 @@ import * as vscode from 'vscode'; +/** The extension settings, read together so a provider never queries them one at a time. */ export interface Settings { completion: boolean; hover: boolean; @@ -12,6 +13,7 @@ export interface Settings { lintDisabledRules: string[]; } +/** The current settings. */ export function getSettings(): Settings { const c = vscode.workspace.getConfiguration('pinescript'); return { diff --git a/tests/core/document-model.test.ts b/tests/core/document-model.test.ts index 9cbcbfa..60f97c6 100644 --- a/tests/core/document-model.test.ts +++ b/tests/core/document-model.test.ts @@ -32,14 +32,14 @@ describe('buildModel on a library', () => { it('parses types and enums', () => { const t = m.types.find((t) => t.name === 'Level')!; expect(t.fields).toEqual([ - { name: 'price', type: 'float', default: null }, - { name: 'name', type: 'string', default: '"level"' }, + { name: 'price', type: 'float', default: null, line: 17 }, + { name: 'name', type: 'string', default: '"level"', line: 18 }, ]); expect(t.docs.fields.price).toBe('The level.'); const e = m.enums.find((e) => e.name === 'Side')!; expect(e.members).toEqual([ - { name: 'long', title: '"Long"' }, - { name: 'short', title: null }, + { name: 'long', title: '"Long"', line: 22 }, + { name: 'short', title: null, line: 23 }, ]); expect(e.docs.enum).toBe('Trade direction.'); }); @@ -95,3 +95,53 @@ describe('buildModel on a consumer script', () => { expect(m.variables.some((v) => v.name === 'color')).toBe(false); }); }); + +describe('buildModel: declaration positions and loop variables', () => { + it('reports the column of the name, not a letter inside a keyword', () => { + const m = buildModel('var a = 1\nvarip r = 2\nfloat t = 3\n'); + const byName = Object.fromEntries(m.variables.map((v) => [v.name, v])); + expect(byName.a?.column).toBe(4); + expect(byName.r?.column).toBe(6); + expect(byName.t?.column).toBe(6); + }); + + it('declares the counter of a for loop as an int scoped to the loop', () => { + const m = buildModel('total = 0\nfor i = 0 to 10\n total += i\nplot(total)\n'); + const counter = m.variables.find((v) => v.name === 'i')!; + expect(counter).toMatchObject({ declaredType: 'int', line: 1, column: 4 }); + expect(counter.scope).toEqual({ start: 1, end: 2 }); + }); + + it('declares the element, and the index, of a for in loop', () => { + const single = buildModel('for value in prices\n x = value\n'); + expect(single.variables.find((v) => v.name === 'value')).toMatchObject({ declaredType: null, column: 4 }); + + const pair = buildModel('for [idx, el] in prices\n x = el\n'); + expect(pair.variables.find((v) => v.name === 'idx')).toMatchObject({ declaredType: 'int', column: 5 }); + expect(pair.variables.find((v) => v.name === 'el')).toMatchObject({ declaredType: null, column: 10 }); + }); + + it('keeps a loop scope narrower than the function around it', () => { + const m = buildModel('f() =>\n for i = 0 to 2\n x = i\n 0\n'); + expect(m.variables.find((v) => v.name === 'i')?.scope).toEqual({ start: 1, end: 2 }); + }); +}); + +describe('buildModel: members among comments', () => { + it('records the line each field and member is written on', () => { + const m = buildModel( + [ + 'type Point', + ' // the price', + ' float price', + '', + ' int index', + 'enum Mode', + ' // fast', + ' fast = "F"', + ].join('\n'), + ); + expect(m.types[0]?.fields.map((f) => f.line)).toEqual([2, 4]); + expect(m.enums[0]?.members.map((mem) => mem.line)).toEqual([7]); + }); +}); diff --git a/tests/core/type-inference.test.ts b/tests/core/type-inference.test.ts index 8ce27a2..e23676a 100644 --- a/tests/core/type-inference.test.ts +++ b/tests/core/type-inference.test.ts @@ -73,3 +73,21 @@ describe('planTypeAnnotations', () => { expect(plan.edits.map((e) => e.line)).toEqual([1]); }); }); + +describe('planTypeAnnotations: declarations that used to come out broken', () => { + it('inserts the type after var and varip, not inside them', () => { + const text = 'var a = 1\nvarip r = 2\n'; + const plan = planTypeAnnotations(buildModel(text), ref, null); + const lines = text.split('\n'); + const applied = plan.edits.map( + (e) => lines[e.line]!.slice(0, e.column) + e.insert + lines[e.line]!.slice(e.column), + ); + expect(applied).toEqual(['var int a = 1', 'varip int r = 2']); + }); + + it('gives a value read from a loop counter the the counter type', () => { + const text = 'for i = 0 to 10\n x = i\n'; + const plan = planTypeAnnotations(buildModel(text), ref, null); + expect(plan.edits).toEqual([{ line: 1, column: 4, insert: 'int ' }]); + }); +});