Skip to content

feat(highlighting): colour CI job scripts as shell - #395

Open
X-Guardian wants to merge 1 commit into
eFAILution:betafrom
X-Guardian:feat/highlight-script-blocks
Open

X-Guardian wants to merge 1 commit into
eFAILution:betafrom
X-Guardian:feat/highlight-script-blocks

Conversation

@X-Guardian

Copy link
Copy Markdown
Contributor

Summary

  • Job before_script, script and after_script values are now highlighted as shell in YAML. This is done with injection grammars that work on top of either VS Code's YAML grammar or Red Hat's, plus VS Code's shell grammar.
  • Credit to YAML with Script is in the README. A follow-up PR adds ShellCheck linting of the same script blocks, using the same job-level rule.

Change Type

  • feat
  • fix
  • refactor
  • docs
  • test
  • chore

Context

User-facing impact

  • Script blocks read as shell: commands, keywords, strings, variables and if/fi structure. YAML structure keeps YAML colours: keys, - markers, |/> headers and !reference [...].
  • Only real job scripts are coloured. A spec input named script, include: inputs: script:, and keys nested inside a job (such as under rules:) stay YAML.
  • Layouts covered: inline values, block scalars, indented and compact (same-indent) lists, comments between items, default:, hidden jobs, interpolated job names and names containing colons, legacy global before_script/after_script, and 2- or 4-space indentation.
  • Works whichever YAML grammar is active. This extension's extensionPack installs Red Hat YAML, whose grammar replaces VS Code's.

How the job-level rule works in a grammar

TextMate regexes can't see a key's parent, and the active YAML grammar can't be relied on to say either: VS Code's nests a scope per mapping level, but Red Hat's has none. So the injection tracks structure itself, by indentation, and hands everything that isn't a script back to source.yaml:

  1. Top-level keys. A reserved keyword (spec, include, variables, workflow and so on, matching NON_JOB_TOP_LEVEL_KEYWORDS) opens a meta.gitlab-ci.non-job region. A legacy before_script/after_script opens a script region. Any other key, including default:, opens a meta.gitlab-ci.job region. Each runs until the next top-level key or ---.
  2. Inside a job. Each line at the job's first indentation level opens an entry that claims its deeper lines. A script, before_script or after_script entry is a script region; any other entry is YAML. So a script: nested under rules: sits inside a YAML entry and never matches.
  3. Inside a script. Each - item is its own region, closed by the next item at the same indentation, so an unterminated quote or a trailing \ in one item can't spill into the next. !reference, *alias, [], |/> headers and - markers keep YAML colours.

The injection selector excludes the three region scopes, so it only ever matches at the top level.

The bash grammar only starts a statement after ^, ;, | and similar, so a command after YAML indentation or - would get no colour. A statement rule starts one at each line's first word, using source.shell#typical_statements.

Inside a block scalar every line keeps its YAML indentation, but the bash grammar only ends a heredoc on a terminator at column 0. So done <<EOF … EOF would turn the rest of the block into heredoc text. A second injection, gitlab-ci-script-heredoc.injection.tmLanguage.json, mirrors the bash heredoc rules but allows the terminator to be indented. It is a separate grammar because it must run inside the script region at any depth (inside case, loops and subshells), which the main grammar's selector excludes.

Line continuations. source.shell starts a command name with a \G rule. In a .sh file, \G never matches at the start of a new line. Inside a script region, the while rules (ours, and the built-in YAML grammar's) leave \G at the start of every line. Without a fix, every \-continued line was read as a new command name, so a long tool \ --a x \ --b y command alternated colours line by line. A third injection, gitlab-ci-script-continuation.injection.tmLanguage.json, handles this:

  • It runs only where a line starts inside a shell command, which can only be a continuation line.
  • There it applies line-start copies of source.shell#option and #argument, which win over the \G rule.
  • Its last rule consumes the indentation before a continuation that starts with an operator such as ||.

Checked against real pipelines

Across 919 real CI files, under both YAML grammars, the highlighted lines match the script lines the linter finds. The script blocks are coloured the same as the same script in a .sh file, apart from minor scope differences.

Known limits (in the README)

  • Grammars can't filter by file name, so script: keys in other YAML files are coloured too. The original extension behaves the same way.
  • If YAML with Script is also installed, its injection wins and its own heredoc bug returns, so the README says to disable it.

GitLab scope

  • gitlab.com
  • self-managed GitLab
  • both

Affected areas

  • Component Browser
  • Hover provider
  • Completion provider
  • Validation provider
  • Cache and refresh behavior
  • GitLab API calls/auth/token storage
  • Docs only
  • Syntax highlighting (new)

What changed by bucket

Bucket Files Approach
Grammar syntaxes/gitlab-ci-script.injection.tmLanguage.json Indentation-tracked job, entry, script and item regions; rules for !reference, aliases, [], block headers, sequence markers and statement starts. Script text goes to source.shell, everything else to source.yaml.
Heredocs syntaxes/gitlab-ci-script-heredoc.injection.tmLanguage.json Heredoc rules that accept an indented terminator, injected only inside script regions and outside strings and comments.
Continuations syntaxes/gitlab-ci-script-continuation.injection.tmLanguage.json Line-start option and argument rules, injected only inside shell commands, so \-continued lines aren't read as new commands.
Contribution package.json, .vscode/extensions.json, .gitignore Three contributes.grammars entries with injectTo: ["source.yaml"]. Recommends the TmLanguage Syntax Highlighter extension, which validates grammar files. Dev dependencies vscode-textmate and vscode-oniguruma for the test.
Docs README.md A highlighting paragraph and its limits.
Tests tests/extension-host/suite/scriptGrammar.test.ts, tests/fixtures/grammars/ See below.

Validation

Local checks

  • npm ci (clean install)
  • npm run check-types
  • npm run lint (clean, --max-warnings 0)
  • npm run test:extension-host (42 passing)
  • npx @vscode/vsce ls lists all three grammar files in syntaxes/
  • Manual verification in VS Code Extension Host

Tests

  • scriptGrammar.test.ts tokenises with vscode-textmate, and runs every case twice:
    • VS Code's YAML grammar, loaded from vscode.env.appRoot, so it matches the VS Code version under test;
    • Red Hat's YAML grammar, from a pinned copy in tests/fixtures/grammars/ (MIT, source noted in its README).
  • Cases:
    • a component template with a spec input named script next to a real job script:;
    • include inputs and rules-nested keys;
    • a global variable named script between two jobs;
    • block scalars, compact lists with comments, default:, hidden jobs at 4-space indentation, legacy globals;
    • jobs after column-0 comment blocks, and job names containing colons;
    • an unterminated quote and a trailing \ in one item not spilling into the next;
    • \-continued options and arguments, including inside $( ), after an assignment, and a continuation starting with ||;
    • YAML colouring for block headers, !reference, *alias items and before_script: [];
    • heredocs with indented terminators (unquoted inside a case, and quoted <<-), with the shell after them coloured normally;
    • command names getting entity.name.command.shell.

Breaking Changes

  • No breaking changes
  • Breaking changes (describe below)

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant