Skip to content

lint: require configured codebase lints to run in CI #49

Description

@x1f9-ricon

Problem

Repos can declare a codebase lint portfolio in mise.toml under [_.codebase].lint, but CI can drift and stop running that configured aggregate. A repo then appears to have adopted shared lint rules, while pull requests only run tests or a stale hand-written subset.

This came out of an x1f9 codebase-wide QA pilot over 22 repos using this bounded corpus:

  • mise.toml / .mise.toml
  • .mise/tasks/**
  • .github/workflows/*.yml|*.yaml

Corpus size: 430 files, 1,046,638 bytes (~261k rough tokens).

Related: #41 asks whether the aggregate codebase lint command should be clarified/restored. This issue is the CI-enforcement side of the same drift risk.

Observed examples

Directly enforced

These repos configure [_.codebase].lint and have a workflow that directly runs aggregate lint:

  • KnickKnackLabs/chat@b4e8581
    • mise.toml: [_.codebase] lint = [...]
    • .github/workflows/test.yml: run: codebase lint "$PWD"
  • KnickKnackLabs/desks@d1918e5
    • .github/workflows/test.yml: run: codebase lint "$PWD"
  • KnickKnackLabs/modules@aa0a6be
    • .github/workflows/test.yml: run: codebase lint "$PWD"
  • KnickKnackLabs/sphincters@9aec333
    • .github/workflows/test.yml: run: codebase lint "$PWD"

ricon-family/fold@0e12030 is also effectively enforced because .github/workflows/ci.yml runs mise run test, and .mise/tasks/test runs "$CODEBASE_BIN" lint when no test args are supplied.

Not enforced or unclear

  • KnickKnackLabs/emails@2131b11
    • mise.toml: lint = ["mise-settings", "gum-table", "or-true"]
    • .github/workflows/test.yml: runs mise run test, mise run test-integration, and bun test src/; no aggregate codebase lint path found.
  • KnickKnackLabs/shimmer@ebaeace
    • mise.toml: lint = ["mise-settings", "gum-table", "bats-test-helper", "bats-test-task", "mcr-scope", "or-true", "shellcheck"]
    • .github/workflows/pr-check.yml: runs mise run test 2>&1 || bats test/; no aggregate codebase lint path found.
  • KnickKnackLabs/shiv@9df70a4
    • mise.toml: lint = ["mise-settings", "bats-test-helper", "bats-test-task", "mcr-scope", "or-true", "shellcheck", "gum-table"]
    • workflows run mise run test / completions; no aggregate codebase lint path found.
  • ricon-family/den@0e7c312
    • mise.toml: lint = ["mise-settings", "gum-table"]
    • no normal CI aggregate codebase lint path found in the selected workflow/task corpus.
  • KnickKnackLabs/sessions@27dcc41
    • mise.toml: configured lint list exists.
    • .github/workflows/test.yml has a hard-coded per-rule loop ending in codebase "$target" "$PWD".
    • This covers the current list, but it can drift from [_.codebase].lint; the rule should probably mark this as unclear/warn and prefer aggregate lint.

Proposed lint behavior

Add or extend a codebase lint rule so that when a repo has [_.codebase].lint, CI must run the configured aggregate.

Recognized enforcement should include:

  1. A workflow command like:

    codebase lint "$PWD"
    mise exec -- codebase lint "$PWD"
  2. A workflow running a repo-local aggregate task that clearly invokes codebase aggregate lint, for example:

    run: mise run test

    where .mise/tasks/test invokes:

    codebase lint ...
    "$CODEBASE_BIN" lint

Hard-coded per-rule workflow loops should be warned as drift-prone unless there is a strong reason to accept them.

Possible --fix

If .github/workflows exists, --fix could add a standard step near tests:

- name: Run codebase lints
  run: codebase lint "$PWD"

If the workflow does not provision codebase, reuse the existing lint:github-actions --fix convention that adds the shiv plugin/tool setup when needed.

Acceptance criteria

  • Detect repos with [_.codebase].lint but no aggregate CI enforcement.
  • Pass repos with direct workflow aggregate lint.
  • Pass or deliberately document repos where a workflow runs an aggregate local task that invokes codebase lint.
  • Warn/fail on hard-coded per-rule loops that bypass the configured aggregate.
  • Add BATS fixtures for direct, indirect, missing, and hard-coded-loop cases.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions