Skip to content

fix: broken E2E harness baseline on master (kimi-code guidance test) #2098

Open

Description

@happy-v587

Description

The Validate harness and Compose definitions job (workflow: e2e-harness.yml) fails make harness-check on a clean checkout of current master (e4efb39b):

FAILED e2e/bub/tests/test_harbor_job_config.py::test_kimi_code_on_arm_installs_the_plugin_as_setup_does[authenticated]
FAILED e2e/bub/tests/test_harbor_job_config.py::test_kimi_code_on_arm_installs_the_plugin_as_setup_does[unauthenticated]

Assertion: _kimi_installation(harness) == _kimi_installation(tmp_path / "setup-home") — the harness-installed
plugins/managed/powercontext/skills/powercontext-project-context/SKILL.md does not match the one produced by
powercontext setup kimi-code (content diff inside the generated guidance file; 12 other mounted files are identical).

Why this matters

Every PR that touches anything matched by the harness workflow currently shows a red validate check even when the PR is unrelated, because the baseline is red. Recent master push runs:

Observed on PR #2093 as well; the PR itself does not touch kimi-code, guidance templates, or the harness.

Reproduction

git checkout e4efb39b
make harness-sync
uv run --project e2e/bub python -m pytest e2e/bub/tests/test_harbor_job_config.py -q
# 2 failed, 106 passed on macOS (arm, Python 3.12 via e2e/bub/.venv)

The failure content-diffs at
integrations/kimi-code/plugins/powercontext/skills/powercontext-project-context/.powercontext-guidance.json's pinned
legacy_sha256 of SKILL.md versus what the two install paths render, so either the rendered template or the pinned hash drifted. Note the same mismatch could be platform/locale-dependent if any template text is machine-dependent, which may explain why some environments pass.

Activity

  1. Fengzdadi commented on Oct 20, 2026

    @Fengzdadi
    Contributor

    Fixed on master by #2090 (a8383b4d): the harness installer now fills the __POWERCONTEXT_SCOPE_SCRIPT_JSON__ placeholder that #2043 added to the Kimi Code skill. e2e-harness.yml has passed on every master push since, and the reproduction above reports 108 passed on 0da4436d. This can be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions