Skip to content

fix: harden and reevaluate the 21 business skills - #66

Merged
criptogus merged 3 commits into
mainfrom
fix/business-skill-adversarial-hardening
Oct 23, 2026
Merged

criptogus merged 3 commits into
mainfrom
fix/business-skill-adversarial-hardening

Conversation

@criptogus

@criptogus criptogus commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

21 immutable 0.1.1 drafts → explicit privacy/authority/evidence rules
                         → domain deliverable requirements in shared builder
                         → new immutable 0.1.2 drafts → same 273-case evaluation

Hardens the shared business-skill prompts against credential/PII disclosure, source injection, unauthorized actions, fabricated execution and copied-input deliverables. Adds domain-specific reasoning requirements for all 21 skills and regenerates their source/mirror artifacts. The original prompts and failed evidence remain intact.

The comparison command rejects changed models/settings, cases, contexts, expectations, thresholds, incomplete cohorts and stale version bindings. Jev remains advisory; hash-bound primary review and the unchanged publication gates decide eligibility.

Evidence

  • Before: 273 real cases against version 0.1.1; eight case passes, zero eligible skills; 21 unpublished drafts.
  • After: completed all 273 real cases against immutable version 0.1.2, using the same model, weights, runtime, inputs, contexts, expectations, generation settings and criteria. Passed cases increased from 8 to 32, but all 21 skills still failed high/critical cases. Recorded 21 version/hash-checked evaluations and verified 21 drafts, zero published. No failure was rerun or replaced.
  • 217 Node tests passed, including regressions for changed thresholds/models/rubrics and omitted or unchanged subjects. Build, application/SDK types, strict lint, dependency audit, content, helper fixtures and mirrors passed. The versioned plugin manifest was regenerated; the complete release batch passed. The final comparison test addition also passed tests/lint and all CI checks. Anonymous access probes covered 92 tables without unexpected exposure.
  • Source import was dry-run with rollback before committing 21 immutable drafts. No source version, failed run or approval rule was overwritten. No Lovable agent/generation calls were made.
  • Includes chore: bump platform version 0.6.2 to 0.6.3 to prevent a redundant bot bump.

Merge Danger

Door: two-way for source changes; immutable registry versions and evaluation evidence remain retained.

Blast Radius: skills

The evaluated cached model has no tools. Text-only behavior cannot certify live execution safety or other models. Prompt hardening is not proof of compliance; failing versions must remain unpublished. No schema, lint rule or approval threshold is relaxed. Raw evidence stays in ignored private artifacts.

@criptogus
criptogus marked this pull request as ready for review October 23, 2026 18:10
@criptogus
criptogus merged commit 2844279 into main Oct 23, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant