Skip to content

feat: package SkillOS demo-paper evaluation flow#12

Merged
Huangdingcheng merged 206 commits into
mainfrom
codex/pr11-version-lab-20260526
Jun 1, 2026
Merged

feat: package SkillOS demo-paper evaluation flow#12
Huangdingcheng merged 206 commits into
mainfrom
codex/pr11-version-lab-20260526

Conversation

@dydpop

@dydpop dydpop commented May 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add a repository-level Windows quick start and one-click demo launcher path: install deps, run START_SKILLOS_DEMO.bat, enter local DeepSeek API URL/model/key, then SkillOS opens in the browser.
  • Add demo restore/readiness/evaluation scripts for the five-input SkillOS flow: trajectory, document, api_doc, script, and past_skills.
  • Integrate selected PR Feat/lbs repo version #11 Git/version ideas on the current branch: business-readable diffs, editable version fields, Version Lab, and re-verification when implementation/interface changes.
  • Improve the frontend graph into a more demo-friendly Nebula/Readable/Debug view with node/edge/label controls.
  • Add PR-ready docs for group operation, update summary, upload scope, and SkillsBench five-task analysis.

SkillsBench evidence

Selected 5-task P0 sparse subset:

  • Oracle: 5/5
  • No-skill baseline: 2/5
  • SkillOS generated skills: 3/5 using clearly labeled clean combined evidence

The clearest gain is dialogue-parser, from 0.667 to 1.0. The two clean failures are sales-pivot-analysis and software-dependency-audit, which now define the next repair-loop targets: exact Excel pivot metadata and exact security CSV normalization.

Verification

  • Secret-pattern scan over staged/source docs: no real sk-... key pattern matches.
  • PowerShell parse check for skillos-one-click-launcher/scripts/Start-SkillOSDemo.ps1: passed.
  • python -m compileall for updated backend/eval scripts: passed.
  • Targeted backend tests: 45 passed.
  • Frontend npm run lint: passed.
  • Frontend npm run build: passed, with only the existing Vite large chunk warning.
  • git diff --check: only LF/CRLF warnings on Windows.

PR scope notes

This PR intentionally includes reproducible code, tests, lightweight demo fixtures, scripts, and summary reports. It intentionally does not include raw sb-runs benchmark logs, Docker installers/images/cache, runtime logs, local launcher config, or API keys.

Limitations

  • The SkillOS generated SkillsBench 3/5 result is clean combined evidence, not one uninterrupted all5 generated-skill run.
  • Ctx2Skill support is a demo-paper Ctx2Skill-lite implementation, not full large-scale self-play.
  • The next research/engineering step is to use the two benchmark failures as Skill repair-loop demonstrations.

input_skill_eval_runner added 30 commits May 27, 2026 22:00
input_skill_eval_runner and others added 29 commits May 27, 2026 23:11
…sbench_06_cvss_score_extraction): snapshot v1.0.0
@Huangdingcheng
Huangdingcheng merged commit c48b7a7 into main Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants