Repository navigation
Release prep: E22's demo path, feedback text, walkthrough output and docs rows - #651
Open
LoopGlitch26 wants to merge 2 commits into
Open
LoopGlitch26 wants to merge 2 commits into
LoopGlitch26 wants to merge 2 commits into
Conversation
…docs rows - E22 took everything after "kept " in `tracekit demo --server --keep`'s output as the kept dir; since #600 the demo prints two more lines after it, so the nightly run failed. It reads the first line only. A test runs E22's demo step. - tracekit feedback's no-store message ended at "prints a replay's"; it says "counts" again, and the test pins the line. - limits.md: strict's E17 count is the committed results' (46 of 881, 5.2%), and test_eval_docs checks it. - The walkthrough's verdicts in sections 2, 6 and 8, and five of its terminal recordings, show what verify prints now, recorded in a throwaway HOME. - support.md lists Devin and Kilo; limits.md has a row for tracekit feedback reading only the dev signer's store; the field-report form no longer says 1.5.3 isn't on PyPI.
…run's verdict as verify prints it now
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the 1.6.0 release check found.
Corpus.demo()took everything afterkeptintracekit demo --server --keep's output as the kept dir. Since verify, last and demo: a short plain result by default, every check behind --detail (#592) #600 the demo prints a "verify the bundle again" line and a "see the run" line after it, soshutil.copygot a three-line path. It now reads only the first line.tests/test_examples_v2.py::DemoServer::test_e22_reads_the_kept_dir_from_its_line_aloneruns E22's demo step against the real demo. It fails without the fix (FileNotFoundErroron the three-line path).tracekit feedback: the no-store message ended at "prints a replay's". It says "counts" again, andtest_feedbackpins the whole phrase (it fails on the cut-off text).docs/limits.md, strict pack row: "in 1.5.3, 46 of E17's 849, 5.4%" is now the committed results' count, 46 of 881 (5.2%), as indocs/evaluation.md.test_eval_docsnow checks this row, so the number can't drift again.tracekit verify/tracekit lastprint today (VERIFIED: …, then NOT PROOF, NOT ANCHORED or ANCHORED). All of it was recorded in a throwaway HOME. A v1 bundle prints no anchoring line, so section 8 and recordings 06 and 10 have none. The page's intro says those verdicts were recorded again.docs/support.md: Devin joins the hooks tested against documented payloads. Kilo Code's plugin is listed as run with the Kilo CLI 7.8.18 on Linux (docs/coding-agents.mdsays so).docs/limits.md: new row:tracekit feedbackreads only the dev signer's store..github/ISSUE_TEMPLATE/field-report.yml: no longer says 1.5.3 isn't on PyPI. It sayspip install -U tracekit-ai.Not done: walkthrough section 4 and recordings 04-policy / 05-export-verify still show the old verdict. Their run includes a call a person approves, and the agent is not allowed to approve it (TK-D012), so recording them needs the owner.
Tests:
tests/test_docs.py,tests/test_eval_docs.py,tests/test_feedback.py,tests/test_examples_v2.py::DemoServerpass, andmake lintis clean. I did not run the full suite, E22 end to end, ormake test-ts.