You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
🚨 BREAKING CHANGE(sdk): move annotation-queue run calls to the platform items API - #815
This breaks langstar-sdk callers in three ways. get_run_from_annotation_queue and RunWithAnnotationQueueInfo no longer exist (use list_annotation_queue_items). add_runs_to_annotation_queue now returns Result<Vec<AnnotationQueueItem>> and delete_run_from_annotation_queue returns Result<bool>, where both returned Result<()>. And the queue types read snake_case JSON instead of camelCase.
LangSmith deprecated the queue run endpoints that langstar queue add-runs, remove-run and items call; those three commands now call the replacement items endpoints, and queue items gains a --status option.
sdk/src/client.rs: the annotation-queue run methods call /api/v1/platform/annotation-queues/{queue_id}/items (add, list, count, patch, placement, items/delete). add_runs_to_annotation_queue now returns the created items; delete_run_from_annotation_queue returns bool (false when the queue holds no item for the run). get_run_from_annotation_queue and RunWithAnnotationQueueInfo are gone, replaced by list_annotation_queue_items and the item types in sdk/src/annotation_queues.rs.
cli/src/commands/queue.rs: add-runs, remove-run and items keep their names and flags. items gains --status, lists by cursor instead of by index, and shows item ID, run ID, type and review section (the items API returns no run name or status). remove-run exits non-zero if the run is not in the queue. queue items --json now prints queue items (id, run_id, item_type, a status naming the review section) where it printed run objects with name and status, so a script that reads .run.name or .name must change.
Fixed on the way: the queue types had rename_all = "camelCase", but LangSmith sends snake_case, so queue create, get and list failed to decode. Requests now send snake_case too. test_queue_items_lifecycle in cli/tests/queue_command_test.rs caught the decode failure.
Docs: docs/queues.md, openspec/specs/queue/spec.md, and the annotation-queue endpoint fragment under reference/api-specs/langsmith/ (re-extracted from the spec already in the repo).
Tests
cli/tests/queue_command_test.rs: live lifecycle (create queue, add, list, count, placement, patch, remove, delete) asserting on returned items, plus parser tests.
sdk/tests/annotation_queue_items_test.rs: mocked request shapes, 100-item chunking, delete-by-run lookup, and paging (following cursors across pages, stopping at the limit, continuing past an empty page, stopping when the API repeats a cursor).
mise run check passes.
Upstream references
The Python SDK's generated client, annotation_queues/items.py at 47c553c, calls the same endpoints. Its hand-written add_runs_to_annotation_queue calls /runs/by-key when given RunKeys (a run's ID, project ID and start time together) and the deprecated /runs when given plain run IDs. Langstar's add_runs_to_annotation_queue sends only the run ID; a caller who has the project ID and start time can use add_items_to_annotation_queue.
The Go SDK's AnnotationQueueItemService has methods for every items endpoint. Langstar has all of them except NewStatus (items/{id}/status), which no CLI command needs.
langsmith-cli, langsmith-mcp-server: not checked beyond a grep for the platform endpoints; they have no annotation-queue item commands.
Decisions for codekiln
I marked the PR a breaking change (major bump, per docs/dev/git-scm-conventions.md) because it removes public SDK items, changes two return types and changes how the queue types decode. Alternative: keep deprecated shims for one release and ship it as a fix.
queue items drops the run's Name and its success or error Status, because the items API returns only the run ID; the Status column now shows the review section. Alternative: look each run up with runs query (one more request per page).
remove-run lists every review section to find the queue item for a run ID, since the new endpoint deletes by item ID.
add-runs sends only the run ID, so right after a run is ingested LangSmith answers 404 until it can find the run. The CI examples in docs/queues.md add runs from a query, so they are unaffected; a script that adds a run it just created must retry. Alternative: take --project-id and --start-time so LangSmith can locate the run directly.
Add, list, count, patch, placement and delete now call
/api/v1/platform/annotation-queues/{queue_id}/items, replacing the deprecated
/runs* and /run/{index} endpoints. The queue types now use the snake_case
field names the API sends; the camelCase renames made create, get and list
fail to decode.
#765 746.4-aq-items Move annotation-queue run calls to the platform items API
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
codekiln
added
enhancement
New feature or request
wip
Not ready yet: an open PR short of the ready bar, or an issue an open PR closes
labels
Oct 13, 2026
Share the cursor traversal between delete-by-run and `queue items`, cover it
with mocked page tests, keep source_proposed_example_id on returned items, and
put the review section in `queue items --json`.
#765 746.4-aq-items Move annotation-queue run calls to the platform items API
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Count endpoint status enum omits the documented all value
sdk/src/client.rs:1086
The count endpoint accepts the valid status=all bucket (as documented in the checked-in OpenAPI fragment), but this parameter uses the list-status enum, which can represent only the three review sections. SDK callers therefore cannot request the queue's total item count. Please introduce a count-specific status type (or a dedicated count-all API) that includes all without making all available to the list endpoint, which rejects it.
The reason will be displayed to describe this comment to others. Learn more.
from AI(claude-code, claude-sonnet-5-5, langstar i765-aq-items)
Two findings on the PR body, two on the code, and one suggested squash subject.
1. The Upstream references bullets name sources with no link and one claim with no source. Quoted: "langsmith-sdk (Python): _openapi_client/resources/annotation_queues/items.py calls the same endpoints. Its hand-written add_runs_to_annotation_queue still uses the deprecated /runs and /runs/by-key, and prefers a RunKey (run_id, session_id, start_time)." and "langsmith-go: AnnotationQueues.Items has New, Update, List, NewStatus, DeleteAll, GetCount, GetPlacement." Each sends the reader to a repository to find a file, with no URL and no commit. I checked the langsmith-go claim in the local clone (annotationqueueitem.go has exactly those methods on the platform paths). There is no local langsmith-sdk clone, so I could not check the Python claim about /runs/by-key and RunKey; the PR should link the file at a commit so the reader can. Rule: My/AI/Rule/How to Communicate Effectively With Me/A pointer carries a link, an id with a slug, and a reason. Rewrite for the two bullets:
The Python SDK's generated client, annotation_queues/items.py at , calls the same endpoints. Its hand-written add_runs_to_annotation_queue still calls the deprecated /runs and /runs/by-key endpoints and sends a run's ID, project ID and start time together. Langstar's add_runs_to_annotation_queue sends only the run ID; a caller who has the project ID and start time can use add_items_to_annotation_queue.
The Go SDK's AnnotationQueueItemService has methods for every items endpoint. Langstar has all of them except NewStatus (items/{id}/status), which no CLI command needs.
2. The body opens with a list of files. The first section is "What changed" and its first bullet is sdk/src/client.rs followed by an endpoint path and six parenthesised names. A reader learns which files changed before learning what a person can now do. Rule: My/AI/Rule/How to Communicate Effectively With Me/Get to the point. Rewrite for the opening sentence, placed above the file bullets: "LangSmith deprecated the queue run endpoints that langstar queue add-runs, remove-run and items call; those three commands now call the replacement items endpoints, and queue items gains a --status option." The third bullet under "Decisions for codekiln", "Run-only add-runs fails with 404 until the run is queryable", is a fact rather than a decision and "Run-only" is unexplained; say "add-runs sends only the run ID, so LangSmith answers 404 until it can find the run; the live test waits for that." Rule: My/Pref/Writing/Be specific and explicit.
3 and 4 are the inline comments on FRAGMENTS.md (schemas fragment not re-extracted, a factual gap) and on sdk/src/client.rs (one wasted request per section).
Suggested squash-merge subject: keep the title as it stands, "🩹 fix(sdk): move annotation-queue run calls to the platform items API".
Facts checked and found accurate: the snake_case fix against AnnotationQueueSchema, the status requirement and page_size default of 20 and maximum of 100 on the list endpoint, the 100-item limit on add, and the item_ids body on items/delete.
from AI(claude-code, claude-sonnet-5-5, langstar i765-aq-items)
Edited the PR body for findings 1 and 2.
Opening of "What changed", added above the file bullets: "LangSmith deprecated the queue run endpoints that langstar queue add-runs, remove-run and items call; those three commands now call the replacement items endpoints, and queue items gains a --status option."
Python bullet, now linked at langsmith-sdk commit 47c553c: "calls /runs/by-key when given RunKeys (a run's ID, project ID and start time together) and the deprecated /runs when given plain run IDs." I checked this in client.py at that commit, and it differs a little from the suggested wording: the /runs path is the one marked deprecated, and /runs/by-key is what RunKeys use.
Go bullet, now linked at langsmith-go commit 59bb42c: "has methods for every items endpoint. Langstar has all of them except NewStatus (items/{id}/status), which no CLI command needs."
Third decision bullet: "add-runs sends only the run ID, so LangSmith answers 404 until it can find the run; the live test waits for that."
The reason will be displayed to describe this comment to others. Learn more.
from AI(claude-code, claude-sonnet-5-5, langstar i765-aq-items)
Round 3. The cursor loop, the repeated-cursor guard and the new test are correct; the sdk test file passes (9 of 9). Two factual findings, both about text that claims more than the repo shows.
Body, Tests section. The line sdk/tests/annotation_queue_items_test.rs: mocked request shapes, 100-item chunking, delete-by-run lookup. predates d7fa533 and 9cb530d and leaves out the paging tests (cursor following, the limit across pages, an empty page, a repeated cursor). It costs the reader a trip into the test file to learn that paging is covered. Rule: Be specific and explicit. Rewrite: sdk/tests/annotation_queue_items_test.rs: mocked request shapes, 100-item chunking, delete-by-run lookup, and paging (following cursors across pages, stopping at the limit, continuing past an empty page, stopping when the API repeats a cursor).
…nverified API fact
#765 746.4-aq-items Move annotation-queue run calls to the platform items API
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
from AI(claude-code, claude-sonnet-5-5, langstar i765-aq-items)
Updated the Tests bullet in the PR body. It now reads: "sdk/tests/annotation_queue_items_test.rs: mocked request shapes, 100-item chunking, delete-by-run lookup, and paging (following cursors across pages, stopping at the limit, continuing past an empty page, stopping when the API repeats a cursor)."
…pts and assistants (#816)
* 📚 docs(openspec): propose export, import, compare and verify for prompts and assistants
Add the OpenSpec change export-import-prompts-and-assistants: a new prompt
capability and assistant additions for writing a prompt commit or an
assistant to a file, recreating it in another workspace or deployment,
comparing a copy with the file, and checking that the copy can run.
The task list splits the work into one pull request per group.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): fix the claims and headings codekiln-review flagged on the transfer design
Add the SDK fields and the deployment URL fix the assistant commands
need (tasks 2.3 and 3.3), check a prompt tag through the commits
endpoint, match the model provider by class name and report a missing
saved model configuration as a note (Open Question 7), and turn topic
headings into decisions.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): carry the commit description, repo details and metadata removals through import
Export now writes the commit description, and import sends it on each
commit it creates. With --update, prompt import also sets the destination
repo's description, readme, tags and visibility through
PATCH /api/v1/repos/{owner}/{repo}, and a repo-only difference updates the
repo without adding a commit. Assistant import sends each metadata key the
destination has and the file lacks with a null value, because the Agent
Server merges metadata on update, and reports any key that remains. Task
3.3 adds graph_id to UpdateAssistantRequest, task 3.4 adds the commit
description and repo update to the SDK, and task 4.1 tests sorted keys
with values built in different insertion orders.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): add tag moves and the change warning to the tasks, and order assistant compare before its record
Task 3.2 now moves a tag with PATCH /api/v1/repos/{owner}/{repo}/tags/{tag_name}, task 1.1 extracts that endpoint and the repo update, task 7.2 tests the warning that the destination changed since the last import, and tasks 9.1 and 9.2 add --assistant-id so assistant compare and verify can name an assistant before import exists. The manifest example uses the PromptPlayground id the upstream fixture holds, and design.md links tasks.md relatively.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): resolve an assistant's deployment through the deployment's url
The assistant spec said a deployment without source_config.custom_url
fails, which task 2.3 changes. Add a MODIFIED requirement that sends
assistant requests to the control plane's top-level url, falling back to
custom_url, and align task 2.3, the design risk and the proposal.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): run verify's checks before import writes and fix the SDK fields tags and deployments need
Add the deployment url to the SDK Deployment and the commit id that the
tag endpoints need, add --workspace-id to assistant verify, test the
assistant dry run, scope the round-trip goal to the fields the file
carries, clear an assistant's description and context on update or
report them, and run verify's checks before prompt and assistant import
write anything, with Open Question 8 on that choice.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): build the destination deployment from its own graph config and move commit tags on import
Swap tasks 2.2 and 2.3 so the destination deployment fixture comes after
the deployment URL it needs, and build that deployment from its own
langgraph.json so the fixture never returns the source deployment. Treat a
commit tag that is missing or points elsewhere as a difference that
--update moves. Lead the assistant update paragraph with what the Agent
Server keeps. Test that prompt import without --workspace-id fails while
LANGSMITH_WORKSPACE_ID is set.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): find tag commits through the commit list and take the model class from the manifest
Plan an SDK call for the commit list so the tag move can name a commit
it did not create, take requires.model_provider from the chat model
constructor's id, add tests for --commit prod and --assistant-id, and
reword two design sentences.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): exit with 3 for a difference, leave clap's 2 for usage errors, and drop unused create fields
clap exits with 2 on a usage error, so compare, verify and import now
exit with 3 for a difference or a failed check and 1 for any other
error. prompt import checks --workspace-id itself and exits with 1.
Task 3.3 drops assistant_id and if_exists, task 1.1 extracts the commit
list, and tasks 5.1 and 8.1 test a missing prompt and assistant.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): point the end-to-end test at the destination workspace, test requires, and let group 2 go ahead without task 2.1
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): run only verify's dependency checks before import and let update fall back to a reported difference
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): say import asks the Agent Server to remove what the file lacks
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): test the JSON report in compare and verify, and name what import checks before it writes
Task 4.3 tests only the report and its exit statuses, since group 4 adds
no command that can find a difference. Tasks 6.1 and 9.2 now test the
`-f json` report on stdout, progress on stderr and clap's status 2.
design.md names the checks import runs, links open questions 7 and 8 by
heading and links #807 by title.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): pass the model provider key to pull_prompt as a secrets map
The pinned LangSmith Python SDK's pull_prompt reads no secret from the
environment by default and raises when a manifest holds a secret
reference and no secrets map arrives, so the structured output script
passes the key it reads from the environment as a secrets map.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): record a new assistant's ID before reading it back
An assistant's record is the only link from the file to its copy, so
assistant import writes the new ID before the read-back comparison. An
import whose read-back finds a difference then leaves the link, and the
next import finds that assistant instead of creating a duplicate.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): reject an export file of another kind or format version
Import, compare and verify read only a file whose format_version is 1
and whose kind names their own resource, and exit with status 1 naming
the field otherwise. Task 4.1 adds the check with unit tests, and task
6.1 adds CLI tests for both cases.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): record the prompt commit before reading it back, and test both read-back failures
Prompt import now records the commit hash before the read-back, as assistant import does, so the next import does not report langstar's own commit as someone else's change. Task 7.2 adds an httpmock test for a field LangSmith drops, task 10.1 a CLI test for a prompt export given to assistant import, and design.md splits the verify paragraph into the secret check and the model configuration note.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): test an assistant file given to prompt import with import, and name the format-version requirement
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): give the destination record its prompt purpose, make developers the actors of task sentences, and drop the #728 field claim
The proposal now says the record finds an assistant's copy and tells whether someone pushed to a prompt. Sentences that had a task push or record now name a developer working that task. The #728 paragraph says export and import share PromptClient calls instead of promising prompt get --all shows an export file's fields. The assistant spec's removal requirement heading names import and the Agent Server.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): mark the SDK field additions as breaking, compare every manifest key, and check prompt references against the file
Adding a public field to Deployment, CommitData, CommitRequest, Assistant or the assistant request structs breaks a struct literal in another crate, so those tasks update the literals in this repository and their pull requests carry a BREAKING CHANGE footer. Compare drops no key by name inside the repo, commit and assistant objects, so a manifest's id arrays count. Assistant compare, verify and import resolve each requires.prompts pointer against the file's assistant object and stop with status 1 on a mismatch.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): ask codekiln whether the SDK field additions are a breaking change
Add Open Question 9 with the recommendation to ship the SDK field additions
as feat changes, and point tasks.md, the design's risk entry and the
proposal at it.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): weigh non_exhaustive and #815's choice in the SDK breaking-change question
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): list CommitManifestResponse among the changed SDK structs and give each rewritten sentence an actor
Adds CommitManifestResponse, which PromptClient::get_commit returns, to the structs that question 9 and tasks.md group 3 name, and has task 3.1 extend it. Names the #815 link by its title, drops "a third option", links question 9 by its text, and limits prompt compare to the file's repo and commit objects.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): have prompt import and verify check requires against the manifest
Someone can edit an export file's manifest in Git and leave requires.secrets as it was, so import and verify work out both requires lists again from the manifest and stop with status 1 when they differ. Adds a scenario and a verify test for it, and names CommitManifestResponse in the proposal's impact list.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): merge the destination graph config to main before its fixture and test import's requires check
Task 2.3 now adds the destination test graph's langgraph.json in a pull
request of its own, because the control plane builds test deployments
from main; the fixture moves to 2.4 with a name prefix of its own.
Task 7.1 tests import against an edited manifest. Question 9's context
leads with its argument, the SDK risk names its structs, and the compare
paragraph says what compare lists.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): say that non_exhaustive breaks langstar's own CLI, order task 2.3's pull request before group 2, and shorten question 5's context
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): name the destination deployment's integration-test prefix and its cleanup, list Assistant among the CLI's struct literals, and rename the workspace-id requirement
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): register the destination test graph under a second graph ID for the assistant update tests, and list group 3's SDK structs by name
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): merge task 2.2 before task 2.3 so its check lists assistants through langstar, and cite find_reusable_deployment and the 409 for the destination deployment
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): state the test-environment merge order once in the tasks preamble, and trim it from task 2.3, task 3.3 and question 5
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): name the 409 risk behind the destination deployment, test --prompt-ref export, and handle prompts without a model
Design question 5 and Risks / Trade-offs now say the control plane may refuse a second deployment from this repository with 409, and that task 2.3 finds out and sends question 5 back to codekiln. Task 8.1 tests that export writes the --prompt-ref pointer and prompt name. A prompt without a model exports requires.model_provider as null, and import and verify skip the saved model configuration check for it.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): rename requires.model_provider to requires.model_class, untie two design sentences, and specify export to stdout, a missing copy at compare, and tag delete
Rename the export file's requires.model_provider to requires.model_class,
so it no longer shares a name with LangSmith's commit.model_provider.
Rewrite the design's two requires-check sentences so the action comes
first. Add scenarios that compare exits with status 1 when the prompt or
the recorded assistant is missing, require export without --output to
print only the JSON document on stdout, and add a tag delete call to
task 3.2 for the end-to-end test in task 10.3.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): make Deployment.url optional, and add the repo's readme and tags to Prompt
The control-plane spec declares Deployment.url nullable, so task 2.2
adds it as Option<String> with serde default and tests a null and a
missing url. PromptClient::get returns Prompt, which has no readme or
repo tags, so task 3.4 adds them and reads them back through get, and
question 9 and the proposal list Prompt among the structs that gain
fields.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): list Prompt among group 3's SDK structs, and limit identical exports to an unchanged repo and tags
Task 3.4 adds readme and tags to Prompt, which test_prompt_serialization
builds with a struct literal, so group 3's preamble lists Prompt. The
repo details and the tags that point at a commit change without a new
commit, so the identical-export requirement now holds only while they
stay unchanged.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): name exhaustive destructuring alongside struct literals as what a new SDK field breaks
A new public field also breaks a crate that destructures the struct
with a pattern naming every field. The proposal, the design's risk and
question 9's context, and group 3's preamble now say so; question 9's
recommendation is unchanged and stays with codekiln.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): give the 409 check a developer as subject, split task 5.1 into its tests, and drop the import-skips-comparison negation
Name a developer as who deploys in task 2.3, read the commit description through the commit read from task 3.1, fix the print_message permalink line, and untangle "calls that list".
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): record a new assistant's ID as pending before import creates it with if_exists do_nothing
Import chooses the assistant ID, writes it to the destination record as pending, and creates the assistant with that assistant_id and if_exists set to do_nothing, so an import that stops between the create and the record write leaves no second assistant on retry. Task 3.3 adds both fields to CreateAssistantRequest and task 10.1 tests the retry.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): sort a commit's tags in the export file, give the structured output check its prompts, and name which record write import can lose
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): resume a prompt import that stopped before its first commit, delete destination tags the file lacks on --update, and split the task test lists
Split the destination record paragraph in design.md into three, list each
task's tests as sub-bullets, complete the OpenAI key setup in design.md, and
gate group 3's SDK integration tests on the integration-tests feature rather
than a bare #[ignore].
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): leave model_config and model_provider out of compare, cite what model_provider holds, and split tasks 2.2, 4.2, 6.2 and 9.2 into lists
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): replace the destination record by rename, drop source.ref from the export example, and warn that assistant export copies credentials kept in configuration
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): lead the compare sentence with what compare checks, move the model_provider citation to export, require the prompt record rename, and spell out the assistant export docs advice
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): record a matching commit an interrupted import pushed, ask question 1 about status 3, and give the 6.2 and 9.2 tests their assertions
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): promise only that prompt export writes no value of a referenced secret, and warn in the prompt export docs about credentials written as literal settings
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): refuse to export a prompt whose model holds a literal API key
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): say export refuses a literal API key in task 5.2, test that refusal, and narrow what an interrupted import finishes
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): untangle the destination-changed warning, match saved model configurations on the model class, and fill source.workspace_id from the repo's tenant_id
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): give the deployment url fix a Decision, fold the model_config paragraph into compare, untangle the assistant credential mitigation, and list the fields upstream update patches
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): test mocked commands as unit tests through mockito and with_base_urls, rerun the stale prompt endpoint extract, split the group 3 introduction, and list tenant_id among the new Prompt fields
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): test that prompt verify and assistant verify report a changed field as compare does
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): point the langgraph get docstring link at the created_by line
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): install the structured output check's Python packages through uv, name the missing-URL error in task 2.2, and split the SDK impact and first goal
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): drop the deployment url change, since GitHub deployments already carry custom_url, and split task 1.2
The control-plane OpenAPI spec says SourceConfig.custom_url is set automatically
for github source deployments, and test_assistant_description_field already
passes against the GitHub-built test deployment. Task 2.2 now only re-enables
test_assistant_create_basic. Also fix the question 3 context about the CI job's
mise install, and split task 1.2 into sub-bullets.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): give assistant compare and verify --record, sort repo tags and secrets in the export, and test JSON Pointer escaping
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): lead the structured output check with its reason, name every sorted list in the export, and drop the unsourced claim about how users run prompts
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): open the assistant Context paragraph with the SDK gap the export needs
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): test compare --commit against an older tag and --update creating missing commit tags
Copilot listed both as previously missed. Update the design's task 7.3
and 8.2 links for the two added lines.
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): redact a literal API key in compare output, and point the structured output check at the destination workspace
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): redact API keys in the shared report task, say compare redacts them, and name the destination workspace in the structured output check
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): redact API keys inside any printed difference value, name both resource types in the shared report task, and say fnox.toml also sets the source workspace
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): redact any string a key ending in api_key holds, print secret references as they are, and redact assistant compare output too
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): refuse to export an assistant holding a literal API key, and match the api_key suffix without regard to case
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): redact only a string value under an api_key key, print an assistant's secret reference unchanged, and test the context branch of the assistant export refusal
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): say what `<redacted>` replaces, lead the assistant secret reference requirement with its SHALL, and justify printing a secret reference by what it holds
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): lead the prompt secret reference requirement with its SHALL, as the assistant one does
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): state the redaction case match and its reach as SHALLs in both specs
#807 Export, import, compare, and verify prompts and assistants across LangSmith environments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): refuse an export that holds any credential, found by key name or value shape, and redact the same strings in reports
Export now checks every string it would write, not only settings ending
in api_key: key names with secret, token, password, credential,
authorization or cookie, and values shaped like bearer tokens, private
keys, URLs and connection strings with passwords, JSON Web Tokens and
provider API keys. It refuses with status 1 instead of redacting, and
--not-a-secret lets one reviewed string through.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): ship the SDK field additions as feat commits, as the release docs and #726 did, and drop open question 9
docs/dev/procedures.md counts changed signatures and removed fields as
breaking, not added ones; docs/dev/git-scm-conventions.md and
docs/dev/ci-cd.md give feat a minor bump; the Prepare Release workflow
picks the version from commit types; release.toml keeps langstar-sdk off
crates.io.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): say plainly what the credential check flags and when export refuses
Lead the design paragraph with what export does, split the
"never redacts and writes the rest" sentence that read two ways, state
the --not-a-secret rules without stacked negatives, and rename the
requirements and scenarios whose subjects could not act.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): name who commits the SDK fields and link the release docs the design cites
Retitle the release decision so it says who acts and what release
follows, replace the group 3 ordinal with a linked task pointer, drop
"ship", and turn each cited release-doc line into a link pinned to main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): open the SDK release decision with what developers commit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): flag plural key names, redact the prompt reference error and support one import at a time
Match `keys` as well as `key` after the credential prefixes so
`api_keys` is flagged as task 4.4 tests, print `<redacted>` for a
flagged value in the prompt reference mismatch error, and say that
langstar supports one import at a time per deployment, with #828 for
making concurrent imports safe. Also retitle the verify decision and
give the ci-cd.md sentence a subject that can act.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): name what one-import-at-a-time protects and untangle the credential sentences
Say that langstar supports one import of an export file at a time, what
two concurrent imports do to the record and to a first assistant import,
and link #828. Split the redaction sentence whose object came last, use
"lets through" for the check and "names" for the user, give the sk-
scenarios a token long enough to flag, drop the "two more signs" count,
and stop the proposal's link text from repeating its sentence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): define prompt-reference syntax, unknown-graph handling and assistant import without --update
Copilot listed these as previously missed findings on #816:
- Prompt names under requires.prompts take [<owner>/]<handle>[:<tag>]; no owner means the current workspace, no tag means latest.
- Verify counts a 422 naming graph_id from the assistant search as a missing graph; task 1.2 records what the live server returns.
- Assistant import creates an assistant only on a first import, and --assistant-id without --update compares and changes nothing.
- A separate requirement covers --record for assistant import, compare and verify.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): report import's action in JSON, settle a resumed import's repo differences and pin the structured output check
Copilot listed these as previously missed findings on #816:
- prompt import and assistant import JSON reports carry action and the destination's commit hash or assistant ID.
- A resumed first import pushes the commit and tags, leaves differing repo details alone and exits with status 3 without --update.
- The structured output script pins its dependencies and validates with jsonschema.
- Task 1.2 waits on the destination workspace and key from task 2.1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📝 docs(openspec): decide six #807 design questions and leave one for codekiln
Compare, verify and import exit with 3 on a difference; import records
destination IDs beside the export; --prompt-ref names prompt references;
prompt import requires --workspace-id; a missing saved model
configuration is a note; a failed dependency check stops import first.
The structured output check skips without an OpenAI key, and the tests
import into the existing workspace and deployment until codekiln answers
the one remaining question.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): run the tests in the existing workspace and deployment and drop the last open question
The structured output check skips without an OpenAI key, and the tests
import into the single existing workspace and test deployment. Trying
export and import on real assistants and prompts before implementation
settles whether a second workspace and deployment are needed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): run the structured output check from a committed uv script lockfile
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): record the export and compare trial in the transfer design
Decision 8 now states the trial's result: one workspace and one test
deployment are enough, and a local langgraph dev server can serve as an
import target. Add scenarios and tasks for refusing a graph's default
assistant, the -/ owner in prompt names and exporting a private prompt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): run the transfer tests in one workspace until a second one exists
Copilot pointed out that the tasks called the second workspace and
deployment optional while task 3.3 needed the destination fixture from
2.4, and task 7.3 read LANGSTAR_TEST_DEST_WORKSPACE_ID, which only 2.1
sets. Until those tasks merge, the destination workspace and deployment
are the existing test ones. The graph moves in 3.3 and 10.2 and the
destination-only prompt in 9.2 carry an ignore naming the task that adds
their environment, and the structured output script takes the workspace
ID the import test used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): drop the local langgraph dev server as an assistant import target
Copilot pointed out that --deployment resolves a name or ID only through
the control plane's deployment list (resolve_deployment_url in
cli/src/deployment_utils.rs), so the proposed commands cannot reach a
local langgraph dev server. Nothing else in the change depends on one, so
the design drops the claim and lists a server the control plane does not
list as a non-goal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): require a second workspace and deployment before the change is archived
#807 asks the assistant tests to create an assistant on a separate
deployment and to exercise an assistant whose prompts were imported into a
separate workspace. The design made tasks 2.1, 2.3 and 2.4 optional, so the
change could be archived with those tests still ignored.
Developers still build and run everything in one workspace and deployment.
Tasks 2.1, 2.3 and 2.4 now merge before the archive, each removes the
`#[ignore]` from the tests that waited for it, and the archiving pull
request links a CI run in which they passed. A 409 from the control plane
in 2.3 goes to codekiln as a decision.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* 📚 docs(openspec): give the test-environment decision subjects that can act
The pr-language-review skill found documents and tasks doing the verbs:
"the design made tasks optional", "those tasks land", "they merge", "a 409
goes to codekiln". Developers merge the tasks, the control plane answers
with a 409, and codekiln decides.
Also drops the coined "unignore" from the Workflow follow-up and splits the
409 clause out of the sentence it was knotted into.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* 📚 docs(openspec): name the tasks the archive waits on, and shorten the heading
The decision said "tasks 2.1, 2.3 and 2.4" with no link and no title, which
costs a reader a trip into tasks.md to learn what they add. Each one now
carries the link and the task title the rest of design.md uses.
The heading was a sentence long enough to read twice. It now says what
developers do: add the second workspace and deployment before they archive
the change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* 📚 docs(openspec): rewrite the verify and import-check sections for a reader who has not read the LangSmith API
codekiln read three paragraphs and could not tell what they were saying.
The verify section now opens with the question a user has after an import —
will the copy run here — and then says what verify asks the destination and
what it prints. Each API call arrives after the sentence that says why
langstar makes it. The assistant graph check explains that every graph a
deployment serves has a default assistant, so an empty search answer means
the deployment does not serve the graph; the 422 paragraph says the server
may refuse an unknown graph ID instead of answering with nothing.
The import section now says what import does — run verify's checks before
it creates anything — and gives the cleanup it saves afterwards.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* 📚 docs(openspec): send an assistant's settings in one field on import
A newer Agent Server refuses to create an assistant that sets both
config.configurable and context, while every read returns both. Import
now sends configurable alone when the two match, the one present when
only one is, and stops with status 1 when they differ.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 📚 docs(openspec): rewrite the rest of design.md for the faults codekiln marked
Same sweep as the verify and import-check sections, across the remaining
decisions.
The destination-record decision opened with the reason for a separate file
instead of saying what import writes. The exit-status decision opened with a
sentence that had to be read twice and ended in "neither success nor a
broken run". The prompt-reference decision said the same thing in its first
two sentences. The one-field heading read "in one field, never in both".
The compare decision and the structured-output script were each one
paragraph long enough to lose a reader; both are split by what they cover.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* 📚 docs(openspec): compare an assistant's settings from whichever field holds them
A file that holds settings in config.configurable alone imported cleanly
and then failed its own read-back: the Agent Server fills context from
configurable, so the comparison found a context the file lacked and import
exited with status 3. The same happened the other way round for a file that
holds only context.
Compare and the read-back now take the file's settings from whichever field
the file holds and the destination's from whichever the server returned,
and a field one side lacks is no difference.
--update no longer sends an empty context beside a config.configurable.
That would be refused by a newer server, or would wipe out the settings the
server had just filled in. Import clears context only for a file that holds
settings in neither field.
New requirements and scenarios in specs/assistant for both rules, and tests
in tasks 10.1 and 10.2 for a config-only file's round trip and for an
--update that leaves the field it did not send alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* 📚 docs(openspec): flag a short Basic credential, and read the settings alias in the shared comparison
Copilot found two holes.
The credential check asked for at least 16 base64 characters after
`Basic `, and `Basic dXNlcjpwYXNz` carries twelve and decodes to
`user:pass`. Under a key the name rule ignores, export would have written
it to Git. The check now decodes the payload and flags it when the result
is printable text with a colon in it, however short. `Basic instructions
follow.` still passes, because its payload decodes to nothing readable.
The settings alias lived in the assistant spec, while task 4.2's shared
comparison still compared every key under `assistant`, so a one-field file
would have been reported as a difference after all. Task 4.2 now reads
`config.configurable` and `context` as one value and compares every other
key under `config` as it stands, with tests for both single-field files and
for a changed `config.recursion_limit`.
Also a repeated-import test for each single-field file, and the matching
scenario in specs/assistant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* 📚 docs(openspec): keep a private prompt out of a public destination repo
Copilot found an exposure in the recovery path. When an earlier import left
a repo with no commit and that repo is public, while the file says the
prompt is private, import pushed the manifest and reported the visibility
difference afterwards. By then the prompt's text was public, and no report
takes that back.
Import now pushes nothing in that case without --update, reports
/repo/is_public and exits with status 3. With --update it makes the repo
private before it pushes. The same holds for a repo someone else made.
New requirement and scenarios in specs/prompt, and tests in tasks 7.1
and 7.2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed to evaluate the untested successful CLI --status path, since SDK tests do not cover the CLI-to-SDK mapping or returned JSON statuses.
…ked LangSmith
Copilot noted that the CLI tests check help and a rejected --status value
but never run a valid one. The listing and JSON output of `queue items`
move into collect_items and items_json, and mocked tests run them with
needs-my-review, needs-others-review and archived, asserting the exact
item IDs and statuses printed and that only the named section is listed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🟢 Approval recommended
No active review issues remain for the migration to LangSmith’s platform items API.
No open issues 🎉
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
enhancementNew feature or requestpriority: lowReview last: design docs, or waits on a decision from codekilnwipNot ready yet: an open PR short of the ready bar, or an issue an open PR closes
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
codekiln
added
enhancement
codekiln
left a comment
Claude
from AI(claude-code, claude-sonnet-5-5, langstar i765-aq-items)
Fixes #765
What changed
This breaks
langstar-sdkcallers in three ways.get_run_from_annotation_queueandRunWithAnnotationQueueInfono longer exist (uselist_annotation_queue_items).add_runs_to_annotation_queuenow returnsResult<Vec<AnnotationQueueItem>>anddelete_run_from_annotation_queuereturnsResult<bool>, where both returnedResult<()>. And the queue types read snake_case JSON instead of camelCase.LangSmith deprecated the queue run endpoints that
langstar queue add-runs,remove-runanditemscall; those three commands now call the replacement items endpoints, andqueue itemsgains a--statusoption.sdk/src/client.rs: the annotation-queue run methods call/api/v1/platform/annotation-queues/{queue_id}/items(add, list, count, patch, placement,items/delete).add_runs_to_annotation_queuenow returns the created items;delete_run_from_annotation_queuereturnsbool(false when the queue holds no item for the run).get_run_from_annotation_queueandRunWithAnnotationQueueInfoare gone, replaced bylist_annotation_queue_itemsand the item types insdk/src/annotation_queues.rs.cli/src/commands/queue.rs:add-runs,remove-runanditemskeep their names and flags.itemsgains--status, lists by cursor instead of by index, and shows item ID, run ID, type and review section (the items API returns no run name or status).remove-runexits non-zero if the run is not in the queue.queue items --jsonnow prints queue items (id,run_id,item_type, astatusnaming the review section) where it printed run objects withnameandstatus, so a script that reads.run.nameor.namemust change.rename_all = "camelCase", but LangSmith sends snake_case, soqueue create,getandlistfailed to decode. Requests now send snake_case too.test_queue_items_lifecycleincli/tests/queue_command_test.rscaught the decode failure.docs/queues.md,openspec/specs/queue/spec.md, and the annotation-queue endpoint fragment underreference/api-specs/langsmith/(re-extracted from the spec already in the repo).Tests
cli/tests/queue_command_test.rs: live lifecycle (create queue, add, list, count, placement, patch, remove, delete) asserting on returned items, plus parser tests.sdk/tests/annotation_queue_items_test.rs: mocked request shapes, 100-item chunking, delete-by-run lookup, and paging (following cursors across pages, stopping at the limit, continuing past an empty page, stopping when the API repeats a cursor).mise run checkpasses.Upstream references
add_runs_to_annotation_queuecalls/runs/by-keywhen givenRunKeys (a run's ID, project ID and start time together) and the deprecated/runswhen given plain run IDs. Langstar'sadd_runs_to_annotation_queuesends only the run ID; a caller who has the project ID and start time can useadd_items_to_annotation_queue.NewStatus(items/{id}/status), which no CLI command needs.Decisions for codekiln
docs/dev/git-scm-conventions.md) because it removes public SDK items, changes two return types and changes how the queue types decode. Alternative: keep deprecated shims for one release and ship it as a fix.queue itemsdrops the run's Name and its success or error Status, because the items API returns only the run ID; the Status column now shows the review section. Alternative: look each run up withruns query(one more request per page).remove-runlists every review section to find the queue item for a run ID, since the new endpoint deletes by item ID.add-runssends only the run ID, so right after a run is ingested LangSmith answers 404 until it can find the run. The CI examples indocs/queues.mdadd runs from a query, so they are unaffected; a script that adds a run it just created must retry. Alternative: take--project-idand--start-timeso LangSmith can locate the run directly.🤖 Generated with Claude Code