Skip to content

ci: verify upstream benchmark PR 115 - #1

Closed
maho0638 wants to merge 16 commits into
mainfrom
diag/validate-pr115-20261012
Closed

maho0638 wants to merge 16 commits into
mainfrom
diag/validate-pr115-20261012

Conversation

@maho0638

Copy link
Copy Markdown
Owner

Temporary fork-only CI validation of upstream PR genlayerlabs#115 source 459a3e2 plus diagnostic workflow. Not for merge. Close after validation.

MuncleUscles and others added 16 commits June 11, 2026 17:09
…suggestions (genlayerlabs#94)

Opt-in pytest options --fee-profile=PATH and --fee-profile-headroom (default
1.25). Every deploy/write executed through gltest records the consumed fee
data from Studio receipts (fees.consumed.executionConsumed /
messageFeesConsumed); at session end PATH is written as a JSON fee profile
(max observed x headroom, decimal strings) that frontends pass verbatim as
'suggestions' to createTransactionKit from @genlayer/transaction-kit.

Time-unit allocations are omitted: the backend reports consumed fee amounts
only, not time-unit usage. Testnet receipts expose no consumed-fee data yet,
so profiles measured there come out empty (warned at session end).

Also catches uv.lock up with the genlayer-py<0.20.0 pin relax from genlayerlabs#91.
Admin merged after Python 3.12/3.13 and branch-policy checks passed; optional E2E app check is queued/non-blocking for dev-branch PRs.
* fix(gltest): resolve runners from local tree when GENVM_PREBUILT_DIR set

GenVM v0.6 dropped the monolithic runner bundle the direct/glsim loader
downloads from the (retired) genlayerlabs/genvm releases, so under v0.6 every
load failed with `No GenVM runner bundle for <version>`. When GENVM_PREBUILT_DIR
points at a prebuilt GenVM tree, extract the runner straight from its
content-addressed tar (globbed under any *runners* dir, so runners/ and
executor/<ver>/legacy-runners/ both resolve) and skip the download. Download
path is untouched and stays the default when the env var is unset.

* test(fixtures): migrate to genvm v0.3 SDK
* test: add v0.30 bug hunt regressions

* fix: resolve v0.30 bug hunt regressions
* chore(genvm): migrate to the v0.3 gl_call ABI ⬆️

Squashed: the v0.3.0-rc7 SDK API migration and the follow-up rename of the
raw gl_call request tags to their v1 names.

* fix(glsim): accept both cross-call ABIs

* fix(genvm): give VecDB its explicit distance metric 🐛

The pinned embeddings runner's `VecDB` takes the distance metric as a fourth
type parameter. Three arguments now fail at contract import, which leaves the
deploy without state rather than reporting a type error.

---------

Co-authored-by: Edgars <edgars@genlayerlabs.com>
…labs#108)

genlayer-py dropped the public `TransactionStatus` enum when it layered the
consumer lifecycle over the raw protocol status. gltest imports the protocol
enum from the new surface and keeps `TransactionStatus` as an alias of it so
downstream suites that already pass it to `wait_transaction_status` keep
working; internal code uses the protocol name and `wait_until`.

Success assertions read the state-discriminated `lifecycle` field and fall
back to the flat status that pre-lifecycle receipts still carry, so gltest
works against both SDK generations.
Require the published genlayer-py 0.19 RC used by the lifecycle APIs on this train. Add guarded RC/final branch routing, exact native-CI and tag provenance checks, prerelease GitHub releases, and fast regression coverage for the release policy.
Resolve current asset-bearing GenVM releases ahead of stale cache, use runner-bearing bundles in a fresh cache namespace, and validate ZIP extraction paths.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T08:28:50.355365Z f8e971d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@maho0638 maho0638 closed this Oct 12, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f8e971d4df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/release.sh
Comment on lines +90 to +92
"" )
echo "No Tests workflow run found for $local_sha on $branch. Refusing to release an unverified head." >&2
exit 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Trigger tests on stable release branches

Final releases are required to run this script from a stable vX.Y branch, but .github/workflows/tests.yml triggers on pushes only to v*-dev. Because gh run list --branch "$branch" --commit "$local_sha" filters by both branch and SHA, a stable branch's post-merge HEAD normally has no matching Tests run, so this new empty-result check blocks every final release before versioning begins. Add stable branches to the workflow's push trigger or verify an appropriate required check instead.

Useful? React with 👍 / 👎.

Comment on lines +152 to +154
parsed_headroom = float(fee_profile_headroom)
if parsed_headroom <= 0:
raise ValueError("--fee-profile-headroom must be greater than 0")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject non-finite fee-profile headroom values

When --fee-profile-headroom is nan or inf, float() accepts it and the <= 0 check does not reject it. At session finish, _apply_headroom() then raises while converting the resulting non-finite Decimal through math.ceil; that exception is swallowed by pytest_sessionfinish, so tests can report success without producing the requested profile. Validate the parsed value with math.isfinite() in addition to checking that it is positive.

Useful? React with 👍 / 👎.

Comment on lines +44 to +45
- name: Action | Validate release version policy
run: uv run pytest tests/test_release_version.py

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Run the new bug-hunt regressions in CI

The workflow runs the tests/gltest_cli/, tests/gltest/, and tests/glsim/ directories, one direct-loader file, and this release-policy file, but none of those commands collects the newly added tests/test_bug_hunt_v030.py. Consequently its regressions for simulated-write rollback, snapshot restoration, embeddings dependencies, and incomplete cache recovery never run on pushes or pull requests; add that file or the root test directory to the workflow.

Useful? React with 👍 / 👎.

Comment on lines +157 to +160
monkeypatch.setattr(sdk_loader, "CACHE_DIR", tmp_path / "cache")
version = "v0.6.0"
stale_tree = sdk_loader.CACHE_DIR / "trees" / version
stale_tree.mkdir(parents=True)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Point the cache-recovery test at TREE_CACHE_DIR

Even when this test is executed, changing CACHE_DIR does not change the module-level TREE_CACHE_DIR that _extract_release_tree() actually uses, and the fixture additionally creates its stale tree under trees rather than trees-v2. The call therefore reads or writes the real user cache and never encounters partial-download, so the assertions can pass even if incomplete-tree recovery is broken. Monkeypatch TREE_CACHE_DIR directly and create the stale version beneath it.

Useful? React with 👍 / 👎.

Comment thread glsim/server.py
Comment on lines 869 to +872
finally:
_clear_sim_config_mocks(engine)
engine.restore_snapshot(snapshot_id)
engine._snapshots.pop(snapshot_id, None)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore queued messages with simulated state

When an SDK-format simulated call enqueues a PostMessage and then raises, call_method() exits before draining the queue, while this finally restores only the fields captured by create_snapshot(); _post_queue is not among them. The failed fee estimation therefore leaves the message queued, and the next unrelated successful top-level call drains it and mutates contract state. Include the queue in engine snapshots or explicitly reset it as part of this simulation rollback.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants