test: add API stress tests#823
Open
G4614 wants to merge 1 commit into
Open
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
✅ Files skipped from review due to trivial changes (2)
🚧 Files skipped from review as they are similar to previous changes (4)
📝 WalkthroughWalkthroughThree k6 stress test scripts are added for Boxlite's REST API: a read-only scenario, a box creation/deletion scenario, and a VM lifecycle (create→running→delete) scenario. Six corresponding make targets (deployed and local variants) are registered in Changesk6 Stress Test Scripts, Make Targets, and Documentation
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
|
G4614
force-pushed
the
stress/api-layer-k6
branch
3 times, most recently
from
2df7c4f to
a73b802
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/test/stress/api-create-box.k6.js`:
- Around line 92-103: The cleanup operation is currently gated on the `created`
variable which requires both a 201 status AND a valid box_id in the response.
This means if the create request returns 201 but the response structure is
malformed or missing box_id, the cleanup will be skipped and boxes will be left
behind. Modify the cleanup condition to gate it on the HTTP create success
(checking create.status === 201 directly) rather than the `created` check
result, while still using the fallback logic for boxId extraction. This ensures
cleanup runs whenever the create request succeeds at the HTTP level, regardless
of response validation results.
In `@scripts/test/stress/api-vm-lifecycle.k6.js`:
- Around line 95-109: The code currently checks the box status only once on the
create response and returns early if it is not already running, which prevents
the test from waiting for the running state to be reached. After the initial
create request and the check block that validates the 201 response and box_id
presence, add a polling loop that repeatedly fetches the box status using an
HTTP GET request to the box endpoint and checks if the status equals running,
continuing to poll with a small sleep interval until the running state is
achieved before proceeding to the sleep(HOLD_SECONDS) call. This ensures the
actual "create → wait for running → hold → delete" lifecycle flow is executed as
intended.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 54846999-8557-4108-bc09-6466bd97a93d
📒 Files selected for processing (6)
make/help.mkmake/test.mkscripts/test/stress/README.mdscripts/test/stress/api-create-box.k6.jsscripts/test/stress/api-read.k6.jsscripts/test/stress/api-vm-lifecycle.k6.js
Contributor
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
G4614
force-pushed
the
stress/api-layer-k6
branch
from
a73b802 to
6324d81
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/test/stress/api-vm-lifecycle.k6.js`:
- Around line 74-90: The polling loop in the VM box status check treats a 404
response as immediately terminal by returning false, which fails valid create
flows under eventual consistency where the box may not be queryable immediately
after creation. Remove the early return statement when info.status === 404 (the
check block that returns false on line 85) and instead allow the polling loop to
continue iterating until either the resource becomes available with status 200
and 'running' state, or the deadline expires. This way, transient 404s during
the eventual consistency window will be retried rather than causing premature
failure.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: b88c35bf-fe74-4a7b-95e5-d6f45dcec290
📒 Files selected for processing (6)
make/help.mkmake/test.mkscripts/test/stress/README.mdscripts/test/stress/api-create-box.k6.jsscripts/test/stress/api-read.k6.jsscripts/test/stress/api-vm-lifecycle.k6.js
✅ Files skipped from review due to trivial changes (2)
- make/help.mk
- scripts/test/stress/README.md
🚧 Files skipped from review as they are similar to previous changes (1)
- make/test.mk
G4614
force-pushed
the
stress/api-layer-k6
branch
from
6324d81 to
540fb01
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/test/stress/api-create-box.k6.js`:
- Around line 1-2: Biome cannot parse the ES6 import/export statements in the k6
files because it lacks ES module configuration. Update the biome.json
configuration file to add a sourceType: "module" override specifically for the
scripts/test/stress/**/*.k6.js pattern, or alternatively exclude the
scripts/test/stress directory from the linting scope in the biome.json
configuration to prevent Biome from processing these k6 test files that use ES
module syntax.
In `@scripts/test/stress/api-vm-lifecycle.k6.js`:
- Around line 11-18: Add explicit validation after parsing the environment
variables to ensure that VM_LIMIT, VUS, WAIT_RUNNING_MS, and POLL_SECONDS are
positive finite numbers. After each Number() conversion for these variables, add
checks to validate that the parsed value is greater than 0 and is a finite
number (not NaN, Infinity, or negative), and throw or log an appropriate error
if validation fails. The same validation pattern should also be applied to the
variables mentioned in the "Also applies to" note at line 27-29.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 2cc14ecf-f21b-4b4b-94df-c8439bb1a69b
📒 Files selected for processing (6)
make/help.mkmake/test.mkscripts/test/stress/README.mdscripts/test/stress/api-create-box.k6.jsscripts/test/stress/api-read.k6.jsscripts/test/stress/api-vm-lifecycle.k6.js
✅ Files skipped from review due to trivial changes (1)
- scripts/test/stress/README.md
🚧 Files skipped from review as they are similar to previous changes (2)
- make/help.mk
- make/test.mk
G4614
force-pushed
the
stress/api-layer-k6
branch
from
540fb01 to
dbdc626
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds opt-in k6 stress tests for the deployed BoxLite REST API:
These tests are intentionally not part of the default test matrix because they target deployed APIs and can consume shared dev/runner capacity.
Validation
git diff --checkmake -n test:stress:api-read-localmake -n test:stress:api-create-box-localk6 v2.0.0Notes
api-create-box.k6.jscreates real boxes and deletes each successful create. It should be run deliberately with low rates first. A one-shot canary is available with:Additional validation after opening the draft:
BOXLITE_STRESS_ITERATIONS=1 k6 run scripts/test/stress/api-create-box.k6.jsstress-api-create-*boxes left behindRead-only overload attempt from local network:
Api:135VM lifecycle stress update
Added
api-vm-lifecycle.k6.jsfor real VM lifecycle stress. It uses a closedconstant-vusmodel so each VU can hold at most one running VM at a time.Safety guard:
BOXLITE_STRESS_VM_LIMIT <= BOXLITE_STRESS_RUNNER_CPUS * 8BOXLITE_STRESS_VUS <= BOXLITE_STRESS_VM_LIMITCurrent dev runner
i-0ee6bc569d5a1e9efisc8i.2xlargewith 8 vCPU, so the hard ceiling is 64 simultaneous VMs. The local canary usesVM_LIMIT=2/VUS=2, far below the ceiling.Validation:
git diff --checkmake -n test:stress:api-vm-lifecycle-localRUNNER_CPUS=1 VM_LIMIT=9 VUS=9fails before sending requestsVM_LIMIT=2 VUS=2 DURATION=25s HOLD_SECONDS=10, 4 create/run/delete iterations, 0 failures, nostress-api-vm-*leftoversReal VM stress run, 4 concurrent VMs
Ran a real VM lifecycle stress against dev with
BOXLITE_STRESS_RUNNER_CPUS=8,BOXLITE_STRESS_VM_LIMIT=4,BOXLITE_STRESS_VUS=4,BOXLITE_STRESS_DURATION=90s,BOXLITE_STRESS_HOLD_SECONDS=20.Safety: max simultaneous VMs was 4, below the current dev runner hard ceiling of
8 vCPU * 8 = 64.Result:
http_req_failed: 0.00%http_req_duration: avg 850.73ms, p90 1.78s, p95 1.86s, p99 2.49s, max 2.85siteration_duration: avg 22.37s, p90 25.46s, p95 25.71s, p99 26.59s, max 26.81sstress-api-vm-*returned[]Summary by CodeRabbit
make helpto list the new stress-test targets..k6.jsscripts from formatting/linting.