Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 29 additions & 2 deletions .github/workflows/pr-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,18 +13,45 @@ jobs:
cli-quality:
uses: ./.github/workflows/cli-quality.yml

docker-smoke:
name: docker-smoke
runs-on: blacksmith-4vcpu-ubuntu-2404
steps:
- name: Check out repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Set up Docker Buildx
uses: useblacksmith/setup-docker-builder@v1
with:
max-cache-size-mb: "20480"

- name: Build and load image
uses: useblacksmith/build-push-action@v2
with:
context: .
load: true
tags: verboo:ci-smoke

- name: Verify container version and protocol
run: |
docker run --rm verboo:ci-smoke --version
docker run --rm verboo:ci-smoke --internal-protocol-self-test

# Keep this check name stable for existing branch-protection rules.
smoke-and-tests:
name: smoke-and-tests
needs: cli-quality
needs: [cli-quality, docker-smoke]
if: ${{ always() }}
runs-on: blacksmith-4vcpu-ubuntu-2404

steps:
- name: Require the complete quality gate
env:
QUALITY_RESULT: ${{ needs.cli-quality.result }}
run: test "$QUALITY_RESULT" = success
DOCKER_RESULT: ${{ needs.docker-smoke.result }}
run: |
test "$QUALITY_RESULT" = success
test "$DOCKER_RESULT" = success

- name: Check out repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
Expand Down
5 changes: 5 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@ FROM node:22-slim AS build

WORKDIR /app

# The terminal test dependency node-pty builds from source on Linux.
# Keep its compiler toolchain in the build stage only.
RUN apt-get update && apt-get install -y --no-install-recommends python3 make g++ \
&& rm -rf /var/lib/apt/lists/*

# Copy dependency manifests first for better layer caching
COPY package.json bun.lock .bun-version ./

Expand Down
10 changes: 8 additions & 2 deletions docs/cli-quality.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

The shared `.github/workflows/cli-quality.yml` workflow is required by PR checks
and every release publication job. No test-failure baseline is accepted.
The existing `smoke-and-tests` check explicitly fails when this gate fails or is
cancelled; it cannot become a successful skipped check after a dependency fails.
The existing `smoke-and-tests` check explicitly fails when this gate or the
Docker build/protocol check fails or is cancelled; it cannot become a successful
skipped check after a dependency fails.

## Agent consumption and completion

Expand All @@ -21,6 +22,9 @@ cancelled; it cannot become a successful skipped check after a dependency fails.
SDK output wakes independently of the parent query yielding another message.
- Metadata writes are atomic and serialized. An execution identifier prevents a
late callback from overwriting a resumed agent's metadata or task state.
- Windows metadata replacement retries temporary reader/scanner locks for up to
630 ms, preserving the previous complete file and write order; permanent
failures still propagate and remove the temporary file.
- Display success, provider failure, user interruption and execution-budget
limits distinctly. Closing task details must not cancel running agents.

Expand All @@ -37,6 +41,8 @@ cancelled; it cannot become a successful skipped check after a dependency fails.
| Terminal matrix | Linux, macOS and Windows; Node 22 and 24; 40×12, 80×24 and 120×40; 1/2/8/20 agents; normal and fullscreen |
| Interaction/error cases | Missing/partial/explicit-zero usage, background completion, task menu/detail, draft preservation, resize, Esc, provider failure, configured maxTurns, streaming JSON |
| Other components | Python tests, web typecheck/build, existing native desktop checks |
| Docker | Full image build, version and protocol smoke on every PR; native test compilation tools stay in the build stage |
| Fixture lifecycle | Partial HTTP request cancellation, draining stream handlers and malformed-request failures under Bun and the six Node/platform jobs |

The terminal fixture uses a fresh project and configuration, synthetic OAuth,
strict empty MCP configuration, disabled plugin installation and no provider
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@verboo/code",
"version": "0.15.22",
"version": "0.15.23",
"description": "Verboo Code — coding agent for the Verboo platform",
"type": "module",
"bin": {
Expand Down
1 change: 1 addition & 0 deletions scripts/check-cli-quality.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ if (!terminalOnly) {
run([node, 'scripts/prepare-cli-package.mjs', '--pack-only'])
}
if (!process.argv.includes('--suite')) {
run([node, '--test', 'scripts/e2e/fake-router.test.mjs'])
run([node, 'scripts/setup-pty.mjs'])
run([node, 'scripts/prepare-cli-package.mjs', '--install-only'])
if (process.platform === 'win32') {
Expand Down
9 changes: 7 additions & 2 deletions scripts/cli-quality-workflow.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,15 @@ test('PRs and every publication surface require the shared quality gate', () =>
const pr = parse(readFileSync('.github/workflows/pr-checks.yml', 'utf8'))
const release = parse(readFileSync('.github/workflows/release.yml', 'utf8'))
expect(pr.jobs['cli-quality'].uses).toBe('./.github/workflows/cli-quality.yml')
expect(pr.jobs['smoke-and-tests'].needs).toBe('cli-quality')
expect(pr.jobs['smoke-and-tests'].needs).toContain('cli-quality')
expect(pr.jobs['smoke-and-tests'].needs).toContain('docker-smoke')
expect(pr.jobs['smoke-and-tests'].if).toBe('${{ always() }}')
expect(pr.jobs['smoke-and-tests'].steps[0].env.QUALITY_RESULT).toBe('${{ needs.cli-quality.result }}')
expect(pr.jobs['smoke-and-tests'].steps[0].run).toBe('test "$QUALITY_RESULT" = success')
expect(pr.jobs['smoke-and-tests'].steps[0].env.DOCKER_RESULT).toBe('${{ needs.docker-smoke.result }}')
expect(pr.jobs['smoke-and-tests'].steps[0].run).toContain('test "$QUALITY_RESULT" = success')
expect(pr.jobs['smoke-and-tests'].steps[0].run).toContain('test "$DOCKER_RESULT" = success')
expect(pr.jobs['docker-smoke'].steps.some(step => step.with?.load === true && step.with?.push !== true)).toBe(true)
expect(pr.jobs['docker-smoke'].steps.some(step => step.run?.includes('docker run --rm verboo:ci-smoke --internal-protocol-self-test'))).toBe(true)
expect(release.jobs['cli-quality'].uses).toBe(pr.jobs['cli-quality'].uses)
expect(release.jobs['cli-quality'].with.ref).toBe('${{ needs.verify.outputs.tag }}')
for (const job of ['publish-npm', 'docker', 'desktop-cli-artifacts', 'publish-desktop-cli']) {
Expand Down
23 changes: 21 additions & 2 deletions scripts/e2e/fake-router.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ export async function createFakeRouter({ agents = 2, omitUsage = false, zeroUsag
const unexpected = []
let sequence = 0
const activeAgents = new Set()
const server = createServer(async (req, res) => {
const activeRequests = new Set()
async function handleRequest(req, res) {
const path = new URL(req.url, 'http://fixture').pathname
const json = value => { res.setHeader('Content-Type', 'application/json'); res.end(JSON.stringify(value)) }
if (path === '/') return json({ ok: true })
Expand Down Expand Up @@ -56,7 +57,25 @@ export async function createFakeRouter({ agents = 2, omitUsage = false, zeroUsag
emit({}, 'stop', omitUsage ? undefined : zeroUsage ? { prompt_tokens: 0, completion_tokens: 0 } : partialUsage ? { prompt_tokens: 120 } : { prompt_tokens: 120, completion_tokens: 24, prompt_tokens_details: { cached_tokens: 5 } })
}
res.end('data: [DONE]\n\n')
}
const server = createServer((req, res) => {
const handling = handleRequest(req, res).catch(error => {
// Stopping the CLI may abort a request while its body is arriving.
// HTTP event listeners do not consume rejected async promises themselves.
if (req.aborted && error?.code === 'ECONNRESET') return
unexpected.push(`Fixture handler failed: ${error?.message ?? String(error)}`)
res.destroy()
}).finally(() => activeRequests.delete(handling))
activeRequests.add(handling)
})
await new Promise(resolve => server.listen(0, '127.0.0.1', resolve))
return { origin: `http://127.0.0.1:${server.address().port}`, requests, unexpected, activeAgents, async close() { server.closeAllConnections(); await new Promise(resolve => server.close(resolve)) } }
return {
origin: `http://127.0.0.1:${server.address().port}`, requests, unexpected, activeAgents, activeRequests,
async close() {
const closed = new Promise(resolve => server.close(resolve))
server.closeAllConnections()
await closed
await Promise.all([...activeRequests])
},
}
}
58 changes: 58 additions & 0 deletions scripts/e2e/fake-router.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import assert from 'node:assert/strict'
import { connect } from 'node:net'
import { once } from 'node:events'
import { setTimeout as delay } from 'node:timers/promises'
import { test } from 'node:test'
import { createFakeRouter } from './fake-router.mjs'

async function waitForRequest(router) {
for (let attempt = 0; attempt < 100; attempt++) {
if (router.activeRequests.size) return
await delay(10)
}
throw new Error('Fixture never accepted the partial request')
}

test('a client may abort a partial request without leaking a rejection', async () => {
const router = await createFakeRouter()
const socket = connect({ host: '127.0.0.1', port: Number(new URL(router.origin).port) })
try {
await once(socket, 'connect')
socket.write('POST /router/v1/chat/completions HTTP/1.1\r\nHost: fixture\r\nContent-Length: 1000\r\n\r\n{"messages":')
await waitForRequest(router)
socket.destroy()
await router.close()
assert.deepEqual(router.unexpected, [])
assert.equal(router.activeRequests.size, 0)
assert.equal(router.requests.length, 0)
} finally {
socket.destroy()
await router.close()
}
})

test('fixture shutdown drains in-flight stream handlers', async () => {
const router = await createFakeRouter()
try {
const response = await fetch(`${router.origin}/router/v1/chat/completions`, {
method: 'POST',
body: JSON.stringify({ messages: [{ role: 'user', content: 'AGENT_FIXTURE_0' }], tools: [{}] }),
})
assert.equal(router.activeAgents.size, 1)
await response.body.cancel()
await router.close()
assert.equal(router.activeRequests.size, 0)
assert.equal(router.activeAgents.size, 0)
assert.deepEqual(router.unexpected, [])
} finally { await router.close() }
})

test('malformed fixture requests still fail the network gate', async () => {
const router = await createFakeRouter()
try {
await assert.rejects(fetch(`${router.origin}/router/v1/chat/completions`, { method: 'POST', body: 'not valid json' }))
await router.close()
assert.equal(router.unexpected.length, 1)
assert.match(router.unexpected[0], /Fixture handler failed/)
} finally { await router.close() }
})
103 changes: 101 additions & 2 deletions src/utils/sessionStorage.agentUsage.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { expect, test } from 'bun:test'
import { mkdtemp, rm } from 'node:fs/promises'
import { expect, spyOn, test } from 'bun:test'
import * as fsPromises from 'fs/promises'
import { mkdtemp, readdir, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { getSessionId, getSessionProjectDir, switchSession } from '../bootstrap/state.js'
Expand Down Expand Up @@ -29,3 +30,101 @@ test('metadata keeps legacy fields and rejects late writes from a replaced execu
expect((await readAgentMetadata(id))?.tokenUsage).toEqual(usage)
} finally { switchSession(session, projectDir); await rm(dir, { recursive: true, force: true }) }
})

for (const code of ['EPERM', 'EACCES', 'EBUSY']) {
test(`Windows metadata replacement recovers from a temporary ${code} lock`, async () => {
const dir = await mkdtemp(join(tmpdir(), 'agent-metadata-lock-'))
const session = getSessionId()
const projectDir = getSessionProjectDir()
const platform = Object.getOwnPropertyDescriptor(process, 'platform')!
const id = asAgentId(`metadata-lock-${code}`)
const usage = { ...emptyAgentUsage(), state: 'reported' as const, confirmed: 144 }
const originalRename = fsPromises.rename
let attempts = 0
let renameSpy: ReturnType<typeof spyOn> | undefined
try {
switchSession(session, dir)
await writeAgentMetadata(id, { agentType: 'explore', executionId: 'current' })
Object.defineProperty(process, 'platform', { ...platform, value: 'win32' })
renameSpy = spyOn(fsPromises, 'rename').mockImplementation(async (from, to) => {
attempts++
// A reader continues seeing the previous complete record during retry.
if (attempts <= 2) {
expect(await readAgentMetadata(id)).toEqual({ agentType: 'explore', executionId: 'current' })
throw Object.assign(new Error('fixture sharing violation'), { code })
}
return originalRename(from, to)
})
await writeAgentUsageMetadata(id, 'current', usage)
expect(attempts).toBe(3)
expect((await readAgentMetadata(id))?.tokenUsage).toEqual(usage)
expect((await readdir(dir, { recursive: true })).some(file => file.endsWith('.tmp'))).toBe(false)
} finally {
renameSpy?.mockRestore()
Object.defineProperty(process, 'platform', platform)
switchSession(session, projectDir)
await rm(dir, { recursive: true, force: true })
}
})
}

for (const [platformName, code, expectedAttempts] of [['win32', 'EPERM', 7], ['win32', 'ENOSPC', 1], ['linux', 'EPERM', 1]] as const) {
test(`metadata failure is bounded and preserves the old file for ${platformName}/${code}`, async () => {
const dir = await mkdtemp(join(tmpdir(), 'agent-metadata-error-'))
const session = getSessionId()
const projectDir = getSessionProjectDir()
const platform = Object.getOwnPropertyDescriptor(process, 'platform')!
const id = asAgentId('metadata-error')
const original = { agentType: 'explore', executionId: 'old' }
let renameSpy: ReturnType<typeof spyOn> | undefined
try {
switchSession(session, dir)
await writeAgentMetadata(id, original)
Object.defineProperty(process, 'platform', { ...platform, value: platformName })
const error = Object.assign(new Error('fixture persistent error'), { code })
renameSpy = spyOn(fsPromises, 'rename').mockRejectedValue(error)
await expect(writeAgentMetadata(id, { agentType: 'general-purpose', executionId: 'new' })).rejects.toBe(error)
expect(renameSpy).toHaveBeenCalledTimes(expectedAttempts)
expect(await readAgentMetadata(id)).toEqual(original)
expect((await readdir(dir, { recursive: true })).some(file => file.endsWith('.tmp'))).toBe(false)
renameSpy.mockRestore()
renameSpy = undefined
await writeAgentMetadata(id, { agentType: 'general-purpose', executionId: 'new' })
expect((await readAgentMetadata(id))?.executionId).toBe('new')
} finally {
renameSpy?.mockRestore()
Object.defineProperty(process, 'platform', platform)
switchSession(session, projectDir)
await rm(dir, { recursive: true, force: true })
}
})
}

test('a resumed execution stays newer than a usage write retried under a Windows lock', async () => {
const dir = await mkdtemp(join(tmpdir(), 'agent-metadata-resume-lock-'))
const session = getSessionId()
const projectDir = getSessionProjectDir()
const platform = Object.getOwnPropertyDescriptor(process, 'platform')!
const id = asAgentId('metadata-resume-lock')
let renameSpy: ReturnType<typeof spyOn> | undefined
try {
switchSession(session, dir)
await writeAgentMetadata(id, { agentType: 'explore', executionId: 'old' })
Object.defineProperty(process, 'platform', { ...platform, value: 'win32' })
const originalRename = fsPromises.rename
renameSpy = spyOn(fsPromises, 'rename').mockImplementation(originalRename)
.mockRejectedValueOnce(Object.assign(new Error('fixture reader lock'), { code: 'EPERM' }))
await Promise.all([
writeAgentUsageMetadata(id, 'old', { ...emptyAgentUsage(), confirmed: 144 }),
writeAgentMetadata(id, { agentType: 'general-purpose', executionId: 'new', description: 'resumed' }),
writeAgentUsageMetadata(id, 'old', emptyAgentUsage()),
])
expect(renameSpy).toHaveBeenCalledTimes(3)
expect(await readAgentMetadata(id)).toEqual({ agentType: 'general-purpose', executionId: 'new', description: 'resumed' })
} finally {
renameSpy?.mockRestore()
Object.defineProperty(process, 'platform', platform)
switchSession(session, projectDir)
await rm(dir, { recursive: true, force: true })
}
})
15 changes: 14 additions & 1 deletion src/utils/sessionStorage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,20 @@ async function persistAgentMetadata(path: string, metadata: AgentMetadata): Prom
const temporary = `${path}.${process.pid}.tmp`
try {
await writeFile(temporary, JSON.stringify(metadata))
await rename(temporary, path)
for (let attempt = 0; ; attempt++) {
try {
await rename(temporary, path)
break
} catch (error) {
// Windows readers and antivirus scanners can briefly prevent replacing
// an existing file. Keep the complete temporary file and the per-path
// write queue while retrying; never remove the last good metadata.
const code = (error as NodeJS.ErrnoException).code
if (process.platform !== 'win32' || attempt >= 6 ||
(code !== 'EPERM' && code !== 'EACCES' && code !== 'EBUSY')) throw error
await new Promise(resolve => setTimeout(resolve, 10 * 2 ** attempt))
}
}
} finally { await unlink(temporary).catch(() => {}) }
}
export async function writeAgentMetadata(
Expand Down
Loading