Skip to content

Commit 96aeebb

Browse files
committed
pyproc S0 기준 증거 고정
1 parent 491c4ff commit 96aeebb

5 files changed

Lines changed: 92 additions & 3 deletions

File tree

docs/operations/benchmarking.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ S1은 현재 공개 속도 간판이다. S1L은 외부 후보가 S1의 병렬 wo
4747

4848
| 비교 축 | pyproc | WebVM | JupyterLite | marimo web runtime | caveat |
4949
|---|---|---|---|---|---|
50-
| S0 python ready latency | 측정 필요 | 측정 필요 | 측정 필요 | 측정 필요 | 페이지 또는 런타임 시작부터 첫 Python 명령 성공까지 |
50+
| S0 python ready latency | [측정됨](../../mainPlan/browser-os-north-star/benchmarks/s0-pyproc-2026-07-15.json) | 측정 필요 | 측정 필요 | 측정 필요 | 페이지 또는 런타임 시작부터 첫 Python 명령 성공까지 |
5151
| S1 numpy sharded matmul | Speed Lab 반복 봉투 | N/A 가능 | N/A 가능 | N/A 가능 | 병렬 worker 모델이 다르면 single-lane으로 재정의하지 않음 |
5252
| S1L single-kernel numpy latency | [측정됨](../../mainPlan/browser-os-north-star/benchmarks/s1l-pyproc-2026-07-15.json) | 측정 필요 | [측정됨](../../mainPlan/browser-os-north-star/benchmarks/s1l-jupyterlite-2026-07-15.json) | [측정됨](../../mainPlan/browser-os-north-star/benchmarks/s1l-marimo-wasm-2026-07-15.json) | S1 대체가 아니라 별도 single-lane 보조 축 |
5353
| S2 process map | browser gate | 측정 필요 | 측정 필요 | 측정 필요 | 같은 순수 Python 또는 NumPy 작업 |

mainPlan/browser-os-north-star/03-progress-ledger.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1798,3 +1798,37 @@ NEXT:
17981798
1. pyproc S0 기준 artifact를 생성한다.
17991799
2. WebVM에서 VM boot 뒤 `python3` 첫 출력까지 측정한다.
18001800
3. `benchmarks/s0-compare-2026-07-15.md`를 만든다.
1801+
1802+
## 2026-07-15 - pyproc S0 기준 artifact 고정
1803+
1804+
완료:
1805+
1806+
- `npm run test:browser`를 3회 실행해 pyproc S0 boot ready sample을 확보했다.
1807+
- [s0-pyproc-2026-07-15.json](benchmarks/s0-pyproc-2026-07-15.json)을 생성했다.
1808+
- [s0-compare-2026-07-15.md](benchmarks/s0-compare-2026-07-15.md)를 pyproc 단독 기준표로 생성했다.
1809+
- [06-speed-comparison.md](06-speed-comparison.md), [benchmarking.md](../../docs/operations/benchmarking.md)의 S0 상태를 갱신했다.
1810+
1811+
실측:
1812+
1813+
- Edge 150.0.4078.65, Windows, AMD Ryzen 7 8845HS.
1814+
- `npm run test:browser` 3회 모두 GREEN 47/47.
1815+
- pyproc S0 samples: 3642ms, 3471ms, 3450ms.
1816+
- median 3471ms, p95 3642ms, min 3450ms, max 3642ms, maxErr 0.
1817+
- artifact 생성 commit: `491c4ffe75a381d17dcaf94ab61b315cb0fb6aa7`, `worktreeDirty: false`.
1818+
1819+
검증:
1820+
1821+
- `npm run test:browser` PASS 3회.
1822+
- `npm run bench:artifact -- --scenario S0 --candidate pyproc --browser-version 150.0.4078.65 --engine Pyodide --source "npm run test:browser" --command "npm run test:browser, boot() ready and run sum(range(100))" --note "derived from three browser gate runs; latencyMs is bootMs until Python boot ready" --sample 3642,0 --sample 3471,0 --sample 3450,0 --out mainPlan/browser-os-north-star/benchmarks/s0-pyproc-2026-07-15.json` PASS.
1823+
- `npm run bench:compare -- mainPlan/browser-os-north-star/benchmarks/s0-pyproc-2026-07-15.json --out mainPlan/browser-os-north-star/benchmarks/s0-compare-2026-07-15.md` PASS.
1824+
1825+
판정:
1826+
1827+
- S0 pyproc 기준점이 생겼다.
1828+
- WebVM은 이제 같은 S0 표에 들어갈 수 있다. 단 WebVM의 latency는 Linux VM boot 뒤 `python3` 첫 출력까지로 분리한다.
1829+
1830+
NEXT:
1831+
1832+
1. WebVM에서 S0 `python3` 첫 출력까지 측정한다.
1833+
2. [s0-compare-2026-07-15.md](benchmarks/s0-compare-2026-07-15.md)에 WebVM 행을 합친다.
1834+
3. WebVM S0가 불안정하면 N/A가 아니라 실패 artifact 또는 제한 근거를 남긴다.

mainPlan/browser-os-north-star/06-speed-comparison.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# 06. 속도 정면 비교 계약
22

3-
상태: pyproc S1 기준 artifact 기록. WebVM, JupyterLite, marimo WASM은 같은 S1 계약을 N/A artifact로 봉인. S1L은 pyproc, JupyterLite, marimo WASM을 같은 브라우저에서 측정.
3+
상태: pyproc S0/S1 기준 artifact 기록. WebVM, JupyterLite, marimo WASM은 같은 S1 계약을 N/A artifact로 봉인. S1L은 pyproc, JupyterLite, marimo WASM을 같은 브라우저에서 측정.
44

55
## 목표
66

@@ -30,6 +30,7 @@
3030

3131
| date | scenario | artifact | compare | result |
3232
|---|---|---|---|---|
33+
| 2026-07-15 | S0 pyproc | [s0-pyproc-2026-07-15.json](benchmarks/s0-pyproc-2026-07-15.json) | [s0-compare-2026-07-15.md](benchmarks/s0-compare-2026-07-15.md) | GREEN, 3 samples, median 3471ms, p95 3642ms, maxErr 0 |
3334
| 2026-07-15 | S1 pyproc | [s1-pyproc-2026-07-15.json](benchmarks/s1-pyproc-2026-07-15.json) | [s1-compare-2026-07-15.md](benchmarks/s1-compare-2026-07-15.md) | GREEN, `size=1024`, 3 samples, median 3.95x, shard p95 2606ms, maxErr 0 |
3435
| 2026-07-15 | S1 external candidates | [webvm N/A](benchmarks/s1-webvm-na-2026-07-15.json), [jupyterlite N/A](benchmarks/s1-jupyterlite-na-2026-07-15.json), [marimo-wasm N/A](benchmarks/s1-marimo-wasm-na-2026-07-15.json) | [s1-compare-2026-07-15.md](benchmarks/s1-compare-2026-07-15.md) | 같은 4-worker sharded NumPy matmul 계약 없음 |
3536
| 2026-07-15 | S1L pyproc | [s1l-pyproc-2026-07-15.json](benchmarks/s1l-pyproc-2026-07-15.json) | [s1l-compare-2026-07-15.md](benchmarks/s1l-compare-2026-07-15.md) | GREEN, 3 samples, median 10067ms, p95 10073ms, maxErr 0 |
@@ -42,7 +43,7 @@
4243

4344
| scenario | pyproc command | WebVM | JupyterLite | marimo web runtime | 판정 |
4445
|---|---|---|---|---|---|
45-
| S0 python ready latency | `npm run bench:artifact -- --scenario S0` | 미측정 | 미측정 | 미측정 | 보류 |
46+
| S0 python ready latency | [artifact](benchmarks/s0-pyproc-2026-07-15.json) | 미측정 | 미측정 | 미측정 | WebVM 측정 대기 |
4647
| S1 numpy sharded matmul | `npm run bench:speed -- --out <path>` | [N/A](benchmarks/s1-webvm-na-2026-07-15.json) | [N/A](benchmarks/s1-jupyterlite-na-2026-07-15.json) | [N/A](benchmarks/s1-marimo-wasm-na-2026-07-15.json) | pyproc만 같은 S1 계약 충족 |
4748
| S1L single-kernel numpy latency | [artifact](benchmarks/s1l-pyproc-2026-07-15.json) | 미측정 | [artifact](benchmarks/s1l-jupyterlite-2026-07-15.json) | [artifact](benchmarks/s1l-marimo-wasm-2026-07-15.json) | WebVM 제외 3자 측정 완료 |
4849
| S2 process map | `npm run test:browser` | 미측정 | 미측정 | 미측정 | 보류 |
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
| candidate | ok | samples | ready median ms | ready p95 ms | min ms | max ms | maxErr | browser | commit | source |
2+
|---|---:|---:|---:|---:|---:|---:|---:|---|---|---|
3+
| pyproc | GREEN | 3 | 3471 | 3642 | 3450 | 3642 | 0 | 150.0.4078.65 | 491c4ffe | mainPlan/browser-os-north-star/benchmarks/s0-pyproc-2026-07-15.json |
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
{
2+
"schemaVersion": 1,
3+
"scenario": "S0",
4+
"candidate": "pyproc",
5+
"name": "python ready latency",
6+
"command": "npm run test:browser, boot() ready and run sum(range(100))",
7+
"commit": "491c4ffe94e2651c16f58c6e40c64dd4eb031729",
8+
"worktreeDirty": false,
9+
"startedAt": "2026-07-15T01:15:51.923Z",
10+
"finishedAt": "2026-07-15T01:15:51.924Z",
11+
"browser": {
12+
"path": null,
13+
"version": "150.0.4078.65"
14+
},
15+
"host": {
16+
"platform": "win32",
17+
"release": "10.0.26200",
18+
"cpuModel": "AMD Ryzen 7 8845HS w/ Radeon 780M Graphics ",
19+
"cpuCount": 16,
20+
"totalMemBytes": 33557934080,
21+
"freeMemBytes": 14668394496
22+
},
23+
"engine": {
24+
"name": "Pyodide"
25+
},
26+
"source": "npm run test:browser",
27+
"note": "derived from three browser gate runs; latencyMs is bootMs until Python boot ready",
28+
"ok": true,
29+
"metrics": {
30+
"samples": [
31+
{
32+
"latencyMs": 3642,
33+
"maxErr": 0
34+
},
35+
{
36+
"latencyMs": 3471,
37+
"maxErr": 0
38+
},
39+
{
40+
"latencyMs": 3450,
41+
"maxErr": 0
42+
}
43+
],
44+
"sampleCount": 3,
45+
"medianMs": 3471,
46+
"p95Ms": 3642,
47+
"minMs": 3450,
48+
"maxMs": 3642,
49+
"maxErr": 0
50+
}
51+
}

0 commit comments

Comments
 (0)