From 45b8b69937c1762556d5abaed7bc925a21da06c5 Mon Sep 17 00:00:00 2001 From: chenghan Date: Wed, 22 Jul 2026 11:18:26 +0800 Subject: [PATCH 1/2] =?UTF-8?q?docs(skill):=20=E9=81=B5=E8=A7=84=E9=87=8D?= =?UTF-8?q?=E5=A1=91=EF=BC=8C=E7=AE=80=E8=80=8C=E5=8F=AF=E8=A1=8C=20=C2=B7?= =?UTF-8?q?=20conform=20SKILL.md=20to=20skill-creator=20spec?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add YAML frontmatter (name + description) as required by skill-creator - Move 'when to use' triggers into description frontmatter - Use imperative form throughout - Reduce body to 88 lines (under 500 limit) - Add agents/openai.yaml with display_name, short_description, default_prompt - Remove verbose sections that Codex already knows --- .agents/skills/local-ocr/SKILL.md | 75 +++++++++------------ .agents/skills/local-ocr/agents/openai.yaml | 6 ++ 2 files changed, 37 insertions(+), 44 deletions(-) create mode 100644 .agents/skills/local-ocr/agents/openai.yaml diff --git a/.agents/skills/local-ocr/SKILL.md b/.agents/skills/local-ocr/SKILL.md index 3ca6c5c..2ddbaa1 100644 --- a/.agents/skills/local-ocr/SKILL.md +++ b/.agents/skills/local-ocr/SKILL.md @@ -1,26 +1,25 @@ -# light-ocr: Local OCR Skill +--- +name: local-ocr +description: Extract text from local images (PNG, JPEG) with precise coordinates, confidence scores, and stable error handling using the light-ocr CLI. Use when needing to read small/dense text in screenshots, receipts, labels, forms, or documents that a multimodal model may misread; when exact text plus bounding box coordinates are needed for field extraction, redaction, counting, or layout analysis; or when deterministic offline OCR is required without network or API calls. +--- -Use this skill when you need to extract text from local images (PNG, JPEG) with precise coordinates, confidence scores, and stable error handling — without writing Node.js integration code or relying on a multimodal model's ability to read small text. +# local-ocr -## When to use OCR instead of a multimodal model - -- Small text, dense text, or text in screenshots/labels/receipts/forms that a multimodal model misreads or hallucinates. -- When you need exact text + bounding box coordinates (for field extraction, redaction, counting, or downstream layout analysis). -- When you need deterministic, offline, reproducible results (no network, no API calls). +`light-ocr` is a local OCR engine for Node.js. It runs offline, returns text with coordinates and confidence, and follows a strict stdout/stderr contract for scripting. ## Commands ```bash -# Full OCR: recognize text + coordinates (default action) +# Full OCR: text + coordinates (default action) light-ocr image.png --format json -light-ocr image.png --format text # just text, no coordinates +light-ocr image.png --format text # text only, no coordinates -# Region-only recognition (ROI) +# Region-only recognition (ROI in pageSpace pixels) light-ocr recognize image.png --region 100,80,640,320 --format json -# Detect-only: just text region boxes, no recognition -light-ocr detect image.png # output is always JSON -light-ocr detect image.png --crop # include PNG crop per box +# Detect-only: text region boxes, no recognition (always JSON) +light-ocr detect image.png +light-ocr detect image.png --crop # attach PNG crop per box # Diagnostics (no image read) light-ocr info --model-info # full EngineInfo JSON @@ -36,7 +35,7 @@ light-ocr recognize image.png --provider cpu # force CPU ## Output schema -All `recognize`/`detect` output uses `--schema-version 1` (default). The envelope: +All `recognize`/`detect` output wraps in a `schemaVersion: 1` envelope: ```json { @@ -46,14 +45,14 @@ All `recognize`/`detect` output uses `--schema-version 1` (default). The envelop } ``` -- `recognize`: `pages[0].lines[]` with `{ id: "L0", text, confidence, box: [4 points] }` -- `detect`: `pages[0].detections[]` with `{ id: "D0", score, box: [4 points] }` -- `--format text`: only recognized text, one line per line (no coordinates) +- `recognize`: `pages[0].lines[]` → `{ id: "L0", text, confidence, box: [4 points] }` +- `detect`: `pages[0].detections[]` → `{ id: "D0", score, box: [4 points] }` +- `--format text`: recognized text only, one line per line - `--format jsonl`: one page record per line (for streaming/batch) ## Choosing what to run -| You want | Command | +| Goal | Command | | --- | --- | | Full text from an image | `recognize --format text` | | Text + coordinates | `recognize --format json` | @@ -64,7 +63,7 @@ All `recognize`/`detect` output uses `--schema-version 1` (default). The envelop ## Exit codes -| Code | Meaning | Agent action | +| Code | Meaning | Action | | --- | --- | --- | | 0 | Success | Parse stdout | | 64 | Usage error | Fix command syntax | @@ -72,35 +71,23 @@ All `recognize`/`detect` output uses `--schema-version 1` (default). The envelop | 66 | Invalid image | Try different image | | 67 | Unsupported capability | Check `info --model-info` | | 68 | Model/bundle error | Reinstall package | -| 69 | Resource limit exceeded | Smaller image or region | +| 69 | Resource limit exceeded | Use smaller image or `--region` | | 70 | Environment/package failure | Check native addon | | 71 | Inference failure | Retry or report bug | | 72 | Internal error | Report bug | -## How to handle failures - -- **Empty result**: No text found. The image may have no text, or text is too small/low-contrast. Try `--region` on specific areas, or check with `detect` first. -- **Low confidence**: Lines with `confidence < 0.5` may be unreliable. Don't present inferred or guessed text as OCR output — always cite the actual `text` field and its `confidence`. -- **Resource limit (exit 69)**: Image too large. Use `--region` to process a sub-area, or downscale before passing. -- **Unsupported capability (exit 67)**: The requested provider or feature isn't available. Run `info --model-info` to see what's supported. - -## Important rules - -1. **Never fabricate OCR text.** Only use text from the `text` field of the result. If confidence is low, say so — don't guess. -2. **Cite coordinates when relevant.** Box coordinates are in `pageSpace` (top-left origin, x right, y down, post-EXIF pixels). Use them for field extraction, redaction, or counting. -3. **Use `--schema-version 1`** for reproducible output. Don't parse help text programmatically. -4. **Prefer `detect` first** if you only need to locate text regions (faster, no recognition). -5. **Use `--region`** to avoid processing huge images unnecessarily — detect first, then recognize specific regions. -6. **Check exit codes** before parsing stdout. Non-zero exit means stdout may be empty; stderr has the error. +## Failure handling -## Validation script - -```bash -# Quick smoke test: recognize a known image and check exit code -light-ocr test-image.png --format text && echo "OK: $(light-ocr test-image.png --format text | wc -l) lines" -``` +- **Empty result**: No text found. Try `detect` first to see if any regions were detected, then `recognize --region` on specific areas. +- **Low confidence** (< 0.5): Do not present inferred text as OCR output. Always cite the actual `text` field and `confidence`. +- **Resource limit** (exit 69): Image too large. Use `--region` to process a sub-area. +- **Unsupported capability** (exit 67): Run `info --model-info` to check available providers. -## Related +## Rules -- [CLI design](docs/cli-design.md) — full flag reference, coordinate semantics, exit codes -- [Roadmap N1](docs/roadmap.md) — product context and acceptance criteria +1. Never fabricate OCR text. Only use the `text` field from results. If confidence is low, state it. +2. Cite coordinates when relevant. Box coordinates are in `pageSpace` (top-left origin, x right, y down, post-EXIF pixels). +3. Use `--schema-version 1` for reproducible output. Do not parse help text programmatically. +4. Prefer `detect` first if only locating text regions (faster, no recognition). +5. Use `--region` to avoid processing huge images unnecessarily. +6. Check exit codes before parsing stdout. Non-zero exit means stdout may be empty; read stderr for the error. diff --git a/.agents/skills/local-ocr/agents/openai.yaml b/.agents/skills/local-ocr/agents/openai.yaml new file mode 100644 index 0000000..2807f85 --- /dev/null +++ b/.agents/skills/local-ocr/agents/openai.yaml @@ -0,0 +1,6 @@ +interface: + display_name: "Local OCR" + short_description: "Offline OCR with coordinates via light-ocr CLI" + default_prompt: "Use $skill-local-ocr to extract text and coordinates from a local image." +policy: + allow_implicit_invocation: true From 27b986df60c14066ff0ded56a271ccce4612e6b2 Mon Sep 17 00:00:00 2001 From: chenghan Date: Wed, 22 Jul 2026 11:56:15 +0800 Subject: [PATCH 2/2] =?UTF-8?q?docs(skill):=20=E5=9B=A0=E5=A2=83=E6=96=BD?= =?UTF-8?q?=E6=B3=95=EF=BC=8C=E5=9C=BA=E6=99=AF=E5=85=88=E8=A1=8C=20=C2=B7?= =?UTF-8?q?=20redesign=20SKILL.md=20with=20scenario-driven=20workflows?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Replace command-listing with real usage scenarios: screenshot small text, form/receipt field extraction (detect-then-recognize two-step pattern), counting regions, verifying multimodal output, batch processing - Add decision flow diagram for command selection - Keep under 500 lines, imperative form, skill-creator spec compliant --- .agents/skills/local-ocr/SKILL.md | 129 +++++++++++++++++++----------- 1 file changed, 81 insertions(+), 48 deletions(-) diff --git a/.agents/skills/local-ocr/SKILL.md b/.agents/skills/local-ocr/SKILL.md index 2ddbaa1..9beb21b 100644 --- a/.agents/skills/local-ocr/SKILL.md +++ b/.agents/skills/local-ocr/SKILL.md @@ -5,62 +5,103 @@ description: Extract text from local images (PNG, JPEG) with precise coordinates # local-ocr -`light-ocr` is a local OCR engine for Node.js. It runs offline, returns text with coordinates and confidence, and follows a strict stdout/stderr contract for scripting. +`light-ocr` is a local OCR engine. It runs offline, returns text with coordinates and confidence, and follows a strict stdout/stderr contract for scripting. -## Commands +## Scenarios + +### Screenshot with small text + +A user shares a screenshot and asks about specific text that is too small or dense to read visually. ```bash -# Full OCR: text + coordinates (default action) -light-ocr image.png --format json -light-ocr image.png --format text # text only, no coordinates +# Step 1: recognize the full image +light-ocr screenshot.png --format json +``` -# Region-only recognition (ROI in pageSpace pixels) -light-ocr recognize image.png --region 100,80,640,320 --format json +If the result has low confidence or missing text in a region: -# Detect-only: text region boxes, no recognition (always JSON) -light-ocr detect image.png -light-ocr detect image.png --crop # attach PNG crop per box +```bash +# Step 2: re-run on the specific region (coordinates from step 1 boxes) +light-ocr recognize screenshot.png --region 100,80,640,320 --format json +``` -# Diagnostics (no image read) -light-ocr info --model-info # full EngineInfo JSON -light-ocr info --version # npm/core/model triple +### Form or receipt field extraction -# stdin -cat image.png | light-ocr recognize --stdin --type image/png --format json +Need to extract specific fields (names, amounts, dates) from a form or receipt image. -# Execution provider -light-ocr recognize image.png --provider auto # default: auto-select best -light-ocr recognize image.png --provider cpu # force CPU +```bash +# Step 1: detect where text regions are (fast, no recognition) +light-ocr detect receipt.png + +# Step 2: recognize only the region containing the target field +light-ocr recognize receipt.png --region 50,200,300,80 --format json ``` -## Output schema +This two-step pattern saves time on large images: detect first, then recognize only the regions of interest. + +### Counting text regions + +Need to count how many text lines or regions exist in an image. + +```bash +light-ocr detect image.png | python -c "import json,sys; print(len(json.load(sys.stdin)['pages'][0]['detections']))" +``` + +### Verifying multimodal model output + +A multimodal model claims to read text from an image. Verify the claim against deterministic OCR. + +```bash +light-ocr image.png --format text +``` + +Compare the text output with the model's claim. If they differ, trust the OCR `text` field — do not fabricate. + +### Batch processing via shell -All `recognize`/`detect` output wraps in a `schemaVersion: 1` envelope: +Process multiple images sequentially with JSONL output. + +```bash +for f in *.png; do + light-ocr recognize "$f" --format jsonl +done +``` + +Each line is one page record. Check exit codes: a non-zero exit for one image does not stop the loop, but stdout for that image may be empty. + +## Decision flow + +``` +Need text from an image? +├── Know which region? → recognize --region x,y,w,h --format json +├── Need full text only? → recognize --format text +├── Need text + coordinates? → recognize --format json +├── Only need where text is? → detect +├── Large image, unsure where text is? → detect first, then recognize --region +└── Need engine info or version? → info --model-info / info --version +``` + +## Output schema ```json { "schemaVersion": 1, "source": { "kind": "image", "mediaType": "...", "identity": {}, "appliedTransforms": {} }, - "pages": [{ "index": 0, "width": ..., "height": ..., "coordinateSpace": "pageSpace", "structure": "ocr-order|detect", "lines|detections": [] }] + "pages": [{ + "index": 0, + "width": 640, "height": 480, + "coordinateSpace": "pageSpace", + "structure": "ocr-order", + "lines": [{ "id": "L0", "text": "HELLO", "confidence": 0.99, "box": [4 points] }] + }] } ``` -- `recognize`: `pages[0].lines[]` → `{ id: "L0", text, confidence, box: [4 points] }` -- `detect`: `pages[0].detections[]` → `{ id: "D0", score, box: [4 points] }` -- `--format text`: recognized text only, one line per line +- `box` is 4 points in `pageSpace` (top-left origin, x right, y down, post-EXIF pixels) +- `detect` replaces `lines` with `detections[]` (`{ id, score, box }`) and sets `structure: "detect"` +- `--format text`: recognized text only, one line per line, no coordinates - `--format jsonl`: one page record per line (for streaming/batch) -## Choosing what to run - -| Goal | Command | -| --- | --- | -| Full text from an image | `recognize --format text` | -| Text + coordinates | `recognize --format json` | -| Just where text is (no text) | `detect` | -| Text in a specific area | `recognize --region x,y,w,h` | -| Engine/provider info | `info --model-info` | -| Quick version check | `info --version` | - ## Exit codes | Code | Meaning | Action | @@ -76,18 +117,10 @@ All `recognize`/`detect` output wraps in a `schemaVersion: 1` envelope: | 71 | Inference failure | Retry or report bug | | 72 | Internal error | Report bug | -## Failure handling - -- **Empty result**: No text found. Try `detect` first to see if any regions were detected, then `recognize --region` on specific areas. -- **Low confidence** (< 0.5): Do not present inferred text as OCR output. Always cite the actual `text` field and `confidence`. -- **Resource limit** (exit 69): Image too large. Use `--region` to process a sub-area. -- **Unsupported capability** (exit 67): Run `info --model-info` to check available providers. - ## Rules -1. Never fabricate OCR text. Only use the `text` field from results. If confidence is low, state it. -2. Cite coordinates when relevant. Box coordinates are in `pageSpace` (top-left origin, x right, y down, post-EXIF pixels). -3. Use `--schema-version 1` for reproducible output. Do not parse help text programmatically. -4. Prefer `detect` first if only locating text regions (faster, no recognition). -5. Use `--region` to avoid processing huge images unnecessarily. -6. Check exit codes before parsing stdout. Non-zero exit means stdout may be empty; read stderr for the error. +1. Never fabricate OCR text. Only use the `text` field from results. If confidence < 0.5, state it. +2. Cite coordinates when relevant. Box coordinates are in `pageSpace`. +3. Use `--schema-version 1` for reproducible output. Do not parse help text. +4. Prefer `detect` first on large images, then `recognize --region` on areas of interest. +5. Check exit codes before parsing stdout. Non-zero exit means stdout may be empty; read stderr.