Initial import of GPT-created project

Arturo R Montesinos · Arturo R Montesinos · commit d8772555c84d · 2025-10-11T20:34:42.000+02:00
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -0,0 +1,20 @@
+name: CI
+on:
+  push:
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e .
+          pip install pytest
+      - name: Tests
+        run: pytest -q
diff --git a/.gitignore b/.gitignore
@@ -0,0 +1,10 @@
+__pycache__/
+.pytest_cache/
+.venv/
+dist/
+build/
+*.egg-info/
+.coverage
+.DS_Store
+.idea/
+.vscode/
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -0,0 +1,15 @@
+repos:
+  - repo: https://github.com/psf/black
+    rev: 24.8.0
+    hooks:
+      - id: black
+  - repo: https://github.com/pycqa/ruff-pre-commit
+    rev: v0.6.3
+    hooks:
+      - id: ruff
+        args: [--fix]
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.10.0
+    hooks:
+      - id: mypy
+        additional_dependencies: []
diff --git a/LICENSE b/LICENSE
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/Makefile b/Makefile
@@ -0,0 +1,7 @@
+.PHONY: format lint test
+format:
+	pre-commit run --all-files || true
+lint:
+	ruff src tests
+test:
+	pytest -q
diff --git a/README.md b/README.md
@@ -0,0 +1,61 @@
+# TypeSketch — infer a type‑oriented YAML from JSON
+> A tiny CLI that reads JSON from stdin and emits a concise, type‑oriented YAML **TypeSketch** — perfect for docs, quick API archeology, and codegen stubs.
+
+![status-badge](https://img.shields.io/badge/status-experimental-blue) ![license](https://img.shields.io/badge/license-MIT-green)
+
+## Why
+When exploring unfamiliar APIs, we often need a *human‑readable* schema sketch rather than a full JSON Schema. TypeSketch summarizes field shapes, union types, and common formats (url, datetime, email, html-string) directly from example payloads.
+
+## Features
+- Infer scalar/union types and simple format hints
+- Expand arrays of objects as proper YAML lists (no angle‑brackets)
+- Merge multiple objects into a unified shape sorted by field popularity
+- Configurable array sampling, depth, and indentation
+- Zero-deps runtime (only Python stdlib)
+
+## Quickstart
+```bash
+pipx install .   # or: pip install -e .
+echo '{"id":1,"name":"X","tags":["a",2],"url":"https://x"}' | typesketch --root item
+```
+
+Output:
+```yaml
+item:
+  id: int
+  name: string
+  tags:
+    - string | int
+  url: url
+```
+
+## CLI
+```bash
+# Read JSON from stdin
+curl https://api.example.com/shops | typesketch
+
+# Use a custom root key and show stats
+cat data.json | typesketch --root Shops --stats
+
+# Treat single object as a one-item list
+cat item.json | typesketch --force-list
+```
+
+See `typesketch --help`.
+
+## Software Curator Principles
+- **Curation-first**: ADRs under `docs/adr/` document naming, heuristics, and trade-offs.
+- **Reproducible**: Unit tests and golden fixtures in `tests/`.
+- **Auditable**: Minimal, readable functions with explicit heuristics and TODOs for future refinement.
+- **Extendable**: Clear seams for adding new detectors (currency, color-hex, uuid, etc.).
+
+## Roadmap
+- [ ] JSON Schema / OpenAPI emitters
+- [ ] Presence/requiredness stats
+- [ ] Friendly collapse rules for very deep objects
+- [ ] YAML → Markdown renderer for docs sites
+- [ ] Pluggable detector registry
+- [ ] Streaming mode for very large inputs
+
+## License
+MIT — see `LICENSE`.
diff --git a/docs/adr/ADR-0001-name-and-scope.md b/docs/adr/ADR-0001-name-and-scope.md
@@ -0,0 +1,5 @@
+# ADR-0001: Name and Scope
+**Status**: Accepted
+
+We name the tool **TypeSketch**: a concise type‑oriented YAML derived from JSON samples.
+Scope: exploration and documentation; not a full validator. JSON Schema/OpenAPI output may be added later.
diff --git a/docs/adr/ADR-0002-language-and-packaging.md b/docs/adr/ADR-0002-language-and-packaging.md
@@ -0,0 +1,4 @@
+# ADR-0002: Language & Packaging
+**Status**: Accepted
+
+Python 3.9+ for a zero‑deps CLI (`typesketch`) installed via pip/pipx.
diff --git a/docs/adr/ADR-0003-heuristics.md b/docs/adr/ADR-0003-heuristics.md
@@ -0,0 +1,8 @@
+# ADR-0003: Inference Heuristics
+**Status**: Accepted
+
+- Scalar types: int, number, boolean, null, string.
+- Simple format hints: url, email, datetime, html-string.
+- Arrays: merge shapes of item objects; scalar arrays use union (`string | int`).
+- Key ordering: by frequency desc, then alpha as tiebreaker.
+- Limits: arrays are sampled (default 200). No deep recursion guards yet.
diff --git a/docs/curation/README.md b/docs/curation/README.md
@@ -0,0 +1,2 @@
+# Curation Notes
+This folder captures the Software Curator rationale behind architectural decisions and heuristics.
diff --git a/examples/poi.json b/examples/poi.json
@@ -0,0 +1,26 @@
+[
+  {
+    "id": 1,
+    "name": "Shop A",
+    "position": {
+      "lat": 1.0,
+      "lng": 2.0
+    },
+    "tags": [
+      "food",
+      "open"
+    ]
+  },
+  {
+    "id": 2,
+    "name": "Shop B",
+    "position": {
+      "lat": 3.0,
+      "lng": 4.0
+    },
+    "tags": [
+      "tech",
+      42
+    ]
+  }
+]
diff --git a/examples/run.sh b/examples/run.sh
@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+set -euo pipefail
+cat "$(dirname "$0")/poi.json" | typesketch --root POIs
diff --git a/pyproject.toml b/pyproject.toml
@@ -0,0 +1,20 @@
+[build-system]
+requires = ["setuptools>=64", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "typesketch"
+version = "0.1.0"
+description = "Emit a type-oriented YAML sketch from JSON"
+authors = [{name="Software Curator", email="maintainers@example.com"}]
+license = {text = "MIT"}
+readme = "README.md"
+requires-python = ">=3.9"
+dependencies = []
+
+[project.scripts]
+typesketch = "typesketch.cli:main"
+
+[tool.ruff]
+line-length = 100
+select = ["E","F","I"]
diff --git a/src/typesketch/__init__.py b/src/typesketch/__init__.py
@@ -0,0 +1 @@
+__all__ = ['infer', 'emit_yaml']
diff --git a/src/typesketch/cli.py b/src/typesketch/cli.py
@@ -0,0 +1,41 @@
+from __future__ import annotations
+import sys, json, argparse
+from typing import Any, Dict, List
+from .infer import merge_objects
+from .emit_yaml import emit_yaml
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description="Emit a TypeSketch (type-oriented YAML) from JSON on stdin.")
+    ap.add_argument('--root', help='Wrap output under this root key')
+    ap.add_argument('--force-list', action='store_true', help='Treat single-object input as a one-item list')
+    ap.add_argument('--array-samples', type=int, default=200, help='Max array items to sample')
+    ap.add_argument('--indent', type=int, default=2, help='Spaces per indent')
+    ap.add_argument('--stats', action='store_true', help='Emit key-frequency stats as YAML comments (TODO)')
+    args = ap.parse_args()
+
+    data = json.load(sys.stdin)
+
+    # Shape inference
+    if isinstance(data, list):
+        if data and isinstance(data[0], dict):
+            merged = merge_objects(data, sample_n=args.array_samples, max_array=args.array_samples)
+            shape = [merged]
+        else:
+            # for scalar arrays, show union type only
+            # promote to object with single key 'items' for readability
+            from .infer import guess_type
+            tset = set()
+            for it in data[:args.array_samples]:
+                tset |= guess_type(it)
+            ts = [t for t in tset if isinstance(t, str)]
+            shape = [{ 'items': f"array<{' | '.join(sorted(ts))}>" }]
+    elif isinstance(data, dict):
+        merged = merge_objects([data], sample_n=args.array_samples, max_array=args.array_samples)
+        shape = merged if not args.force_list else [merged]
+    else:
+        shape = {'value': type(data).__name__.lower()}
+
+    sys.stdout.write(emit_yaml(shape, root=args.root, indent=args.indent))
+
+if __name__ == '__main__':
+    main()
diff --git a/src/typesketch/detectors.py b/src/typesketch/detectors.py
@@ -0,0 +1,17 @@
+from __future__ import annotations
+import re
+
+RE_URL = re.compile(r'^https?://', re.I)
+RE_EMAIL = re.compile(r'^[^@\s]+@[^@\s]+\.[^@\s]+$')
+RE_ISO_DT = re.compile(r'^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}')
+
+def detect_string_format(s: str) -> str:
+    if RE_URL.match(s):
+        return "url"
+    if RE_EMAIL.match(s):
+        return "email"
+    if RE_ISO_DT.match(s):
+        return "datetime"
+    if "<" in s and ">" in s and "/" in s:
+        return "html-string"
+    return "string"
diff --git a/src/typesketch/emit_yaml.py b/src/typesketch/emit_yaml.py
@@ -0,0 +1,42 @@
+from __future__ import annotations
+from typing import Any, Dict, List, Union
+
+def _to_yaml(obj: Any, indent: int = 0, step: int = 2) -> List[str]:
+    sp = " " * (indent * step)
+    lines: List[str] = []
+    if isinstance(obj, dict):
+        for k, v in obj.items():
+            if isinstance(v, dict):
+                lines.append(f"{sp}{k}:")
+                lines.extend(_to_yaml(v, indent+1, step))
+            elif isinstance(v, list):
+                lines.append(f"{sp}{k}:")
+                for item in v:
+                    if isinstance(item, dict):
+                        lines.append(f"{sp}{' ' * step}-")
+                        lines.extend(_to_yaml(item, indent+2, step))
+                    else:
+                        lines.append(f"{sp}{' ' * step}- {item}")
+            else:
+                lines.append(f"{sp}{k}: {v}")
+        return lines
+    elif isinstance(obj, list):
+        for item in obj:
+            if isinstance(item, dict):
+                lines.append(f"{sp}-")
+                lines.extend(_to_yaml(item, indent+1, step))
+            else:
+                lines.append(f"{sp}- {item}")
+        return lines
+    else:
+        lines.append(f"{sp}{obj}")
+        return lines
+
+def emit_yaml(shape: Any, root: str | None = None, indent: int = 2) -> str:
+    lines: List[str] = []
+    if root:
+        lines.append(f"{root}:")
+        lines.extend(_to_yaml(shape, 1, indent))
+    else:
+        lines.extend(_to_yaml(shape, 0, indent))
+    return "\n".join(lines) + "\n"
diff --git a/src/typesketch/infer.py b/src/typesketch/infer.py
@@ -0,0 +1,78 @@
+from __future__ import annotations
+from collections import Counter, defaultdict
+from typing import Any, Dict, List, Tuple, Union, Iterable
+from .detectors import detect_string_format
+
+TypeMarker = Union[str, Tuple[str, dict]]  # e.g., "int" or ("object", {...}) or ("array<object>", {...})
+
+def _union_str(types: Iterable[str]) -> str:
+    ts = sorted(set(types))
+    return " | ".join(ts) if len(ts) > 1 else (ts[0] if ts else "unknown")
+
+def guess_type(value: Any, max_array: int = 200) -> set[TypeMarker]:
+    if value is None:
+        return {"null"}
+    if isinstance(value, bool):
+        return {"boolean"}
+    if isinstance(value, int):
+        return {"int"}
+    if isinstance(value, float):
+        return {"number"}
+    if isinstance(value, str):
+        return {detect_string_format(value)}
+    if isinstance(value, list):
+        items = value[:max_array]
+        if not items:
+            return {"array<empty>"}
+        tset: set[TypeMarker] = set()
+        obj_items = []
+        for it in items:
+            it_types = guess_type(it, max_array=max_array)
+            tset |= it_types
+            if isinstance(it, dict):
+                obj_items.append(it)
+        if obj_items:
+            merged = merge_objects(obj_items, max_array=max_array)
+            return {("array<object>", merged)}
+        else:
+            # collapse scalar unions inside array
+            leaf_types = [t for t in tset if isinstance(t, str)]
+            return {f"array<{_union_str(leaf_types)}>"}
+    if isinstance(value, dict):
+        return {("object", value)}
+    return {"unknown"}
+
+def merge_objects(objs: List[dict], sample_n: int = 200, max_array: int = 200) -> Dict[str, Any]:
+    key_counts: Counter[str] = Counter()
+    field_vals: defaultdict[str, list] = defaultdict(list)
+
+    for o in objs[:sample_n]:
+        if not isinstance(o, dict):
+            continue
+        for k, v in o.items():
+            key_counts[k] += 1
+            field_vals[k].append(v)
+
+    keys = sorted(field_vals.keys(), key=lambda k: (-key_counts[k], k))
+    merged: Dict[str, Any] = {}
+    for k in keys:
+        vals = field_vals[k]
+        local_types: set[str] = set()
+        object_shapes: list[dict] = []
+        array_object_shape: dict | None = None
+        for v in vals:
+            tset = guess_type(v, max_array=max_array)
+            for t in tset:
+                if isinstance(t, tuple) and t[0] == "object":
+                    object_shapes.append(t[1])
+                elif isinstance(t, tuple) and t[0] == "array<object>":
+                    array_object_shape = t[1]
+                elif isinstance(t, str):
+                    local_types.add(t)
+        if object_shapes:
+            merged[k] = merge_objects(object_shapes, sample_n=sample_n, max_array=max_array)
+        elif array_object_shape is not None:
+            merged[k] = [array_object_shape]  # list with merged item shape
+        else:
+            merged[k] = _union_str(local_types) if local_types else "unknown"
+    return merged
diff --git a/tests/test_cli_smoke.py b/tests/test_cli_smoke.py
diff --git a/tests/test_emit_yaml.py b/tests/test_emit_yaml.py

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	`+# Curation Notes`
	`2`	`+This folder captures the Software Curator rationale behind architectural decisions and heuristics.`