Skip to content

docs: simplify demo and clarify supported audiences #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
snchimata merged 2 commits into from
Nov 12, 2026
Merged

docs: simplify demo and clarify supported audiences #49

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
06dfb36
docs: simplify demo and clarify supported audiences
snchimata Nov 12, 2026
b09b81a
docs: shorten Python SDK heading
snchimata Nov 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 4 additions & 34 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,47 +27,17 @@

![Terminal demo: Python fit() code and verified 2,366 to 1,376 token savings from the first frame, followed by Core-first processing and optional Tokenfold Select. Select illustration only.](https://raw.githubusercontent.com/snchimata/tokenfold/a5aafaf9fb48047ed03e42e9cc379de49118b25d/docs/assets/tokenfold-demo.gif)

<details>
<summary><strong>Demo scope and regeneration</strong></summary>

The 13-second FIT -> CORE -> SELECT loop returns directly to the code, with no
closing slide. It opens on the executed Python SDK `fit()` example and its
verified result: explicit folding reduces already-compact JSON from 2,366 to
1,376 tokens (**41.8%**), meeting a 1,500-token budget without dropping any of
the 30 records. The compact baseline exceeds that same budget.

The following Core frame starts from the formatted API-response fixture:
3,812 to 1,376 tokens (**63.9%**), with exact JSON-value recovery. These are
different input representations, not interchangeable SDK and CLI baselines.
The workflow builds the Core baseline first; Select requires a still-over-budget,
eligible request, an approved runtime and explicit per-fit opt-in. Select is an
**illustration, not live inference**, and remains experimental and unqualified.
Fixture savings are not universal, and reversible data is not model-answer proof.

The [renderer](scripts/render-demo.py) requires Pillow, a locally built CLI and
the current SDK installed in a Python >=3.12 environment. It checks the fixture
hash, Core counts, decode equality and SDK receipt before rendering:

```bash
python scripts/render-demo.py --sdk-python /path/to/sdk-env/python
```

The [VHS tape](docs/tokenfold-demo.tape) is a separate, Core-only terminal
recording and writes to `tmp/tokenfold-core-terminal.gif`.

</details>

## Table of contents

[Python quickstart](#quick-start) | [Python fit workflow](#native-sdk-fitting) | [Select](#tokenfold-select) | [Benchmark highlights](#benchmark-highlights) | [Other integrations](#other-integrations) | [How it works](#how-it-works) | [Core](#tokenfold-core) | [Full results](#measured-results) | [Extended tooling](#extended-tooling) | [Safety and auditability](#safety-and-auditability) | [Reproduce the results](#reproduce-the-results) | [Contributing](#contributing) | [License](#license)

> **Built for:** Python application developers and AI teams fitting JSON, tool
> definitions, and grouped context into LLM requests. TypeScript, coding agents,
> CLI, proxy, and Rust integrations use the same underlying Core engine.
> **Built for:** Python, TypeScript, and Rust developers and AI teams fitting
> JSON, tool definitions, and grouped context into LLM requests. Coding-agent,
> CLI, and proxy integrations use the same underlying Core engine.

## Quick start

### Published Python SDK
### Python SDK

Python >=3.12. Install the current SDK release; the base workflow needs no
repository checkout, model, GPU, or ML framework:
Expand Down
4 changes: 2 additions & 2 deletions python-tests/test_readme_layout.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,15 +56,15 @@ def test_fit_onboarding_and_migration_are_visible(self):
self.assertNotIn('Start model-free with Core.', intro)

def test_python_first_release_tracks_are_separate(self):
published = self.text.split('### Published Python SDK\n', 1)[1]
published = self.text.split('### Python SDK\n', 1)[1]
published, fitting = published.split('### Python fit workflow\n', 1)
self.assertIn('python -m pip install "tokenfold==0.6.0"', published)
self.assertIn('from tokenfold import Data, fit', fitting)
self.assertIn('from tokenfold.lowlevel import InputFormat, compress, decode', self.text)
self.assertNotIn('Path(', published)
self.assertIn('Python >=3.12', published)
self.assertIn('representation="folded"', fitting)
self.assertLess(self.text.index('### Published Python SDK'), self.text.index('```js'))
self.assertLess(self.text.index('### Python SDK'), self.text.index('```js'))
self.assertLess(self.text.index('## Tokenfold Select'), self.text.index('## Benchmark highlights'))
self.assertLess(self.text.index('## Benchmark highlights'), self.text.index('## Other integrations'))
self.assertNotIn('google.com/search', self.text)
Expand Down
Loading