Skip to content

Inline datasets: CSV carried in the dashboard YAML (0.11.0) - #11

Merged
dankor merged 1 commit into
mainfrom
inline-datasets
Aug 21, 2026
Merged

dankor merged 1 commit into
mainfrom
inline-datasets

Conversation

@dankor

@dankor dankor commented Aug 21, 2026

Copy link
Copy Markdown
Owner

A chart's dataset: name now resolves two ways — a managed dataset in the store, or an inline one defined in the dashboard's own optional datasets: block:

datasets:
  order_data: |
    id,status,amount
    1,paid,42

The point is prototyping without leaving the editor: invent a table, chart it, iterate, with no upload step and nothing to clean up. A dashboard that carries its own data is also a single file you can hand to someone, and the assistant's prompt now teaches the block, so "mock me up some sales data and chart it" is one turn.

The block holds CSV and nothing else. A path or a header-only line is rejected with a message saying so, because either would otherwise parse as a valid one-column, zero-row table and fail much later complaining about a missing column.

An inline name overrides a managed one: the block is part of the file being rendered, so quietly reading someone else's stored data would be the surprising choice. Anything it doesn't define falls through to the store as before.

Each block converts to Parquet once and is cached by content checksum in the temp dir, so an unchanged block costs a stat, an edit lands on a fresh path, and two dashboards with the same sample data share a file. The write is a rename, which is atomic, so concurrent renders can't read a half-written file.

Also:

  • Dashboard.dataset_names() reports only managed names, so a chart on inline data doesn't pin a stored dataset against the portal's delete-guard.
  • The edit modal's column dropdowns resolve inline datasets, which would otherwise be empty for such a chart.
  • The starter dashboard is a file, not a 440-line string in app.py. It lives in demo/dashboards/, is read at import, and docker-compose maps that folder in as the demo path — so what you open in the browser is the file in git, nothing is copied or seeded, and _seed_demo_path is gone. The path is hardcoded and the file required: missing it is a broken checkout, so the import fails and says so. compose mounts the folder at both /app/demo (what the app reads) and /paths/demo (the path you browse), so an edit shows up in both without a rebuild; a test asserts any service mounting the source mounts the demo folder too, which is the drift that broke the container.
  • The starter dashboard carries its own data, and files/ is gone entirely. The sample now lives in the dashboard that uses it, so the Parquet (a derived binary in git), the CSV, the generator that made them, the Dockerfile COPY and the compose bind-mounts were all machinery around a file nothing read any more. The orders dataset is no longer seeded at startup in either mode.
  • The docs show YAML rather than Python. The examples in architecture.md and the README passed a .csv path, which has not worked since datasets moved to Parquet — scan() calls scan_parquet, so every one of those snippets failed on its first line. Rather than correct paths in Python that duplicated the YAML anyway, the product-facing examples are now the YAML a user actually writes. The README's "Use it as a library" section is dropped for now; the contributor guides (SKILL.md, PARAM_SKILL.md) keep their Python, since they show how to write chart and widget code.
  • Two dashboard examples in the docs did not parse: each placed one chart twice, which is only legal as a contiguous vertical span. Found by rendering every complete example in the docs rather than reading them; all three now parse and render. Stale claims went with them — the README and compose no longer promise a seeded orders dataset, which nothing creates any more.
  • The route tests carry their own data too. They had leaned on that seeded dataset, which meant they were passing off a leftover file on the developer's disk and would have failed on a clean checkout; the suite now runs with no dataset store present at all.

Minor bump: datasets: is optional, so existing dashboards are untouched.

523 tests pass.

A chart's `dataset:` name now resolves two ways — a managed dataset in the
store, or an inline one defined in the dashboard's own optional `datasets:`
block:

    datasets:
      order_data: |
        id,status,amount
        1,paid,42

The point is prototyping without leaving the editor: invent a table, chart it,
iterate, with no upload step and nothing to clean up. A dashboard that carries
its own data is also a single file you can hand to someone, and the assistant's
prompt now teaches the block, so "mock me up some sales data and chart it" is
one turn.

The block holds CSV and nothing else. A path or a header-only line is rejected
with a message saying so, because either would otherwise parse as a valid
one-column, zero-row table and fail much later complaining about a missing
column.

An inline name overrides a managed one: the block is part of the file being
rendered, so quietly reading someone else's stored data would be the surprising
choice. Anything it doesn't define falls through to the store as before.

Each block converts to Parquet once and is cached by content checksum in the
temp dir, so an unchanged block costs a stat, an edit lands on a fresh path, and
two dashboards with the same sample data share a file. The write is a rename,
which is atomic, so concurrent renders can't read a half-written file.

Also:
- `Dashboard.dataset_names()` reports only managed names, so a chart on inline
  data doesn't pin a stored dataset against the portal's delete-guard.
- The edit modal's column dropdowns resolve inline datasets, which would
  otherwise be empty for such a chart.
- The starter dashboard is a file, not a 440-line string in `app.py`. It lives
  in `demo/dashboards/`, is read at import, and docker-compose maps that folder
  in as the `demo` path — so what you open in the browser is the file in git,
  nothing is copied or seeded, and `_seed_demo_path` is gone. The path is
  hardcoded and the file required: missing it is a broken checkout, so the
  import fails and says so. compose mounts the folder at both `/app/demo` (what
  the app reads) and `/paths/demo` (the path you browse), so an edit shows up in
  both without a rebuild; a test asserts any service mounting the source mounts
  the demo folder too, which is the drift that broke the container.
- The starter dashboard carries its own data, and `files/` is gone entirely.
  The sample now lives in the dashboard that uses it, so the Parquet (a derived
  binary in git), the CSV, the generator that made them, the Dockerfile `COPY`
  and the compose bind-mounts were all machinery around a file nothing read any
  more. The `orders` dataset is no longer seeded at startup in either mode.
- The docs show YAML rather than Python. The examples in `architecture.md` and
  the README passed a `.csv` path, which has not worked since datasets moved to
  Parquet — `scan()` calls `scan_parquet`, so every one of those snippets failed
  on its first line. Rather than correct paths in Python that duplicated the
  YAML anyway, the product-facing examples are now the YAML a user actually
  writes. The README's "Use it as a library" section is dropped for now; the
  contributor guides (`SKILL.md`, `PARAM_SKILL.md`) keep their Python, since
  they show how to write chart and widget code.
- Two dashboard examples in the docs did not parse: each placed one chart twice,
  which is only legal as a contiguous vertical span. Found by rendering every
  complete example in the docs rather than reading them; all three now parse and
  render. Stale claims went with them — the README and compose no longer promise
  a seeded `orders` dataset, which nothing creates any more.
- The route tests carry their own data too. They had leaned on that seeded
  dataset, which meant they were passing off a leftover file on the developer's
  disk and would have failed on a clean checkout; the suite now runs with no
  dataset store present at all.

Minor bump: `datasets:` is optional, so existing dashboards are untouched.

523 tests pass.
@dankor
dankor merged commit b812cf8 into main Aug 21, 2026
2 checks passed
@dankor
dankor deleted the inline-datasets branch August 21, 2026 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant