Skip to content

docs: add an Open items list under Experiments - #36

Merged
yodablocks merged 1 commit into
masterfrom
docs/open-items
Oct 5, 2026
Merged

yodablocks merged 1 commit into
masterfrom
docs/open-items

Conversation

@yodablocks

Copy link
Copy Markdown
Owner

Docs only: a short Open items list at the end of the Experiments section, so someone resuming this work (including me) can see where to pick it up. The notes were previously scattered across the experiment READMEs and outside this repo.

Five items, none started in the repo:

  1. Scale test: begun outside this repo with EU legal acts from Cellar and stopped; embedding runs at about 5 chunks per second on the Apple-GPU Mac used here, which overheats under sustained load, so only about a 4x step is realistic there.
  2. Local re-ranker: trails Jev by about 5 questions; the local LLM judge was stopped before a result, so the wording idea is untested.
  3. Jev's run-to-run variation: three passes only.
  4. The 28 first-set questions drafted from the searched chunks.
  5. Cross-document answers: not covered by any experiment.

The figures repeat what the experiment READMEs already say (5.2 chunks per second, about 17,500 acts, about 250,000 chunks in total, 5 questions behind). No link points to the scale-test repo because it is local only. No code changes.

@yodablocks
yodablocks merged commit 3185332 into master Oct 5, 2026
4 checks passed
@yodablocks
yodablocks deleted the docs/open-items branch October 5, 2026 04:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant