Skip to content

QVAC-24055 doc: flag built-in RAG vector store as prototype-only - #4034

Open
BrunoCampana wants to merge 3 commits into
mainfrom
docs/rag-not-prod
Open

QVAC-24055 doc: flag built-in RAG vector store as prototype-only#4034
BrunoCampana wants to merge 3 commits into
mainfrom
docs/rag-not-prod

Conversation

@BrunoCampana

@BrunoCampana BrunoCampana commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Our built-in RAG vector store (the rag* workspace functions in @qvac/sdk) is intended for prototypes, not production — for production, the recommended path is an external vector DB (MongoDB, SQLite, LanceDB, Chroma). The docs never said so, so a developer reading the RAG guide or the API reference had no signal to look elsewhere.

How does it solve it?

Aligned message across every surface where that decision is made:

  • Guide (docs/website/content/docs/ai-capabilities/rag.mdx): callouts under Pipeline > Built-in vector store and above the Built-in vector store example. Also fixed the External vector DB pipeline bullet: it listed ragChunk() as a primitive, but no external example uses it.
  • Source of truth (packages/sdk/src/client/api/rag.ts): the 9 workspace rag* functions (ragChunk, ragIngest, ragSaveEmbeddings, ragSearch, ragReindex, ragDeleteEmbeddings, ragListWorkspaces, ragCloseWorkspace, ragDeleteWorkspace) carry a new @prototype TSDoc tag — the first-class inverse of @deprecated. Reaches IDE hover and .d.ts immediately.
  • API reference generator (docs/website/scripts/api-docs/*): parses the custom @prototype block tag at both per-overload and top-level surfaces (extract.ts + types.ts), and renders it as a > 🧪 **Prototype only**: <msg> blockquote in single-page.njk — visually parallel to the existing > ⚠️ **Deprecated**: <msg> treatment. Rendering test added.
  • Existing reference pages (index.mdx, v0.17.x.mdx, v0.16.x.mdx): rendered by hand in the same blockquote form so every version reads consistently today. v0.16.x and v0.17.x are frozen snapshots the pipeline does not regenerate; index.mdx gets the manual render until the next release regen.

Naming choice: @prototype over @alpha/@beta (the built-in path is not scheduled to graduate — the strategy is external DB) and over @experimental (industry-ambiguous). Custom tag, no TSDoc registration needed — follows the existing @overloadLabel precedent in this pipeline.

Expected outcome: every rag* workspace function now renders with a prominent 🧪 **Prototype only** blockquote in the API reference — same visual treatment as @deprecated, so readers instantly recognize a stability signal. The three currently versioned pages already show that render; future regenerations produce it automatically from the source.

Preview / test

See preview in our testing env:
https://qvac-docs-staging-test-1-di1kk.kinsta.page/ai-capabilities/rag/#pipeline

https://qvac-docs-staging-test-1-di1kk.kinsta.page/ai-capabilities/rag/#built-in-vector-store

https://qvac-docs-staging-test-1-di1kk.kinsta.page/reference/api/#ragchunk

- The built-in RAG vector store (workspace-backed rag* functions) is
  intended for prototypes, not production. The docs did not say so;
  this makes it explicit across every surface where a developer might
  decide to use it.

- Guide (docs/website/content/docs/ai-capabilities/rag.mdx):
  - Callout in Pipeline attached to the "Built-in vector store" bullet.
  - Reduced Callout at the top of Examples > Built-in vector store, so
    TOC-jumpers landing there also see the notice.
  - Fixed the "External vector DB" Pipeline bullet: ragChunk() was
    listed as available on the external path, but no external example
    (rag-mongodb, rag-sqlite, rag-lancedb, rag-chromadb) uses it — only
    the built-in path (rag-hyperdb) does.

- Source of truth (packages/sdk/src/client/api/rag.ts): prefixed the
  TSDoc description of all 9 workspace rag* functions (ragChunk,
  ragIngest, ragSaveEmbeddings, ragSearch, ragReindex,
  ragDeleteEmbeddings, ragListWorkspaces, ragCloseWorkspace,
  ragDeleteWorkspace) with the same notice. Reaches IDE hover today
  and auto-generated future reference pages. loadModel, embed and
  unloadModel are intentionally not annotated — the task scope
  excludes embedding.

- Past reference snapshots, which are frozen and not regenerated by
  the tooling: added the same one-line notice at the top of each rag*
  function section in v0.16.x, v0.17.x and index.mdx (v0.18.x latest).
  Text kept byte-identical to the TSDoc so every surface reads the
  same.
- Followup to the previous commit on this branch, which flagged the
  built-in RAG vector store as prototype-only. That commit used a
  bold prose prefix ("**Prototype only** — …") on every affected
  function. In the auto-generated API reference the prefix looks like
  ordinary description text and does not stand out; @deprecated has a
  dedicated visual treatment ("> ⚠️ **Deprecated**: …" blockquote) but
  its inverse — "not production grade, and not intended to graduate"
  — did not exist. This commit introduces that inverse as a first-
  class TSDoc tag and adopts it across the surfaces the previous
  commit touched.

- New custom @prototype block tag, mirroring @deprecated end-to-end:
  - types.ts: prototype?: string on ApiFunction and ApiOverload.
  - extract.ts: parse @prototype at two sites (per-overload and
    top-level), same shape as @deprecated. Custom tags are already
    consumed without registration in this pipeline (@overloadLabel
    precedent), so no typedoc.config.json change is needed.
  - single-page.njk: emit "> 🧪 **Prototype only**: {msg}"
    immediately after the @deprecated blockquote on both fn and ov
    surfaces. Header "Fields shown" line updated to list "prototype".

- Semantic choice: @prototype over @Alpha / @beta / @experimental.
  The RAG built-in path is not scheduled to graduate to production —
  the strategy is external vector DB — so maturity-gate tags
  (alpha/beta) would be misleading. @experimental is ambiguous in
  the industry ("may change" vs. "not for prod"). @prototype names
  the intent exactly.

- Source migration (packages/sdk/src/client/api/rag.ts): the 9
  workspace rag* functions move from the bold-prose prefix to the
  new tag ("@prototype The built-in …"). The message text is
  identical to the previous commit's; only the delivery mechanism
  changes. IDE hover, .d.ts, and the next full regen of index.mdx
  will now render the callout as a proper blockquote.

- Reference pages (index.mdx, v0.17.x.mdx, v0.16.x.mdx): converted
  the bold-prose prefix to the blockquote form the generator now
  produces ("> 🧪 **Prototype only**: The built-in …"), 9 × 3 = 27
  replacements. v0.16.x and v0.17.x are frozen snapshots that the
  pipeline does not regenerate; index.mdx would be regenerated on
  the next release cut. Aligning all three by hand keeps the reader
  experience consistent across the version switcher until then.

- Tests: added the mirror of the deprecated-callout snapshot check
  for @prototype in api-docs-rendering.test.ts, and updated the
  "Fields shown" line in the rendering snapshot. 247 tests pass
  (bun run test in docs/website). The one failing suite,
  tsdoc-completeness.test.ts, was already broken before this branch:
  TypeDoc's entry point moved from packages/sdk/index.ts to
  packages/sdk/src/index.ts in commit 4994117 ("chore[bc]: use
  @qvac/inference for SDK") and typedoc.config.json still points to
  the old path. Out of scope here.
@github-actions

Copy link
Copy Markdown
Contributor

Review Status

Current Status: ❌ PENDING
Approvals so far: none

Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member.

@github-actions

Copy link
Copy Markdown
Contributor

License compliance — clean

No new dependency license findings in this PR.

Warn-only (shadow) mode — this check does not block merges yet.

Updated automatically by the canonical license compliance workflow.

NOTICE presence (advisory)

Missing NOTICE (advisory, does not block):

  • ./.github/actions/release-merge-guard
  • ./docs/website
  • ./packages/ggml-coload-smoke
  • ./packages/fabric/test/integration
  • ./packages/inference-addon-cpp/mobile
  • ./packages/sdk/e2e
  • ./packages/llm-llamacpp/benchmarks/performance
  • ./packages/llm-llamacpp/benchmarks/server
  • ./packages/vla-ggml/sim/server
  • ./packages/embed-llamacpp/benchmarks/performance
  • ./packages/embed-llamacpp/benchmarks/server
  • ./packages/asr-ggml/benchmarks/server

@BrunoCampana
BrunoCampana requested review from GSServita and removed request for Victor-Rodzko August 24, 2026 19:19

@opaninakuffo opaninakuffo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flags the built-in RAG store as prototype-only in the guide, the 9 rag* TSDocs, and a new @prototype path in the API-docs generator. generator plumbing matches @deprecated (extract + njk + rendering test). two gaps: ragChunk isn't the store, and @overloadLabel's real precedent is packages/sdk/tsdoc.json (this tag isn't registered there).

Comment thread packages/sdk/src/client/api/rag.ts
Comment thread docs/website/scripts/api-docs/extract.ts
- Review feedback on PR #4034 from @opaninakuffo. Two changes: fix an
  overreach in the @prototype scope, and register the custom tag with
  the TSDoc validator.

- Drop @prototype from ragChunk (rag.ts) and from the matching
  blockquote under `### ragChunk` in the three versioned reference
  pages (index.mdx, v0.17.x.mdx, v0.16.x.mdx). Rationale: ragChunk is
  a pure text splitter — no workspace, no HyperDB, no store — its
  backend handler is LLMChunkAdapter. The @prototype callout is
  specifically about the built-in vector store; attaching it to
  ragChunk was a category error that implied the chunker itself is
  unstable. Tag stays on the other 8 workspace rag* functions.

- Register @prototype in packages/sdk/tsdoc.json next to
  @overloadLabel (both tagDefinitions and supportForTags). Corrects
  the earlier commit's claim that this pipeline "consumes custom tags
  without registration": the @overloadLabel precedent is in fact
  registered — in tsdoc.json (which TypeDoc auto-loads from the SDK's
  tsconfig directory), not in typedoc.config.json. Without this,
  every `docs:generate-api` run would emit an "Unknown block tag
  @prototype" warning per annotated function (8 per regen). No
  behaviour change; noise-only fix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants