Skip to content

fix(rag): make OCI embedding dimension + input_type deterministic (closes #292)#293

Merged
fede-kamel merged 2 commits into
mainfrom
fix/292-oci-embeddings-output-dimensions
May 30, 2026
Merged

fix(rag): make OCI embedding dimension + input_type deterministic (closes #292)#293
fede-kamel merged 2 commits into
mainfrom
fix/292-oci-embeddings-output-dimensions

Conversation

@fede-kamel
Copy link
Copy Markdown
Contributor

Closes #292.

The report

RAG queries against a cohere.embed-v4.0-indexed Oracle table failed with:

ORA-51803: Vector dimension count must match the dimension count specified in
the column definition (expected 1536 dimensions, specified 1024 dimensions).

The original hypothesis was that OCIEmbeddings.embed_query() hardcoding
input_type="SEARCH_QUERY" caused v4 to return 1024-dim query vectors against a
1536-dim (SEARCH_DOCUMENT-indexed) table.

What the investigation actually found

I reproduced against live OCI Generative AI (us-chicago-1) on two separate tenancies.
input_type does not change the output dimension — that hypothesis is wrong:

model SEARCH_DOCUMENT SEARCH_QUERY
cohere.embed-v4.0 1536 1536
cohere.embed-english-v3.0 1024 1024
cohere.embed-multilingual-v3.0 1024 1024
cohere.embed-english-light-v3.0 384 384

The real lever is Cohere v4's Matryoshka output_dimensions, which Locus never
set. v4 returns exactly the requested size, and OCI's server-side default applies when
it's omitted:

v4 default (no output_dimensions): 1536
v4 output_dimensions=256:  256
v4 output_dimensions=512:  512
v4 output_dimensions=1024: 1024   ← the reported 1024
v4 output_dimensions=1536: 1536

So the ORA-51803 happens when one path (an ingestion pipeline, an older server-side
default, or a foreign OracleVS writer) lands at one output_dimensions and another
path lands at a different one. Because Locus left the dimension to an implicit OCI
default, the stored column and query vectors could silently diverge. input_type was a
red herring — but the code did genuinely hardcode it, so that's fixed too.

Fix

Primary — deterministic dimension (output_dimensions):

  • Add OCIEmbeddingConfig.output_dimensions (default None). It's forwarded to
    EmbedTextDetails on every embed path through a shared _build_embed_details helper,
    and only when set, so non-Matryoshka models (v3, fixed dims) never receive the
    field.
  • config.dimension now prefers output_dimensions, so the vector store sizes its
    column to exactly the embedding dimension being produced.

Secondary — honor input_type (the original hardcode + cleanup):

  • embed_query() no longer hardcodes SEARCH_QUERY; it reads a new
    OCIEmbeddingConfig.query_input_type (default SEARCH_QUERY).
  • embed_documents() reads input_type from config instead of hardcoding
    SEARCH_DOCUMENT.
  • Removed the dead original_type local and the incorrect "Can't modify frozen config"
    comment in embed_query (the config is a plain BaseModel, never frozen).

Tests

Unittests/unit/test_rag_embeddings_oci.py:

  • output_dimensions omitted by default (never attached for non-Matryoshka models)
  • output_dimensions forwarded on all three paths (embed/embed_query/embed_documents)
  • config.dimension reflects output_dimensions (wins over the model hint)
  • query_input_type / input_type are honored, not hardcoded
  • config defaults assert the two new fields

Full unit suite: 5023 passed, 0 failed.

Integrationtests/integration/rag/test_oci_embeddings.py: 9 passed against live
OCI GenAI.

Live verification of the fix (real cohere.embed-v4.0, us-chicago-1):

config embed_documents embed_query config.dimension
default (unpinned) 1536 1536 1536
output_dimensions=1024 1024 1024 1024

Both paths and the column dimension stay in lockstep — the ORA-51803 divergence can no
longer arise from an implicit server default.

…290)

Bumps locus-sdk to 0.2.0b25. Fixes a Pydantic v2 regression where an
explicit wallet_password="" (the python-oracledb auto-login / cwallet.sso
idiom) was silently dropped by a truthy SecretStr guard, forcing the
encrypted ewallet.pem path and failing at connect with DPY-6005 /
OSError: [Errno 22]. The guard is now `is not None` across all seven
Oracle pool builders (closes #289).

Signed-off-by: Federico Kamelhar <federico.kamelhar@oracle.com>
…oses #292)

OCIEmbeddings.embed_query() hardcoded input_type="SEARCH_QUERY" (ignoring
config) and embed_documents() hardcoded "SEARCH_DOCUMENT"; embed_query also
carried a dead `original_type` local and a misleading "frozen config" comment.

More importantly, the real dimension-mismatch trigger behind the report is
Cohere v4's Matryoshka output_dimensions, which Locus never set. cohere.embed-
v4.0 returns exactly the requested size (256/512/1024/1536) and OCI's
server-side default (1536 today) applies otherwise — so a table indexed at one
output_dimensions and queried at another raises ORA-51803. input_type does NOT
change the output dimension (verified live: v4 returns 1536 for both
SEARCH_QUERY and SEARCH_DOCUMENT; v3 returns 1024 for both).

Changes:
- Add OCIEmbeddingConfig.output_dimensions (default None). Forwarded to
  EmbedTextDetails on every embed path via a shared _build_embed_details
  helper, and only when set, so non-Matryoshka models never see the field.
- config.dimension now prefers output_dimensions, so the vector column is
  sized to match the pinned embedding dimension.
- Add OCIEmbeddingConfig.query_input_type (default SEARCH_QUERY) and read
  input_type from config in embed_documents; removes the hardcodes and the
  dead/misleading code in embed_query.

Tests: query_input_type / input_type configurability, output_dimensions
omitted-by-default + forwarded on all three paths, and config.dimension
reflecting output_dimensions.

Signed-off-by: Federico Kamelhar <federico.kamelhar@oracle.com>
@oracle-contributor-agreement oracle-contributor-agreement Bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label May 30, 2026
@fede-kamel fede-kamel merged commit c83e08c into main May 30, 2026
10 checks passed
@fede-kamel fede-kamel deleted the fix/292-oci-embeddings-output-dimensions branch May 30, 2026 13:02
fede-kamel added a commit that referenced this pull request May 30, 2026
…ype fix (#293) (#294)

Bumps locus-sdk to 0.2.0b26. Makes the OCI embedding dimension deterministic
via a new output_dimensions config knob (Cohere v4 Matryoshka), so a vector
column and its query vectors can no longer diverge into ORA-51803; also honors
configured input_type / query_input_type instead of hardcoding it (closes #292).

Signed-off-by: Federico Kamelhar <federico.kamelhar@oracle.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

1 participant