Skip to content

fix: Ensure unique chunk ids#67

Merged
MichalSteczko merged 7 commits into
mainfrom
fix/fix_unique_chunk_ids
May 29, 2026
Merged

fix: Ensure unique chunk ids#67
MichalSteczko merged 7 commits into
mainfrom
fix/fix_unique_chunk_ids

Conversation

@MichalSteczko
Copy link
Copy Markdown
Collaborator

Description

We had to add additional portion of information during chunk index creation to prevent hash conflicts.

Motivation

PG vector does not support the same chunk ID's within the batch and throws an error.

Changes

  • there is a probability near 0 that the chunk index will be the same within the batch
  • unit test cases have been created

Testing

Manual testing with remote OGX server and PG vector db.

Checklist

  • Tests added/updated
  • Code follows style guide
  • All checks passing

Signed-off-by: MichalSteczko <msteczko@redhat.com>
Signed-off-by: MichalSteczko <msteczko@redhat.com>
Signed-off-by: MichalSteczko <msteczko@redhat.com>
Signed-off-by: MichalSteczko <msteczko@redhat.com>
Comment thread ai4rag/rag/vector_store/ogx.py Outdated
Comment thread ai4rag/rag/vector_store/ogx.py Outdated
Comment thread ai4rag/rag/vector_store/ogx.py Outdated
Comment thread ai4rag/rag/vector_store/ogx.py Outdated
Comment thread ai4rag/rag/vector_store/ogx.py Outdated
Comment thread ai4rag/rag/vector_store/ogx.py Outdated
Comment thread ai4rag/rag/vector_store/ogx.py Outdated
Comment thread ai4rag/rag/vector_store/ogx.py Outdated
Comment thread ai4rag/rag/vector_store/ogx.py Outdated
Signed-off-by: MichalSteczko <msteczko@redhat.com>
Signed-off-by: MichalSteczko <msteczko@redhat.com>
@MichalSteczko MichalSteczko merged commit 7ede0c2 into main May 29, 2026
4 checks passed
@MichalSteczko MichalSteczko deleted the fix/fix_unique_chunk_ids branch May 29, 2026 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants