Simple Serverless RAG

A serverless RAG (Retrieval-Augmented Generation) system using OpenSearch for vector search and Claude/Gemini for answer generation.

How it works

Two modes: local (develop and test) and AWS (production serverless).

Architecture

Ingestion flow

1. A text document is sent to the system
2. The text is split into chunks
3. Each chunk becomes an embedding (a numeric vector representing the text's meaning)
4. Chunks + vectors are stored in OpenSearch

Query flow (RAG)

1. The user asks a question in the chat
2. The question is converted to an embedding
3. OpenSearch finds the semantically closest chunks
4. The LLM (Claude via Bedrock or Gemini) generates an answer based on those chunks
5. The answer is displayed together with the sources used

---

Running locally

Prerequisites

• Docker
• Python 3.12+
• Gemini API key (GEMINI_API_KEY) in .env

1. Configure the environment

Copy the example file and fill in the variables:

cp .env.example .env
# edit .env and set your GEMINI_API_KEY

2. Install dependencies

make install

Creates the .venv and installs all dependencies (OpenSearch, fastembed, Streamlit, etc.).

3. Start local infrastructure

make local-up

Starts OpenSearch (port 9200) and ElasticMQ (port 9324) via Docker.

4. Ingest documents

make ingest FILES="documents/*.txt"

Supports .txt and .md. For a single file:

make ingest FILES="documents/my-file.txt"

The doc_id is the filename without extension. Re-ingesting the same file replaces previous chunks.

5. Run the chat

make ui

Opens the chat at http://localhost:8501.

---

Environment variables (.env)

Variable	Description	Default
GEMINI_API_KEY	Gemini API key	required
GEMINI_MODEL_ID	Gemini model to use	gemini-2.5-flash-lite
OPENSEARCH_ENDPOINT	OpenSearch endpoint	http://localhost:9200
OPENSEARCH_INDEX	Index name	rag_chunks_local
OPENSEARCH_AUTH	Auth mode (local or sigv4)	local
EMBEDDING_PROVIDER	Embedding provider (fastembed, local, bedrock, mock)	fastembed
EMBEDDING_DIM	Vector dimension	384
LLM_PROVIDER	LLM provider (gemini, bedrock, or mock)	bedrock

---

Available commands

make install                         # create .venv and install dependencies
make local-up                        # start OpenSearch + ElasticMQ
make local-down                      # stop containers
make ingest FILES="documents/*.txt"  # index documents
make ui                              # open the chat
make test                            # run unit tests
make lint                            # check code with ruff
make format                          # format code with ruff

---

Embedding providers

Provider	When to use
fastembed	Local, no cloud, real semantics (recommended for dev)
local	Local sentence-transformers, requires PyTorch
bedrock	AWS Bedrock (Titan Embeddings), requires AWS credentials
mock	Deterministic random vectors, for testing the pipeline only

The default fastembed model is BAAI/bge-small-en-v1.5 (~67MB, downloaded automatically on first run).

---

Project structure

.
├── ask/              # Lambda: receives question, searches chunks, generates answer
├── ingest/           # Lambda: receives document and enqueues to SQS
├── ingest_worker/    # Lambda: processes queue, generates embeddings and indexes
├── query/            # Lambda: vector search without answer generation
├── shared/           # Shared layer: OpenSearch client + embeddings
├── ui/               # Streamlit chat (local interface)
├── script/           # Utility scripts (local file ingestion)
├── documents/        # Sample documents to ingest
├── tests/            # Unit and integration tests
├── template.yaml     # SAM template (AWS deploy)
└── docker-compose.yml

---

AWS deploy (production)

The project is serverless via AWS SAM: API Gateway + Lambda + SQS + OpenSearch Serverless (AOSS) + Bedrock.

make build
make deploy

On AWS, embeddings are generated by Bedrock Titan Embeddings V2 and answers by Claude via Bedrock.

To use the AWS deploy, set OPENSEARCH_AUTH=sigv4 and point OPENSEARCH_ENDPOINT to the AOSS endpoint.

Deploy evidence