A hands-on, production-shaped reference application that explores the full surface of Spring AI on Java 25 and Spring Boot 3.5. It brings together multi-provider chat, retrieval-augmented generation (RAG), multimodality, tool-calling agents with human-in-the-loop approval, the Model Context Protocol (MCP), and a full observability stack.
Status: learning / reference project. It is intentionally broad — each package is a self-contained demonstration of one Spring AI capability, wired so the pieces can be combined.
- Features
- Architecture
- Tech Stack
- Project Layout
- Prerequisites
- Configuration & Secrets
- Running Locally
- API Overview
- RAG Pipeline
- The Security-Review Agent
- Observability
- Security & Prompt-Injection Defenses
- Testing with the Postman Collection
- Notes & Known Rough Edges
- License
Chat & Providers
- Pluggable providers behind Spring AI's
ChatClient: OpenAI, Google Vertex AI Gemini, Hugging Face (dedicated inference endpoints), Ollama (local), and the Docker Model Runner (OpenAI-compatible local endpoint). - Streaming responses, structured output (POJO /
List/Mapbinding), and a "raw" OpenAI Java client comparison. - Conversational memory two ways: JDBC-backed message-window memory and
semantic memory backed by a pgvector
VectorStore.
Retrieval-Augmented Generation (RAG)
- A simple path using
QuestionAnswerAdvisorover an in-memorySimpleVectorStore. - An advanced path using
RetrievalAugmentationAdvisorwith:- domain-synonym query transformation,
- multi-query expansion,
- pgvector similarity retrieval,
- Cohere reranking, neighbour-chunk stitching, and citation-header post-processing.
- PDF ingestion (page/paragraph readers) plus a filesystem watcher that ingests new PDFs dropped into a directory at runtime.
- An "internal Q&A over docs" scenario and a "mini employee handbook" scenario.
Multimodality
- Image → text: product-image description.
- Text → image: marketing-asset / campaign image generation with DALL·E 3.
- Speech → text: meeting-notes transcription (Whisper) with structured summarization.
- Text → speech: ticket-status voice responses (TTS).
Agents & Tools
- A Security-Review Agent that plans and executes a multi-step review of a system, calling tools for: threat-model diagram extraction (draw.io / image), RAG over internal security docs, web/OWASP lookups (via Google Vertex AI Search), and a security-posture lookup.
- Human-in-the-loop checkpoints (approve diagram extraction, approve final report), with durable review state persisted to Postgres.
- A vision variant and a deterministic chain-workflow variant of the agent.
- MCP: the main app is an MCP client; the
posture-serviceis a standalone MCP server exposing posture tools.
Cross-cutting
- Custom
ChatClientadvisors: system-prompt injection, validation, error-wrapping,SafeGuardAdvisor(sensitive-term guard), content sanitization, and a simple logger. - Full observability: Micrometer + Prometheus metrics, OpenTelemetry tracing to Jaeger/Tempo, Loki/Promtail logs, and provisioned Grafana dashboards.
┌──────────────────────────────────────────────┐
HTTP / Postman ───▶ │ gen-ai-java-spring (port 8081) │
│ │
│ Chat ── RAG ── Multimodality ── Agents │
│ │ │ │ │ │
│ │ │ │ ├── Tools: diagram / rag / web / posture
│ │ │ │ └── MCP client ─────────────┐
│ ▼ ▼ ▼ │
│ ChatClient (OpenAI / Vertex / HF / Ollama / DMR) │
└──────┬───────────────────────────────┬───────────────────────┘
│ │
┌────────▼────────┐ ┌─────────▼──────────┐
│ Postgres + │ │ posture-service │
│ pgvector (5433) │ │ MCP server (8082) │
│ · rag store │ └────────────────────┘
│ · chat memory │
└─────────────────┘
Observability: Micrometer → Prometheus (9090) · OTLP → Jaeger (16686) ·
Logs → Loki (3100)/Promtail · Dashboards → Grafana (3001)
| Area | Choice |
|---|---|
| Language / build | Java 25, Maven, Spring Boot 3.5.8 |
| AI framework | Spring AI 1.1.0 (BOM-managed) |
| Providers | OpenAI, Vertex AI Gemini, Hugging Face, Ollama, Docker Model Runner |
| Vector store | PostgreSQL 16 + pgvector (HNSW, cosine, 1536-dim) |
| Reranking | Cohere rerank-english-v3.0 |
| Memory | Spring AI JDBC chat-memory + vector-store chat-memory |
| Agent protocol | Model Context Protocol (MCP) client + server |
| Observability | Micrometer, Prometheus, OpenTelemetry, Jaeger, Tempo, Loki, Grafana |
| Misc | Lombok, Apache HttpClient 5, JTokkit (token counting), PDFBox reader |
.
├── pom.xml # main app (gen-ai-java-spring)
├── src/main/java/com/genai/java/spring
│ ├── config/ # AIProviderConfig — all ChatClient/VectorStore beans
│ ├── chat/ # provider controllers, memory, structured output, jailbreak demo
│ ├── multimodality/ # image/speech/text-to-* controllers
│ ├── rag/ # simple + advanced RAG, rerank, ingestion, PDF watcher
│ └── aiagent/ # security-review agent, tools, MCP, approvals, state
├── src/main/resources
│ ├── application.yml # central configuration
│ ├── templates/*.st # StringTemplate prompt templates
│ ├── rag/ # sample PDFs + employee-handbook docs
│ └── static/ # sample product images, meeting-notes.mp3
├── posture-service/ # standalone Spring Boot MCP server (port 8082)
├── docker/
│ ├── postgres/pgvector.yml # Postgres + pgvector (port 5433)
│ └── observability/ # Prometheus, Grafana, Loki, Promtail, Tempo, Jaeger
├── ollama/ # custom Mistral Modelfile + manifest
├── huggingface/huggingface.sh # shared HF inference API example
└── gen-ai-with-java-spring.postman_collection.json
- JDK 25 (the build targets
--release 25) - Maven 3.9+
- Docker / Docker Desktop (for Postgres + observability)
- API keys as needed (see below). Only the keys for the providers you actually call are required.
- (optional) Ollama for local models, Cohere key for reranking, a GCP project for Vertex AI Gemini / Vertex AI Search.
All secrets are read from environment variables — nothing sensitive is committed. Export what you need before running:
export OPENAI_API_KEY=sk-... # chat, embeddings, DALL·E, Whisper, TTS
export HUGGINGFACE_API_KEY=hf_... # Hugging Face inference endpoints
export COHERE_API_KEY=... # RAG reranking
export GCP_PROJECT=... # Vertex AI Search (web tool)
export DATASTORE_ID=... # Vertex AI Search data store
# Vertex AI Gemini uses Application Default Credentials (gcloud auth application-default login)Key knobs live in src/main/resources/application.yml:
app.ai.provider— default provider for shared endpoints.app.rag.*— chunking, top-k, similarity threshold, query-expansion count, rerank settings.spring.ai.*— per-provider model names (e.g.gpt-4o-mini,gemini-2.0-flash,mistral:7b).spring.datasource.*— points at Postgres onlocalhost:5433(matches the compose file).
Heads-up — machine-specific paths. A few settings currently point at absolute local directories (
app.agent.upload-dir,app.rag.pdf.pathunder/Users/Shared/...) and the Vertexproject-idis hard-coded. Update these to your own paths/project before running the agent and filesystem-watcher features. See Notes & Known Rough Edges.
docker compose -f docker/postgres/pgvector.yml up -dBrings up pgvector/pgvector:pg16 on host port 5433. The vector tables (rag_vector_store,
vector_store) and chat-memory schema are auto-created on first run (initialize-schema: true).
docker compose -f docker/observability/observability-compose.yml up -d| Service | URL |
|---|---|
| Grafana | http://localhost:3001 |
| Prometheus | http://localhost:9090 |
| Jaeger UI | http://localhost:16686 |
| Loki | http://localhost:3100 |
Grafana ships with provisioned datasources and two dashboards (AI metrics, AI logs).
ollama pull mistral:7b
# optional: build the custom email-drafting model
ollama create custom-mistral-model -f ./ollama/custom-mistral-modelfile
ollama run custom-mistral-modelSee ollama.md for how the Modelfile is layered on top of the base model.
mvn spring-boot:run
# or
mvn clean package && java -jar target/gen-ai-java-spring-0.0.1-SNAPSHOT.jarThe app starts on http://localhost:8081.
The agent's posture tool calls a separate MCP server:
cd posture-service
mvn spring-boot:run # starts on http://localhost:8082Base URL: http://localhost:8081. A full Postman collection
is included.
Chat
| Method & Path | Description |
|---|---|
POST /api/openai/chat |
OpenAI chat (+ streaming, structured) |
POST /api/vertexai/chat |
Google Vertex AI Gemini |
POST /api/huggingface/chat |
Hugging Face inference endpoint |
POST /api/ollama/chat |
Local Ollama model |
POST /api/docker-model-runner/chat |
Docker Model Runner (OpenAI-compatible) |
POST /api/common/chat |
Provider-agnostic entry point |
POST /api/orders/chat |
Tool-calling + chat memory demo |
Multimodality
| Method & Path | Description |
|---|---|
POST /api/products |
Image → text (product description) |
POST /api/marketing-assets/... |
Text → image (DALL·E 3) |
POST /api/meeting-assistant/... |
Speech → text + summarization (Whisper) |
POST /api/tickets/... |
Text → speech (TTS) |
RAG
| Method & Path | Description |
|---|---|
POST /api/internal-qa |
Advanced RAG over internal PDFs |
POST /api/employee-handbook |
RAG over the mini employee handbook |
POST /api/it-support |
In-prompt-context retrieval demo |
Agent
| Method & Path | Description |
|---|---|
POST /api/security-review |
Kick off a security review |
POST /api/security-review/{id}/ask |
Follow-up question in a review |
POST /api/security-review/{id}/human-approve-diagram-extract |
Approve extracted threat model |
POST /api/security-review/{id}/human-approve-final-report |
Approve the final report |
GET /api/security-review/{id} |
Fetch review state |
The advanced pipeline is assembled in
AIProviderConfig as a
RetrievalAugmentationAdvisor:
query
└─ DomainSynonymTransformer # expand PTO → "paid time off", MFA → "multi-factor auth", …
└─ MultiQueryExpander # generate N paraphrased queries (LLM, temperature 0)
└─ VectorStoreDocumentRetriever # pgvector top-k + similarity threshold
└─ RerankPostProcessor # Cohere rerank-english-v3.0, keep top-n
└─ NeighbourStitchPostProcessor # stitch adjacent chunks for context
└─ CitationHeaderPostProcessor # prepend source citations
└─ ContextualQueryAugmenter # (disallows empty context)
Ingestion (RagIngestionService + PdfWatcherService) reads PDFs, splits them with a
TokenTextSplitter, embeds with OpenAI, and upserts into the rag_vector_store table. Drop a new
PDF into the configured watch directory and it is ingested without a restart.
aiagent/ implements an agent that reviews a system design for security issues:
- Plan the review (
Plan/PlanSteprecords). - Extract a threat-model diagram from an uploaded draw.io / image file (
DiagramTools). - Pause for human approval of the extracted model.
- Gather evidence via tools — internal-docs RAG (
RagTools), web/OWASP lookups (WebTools, backed by Vertex AI Search), and security posture (PostureToolsover MCP). - Draft a report and pause for human approval before finalizing.
Review state (ReviewState, checkpoints, pending approvals, stored messages) is persisted to
Postgres so a review can be resumed. Two variants exist: SecurityReviewAgentVision (image input)
and SecurityReviewAgentChainWorkflow (deterministic chained steps, toggled by
app.agent.use-chain-workflow).
- Metrics — Actuator exposes
/actuator/prometheus; Micrometer publishes AI-specific meters. - Tracing — Micrometer Tracing → OpenTelemetry → OTLP (
localhost:4317) → Jaeger/Tempo. - Logs — structured logging shipped by Promtail into Loki.
- Dashboards — Grafana auto-provisions "AI metrics" and "AI logs" dashboards.
Sampling is set to 1.0 (trace everything) for local development — dial this down for real workloads.
This project deliberately includes a small red-team/blue-team surface:
SafeGuardAdvisorblocks prompts containing sensitive terms (credentials, "ignore previous instructions", etc.).ContentSanitizerAdvisorscrubs tool/document content before it reaches the model.chat/openai/jailbreak/demo(withBankingTools) demonstrates a jailbreak scenario and the guardrails that mitigate it.
These are educational demonstrations, not a substitute for a real security review.
Import gen-ai-with-java-spring.postman_collection.json
into Postman. It contains ready-to-run requests for every endpoint above, grouped by capability.
The project was built iteratively with heavy manual and Postman-driven testing across each
provider and RAG configuration.
- Machine-specific paths:
app.agent.upload-dirandapp.rag.pdf.pathpoint at/Users/Shared/...; the Vertexproject-idinapplication.ymlis hard-coded. Change these for your environment (or externalize them to env vars) before using the agent / PDF-watcher features. - Java 25 + Spring Boot 3.5.8 + Spring AI 1.1.0 are aligned and current; if you build on an
older JDK, adjust
<java.version>and the compiler<release>inpom.xml. - pgvector dimensions are fixed at
1536(OpenAItext-embedding-3-small/ada-002). If you switch embedding models, updatespring.ai.vectorstore.pgvector.dimensionsand rebuild the store. app.rag.force-rebuild: truere-ingests on startup — set tofalseonce your store is warm.- Some provider endpoints (Hugging Face dedicated endpoint URL, Vertex data store) are placeholders and must be pointed at your own resources.
No license file is currently included. Add one (e.g. MIT) before publishing or reusing.