Skip to content

Repository files navigation

Gen AI with Java & Spring Boot — Agents, RAG & Multimodality

A hands-on, production-shaped reference application that explores the full surface of Spring AI on Java 25 and Spring Boot 3.5. It brings together multi-provider chat, retrieval-augmented generation (RAG), multimodality, tool-calling agents with human-in-the-loop approval, the Model Context Protocol (MCP), and a full observability stack.

Status: learning / reference project. It is intentionally broad — each package is a self-contained demonstration of one Spring AI capability, wired so the pieces can be combined.


Table of Contents


Features

Chat & Providers

  • Pluggable providers behind Spring AI's ChatClient: OpenAI, Google Vertex AI Gemini, Hugging Face (dedicated inference endpoints), Ollama (local), and the Docker Model Runner (OpenAI-compatible local endpoint).
  • Streaming responses, structured output (POJO / List / Map binding), and a "raw" OpenAI Java client comparison.
  • Conversational memory two ways: JDBC-backed message-window memory and semantic memory backed by a pgvector VectorStore.

Retrieval-Augmented Generation (RAG)

  • A simple path using QuestionAnswerAdvisor over an in-memory SimpleVectorStore.
  • An advanced path using RetrievalAugmentationAdvisor with:
    • domain-synonym query transformation,
    • multi-query expansion,
    • pgvector similarity retrieval,
    • Cohere reranking, neighbour-chunk stitching, and citation-header post-processing.
  • PDF ingestion (page/paragraph readers) plus a filesystem watcher that ingests new PDFs dropped into a directory at runtime.
  • An "internal Q&A over docs" scenario and a "mini employee handbook" scenario.

Multimodality

  • Image → text: product-image description.
  • Text → image: marketing-asset / campaign image generation with DALL·E 3.
  • Speech → text: meeting-notes transcription (Whisper) with structured summarization.
  • Text → speech: ticket-status voice responses (TTS).

Agents & Tools

  • A Security-Review Agent that plans and executes a multi-step review of a system, calling tools for: threat-model diagram extraction (draw.io / image), RAG over internal security docs, web/OWASP lookups (via Google Vertex AI Search), and a security-posture lookup.
  • Human-in-the-loop checkpoints (approve diagram extraction, approve final report), with durable review state persisted to Postgres.
  • A vision variant and a deterministic chain-workflow variant of the agent.
  • MCP: the main app is an MCP client; the posture-service is a standalone MCP server exposing posture tools.

Cross-cutting

  • Custom ChatClient advisors: system-prompt injection, validation, error-wrapping, SafeGuardAdvisor (sensitive-term guard), content sanitization, and a simple logger.
  • Full observability: Micrometer + Prometheus metrics, OpenTelemetry tracing to Jaeger/Tempo, Loki/Promtail logs, and provisioned Grafana dashboards.

Architecture

                         ┌──────────────────────────────────────────────┐
   HTTP / Postman  ───▶  │        gen-ai-java-spring  (port 8081)        │
                         │                                              │
                         │  Chat ── RAG ── Multimodality ── Agents      │
                         │    │       │          │            │         │
                         │    │       │          │            ├── Tools: diagram / rag / web / posture
                         │    │       │          │            └── MCP client ─────────────┐
                         │    ▼       ▼          ▼                                        │
                         │  ChatClient (OpenAI / Vertex / HF / Ollama / DMR)              │
                         └──────┬───────────────────────────────┬───────────────────────┘
                                │                               │
                       ┌────────▼────────┐            ┌─────────▼──────────┐
                       │ Postgres +      │            │  posture-service   │
                       │ pgvector (5433) │            │  MCP server (8082) │
                       │  · rag store    │            └────────────────────┘
                       │  · chat memory  │
                       └─────────────────┘

   Observability:  Micrometer → Prometheus (9090) · OTLP → Jaeger (16686) ·
                   Logs → Loki (3100)/Promtail · Dashboards → Grafana (3001)

Tech Stack

Area Choice
Language / build Java 25, Maven, Spring Boot 3.5.8
AI framework Spring AI 1.1.0 (BOM-managed)
Providers OpenAI, Vertex AI Gemini, Hugging Face, Ollama, Docker Model Runner
Vector store PostgreSQL 16 + pgvector (HNSW, cosine, 1536-dim)
Reranking Cohere rerank-english-v3.0
Memory Spring AI JDBC chat-memory + vector-store chat-memory
Agent protocol Model Context Protocol (MCP) client + server
Observability Micrometer, Prometheus, OpenTelemetry, Jaeger, Tempo, Loki, Grafana
Misc Lombok, Apache HttpClient 5, JTokkit (token counting), PDFBox reader

Project Layout

.
├── pom.xml                          # main app (gen-ai-java-spring)
├── src/main/java/com/genai/java/spring
│   ├── config/                      # AIProviderConfig — all ChatClient/VectorStore beans
│   ├── chat/                        # provider controllers, memory, structured output, jailbreak demo
│   ├── multimodality/               # image/speech/text-to-* controllers
│   ├── rag/                         # simple + advanced RAG, rerank, ingestion, PDF watcher
│   └── aiagent/                     # security-review agent, tools, MCP, approvals, state
├── src/main/resources
│   ├── application.yml              # central configuration
│   ├── templates/*.st               # StringTemplate prompt templates
│   ├── rag/                         # sample PDFs + employee-handbook docs
│   └── static/                      # sample product images, meeting-notes.mp3
├── posture-service/                 # standalone Spring Boot MCP server (port 8082)
├── docker/
│   ├── postgres/pgvector.yml        # Postgres + pgvector (port 5433)
│   └── observability/               # Prometheus, Grafana, Loki, Promtail, Tempo, Jaeger
├── ollama/                          # custom Mistral Modelfile + manifest
├── huggingface/huggingface.sh       # shared HF inference API example
└── gen-ai-with-java-spring.postman_collection.json

Prerequisites

  • JDK 25 (the build targets --release 25)
  • Maven 3.9+
  • Docker / Docker Desktop (for Postgres + observability)
  • API keys as needed (see below). Only the keys for the providers you actually call are required.
  • (optional) Ollama for local models, Cohere key for reranking, a GCP project for Vertex AI Gemini / Vertex AI Search.

Configuration & Secrets

All secrets are read from environment variables — nothing sensitive is committed. Export what you need before running:

export OPENAI_API_KEY=sk-...          # chat, embeddings, DALL·E, Whisper, TTS
export HUGGINGFACE_API_KEY=hf_...      # Hugging Face inference endpoints
export COHERE_API_KEY=...              # RAG reranking
export GCP_PROJECT=...                 # Vertex AI Search (web tool)
export DATASTORE_ID=...                # Vertex AI Search data store
# Vertex AI Gemini uses Application Default Credentials (gcloud auth application-default login)

Key knobs live in src/main/resources/application.yml:

  • app.ai.provider — default provider for shared endpoints.
  • app.rag.* — chunking, top-k, similarity threshold, query-expansion count, rerank settings.
  • spring.ai.* — per-provider model names (e.g. gpt-4o-mini, gemini-2.0-flash, mistral:7b).
  • spring.datasource.* — points at Postgres on localhost:5433 (matches the compose file).

Heads-up — machine-specific paths. A few settings currently point at absolute local directories (app.agent.upload-dir, app.rag.pdf.path under /Users/Shared/...) and the Vertex project-id is hard-coded. Update these to your own paths/project before running the agent and filesystem-watcher features. See Notes & Known Rough Edges.


Running Locally

1. Infrastructure (Postgres + pgvector)

docker compose -f docker/postgres/pgvector.yml up -d

Brings up pgvector/pgvector:pg16 on host port 5433. The vector tables (rag_vector_store, vector_store) and chat-memory schema are auto-created on first run (initialize-schema: true).

2. Observability stack (optional)

docker compose -f docker/observability/observability-compose.yml up -d
Service URL
Grafana http://localhost:3001
Prometheus http://localhost:9090
Jaeger UI http://localhost:16686
Loki http://localhost:3100

Grafana ships with provisioned datasources and two dashboards (AI metrics, AI logs).

3. Local models with Ollama (optional)

ollama pull mistral:7b
# optional: build the custom email-drafting model
ollama create custom-mistral-model -f ./ollama/custom-mistral-modelfile
ollama run custom-mistral-model

See ollama.md for how the Modelfile is layered on top of the base model.

4. The main application

mvn spring-boot:run
# or
mvn clean package && java -jar target/gen-ai-java-spring-0.0.1-SNAPSHOT.jar

The app starts on http://localhost:8081.

5. The posture MCP microservice

The agent's posture tool calls a separate MCP server:

cd posture-service
mvn spring-boot:run   # starts on http://localhost:8082

API Overview

Base URL: http://localhost:8081. A full Postman collection is included.

Chat

Method & Path Description
POST /api/openai/chat OpenAI chat (+ streaming, structured)
POST /api/vertexai/chat Google Vertex AI Gemini
POST /api/huggingface/chat Hugging Face inference endpoint
POST /api/ollama/chat Local Ollama model
POST /api/docker-model-runner/chat Docker Model Runner (OpenAI-compatible)
POST /api/common/chat Provider-agnostic entry point
POST /api/orders/chat Tool-calling + chat memory demo

Multimodality

Method & Path Description
POST /api/products Image → text (product description)
POST /api/marketing-assets/... Text → image (DALL·E 3)
POST /api/meeting-assistant/... Speech → text + summarization (Whisper)
POST /api/tickets/... Text → speech (TTS)

RAG

Method & Path Description
POST /api/internal-qa Advanced RAG over internal PDFs
POST /api/employee-handbook RAG over the mini employee handbook
POST /api/it-support In-prompt-context retrieval demo

Agent

Method & Path Description
POST /api/security-review Kick off a security review
POST /api/security-review/{id}/ask Follow-up question in a review
POST /api/security-review/{id}/human-approve-diagram-extract Approve extracted threat model
POST /api/security-review/{id}/human-approve-final-report Approve the final report
GET /api/security-review/{id} Fetch review state

RAG Pipeline

The advanced pipeline is assembled in AIProviderConfig as a RetrievalAugmentationAdvisor:

query
  └─ DomainSynonymTransformer      # expand PTO → "paid time off", MFA → "multi-factor auth", …
       └─ MultiQueryExpander        # generate N paraphrased queries (LLM, temperature 0)
            └─ VectorStoreDocumentRetriever  # pgvector top-k + similarity threshold
                 └─ RerankPostProcessor       # Cohere rerank-english-v3.0, keep top-n
                      └─ NeighbourStitchPostProcessor  # stitch adjacent chunks for context
                           └─ CitationHeaderPostProcessor  # prepend source citations
                                └─ ContextualQueryAugmenter  # (disallows empty context)

Ingestion (RagIngestionService + PdfWatcherService) reads PDFs, splits them with a TokenTextSplitter, embeds with OpenAI, and upserts into the rag_vector_store table. Drop a new PDF into the configured watch directory and it is ingested without a restart.


The Security-Review Agent

aiagent/ implements an agent that reviews a system design for security issues:

  1. Plan the review (Plan / PlanStep records).
  2. Extract a threat-model diagram from an uploaded draw.io / image file (DiagramTools).
  3. Pause for human approval of the extracted model.
  4. Gather evidence via tools — internal-docs RAG (RagTools), web/OWASP lookups (WebTools, backed by Vertex AI Search), and security posture (PostureTools over MCP).
  5. Draft a report and pause for human approval before finalizing.

Review state (ReviewState, checkpoints, pending approvals, stored messages) is persisted to Postgres so a review can be resumed. Two variants exist: SecurityReviewAgentVision (image input) and SecurityReviewAgentChainWorkflow (deterministic chained steps, toggled by app.agent.use-chain-workflow).


Observability

  • Metrics — Actuator exposes /actuator/prometheus; Micrometer publishes AI-specific meters.
  • Tracing — Micrometer Tracing → OpenTelemetry → OTLP (localhost:4317) → Jaeger/Tempo.
  • Logs — structured logging shipped by Promtail into Loki.
  • Dashboards — Grafana auto-provisions "AI metrics" and "AI logs" dashboards.

Sampling is set to 1.0 (trace everything) for local development — dial this down for real workloads.


Security & Prompt-Injection Defenses

This project deliberately includes a small red-team/blue-team surface:

  • SafeGuardAdvisor blocks prompts containing sensitive terms (credentials, "ignore previous instructions", etc.).
  • ContentSanitizerAdvisor scrubs tool/document content before it reaches the model.
  • chat/openai/jailbreak/demo (with BankingTools) demonstrates a jailbreak scenario and the guardrails that mitigate it.

These are educational demonstrations, not a substitute for a real security review.


Testing with the Postman Collection

Import gen-ai-with-java-spring.postman_collection.json into Postman. It contains ready-to-run requests for every endpoint above, grouped by capability. The project was built iteratively with heavy manual and Postman-driven testing across each provider and RAG configuration.


Notes & Known Rough Edges

  • Machine-specific paths: app.agent.upload-dir and app.rag.pdf.path point at /Users/Shared/...; the Vertex project-id in application.yml is hard-coded. Change these for your environment (or externalize them to env vars) before using the agent / PDF-watcher features.
  • Java 25 + Spring Boot 3.5.8 + Spring AI 1.1.0 are aligned and current; if you build on an older JDK, adjust <java.version> and the compiler <release> in pom.xml.
  • pgvector dimensions are fixed at 1536 (OpenAI text-embedding-3-small/ada-002). If you switch embedding models, update spring.ai.vectorstore.pgvector.dimensions and rebuild the store.
  • app.rag.force-rebuild: true re-ingests on startup — set to false once your store is warm.
  • Some provider endpoints (Hugging Face dedicated endpoint URL, Vertex data store) are placeholders and must be pointed at your own resources.

License

No license file is currently included. Add one (e.g. MIT) before publishing or reusing.

About

Production-shaped Spring AI reference app (Java 25, Spring Boot 3.5): multi-provider LLM chat (OpenAI, Vertex AI, Hugging Face, Ollama), advanced RAG over pgvector with Cohere reranking, multimodality, a human-in-the-loop tool-calling Security-Review Agent, MCP client+server, and full Prometheus/Grafana/OpenTelemetry observability.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages