feat: streams in Redis + add Ollama provider

ARCHITECTURE.md

@@ -0,0 +1,105 @@
+# RsChat Architecture
+
+## Overview
+
+RsChat is a real-time chat application that provides resumable streaming conversations with LLM providers. The architecture is designed for high performance, scalability across multiple server instances, and resilient streaming that can survive network interruptions.
+
+## Core Architecture
+
+### Frontend (React/TypeScript)
+- **Location**: `web/`
+- **Streaming**: Server-Sent Events (SSE)
+- **State Management**: React Context for streaming state (`web/src/lib/context/StreamingContext.tsx`)
+- **API Integration**: Type-safe API calls (generated from OpenAPI: `web/src/lib/api/types.d.ts`)
+
+### Backend (Rust/Rocket)
+- **Location**: `server/`
+- **Framework**: Rocket with async/await support
+- **Database**: PostgreSQL for persistent storage
+- **Cache/Streaming**: Redis for stream management and caching
+
+## LLM Streaming Architecture
+
+### Dual-Stream Approach
+
+RsChat uses a hybrid streaming architecture that provides both real-time performance and cross-instance resumability:
+
+1. **Server**: Redis Streams for resumability and multi-instance support
+2. **Client**: Server-Sent Events (SSE) read from the Redis streams
+
+### Key Components
+
+#### 1. LlmStreamWriter (`server/src/stream/llm_writer.rs`)
+
+The core component that processes LLM provider streams and manages Redis stream output.
+
+**Key Features:**
+- **Batching**: Accumulates chunks from the provider stream, up to a max length or timeout
+- **Background Pings**: Sends regular keepalive pings
+- **Database Integration**: Saves final responses to PostgreSQL
+
+#### 2. Redis and SSE Stream Structure
+
+**Redis Key for Chat Streams**: `user:{user_id}:chat:{session_id}`
+
+**Chat Stream Message Types**:
+- `start`: Stream initialization
+- `text`: Accumulated text chunks
+- `tool_call`: LLM tool invocations (JSON stringified)
+- `error`: Error messages
+- `ping`: Keepalive messages
+- `end`: Stream completion
+- `cancel`: Stream cancellation
+
+#### 3. Stream Lifecycle
+
+```
+Client Request → SSE Connection → LlmStreamWriter.create()
+                     ↓
+LLM Provider Stream → Batching Data Chunks → Redis XADD
+                     ↓
+Background Ping Task (intervals)
+                     ↓
+Stream End → Database Save → Redis DEL
+```
+
+
+### Resumability Features
+
+#### Cross-Instance Support
+- Redis streams provide shared state across server instances
+- Background ping tasks maintain stream liveness
+- Stream cancellation detected via Redis XADD failures
+
+## Data Flow
+
+### 1. New Chat Request
+```
+Client → POST /api/chat/{session_id}
+       → Send request to LLM Provider
+       → SSE Response Stream created
+       → LlmStreamWriter.create()
+       → Redis Stream created
+```
+
+### 2. Stream Processing
+```
+LLM Chunk → Process text, tool calls, usage, and error chunks
+          → Batching Logic
+          → Redis XADD (if conditions met)
+          → Continue SSE Stream
+```
+
+### 3. Stream Completion
+```
+LLM End → Final Database Save
+        → Redis Stream End Event
+        → Redis Stream Cleanup
+        → SSE Connection Close
+```
+
+### 4. Reconnection/Resume
+```
+Client Reconnect → Check ongoing streams via GET /api/chat/streams
+                 → Reconnect to stream (if active)
+```

server/Cargo.toml

@@ -18,10 +18,10 @@ strip = true
 [dependencies]
 aes-gcm = "0.10.3"
 astral-tokio-tar = "0.5.2"
-async-stream = "0.3.6"
 bollard = { version = "0.19.1", features = ["ssl"] }
 chrono = { version = "0.4.41", features = ["serde"] }
 const_format = "0.2.34"
+deadpool = { version = "0.12.2", features = ["rt_tokio_1"] }
 diesel = { version = "2.2.10", features = [
     "postgres",
     "chrono",
@@ -33,8 +33,12 @@ diesel-derive-enum = { version = "3.0.0-beta.1", features = ["postgres"] }
 diesel_as_jsonb = "1.0.1"
 diesel_async_migrations = "0.15.0"
 dotenvy = "0.15.7"
+dyn-clone = "1.0.19"
 enum-iterator = "2.1.0"
-fred = { version = "10.1.0", default-features = false, features = ["i-keys"] }
+fred = { version = "10.1.0", default-features = false, features = [
+    "i-keys",
+    "i-streams",
+] }
 hex = "0.4.3"
 jsonschema = { version = "0.30.0", default-features = false }
 rand = "0.9.1"
@@ -54,7 +58,6 @@ schemars = { version = "0.8.22", features = ["chrono", "uuid1"] }
 serde = { version = "1.0.219" }
 serde_json = "1.0.140"
 subst = { version = "0.3.8", features = ["json"] }
-tempfile = "3.20.0"
 thiserror = "2.0.12"
 tokio = { version = "1.45.1" }
 tokio-stream = "0.1.17"

server/src/api.rs

@@ -1,6 +1,7 @@
 mod api_key;
 mod auth;
 mod chat;
+mod info;
 mod provider;
 mod secret;
 mod session;
@@ -9,6 +10,7 @@ mod tool;
 pub use api_key::get_routes as api_key_routes;
 pub use auth::get_routes as auth_routes;
 pub use chat::get_routes as chat_routes;
+pub use info::get_routes as info_routes;
 pub use provider::get_routes as provider_routes;
 pub use secret::get_routes as secret_routes;
 pub use session::get_routes as session_routes;