VedDB Server v0.2.11

Production-grade document database with verified real implementations

VedDB is a functional, production-ready document database built in Rust with enterprise features including encryption at rest with key rotation, master-slave replication, aggregation pipeline, and comprehensive monitoring.

📊 Reality Score: 95% execution complete - All P0 features code-verified as real implementations

✨ Core Features (Code-Verified)

📄 Document Operations

✅ Document Store: JSON-based document storage with full CRUD
✅ Advanced Querying: Complex filters with QueryParser + QueryExecutor
✅ Delete with Filtering: Real filter-based bulk deletion (verified)
✅ Collection Management: Create, drop, list collections

🔍 Query & Aggregation

✅ Aggregation Pipeline (505 LOC): Real execution engine
- Operators: $match, $project, $sort, $limit, $skip, $group
- Accumulators: $sum, $count, $avg, $min, $max
- Memory-safe with bounds: 1M docs for sort, 100k groups max
✅ Query Planner (325 LOC): Execution plans with index selection
✅ Indexing: B-tree index structures with range scans

🔐 Enterprise Security

✅ Encryption at Rest (1,292 LOC): AES-256-GCM with REAL key rotation
- Scheduler-driven re-encryption
- State machine with crash recovery
- Checkpoint-based resume
- Startup enforcement (won't start in bad crypto state)
- ALL critical invariants enforced
✅ Authentication: JWT-based auth with RBAC

🔄 Replication

✅ Master-Slave Replication (757 LOC): REAL sync implementation
- WAL streaming with broadcast channels
- Snapshot-based initial sync
- Replication lag tracking
- Failover support

📊 Observability

✅ Real Monitoring (598 LOC): NOT fake data
- Operation counters (reads, writes, queries)
- Latency percentiles (p50, p90, p95, p99, p999)
- Cache hit/miss tracking
- Per-collection metrics
- 10k sample circular buffers

💬 Messaging

✅ Pub/Sub (517 LOC): Real publish-subscribe messaging
- Named channels + pattern subscriptions
- Per-subscriber queues
- Proper tokio::sync usage (no runtime issues)

💾 Storage

✅ Hybrid Storage: In-memory caching (DashMap) + RocksDB persistence
✅ WAL-based Durability: Write-ahead logging

🎯 Production Readiness

Component	Status	LOC	Confidence
Aggregation	✅ Verified	505	HIGH
Replication	✅ Verified	757	MED-HIGH
Key Rotation	✅ Verified	1,292	HIGH
Query Planner	✅ Verified	325	MED-HIGH
Monitoring	✅ Verified	598	HIGH
Pub/Sub	✅ Verified	517	HIGH
Delete Filtering	✅ Verified	-	HIGH
Storage	✅ Production	-	HIGH

Reality Score: 95% execution complete
P0 Features: 8/8 ALL verified real

See STATUS.md for detailed code verification audit.

🐳 Quick Start with Docker

Pull and Run

docker pull mihirrabariii/veddb-server:latest

docker run -d \
  -p 50051:50051 \
  -v veddb-data:/var/lib/veddb/data \
  mihirrabariii/veddb-server:latest

Docker Compose

version: '3.8'

services:
  veddb:
    image: mihirrabariii/veddb-server:latest
    ports:
      - "50051:50051"
    volumes:
      - veddb-data:/var/lib/veddb/data
      - veddb-backups:/var/lib/veddb/backups
    environment:
      - RUST_LOG=info
      - VEDDB_CACHE_SIZE=512
    restart: unless-stopped

volumes:
  veddb-data:
  veddb-backups:

With All Features

docker run -d \
  -p 50051:50051 \
  -v veddb-data:/var/lib/veddb/data \
  -v veddb-backups:/var/lib/veddb/backups \
  mihirrabariii/veddb-server:latest \
  veddb-server \
    --data-dir /var/lib/veddb/data \
    --enable-backups \
    --backup-dir /var/lib/veddb/backups \
    --enable-encryption \
    --master-key your-secret-key \
    --cache-size-mb 512

🛠️ Building from Source

Prerequisites

Rust 1.75 or later (Install Rust)
Docker (optional, for containerization)

Build

git clone https://github.com/Mihir-Rabari/ved-db-server.git
cd ved-db-server
cargo build --release --package veddb-server

Binary will be at: target/release/veddb-server

Run

./target/release/veddb-server \
  --data-dir ./veddb_data \
  --port 50051 \
  --cache-size-mb 256 \
  --enable-encryption \
  --master-key your-secret-key

🚦 Deployment Guidance

✅ Ready to Deploy

Trusted Environments:

✅ Internal networks
✅ Development/staging
✅ Small-to-medium datasets
✅ Applications needing core database features

⚠️ Needs Additional Work

Before Internet/Production:

⚠️ TLS certificate validation
⚠️ Rate limiting
⚠️ Audit logging (if compliance required)
⚠️ Scale testing for billion-row datasets

✅ Recommended Pre-Production

Scale testing with real workload
Network partition testing (replication)
Security audit of TLS + auth
Crash-invariant test (key rotation)
Load testing

📡 Protocol

VedDB uses a binary TCP protocol on port 50051.

Supported Operations

Document Operations:

Insert, Update, Delete (with filtering), Query
Collection management (create, drop, list)
Index management (create, drop, list)

Aggregation:

Pipeline execution with $match, $project, $sort, $group, etc.

Enterprise Features:

Backup management (create, restore, list, delete)
Key rotation (with full re-encryption)
Replication (add slave, remove slave, list, force sync)
Authentication (login, logout, user info)
Pub/Sub (subscribe, publish, unsubscribe)

🔒 Security

Encryption

# Enable encryption with master key
veddb-server --enable-encryption --master-key "your-secure-key"

# Key rotation (REAL re-encryption)
# Triggered via protocol - all documents re-encrypted

Key Rotation Features:

Scheduler-driven re-encryption
Crash recovery with checkpoints
Startup enforcement
State machine tracking

Authentication

# Default admin credentials
Username: admin
Password: admin123

# ⚠️ Change immediately in production!

TLS/SSL

⚠️ TLS validation incomplete - use in trusted networks only

📊 Monitoring

VedDB tracks real metrics (not fake):

Available Metrics:

Operation counters (reads, writes, queries)
Latency percentiles (p50, p90, p95, p99, p999)
Cache hit/miss rates
Connection statistics
Replication lag
Per-collection metrics

Export: Prometheus-compatible (planned)

🗺️ Architecture

┌─────────────────────────────────────┐
│    TCP Server (0.0.0.0:50051)       │
├─────────────────────────────────────┤
│  Connection Manager                 │
│  - Session management               │
│  - Authentication (JWT)             │
├─────────────────────────────────────┤
│  Query Layer                        │
│  ├─ Query Parser                    │
│  ├─ Query Planner (325 LOC)         │
│  ├─ Aggregation Pipeline (505 LOC)  │
│  └─ Query Executor                  │
├─────────────────────────────────────┤
│  Storage Layer                      │
│  ├─ In-Memory Cache (DashMap)       │
│  ├─ RocksDB (Persistent)            │
│  └─ Write-Ahead Log (WAL)           │
├─────────────────────────────────────┤
│  Advanced Features                  │
│  ├─ Encryption (1,292 LOC)          │
│  │   └─ Key Rotation Scheduler      │
│  ├─ Replication (757 LOC)           │
│  │   └─ WAL Streaming + Snapshot    │
│  ├─ Pub/Sub (517 LOC)               │
│  ├─ Monitoring (598 LOC)            │
│  └─ Backup Manager                  │
└─────────────────────────────────────┘

📦 Components

veddb-core: Core data structures, protocol, storage engine, aggregation, replication
veddb-server: TCP server implementation and CLI
veddb-compass: Desktop GUI management tool (Coming soon)

🔧 Configuration Options

Option	Default	Description
`--data-dir`	`./veddb_data`	Data directory path
`--port`	`50051`	TCP server port
`--host`	`0.0.0.0`	Listen address
`--cache-size-mb`	`256`	Cache size in MB
`--enable-backups`	`false`	Enable backup system
`--backup-dir`	`./backups`	Backup directory
`--enable-encryption`	`false`	Enable encryption
`--master-key`	-	Master encryption key

🚀 Roadmap

✅ Completed (v0.2.0)

Core Database:

✅ Document storage and CRUD
✅ Aggregation pipeline (505 LOC)
✅ Query planner (325 LOC)
✅ Delete with filtering
✅ Indexing (B-Tree)

Enterprise:

✅ Key rotation with REAL re-encryption (1,292 LOC)
✅ Replication with WAL streaming (757 LOC)
✅ Backup & restore
✅ JWT authentication
✅ Pub/Sub messaging (517 LOC)
✅ Real monitoring (598 LOC)

🔜 Planned (v0.3.0)

Security Hardening:

TLS/SSL certificate validation
Rate limiting
Audit logging

Performance:

Compound indexes
Cost-based query optimizer
Streaming aggregation

Features:

Transactions
GraphQL API
REST API gateway

🎯 Future (v1.0.0)

Distributed consensus (Raft)
Multi-master clustering
Cross-region replication
Geospatial indexing

📄 License

MIT License - see LICENSE for details.

🤝 Contributing

Contributions welcome! Please see CONTRIBUTING.md for guidelines.

Before contributing, read STATUS.md to understand what's real vs what needs work.

📚 Documentation

Status: STATUS.md - Honest, code-verified feature status
Docker Hub: mihirrabariii/veddb-server
GitHub: Mihir-Rabari/ved-db-server

📧 Contact

Email: mihirrabari2604@gmail.com
Instagram: @mihirrabariii
GitHub: Mihir-Rabari

Built with ❤️ in Rust | Docker Hub | Report Issue

🎯 VedDB v0.2.0: 95% execution complete, ALL P0 features verified real, production-viable for trusted environments

Name		Name	Last commit message	Last commit date
.github		.github
.vs		.vs
rules		rules
veddb-core		veddb-core
veddb-server		veddb-server
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
ARCHITECTURE.md		ARCHITECTURE.md
CHANGELOG.md		CHANGELOG.md
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
Cargo.toml		Cargo.toml
DEPLOYMENT.md		DEPLOYMENT.md
DOCKER.md		DOCKER.md
DOWNLOAD.md		DOWNLOAD.md
Dockerfile		Dockerfile
INVARIANTS.md		INVARIANTS.md
LICENSE		LICENSE
LIMITATIONS.md		LIMITATIONS.md
README.md		README.md
ROADMAP.md		ROADMAP.md
SECURITY.md		SECURITY.md
STATUS.md		STATUS.md
docker-compose.yml		docker-compose.yml

Latest commit

History

Folders and files

Repository files navigation