Refactor: Apply Single Responsibility Principle throughout codebase

[martgra]

This comprehensive refactoring decouples components and ensures each class/module
has a single, well-defined responsibility.

Key Changes

1. Protocol Abstractions (Decoupling)

• EmbeddingProvider protocol: Abstracts embedding operations

  • Decouples from OpenAI-specific implementation
  • Enables easy swapping of embedding providers (Cohere, local models, etc.)

• VectorStoreRepository protocol: Abstracts vector storage

  • Decouples from ChromaDB-specific implementation
  • Enables easy swapping of vector databases (Pinecone, Weaviate, etc.)

2. Infrastructure Implementations

• OpenAIEmbeddingProvider: OpenAI implementation of EmbeddingProvider
• ChromaVectorStoreRepository: ChromaDB implementation of VectorStoreRepository

3. Domain Services (Single Responsibility)

• XMLParsingService: Parse XML files and extract articles

  • Responsibility: XML parsing only

• ChunkingService: Split articles into token-sized chunks

  • Responsibility: Article chunking coordination

• EmbeddingService: Generate embeddings using provider

  • Responsibility: Embedding coordination
  • Decoupled from progress tracking (uses optional callbacks)
  • Decoupled from specific providers (uses EmbeddingProvider)

• FileProcessingService: Orchestrate parse → chunk → embed → index

  • Responsibility: Single file processing pipeline
  • Coordinates the domain services

4. Pipeline Orchestrator

• PipelineOrchestrator: Coordinate high-level pipeline stages
  • Responsibility: Sync → Identify → Process → Cleanup
  • Uses dependency injection for all services
  • Clean separation from infrastructure concerns

5. Simplified pipeline.py

• Reduced from 362 lines to 126 lines
• Changed from 7+ responsibilities to 1 responsibility
• New responsibility: Dependency injection and wiring
• Provides backward-compatible run_pipeline() function
• All business logic moved to services and orchestrator

Benefits

Testability

• Each service can be tested in isolation
• Easy to mock dependencies using protocols
• No need for complex integration test setup

Maintainability

• Clear separation of concerns
• Each class/module has single, obvious responsibility
• Easier to understand and modify

Flexibility

• Easy to swap implementations (OpenAI → Cohere, ChromaDB → Pinecone)
• Can reuse services in different contexts
• Progress tracking decoupled via callbacks

Code Quality

• Reduced coupling between components
• Better adherence to SOLID principles
• More focused, cohesive modules

Architecture

CLI (cli.py)
  ↓
Pipeline Factory (pipeline.py)
  ↓
PipelineOrchestrator
  ├── FileProcessingService
  │   ├── XMLParsingService
  │   ├── ChunkingService
  │   ├── EmbeddingService (uses EmbeddingProvider)
  │   └── VectorStoreRepository
  └── VectorStoreRepository

Protocols (Abstractions):
  - EmbeddingProvider
  - VectorStoreRepository

Implementations:
  - OpenAIEmbeddingProvider
  - ChromaVectorStoreRepository

Backward Compatibility

• Existing CLI unchanged
• run_pipeline() function signature unchanged
• All existing functionality preserved