MedLen - Medical OCR Pipeline

MedLen is a professional, high-performance Optical Character Recognition (OCR) pipeline specifically designed for medical documents. It leverages the Chandra-OCR model via GGUF quantization to extract structured tables from laboratory results, prescriptions, and medical reports.

---

🚀 Key Features

• Chandra GGUF Engine: Uses high-performance quantized inference for accurate medical table extraction.
• Intelligent Preprocessing:
  • Automatic conversion of HEIC, TIFF, and JPG to optimized PNG.
  • Image enhancement for better OCR visibility (contrast/sharpness).
  • Smart resizing aligned to the model's patch sizes (multiples of 28).
• Automated Orchestration:
  • Automatically checks and downloads model weights from Hugging Face if missing.
  • Manages Docker environment (docker-compose) and ensures server health before processing.
• Structured Data Mapping:
  • Post-processes OCR tables to map parameter names to a standardized schema using synonym dictionaries.
  • Splits multi-table documents into distinct structured reports (Hematology, Biochemistry, etc.).
• Aesthetic CLI: Minimal, professional terminal logging with dynamic spinners and detailed phase tracking (5 phases).
• Flexible Outputs: Saves results in HTML (raw), Markdown (readable), and JSON (structured).

---

🛠 Project Structure

MedLen/
├── main.py              # Main Controller & CLI
├── docker-compose.yml   # Multi-GPU support (llama.cpp server)
├── .env                 # Centralized configuration
├── utils/               # Internal modules
│   ├── ocr_client.py    # API & Health communication
│   ├── resize.py        # Image enhancement & resizing logic
│   ├── converter.py     # Multi-format image loader
│   ├── extractor.py     # Table extraction & schema mapping engine
│   ├── setup_models.py  # Model weight management
│   └── clean_cache.py   # Maintenance utility
├── mapping_context/     # Schema & Synonym definitions
├── database/            # Data storage
│   ├── input/           # Place raw images here
│   ├── mid_img/         # Preprocessed intermediate images
│   ├── mid_ocr/         # Intermediate OCR results (HTL)
│   └── output/          # Final Structured Data (JSON, MD)
└── models/              # Model weights storage

---

📦 Prerequisites

• Python: 3.12+ (managed by uv)
• Docker: For running the Chandra-GGUF server
• NVIDIA GPU: Recommended for CUDA-accelerated inference

---

⚡ Quick Start

1. Initialization

Note: For a detailed setup guide, including potentially missing API keys and manual model download steps, please refer to the Reproduction Guide. For a technical deep dive, see the System Overview. If you encounter issues, check context Troubleshooting Guide.

Clone the repository and install dependencies using uv:

uv sync

2. Configure Environment

Create a .env file (see env.example) to customize paths and model settings.

3. Run the Pipeline

The pipeline is designed to be fully automated. You don't need to manually start Docker or download models; main.py handles it for you.

Batch Mode (Process all files in database/input/):

uv run python main.py

Single File Mode:

uv run python main.py --file "path/to/your/image.jpg"

Limited Batch:

uv run python main.py --limit 5

---

💻 Web Demo

MedLen includes a modern web interface for interactive visualizations and testing.

1. Start the Backend API

The backend acts as a bridge between the pipeline and the UI. It will strictly enforce one request at a time to manage GPU resources.

uv run python demo/backend/server.py

Server runs on: http://localhost:8000

2. Start the Frontend

In a new terminal window:

cd demo/frontend
npm install  # (First time only)
npm run dev

App runs on: http://localhost:3000

3. Usage

Open http://localhost:3000 in your browser. You can drag and drop medical documents to see the real-time processing pipeline, including de-identification steps and final structured output.

---

📄 Output Formats

Every processed image generates two sets of outputs:

1. Intermediate (database/mid_ocr/):

   • .html: Full structured table as raw HTML.
   • .md: Markdown representation.

2. Final Structured Data (database/output/):

   • Structured by Section: If a document contains multiple tables (e.g. Hematology + Biochemistry), they are automatically split into separate files.
   • .json: Machine-readable format mapped to a canonical schema (Standard Parameter Names, Results, Units, Normal Ranges).
   • .md: Clean human-readable report for each section.

🧠 Schema & Mapping

The pipeline uses mapping_context/ to map diverse hospital formats to a standard schema:

• *SYNONYM*.csv files: Define mappings from various parameter names (e.g., "Glu", "Glucose máu", "Đường huyết") to a canonical key (e.g., "Glu").
• extractor.py: Uses exact and fuzzy matching to standardize data during the extraction phase.

---

🧼 Maintenance

To keep the workspace clean from temporary Python files:

uv run python utils/clean_cache.py

---

⚖️ License

Internal Project - All rights reserved.