Submission — pradeepbugga

submissions/pradeepbugga/MEMO.md

@@ -0,0 +1,81 @@
+
+# MIB Doc Challenge — Technical Memo
+
+**Author:** pradeepbugga · **Solution repo:** https://github.com/pradeepbugga/mib-doc-challenge
+
+## Approach
+
+The goal of this challenge is to build a system that automates a legacy intake system, rapidly providing 
+adjudication recommendations given a collection of visa information.  This project is inherently challenging due to 
+a few factors:
+- the data is messy (forms are provided with a variety of distortions and damages as one would expect in practice)
+- the algorithms for extraction (especially OCR) are capped to a latency of 6 seconds per PDF, no network, and limiting hardware requirements
+- the adjudication rules are only partially defined in the field manual
+- information within a visa packet can be conflicting 
+
+Our approach toward this ill-defined problem is to establish ground truths then steadily build upon those truths.
+Practically, this means we first prioritize page quality (i.e. machine-readable text vs scan images that require OCR), even though
+that may disagree with page priority in the field manual.  An applicant name in machine readable text will be trusted for our prediction pipeilne
+more than an applicant name in a scan.
+
+This also means that we prioritize independent fields over derived fields (fields that rely on other fields).  Applicant name, species,
+home world, and declared purpose (along with registry status), are indpeendent fields because there are no records in the training dataset that lack 
+explicit evidence for these.  On the other hand, risk flags and fee status are both very likely derived because there are 
+a number of examples in the training dataset where these fields are not mentioned yet have non-null values.  (Note: this assumption
+rests on training dataset having all relevant evidence in the packet).
+
+We show the overall pipeline below:
+
+
+```
+predict.py → packet_pipeline → per-page: quality → OCR → classify → extract
+           → corroboration → derivation → adjudication
+```
+
+In this strategy, we extract fields from machine-readable layer or OCR (repairing where needed),
+normalize fields against controlled vocabulary, then corroborate fields as different forms may have different values.
+Finally we perform any derivation (i.e. if registry_status = EMBARGO, risk_flag = planetary_embargo) then perform adjudication 
+as per the field manual policy.
+
+For OCR, we observe a variety of distortions, namely shear, rotation, scan line tear, and blur.  We developed low latency methods
+to tackle these issues.  For shear and rotation, we first detected line segments then measured angles versus a horizontal and 
+vertical line reference.  For tear, we identified the leftmost (or rightmost) line segment, then flattened them to a common x coordinate.
+For blur, we found that the degree of blur was too much for deblurring CV algorithms.  One idea was to generate synthetic blurs
+of prefix-value possibilites for controlled vocabulary fields (i.e. purpose) then pattern match.  Unfortunately, exactly matching degree of blur
+proved more difficult than expected.  For all the repair scripts, we tested OCR before and after, only accepting a repair if it proved beneficial.
+The downside of these fallbacks is that they really push against the latency constraints (6 s / PDF).  
+
+After extraction, we regex to field specific patterns then normalize the OCR value to a controlled vocabulary.  Our thresholds are chosen 
+such that fields that could be catastrophic (i.e. risk_flags) are more tolerant.  
+
+Derivation and adjudication thereafter follow combining our established field values, the field manual, and maximization of training dataset accuracy.
+
+
+## Results (training set, 1,000 packets)
+
+
+| Section | Score |
+| --- | --- |
+| Field extraction | 41.81 / 50 |
+| Classification | 65.17 / 80 |
+| Calibration | 13.60 / 20 |
+| Missing-case penalty | −0.00 / 10 (0 missing cases) |
+| **Total** | **120.58 / 150** |
+
+Mean confidence Brier 0.1599; 26 catastrophic false approvals.
+
+Per-field accuracy:
+
+| Field | Accuracy | Field | Accuracy |
+| --- | --- | --- | --- |
+| `species_code` | 94.0% | `arrival_date` | 83.6% |
+| `declared_purpose` | 91.5% | `applicant_name` | 81.4% |
+| `home_world` | 90.4% | `risk_flags` | 76.6% |
+| `visa_class` | 87.6% | `fee_status` | 62.7% |
+| `sponsor_id` | 85.8% | | |
+
+
+## With another week
+
+The biggest loss in our pipeline is fully predicted all risk flags and fee status values.
+More thought is required to understand how to approach this. 

submissions/pradeepbugga/SUBMISSION.md

@@ -0,0 +1,74 @@
+# Submission — pradeepbugga
+
+## Public solution repository
+
+https://github.com/pradeepbugga/mib-doc-challenge
+
+The solution lives on the `main` branch of that repository and includes a
+`Dockerfile` at the repository root.
+
+## Building and running
+
+```bash
+docker build -t mib-submission .
+
+mkdir -p /tmp/mib-output
+docker run --rm --network none \
+  --mount type=bind,src="$PWD/data/validation",dst=/input,readonly \
+  --mount type=bind,src="/tmp/mib-output",dst=/output \
+  mib-submission /input /output/predictions.jsonl
+```
+
+The image entrypoint is `run.sh`, which takes `<input_pdf_dir>` and
+`<output_predictions_path>` positionally, exactly as required by
+`DOCKER_SUBMISSION.md`.
+
+## Verified against the scoring contract
+
+Measured locally with the exact runtime flags from `DOCKER_SUBMISSION.md`
+(`--network none --cpus 4 --memory 8g --pids-limit 512 --read-only
+--tmpfs /tmp:rw,nosuid,nodev,size=2g`):
+
+| Check | Limit | Measured |
+| --- | --- | --- |
+| Uncompressed image size | 4 GiB | 1.05 GiB |
+| Average runtime per PDF | 6 s | 5.51 s (25-PDF sample) |
+| Projected 5,000-PDF runtime | 30,000 s | ~27,550 s (~7.65 h) |
+| Predictions file size | 25 MiB | 1.6 MiB |
+| Model artifacts | 250 MiB / 1 GiB | none |
+
+The container runs correctly with a read-only root filesystem and no network.
+Predictions produced inside the container are byte-for-byte identical to those
+produced on the host for a 25-PDF sample (25/25 identical), and
+`scripts/validate_submission.py` reports 5,000 valid records with 0 missing
+case ids against `data/validation_manifest.csv`.
+
+Note on timing: this is meaningfully tighter than an earlier build's measured
+1.43s/PDF. The gap is from additional scored-candidate OCR retries added since
+(tear-scanline repair, line-level retry, geometry correction), each of which
+only fires conditionally and only costs extra OCR passes on pages that need
+them — still comfortably under the 6s/PDF budget and the 8h20m hard cap, but
+with less margin than before, and worth watching if further retry-heavy
+features are added.
+
+## Runtime characteristics
+
+- Base image: `python:3.12-slim`, plus `tesseract-ocr`, `tesseract-ocr-eng`,
+  `tesseract-ocr-osd`, and `libgl1`.
+- Python dependencies are version-pinned in `requirements.txt`: PyMuPDF,
+  OpenCV (headless), NumPy, pytesseract, Pillow.
+- No network access at runtime, no API keys, no external services, no GPU.
+- No model artifacts — the system is classical CV plus offline OCR plus
+  hand-written rules, so the model-size limits do not apply.
+- Runs with a read-only container root filesystem; scratch files go to `/tmp`
+  and output to the mounted output path.
+- Worker count is capped to fit the 4 vCPU budget (`default_worker_count()`
+  in `scripts/predict.py`).
+
+## Contents of this folder
+
+| File | What it is |
+| --- | --- |
+| `predictions.jsonl` | Predictions for all 5,000 validation packets |
+| `MEMO.md` | Technical memo: approach, failure modes, next steps |
+| `SUBMISSION.md` | This file |