Docs AI Operating Model

Quality gates, PromptOps, governance, and evaluation for AI-assisted documentation.

This is the flagship AI DocsOps project in my documentation portfolio. It shows how a docs team can move from ad hoc AI experiments to a governed, measurable operating model: prompts as code, human review lanes, CI quality gates, and evaluation datasets that define what "good" means before AI-assisted content reaches readers.

Portfolio note

All content in this repository is generic and non-proprietary. It is meant to demonstrate senior-level thinking and execution for documentation leaders, technical writers, and teams adopting AI-assisted documentation responsibly.

---

Why this repo exists

AI-assisted documentation fails when teams treat the model as the workflow. It works when teams define the operating model around the model:

• Quality gates that keep docs buildable, reviewable, and consistent
• PromptOps practices that version prompts, metadata, and output contracts
• Governance that controls privacy, IP, compliance, and review risk
• Evaluation that turns quality from opinion into repeatable checks

This repo is designed to make those decisions visible in a docs-as-code environment.

---

Operating model at a glance

Pillar	What it demonstrates	Where to look
Quality gates	CI checks for builds, prose, prompt schemas, and evaluation smoke tests	docs/implementation/quality-gates.md
PromptOps	Prompts stored as versioned assets with metadata and reviewable contracts	prompts/, docs/promptops/
Governance	AI policy, risk register, data classification, and human review lanes	docs/governance/
Evaluation	Task datasets, rubrics, and lightweight rule-based scoring	eval/, docs/evaluation/

---

What it demonstrates

Docs-as-code and DocsOps

• Site authored in Markdown and published with MkDocs
• CI quality gates: strict site build, prose linting, prompt validation, and evaluation scaffolding
• A consistent information architecture: strategy -> governance -> PromptOps -> evaluation -> implementation

AI documentation governance

• Use-case catalog and decision criteria for choosing where AI belongs
• Guardrails for data classification, risk review, and human approval lanes
• Metrics for adoption, quality, efficiency, and customer impact

PromptOps

• Prompt files stored with metadata and versioning
• Schema validation in CI so broken prompt definitions fail early
• Patterns for prompt structure, source constraints, and output contracts

Evaluation

• Task-based evaluation dataset for realistic documentation scenarios
• Rule-based checks for structure, safety, and traceability
• Optional LLM-as-judge extension points that stay vendor-agnostic

---

Quick start (local)

# 1) Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate

# 2) Install deps
pip install -r requirements.txt
pip install -r requirements-docs.txt

# 3) Validate the prompt library
python -m docsai_toolkit validate-prompts

# 4) Build the site (strict)
mkdocs build --strict

# 5) Run a lightweight evaluation pass
python -m docsai_toolkit eval --dataset eval/datasets/smoke.yml --dry-run

---

Publish the docs site (GitHub Pages)

This repo includes a Pages workflow that:

1. Builds the MkDocs site
2. Uploads it as a Pages artifact
3. Deploys it with actions/deploy-pages

In your repo settings, use:

• Settings -> Pages -> Source -> GitHub Actions

---

Repository map

docs/                 # published site: strategy, governance, implementation
prompts/              # prompt library in YAML
eval/                 # evaluation datasets and rubrics
adr/                  # architecture decision records
src/docsai_toolkit/   # CLI utilities for validation and evaluation scaffolding
.github/workflows/    # CI and Pages deployment
styles/               # Vale prose linting rules

---

Portfolio context

Recommended GitHub profile pin order:

1. docs-portfolio - portfolio front door
2. docs-ai-operating-model - flagship AI DocsOps operating model
3. openapi-to-human-reference - API documentation transformation example
4. docsops-quality-gate - reusable PR quality gate companion project

The emphasis is clarity and consolidation: fewer repos, stronger positioning, and a visible through-line from documentation strategy to AI-assisted execution.

---

Suggested next polish

• Add a short case study showing how a docs team would adopt this model over 30/60/90 days
• Add a 60-90 second demo video showing PR -> CI -> Pages -> prompt validation -> evaluation report

---

License

MIT (see LICENSE).