Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [0.5.0] - 2026-02-04

### Added
- **Academic extension** (`codex.academic`): theorems (8 variants), proofs, exercises with hints/solutions, exercise sets, algorithms, abstracts with keywords, equation groups
- **Academic extension** (`cdx.academic`): theorems (8 variants), proofs, exercises with hints/solutions, exercise sets, algorithms, abstracts with keywords, equation groups
- Academic cross-reference marks: `theorem-ref`, `equation-ref`, `algorithm-ref` for typed internal links
- Admonition blocks from fenced Divs (note, warning, tip, danger, important, caution)
- Figure container blocks with `figcaption` children and subfigure support
Expand Down Expand Up @@ -108,10 +108,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [0.1.0] - 2025-01-25

### Added
- Initial Pandoc custom writer for Codex format
- Initial Pandoc custom writer for CDX format
- Block types: paragraph, heading, list, codeBlock, blockquote, table, math, image, horizontalRule
- Inline marks: bold, italic, code, link, strikethrough, underline, superscript, subscript
- Dublin Core metadata extraction from YAML frontmatter
- Pandoc reader for Codex JSON back to any output format
- Pandoc reader for CDX JSON back to any output format
- Full pipeline script for creating `.cdx` archives
- Integration test suite with 12 test cases
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ make test-golden
make test-reader

# Test with a specific input file
pandoc input.md -t codex.lua -o output.cdx
pandoc input.md -t cdx.lua -o output.cdx
```

Before submitting a PR, run the full suite:
Expand Down Expand Up @@ -90,7 +90,7 @@ See the [Project Structure section in README.md](README.md#project-structure) fo

## Specification Reference

This writer implements the [Codex Document Format Specification](https://github.com/Entrolution/codex-file-format-spec). When implementing new features:
This writer implements the [CDX Document Format Specification](https://github.com/Entrolution/codex-file-format-spec). When implementing new features:

- Reference the relevant spec section
- Note any deviations or extensions
Expand Down
2 changes: 1 addition & 1 deletion LICENSE-APACHE
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ APPENDIX: How to apply the Apache License to your work.
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright 2025 Codex Document Format Contributors
Copyright 2025 CDX Document Format Contributors

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion LICENSE-MIT
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2025 Codex Document Format Contributors
Copyright (c) 2025 CDX Document Format Contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
8 changes: 4 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Makefile for Codex Pandoc Writer
# Makefile for CDX Pandoc Writer

PANDOC := pandoc
WRITER := codex.lua
WRITER := cdx.lua
JQ := jq

# Test input files
Expand All @@ -14,7 +14,7 @@ TEST_CDX := $(patsubst tests/inputs/%.md,tests/outputs/%.cdx,$(TEST_INPUTS))
all: test

help:
@echo "Codex Pandoc Writer"
@echo "CDX Pandoc Writer"
@echo ""
@echo "Targets:"
@echo " test-all Run everything (lint + test + golden + reader + validate)"
Expand Down Expand Up @@ -108,7 +108,7 @@ validate-schema: test-json
@echo "Validating against spec schemas..."
@for f in tests/outputs/*.json; do \
echo "Checking $$f..."; \
$(JQ) -e '.manifest.codex' $$f > /dev/null || { echo " WARN: no manifest.codex version"; continue; }; \
$(JQ) -e '.manifest.cdx' $$f > /dev/null || { echo " WARN: no manifest.cdx version"; continue; }; \
$(JQ) -e '.content.blocks | type == "array"' $$f > /dev/null || { echo " FAIL: content.blocks is not an array"; exit 1; }; \
$(JQ) -e '.dublin_core.terms | type == "object"' $$f > /dev/null || { echo " FAIL: dublin_core.terms is not an object"; exit 1; }; \
$(JQ) -e '[.content.blocks[] | .type] | all(. != null and . != "")' $$f > /dev/null || { echo " FAIL: block missing type field"; exit 1; }; \
Expand Down
30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
[![CI](https://github.com/Entrolution/cdx-pandoc/actions/workflows/ci.yml/badge.svg)](https://github.com/Entrolution/cdx-pandoc/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue.svg)](LICENSE-MIT)

Pandoc custom writer and reader for [Codex Document Format](https://github.com/Entrolution/codex-file-format-spec) (`.cdx`) files.
Pandoc custom writer and reader for [CDX Document Format](https://github.com/Entrolution/codex-file-format-spec) (`.cdx`) files.

## Overview

This writer enables conversion from any Pandoc-supported format (Markdown, LaTeX, Word, etc.) to Codex Document Format. The conversion happens in two phases:
This writer enables conversion from any Pandoc-supported format (Markdown, LaTeX, Word, etc.) to CDX Document Format. The conversion happens in two phases:

1. **Lua Writer** (`codex.lua`) - Converts Pandoc AST to Codex JSON structures
1. **Lua Writer** (`cdx.lua`) - Converts Pandoc AST to CDX JSON structures
2. **Packaging** - Shell wrapper creates the final .cdx ZIP archive

## Requirements
Expand All @@ -26,7 +26,7 @@ This writer enables conversion from any Pandoc-supported format (Markdown, LaTeX
Generate the intermediate JSON structure:

```bash
pandoc input.md -t codex.lua -o output.json
pandoc input.md -t cdx.lua -o output.json
```

### Full Pipeline (Recommended)
Expand All @@ -48,9 +48,9 @@ Any format Pandoc can read:
- reStructuredText (.rst)
- And many more...

## Reading Codex Documents
## Reading CDX Documents

Convert Codex JSON back to any Pandoc-supported output format:
Convert CDX JSON back to any Pandoc-supported output format:

```bash
pandoc -f cdx-reader.lua output.json -o document.md
Expand All @@ -64,7 +64,7 @@ The reader handles core block types (paragraphs, headings, lists, code blocks, b

### Block Types Supported

| Pandoc | Codex | Notes |
| Pandoc | CDX | Notes |
|--------|-------|-------|
| Para | paragraph | Text content |
| Header | heading | Levels 1-6, preserves IDs |
Expand Down Expand Up @@ -96,7 +96,7 @@ The reader handles core block types (paragraphs, headings, lists, code blocks, b

### Inline Formatting

| Pandoc | Codex Mark |
| Pandoc | CDX Mark |
|--------|------------|
| Strong | bold |
| Emph | italic |
Expand Down Expand Up @@ -224,7 +224,7 @@ The writer produces a JSON structure with three sections:
```json
{
"manifest": {
"codex": "0.1",
"cdx": "0.1",
"id": "pending",
"state": "draft",
"created": "2025-01-28T10:00:00Z",
Expand All @@ -248,7 +248,7 @@ The writer produces a JSON structure with three sections:
}
```

The packaging script extracts these into the proper Codex directory structure.
The packaging script extracts these into the proper CDX directory structure.

## Development

Expand All @@ -267,8 +267,8 @@ make test-reader # Test round-trip (JSON → markdown)

```
cdx-pandoc/
├── codex.lua # Main Pandoc custom writer
├── cdx-reader.lua # Codex → Pandoc reader
├── cdx.lua # Main Pandoc custom writer
├── cdx-reader.lua # CDX → Pandoc reader
├── lib/
│ ├── blocks.lua # Writer: block type converters
│ ├── inlines.lua # Writer: inline/text node converters
Expand Down Expand Up @@ -336,14 +336,14 @@ Convert:

### "Cannot find library" error

Ensure you're running pandoc from the project root directory, or that the `lib/` directory is in the same location as `codex.lua`.
Ensure you're running pandoc from the project root directory, or that the `lib/` directory is in the same location as `cdx.lua`.

### Citations not appearing in bibliography

Add the `--citeproc` flag to enable Pandoc's citation processor:

```bash
pandoc input.md --citeproc -t codex.lua -o output.json
pandoc input.md --citeproc -t cdx.lua -o output.json
```

### Empty or missing metadata
Expand All @@ -368,7 +368,7 @@ Inline math uses single dollar signs: `$x = y$`. Inline math is preserved as a `

### Reader round-trip loses semantic data

The reader converts Codex back to standard Pandoc elements. Most block types survive round-trip faithfully:
The reader converts CDX back to standard Pandoc elements. Most block types survive round-trip faithfully:

- **Core blocks**: paragraphs, headings, lists, tables, code, math, images, figures, definition lists, admonitions all round-trip cleanly.
- **Academic blocks**: theorems (variant, id, title), proofs (of, method), exercises (difficulty, hints, solutions), algorithms (title, pseudocode), abstracts (keywords), and equation groups (reconstructed LaTeX environments) all survive via Pandoc Div attributes.
Expand Down
4 changes: 2 additions & 2 deletions cdx-reader.lua
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
-- Codex Document Format → Pandoc Reader
-- Converts Codex JSON to Pandoc AST for output to any format.
-- CDX Document Format → Pandoc Reader
-- Converts CDX JSON to Pandoc AST for output to any format.
--
-- Usage:
-- pandoc -f cdx-reader.lua output.json -o document.md
Expand Down
12 changes: 6 additions & 6 deletions codex.lua → cdx.lua
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
-- codex.lua
-- Pandoc custom writer for Codex Document Format
-- cdx.lua
-- Pandoc custom writer for CDX Document Format
--
-- Usage:
-- pandoc input.md -t codex.lua -o output.json
-- pandoc input.md -t cdx.lua -o output.json
--
-- Output: JSON containing manifest, content, and dublin_core sections
-- that can be unpacked into a Codex directory structure and packaged.
-- that can be unpacked into a CDX directory structure and packaged.

-- Get the directory containing this script
local script_dir = PANDOC_SCRIPT_FILE and (PANDOC_SCRIPT_FILE:match("(.*/)" ) or "") or ""
Expand Down Expand Up @@ -52,7 +52,7 @@ academic.set_extension_tracker(track_extension)
blocks.set_academic(academic)

-- Spec version
local CODEX_VERSION = "0.1"
local CDX_VERSION = "0.1"
local CONTENT_VERSION = "0.7.1"

-- Generate ISO 8601 timestamp
Expand All @@ -70,7 +70,7 @@ local function create_manifest()
local now = iso_timestamp()

return {
codex = CODEX_VERSION,
cdx = CDX_VERSION,
id = "pending",
state = "draft",
created = now,
Expand Down
6 changes: 3 additions & 3 deletions lib/academic.lua
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
-- lib/academic.lua
-- Writer-side academic extension block conversion
-- Converts Pandoc Divs/RawBlocks to academic:* Codex blocks
-- Converts Pandoc Divs/RawBlocks to academic:* CDX blocks

local utils = dofile((PANDOC_SCRIPT_FILE and (PANDOC_SCRIPT_FILE:match("(.*/)" ) or "") or "") .. "lib/utils.lua")
local has_class = utils.has_class
Expand All @@ -12,7 +12,7 @@ local M = {}
-- Module references (set by init)
local blocks = nil

-- Extension tracker function (set by codex.lua)
-- Extension tracker function (set by cdx.lua)
local track_extension = function() end

function M.set_blocks(mod)
Expand Down Expand Up @@ -392,7 +392,7 @@ end
-- Convert a Div block to its academic type
-- @param block Pandoc Div element
-- @param academic_type The classified type string
-- @return Codex block
-- @return CDX block
function M.convert_div(block, academic_type)
track_extension(utils.EXT_ACADEMIC)
if theorem_variants[academic_type] then
Expand Down
16 changes: 8 additions & 8 deletions lib/bibliography.lua
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ local meta_to_string = utils.meta_to_string

local M = {}

-- Map CSL types to Codex entry types
-- Map CSL types to CDX entry types
M.TYPE_MAP = {
["article-journal"] = "article-journal",
["article-magazine"] = "article-magazine",
Expand Down Expand Up @@ -139,8 +139,8 @@ function M.extract_date(date_val)
return result.year and result or nil
end

-- Simple string fields to copy from CSL reference to Codex entry
-- Each entry is {csl_field, codex_field} (codex_field defaults to csl_field)
-- Simple string fields to copy from CSL reference to CDX entry
-- Each entry is {csl_field, cdx_field} (cdx_field defaults to csl_field)
local SIMPLE_FIELDS = {
{"title"},
{"container-title"},
Expand All @@ -155,9 +155,9 @@ local SIMPLE_FIELDS = {
{"abstract"},
}

-- Extract a single CSL reference entry to Codex format
-- Extract a single CSL reference entry to CDX format
-- @param ref CSL reference (MetaMap or table)
-- @return Codex bibliography entry
-- @return CDX bibliography entry
function M.extract_entry(ref)
if not ref then
return nil
Expand All @@ -180,7 +180,7 @@ function M.extract_entry(ref)
return nil
end

-- Type (map to Codex types or keep as-is)
-- Type (map to CDX types or keep as-is)
local ref_type = get_field("type")
entry.type = M.TYPE_MAP[ref_type] or ref_type or "other"

Expand All @@ -198,9 +198,9 @@ function M.extract_entry(ref)
-- Simple string fields
for _, field in ipairs(SIMPLE_FIELDS) do
local csl_name = field[1]
local codex_name = field[2] or csl_name
local cdx_name = field[2] or csl_name
local val = get_field(csl_name)
if val then entry[codex_name] = val end
if val then entry[cdx_name] = val end
end

return entry
Expand Down
14 changes: 7 additions & 7 deletions lib/blocks.lua
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
-- lib/blocks.lua
-- Convert Pandoc blocks to Codex block structures
-- Convert Pandoc blocks to CDX block structures

local utils = dofile((PANDOC_SCRIPT_FILE and (PANDOC_SCRIPT_FILE:match("(.*/)" ) or "") or "") .. "lib/utils.lua")
local has_class = utils.has_class
Expand All @@ -15,10 +15,10 @@ local inlines = nil
-- Academic module (will be set by init, optional)
local academic = nil

-- Extension tracker function (set by codex.lua)
-- Extension tracker function (set by cdx.lua)
local track_extension = function() end

-- Bibliography context (set by codex.lua before conversion)
-- Bibliography context (set by cdx.lua before conversion)
local bib_context = {
csl_entries = {},
style = "unknown"
Expand All @@ -45,9 +45,9 @@ function M.set_bibliography_context(csl_entries, style)
bib_context.style = style or "unknown"
end

-- Convert a list of Pandoc blocks to Codex blocks
-- Convert a list of Pandoc blocks to CDX blocks
-- @param blocks Pandoc block list
-- @return Array of Codex blocks
-- @return Array of CDX blocks
function M.convert(blocks)
if not blocks then
return {}
Expand Down Expand Up @@ -109,9 +109,9 @@ block_handlers.LineBlock = function(block)
}
end

-- Convert a single Pandoc block to a Codex block
-- Convert a single Pandoc block to a CDX block
-- @param block Pandoc block element
-- @return Codex block table or nil
-- @return CDX block table or nil
function M.convert_block(block)
local tag = block.t or block.tag
local handler = block_handlers[tag]
Expand Down
8 changes: 4 additions & 4 deletions lib/inlines.lua
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
-- lib/inlines.lua
-- Convert Pandoc inline elements to Codex text nodes
-- Convert Pandoc inline elements to CDX text nodes

-- Load shared utilities
local utils = dofile((PANDOC_SCRIPT_FILE and (PANDOC_SCRIPT_FILE:match("(.*/)" ) or "") or "") .. "lib/utils.lua")
Expand All @@ -8,7 +8,7 @@ local has_class = utils.has_class

local M = {}

-- Extension tracker function (set by codex.lua)
-- Extension tracker function (set by cdx.lua)
local track_extension = function() end

-- Set the extension tracker function
Expand Down Expand Up @@ -505,11 +505,11 @@ function M.merge_adjacent(nodes)
return result
end

-- Convert a list of Pandoc inlines to Codex text nodes
-- Convert a list of Pandoc inlines to CDX text nodes
-- This is the main entry point
-- @param inlines Pandoc inline list
-- @param ctx Optional context for state accumulation (defaults to global)
-- @return Array of Codex text nodes
-- @return Array of CDX text nodes
function M.convert(inlines, ctx)
if not inlines then
return {}
Expand Down
2 changes: 1 addition & 1 deletion lib/json.lua
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
-- lib/json.lua
-- JSON encoding utilities for Codex Pandoc writer
-- JSON encoding utilities for CDX Pandoc writer

local M = {}

Expand Down
Loading
Loading