family-tree

Convert GEDCOM files to lossless, AI-ready JSON. Preserve family genealogy data with perfect fidelity for current and future integrations.

🎯 Philosophy

Lossless preservation: Every piece of data from your GEDCOM files is retained, including unfamiliar tags stored in rawTags fields.

AI-ready JSON: Clean, machine-friendly structure designed for embeddings, RAG systems, and future analysis pipelines.

Future-proof: Scalable architecture ready for integrations with webtrees, FamilySearch API, and AI pipelines.

Non-technical friendly: Simple scripts and clear documentation so anyone can convert and analyze family data.

📋 Prerequisites

Node.js 20.0.0+ (download)
Your GEDCOM (.ged) files from Ancestry.com or other genealogy software

🚀 Quick Start

1. Clone or download this repository

cd family-tree

2. Install dependencies

npm install

3. Add your GEDCOM (local)

Place your .ged files in data/raw/. These files are ignored by default and should not be committed.

4. Convert GEDCOM to JSON

npm run convert -- data/raw/yourfile.ged

Output: A timestamped JSON file in data/json/yourfile-YYYY-MM-DD.json

5. Analyze the results

npm run stats -- data/json/yourfile-YYYY-MM-DD.json

Prints family counts, date ranges, top surnames, and birthplaces.

6. Validate quality (optional)

npm run validate -- data/json/yourfile-YYYY-MM-DD.json

Checks each person record against the JSON schema.

7. Deep verification (recommended)

npm run verify

Writes a dated report to data/verification-report-YYYY-MM-DD.json.

8. Build multi-source historical context (optional, long-running)

npm run fetch-sources
npm run build-context

This pipeline first fetches period+place facts from external sources (DPLA, Europeana, Chronicling America, LOC, OWID, and optional World History Encyclopedia), then uses Ollama only to synthesize those facts into narrative-ready context under data/historical-context/. Typical runtime is 20-60 minutes for a full tree range, which is expected. See docs/setup-context-sources.md for API key setup and run order.

Useful scoped builds:

npm run context:status
npm run build-context:world
npm run build-context:usa
npm run build-context:ohio
npm run build-context:indiana
npm run enrich-context
npm run build-context:reset

📁 Folder Structure

family-tree/
├── data/
│   ├── raw/              # Original .ged files (never modified)
│   ├── json/             # Converted JSON output (timestamped)
│   └── archive/          # Older versions for reference
├── scripts/
│   ├── ged-to-json.js    # Main conversion engine
│   ├── validate.js       # Schema validation
│   └── stats.js          # Statistical analysis
├── schema/
│   └── person.schema.json # JSON schema for validation
├── docs/
│   ├── data-dictionary.md # Explains every JSON field
│   └── integrations.md    # Future API integrations
└── README.md             # This file

📊 Output Format

Converted JSON follows a standardized structure:

{
  "meta": {
    "source": "myfamily.ged",
    "convertedAt": "2024-01-15T14:30:00Z",
    "gedcomVersion": "5.5.1",
    "totalIndividuals": 250,
    "totalFamilies": 80
  },
  "individuals": [
    {
      "id": "@I1@",
      "name": {
        "full": "John Henry Smith",
        "given": "John Henry",
        "surname": "Smith"
      },
      "sex": "M",
      "birth": {
        "date": "12 MAR 1845",
        "dateISO": "1845-03-12",
        "place": "Boston, Massachusetts, USA"
      },
      "death": { ... },
      "familiesAsSpouse": ["@F1@"],
      "familiesAsChild": ["@F2@"]
    }
  ],
  "families": [ ... ],
  "sources": [ ... ],
  "notes": [ ... ],
  "repositories": [ ... ]
}

Key features:

✅ Both raw GEDCOM dates AND ISO 8601 conversions
✅ Date qualifiers (ABT, BEF, AFT, BET) preserved
✅ All unrecognized tags stored in rawTags — nothing is lost
✅ Warnings logged for unparseable or unfamiliar GEDCOM elements

🔧 Commands

Convert a GEDCOM file

npm run convert -- data/raw/myfile.ged

Outputs to data/json/myfile-YYYY-MM-DD.json. Re-running produces the same output (idempotent).

Print statistics

npm run stats -- data/json/myfile-YYYY-MM-DD.json

Shows:

Person and family counts
Male/female/unknown breakdown
Earliest and latest birth/death years
Top 10 surnames
Top 10 birthplaces

Validate JSON quality

npm run validate -- data/json/myfile-YYYY-MM-DD.json

Checks each person record against the schema. Reports any validation errors.

Scaffold local document workspace

npm run scaffold

Creates documents/by-person/... folders and per-person metadata.json (local-only).

Inventory media references from GEDCOM

npm run media-inventory

Writes data/media-inventory.json showing which referenced media files are present locally vs missing.

📖 Documentation

data-dictionary.md — Complete reference for every JSON field
integrations.md — Roadmap for webtrees, FamilySearch API, and AI pipelines

🔐 Privacy

⚠️ Important: GEDCOM files may contain private information about living relatives. Consider these options:

Use a private repository: Store sensitive .ged files in a private Git repository or locally only
Anonymize before sharing: Remove dates and details for living people before publishing
Separate public/private data: Keep sensitive files in data/archive/private/

Your .git/config can use core.sparseCheckout to selectively version files.

This repo is configured for public publishing: .gitignore excludes data/raw/, data/json/*.json, and per-person documents/by-person/ content by default. See data/README.md and documents/README.md.

🗺️ Roadmap

GEDCOM → JSON conversion with lossless preservation
JSON schema validation
Statistical analysis
webtrees integration — Import JSON directly into webtrees
FamilySearch API integration (read-only) — Matching + download tooling
AI/RAG pipeline — Generate embeddings and query with LLMs
GEDCOM-X support — Upgrade to modern GEDCOM-X format
Web UI — Visual tree explorer and editor
Mobile app — React Native companion for mobile browsing

💡 Use Cases

📱 Backup & preservation: Store family data in durable JSON format
🤖 AI analysis: Generate embeddings for RAG systems
🌳 Web publishing: Easily import into webtrees for family website
📊 Research: Analyze genealogy data programmatically
🔗 Integration: Connect with FamilySearch, Ancestry, other platforms
🕐 Version control: Track genealogy changes over time with Git

⚙️ Technical Details

Parser: parse-gedcom — Fast, standard GEDCOM parsing
Validation: ajv — JSON Schema validation
Date parsing: date-fns — Robust date handling
Format: GEDCOM 5.5.1 (standard format from Ancestry.com)

🐛 Troubleshooting

Q: "File not found" error

A: Make sure your .ged file path is relative to the project root, e.g., data/raw/myfile.ged

Q: Warnings logged for many tags

A: This is normal! The converter preserves all GEDCOM tags, even non-standard ones. Check data/json/warnings-YYYY-MM-DD.log for details. They're safe in rawTags.

Q: Dates not parsing correctly

A: GEDCOM supports many date formats. If a date doesn't convert to ISO 8601, it's preserved as-is in the date field for manual inspection.

Q: How do I merge multiple GEDCOM files?

A: Not yet automated, but you can merge the individuals, families, and other arrays in the JSON files manually. Check integrations.md for upcoming merge tools.

Q: Why does context generation take so long?

A: fetch-sources + build-context process many 5-year windows across multiple scopes and enforce API/model rate limits. A full run commonly takes 20-60 minutes. Run npm run update-context quarterly for incremental refresh.

📝 License

MIT — Feel free to use, modify, and share.

🤝 Contributing

Ideas? Found a bug? Please open an issue or submit a pull request!

Happy genealogy hunting! 🌳

Before You Push Your Own Fork Public

Delete any personal data files from data/ and documents/ before committing (they are gitignored but exist in your working tree)
Confirm no private files were accidentally committed: git ls-files data/json/ data/raw/ documents/by-person/
Copy .env.example to .env and fill in your credentials (never commit .env)
Run npm run scaffold after converting your GEDCOM to create person folders
Run npm run verify to check your data quality before running any API scripts

Navigation Menu

Uh oh!

family-tree

🎯 Philosophy

📋 Prerequisites

🚀 Quick Start

1. Clone or download this repository

2. Install dependencies

3. Add your GEDCOM (local)

4. Convert GEDCOM to JSON

5. Analyze the results

6. Validate quality (optional)

7. Deep verification (recommended)

8. Build multi-source historical context (optional, long-running)

📁 Folder Structure

📊 Output Format

🔧 Commands

Convert a GEDCOM file

Print statistics

Validate JSON quality

Scaffold local document workspace

Inventory media references from GEDCOM

📖 Documentation

🔐 Privacy

🗺️ Roadmap

💡 Use Cases

⚙️ Technical Details

🐛 Troubleshooting

📝 License

🤝 Contributing

Before You Push Your Own Fork Public

Uh oh!

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

family-tree

🎯 Philosophy

📋 Prerequisites

🚀 Quick Start

1. Clone or download this repository

2. Install dependencies

3. Add your GEDCOM (local)

4. Convert GEDCOM to JSON

5. Analyze the results

6. Validate quality (optional)

7. Deep verification (recommended)

8. Build multi-source historical context (optional, long-running)

📁 Folder Structure

📊 Output Format

🔧 Commands

Convert a GEDCOM file

Print statistics

Validate JSON quality

Scaffold local document workspace

Inventory media references from GEDCOM

📖 Documentation

🔐 Privacy

🗺️ Roadmap

💡 Use Cases

⚙️ Technical Details

🐛 Troubleshooting

📝 License

🤝 Contributing

Before You Push Your Own Fork Public