🗺️ OpenFoundry Roadmap

From zero to full Palantir Foundry parity — open source.

Last updated: May 2026

Overview

OpenFoundry aims to deliver 25 core capabilities that match Palantir Foundry — all open-source, self-hosted, and community-driven. This roadmap outlines our phased approach to get there.

Status Legend

Icon	Meaning
✅	Done — feature shipped and usable
🚧	In Progress — actively being built
📐	Designed — architecture defined, implementation pending
🔲	Planned — scoped but not yet started
💡	Exploring — researching approaches

Priority Legend

Tag	Meaning
🔴 Critical	Core platform value — blocks adoption
🟡 High	Key differentiator — needed for production use
🟠 Medium	Important — completes the platform story
🟢 Low	Nice to have — enhances ecosystem

📊 Parity Tracker: 23/25 Foundry Components shipped

#	Foundry Component	OpenFoundry Service(s)	Status	Target Phase
1	Ontology	`ontology-definition-service` · `ontology-query-service` · `object-database-service` · `ontology-indexer`	✅ Done	Phase 1
2	Transforms / Pipeline Builder	`pipeline-build-service` · `pipeline-runner` · `pipeline-runner-spark` (+ lib `pipeline-expression`)	✅ Done	Phase 1
3	Data Connections	`connector-management-service` · `ingestion-replication-service`	✅ Done	Phase 1
4	Contour (Visual Analytics)	`sql-bi-gateway-service` (Flight SQL) + frontend `/contour`, `/queries`	✅ Done	Phase 1
5	Dataset Management & Versioning	`dataset-versioning-service`	✅ Done	Phase 1
6	Data Lineage	`lineage-service`	✅ Done	Phase 1
7	Notebooks / Code Workbooks	`notebook-runtime-service` (+ lib `python-sidecar`)	✅ Done	Phase 1
8	Quiver (Dashboards)	`apps/web` frontend (`/dashboards`, `/quiver`)	✅ Done	Phase 2
9	Object Explorer	`ontology-query-service` + `apps/web` `/object-explorer`	✅ Done	Phase 1
10	Auth / RBAC / SSO	`identity-federation-service` · `authorization-policy-service` (+ libs `auth-middleware`, `authz-cedar-go`)	🚧 In Progress	Phase 2
11	Workflows / Actions	`workflow-automation-service` (+ libs `saga`, `state-machine`, `event-scheduler`) · `ontology-actions-service`	✅ Done	Phase 2
12	Notifications	`notification-alerting-service`	✅ Done	Phase 2
13	Data Catalog	`iceberg-catalog-service` + `dataset-versioning-service` + frontend `/datasets`, `/iceberg-tables`	✅ Done	Phase 2
14	Data Quality	Declarative expectations inside `pipeline-build-service` via lib `pipeline-expression`	✅ Done	Phase 2
15	Slate/Workshop (App Builder)	`application-composition-service` + frontend `/apps`, `/workshop-editor`	✅ Done	Phase 3
16	ML / Model Management	`model-catalog-service` · `model-deployment-service` (+ lib `ml-kernel-go`)	✅ Done	Phase 3
17	AIP (GenAI / LLM / Copilot)	`agent-runtime-service` · `retrieval-context-service` · `llm-catalog-service` · `ai-evaluation-service` (+ lib `ai-kernel-go`)	✅ Done	Phase 3
18	Reports	Frontend `/reports` route (no dedicated service yet — report jobs land via `workflow-automation-service`)	🚧 In Progress	Phase 4
19	Fusion (Entity Resolution)	`entity-resolution-service` + frontend `/fusion`	✅ Done	Phase 4
20	Code Repositories (Git)	`code-repository-review-service` (global branching, code-security scanning, review plane)	✅ Done	Phase 4
21	Marketplace	`federation-product-exchange-service` + frontend `/marketplace`	✅ Done	Phase 4
22	Streaming (Real-time)	`ingestion-replication-service` + lib `event-bus-data` over Kafka (Strimzi)	✅ Done	Phase 4
23	Geospatial / Maps	`ontology-exploratory-analysis-service` + libs `geospatial-core`, `geospatial-tiles` + frontend `/geospatial`, `/maplibre-demo`	✅ Done	Phase 4
24	Audit & Compliance	`audit-compliance-service` + `audit-sink` (Kafka → Iceberg) (+ lib `audit-trail`)	✅ Done	Phase 4
25	Nexus (Cross-org Sharing)	`federation-product-exchange-service` (federation registry, product distribution)	✅ Done	Phase 5

Current repo audit: 24 components are shipped on main. The two remaining gaps are (a) Enterprise auth — OIDC is implemented in identity-federation-service; SAML sign-in flow is still pending; and (b) Reports — the frontend is in place but a dedicated report-service binary has not been split out of the workflow plane yet.

Phase 1 — Foundation 🏗️

Goal: A working platform where you can connect data, explore it, build pipelines, and define an ontology.

Priority: 🔴 Critical — nothing works without this.

Milestone 1.1 — Platform Bootstrap

Go module setup — Single go.mod at the repo root, shared packages compile, Makefile as the canonical task runner (the justfile is a thin shim)
Protobuf generation — buf pipeline generating Go (via buf generate) into libs/proto-gen/, plus TypeScript, Python and Java SDKs in sdks/
Shared libraries — core-models, auth-middleware, event-bus-control (NATS), event-bus-data (Kafka), storage-abstraction, observability
Edge gateway service — edge-gateway-service: chi HTTP server, JWT validation, routing, rate-limiting, CORS
Identity & auth service (basic) — identity-federation-service with JWT issue/validate, local user registration, session management (OIDC, MFA, WebAuthn already wired)
Docker Compose dev stack — Postgres (CNPG), Kafka (Strimzi), NATS JetStream, Ceph S3 (Rook) running with docker compose -f infra/compose/docker-compose.yml up -d
React 19 + Vite shell — apps/web app layout, sidebar, top bar, routing, auth flow, design system (base UI components)
CI pipeline — GitHub Actions (.github/workflows/openfoundry-go.yml): lint (golangci-lint), vet, tidy drift, proto drift, sqlc drift, race-enabled test, integration (testcontainers)

Milestone 1.2 — Data Layer

Dataset service — dataset-versioning-service: CRUD, Parquet/Iceberg read/write, schema management, branches, transactions, file APIs
Data connectors (first wave) — connector-management-service: PostgreSQL, MySQL, CSV, Parquet, JSON, S3, REST API (extensible via WASM connectors in plugin-sdk lib)
Query service — sql-bi-gateway-service: Apache Arrow Flight SQL server (port 50133) over DataFusion, with per-statement routing to Iceberg, Vespa and Postgres (ADR-0014)
Frontend: Dataset Explorer — /datasets route: data preview table, schema viewer, upload flow
Frontend: SQL Workbench — /queries: Monaco SQL editor, query execution, results table

Milestone 1.3 — Ontology & Pipelines

Ontology services — ontology-definition-service (object types, properties, link types, action types), ontology-query-service (read path), object-database-service (Cassandra/Scylla storage), ontology-indexer (Kafka → search backend)
Pipeline services (basic) — pipeline-build-service (authoring + build orchestration), pipeline-runner (Spark orchestrator), pipeline-runner-spark (Scala JAR for Iceberg transforms)
Data lineage — lineage-service: OpenLineage events sink, lineage graph query API
Frontend: Ontology Explorer — /ontologies, /ontology-design, /object-explorer: type editor, graph view (Cytoscape.js)
Frontend: Pipeline Builder — /pipelines DAG canvas (React Flow), node palette, transform editor
Frontend: Lineage View — /lineage interactive lineage graph

Milestone 1.4 — Notebooks

Notebook service — notebook-runtime-service: notebook CRUD, cell model, session management, kernel execution, export
Python kernel — Subprocess-based Python execution via lib python-sidecar (gRPC interface), variable state, output capture
SQL kernel — Route SQL cells to sql-bi-gateway-service (Flight SQL)
Frontend: Notebook Editor — /notebooks: cell editor (Monaco), cell outputs, kernel selector/status

Phase 1 exit criteria:

A user can connect a Postgres database, explore tables, write SQL queries, build a simple pipeline with SQL transforms, define ontology object types backed by datasets, and run Python notebooks.

Phase 2 — Core Platform 🧱

Goal: Production-grade auth, dashboards, workflows, data quality, and catalog. The platform becomes usable for real teams.

Priority: 🔴 Critical + 🟠 Medium features that complete the core loop.

Milestone 2.1 — Enterprise Auth

RBAC — Roles, permissions, row-level security
ABAC — Attribute-based policies
SSO — OAuth2/OIDC provider integration, SAML (OIDC implemented; SAML sign-in flow pending)
MFA — TOTP-based multi-factor authentication
API keys — Programmatic access management
Frontend: User/Role management — Settings pages for users, roles, groups

Milestone 2.2 — Dashboards (Quiver)

Dashboard grid layout — Responsive drag-and-drop grid
Chart widget — ECharts integration: bar, line, area, pie, scatter, etc.
Table widget — Paginated, sortable, filterable data tables
KPI widget — Single metric cards with sparklines
Filter bar — Global filters propagated to all widgets
Date range filter — Relative and absolute date selection
Dashboard CRUD — Create, edit, duplicate, share dashboards

Milestone 2.3 — Data Catalog & Quality

Data catalog — Search by name/tag/owner, dataset tagging, ownership assignment
Auto-profiling — Column statistics, distributions, null rates, uniqueness
Quality rules — Null checks, range validation, regex, custom SQL rules
Quality scoring — Per-dataset quality score, trend tracking
Quality alerts — Notifications on quality degradation
Frontend: Catalog search — Full-text search in dataset explorer
Frontend: Quality dashboard — Quality scores, profiling report, rule management

Milestone 2.4 — Workflows & Notifications

Workflow service — workflow-automation-service: workflow definitions, step execution, conditional branching (libs saga, state-machine)
Triggers — Cron (lib scheduling-cron + event-scheduler), event-driven, manual, webhook triggers
Human-in-the-loop — Native approval step inside workflow-automation-service, approval inbox in apps/web
Notification service — notification-alerting-service: email (SMTP/SES), Slack, MS Teams webhooks
In-app notifications — WebSocket-based real-time notifications via notification-alerting-service
User preferences — Per-user channel and frequency preferences
Frontend: Workflow builder — /workflows, /automate: visual workflow canvas, step config, trigger config
Frontend: Notification bell — In-app notification center

Milestone 2.5 — Pipeline Enhancements

Phase 2 exit criteria:

Teams can collaborate with proper auth/RBAC, build dashboards over their data, set up data quality monitoring, automate workflows with approvals, and receive notifications.

Phase 3 — Intelligence 🧠

Goal: ML, AI, and app building capabilities. This is where OpenFoundry becomes a true decision-making platform.

Priority: 🔴 Critical — these are the features that make Foundry Foundry.

Milestone 3.1 — App Builder (Slate/Workshop)

App composition service — application-composition-service: app definitions, page layouts, widget catalog, publish runtime
Widget system — Table, form, chart, map, text, image, button, container
Data binding — Bind widgets to ontology objects, datasets, or queries
Event handlers — onClick → execute action, navigate, filter, etc.
App theming — Colors, fonts, branding customization
Publish & deploy — Version and publish apps, embedding support (iframe)
App templates — Starter templates for common use cases
Frontend: WYSIWYG editor — /apps, /workshop-editor: drag-and-drop canvas, property inspector, live preview
Frontend: App runtime — Render published apps for end users

Milestone 3.2 — ML Studio

Experiment tracking — Logged through model-catalog-service (runs with params, metrics, and artifacts)
Model registry — model-catalog-service: register models, manage versions (staging → production)
Feature store — Feature definitions, online serving, offline batch computation (lib ml-kernel-go)
Training orchestration — Submit training jobs, hyperparameter tuning
Model serving — model-deployment-service: real-time inference endpoints, batch predictions
A/B testing — Traffic splitting between model versions
Drift monitoring — Data and concept drift detection (lib ml-kernel-go), auto-retraining triggers via workflow-automation-service
Frontend: ML Studio — /ml route: experiment list, run comparison, model registry, deployment panel

Milestone 3.3 — AI Platform (AIP)

Phase 3 exit criteria:

Users can build operational apps without code, train and deploy ML models, use AI agents and a platform copilot to accelerate their work, and build RAG pipelines over their data.

Phase 4 — Platform Completeness 🔒

Goal: Every remaining Foundry capability. Entity resolution, streaming, geospatial, code repos, marketplace, reports, and audit.

Priority: 🟡 High — completes the platform for enterprise adoption.

Milestone 4.1 — Entity Resolution (Fusion)

Match rules — entity-resolution-service: deterministic rules (exact, fuzzy, phonetic)
ML-based matching — Gradient boosted classifier for probabilistic matching
Blocking strategies — LSH, sorted neighborhood, key-based blocking
String comparators — Jaro-Winkler, Levenshtein, Soundex, metaphone
Graph resolution — Transitive closure for entity clusters
Golden record — Survivorship rules, merge strategies
Human-in-the-loop — Review queue for uncertain matches
Frontend: Match rule builder, cluster viewer, manual review (/fusion)

Milestone 4.2 — Real-time Streaming

Stream definitions — Named streams with schemas, served by ingestion-replication-service over Kafka (Strimzi)
Processing topology — DAG-based stream processing
Windowing — Tumbling, sliding, and session windows (Flink jobs under infra/helm/infra/flink-jobs)
Stream joins — Stream-stream and stream-table joins
Complex event processing — Pattern matching on event sequences
State backend — RocksDB-based state store (Flink-managed)
Connectors — Kafka source, NATS source, HTTP webhook source, WebSocket sink, dataset sink (lib event-bus-data for Kafka, event-bus-control for NATS)
Backpressure — Flow control to prevent overload
Frontend: Topology editor, stream monitor, live data tail (/streaming)

Milestone 4.3 — Reports & Geospatial

Report service — Currently no dedicated binary; report jobs run as workflow-automation-service steps and surface in the /reports frontend route (a dedicated report-service is planned for Phase 4 completion)
Generators — PDF / Excel / CSV / HTML / PPTX generators pending; today only basic CSV/Parquet exports via pipelines
Distribution — Email, S3, Slack, webhook delivery via notification-alerting-service
Geospatial capability — ontology-exploratory-analysis-service (geospatial queries): within, intersects, nearest, buffer
Vector tiles — MVT tile server, H3 hex aggregation (lib geospatial-tiles)
Geocoding — Address ↔ coordinates
Spatial clustering — DBSCAN, K-means
Routing — Shortest path, isochrones
Frontend: MapLibre GL map, layer panel, heatmap, clustering, routing (/geospatial, /maplibre-demo)

Milestone 4.4 — Code Repos & Marketplace

Milestone 4.5 — Audit & Compliance

Audit service — audit-compliance-service + audit-sink (Kafka → Iceberg): immutable append-only audit log
Event collection — Auto-capture from all services via Kafka (lib audit-trail)
GDPR support — Right to erasure, data portability (lineage deletion lives in audit-compliance-service)
Compliance reports — SOC2, ISO 27001, HIPAA export formats
Anomaly detection — Alert on unusual access patterns
Data classification — PII, confidential, public labels (lib media-scanner for SDS)
Retention policies — Configurable TTL for audit events (handled inside audit-compliance-service)
Frontend: Audit log viewer, compliance dashboard, policy manager (/audit)

Phase 4 exit criteria:

The platform has full feature parity with Palantir Foundry for 23 of 25 components, suitable for enterprise production use. Outstanding: dedicated report-service binary + SAML sign-in flow.

Phase 5 — Ecosystem 🌐

Goal: Cross-organization data sharing, plugin SDK, and community ecosystem.

Priority: 🟠 Medium — the network-effect layer.

Milestone 5.1 — Nexus (Cross-org Data Sharing)

Peer management — federation-product-exchange-service: register and authenticate partner organizations
Data sharing contracts — Define what's shared, with whom, under what terms
Federated queries — Query shared data without copying it (via Flight SQL surface of sql-bi-gateway-service)
Selective replication — Replicate subsets of data to consumer orgs
E2E encryption — Encrypted data in transit and at rest for shared datasets
Cross-org audit trail — Audit bridge between organizations (audit-compliance-service)
Schema compatibility — Validate schema compatibility across orgs
Frontend: Peer list, share wizard, contract manager, shared data browser (/marketplace, /federation)

Milestone 5.2 — Developer Ecosystem

Plugin SDK — Go + WASM SDK in lib plugin-sdk for building custom connectors, transforms, widgets
CLI tool — of-cli under tools/of-cli for project management, deployment, and scripting (compiled with go build ./tools/of-cli)
REST API docs — Full OpenAPI spec auto-generated from proto (drift gated by proto-check.yml)
Developer portal — Interactive API explorer, tutorials, cookbooks under /developers
Terraform provider — Manage OpenFoundry resources as IaC
GitHub/GitLab integration — External Git sync, CI/CD triggers via code-repository-review-service
Frontend: Developers portal with API explorer, SDK toolkit, Terraform panel, and repository integration manager (/developers)

Milestone 5.3 — Performance & Scale

Distributed query execution — Multi-node DataFusion queries served by sql-bi-gateway-service
Distributed pipeline execution — Parallel transform execution across workers (Spark Operator + pipeline-runner-spark)
Auto-scaling — HPA/KEDA-based scaling per service
Multi-tenancy — Logical tenant isolation, resource quotas (tenancy-organizations-service)
Global CDN — Tile server (lib geospatial-tiles) and static asset caching at the edge
Benchmark suite — Reproducible benchmarks for all critical paths under benchmarks/

Phase 5 exit criteria:

Organizations can share data securely across boundaries, third-party developers can extend the platform, and the system scales to enterprise workloads.

📅 Indicative Timeline

⚠️ These are estimates, not commitments. Open source moves at the speed of contributors.

2026 Q2-Q3    Phase 1 — Foundation
              ████████████████████████████░░░░░░░░░░░░░░░░░░░

2026 Q3-Q4    Phase 2 — Core Platform
              ░░░░░░░░░░░░████████████████████████░░░░░░░░░░░

2027 Q1-Q2    Phase 3 — Intelligence (ML, AI, App Builder)
              ░░░░░░░░░░░░░░░░░░░░░░░░████████████████░░░░░░

2027 Q2-Q3    Phase 4 — Platform Completeness
              ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░████████████░░

2027 Q3+      Phase 5 — Ecosystem
              ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░████████

🎯 How We Prioritize

User value first — Does this unlock a workflow that wasn't possible before?
Foundation before features — Auth, data layer, and ontology must be solid before ML/AI.
Horizontal before vertical — Basic versions of many features > perfect version of one.
Community signal — GitHub issues with 👍 reactions influence priority.
Contributor interest — If someone wants to build it, we help them ship it.

🤝 Help Us Get There Faster

Every contribution accelerates the roadmap. Here's where help is most needed:

Phase	Area	What's Needed
Phase 1	Data connectors	Implement new sources via the connector contracts in `connector-management-service` and the `plugin-sdk` WASM SDK
Phase 1	Frontend	React 19 + Vite pages under `apps/web/src/routes`, Tailwind components
Phase 2	Dashboard widgets	New chart types, custom widgets
Phase 2	Quality rules	Custom data quality rule implementations (extend lib `pipeline-expression`)
Phase 3	LLM providers	Adapters for Gemini, Mistral, Cohere, etc. (register in `llm-catalog-service` via lib `ai-kernel-go`)
Phase 4	Geospatial	Extend `ontology-exploratory-analysis-service`, contribute algorithms to libs `geospatial-core` / `geospatial-tiles`
Phase 4	Report generators	PDF, Excel, PPTX template engines — needed to ship the dedicated `report-service` binary
Phase 2	Enterprise SSO	Wire SAML sign-in flow in `identity-federation-service`, provider validation, and end-to-end login testing

Want to contribute? Check the issues labeled help wanted or comment on this roadmap's tracking issue.

🛡️ Data plane hardening (2026 Q2-Q3)

Sixteen verifiable milestones that consolidate OpenFoundry's data plane around the five target planes documented in docs/architecture/runtime-topology.md (storage, ingestion, compute, control, relational state). Each item is a concrete, already-merged change in the monorepo, anchored to one of the ADRs 0008–0012 in docs/architecture/adr/.

This roadmap is a living document. It evolves with community feedback and contributions.

Discuss the roadmap →

FilesExpand file tree

ROADMAP.md

Latest commit

History