feat: Source Discovery API for multi-source ontology management#1
Open
vbrevik wants to merge 41 commits into
Open
feat: Source Discovery API for multi-source ontology management#1vbrevik wants to merge 41 commits into
vbrevik wants to merge 41 commits into
Conversation
The ontology-manager references ontology content from the multi-perspective-context-ontology repo via a symlink at data/mpcg-ontology. This is a local path reference, not tracked. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Historical planning docs for @mpcg/core, @mpcg/api, @mpcg/web packages that were superseded by this repo. Preserved for reference. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
data/sources.json lists available ontology data sources (symlinked). Two sources configured: - system-ontology: platform operations ontology (extracted to ontology-data repo) - mpcg-ontology: multi-perspective context ontology This establishes the pattern for the tool to work with any ontology data source, decoupling tool from data. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Create ontology_sources table with CHECK and partial unique indexes - Add source_id column to classes, properties, relationship_types - Replace unique constraints with NULL-safe partial indexes - Add 12 migration verification tests Plan: section-01-migration.md Co-Authored-By: Claude <noreply@anthropic.com>
- OntologySource DB row model (FromRow) - SourcesConfig, SourceEntry, SourceManifest filesystem models - SourceResponse, ActiveSourcesResponse, SetActiveInput API types - 5 unit tests for serde deserialization Plan: section-02-models.md Co-Authored-By: Claude <noreply@anthropic.com>
…ment - Add service with discover_sources, get_active_sources, set_active_sources, sync_sources_to_db - Filesystem discovery with symlink detection, manifest parsing, and 500ms timeout - Transaction-wrapped atomic flag updates for base/extension assignment - Batch DB enrichment query (replaces N+1 pattern) - 5 unit tests (filesystem discovery) + 8 integration tests (DB operations) Plan: section-03-service.md Co-Authored-By: Claude <noreply@anthropic.com>
- Router factory with GET /, GET /active, PUT /active endpoints - Pure delegation to OntologySourceService — no business logic in handlers Plan: section-04-routes.md Co-Authored-By: Claude <noreply@anthropic.com>
…t harness - Add ontology_data_dir config field with "./data" default - Create OntologySourceService in main.rs, nest routes under /ontology-sources - Update TestServices, create_test_config, jwt_helpers with new config field Plan: section-05-config-integration.md Co-Authored-By: Claude <noreply@anthropic.com>
- Add test_config_default_data_dir and test_service_in_test_services - Complete test suite: 22 tests across models, service, migration, and config Plan: section-06-tests.md Co-Authored-By: Claude <noreply@anthropic.com>
- Migration: add is_system BOOLEAN to classes, create source_conflicts table with indexes - Update Class, Property, RelationshipType structs with source_id and is_system fields Plan: section-01-migration.md Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Backend rate limit module updates, migration, and test coverage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Backup agent improvements, docker-compose updates, and ports documentation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sidebar layout, workspace switcher, ontology API, user roles, targeting module, permission engine, and route updates. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Security phase 2 scope, rate limiting reports, network segmentation, immutable backups, disaster recovery, and test coverage reports. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Source discovery API plans, import engine spec/plan/sections, ontology browser and source management UI specs, and project manifest. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Intermediate representation structs, API response types, JSON deserialization structs, ImportParams, and ImportError with sanitized error responses. Updates ClassWithParent for schema compatibility. 15 unit tests. Plan: section-02-models.md Co-Authored-By: Claude <noreply@anthropic.com>
JSON adapter parses system-ontology format (classes, properties, relationship_types). Schema adapter walks MPCG taxonomy trees with active type detection from JSON Schema. Includes dispatch, validation with error accumulation, and topological sort. 16 adapter tests. Plan: section-03-adapters.md Co-Authored-By: Claude <noreply@anthropic.com>
Clean-swap transactions, cross-source class resolution, conflict detection for extensions, flag management with previous holder cleanup, and topological insertion order. DB-sourced timestamps. Plan: section-04-service.md Co-Authored-By: Claude <noreply@anthropic.com>
POST /:id/import and DELETE /:id/import handlers merged under /ontology-sources with auth+CSRF middleware. ImportService created in main.rs, added to TestServices in test harness. Plan: section-05-routes-integration.md Co-Authored-By: Claude <noreply@anthropic.com>
8 sqlx::test integration tests: import, unload, clean swap reimport, flag atomicity, nonexistent source, extension without base, and 4 migration verification tests. Total: 39 new tests across the feature. Plan: section-06-tests.md Co-Authored-By: Claude <noreply@anthropic.com>
API reference, supported formats, behavior documentation, and architecture overview for the import engine feature. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Install @headless-tree/core, @headless-tree/react, @tanstack/react-virtual - Create route at /admin/ontology/browser - Scaffold 14 placeholder component/hook files Plan: section-01-infrastructure.md Co-Authored-By: Claude <noreply@anthropic.com>
- Add buildClassTree pure function for flat-to-hierarchy conversion - Implement useClassTree hook with TanStack Query, search/source filtering - Implement useClassDetail hook with class detail, properties, version queries - Add mutation functions for description update and property CRUD - 26 tests covering tree building, filtering, and hook behavior Plan: section-02-data-layer.md Co-Authored-By: Claude <noreply@anthropic.com>
- Add OntologyBrowserProvider with localStorage persistence and stale selection recovery (ref-optimized to avoid O(n) scan on selection) - Add resizable split-panel layout with independent error boundaries - Update Shadcn resizable wrapper for react-resizable-panels v4 API - Fix collapse toggle ARIA: moved button outside role="separator" - Fix toast double-dismiss timer that bypassed exit animations - Wire OntologyBrowser into /admin/ontology/browser route - 11 tests (5 context + 6 layout) Plan: section-03-layout-and-context.md Co-Authored-By: Claude <noreply@anthropic.com>
- Add ClassTree with headless-tree + react-virtual integration - Add ClassTreeNode with chevron, name/description, badges - Add ClassTreeSearch with 300ms debounced search + source filter - Remove selectionFeature to prevent split-brain with context - Fix ARIA: use native select semantics instead of role="combobox" - Update SourceBadge stub with sourceId prop - 23 tests (9 tree + 9 node + 5 search) Plan: section-04-tree-component.md Co-Authored-By: Claude <noreply@anthropic.com>
- Add ClassDetail container with placeholder/skeleton/data states - Add ClassHeader with name, parent link, source badge, description - Add ClassProperties with add/edit/delete UI and AlertDialog confirm - Add ClassConflicts with graceful degradation for conflict data - Update ClassLink and SourceBadge stubs with proper props - 20 tests (5 container + 5 header + 7 properties + 3 conflicts) Plan: section-05-detail-panel.md Co-Authored-By: Claude <noreply@anthropic.com>
- Add click-to-edit description in ClassHeader with 2 missing tests - Check isPlaceholderData in ClassDetail to prevent stale data display - Add versionId prop to ClassProperties for section-07 wiring Co-Authored-By: Claude <noreply@anthropic.com>
…ink) - SourceBadge with deterministic color hash, abbreviation, click callback - ConflictBadge with AlertTriangle icon and tooltip - ClassLink as button with onNavigate callback - Wired onNavigate/onSourceClick through ClassHeader to ClassDetail Plan: section-06-shared-components.md Co-Authored-By: Claude <noreply@anthropic.com>
- EditableText reusable component with async save, multiline, loading - CreateClassDialog with validation, parent class selector - Optimistic updates with rollback and toast notifications on all mutations - Wire EditableText into ClassHeader for description editing - New Class button in ClassTree header Plan: section-07-inline-editing.md Co-Authored-By: Claude <noreply@anthropic.com>
- useLocalStorage reusable hook with JSON parse/stringify and error fallback - Expanded nodes persistence with 300ms debounce and unmount flush - Source filter persistence via useLocalStorage in useClassTree - 11 persistence tests covering restoration, corruption, debounce Plan: section-08-state-persistence.md Co-Authored-By: Claude <noreply@anthropic.com>
- 6 component integration tests for tree-detail cross-component interaction - 7 Playwright E2E specs for full browser workflow - Tests cover selection, navigation, stale recovery, search Plan: section-09-integration-testing.md Co-Authored-By: Claude <noreply@anthropic.com>
Promote ontology browser from /admin/ontology/browser to /ontology as a top-level route with primary nav placement and tabbed detail panel. Co-Authored-By: Claude <noreply@anthropic.com>
- Create ontology.index.tsx as new top-level route - Convert admin/ontology/browser.tsx to redirect to /ontology - Auto-regenerated routeTree.gen.ts Plan: section-01-route-and-redirect.md Co-Authored-By: Claude <noreply@anthropic.com>
- Second item after Dashboard, before Projects - Uses Network icon from lucide-react - Links to /ontology Plan: section-02-navigation.md Co-Authored-By: Claude <noreply@anthropic.com>
- Detail tab shows existing ClassHeader, ClassProperties, ClassConflicts
- Graph and Relationships tabs show visual placeholder stubs
- Tab bar pinned at top, content scrolls
- key={selectedClassId} resets to Detail tab on class change
Plan: section-03-detail-panel-tabs.md
Co-Authored-By: Claude <noreply@anthropic.com>
- 5 new ClassDetail tab tests (3 tabs render, default active, placeholders, no tabs when empty) - Update Playwright E2E to use /ontology instead of /admin/ontology/browser Plan: section-04-testing.md Co-Authored-By: Claude <noreply@anthropic.com>
- Fix import engine route syntax ({id} -> :id) for axum 0.7
- Add ontologyFetch helper with credentials/CSRF for authenticated API calls
- Update api.test.ts assertions for new fetch signature
- Activate mpcg-ontology in sources.json
- Add frontend Dockerfile with pnpm, mount MPCG ontology in docker-compose
- Change db port from 5433 to 5434 (port conflict)
Co-Authored-By: Claude <noreply@anthropic.com>
- Ontology browser implementation: contracts, code reviews, usage guide - Ontology-as-core: spec, research, interview, plan, TDD, sections - Deep-implement and deep-plan config files Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/api/ontology-sourcesendpoints for discovering, listing, and managing ontology data sourcesontology_sourcestable,source_idcolumn on classes/properties/relationship_types, partial unique indexesOntologySourceServicewith filesystem discovery (symlink detection, manifest parsing, 500ms timeout), DB sync, and transactional active source managementontology_data_dirsetting, route registration inmain.rs, test harness updatesEndpoints
/api/ontology-sources/api/ontology-sources/active/api/ontology-sources/activeTest plan
cargo test --lib ontology_sources(10 tests)cargo test --test ontology_sources_test --no-runcargo test --test ontology_sources_testdata/sources.json, hit endpoints via curl🤖 Generated with Claude Code