Packaging and Merging Updates

[EvanDietzMorris]

PyPI Packaging & Project Modernization

This restructures ORION into a distributable Python package published on PyPI as robokop-orion.

• Renamed Common/ → orion/ to follow packaging conventions
• Moved cli/ → orion/cli/ so cli scripts are part of the distributed package
• Renamed load_manager.py & SourceDataManager → ingest_pipeline.py & IngestPipeline
• Removed requirements.txt, all dependencies are now declared in pyproject.toml
• Added pyproject.toml, uv.lock, .python-version,
• Using uv as the build backend for packaging and development
• Separated core dependencies, dev, and those used only by parsers.
• CLI entry points registered as [project.scripts]:
  • orion-build - build complete knowledge graphs from a Graph Spec
  • orion-ingest - run the ingest pipeline for individual data sources
  • orion-merge - merge KGX node/edge files
  • orion-meta-kg - generate MetaKG and test data files
  • orion-redundant-kg - generate redundant edge files
  • orion-ac - generate AnswerCoalesce files
  • orion-neo4j-dump / orion-memgraph-dump - generate database dumps

CI/CD

• New pypi.yml workflow - publishes to TestPyPI and PyPI on GitHub release using trusted publishing
• Updated test.yml - migrated from Python 3.9 + pip to Python 3.12 + uv

Docker & Deployment

• Dockerfile updated: uses uv to install the package and manage entry points
• Restructured neo4j image and python usage to be able to use newer python versions (3.12)
• Docker Compose and Helm charts now use executables like orion-build instead of python commands

Merging Improvements

• Significantly improved merging algorithm
• Added custom merging option for retrieval sources
• Merging now always treats lists as sets
• Simplified merging buffer with stricter buffer size enforcement
• Better knowledge source merging with improved tests

Other Improvements

• Added add_edge_id and edge_id_type support (including UUID option) at the graph spec level
• Prevented duplicate edges in example data
• Improved handling of list qualifier values for MetaKG
• Increased default normalization batch size
• Updated documentation (README, contributor guide, Jupyter notebook) to reflect new structure and commands
• Updated .gitignore to exclude .DS_Store, .pytest_cache/, dist/, and *.egg-info/