Skip to content

Add an output connector for MilvusDB #164

@zxqfd555

Description

@zxqfd555

Is your feature request related to a problem? Please describe.
Following user feedback, there is a need for a connector from Pathway to MilvusDB. At the moment, Pathway does not provide a native way to write data there, which limits integration with vector search and retrieval workflows built on top of Milvus.

This connector is expected to be an output connector.

Describe the solution you'd like
There are two possible implementation approaches:

  1. Python-based wrapper. Implement the connector in Python using the official Milvus Python client.
  2. Rust Engine-level implementation. Implement a native connector on the Rust engine side using the officially provided Milvus Rust crate.

At this stage, it would be valuable to gather more information about expected usage patterns, required performance characteristics, and reliability expectations in order to decide how to best balance efficiency, stability, and maintenance effort.

Describe alternatives you've considered
The previous section outlines two alternative approaches: Python-based implementation versus a runtime engine-level implementation, between which a decision needs to be made.

Additional context
Since this is a new connector being added to the system, no backward compatibility issues are expected.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions