-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Description
Is your feature request related to a problem? Please describe.
Following user feedback, there is a need for a connector from Pathway to MilvusDB. At the moment, Pathway does not provide a native way to write data there, which limits integration with vector search and retrieval workflows built on top of Milvus.
This connector is expected to be an output connector.
Describe the solution you'd like
There are two possible implementation approaches:
- Python-based wrapper. Implement the connector in Python using the official Milvus Python client.
- Rust Engine-level implementation. Implement a native connector on the Rust engine side using the officially provided Milvus Rust crate.
At this stage, it would be valuable to gather more information about expected usage patterns, required performance characteristics, and reliability expectations in order to decide how to best balance efficiency, stability, and maintenance effort.
Describe alternatives you've considered
The previous section outlines two alternative approaches: Python-based implementation versus a runtime engine-level implementation, between which a decision needs to be made.
Additional context
Since this is a new connector being added to the system, no backward compatibility issues are expected.