Skip to content

Local mode: recommend/discovery/context/feedback queries mis-order and mis-threshold on Euclid/Manhattan #1378

Description

@winklemad

Current Behavior

Recommend / discovery / context / relevance-feedback queries score points from the internal core distance, which is oriented so a higher score is always better regardless of the collection metric. On Euclid/Manhattan collections in local mode, two paths in LocalCollection.search treat those scores using the raw distance order instead, producing wrong results.

1. Relevance-feedback returns the farthest points first (reversed order).

c = QdrantClient(":memory:")
c.create_collection("rf", vectors_config=models.VectorParams(size=2, distance=models.Distance.EUCLID))
c.upsert("rf", points=[
    models.PointStruct(id=1, vector=[0.1, 0.0]),   # nearest to target [0, 0]
    models.PointStruct(id=5, vector=[5.0, 0.0]),
    models.PointStruct(id=9, vector=[9.0, 0.0]),   # farthest
])
c.query_points("rf", query=models.RelevanceFeedbackQuery(relevance_feedback=models.RelevanceFeedbackInput(
    target=[0.0, 0.0],
    feedback=[models.FeedbackItem(example=[3.0, 0.0], score=0.5),
              models.FeedbackItem(example=[4.0, 0.0], score=0.5)],
    strategy=models.NaiveFeedbackStrategy(naive=models.NaiveFeedbackStrategyParams(a=1.0, b=1.0, c=1.0)),
)), limit=3).points
# ids -> [9, 5, 1]   (expected [1, 5, 9])

2. score_threshold drops all results for the whole higher-is-better family (recommend/discovery/context/feedback).

c.create_collection("t", vectors_config=models.VectorParams(size=2, distance=models.Distance.EUCLID))
c.upsert("t", points=[models.PointStruct(id=i, vector=[float(i), 0.0]) for i in (1, 2, 3, 4)])
c.query_points("t",
    query=models.RecommendQuery(recommend=models.RecommendInput(
        positive=[[1.0, 0.0]], negative=[], strategy=models.RecommendStrategy.BEST_SCORE)),
    limit=10, score_threshold=0.13).points
# returns []   (expected the points scoring >= 0.13)

Expected Behavior

Feedback results ordered nearest-first ([1, 5, 9]), and score_threshold keeps the points scoring at/above the threshold for the recommend/discovery/context/feedback family.

Root cause

Both the result ordering and the score_threshold check derive their direction from distance_to_order(distance) (the raw distance order). For the recommend/discovery/context/feedback family the scores are higher-is-better on every metric, so on Euclid/Manhattan the ordering special-case must apply (it omitted NaiveFeedbackQuery) and the threshold direction must match — instead the threshold followed the raw order and inverted.

I have a fix that derives a single higher-is-better flag from the query type and uses it for both the ordering and the threshold, plus regression tests over both metrics.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions