Skip to content

Add Terms Query Support for Calcite RelNode Conversion#2

Open
abhishek00159 wants to merge 3 commits intovinaykpud:feature/datafusion-dslfrom
abhishek00159:feature/datafusion-dsl-term-query
Open

Add Terms Query Support for Calcite RelNode Conversion#2
abhishek00159 wants to merge 3 commits intovinaykpud:feature/datafusion-dslfrom
abhishek00159:feature/datafusion-dsl-term-query

Conversation

@abhishek00159
Copy link
Copy Markdown

Summary

This PR adds support for converting OpenSearch terms queries into Calcite RelNode format, enabling the query-dsl-calcite plugin to handle multi-value equality checks.

Changes

New Translator: Implemented TermsQueryTranslator that converts TermsQueryBuilder to Calcite SEARCH(IN) operator
Registration: Added the new translator to QueryRegistryFactory
Documentation: Updated README with terms query mapping (terms → SEARCH($field, value1, value2, ...))
Tests: Added comprehensive integration tests covering:
• Basic terms query conversion
• Validation for unsupported boost parameter
• Validation for unsupported _name parameter
• Validation for unsupported value_type parameter

Technical Details

The TermsQueryTranslator converts a terms query like:
json
{
"terms": {
"category": ["electronics", "computers", "laptops"]
}
}

Into a Calcite logical plan:
LogicalFilter(condition=[SEARCH($0, Sarg['electronics':VARCHAR, 'furniture':VARCHAR]:VARCHAR)])
LogicalTableScan(table=[[products]])

Testing:

curl -X PUT "localhost:9200/products"                                                                                                              

{"acknowledged":true,"shards_acknowledged":true,"index":"products"}%                                                                                                                 
abhissom@c889f3ee19b7 OpenSearch % curl -X POST "localhost:9200/products/_doc/1" -H 'Content-Type: application/json' -d'                                                              
{                                                                                                                                                                                     
  "name": "Laptop",                                                                                                                                                                   
  "category": "electronics",                                                                                                                                                          
  "price": 999                                                                                                                                                                        
}'
{"_index":"products","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}%                                       
abhissom@c889f3ee19b7 OpenSearch % curl -X POST "localhost:9200/products/_doc/2" -H 'Content-Type: application/json' -d'                                                              
{                                                                                                                                                                                     
  "name": "Phone",                                                                                                                                                                    
  "category": "electronics",                                                                                                                                                          
  "price": 699                                                                                                                                                                        
}'                                                                                                                                                                                    
 

curl -X POST "localhost:9200/products/_doc/3" -H 'Content-Type: application/json' -d'

{

  "name": "Desk",

  "category": "furniture",

  "price": 299

}'
{"_index":"products","_id":"2","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1}{"_index":"products","_id":"3","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":2,"_primary_term":1}%                                                                               
abhissom@c889f3ee19b7 OpenSearch % curl -X GET "localhost:9200/products/_search" -H 'Content-Type: application/json' -d'                                                              
{                                                                                                                                                                                     
  "query": {                                                                                                                                                                          
    "terms": {                                                                                                                                                                        
      "category": ["electronics", "furniture"]                                                                                                                                        
    }                                                                                                                                                                                 
  }                                                                                                                                                                                   
}'
{"took":230,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":10,"relation":"eq"},"max_score":null,"hits":[{"_id":"0","_score":null,"_source":{"category":"Apple","category.keyword":1299,"name":"US"},"fields":{"category":["Apple"],"category.keyword":[1299],"name":["US"]}},{"_id":"1","_score":null,"_source":{"category":"Apple","category.keyword":999,"name":"EU"},"fields":{"category":["Apple"],"category.keyword":[999],"name":["EU"]}},{"_id":"2","_score":null,"_source":{"category":"Apple","category.keyword":1099,"name":"US"},"fields":{"category":["Apple"],"category.keyword":[1099],"name":["US"]}},{"_id":"3","_score":null,"_source":{"category":"Apple","category.keyword":1199,"name":"EU"},"fields":{"category":["Apple"],"category.keyword":[1199],"name":["EU"]}},{"_id":"4","_score":null,"_source":{"category":"Samsung","category.keyword":799,"name":"US"},"fields":{"category":["Samsung"],"category.keyword":[799],"name":["US"]}},{"_id":"5","_score":null,"_source":{"category":"Samsung","category.keyword":899,"name":"EU"},"fields":{"category":["Samsung"],"category.keyword":[899],"name":["EU"]}},{"_id":"6","_score":null,"_source":{"category":"Samsung","category.keyword":699,"name":"US"},"fields":{"category":["Samsung"],"category.keyword":[699],"name":["US"]}},{"_id":"7","_score":null,"_source":{"category":"Dell","category.keyword":599,"name":"US"},"fields":{"category":["Dell"],"category.keyword":[599],"name":["US"]}},{"_id":"8","_score":null,"_source":{"category":"Dell","category.keyword":649,"name":"EU"},"fields":{"category":["Dell"],"category.keyword":[649],"name":["EU"]}},{"_id":"9","_score":null,"_source":{"category":"Dell","category.keyword":549,"name":"US"},"fields":{"category":["Dell"],"category.keyword":[549],"name":["US"]}}]}}%   

Output:

[2026-03-26T21:54:10,075][INFO ][o.o.d.q.DefaultQueryPlanExecutor] [runTask-0] Executing RelNode:
LogicalFilter(condition=[SEARCH($0, Sarg['electronics':VARCHAR, 'furniture':VARCHAR]:VARCHAR)])
  LogicalTableScan(table=[[products]])

Validation

The implementation enforces that only default values are used for boost, query name, and value type parameters, throwing exceptions for unsupported configurations.

Check List

  • [ YES ] Functionality includes testing.
  • [ NA ] API changes companion pull request created, if applicable.
  • [ NA ] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

TermsQueryBuilder termsQuery = (TermsQueryBuilder) query;

if (termsQuery.boost() != AbstractQueryBuilder.DEFAULT_BOOST) {
throw new RuntimeException("Terms query does not support non-default boost");
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to throw ConversionException to follow the same convention used in other places

Comment on lines +156 to +159
String indexName = "test-terms-valuetype";
String mapping = "{\"properties\": {\"category\": {\"type\": \"keyword\"}}}";
client().admin().indices().prepareCreate(indexName).setMapping(mapping).get();
ensureGreen(indexName);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be done in the setup method of IT instead of every method?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants