Skip to content

Deletes are not merged correctly with Lance format on MOR table type #18558

@yihua

Description

@yihua

Bug Description

What happened:

Running TestLanceDataSource#testSqlCommands with DELETE command throws error, indicating that the deletes are not merged correctly with Lance format on MOR table type:

org.opentest4j.AssertionFailedError: Row count mismatch ==> 
Expected :3
Actual   :4
<Click to see difference>


	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:563)
	at org.apache.hudi.functional.TestLanceDataSource.checkAnswer(TestLanceDataSource.scala:74)

What you expected:

Deletes should be merged correctly with Lance format on MOR table type.

Steps to reproduce:
Run the above functional tests with MOR and DELETE command enabled.

Environment

Hudi version: master
Query engine: Spark
Relevant configs:

Logs and Stack Trace

No response

Metadata

Metadata

Assignees

Labels

type:bugBug reports and fixes

Type

No type
No fields configured for issues without a type.

Projects

Status

In progress

Relationships

None yet

Development

No branches or pull requests

Issue actions