`TableProvider::delete_from` problem with pushed down filters

[mjgarton]

Describe the bug

I'm attempting to implement the new delete_from method in my custom TableProvider implementation. Despite a DELETE query with a where clause that matches a single record, I'm getting no filter expressions passed to delete_from leading to all data being deleted instead of a single record.

What I currently believe is happening:

1. My table provider supports push down filters.
2. The datafusion pushes down the filters, so that instead of the ExecutionPlan being a Filter at the top level and having TableScan as its input value, the ExecutionPlan becomes TableScan which now contains the filters instead.
3. In extract_dml_filters , the call to TreeNode::apply never reaches the filters inside the TableScan because LogicalPlan::apply_children calls ``LogicalPlan::inputs` which returns no children in the case where the top level of the LogicalPlan is a TableScan.

If this explanation does not help, I will try to make a minimal reproduction case to show the problem.

To Reproduce

1. Try to delete a single record from a table via a TableProvider that supports Exact pushdown filters.
2. Observe all records being deleted.

Expected behavior

Filters should be correctly passed to delete_from

Additional context

No response

[kosiew]

@mjgarton
Your explanation does help 😄

[mjgarton]

Thanks @kosiew . I believe this issue may also apply to update processing.

Please let me know if there's anything I can do to help. I would try to fix this myself, but I don't know what the right approach is.

The current approach to deletion in my application is to use a custom planner, and internally it simply executes the input plan to find the records that need deleting. This approach works, but I suspect isn't very optimal because it reads everything before deleting it.

[kosiew]

I believe this issue may also apply to update processing.

Yes, it does.

You can take a look at #19884.
It's still Draft because I am waiting for CI to 🟢 it.