Ensure Lance file format support on Spark 4.1

### Bug Description

We need to make sure that the lance-spark dependency is updated for Spark 4.1 and the tests are enabled to ensure they work.

### Environment

**Hudi version:**
**Query engine:** (Spark/Flink/Trino etc)
**Relevant configs:**


### Logs and Stack Trace

_No response_