Part of #5.
user_schema is loaded at src/daflip/services.py:566-568, but the chunked branch at services.py:575-579 neither receives nor uses it. The schema is read, validated, and discarded.
This also removes the natural workaround for schema-drift problems in chunked writes: a user who knows their column types cannot supply them. Related to the null-type inference bug fixed in fix-chunked-parquet-null-schema, where a user-supplied schema would have been the obvious escape hatch.
Fix: pass the schema into _handle_chunked_conversion and, when present, use it as the ParquetWriter schema and cast each chunk to it instead of inferring from the first chunk.
Part of #5.
user_schemais loaded atsrc/daflip/services.py:566-568, but the chunked branch atservices.py:575-579neither receives nor uses it. The schema is read, validated, and discarded.This also removes the natural workaround for schema-drift problems in chunked writes: a user who knows their column types cannot supply them. Related to the null-type inference bug fixed in
fix-chunked-parquet-null-schema, where a user-supplied schema would have been the obvious escape hatch.Fix: pass the schema into
_handle_chunked_conversionand, when present, use it as theParquetWriterschema and cast each chunk to it instead of inferring from the first chunk.