Overview
Add support for custom NULL marker configuration to match BigQuery's --null_marker parameter functionality.
Current Behavior
The emulator hardcodes NULL detection to only recognize empty strings ("") and the string "null" (case-insensitive).
Expected Behavior
Users should be able to specify a custom string that represents NULL values in their CSV data, matching BigQuery's behavior.
Common Null Markers
"N/A"
"\\N" (MySQL convention)
"NULL"
"-"
"NA"
Implementation Requirements
- Add
NullMarker field to JobConfigurationLoad structure
- Check value against custom marker before type conversion
- Default to empty string if not specified
- Maintain backward compatibility with existing
"null" detection
Test Cases
- CSV with
"N/A" and nullMarker="N/A" → should be NULL
- CSV with
"\\N" and nullMarker="\\N" → should be NULL
- CSV with
"N/A" and no nullMarker set → should be string "N/A"
- Ensure empty string always represents NULL (default behavior)
Documentation Reference
Overview
Add support for custom NULL marker configuration to match BigQuery's
--null_markerparameter functionality.Current Behavior
The emulator hardcodes NULL detection to only recognize empty strings (
"") and the string"null"(case-insensitive).Expected Behavior
Users should be able to specify a custom string that represents NULL values in their CSV data, matching BigQuery's behavior.
Common Null Markers
"N/A""\\N"(MySQL convention)"NULL""-""NA"Implementation Requirements
NullMarkerfield toJobConfigurationLoadstructure"null"detectionTest Cases
"N/A"andnullMarker="N/A"→ should be NULL"\\N"andnullMarker="\\N"→ should be NULL"N/A"and no nullMarker set → should be string "N/A"Documentation Reference