Skip to content

Normalize StructSource table data before loading#485

Open
oshiro3 wants to merge 1 commit into
goccy:mainfrom
oshiro3:fix_load_repeated_record
Open

Normalize StructSource table data before loading#485
oshiro3 wants to merge 1 commit into
goccy:mainfrom
oshiro3:fix_load_repeated_record

Conversation

@oshiro3

@oshiro3 oshiro3 commented May 26, 2026

Copy link
Copy Markdown

Overview

I have created a patch to resolve Issue #483; please review it.
Please comment if you have any concerns regarding the fix or the tests.

This pull request fixes intermittent field misalignment that occurs when reading REPEATED RECORD data via StructSource.

In the previous StructSource, table data was passed directly to AddTableData, which meant that nested RECORD values backed by maps could be inserted without being normalized based on the table schema. Because the iteration order of Go maps is non-deterministic, iterated STRUCT fields were
stored in an unpredictable order, causing subsequent UNNEST field accesses (e.g., item.skip) to read values from different fields.

This fix normalizes the table data using types.NewTableWithSchema within the addProject read path before insertion.

Testing

We added an internal server test that directly calls addProject with a REPEATED RECORD column to verify that UNNEST(items) returns fields in the expected schema order.

go test ./server -run TestAddProjectNormalizesRepeatedRecordData -count=1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant