`mvexpand` over an array whose elements are records lost the column it
expanded. Uncollect explodes a record element into one column per record
field, so the row type becomes the element's field names and the
`field(arrayFieldName)` lookup right after it cannot find the array's own
name: `mvexpand EVENTS` over ARRAY<ROW<NAME, TIME>> fails with
"field [EVENTS] not found; input fields are: [ID, TAGS, NAME, TIME]".
The exploded columns are re-wrapped into a single ROW column named after the
array, so the expanded column keeps its name and a leaf stays addressable as
`<array>.<leaf>` downstream. The projection is forced, because it is an
identity on field references and RelBuilder would otherwise elide it and
drop the rewrap.
An element that is not a record needs nothing: ARRAY<scalar>, and the
ARRAY<ANY> a `nested` mapping declares, already yield exactly one column
carrying the array's name. So this only adds a case, leaving every shape
that works today on its existing path.
Tests pin both halves against a table with an ARRAY<ROW> column and an
ARRAY<VARCHAR> column: the record array gains the `ROW($0, $1)` rewrap above
Uncollect and fails without it, and the scalar array plans exactly as before,
with no rewrap.
Signed-off-by: Marc Handalian <handalm@amazon.com>
mvexpandover an array whose elements are records lost the column it expanded. Uncollect explodes a record element into one column per record field, so the row type becomes the element's field names and thefield(arrayFieldName)lookup right after it cannot find the array's own name:mvexpand EVENTSover ARRAY<ROW<NAME, TIME>> fails with "field [EVENTS] not found; input fields are: [ID, TAGS, NAME, TIME]".The exploded columns are re-wrapped into a single ROW column named after the array, so the expanded column keeps its name and a leaf stays addressable as
<array>.<leaf>downstream. The projection is forced, because it is an identity on field references and RelBuilder would otherwise elide it and drop the rewrap.An element that is not a record needs nothing: ARRAY, and the ARRAY a
nestedmapping declares, already yield exactly one column carrying the array's name. So this only adds a case, leaving every shape that works today on its existing path.Tests pin both halves against a table with an ARRAY column and an ARRAY column: the record array gains the
ROW($0, $1)rewrap above Uncollect and fails without it, and the scalar array plans exactly as before, with no rewrap.Description
[Describe what this change achieves]
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.