[dongatnb]

Optimize performance of the following snippet in a hive query:

        transform(
            a.buckets,
            x->if(
                array_contains(c.names, split(x, '-') [0]),
                x,
                null
            )
        ) as buckets,

and

        transform(
            split(bucket, ','),
            x->if(
                array_contains(c.names, split(x, '-') [0]),
                x,
                null
            )
        ) as buckets,

array_contains has time complexity o(N) and it is applied to multiple elements in an array which sums to O(MN) time complexity.

Solution: Convert c.names into a HashSet which can be used across multiple rows.