The last line of code in pyspark-broadcast-dataframe.py:
filteDf= df.where((df['state'].isin(broadcastStates.value)))
gives this error:
SparkRuntimeException Traceback (most recent call last)
in <cell line: 29>()
27
28 # Broadcast variable on filter
---> 29 filteDf= df.where((df['state'].isin(broadcastStates.value)))
4 frames
/usr/local/lib/python3.10/dist-packages/pyspark/errors/exceptions/captured.py in deco(*a, **kw)
183 # Hide where the exception came from that shows a non-Pythonic
184 # JVM exception message.
--> 185 raise converted from None
186 else:
187 raise
SparkRuntimeException: [UNSUPPORTED_FEATURE.LITERAL_TYPE] The feature is not supported: Literal for '{FL=Florida, NY=New York, CA=California}' of class java.util.HashMap.
The last line of code in pyspark-broadcast-dataframe.py:
filteDf= df.where((df['state'].isin(broadcastStates.value)))
gives this error:
SparkRuntimeException Traceback (most recent call last)
in <cell line: 29>()
27
28 # Broadcast variable on filter
---> 29 filteDf= df.where((df['state'].isin(broadcastStates.value)))
4 frames
/usr/local/lib/python3.10/dist-packages/pyspark/errors/exceptions/captured.py in deco(*a, **kw)
183 # Hide where the exception came from that shows a non-Pythonic
184 # JVM exception message.
--> 185 raise converted from None
186 else:
187 raise
SparkRuntimeException: [UNSUPPORTED_FEATURE.LITERAL_TYPE] The feature is not supported: Literal for '{FL=Florida, NY=New York, CA=California}' of class java.util.HashMap.