As the title suggests, in the example there were the following code: rdd = sc.parallelize(data) rdd = rdd.coalesce(max_partitions) What is sc? I can't see any import nor assignment before it's been called.
As the title suggests, in the example there were the following code:
rdd = sc.parallelize(data)
rdd = rdd.coalesce(max_partitions)
What is sc? I can't see any import nor assignment before it's been called.