Spark Plan

The root SparkPlan is wrapped by ReturnAnswer, which is always transformed first by SpecialLimits.

The transformed root SparkPlan is sparkPlan in QueryExecution. During the physical plan transformation, all children logical plan is transformed by planLater.

Each SparkPlan has an outputPartitioning, which may inherit from its children (UnaryNode). Please note that it is a function, and thus when a spark plan has children changed, e.g., ShuffleExchange inserted, its outputPartitioning may also change. Refer to EnsureRequirements for its detailed usage.

The abstract notable methods of a SparkPlan are as follows: doPrepare(): Unit – Allows a certain operation to ensure that before the query is actually executed, some setup code is run. An example of this is BroadcastHashJoin where you may want to broadcast the result of one side of the join before performing the actual computation of the join. doExecute(): RDD[InternalRow]