ShuffleExternalSorter

Used in UnsafeShuffleWriter, and leverage ShuffleInMemorySorter for implementation.

It copy the data to allocated memory, and invoke ShuffleInMemorySorter to do the real work.

An external sorter that is specialized for sort-based shuffle, used by UnsafeShuffleWriter. Unlike ExternalSorter, this sorter does not merge its spill files. Instead, this merging is performed in UnsafeShuffleWriter, which uses a specialized merge procedure that avoids extra serialization/deserialization.

Its closeAndWriteOutput will ask the sorted record iter from ShuffleInMemorySorter and write to file.

During insert into ShuffleInMemorySorter, if the records is larger than threashold, it will spill to disk, which instruct the in memory sorter to spill.

During closeAndWriteOutput, it get all spills, and merge sort them.