Learning Apache Spark
Introduction
- Download, Compile and Run
- Secure the Spark
  - Long Running Service
RDD
Data Structures
Storage
MemoryManagement
Serializer
- SerializerManager
- UnsafeRowSerializer
Shuffle
- ShuffleManager
  - SortShuffleManager
- ShuffleDependency
  - ShuffleRDD
  - CoGroupedRDD
  - SubtractedRDD
  - ShuffledRowRDD
- ShuffleHandle
- ShuffleWriter
  - BypassMergeSortShuffleWriter
    - DiskBlockObjectWriter
  - SortShuffleWriter
    - ExternalSorter
  - UnsafeShuffleWriter
    - ShuffleExternalSorter
      - ShuffleInMemorySorter
- ShuffleReader
- ShuffleClient
- ShuffleBlockFetcherIterator
- RetryingBlockFetcher
Scheduler
Dataset/DataFrame
Catalyst
- Analyzer
  - GlobalAggregates
- Expression
  - Generator
    - Explode
  - Named Expressions
    - AttributeReference
    - Alias
  - Standard Expressions
  - BoundReference
  - windowExpressions
    - WindowFrame
    - WindowSpec
      - WindowSpecDefinition
    - FrameBoundary
    - WindowFunction
      - OffsetWindowFunction
      - AggregateWindowFunction
    - WindowExpression
  - AggregateExpression
    - AggregateMode
  - AggregateFunction
    - DeclarativeAggregate
      - TypedAggregateExpression
    - ImperativeAggregate
      - ScalaUDAF
  - ScalaUDF
  - CodegenFallback
  - Struct
- QueryPlan
  - Logical Plan
  - Spark Plan
    - GenerateExec
    - Exchange
    - Aggregate
      - HashAggregateExec
      - SortAggregateExec
      - AggregationIterator
        
        SortBasedAggregationIterator
        
        TungstenAggregationIterator
    - Join
    - SortExec
    - WindowExec
    - ExpandExec
  - LocalRelation
- Projection
- QueryExecution
- ExecutedPlan
- Misc.
- External Data Sources
- types
  - AbstractDataType
    - DataType
      - UserDefinedType
      - ObjectType
- Subquery
SQL
- HiveSupport
  - UDF
Tungsten
StructuredStreaming
MLLib
Misc.
Secure Cluster
Debug
Structured Streaming

Powered by GitBook

Stage

Stage

A stage consists of a number of task that can be executed in parallel.

results matching ""

No results matching ""