Learning Apache Spark
Introduction
- Download, Compile and Run
- Secure the Spark
  - Long Running Service
RDD
Data Structures
Storage
MemoryManagement
Serializer
- SerializerManager
- UnsafeRowSerializer
Shuffle
- ShuffleManager
  - SortShuffleManager
- ShuffleDependency
  - ShuffleRDD
  - CoGroupedRDD
  - SubtractedRDD
  - ShuffledRowRDD
- ShuffleHandle
- ShuffleWriter
  - BypassMergeSortShuffleWriter
    - DiskBlockObjectWriter
  - SortShuffleWriter
    - ExternalSorter
  - UnsafeShuffleWriter
    - ShuffleExternalSorter
      - ShuffleInMemorySorter
- ShuffleReader
- ShuffleClient
- ShuffleBlockFetcherIterator
- RetryingBlockFetcher
Scheduler
Dataset/DataFrame
Catalyst
- Analyzer
  - GlobalAggregates
- Expression
  - Generator
    - Explode
  - Named Expressions
    - AttributeReference
    - Alias
  - Standard Expressions
  - BoundReference
  - windowExpressions
    - WindowFrame
    - WindowSpec
      - WindowSpecDefinition
    - FrameBoundary
    - WindowFunction
      - OffsetWindowFunction
      - AggregateWindowFunction
    - WindowExpression
  - AggregateExpression
    - AggregateMode
  - AggregateFunction
    - DeclarativeAggregate
      - TypedAggregateExpression
    - ImperativeAggregate
      - ScalaUDAF
  - ScalaUDF
  - CodegenFallback
  - Struct
- QueryPlan
  - Logical Plan
  - Spark Plan
    - GenerateExec
    - Exchange
    - Aggregate
      - HashAggregateExec
      - SortAggregateExec
      - AggregationIterator
        
        SortBasedAggregationIterator
        
        TungstenAggregationIterator
    - Join
    - SortExec
    - WindowExec
    - ExpandExec
  - LocalRelation
- Projection
- QueryExecution
- ExecutedPlan
- Misc.
- External Data Sources
- types
  - AbstractDataType
    - DataType
      - UserDefinedType
      - ObjectType
- Subquery
SQL
- HiveSupport
  - UDF
Tungsten
StructuredStreaming
MLLib
Misc.
Secure Cluster
Debug
Structured Streaming

Powered by GitBook

NarrowDependency

Narrow Dependency

In UnionRDD, one child rdd partition corresponds to one partition of one parent rdd.

results matching ""

No results matching ""