Learning Apache Spark
Introduction
- Download, Compile and Run
- Secure the Spark
  - Long Running Service
RDD
Data Structures
Storage
MemoryManagement
Serializer
- SerializerManager
- UnsafeRowSerializer
Shuffle
- ShuffleManager
  - SortShuffleManager
- ShuffleDependency
  - ShuffleRDD
  - CoGroupedRDD
  - SubtractedRDD
  - ShuffledRowRDD
- ShuffleHandle
- ShuffleWriter
  - BypassMergeSortShuffleWriter
    - DiskBlockObjectWriter
  - SortShuffleWriter
    - ExternalSorter
  - UnsafeShuffleWriter
    - ShuffleExternalSorter
      - ShuffleInMemorySorter
- ShuffleReader
- ShuffleClient
- ShuffleBlockFetcherIterator
- RetryingBlockFetcher
Scheduler
Dataset/DataFrame
Catalyst
- Analyzer
  - GlobalAggregates
- Expression
  - Generator
    - Explode
  - Named Expressions
    - AttributeReference
    - Alias
  - Standard Expressions
  - BoundReference
  - windowExpressions
    - WindowFrame
    - WindowSpec
      - WindowSpecDefinition
    - FrameBoundary
    - WindowFunction
      - OffsetWindowFunction
      - AggregateWindowFunction
    - WindowExpression
  - AggregateExpression
    - AggregateMode
  - AggregateFunction
    - DeclarativeAggregate
      - TypedAggregateExpression
    - ImperativeAggregate
      - ScalaUDAF
  - ScalaUDF
  - CodegenFallback
  - Struct
- QueryPlan
  - Logical Plan
  - Spark Plan
    - GenerateExec
    - Exchange
    - Aggregate
      - HashAggregateExec
      - SortAggregateExec
      - AggregationIterator
        
        SortBasedAggregationIterator
        
        TungstenAggregationIterator
    - Join
    - SortExec
    - WindowExec
    - ExpandExec
  - LocalRelation
- Projection
- QueryExecution
- ExecutedPlan
- Misc.
- External Data Sources
- types
  - AbstractDataType
    - DataType
      - UserDefinedType
      - ObjectType
- Subquery
SQL
- HiveSupport
  - UDF
Tungsten
StructuredStreaming
MLLib
Misc.
Secure Cluster
Debug
Structured Streaming

Powered by GitBook

Introduction

Learning Apache Spark

This is the Book for Spark Architecture and Source Code Analysis based on Spark-2.0.

results matching ""

No results matching ""