Learning Apache Spark
Introduction
Download, Compile and Run
Secure the Spark
Long Running Service
RDD
Lineage
Dependency
ShuffleDependency
NarrowDependency
OneToOneDependency
RangeDependency
Partition
Partitioner
PairRDDFunctions
Data Locality
Cacheing
StorageLevel
Checkpoint
Data Structures
AppendOnlyMap
ExternalSorter
ShuffleInMemorySorter
ShuffleExternalSorter
UnsafeKVExternalSorter
UnsafeExternalRowSorter
UnsafeInMemorySorter
UnsafeExternalSorter
ChainedIterator
UnsafeSorterSpillMerger
ExternalAppendOnlyMap
BytesToBytesMap
MapIterator
UnsafeFixedWidthAggregationMap
Batch
RowBasedKeyValueBatch
FixedLengthRowBasedKeyValueBatch
VariableLengthRowBasedKeyValueBatch
ColumnarBatch
FastHashMap
TimSort
Storage
BlockManger
BlockManagerMaster
BlockManagerMasterEndpoint
MemoryManagement
MemoryAllocator
TaskMemoryManager
MemoryManager
Serializer
SerializerManager
UnsafeRowSerializer
Shuffle
ShuffleManager
SortShuffleManager
ShuffleDependency
ShuffleRDD
CoGroupedRDD
SubtractedRDD
ShuffledRowRDD
ShuffleHandle
ShuffleWriter
BypassMergeSortShuffleWriter
DiskBlockObjectWriter
SortShuffleWriter
ExternalSorter
UnsafeShuffleWriter
ShuffleExternalSorter
ShuffleInMemorySorter
ShuffleReader
ShuffleClient
ShuffleBlockFetcherIterator
RetryingBlockFetcher
Scheduler
Job DAG
Stage
TaskSet
Result Task
ShuffleMapTask
DAGScheduler
TaskScheduler
SchedulerBackend
Dataset/DataFrame
Encoder
ExpressionEncoder
RowEncoder
KeyValueGroupedDataset
Encoders
CatalystSerde
Row/InternalRow
Column
Catalyst
Analyzer
GlobalAggregates
Expression
Generator
Explode
Named Expressions
AttributeReference
Alias
Standard Expressions
BoundReference
windowExpressions
WindowFrame
WindowSpec
WindowSpecDefinition
FrameBoundary
WindowFunction
OffsetWindowFunction
AggregateWindowFunction
WindowExpression
AggregateExpression
AggregateMode
AggregateFunction
DeclarativeAggregate
TypedAggregateExpression
ImperativeAggregate
ScalaUDAF
ScalaUDF
CodegenFallback
Struct
QueryPlan
Logical Plan
Expand
Window
LogicalRelation
LogicalRDD
Convert to SQL
RunnableCommand
Aggregate
Generate
Spark Plan
GenerateExec
Exchange
BroadcastExchangeExec
ShuffleExchange
ExchangeCoordinator
Aggregate
HashAggregateExec
SortAggregateExec
AggregationIterator
SortBasedAggregationIterator
TungstenAggregationIterator
Join
SortMergeJoinExec
HashJoin
ShuffledHashJoinExec
BroadcastHashJoinExec
BroadcastNestedLoopJoinExec
CartesianProductExec
SortExec
WindowExec
ExpandExec
LocalRelation
Projection
QueryExecution
ExecutedPlan
EnsureRequirements
Distribution
Partitioning
CollapseCodegenStages
ReuseExchange
PlanSubqueries
Misc.
External Data Sources
types
AbstractDataType
DataType
UserDefinedType
ObjectType
Subquery
SQL
HiveSupport
UDF
Tungsten
CodeGen-Expression
CodeGenerator
GenerateSafeProjection
GeneratePredicate
GenerateOrdering
GenerateUnsafeProjection
GenerateUnsafeRowJoiner
GenerateMutableProjection
GenerateColumnAccessor
CodegenSupport
WholeStageCodegenExec
Examples
BufferedRowIterator
SortExec
InputAdapter
StructuredStreaming
StreamExecution
Sink
StateStoreSaveExec
StateStoreRestoreExec
IncrementalExecution
StatefulAggregationStrategy
StateStoreRDD
StateStoreCoordinator
StateStore
ProcessingTimeExecutor
DataSource
Source
StreamingExecutionRelation
StreamingRelation
TimeWindow
MLLib
Pipeline
PipelineStage
Transformers
Estimator
CrossValidator
Predictor
Classifier
ProbabilisticClassifier
RandomForestClassifier
NaiveBayes
LogisticRegression
DecisionTreeClassifier
GeneralizedLinearModels (GLMs)
LogisticRegressionModel
SVMModel
RidgeRegressionModel
LinearRegressionModel
Model
PipelineModel
PredictionModel
RegressionModel
ClassificationModel
Algorithms
Tree
DecisionTree
Ensemble
RandomForest
GradientBoostedTrees
Feature Scaling and Normalization
GeneralizedLinearRegression
IterativelyReweightedLeastSquares
WeightedLeastSquaresModel
Misc.
Compression
REPL stack
Hive
Secure Cluster
Debug
Structured Streaming
DataStreamReader
StreamingRelation
StreamingQueryManager
StreamingQuery
Powered by
GitBook
GlobalAggregates
GlobalAggregates
df.select(avg("column")) will be rewritten same as df.agg(avg("column"))
results matching "
"
No results matching "
"