Apache Spark and Scala Training

Curriculum:

1.Spark And Hadoop Distributed File System

• What is Spark
• Comparison with Hadoop
• Components of Spark

2.Spark Components, Common Spark Algorithms-Iterative Algorithms, Graph Analysis, Machine Learning

• Apache Spark- Introduction, Consistency, Availability, Partition
• Unified Stack Spark
• Spark Components
• Comparison with Hadoop – Scalding example, mahout, storm, graph

3.Running Spark On A Cluster, Writing Spark Applications Using Python, Java, Scala

• Explain python example
• Show installing a spark
• Explain driver program
• Explaining spark context with example
• Define weakly typed variable
• Combine scala and java seamlessly
• Explain concurrency and distribution
• Explain what is the trait
• Explain higher-order function with example
• Define OFI scheduler
• Advantages of Spark
• Example of Lamda using spark
• Explain Mapreduce with example
  

4.RDD And Its Operation

• Difference between RISC and CISC
• Define Apache Mesos
• Cartesian product between two RDD
• Define count
• Define Filter
• Define Fold
• Define API Operations
• Define Factors

5.Spark, Hadoop, And The Enterprise Data Centre, Common Spark Algorithms

• How Hadoop cluster is different from spark
• Define writing data
• Explain sequence file and its usefulness
• Define protocol buffers
• Define text file, CSV, Object Files, and File System
• Define sparse metrics
• Explain RDD and Compression
• Explain data stores and its usefulness

6.Spark Streaming

• Define Elastic Search
• Explain Streaming and its usefulness
• Apache bookeeper
• Define Dstream
• Define mapreduce word count
• Explain Paraquet
• Scala ORM
• Define Mlib
• Explain multi graphix and its usefulness
• Define property graph

7.Spark Persistence In Spark

• Persistence
• Motivation
• Example
• Transformation
• Scala and Python
• Examples – K-means
• Latent Dirichlet Allocation (LDA)

8.Broadcast And Accumulator

• Motivation
• Broadcast Variables
• Example: Join
• Alternative if one table is small
• Better version with broadcast
• How to create a Broadcast
• Accumulators motivation
• Accumulator Rules
• Custom accumulators
• Another common use
• Creating an accumulator using spark context object

9.Spark SQL And RDD

• Spark SQL main capabilities
• Spark SQL usage diagram
• Spark SQL
• Important topics in Spark SQL- Data frames
• Twitter language analysis

10.Introduction Of Scala

• Scala Overview

11.Pattern Matching

• Advantages of Scala
• REPL (Read Evaluate print loop)
• Language Features
• Type Interface
• Higher order function
• Option
• Pattern Matching
• Collection
• Currying
• Traits
• Application Space

12.Executing The Scala Code

• Uses of scala interpreter
• Example of static object timer in scala
• Testing of String equality in scala
• Implicit classes in scala with examples
• Recursion in scala
• Currying in scala with examples
• Classes in scala

13.Classes Concept In Scala

• Constructor
• Constructor overloading
• Prop erties
• Abstract classes
• Type hierarchy in Scala
• Object equality
• Val and var methods

14.Case Classes And Pattern Matching

• Sealed traits
• Case classes
• Constant pattern in case classes
• Wild card pattern
• Variable pattern
• Constructor pattern
• Tuple pattern

15.Concepts Of Traits With Example

• Java equivalents
• Advantages of traits
• Avoiding boilerplate code
• Linearization of traits
• Modelling a real world example

16.Scala Java Interoperability

• How traits are implemented in scala and java
• How extending multiple traits is handled

17.Scala Collections

• Classification of scala collections
• Iterable
• Iterator and iterable
• List sequence example in scala

18.Mutable Collections Vs. Immutable Collections

• Array in scala
• List in scala
• Difference between list and list buffer
• Array buffer
• Queue in scala
• Dequeue in scala
• Mutable queue in scala
• Stacks in scala
• Sets and maps in scala
  Tuples

19.Use Case Bobsrockets Package

• Different import types
• Selective imports
• Testing-Assertions
• Scala test case- scala test fun. Suite
• Junit test in scala
• Interface for Junit via Junit 3 suite in scala test
• SBT
• Directory structure for packaging scala application