1 of 9

A Desperate Weekend �Learning Spark

(Everything I learned about Spark in 48 hours)

2 of 9

One day I was ready to leave my job

3 of 9

So I Interviewed Elsewhere

4 of 9

And they had “Homework” due Monday (On a Friday)

5 of 9

And One Frantic Weekend Later… I learned Spark

6 of 9

So Let’s Get Started - Getting Setup with - Python 3 2

​

Why not Python 2?

https://pythonclock.org

Use Python3 for new projects.

I’m using Python 2 for right now OK? I’m sorry.

7 of 9

Getting Setup

​

  • Make a virtual environment: �virtualenv venv�
  • Turn on the virtual environment:�source ./venv/bin/activate�
  • Install pyspark�pip install pyspark�

8 of 9

Have some data

9 of 9

Use The Data - DEMO TIME!!!!

Code here - https://gist.github.com/fernando-mc/598ab90a9d0efac12c8f299d98de82ed