JavaScript isn't enabled in your browser, so this file can't be opened. Enable and reload.

1 of 28

Data Analysis

with Python

Full tutorial for beginners

2 of 28

Hands-on, online Data Science training.

3 of 28

4 of 28

About this tutorial

Let’s quickly review the contents of this tutorial. In the description of this video, we’ve included direct links to each section, so you can jump between them.

This is the first section, and we’re going to discuss what is Data Analysis. We’ll also talk about Data Analysis *with Python* and why programming tools like Python, SQL and Pandas are important.

In the following section, we’ll show you a real example of Data Analysis using Python, so you can see the power of it. We’ll not “explain” the tools in detail, it’s just a quick demonstration for you to understand what this tutorial is about. The following sections will be the ones explaining each tool in detail.

There are two more sections that I want to specially point out. The first one is Section no 3, “Jupyter tutorial”. This is not mandatory, you can skip it if you already know how to use Jupyter Notebooks.

And also the last section, Python in under 10 minutes. This is just a recap of Python if you’re coming from other languages. You might want to take this FIRST if that’s the case. Again, you can use the links in the video description to jump straight to it.

5 of 28

What is Data Analysis?

6 of 28

What is Data Analysis

> A process of inspecting, cleansing, transforming and modeling data with the goal of discovering useful information, informing conclusion and supporting decision-making.

Definition by Wikipedia.

7 of 28

What is Data Analysis

> A process of inspecting, cleansing, transforming and modeling data with the goal of discovering useful information, informing conclusion and supporting decision-making.

Definition by Wikipedia.

8 of 28

What is Data Analysis

> A process of inspecting, cleansing, transforming and modeling data with the goal of discovering useful information, informing conclusion and supporting decision-making.

Definition by Wikipedia.

9 of 28

What is Data Analysis

> A process of inspecting, cleansing, transforming and modeling data with the goal of discovering useful information, informing conclusion and supporting decision-making.

Definition by Wikipedia.

10 of 28

What is Data Analysis

> A process of inspecting, cleansing, transforming and modeling data with the goal of discovering useful information, informing conclusion and supporting decision-making.

Definition by Wikipedia.

11 of 28

Data Analysis Tools

12 of 28

Auto-managed closed tools

Programming Languages

13 of 28

👎 Closed Source 🙅‍♂️

👎 Expensive 💸

👎 Limited 😩

👍 Easy to learn 👩‍💻

👍 Open Source 🤩

👍 Free (or very cheap) 🤑

👎 Extremely Powerful 💪

👎 Steep learning curve 👩‍💻

Auto-managed closed tools

Programming Languages

The main advantage of closed tools like Tableau or Excel is that they’re generally easy to learn. There’s a company writing documentation, providing support and driving the creation of the product. The biggest disadvantage is that the scope of the tool is limited, you can’t cross the boundaries of it.

In contrast, using Python and the universe of PyData tools, gives you amazing flexibility. Do you need to read data from a closed API using secret key authentication? You can do it. Do you need to consume data directly from AWS Kinesis? You can do it. A programming language is the most powerful tool you can learn. Another important advantage is the general scope of a programming language. What happens if Tableau, for example, goes out of business? Or if you just get bored from it and feel like your career is stuck? Learning how to process data using a programming language gives you freedom.

The main disadvantage of a programming language is that it’s not as simple to learn as with a tool. You need to learn the basics of coding first, and it takes time.

14 of 28

Why Python

for Data Analysis?

15 of 28

Why Python for Data Analysis?

Why would we choose Python over R or Julia?

👍 very simple and intuitive to learn

👍 “correct” language

👍 powerful libraries (not just for Data Analysis)

👍 free and open source

👍 amazing community, docs and conferences

16 of 28

When to choose R?

Python, sadly, is not always the answer

When R Studio is needed
When dealing with advanced statistical methods
When extreme performance is needed

17 of 28

The Data Analysis

Process

18 of 28

Building Machine Learning Models
Feature Engineering
Moving ML into production
Building ETL pipelines
Live dashboard and reporting
Decision making and real-life tests

Exploration
Building statistical models
Visualization and representations
Correlation vs Causation analysis
Hypothesis testing
Statistical analysis
Reporting

Hierarchical Data
Handling categorical data
Reshaping and transforming structures
Indexing data for quick access
Merging, combining and joining data

Missing values and empty data
Data imputation
Incorrect types
Incorrect or invalid values
Outliers and non relevant data
Statistical sanitization

Data Extraction

Data Cleaning

Data Wrangling

Analysis

SQL
Scrapping
File Formats

CSV
JSON
XML

Consulting APIs
Buying Data
Distributed Databases

Action

The process starts by getting the data. Where is your data coming from? Usually, it’s in your own database. But it could also come from files stored in different formats or web APIs.

Once we’ve collected the data we’ll need to clean it. If the source of the data is your own database, then it’s probably already in shape. If you’re using more extreme sources, like web scraping, then the process will be more tedious.

With our data cleaned, we’ll now need to rearrange and reshape the data for better analysis. Transforming fields, merging tables, combining data from multiple sources, etc. The objective of this process is to get the data ready for the next step.

The process of analysis involves extracting patterns from the data that is now clean and in shape. Capturing trends or anomalies. Statistical analysis will be fundamental in this process.

Finally, it’s time to do something with that analysis. If this was a Data Science project, we could be ready to implement Machine Learning models. If we focus strictly on Data Analysis, we’ll probably need to build reports, communicate our results and support decision making.

Let me finish by saying that, in real life, this process isn’t so linear. We’re usually jumping back and forth between the steps, and it looks more like a cycle than a straight line.

19 of 28

Data Analysis

Vs

Data Science

20 of 28

DATA ANALYSIS VS DATA SCIENCE

The traditional view

21 of 28

Python & PyData Ecosystem

22 of 28

PYTHON ECOSYSTEM:

The libraries we use...

pandas: The cornerstone of our Data Analysis job with Python
matplotlib: The foundational library for visualizations. Other libraries we’ll use will be built on top of matplotlib.
numpy: The numeric library that serves as the foundation of all calculations in Python.
seaborn: A statistical visualization tool built on top of matplotlib.
statsmodels: A library with many advanced statistical functions.
scipy: Advanced scientific computing, including functions for optimization, linear algebra, image processing and much more.
s cikit-learn: The most popular machine learning library for Python (not deep learning)

23 of 28

How Python Data

Analysts Think

24 of 28

EXCEL, TABLEAU, ETC.

They’re all visual tools...

25 of 28

Thinking like a

Python Data Analyst

26 of 28

And finally,

why Python?

27 of 28

28 of 28

About this tutorial

What is Data Analysis
Real Example Data Analysis with Python
How to use Jupyter Notebooks
Intro to NumPy (exercises included)
Intro to Pandas (exercises included)
Data Cleaning
Reading Data SQL, CSVs, APIs, etc
Python in Under 10 Minutes