Topic 5: DataFrame

Learning Outcomes:

(a) Identify basic Pandas DataFrame properties and methods (shape, columns, index, info(), head() and tail())

CP125

Topic 5: DataFrame

Table of Contents

• why Pandas
• Introduction to Pandas
• Panda Series and DataFrame
• Importing Pandas module
• Constructing DataFrame
• Retrieve information about Pandas DataFrame

​

CP125

Why Pandas?

What is Pandas

​

• Derive form the term Panel Data and also refers to Python Data Analysis

​

• Powerful Python library for handling and manipulating tables data or tabular data

​

• Pandas represent data in the form of - Series and DataFrame

​

• In summary, Pandas is a Python library used for data manipulation and analysis

Pandas Series and DataFrame

Pandas provides two types of classes for handling data:

1. Series: a one-dimensional labeled array holding data of any type�such as integers, strings, Python objects etc.
2. DataFrame: a two-dimensional data structure that holds data like a two-dimensional array or a table with rows and columns.

Pandas Data Structure

Series

DataFrame

Array-like object containing sequence of value

Similar to a column in table

​

label 1

label 2

label 3

3

4

5

0

1

2

3

4

5

0

1

2

Collection of series

row index

row index

Column label

Pandas Dataframe Overview

DataFrame is basically Pandas data table

DataFrame has similar structure to data representation in spreadsheet software

Spreadsheet

DataFrame

Data Type in Pandas

• A DataFrame is a 2-Dimensional data structure
• It can store data of different types including characters, integers, floating point values, categorical data and more in columns.

column names

types

Data types in Pandas (Continue)

Importing Pandas module

Getting Started with Pandas

import pandas as pd

import statement

module name

alias

renaming pandas as pd as alias is considered as standard practice and use everytime pandas module is loaded into the program

Constructing Dataframe

To create a Pandas DataFrame, you can use Python to construct it from

• Lists or dictionaries,
• Import data from files like CSV or Excel,
• Connect to SQL databases,
• Fetch JSON data,
• Retrieve data from URLs, or even from
• Python dictionaries of Pandas Series

​

we will only focus on list and csv file

Constructing Dataframe (Cont’)

• From List

​

data = [['Alice', 25], ['Bob', 30], ['Charlie', 35]]

​

df = pd.DataFrame(data,columns=[‘name’,’age’])

​

​

​

• From csv File

df=pd.read_csv(file_path)

​

Convert List to DataFrame

Output

Load csv file into Pandas DataFrame

Pandas support many different data sources and file formats.

replace the * with the appropriate file format.

import pandas as pd

df = pd.read_csv(‘file_path’)

​

Retrieve DataFrame Basic Information

To get a glimpse of the data, we can use the following methods

• df.shape #return tuple (row,col)
• df.columns #index column information
• df.index #row start index,stop index and step
• df.info() #summary of the DataFrame
• df.head() #Default to first 5 rows
• df.tail() #Default to last 5 rows

Dataframe from List

1. Shape of

Dataframe

Return the number of rows and columns in the DataFrame

df.shape

2. Get index in the Dataframe

Return start, end and steps in index

df.index

3. Get all columns in Dataframe

Return Index object containing the columns name

df.columns

4. Get Info

of the Dataframe

​

Return index information, number of non-Null values in each columns, the data types and memory usage

df.info()

5a. Get First 5

Rows of

the Dataframe

Default - Return first 5 rows

df.head()

5b. Get First n rows

of the Dataframe

Return first n rows

df.head(n)

6a. Get Last 5 Rows of the Dataframe

Default - Return last 5 rows

df.tail()

6b. Get Last n Rows

of the Dataframe

Return last n rows

df.tail(n)

Dataframe from CSV files

​

Return the number of rows and columns in the DataFrame

df.shape

#Reading CSV files

1. Shape of

Dataframe

2. Get index in the Dataframe

3. Get all columns in Dataframe

Return start, end and steps in index

df.index

Return Index object containing the columns name

df.columns

4. Get Info

of the Dataframe

​

5a. Get First 5

Rows of

the Dataframe

Return index information, number of non-Null values in each columns, the data types and memory usage

df.info()

Default - Return first 5 rows

df.head()

5b. Get First n rows

of the Dataframe

6a. Get Last 5 Rows of the Dataframe

Return first n rows

df.head(n)

Default - Return last 5 rows

df.tail()

6b. Get Last n Rows

of the Dataframe

Return last n rows

df.tail(n)

Summary

• Learning to create Pandas DataFrame from list and CSV files
• Using Pandas DataFrame properties and methods to retrieve basic information about the DataFrame.