1 of 29

Data Manipulation

with Pandas Library

Ahmad Ahmadzada

2 of 29

Data Manipulation using Pandas library

3 of 29

Plan:

1) What is Data Manipulation?

2) What is Pandas?

3) Common functions in Pandas

4) Practical example with Pandas

4 of 29

Data manipulation is the process of organizing information to make it readable and understandable. Engineers perform data manipulation using data manipulation language (DML) capable of adding, deleting, or altering data. 

While working with disparate data, you need to organize, clean, and transform it to use it in your decision-making process. This is where data manipulation fits in. Data manipulation allows you to manage and integrate data helping drive actionable insights.

5 of 29

Pandas is an open-source library that is made mainly for working with relational or labeled data both easily and intuitively. It provides various data structures and operations for manipulating numerical data and time series. This library is built on top of the NumPy library. Pandas is fast and it has high performance & productivity for users.

6 of 29

Advantages of Pandas:

  • Fast and efficient for manipulating and analyzing data.
  • Data from different file objects can be loaded.
  • Easy handling of missing data (represented as NaN) in floating point as well as non-floating point data
  • Data set merging and joining.
  • Flexible reshaping and pivoting of data sets
  • Powerful group by functionality for performing split-apply-combine operations on data sets.

7 of 29

Pandas generally provide two data structures for manipulating data, They are: 

  • Series is a one-dimensional labeled array capable of holding data of any type (integer, string, float, python objects, etc.). The axis labels are collectively called indexes. Pandas Series is nothing but a column in an excel sheet. Labels need not be unique but must be a hashable type. The object supports both integer and label-based indexing and provides a host of methods for performing operations involving the index.
  • DataFrame is a two-dimensional size-mutable, potentially heterogeneous tabular data structure with labeled axes (rows and columns). A Data frame is a two-dimensional data structure, i.e., data is aligned in a tabular fashion in rows and columns. Pandas DataFrame consists of three principal components, the data, rows, and columns.

8 of 29

9 of 29

Some common functions in Pandas library

​

​

​

pd.DataFrame(np.random.rand(20,5))

5 columns and 20 rows of random floats

pd.Series(my_list)

Create a series from an iterable my_list

df.index = pd.date_range('1900/1/30', periods=df.shape[0])

Add a date index

df.head(n)

First n rows of the DataFrame

df.tail(n)

Last n rows of the DataFrame

df.info()

Index, Datatype and Memory information

10 of 29

df.columns = ['a','b','c']

Rename columns

df.rename(columns={'old_name': 'new_ name'})

Selective renaming

df.set_index('column_one')

Change the index

df.sort_values(col1)

Sort values by col1 in ascending order

df.sort_values(col2,ascending=False)

Sort values by col2 in descending order.5

pd.read_csv(filename)

From a CSV file

11 of 29

Data Visualization

using Python

Ahmad Ahmadzada

12 of 29

Creating various graphs using visualization libraries

13 of 29

Python Data Visualization Libraries

Matplotlib

Seaborn

Ggplot

Plotly

Geoplotlib

Bokeh

Folium

Altair

Pygal

Gleam

14 of 29

What is Matplotlib?

  • Matplotlib is a plotting library for the Python programming language.

​

It provides an interface for creating a wide variety of static, animated, and interactive visualizations in Python.

​

Matplotlib is designed to be compatible with different types of data, including arrays, lists, and pandas DataFrames, and it supports a wide range of plot types, including line plots, bar plots, scatter plots, histograms, and pie charts, among others.

15 of 29

What is Plotly?

  • Plotly is an open-source graphing library for Python, R, Julia, and other programming languages that allows you to create interactive, web-based data visualizations. It provides a high-level interface for drawing attractive and informative statistical graphics, with a focus on producing interactive plots that can be easily shared and embedded in web applications or documents.
  • Plotly offers a wide variety of chart types, such as scatter plots, line charts, bar charts, pie charts, heatmaps, and more. It also provides advanced features like 3D charts, geographic maps, and the ability to create animations or combine multiple plots into a single figure.

​

16 of 29

Seaborn

  • Built-in functions for visualizing univariate and bivariate distributions, such as histograms, kernel density plots, scatter plots, and violin plots.
  • Support for visualizing multivariate data using pair plots, heatmaps, and cluster maps.
  • Functions to visualize linear relationships between variables using regression plots and residual plots.
  • Easy integration with Pandas DataFrames, allowing you to use column names as input for creating visualizations.
  • Built-in themes and color palettes to quickly style your plots.

​

17 of 29

Types of Graph

Line Graphs

Bar Charts

Pie Charts

Histogram

Scatter plot

Area Charts 

Radar chart

Box Plots

18 of 29

Using Salary DataSet for Visualization

19 of 29

Bar Chart

20 of 29

Scatter Plot

21 of 29

Pie Chart

22 of 29

23 of 29

scatter plot with plotly

24 of 29

Histogram

25 of 29

Scatter Plot with Seaborn

26 of 29

Hist Plot using Seaborn

27 of 29

Line graph

28 of 29

29 of 29

Thank you for your attention