1 of 19

GOAL

Introduce methods of organizing data and using graphs to draw meaningful correlations from the data

Basic Scientific Computing for Research Projects

2 of 19

DJ - Maybe this can be the Potential Outline

Show How to plot 2 columns for a dataset

(maybe do the dogs file plot pie graph of dog breeds?)

Feel free to use or delete any slides after here… ;-)

3 of 19

3

PART 1

Data Sets

4 of 19

Data Sets

Common Data Set formats:

.csv - spreadsheet like data
.json - structured or nested data
.xlsx - excel format
.zip - compressed folder with multiple files
.txt - plain text data

Public Datasets

Kaggle

website : https://www.kaggle.com/datasets

Google Dataset Search

website : https://datasetsearch.research.google.com

Data.gov

website : https://data.gov

�

5 of 19

5

JSON Files

5

JSON = JavaScript Object Notation

JSON files are very popular for storing and transmitting data between applications.

So what makes a JSON file? key:value pairs!

Keys must always be strings
Strings must always be in double quotes
JSON objects (Python dictionaries) can be assigned as values & are written with { }
JSON arrays (Python lists) are similar to objects, but they are defined using [ ]

When we refer to JSON files, we are really just referring to files that follow a set of guidelines for representing data!

JSON files behave very similar to dictionaries & are great to work with once you’re familiar with them!

6 of 19

6

CSV Files

6

CSV stands for Comma Separated Values.

As you may have guessed, CSV files contain a series of items, each of which is separated by commas (see image on left).

CSV files are another way to format data within a file so that it is organized and easy to work with.

CSV files are plain-text files – i.e. they don’t preserve any formatting. This makes them easy to create and easy to upload into other tools like spreadsheets.

While CSV files are straightforward and consume less memory than JSON files, they do not support the same scalability and versatility and they cannot nest data.

x	y1	y2	y3
1.96	3.90	4.76	5.48
3.28	4.87	5.22	6.39
4.15	5.83	6.26	7.25
4.92	7.08	7.24	8.33
6.31	7.55	8.86	9.09
6.89	8.10	8.94	10.10
8.03	8.88	9.34	11.17

7 of 19

7

Parsing JSON

7

After you open a JSON file you have to be able to extract information from it

This is known as parsing!

To do so, here are the general rules you should follow:

How to Parse JSON
Add import json at the top of your program.
Open the file using with open(), then use json.load() to get the file into a readable format for your program.
Extract the data using similar properties to those we used for dictionaries

8 of 19

8

PART 2

Helpful Built-In Functions

9 of 19

9

Operation	Definition	Syntax	Example
.upper()	The .upper() method returns a copy of the original string with all lowercase letters converted to uppercase. It does not modify the original string (strings in Python are immutable).	string.upper()
.lower()	The .lower() method returns a copy of the original string with all uppercase letters converted to lowercase. Like .upper(), it does not modify the original string.	string.lower()

(output)

10 of 19

10

Operation	Definition	Syntax	Example
.split()	The .split() method splits a string into a list where each word is a list item. You can specify the separator, otherwise, it will split on any whitespace (space, newline, tab, etc.).	string.split()
.strip()	The .strip() method removes any leading (spaces at the beginning) and trailing (spaces at the end) characters (space is the default).	string.strip()

(output) ^

11 of 19

Using the type() function

11

Here’s another tip: use the type() function to help you identify the values of different keys!
In the video, we discussed that we can identify a list by its opening bracket [ or identify a dictionary by its opening curly brace {, but we can also use the type() function if the type is unclear!

This helps us understand how to access the next values and continue the search through the JSON file.

CODE

OUTPUT

As you can see, when the type is ‘dict’, we access the next element using a string that is a key in the dictionary. When the type is ‘list’, we access the next element by indexing with a number, like [0].

12 of 19

12

Let’s Code

13 of 19

Loading Datasets (JSON & CSV)

Upload to Google Drive

Go to google drive press click “New” -> “File Upload”
Then select your .csv or .json file

Mount Google Drive In Colab

from google.colab import drive
drive .mount(‘/content/drive’)
your files will be located in : /content/drive/My Drive/

Load The Data Set(CSV)

import pandas as pd
df = pd.read_csv(‘/content/drive/My Drive/ filename.csv’)

Load The Data Sets(JSON)

import json
with open(‘/content/drive/My Drive/ filename.json’) as f:

data = json.load(f)

14 of 19

Now it's Your Turn!

14

Now it is your turn to come up with an example using the skills you have learned from this module!

This is pretty open ended! Think about your favorite movie, tv show, book, mathematical formula, or maybe video game. Use the tools you have used to draw a conclusion within a certain theme.

Create an example using all the following skills:

Choose from the following JSON/CSV files or find your own:

Open the file
Store the contents of the file in a data structure
Extract/draw conclusions from the file
Write a new piece of data to the file (OPTIONAL)
Print out the files contents in a visually appealing way

15 of 19

Types Of Graphs

Scatter

Code: px.scatter(df, x=..., y=...)

Bar

Code: px.bar(df, x=..., y=...)

Line

Code: px.line(df, x=..., y=...)

Histogram

Code: px.histogram(df, x=...)

Box Plot

Code: px.box(df, x=..., y=...)

16 of 19

Graph Customizations

Adding more parameters leads to other customizations. Below are some parameters you can use for customizing graphs.

title = “Your Chart Title”
color = “another_column”
labels = {"column_x": "X Label", "column_y": "Y Label"}
template="plotly_dark"

17 of 19

17

Tools you have Learned

18 of 19

Tools You Have Learned

18

A data structure is just a way to organize and store data.
There are four main built-in data structures in Python: lists, tuples, sets, and dictionaries.

You also learned how all of these data structures work!

JSON files are files that contain data formatted in a specific way, which resembles a Python dictionary.
CSV files are files that contain data formatted in a specific way as well - the items are separated by commas.
You learned how to parse & slice through data when it is stored in different formats and how to apply that to real world problem solving! Way to go!!!

19 of 19

Thank you!

Any text here?