1 of 18

CSE 163

Section AX

​

TA 1 & TA 2

​

​

Question of the Day: What’s your favorite baked good?

​

​

​

2 of 18

Housekeeping 🏡

Important Dates and Reminders

​

  • Homework 2 due Tuesday, October 20 at 11:59pm on Gradescope
  • Programming Practice 2 due Sunday October 18 at 11:59pm on Gradescope
  • Section Check-in 3 due Friday October 16 at 11:59pm on Gradescope

​

​

​

​

​

3 of 18

Game Plan

What We’ll Cover Today

  • Quick Review
  • Todo: pandas-fundamentals-au26.ipynb

4 of 18

DataFrames Review

5 of 18

Data

Goal: Represent real-world phenomena in a tabular format

CSV

  • Ubiquitous representation of data
  • Comma separated values.
  • First row is columns
    • Every other row is a “record” or instance of data

# cats.csv

import pandas as pd

data = pd.read_csv("cats.csv")

​

print(data)

​

===============================

name color age outdoor

0 fluffy ginger 2 True

1 sparky grey 8 False

2 storm black 1 True

===============================

​

# cats.csv in a list of dicts

cats = [

{"name": "fluffy", "age": 2},

{"name": "sparky", "age": 8},

{"name": "storm", "age": 1}

​

​

6 of 18

Processing:

List of dicts

  • Loop through the list of dicts
    • Every iteration will set your iteration variable to a new row
  • Extract variables with dictionary indexing
  • Notice- Row-by-row processing

# cats.csv in a list of dicts

cats = [

{"name": "fluffy", "age": 2},

{"name": "sparky", "age": 8},

{"name": "storm", "age": 1}

]

​

​

total_age = 0

for c in cats:

# c stores a cat dictionary

# ex: {"name": __, "age": __}

age = c['age']

total_age += age

​

total_age # 11

​

​

7 of 18

DataFrames

  • Tabular Data
    • Similar to list of dictionaries, but more powerful
    • Columns are Series
    • Single rows are Series
  • Able to get entire columns
    • data[‘column_name’]
    • data[ [‘col1’, ‘col2’] ]
  • Aggregate data
    • data[‘column_name’].sum()
      • .mean(), .max(), .unique()...
  • Series-based operations
    • data[‘column’] + …

import pandas as pd

data = pd.read_csv("cats.csv")

​

print(data)

​

===============================

name color age outdoor

0 fluffy ginger 2 True

1 sparky grey 8 False

2 storm black 1 True

===============================

data['age'] # Series([2, 8, 1])

data['age'].sum() # 11

​

data['age'] * 4 # Series([8, 32, 4])

8 of 18

DataFrame Filtering

  • Masks
    • Boolean Series from a mask
      • data[‘age’] > 3
    • Filter by using mask in dataframe
      • data[data[‘age’] > 3]
    • Combine masks with &, |, ~
      • data[mask1 & mask2]

​

data['age'] > 1

# Series([True, True, False])

data[data['age'] > 1]

​

===============================

name color age outdoor

0 fluffy ginger 2 True

1 sparky grey 8 False

===============================

​

data['outdoor'] == True

# Series([True, False, True])

data[data['outdoor'] == True]

===============================

name color age outdoor

0 fluffy ginger 2 True

2 storm black 1 True

===============================

​

9 of 18

DataFrame Filtering

cont.

===============================

name color age outdoor

0 fluffy ginger 2 True

1 sparky grey 8 False

2 storm black 1 True

===============================

​

old_age = data['age'] > 1

# Series([True, True, False])

outdoors = data['outdoor'] == True

# Series([True, False, True])

old_age & outdoors

# Series([True, False, False])

data[old_age & outdoors]

​

===============================

name color age outdoor

0 fluffy ginger 2 True

===============================

​

10 of 18

DataFrame Indexing with .loc

  • Use a row_indexer and column_indexer to specify where to extract data.
  • Data.loc[row_indexer, col_indexer]
  • Can also get you single values
    • data.loc[0, ‘age’]
  • Can use slices too!
    • Data.loc[0:2] gets rows 0, 1, and 2
      • Use “:” for “all”
    • Can use lists
      • data.loc[0, [‘name’, ‘age’]

​

df.loc[0] # same as df.loc[0, :]

# Series(["fluffy", "ginger", 2, True])

​

df.loc[0:1] # same as df.loc[0:1, :]

​

# DataFrame

===============================

name color age outdoor

0 fluffy ginger 2 True

1 sparky grey 8 False

===============================

​

df.loc[0:1, "color"]

# Series(["ginger", "grey"])

​

11 of 18

Groupby Demo

12 of 18

Group By

result = data.groupby('col1')['col2'].sum()

​

​

​

​

​

​

​

​

​

​

​

​

​

12

​

col1

col2

0

A

1

1

B

2

2

C

3

3

A

4

4

C

5

​

col2

C

3

5

​

col2

B

2

​

col2

A

1

4

A

5

B

2

C

8

A

5

B

2

C

8

Data�DataFrame

Split

Apply

Combine�Series

13 of 18

13

Practice problems!

​

14 of 18

Solutions

Individual Activity and Group Activity

1) gsb.columns

​

2) gsb['Name'] and it returns a Series -> type(gsb['Name'])

​

3) gsb[gsb['FlavorScore'] > 70]

​

4) gsb['CreativityScore'] = [100, 80, 70, 90, 100, 20, 80, 90, 100, 70, 70]

​

5) gsb['FlavorScore'].max() - gsb['FlavorScore'].min()

​

6) gsb['TotalScore'] = gsb[['FlavorScore', 'PresentationScore', 'CreativityScore']].mean(axis=1)

​

​

15 of 18

Solutions

7)gsb['CityOfOrigin'].unique()

​

8) gsb[(gsb['CityOfOrigin'] == 'Seattle') | (gsb['CityOfOrigin'] == 'Florence')]

​

9) gsb = gsb.set_index('BakerID')

​

10) gsb.loc[105, ['CityOfOrigin', 'StartedBaking']]

​

11) )gsb.loc[[101, 110], ['Name', 'DessertBaked']]

​

​

​

​

16 of 18

Groupby problems solutions

  1. # first, groupby the city of origin, then access the flavor score column and find the index of the max value

​

grouped_cities = gsb.groupby("CityOfOrigin")["FlavorScore"].idxmax()

​

# then, use the grouping from above as our row indexer in .loc (since we used idxmax()) and specify the other columns we want to display

​

gsb_grouped = gsb.loc[grouped_cities, ["Name", "CityOfOrigin", "FlavorScore"]]

​

​

  1. # groupby both columns (order matters), then call .size() to show the count

​

city_year_count = gsb.groupby(["CityOfOrigin", "StartedBaking"]).size()

​

17 of 18

Groupby problems solutions

3)city_avg = gsb.groupby('CityOfOrigin')['FlavorScore'].mean()

city_avg[city_avg > gsb['FlavorScore'].mean()]

​

18 of 18

18

Section Code

​