1 of 45

Table Fundamentals

1

Data 6 Summer 2024

LECTURE 08

Learning how to work with tables in Python.

Developed by students and faculty at UC Berkeley and Tuskegee University

data6.org/su24/syllabus/#acknowledgements-

2 of 45

Ice Breaker

Would you rather fight:

1 horse-sized duck

OR

100 duck-sized horses

2

3 of 45

Course Input

How many people have:

Looked at HW01
Started HW01

3

4 of 45

Week 2

Announcements!

Homework 1 and Survey 1 have been released and will be

Due on 7/11 @ 11 PM

Reading Assignment 2 due Friday 12pm
Lab 2 due tomorrow (7/10) 11pm
Office Hours take place

Wednesday 9-10 AM @ SOCS 126
Thursday 5:00-7 PM @ Warren 101B

Discussion will be Friday at 10 AM in SOCS 166 this week unless otherwise instructed
Quiz 1

Next week on Monday, in-class

4

5 of 45

Week 2

Announcements!

SRI Research Consent Form

Optional but it will greatly help the CalTeach SRI researchers

5

6 of 45

Today’s Roadmap

Lecture 08, Data 6 Summer 2024

Table Attributes and Methods
select, drop, and add columns
Create new tables
More Table Methods and Documentation
Demo: WNBA Data

6

7 of 45

Data 6 Content

7

Module 1

Introduction to Python and Jupyter Notebooks

Module 2

Data Visualization

Module 3

Working with Tables

Module 4

Simulation and Randomness

8 of 45

Exploratory Data Analysis

John Tukey (1915-2000) was a Princeton Mathematician & Statistician�and an Early Data Scientist. He coined the phrase Exploratory Data Analysis (EDA).

8

[Data Analysis & Statistics, Tukey 1965; Image from LIFE Magazine]

EDA is like detective work:

Exploratory data analysis is an attitude, a state of�flexibility, a willingness to look for those things that�we believe are not there, as well as those that�we believe to be there.

Today we will perform EDA by analyzing tables using Python.

9 of 45

Review: Table

Review: In this course, we often start with tabular data and perform EDA:

Columns
Rows (also known as records).

9

	Estimated total state population	Estimated high school graduate or higher (%)	Estimated bachelor's degree or higher (%)
Alabama	3,344,006	86.9	26.2
California	26,665,143	83.9	34.7
Florida	15,255,326	88.5	30.5
New York	13,649,157	87.2	37.5
Texas	18,449,851	84.4	30.7

row

column

10 of 45

Today’s goals

If you are new to programming, it is easy to get lost in the syntax of Python! For today, we will therefore focus on two meta-goals:

How to translate your existing critical thinking skills and propensity for EDA into algorithmic thinking with Python Tables
How to read Python documentation to find what you want
How to debug code systematically

Review: In this course, we often start with tabular data and perform EDA:

Columns
Rows (also known as records).

10

	Estimated total state population	Estimated high school graduate or higher (%)	Estimated bachelor's degree or higher (%)
Alabama	3,344,006	86.9	26.2
California	26,665,143	83.9	34.7
Florida	15,255,326	88.5	30.5
New York	13,649,157	87.2	37.5
Texas	18,449,851	84.4	30.7

11 of 45

Table Attributes and Methods

11

1. Table Attributes and Methods

2. select, drop, and add columns

3. Create new tables

5. More Table Methods and Documentation

6. Demo: WNBA Data

➤

12 of 45

First, assign a Table to a name

To work with this table, we first load it in from a file called data/cal_unis.csv:

schools = Table.read_table('data/cal_unis.csv')

This lecture we will return to the dataset of public four-year colleges and universities in California (source):

12

assigns RHS value to schools name

returns a datascience Table from the given data file

Then we use schools to call Table methods and attributes.

Always assign your table to a name!

13 of 45

Next, use the documentation

We start with asking early questions about the table, such as # rows, # columns, and getting specific columns.

Note the return types!
Note the different syntax (more later).

13

Access Su24 datascience documentation: here

14 of 45

Quick-check 1

Answer on Ed!

14

15 of 45

Quick check

We start with asking early questions about the table, such as # rows, # columns, and getting specific columns.

Note the return types!
Note the different syntax (more later).

Using the documentation, fill in the following code to:

Assign ncols to the number of columns in schools
Assign arr to the Name column, but only use integer arguments

15

Access Su24 datascience documentation: here

16 of 45

Attributes vs. Methods

Table attributes

Properties/ characteristics of the given data type (in this case, datascience.Table)

Table methods

Functions that are called on a given data science table (in this case, schools)
All table methods return copies of information from the original table!

16

Very important Python datascience library paradigm

17 of 45

Quick-check 2

Answer on Ed!

17

18 of 45

What is going to happen? .take, .sort

What will be the output of each cell? Example: A, B, C means the outputs are table A, then table B, then table C.

18

schools�(before cells are run)

19 of 45

What is going to happen? .take, .sort

What will be the output of each cell? Example: A, B, C means the outputs are table A, then table B, then table C.

19

E. Error

D.

A.

B.

C.

20 of 45

Select, drop, and add columns

20

➤

1. Table Attributes and Methods

2. select, drop, and add columns

3. Create new tables

5. More Table Methods and Documentation

6. Demo: WNBA Data

21 of 45

Column-first paradigm

Because columns represent variables, we will first learn a column-first approach to tables:

(Next week, we’ll learn how to edit specific rows; for now we won’t).

tbl.select(...) returns a new table with a subset of columns.
tbl.drop(...) and returns a new table without a subset of columns
tbl.with_columns(...) returns a new table with additional new column(s).

Now to the notebook for two demos!

21

22 of 45

Reorder Columns

How do we reorder the columns to get the following table?

Hint: use one of .select, .drop, or .with_columns.

22

23 of 45

Reorder Columns: Solution

How do we reorder the columns to get the following table?

23

24 of 45

Quick Check 3

Answer on Ed!

Fill in the three blanks so that the correct table is returned.

24

Quick Check

25 of 45

Create new tables

25

➤

1. Table Attributes and Methods

2. select, drop, and add columns

3. Create new tables

4. More Table Methods and Documentation

5. Demo: WNBA Data

26 of 45

What does the following code do?

26

27 of 45

What does the following code do? Solution

27

With this newly created table,

Add some new columns.

Assign the new table with new columns to states.

28 of 45

What does the following code do? Solution

28

Add 3 columns that have these labels…

…and that have these values.

29 of 45

Good programming style

29

Use whitespace to your advantage!

As a human, why is the code above potentially easier to read than the code below?

30 of 45

31 of 45

(1/3) Navigating documentation

Early programmers often find documentation the most challenging.

Suppose we have our schools table:

How do we get all the column labels of schools?
How do we update schools such that the column Name is renamed University?

31

32 of 45

(1/3) Navigating documentation

Early programmers often find documentation the most challenging.

Suppose we have our schools table:

How do we get all the column labels of schools?
How do we update schools such that the column Name is renamed University?

32

Python tuple datatype�(fixed, ordered list of values)

33 of 45

(2/3) Navigating documentation:�Filtering with where

Often times, we will want to filter the rows of a table based on a specific condition, i.e., get only the rows where some condition is true.

33

How do we get a table with all of the UC schools?
How do we get a table with all the schools in Los Angeles?

# 1

# 2

We will learn more complicated uses of .where later in the course (“Predicates”), but for now we filter only on exact matches.

34 of 45

(3/3) Navigating documentation:�Be careful with .show

The following cells exhibit a tricky, but potentially common bug.

When run in sequence, why does the last cell throw an error?

Despite this, what might be a �good use case for using .show?

34

35 of 45

(3/3) Navigating documentation:�Be careful with .show

The following cells exhibit a tricky, but potentially common bug.

When run in sequence, why does the last cell throw an error?

35

.show is purely for display purposes. It does not return anything—i.e., it returns None.

The second cell therefore assigns schools to None!

36 of 45

(3/3) Navigating documentation:�Be careful with .show

The following cells exhibit a tricky, but potentially common bug.

When run in sequence, why does the last cell throw an error?

Despite this, what might be a �good use case for using .show?

36

.show is useful when you want to display intermediate output for debugging purposes.

37 of 45

Questions?

37

38 of 45

Even More Practice

(There are a few extra exercises in today’s lecture notebook for you to check out on your own time.)

38

39 of 45

Demo: WNBA Data

39

➤

1. Table Attributes and Methods

2. select, drop, and add columns

3. Create new tables

4. More Table Methods and Documentation

5. Demo: WNBA Data

40 of 45

WNBA Data

Let’s apply the methods we just learned about to a dataset of WNBA statistics from the 2020 season. The data was downloaded from here.

40

41 of 45

Questions?

41

42 of 45

Even More Practice

Answer on Ed!

42

Fill in the blank so that the correct array is returned. Select all that apply.

'latitude'
-2
2
3

Quick Check

43 of 45

In Conclusion…

43

44 of 45

Week 2

Announcements!

Homework 1 and Survey 1 have been released and will be

Due on 7/11 @ 11 PM

Reading Assignment 2 due Friday 12pm
Lab 2 due tomorrow (7/10) 11pm
Office Hours take place

Wednesday 9-10 AM @ SOCS 126
Thursday 5:00-7 PM @ Warren 101B

Discussion will be Friday at 10 AM in SOCS 166 this week unless otherwise instructed
Quiz 1

Next week on Monday, in-class

44

45 of 45

Recap

Next Time

Tables!
Columns and Variables
Table Methods

Introduction to Visualization

45