1 of 10

Data Wrangling

2 of 10

What does data look like?�Gene Expression Data

3 of 10

What does data look like?�Ecology Data

4 of 10

What does data look like?�Health Record Data

5 of 10

Tidy Data

  • First row typically contains column names. This is often called as header.
  • Each variable forms a column (e.g. age, sex, BMI)
  • Each observation forms a row (e.g. genes, species, person)
  • Each type of observational unit forms a table

​

​

​

Tidy

Untidy

6 of 10

Structured vs Unstructured data

  • Data frames we see in the previous slides are already structured data.
  • However, very rarely is data already available in this form.
  • Often the data is unstructured and does not follow a standard format:
    • Clinical notes written by doctors
    • Web pages
    • PDF
    • Tweets
    • Images

7 of 10

Excel vs R

  • Scalability
    • Excel: Works well for small datasets (<1M rows). Crashes with very large data.
    • R: Can handle millions of rows and integrates with databases.
  • Reproducibility
    • Excel: Analysis often involves manual clicks and edits → hard to reproduce or audit.
    • R: Everything is code. Your entire analysis is a script you can rerun at any time, ensuring reproducibility and transparency.
  • Automation
    • Excel: Manual repetition (copy/paste, updating formulas) is common.
    • R: Once you write a function or script, you can rerun it with new data instantly. Saves enormous time for recurring analyses.
  • Data Wrangling & Visualization
    • Excel: Good for quick manual edits and basic visualization
    • R: Packages available for easier and advanced data cleaning.

​

8 of 10

Data Wrangling

  • Converting unstructured data into structured data that is ready for analysis
    • Importing data from files
    • Joining tables
    • String processing

​

  • We only spent a week on data wrangling, but you will likely spend more time cleaning data in your jobs/research.

​

  • We will not cover everything about data wrangling, but once you learn the basics, you can quickly build up those skills in your jobs with the help of AI.

​

​

​

9 of 10

Further Learning

  • SQL is widely used in data-intensive industries to manage and manipulate large databases. It is often listed as a required job skill for interviewing a data science job.
  • In R, dplyr functions like filter, select, and the joins we will learn this week, mirror SQL operations.
  • There many online resources to learn SQL such as W3Schools SQL Tutorial.

10 of 10