1 of 6

Chapter 7: Education Data Science Pipeline with

Online Science Class Data

Bookclub

3/10/2021

2 of 6

Goals

Learning about key steps in data preparation, processing, wrangling using tidyverse (a set of packages)

​

source: https://whatsthebigdata.com/2016/05/01/data-scientists-spend-most-of-their-time-cleaning-data/

​

source: https://r4ds.had.co.nz/wrangle-intro.html

​

3 of 6

Driving question & data

Question: What ‘explains’ student final grade in online science courses?

​

Variables (data type)

dataframe (source)

  • Final grade for the course (achievement, DV)
  • Subject (5 categories: AnPhA, BioA, FrScA, OcnA, PhysA)

course_data (Gradebook in LMS)

Three student motivation measures from 10 self-report questions (affect)

  1. Interest (int.)
  2. Perceived competence (pc)
  3. Utility value (uv)

pre_survey (from LMS, given before the start of the course)

Number of minutes students spent on the course (log-trace/process data)

course_minutes (from LMS)

Discussion board (text/string, learning process)

LMS

Note: pre_survey, course_data, course_minutes are all imported from {dataedu} package

4 of 6

Outline of data processing & analysis

  1. Import & view data
    1. view(dfSummary(df)) from {summarytools} to get a quick view of each var/column in a dataframe (df)
  2. Process three sets of data (which come as dataframes, dfs)
    • pre_survey: pivot_longer to make df a long format ; mutate to create a new var/column or overwrite
    • course_data → separate subject from CourseSectionOrigID
    • time_spent → mutate to transform to hours, scale to standardize (mean 0, SD = 1)
  3. left_join (merge) the three dfs into new df called dat using student_id & course_id as keys
    • course_dat as main df (place it first/left), and add pre_survey, and time_spent
  4. Find distinct student cases for each student & course combo
    • course_dat is a long format with many gradebook items come in rows
    • We subset dat to include one row per student per course.
  5. Analyze data
    • Relationship between time spent & final grade (scatter plot with {ggplot2}, {ggpubr})
    • Correlation matrix among 3 motivation measures apa.cor.table {apaTables}
    • Linear regression models lm( ) → regression tables with apa.reg.table {apaTables} or tab_model {sjPlot}
      1. Time in minutes
      2. Time in hours (transformed)
      3. Standardized time (transformed)
      4. Standardized time AND subject as predictors

blue = function

{purple} = package

green = output

5 of 6

pivot_longer: make a df from wide format to long format

wide format (pre_survey, original format)

long format (pre_survey, transferred, after using pivot_longer function)

pre_survey %>%

pivot_longer(cols = q1:q10,

names_to = "question",

values_to = "response")

⇒ you specify the highlighted parts

6 of 6

left_join is most common; but there are others to merge 2 dfs

  • a and b are the two dataframes you want to merge
  • by = “x1” specifies x1 is the key that appears in both dfs.
  • You can have multiple variables as key
  • If key variables have different names in the 2 dfs you can specify that in the statement