Welcome (back) to Stat 494: Statistical Genetics!

While we wait to get started...

• Sit wherever you want! (just avoid the back corner tables…)
• Join Slack (if you haven't already) and catch up on recent messages
• Office Hours schedule
• Pictures from Friday's Grading Gallery Walk
• Updated Learning Goals Brainstorm doc
• Sign up for a Journal Club presentation slot
• Review your notes and prepare for today's Journal Club discussion!

Goals for today

• Journal Club # 1
• Why are we doing this?
• Tips for reading scientific articles
• Butler & Nisan discussion
• What's next?

​

• Syllabus Highlights: Co-Grading

​

• Intro to GWAS
• What do genetic data look like?
• (How) can we use linear regression in this setting?
• Wrapping our heads around the size of genetic data

Journal Club #1

Why are we doing this?

• Publishing a research article in a peer-reviewed journal is one of the primary mechanisms by which statistical methods and scientific results are communicated!
• Learning to navigate the format and writing style of research article is a worthwhile skill to develop…
• Conducting research
• Staying up-to-date on new statistical methodology
• Being an informed participant in your healthcare
• Etc.
• … but it takes practice!
• Particular writing style & structure
• Different process for reading

Tips for reading scientific articles

• Break into smaller steps
• Read more than once
• Read in the "wrong" order
• Take notes!! (in the margins, make diagrams, pause & summarize key points, list words you don't know, etc.)
• Ask lots of questions while you read
• "Don't assume they're infallible"
• It's okay if you don't understand everything on first (or second!) read

Advice from UW Stat Gen Seminar

Advice from Dr. Jennifer Raff's "How to read and understand a scientific paper: a guide for non-scientists"

+ WHAT question are they (trying) to answer?

What questions are they (trying) to answer? (Intro)

What do you think? Discuss at your table!

BIG QUESTION(s):

• How useful grades are? Other types of information? Rewards?
• How are people generally motivated? What influence motivation?

SPECIFIC QUESTION(s):

• Can (intrinsic) motivation & performance be maintained/improved via personalized feedback? Vs normal grades? Vs no feedback at all?
• How does getting no feedback at all influence motivation?
• Is this different depending on the task? (quantitative vs qualitative)

Why is this question important? (Intro & Discussion)

• Addressing gaps in existing literature:
• lack of prior studies on "role of availability of diagnostic information relevant to self-assessment of competence on continuing task motivation"
• limitation of prior studies that compared only to "nonrecepit of any information" (not to "receipt of normative evaluation" ie grades)
• limitation of prior studies that used only a single trial
• Important implications for education
• What else? Mental Health, Applying to jobs and grad school (communication), other contexts/jobs where staying motivated is important
• Do you think the paper did a good job of answering this question? Room for improvement in generalizability, Showed students preferred it

What did they do? How did they do it? Why? (Methods*)

Draw a diagram! Some papers even do this for you…

​

Example: Systematic comparison of phenome-wide admixture mapping and genome-wide association in a diverse biobank | medRxiv

What did they do? How did they do it? Why? (Methods*)

recruit

participants

summary statistics

pilot study

conduct hyp. tests

?

Group 1 (Feedback)

Group 2 (Grades)

Group 3 (Nothing)

Session 1: Tasks A+B

Session 2: Tasks A*+B*

Session 3: Tasks A+B

implement "instruments"

provide feedback/grades

provide feedback/grades

(they describe this in Results instead)

Attitudinal Questionnaire

What did they do? How did they do it? Why? (Methods*)

recruit

participants

summary statistics

pilot study

conduct hyp. tests

?

Group 1 (Feedback)

Group 2 (Grades)

Group 3 (Nothing)

Session 1: Tasks A+B

Session 2: Tasks A*+B*

Session 3: Tasks A+B

implement instruments

provide feedback/grades

provide feedback/grades

use ANOVA and ANCOVA to compare measures across the three groups

"three city elementary schools serving predominantly middle-class populations"

assess instruments (eg are tasks interesting?)

randomly assigned by class

e.g. average age

Questions? Corrections?

Attitudinal Questionnaire

What did they do? How did they do it? Why? (Methods*)

recruit

participants

summary statistics

pilot study

conduct hyp. tests

?

Group 1 (Feedback)

Group 2 (Grades)

Group 3 (Nothing)

Session 1: Tasks A+B

Session 2: Tasks A*+B*

Session 3: Tasks A+B

implement instruments

provide feedback/grades

provide feedback/grades

use ANOVA and ANCOVA to compare measures across the three groups

"three city elementary schools serving predominantly middle-class populations"

assess instruments (eg are tasks interesting?)

randomly assigned by class

e.g. average age

Would you do anything differently?

Attitudinal Questionnaire

What did they do? How did they do it? Why? (Methods*)

​

Some ideas that came up during our in-class discussion:

• Corrections to my methods diagram:
• Add time between sessions
• Clarification questions about their methods:
• Why did they report p-values the way they did? (eg "p < 0.08")
• Why didn't they try to get an even split between boys and girls in the three groups?
• Why is there a different amount of time between sessions? Could this impact results?
• What is ANOVA/ANCOVA?
• Limitations:
• Didn't look at combined grade & feedback (which is so common in education)?
• Age of participants → results might not generalize to other age groups
• Would results change if regular classroom teachers had administered the activities/surveys rather than an outside grad student?
• Possible concerns about multiple testing (they conducted a LOT of hypothesis tests!)
• Calculating means and standard deviations on what is really a categorical variable (1 = low agreement, …, 7 = high agreement)
• Possible sampling bias (focus on schools in predominantly middle class area)

What did they find? (Results)

Task A ("quantitative"):

• Comments & grades groups performed similarly
• No feedback group performed worse

"Thus the hypothesis that performance on the quantitative task would be higher after receipt of some feedback than after nonreceipt of feedback was supported by the results."

What did they find? (Results)

Task B ("qualitative"):

• Comments group scored higher than the other two
• No "significant" difference between grades and no feedback groups

​

​

What else stood out to you?

Would you present these results any differently?

What should come next?

• What questions/ideas do you have?
• Check out how others have followed up on this work in the literature:

Debriefing Journal Club #1

Take a few minutes to write responses to the following:

• How does this article support, or challenge, your personal views / prior experience with grading and feedback?
• How was your experience reading this article? Was it more/less challenging than you were expecting? Did it take more/less time than you were expecting? What was helpful?
• What might you try differently next week?

Debriefing Journal Club #1

Tips for discussion leaders:

• Follow the tips in the UW Stat Gen Seminar "How to" Guide:
• Spend 20 – 30 minutes presenting, keeping the number of slides to a minimum (~6)
• Use the rest of the time (either at the end, or integrated throughout) for discussion
• Ideas for discussion: anything you found particularly interesting, connections you noticed between the paper and things we've covered in class (or other classes you've taken), potential limitations of the approach the authors took, parts of the paper you found confusing, etc.
• Remember: it's ok if you don't understand everything in the paper!
• Sign up here: STAT 494 - Journal Club Schedule

Syllabus Highlights: Co-Grading

About STAT 494: grading system

We'll be using a "non-traditional" grading system in this class known as

ungrading / co-grading

Why? There are a lot of problems with traditional grades

• They are not good incentive (and they incentivize the wrong things)
• They discourage risk-taking/making mistakes
• They are not always good feedback
• They are not always good markers of learning
• They feel terminal, communicating that learning is done (but it never is!)
• They are inconsistent and unfair (they may appear "objective," but they're not)

About STAT 494: grading system

We'll be using a "non-traditional" grading system in this class known as

ungrading / co-grading

Why? Also…

• Many studies have shown that feedback is more useful for learning than grades
• I want to engage with your work, rather than evaluate it
• I want you to build skills in self-reflection and self-assessment, both of which will be critical to your life/career after Macalester (and will help your learning now)
• You all have different goals, and you are the best judge of how you are doing

"the saddest and most ironic practice in schools is how hard we try to measure how students are doing and how rarely we ever ask them" – Amy Fast

What's Next?

Before our next class:

• If you didn't finish over the weekend:
• Finish the Syllabus Reflection
• Update your versions of R/RStudio (instructions linked on course website)
• Sign up for a Journal Club presentation slot (Moodle > Resources > General Course Materials)
• Prep for Lab 1
• Least Squares review (Stat 155 Notes 3.3 – 3.4)
• Least Squares Estimation Videos: Intro (5 mins) + Example (14 mins)
• Start prepping for Journal Club #2
• Discussion leaders: schedule a meeting with me before you present!