4 of 9

Invitation to extra credit opportunity�Invitation to participate in a research study

Today, I'm going to tell you about an extra credit opportunity

That opportunity includes an invitation to participate in a research study.

What will be asked of you to earn the extra credit:

Fill out a survey this week that will take about 10-15 minutes to complete
Fill out another one during week 10 / finals week (again, 10-15 minutes)

What about the research study

The survey includes an opportunity to opt-in or opt-out of the study
You don't have to opt-in to the study to get the extra credit
We hope that you will, but if you don't, it does not affect your grade in the course

5 of 9

What is this research study?

Who: Collaboration among faculty from three Universities:

Phill Conrad, UCSB
Chris Hundhausen, Oregon State University
Sola Adesope, Washington State University

What:

Studying use of "brownfield projects" in courses like this one.
Brownfield means the code existed before you see it, and lives on after the course

Why:

To learn how to better prepare students like you for industry careers
To learn how to assess your work (esp. group work) more accurately and fairly

How:

Collecting information from GitHub repos, Slack conversations, CATME surveys, grades
Data is anonymized before it is reported to protect your privacy

6 of 9

Sample work so far

SIGCSE 2021

Evaluating Commit, Issue and Product Quality� in Team Software Development Projects

How do you measure the quality of a commit message, or an issue on a Kanban board?
Important for accurate feedback, and fair grades.

ITICSE 2021

Teaching Testing with �Modern Technology Stacks in �Undergraduate Software Engineering Courses

How do you incorporate the kinds of software testing done in industry into a course like CS156?
Honorable mention for best paper (out of 84 accepted, 275 submitted)

SIGCSE 2023

Investigating Reflection in �Undergraduate Software Development Teams: �An Analysis of Online Chat Transcripts

Do students engage in reflection in their team chat channels on Slack?
Not much as it turns out; this is an opportunity for improvement

Journal: Computer Science Education�(Taylor and Francis), May 2022

Assessing individual contributions to �software engineering projects: �a replication study

Are individual members of a team each "pulling their weight? �Can you tell from CATME scores? �GitHub activity?
We replicate and extend earlier work by a CSU Chico professor

TOCE (to appear)

Combining GitHub, Chat, and Peer Evaluation Data to Assess Individual Contributions to Team Software Development Projects

Extends the May 2022 replication study to include chat data and other new metrics of equitable team contribution.

7 of 9

A few more notes

If you participate, we protect your autonomy and privacy

Until final course grades are submitted, your instructor (Conrad)�will only have access to whether you filled out the survey or not

no access to your actual answers
no access to whether you opted-in to research participation or not

After final course grades are submitted

answers and opt-in/opt-out are revealed
but individual information is not published in the papers
only aggregate data is reported

You must be > 18 years of age to opt-in to the study.�But anyone can earn the extra credit.

We really hope most folks opt-in; �we need 100% of folks on a team in order to use the data for that team.

8 of 9

How to participate

Link to the survey posted on Canvas and the course Slack

The survey asks for a "Random Participant Identifier (RPID)"

Don't enter your perm number for that
The RPID has four digits
It can be found on Canvas as RPID in your "grades"
Don't enter the comma! (e.g. 1234 not 1,234)

�

Y9876

9 of 9

We hope that everyone will participate!

Last year WSU had 100% participation in the survey.

UCSB's rate was pretty high on individual student basis

But the non-consenters/non-responders were spread across teams:

To be ethical, we had to excluded data for the whole team�if even one person withheld consent, or just didn't fill out the survey.
So, we are really hoping for as many "yes" answers as possible.
However, it is important that you do not feel coerced in any way�Federal Law and UC policy require "informed consent"
That's why you get the extra credit no matter what
That's why we want to take the time to answer your questions.

What questions do you have?

1 of 9