SEISMIC
Evaluation Activities
Summer Meeting
Measurement Fellows and Undergraduate Scholars
Future Evaluation Activities
Discussion
Summer Meeting
Findings from qualitative data collected during the meeting
Summer Meeting was held June 17-June 25, 2021
Data collected
Voices heard include those of faculty, staff, post-docs, graduate and undergraduate students
Summer Meeting Report
Findings – Major Themes
Findings – Major Themes (continued)
Conclusions and Recommendations
Conclusions and Recommendations
Measurement Fellows and Undergraduate Scholars
.
Post experience interviews
During October and November 2021, CEDER interviewed
about their summer research experiences
Individual and Focus Group Interviews
Most of the Measurement Fellows learned about the experience through a graduate student email chain.
Most of the Undergraduate Scholars learned about the experience through a professor, advisor, or other mentor
How did they learn about the opportunity?
Goals for the Experience
“My goals were to (1) learn more about the equity, diversity, and inclusion issues faced by students in STEM courses, (2) participate in some professional development workshops, and (3) develop a lasting, meaningful relationship with mentor(s).” SEISMIC UG Scholar
“I'm trying to think, my goals for the summer were to be involved in some research opportunity to try and fill the gap between semesters. But also, this specific organization attracted me because I've done mathematics research, and worked at a community college, and went to a community college, and it just seemed like this was something that I already had interest in. So, it was good to see that there's a program that I could have joined to see kind of what is the research in introductory courses in STEM, like how students get introduced to these topics, and what inequities may be there” SEISMIC MF
Quality of the Experience
SEISMIC Measurement Scholars and Undergraduate Scholars’ feedback on their summer research experiences was mostly positive.
“I really enjoyed it. I think the group that I was assigned was very supportive. At SEISMIC, I think you get the opportunity to either pursue an independent project or you can work with a preexisting project. And I decided to work with a ongoing project in this area of determining the impact of class composition on performance. And I really enjoyed it a lot. I think they're very supportive. Ad they really encouraged me to be creative and offer my insights despite being only starting out in the Fellowship, which I think really helped validate my opinions. I don't know, I think that was a really good experience.” SEISMIC MF
“SEISMIC was the first place where I got to do something related to research. It was a good place to start since everyone was very kind and happy to help. I was also able to get a good overview of many things.” SEISMIC UG Scholar
Quality of the Experience
Professional Development Sessions
“They were VERY useful. They give us overview of many things. I really enjoyed how we learned about types of research, coding with R, resume and cover letter writing, and others. It was a great overview of many things. SEISMIC UG Scholar
Mentoring Experience
“I received professional development mentoring from the SEISMIC mentors. I cannot say enough how meaningful the advice of my mentor has been, especially regarding my applications to graduate programs and my development as a scientist and a communicator.” SEISMIC UG Scholar
Quality of the Experience - Funding
The funding the fellowship provided was critical
“If I didn't get the fellowship, I wouldn't have been able to work on the survey over the summer. I would've had to do something else. It wouldn't have been a priority. And this project, which was sort of just a side volunteer project that I was doing, has now become like my main task with my GSR. And I think seeing some traction with SEISMIC is what helped me get traction to be able to get a GSR to work on it.”
“So that was really helpful to feel like I had something to do over the summer that was productive and feel like I was compensated for it and not just watch the savings drop over the summer, wait till September 1st to breathe again.”
Quality of the Experience – Measurement Fellow Showcase
Feedback on the Measurement Fellow Showcase was very positive
“I think having this culminating event put my work into perspective. I think that being encouraged to present my work to others highlighted the impact of the work I could be making. Because, I don't know, I feel with the enthusiasm from the audience members, after hearing my presentation, it made me actually more excited to pursue this research even more.”
Quality of the Experience
Overall Strengths
“I thought that the SEISMIC mentors and project leads were incredibly kind and genuinely cared about our welfare and futures. The professional development sessions were very helpful. I thought the best part of the program was the opportunity to present our work at the end, to both our peers and other members of the SEISMIC community. This was both fun (because we got to share our work with our fellow SEISMIC scholars and hear about what others had been working on) and an incredible opportunity to practice our presentation and communication skills.” SEISMIC UG Scholar
“I think the best experience to maybe highlight is we just did like two iterations of a survey and SEISMIC community members gave us extensive feedback on both iterations. Which was, they were the most responsive and most useful resources to give us feedback on our survey, which I would not have gotten unless I had the fellowship. I don't think they would have was interested to help, but since they knew who I was, and they knew a little bit about the project, they were all very responsive and helpful.” SEISMIC MF
Quality of the Experience
Challenges
One challenge reaped an unexpected opportunity for a UG Scholar to learn about intersectionality
“Because originally the programs mostly made out to be for minorities and I think one thing that really caught me off guard was in the beginning when I was looking at the students that got accepted into program, I was like, oh, doesn't really seem that diverse…But then during the trainings, the seminars that we had prior to the beginning of the program, they also told us…we can't just make assumptions based off exteriors. We don't know the other categories that make up the identity of an individual…As for the individual, who knows from the interior.”
Recommendations
Measurement Fellows
“It felt like there might be conflicting visions within SEISMIC…It’s written into the grant a little bit that this is attempting to do rigorous quantitative research about equity issues, because we've seen a lot of qualitative research in equity and we want to do something else, but there's this natural tendency in equity research to view quantitative researches in some way essentializing or humanizing. And so,... there was just some tension there, a few members didn't like... In some of the more informal discussions I had with graduate students and with junior members, I could see that tension a little bit that how do we frame our quantitative research within this equity framework that we're trying to use? Because it's a little bit outside the norm for the community, it seems like.”
Recommendations
Undergraduate Scholars
Shorter meetings, especially if on Zoom
“Something that would have helped me is to have 2 shorter weekly meetings (~1 hour each) instead of 1 longer meeting of two hours. I found my focus and engagement waning in the last hour of the meeting (this is true of any Zoom call, not just this specific one). I think we could have seen greater participation from everyone if the meetings were shorter and more regular.”
Identify an advocate for junior scholars
I would say, maybe if there was some sort of representative, or advocate for junior scholars, because I feel like right now we don't really have someone standing in for us to share our opinions. And I really think that that is a really important factor, because if we're doing research for students in STEM, especially introductory courses, then why are junior scholars not given a platform to speak? And why do they not feel comfortable participating?
Recommendations
Undergraduate Scholars (continued)
Have two or more individuals on a project
“I know that for some projects we had two individuals working together, two students within every project. But then through other projects, we just had one individual. I know that some projects have limitations on how many people they could take in. But I think it would be nice if we had two individuals at least per project. So that way the students could connect to each other in case they need help or just to talk to each other and say how are you doing? Do you think this would work? And just be able to connect. I feel that would be really great to incorporate. Because I noticed that for one of my peers that was by himself, he was really struggling sometimes. But he didn't know who to talk to because he could talk to his advisor, but it was not the same as talking to a student who's in the same place as you.”
Measurement Fellows felt connected to SEISMIC and all but one wanted to remain connected. They would like to learn, e.g.
“We can keep saying it's there, but what are we going to do about it? And so, knowing what that roadmap looked like or what's currently going on the intervention side might have been helpful.”
Future Interactions with SEISMIC
Future Evaluation Activities
Ideas for 2022
Things are opening up for travel site visits and engaging with local teams
Focus 1 SEISMIC Teams
Possible Evaluation Questions
�Planned Activities
What’s New in 2022
Focus 2: SEISMIC Summer Meeting
Possible Evaluation Questions
TBD [Revise based on last year’s survey and survey described in SEISMIC proposal]
�Proposed Activities
�
What’s New in 2022
Focus 3: Overall
�Proposed Activities
�
What’s New in 2022
Discussion