1 of 14

Measuring Creatively the Impact of Applied Learning on Students’ Affective Development

Ciara Christian, Steve Freeland, Simon Stacey, Hannah Schmitz, Michele Wolff

2 of 14

Project Origin

Honors

College

Interdisciplinary Studies

Shriver

Center

?

The origins of this research project may be found in three different conversations that took place at UMBC during the Academic year 2013/2014. One involved the Honors College, another involved the Shriver Center and the third involved the Interdisciplinary Studies Program (“INDS”). The common thread linking these conversations was Dr. Diane Lee, then Vice Provost and Dean of Undergraduate Education at UMBC (and now emerita).

Michele Wolff, director of the Shriver Center, had met with Dr. lLee to discuss how applied learning could best be integrated/encouraged more broadly throughout UMBC’s curriculum. Simon Stacy, director of the Honors College and Stephen Freeland, director of the Interdisciplinary Studies Program had each (independently) met with Dr. Lee, as their direct supervisor, to discuss how to better incorporate applied learning into their respective academic units. Dr. Lee helped these three individuals meet early in the winter of early 2014 in order to converge their overlapping interests. A primary finding of their first meeting was that numerous individuals, distributed across UMBC, had something's relevant to contribute to any attempt to distill “best practices.” As a result, Wolff, Stacy and Freeland applied (successfully) for a Breaking Ground grant to convene a larger group of relevant faculty and staff

3 of 14

2 Minute Exercise!

Think about an instructional happening (past, present or future) with an applied learning experience*

What were you hoping students would take away from this experience? What would they be able to do as a result of it that they couldn’t otherwise have done? Say as much as you can about your hopes and expectations for their learning.

* For the purposes of today’s presentation, let’s define this as: “something that connects theory to practice/ action to thought.” Examples include: service-learning, internship, cooperative education, performance, research and leadership.

4 of 14

Bloom’s Affective Taxonomy

5 of 14

Our Project Has Identified 9 Affective Functional Competencies for exploration

Critical Agency
(Empathy)
Ethics and Integrity
Innovative Leadership
Intercultural Development and Perspective
Resilience and Adaptability
Self-awareness
Social Responsibility
Teamwork

6 of 14

Our Project Has Identified 9 Affective Functional Competencies for exploration

Critical Agency
(Empathy)
Ethics and Integrity
Innovative Leadership
Intercultural Development and Perspective
Resilience and Adaptability
Self-awareness
Social Responsibility
Teamwork

Identify the AFC (Affective Functional Competency) that most closely aligns with what you hoped students would get from the ALE (Applied Learning Experience) you identified.

7 of 14

Our First Attempt at Measurement: Likert scale

Clarity of purpose: Understands and can articulate clearly intended impact while appreciating that complexities, conflicts and ambiguities will inevitably be encountered

8 of 14

Findings

Students provide inauthentic (socially desirable) responses
This style of question produces survey fatigue
Recipients misinterpret questions’ intent

(this includes, but is not limited to, intercultural effects)

As self awareness grows, individuals score themselves lower!
All the limitations of self-reporting�

9 of 14

Our second attempt at measurement:

Partnering with Students

INDS 430: “Creative Survey Design: An Interdisciplinary Approach to Creating Interactive Assessment Tools”

Involved 8 students as co-creators

(backgrounds in Interdisciplinary Studies, Psychology, Graphic Design/Art, Statistics, Information Systems, Political Science, Health Administration and Policy)

Students developed ideas for tools to measure specific Affective Functional Competencies, and explored their own growth in the affective domain

10 of 14

Findings

Students find this research relevant and engaging...

Our biggest impact was on the students in the class (not as originally envisaged on others, using tools created by the class)

=> we have evolved the class to run as a First Year Seminar starting Fall 2017

This topic is (still) HARD! We are learning from our mistakes

=> For example, empathy is NOT a good subject for assessment of affective learning as it is too broad and complex to hold useful meaning across different instructors and courses

Student reflections are IMPORTANT: qualitative data from self reporting tells us far more than our clumsy, superficially quantitative beginnings...

11 of 14

Paired Mini-Activity #1

Review one another’s statements about what you hoped your students would get from their ALEs, with the goal of disentangling the cognitive and affective components of these hoped-for outcomes. Do this any way you please- by underlining the two kinds of elements in different colors, by putting words in different columns, by circling them, by drawing arrows, etc.

12 of 14

Paired Mini-Activity #2

Using what you produced in the previous activity, each write a Student Learning Outcome (SLO) statement for your ALE that foregrounds and emphasizes its affective elements.

13 of 14

Paired Mini-Activity #3

Try to identify a mechanism/tool/instrument that you can imagine using to assess the achievement by students of your newly-developed SLO. Feel more than free to be unorthodox!

14 of 14

Creating Innovative Measurement Tools

Example: Reading the Mind in the Eyes