Academic Reporting Tools (ART 2.0)

at the �University of Michigan

�

Overview for OUCeL

16 August 2018

​

August (Gus) Evrard

Arthur F. Thurnau Professor, Departments of Physics and Astronomy

ART 2.0 Evangelist and Architect

​

Chris Teplovs

Initial Lead Developer and Architect, ART 2.0

Lecturer, U-M School of Information

​

Mission

​

The mission of Academic Reporting Tools (ART) is to:

​

promote deeper knowledge of the University of Michigan’s curricular history within the Ann Arbor campus community, and,

in so doing,

​

support exploration, discovery, and decision making by U-M students, faculty and staff.

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

4 distinct campuses in Ann Arbor

​

600+ buildings,

18M sq ft

​

2000+ classrooms

​

30 miles of roads

​

1600 trees

...

U-M is a BIG place.

​

How much does any U-M community member really know about it?

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Stories about the curriculum are being told around campus all the time:

​

student–student

adviser–student

faculty–student

faculty–faculty

admin–faculty

….

​

​

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Stories about the curriculum are being told around campus all the time:

​

student–student

adviser–student

faculty–student

faculty–faculty

admin–faculty

….

​

​

No one ever gets an A in that class...

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Stories about the curriculum are being told around campus all the time:

​

student–student

adviser–student

faculty–student

faculty–faculty

admin–faculty

….

​

​

I heard that course is so much work!

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Stories about the curriculum are being told around campus all the time:

​

student–student

adviser–student

faculty–student

faculty–faculty

admin–faculty

….

​

​

My friend told me he’s the most BORING professor...

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Stories about the curriculum are being told around campus all the time:

​

student–student

adviser–student

faculty–student

faculty–faculty

admin–faculty

….

​

How well informed are these stories?

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Simple design:

​

multiple decks of cards, each with relevant descriptive statistics for every

​

• course
• instructor
• major
• student

...

​

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Live demos…

production

development

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

top ⅓

of page

middle

of page

bottom

of page

InstructorInfo card (released Nov 2016)

Process

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

ART 2.0 Steering Team 2015-18

• 18 members offer representation across
• 5 colleges
• Student Life
• Registrar
• Center for Research on Learning and Teaching
• Central Student Government
• Central IT

​

• Bi-weekly, one-hour meetings during Fall, Winter terms

​

• Team members guide development and serve as communicators with their constituencies

​

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

DIG Application Developers

User Experience Designers

Behavioral Scientists

DIG Student Fellows

Faculty Innovators

DIG Communities of Practice within U-M

Scholars (faculty & students) test tool impact

Central IT

makes innovations infrastructure

DIG is a unique digital education ‘accelerator’

helps take good ideas from innovation to infrastructure

Innovations become start up companies

See Tim McKay’s LAK 2017 keynote for more information

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Some bumps in the road…

• push-back at Provost’s Academic Program Group presentation

​

How can you release my data?!

– Unit Dean

​

• compromise on Student Evaluations of Teaching (SET) release

(Student Government, Senate Assembly, and Provost)

• students can view SET summaries
• advisors, faculty and staff cannot

​

• concerns over grade information
• Grade distributions released March 2018
• who can view disaggregated grade performance?

​

• complexity of curricular data: multiple sources, dynamic nature

​

• Provost Office support critical to continued growth

​

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

ART: first year weekly traffic after March 2016 roll-out

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

ART: weekly traffic since Nov 2016 (x 3 annual growth)

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Student engagement and

LA opportunities

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

In-house user experience design a big plus

Mike Wojan

U/X Design Lead

Dana Demsky

U/X Student Fellow

ART logo

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

ART Labs : student fellows performing data analysis

initial project: context for SETs and grades

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

SET summaries: domain differences

Students claim to learn a lot in natural science courses while also reporting not being strongly motivated to take them.

• How might we change their attitudes?
• Would they (claim to) learn even more if we did?

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

under the hood

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

ART 2.0 Under the Hood

• A python-based Django application that uses MySQL as database back-end into which data is loaded from institutional Oracle data warehouse

​

• Deployed via containerized on-premises infrastructure using Redhat’s Openshift Platform ( = Docker + Kubernetes)

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

ART 2.0 Data Ecosystem

ART 2.0 Data Ecosystem

next steps

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

more things to do…

• Examine student behavior

– courses/instructors/degrees viewed vs. chosen

– engagement across different student populations

​

• Connect majors to career outcomes (data sources?)

​

• Personalization (deck of student cards)

– support exploration for intellectual breadth, disciplinary depth

– simplify registration process

– explore proximity to credentials

– Transcript of Future prototype?

​

• “Dig deeper” functionality for faculty, staff, administrators

– challenge: disaggregation by gender, race&ethnicity

​

• Institutionalizing the service

– role-based access to Dig Deeper functions

– design & implement effective, sustainable governance

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

2016 prototype “Dig Deeper” view

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

2016 prototype “Dig Deeper” view

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Biggest challenge is cultural –

Special thanks to:

​

Kushank Raghav

ART 2.0 Lead Developer, Office of Academic Innovation

​

Mike Wojan, Guanchao (Mark) Huang

User Experience Designers, Office of Academic Innovation

​

Amy Homkes-Hayes

Lead Innovation Advocate, Office of Academic Innovation

​

Lisa Emery

Senior Associate University Registrar

​

Keesa Johnson

Design Manager, Office of Academic Innovation

​

Kyle Schulz

Data Scientist, Office of Academic Innovation

​

+ the DIG Student Fellows and Interns

• the ART2.0 Steering Committee

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

finis

Evaluation items:

​

Self-assessed Learning: This course advanced my understanding of the subject matter.

Impact on Students: My interest in the subject has increased because of this course.

Course Organization (course focused): I knew what was expected of me in this course.

Course Organization (instructor focused): The instructor seemed well prepared for class meetings.

Instructor Clarity: The instructor explained material clearly.

Classroom Climate: The instructor treated students with respect.

Student Motivation: I had a strong desire to take this course.

Perceived Workload: As compared with other courses of equal credit, the workload for this course was…

​

For background information and more details, please find the committee reports here:

http://facultysenate.umich.edu/wp-content/uploads/sites/22/2015/02/SET-release-policies-SACUA.pdf

http://facultysenate.umich.edu/wp-content/uploads/sites/22/2015/02/CEIRC-Report-Interim-160213-1.pdf

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Contact information -

evrard@umich.edu

cteplovs@umich.edu

​

​

Coming soon:

Open ART site - virtualized cards for external users

​

Academic Reporting Tools @Michigan : Evrard/Teplovs : 16 Aug 2018 : OUCeL group

Amorphous human image credit:

Marco Lachmann-Anke

pixabay.com/McLac2000