Using Data to Inform Course Development and Assess Impact in a Hybrid Course

Cathy Hearn, Fall & Winter Course Advocate

Rebecca Quintana, Learning Experience Designer

This is the fifth blog post in a series on the School of Education’s 2018 Winter Cohort initiative. In the first post in this series, Professor Donald Peurach introduced the 2018 Winter Cohort: a learning experience in which University of Michigan graduate students collaborated with online learners from across the globe to complete content from the School of Education’s MicroMasters program in Leading Educational Innovation and Improvement.

In this post, Cathy Hearn and Rebecca Quintana discuss research efforts connected to the 2018 Winter Cohort Experience. Faculty and students from the School of Education and staff from the Office of Academic Innovation formed a community of inquiry around the Winter Cohort. The core course team collected weekly reflections from learners in order to improve the course experience week-by-week, to inform longer-term and larger-scope course development. The larger research team collected additional data including learner demographics, interviews, and activity logs with the goal of answering several questions about teaching and learning articulated by the team. A secondary goal was to develop conference papers and presentations in order to share their learning with others. The research team hopes this work serves as a use case for thoughtful and effective data collection and use from innovative educational programs.

Data collection to inform minor weekly changes

During the class sessions, we strove to create a culture of openness to course feedback.

We frequently held informal focus groups with students, soliciting opinions on topics such as the pace of work, study materials, enrichment opportunities, and office hours. Our learners also completed weekly surveys where they were asked to provide both ‘warm’ and ‘cool’ feedback. Reviewing these surveys weekly allowed us to introduce changes or additions on a week-by-week basis. For instance, learners expressed they were interested in receiving feedback from other teams. In response, we designed a lesson where students were able to do precisely that. We invited campus-based and online learners to take part in a live session in which teams exchanged work artefacts and worked through a feedback protocol. After the session, learners expressed an appreciation that we listened to—and acted upon—their input in real time.

Data collection to respond to driving research questions

In addition to these data sources, our research team intentionally recorded other aspects of the cohort experience. This took the form of learner and course staff interviews, pre- and post- course surveys, video and audio recordings, work artefacts, edX analytics, and more. Based on themes that emerged in our early analysis, we are now using these data sources to investigate social interaction, learner diversity, the effects of curating an online course, and ecologies of resources, drawing on Luckin’s (2010) framework. We have now assembled small groups of researchers around each topic and have submitted a proposal to present our findings at the 2019 American Educational Research Association (AERA) Conference. Through the preparation of this conference proposal, our fundamental aim is to share what we have learned with others setting out to do similar work.

Recommendations that stem from findings

Our attention to collecting and analyzing a rich body of data has been important to help us evaluate our course design and content. We feel many of the recommendations that stem from our findings will be useful for others designing and facilitating similar online and blended courses. Our recommendations include:

  • Developing a “Week Zero” or tutorial week aimed at acquainting learners with the platform and with course norms;
  • Highlighting the presence of course leaders and active course peers, which can be vital to learner persistence in an online learning setting;
  • Creating a simple course structure, and emphasizing important information in order to ensure that guidance is clear; and
  • Ensuring all learners feel represented and included in the course by providing additional examples and case studies from outside of U.S. or mainstream contexts.

Our community of inquiry centered around understanding what it means to support online and on-campus learners who are interacting together in an online environment. We also learned a lot about what it means to form and support a diverse group of researchers who are focused on a shared phenomenon of interest – a practice we hope to adopt in future investigations.

Read these other blog posts from the 2018 Winter Cohort of the University of Michigan’s Leading Educational Innovation and Improvement MicroMasters program:



Connect with the School of Education MicroMasters team @UMLeadEdHub


Luckin, R. (2010). Re-designing learning contexts: Technology-rich, learner-centred ecologies. New York: Routledge.

Don Peurach, Barry Fishman, and Tabbye Chavous Appointed to Academic Innovation Advisory Committee

Unique Online Course Aims to Prepare Students, Others for Work with Communities

Written by Nitya Gupta, Michigan News

ANN ARBOR—More than 5,000 University of Michigan students engage in community-based projects each year according to reporting alone, and most likely that number is higher when other informal, undocumented experiences are factored in.

Yet, those who help organize some of U-M’s engagement experiences have long been concerned that some students stepping into local and global communities have little to no training on how to work in such settings.

“Over the last decade, U-M has seen significant growth in the number of students engaged in community-based learning,” said Amy Conger, U-M associate vice provost and director of global engagement. “The interest was growing so quickly we were struggling to keep up with the need to help students thoughtfully prepare for community engagement.”

To help remedy this, experts from eight units across the university and a group of 15 students teamed up to create the Community Engagement: Collaborating for Change massive open online course (MOOC).

The team incorporated input from community stakeholders, students, staff and faculty across campus. Although other MOOCs on various aspects of community engagement exist, it is the first comprehensive course to have been developed by such an interdisciplinary team to prepare online learners for a broad range of engagement experiences.

“Our interdisciplinary team of staff and faculty brought together unique perspectives on community engagement from social work, information, engineering, international studies, student affairs, education and the humanities,” said Kelly Kowatch, director of engaged learning programs and adjunct lecturer at the U-M School of Information. “We also intentionally sought out significant input from student users and community members who are experts on these topics.”

Hosted on Michigan Online, developed in partnership with Academic Innovation, and funded by a Education Abroad Resource Grant from the Office of the Provost, the MOOC is targeted at early career students before they enter a community—either within the U.S. or abroad. Recent data shows 96 percent of U-M students have an engaged learning experience, many of which are in local or global settings that will take the students into places where they will work with community partners.

“We know that properly preparing the thousands of students who have community engagement experiences each year maximizes learning outcomes and community benefit,” said team member Danyelle Reynolds, assistant director for student learning and leadership at the Edward Ginsberg Center.

Unique online course aims to prepare students others for work with communities.

The aim of the MOOC is to teach the following key concepts and strategies:

  • Valuing community context and expertise
  • Understanding how social identities, power, and privilege impact your interactions
  • Approaches to collaborative leadership, such as listening effectively, resolving conflicts, and building mutually-beneficial partnerships
  • Reflecting on your work, and transitioning in and out of communities
  • Effectively managing community-engaged projects

“This MOOC is a way to address the scale of interest, and we hope to help students to approach future community based work with open-mindedness, humility and a willingness to learn,” Conger said.

For Monica Kim, one student who piloted the course over the summer, community engagement is in her background. She is a Ginsberg MacDonald Community Fellow, a three-year fellowship for community engagement granted through the U-M center that works in communities to advance social change. Although she’s already been exposed to some of the ideas in the course, Kim found herself learning new concepts and strategies.

“I think the most important thing I learned from the MOOC is to foster an understanding between community and whoever’s coming into the community to help it,” Kim said. “We need to make sure this relationship is ethical. ‘Ethical’ isn’t a word I used to think about in relation to community engagement, but it’s important because we often talk about privilege, power, balance and making sure we’re not exploiting people or exercising the savior complex.”

The course is designed to benefit anyone, regardless of prior experience in community engagement work, including U-M faculty and staff and volunteers outside the university who want to work more effectively with community organizations, including community-academic partnerships, social change projects, community service and learning, education and work abroad, traditional and community-based participatory research, nonprofit internships, public scholarship and civic performance.

Learners can also mix and match modules and activities within the course to personalize their learning experience. In full, the course takes approximately six weeks to complete, if spending about two to four hours a week on the material. It launches Sept. 21.


More information:

Advisory Committee Named to Inform Academic Innovation Strategy

James DeVaney, Associate Vice Provost for Academic Innovation

I’m very pleased to announce that the Office of Academic Innovation has appointed a new faculty advisory committee and will convene this group for the first time on Tuesday, September 18, 2018. We’re thrilled to benefit from the experience and expertise of this talented group of faculty leaders as we seek to broaden the impact of U-M’s mission by making our learning experiences accessible at scale and ultimately to create a peaceful, equitable and empowered society.

The Academic Innovation Advisory Committee is responsible for advising the the Office of Academic Innovation on policies, activities, and investments that enable the Office of Academic Innovation to fulfill its mission. This new faculty advisory committee will play a critical role in informing our thinking and strategy as we continue to foster a culture of innovation in learning at Michigan and maximize our positive impact on campus and beyond.


The Academic Innovation Advisory Committee members include:

Welcome Academic Innovation Advisory Committee!

  • Arun Agrawal, School for Environment and Sustainability, Professor, School for Environment and Sustainability
  • Barry Fishman, School of Information and School of Education, Professor of Information and Education.
  • Brenda Gunderson, College of Literature, Science and the Arts, Senior Lecturer in Statistics, College of Literature, Science, and the Arts
  • Caren Stalburg, Michigan Medicine, Associate Professor, Departments of Obstetrics & Gynecology and Learning Health Sciences
  • Don Peurach, School of Education, Associate Professor, Educational Policy, Leadership, and Innovation
  • Gautam Kaul, Ross School of Business, Professor of Finance & Fred M. Taylor Professor of Business Administration
  • Harley Etienne, Taubman College, Assistant Professor of Urban and Regional Planning
  • Liz Gerber, Gerald R. Ford School of Public Health, Jack L. Walker, Jr. Professor of Public Policy, Associate Dean for Research and Policy Engagement
  • Meg Duffy, College of Literature, Science and the Arts, Professor, Department of Ecology & Evolutionary Biology
  • Mika LaVaque-Manty, College of Literature, Science and the Arts, Arthur F. Thurnau Professor, Associate Professor of Political Science, Director of the LSA Honors Program
  • Tabbye Chavous, College of Literature, Science and the Arts and School of Education, Professor of Education and Psychology, Director of the National Center for Institutional Diversity (NCID)
  • Tawanna Dillahunt, School of Information and College of Engineering, Assistant Professor, School of Information (courtesy appointment in Electrical Engineering and Computer Science)
  • Vineet Kamat, College of Engineering, Professor of Civil and Environmental Engineering


Faculty innovators are contributing to U-M’s academic innovation network in many ways as we grow rapidly to serve our expanding global U-M community, enabling people to connect, engage and take control of their life-long learning journeys.

In addition to forming the new committee, the Office of Academic Innovation appointed two faculty innovators in residence in the Spring. Professor Barry Fishman and Professor Meghan Duffy are spending their sabbaticals at the Office of Academic Innovation. Both Fishman and Duffy will also serve on the Advisory Committee during their residence.

The Office of Academic Innovation also appointed two senior academic innovation fellows. Chris Brooks, Research Assistant Professor at the School of Information, continues to work closely with the Office of Academic Innovation, where he has been a key contributor and faculty innovator for several years, with a focus on learning analytics and the design of online learning experiences. Will Potter, a lecturer in the Department of English, was appointed senior academic innovation fellow in May and focuses on the intersections between academic innovation, digital storytelling, and public engagement including U-M’s Teach-Out series.

The Office of Academic Innovation is also facilitating several communities of practice which engage faculty from across academic units, with the aim of increasing adoption of innovative pedagogies and tools championed by faculty innovators. Current communities of practice focus on simulations, gameful learning, and public engagement, respectively.

Faculty continue to work closely with the Office of Academic Innovation across a number of initiatives. Currently, more than 90 faculty are working with Academic Innovation across more than 100 projects.

Michigan Online encourages learning lifestyle for U-M community through #HereToLearn

This article was originally posted on 9/4/2018 on Michigan News

Laurel Thomas, Michigan News

Why are you “here to learn?” This question posed to alumni and other University of Michigan learners by the Office of Academic Innovation is followed with a commitment: We’ll help you get there.

U-M’s 583,000 alumni worldwide now enjoy the lifetime benefit of Michigan Online learning opportunities at no cost. They will receive free certification to a growing list of more than 90 faculty-led online learning experiences.

This new alumni benefit expands the program rolled out in May, which also granted free certification options to faculty, staff and current students.

“I am proud that the University of Michigan is the first university to offer these certifications at no cost to alumni from all three of our campuses,” U-M President Mark Schlissel said. “Michigan Online extends our commitment as a public university to lead the way in increasing access to knowledge through academic innovation.”

The Office of Academic Innovation partners with schools and colleges across the university to create learning opportunities led by U-M’s faculty and bring them to the masses via Michigan Online. A portfolio of courses in a wide range of disciplines has led to nearly 6.9 million enrollments worldwide.

“We created Michigan Online to broaden the impact of U-M’s mission to develop leaders and citizens who challenge the present and enrich the future,” said James DeVaney, associate vice provost for academic innovation. “By inviting alumni to take advantage of this new benefit and expanding open access to learners on campus and around the world, we are deepening our commitment to quality learning at scale. Through Michigan Online, we aim to better understand and meet the lifelong and lifewide learning needs of our growing U-M community.”

While access to Michigan Online courses via U-M’s partner platforms generally are free to all learners, the certificates carry price tags. These could run approximately $500 for a six-month series of courses. These fees are now waived for all current students, staff, faculty and alumni from the three campuses.

A certificate confirms successful completion of a course, or collection of courses, and can be included in a resume, mentioned in a job application or shared via social media, such as LinkedIn. Certificates are not official professional or academic credits.

In today’s complex world learning beyond the traditional degree has never been more important, leaders say. Through a newly released video, Michigan Online is engaging alumni and all learners to reflect on their unique reasons for learning by asking that question, “Why are you ‘here to learn?’” The learners are being encouraged to share answers on social media using #HereToLearn and #MichiganOnline.

Silhouette of a young woman walking into a building with the text #HereToLearn on top.Michelle Li, a participant in School of Education Michigan Online courses, has an answer: “Because there is so much to learn and the more I learn, the more I am well positioned to productively address the persistent problems in K-12 education. Because we expect young people to learn in school every day, and as adults, we need to hold ourselves to the same standard.”

Li served as a public high school English teacher in the Boston area for a decade and a half before earning a master’s degree in program evaluation and improvement research at U-M. She now is a facilitator and continuous improvement coach at the Center for Leadership and Educational Equity in Rhode Island, an organization that provides leaders with professional learning and support to create equitable outcomes for students in schools.

U-M Information and Technology Services employee Ken Caldwell has taken several Michigan Online courses, partly for enrichment but also to enhance skills for his job as a marketing and communications professional, and to prepare to be a master’s student in the School of Information.

Among several massive open online courses (MOOCs), he took a five-course Python for Everybody series prior to enrolling in the Master of Science in Information program.

“I took that as a way to sample the type of curriculum UMSI offers,” said Caldwell, a marketing communication specialist.

As for why he is here to learn: “I’m really interested in helping people solve problems with technology. Ensuring information flows smoothly is key.”

Another Michigan Online student taking courses in education is both an alumna and U-M staff member. Evelyn Ventola, field manager for the Longitudinal Study of American Youth at the U-M Center for Political Studies, said the ability to study at a pace that fit around work already made MOOC learning a plus.

“It’s wonderful to see that U-M is taking such an interest in the continued education of its faculty, staff, students and alumni,” she said. “Having free access to these courses has definitely encouraged me to explore the offerings, as I normally find the cost of these programs to be a barrier to participation. I am ‘here to learn’ because I know that education is the surest way to better oneself.”