A Picture is Worth a Thousand Words: Understanding Learners Through Visualization

Kara Foley, MOOC Assessment Fellow, University of Michigan Ross MBA and M.A. Educational Studies

Eejain Huang, Data Visualization Fellow, Combined Program in Education and Psychology, Statistics

Filip Jankovic, Research Assistant & Data Science Course Coordinator, MSE Industrial and Operations Engineering

Why Data Visualization?

In the virtual world, learners’ background and needs are assumed and indistinct, creating one of the biggest challenges for massive online courses: finding out who the learners are.

Here at the Digital Education and Innovation Lab (DEIL), we have devoted a lot of time and resources into developing effective initiatives to transform education. As an essential element in this process, collecting and interpreting learner feedback is at both the beginning and end of every MOOC course.

Screen shot of dataThree surveys were designed with iteration, assessment, research and marketing needs in mind. However, useful information is often hidden from decision makers because of the considerable amount of data gathered from these surveys.

To better explore this large amount of survey data, understand our learners and adapt courses according to their needs and feedback, DEIL created the DEIL Data Team to tackle these issues.

Data visualization was chosen as an ideal approach to interpret big data based on several reasons. First, data visualization can serve as a summarization, a guidance, a metaphor and eventually a gateway to understanding data. Second, data visualization can convey learning analytics in a digestible, actionable format for the audience. Like the old proverb says, “sometimes a picture is worth a thousand words.”

Who We Are and Our Journey

Our team has three members from diverse backgrounds who have contributed different expertise. Eejain Huang is a Ph.D. candidate working in Combined Program in Education and Psychology as well as a masters degree in statistics. Filip Jankovic is a U-M graduate with a Master of Science and Engineering degree in Industrial and Operations Engineering and is currently working with Dr. Christopher Brooks on the Applied Data Science with Python Coursera Specialization. Kara Foley is a recent graduate and has obtained her Master of Business Administration degree and Master of Arts degree in Educational Studies.

Eejain Huang, Filip Jankovic and Kara Foley in discussion while sitting around a table with laptops in front of large screen with a visualization of the globeKara was hired as a MOOC Assessment Fellow in January 2016 to evaluate which data visualization tool(s) would both be feasible to implement and highly automated. With input from additional faculty and staff across the University, we opted to reformat and clean the data using R and construct the visualizations in Tableau. Kara created initial prototypes of dashboards, and saw the challenges of data manipulation and advanced visualization design that needed to be addressed by a larger team with additional skills.

Eejain then began her full-time role in the summer of 2016 to clean the data and reshape it into a format that could be used to create Tableau visualizations. Much work had been spent on optimizing data structures to accelerate processing time, cleaning text responses with regular expressions and creating unique identifiers to link different surveys.

Finally, when the data was ready for creating and refining the visualizations using Tableau, Eejain created the Tableau dashboard. Filip joined our team in August of 2016 bringing with him his expertise in data science guiding best practices for visualization design. Filip helped ensure the visualizations represented the data accurately and effectively – an essential step when creating dashboards that allow decision makers to make well-informed decisions.

As dashboard revisions were made, we shared the visualizations with faculty and staff for their feedback. This involved hosting multiple meetings with different teams within the Office of Academic Innovation, and also conducting individual interviews with people who have specialized expertise in different steps of shaping a MOOC course including Dr. Christopher Brooks, Laura Elgas, Dr. Gautam Kaul, Noni Korf, David Lawrence-Lupton and Dr. Donald Peurach. Through this process we have learned about main priorities, key data points of interest and potential implications of this data.

What Did We Find and Some Insights

Currently, we are still experimenting on different ways to represent data and are seeking the best ways to distribute our findings. However, even at this transitioning stage, we think our findings may serve as an insightful guide for understanding learners and transforming MOOCs.

Illustration of a visualization

Although we recognize that our sample size is a small percentage of our learners, we discovered some trends that instructors may find useful for course design. While the majority of our respondents come from the United States and self-identified as Caucasian/ European / Russian, learners from developing countries constitute the next largest proportion of respondents. We noticed a difference in gender participation in our MOOCs, and in higher education degree attainment. Gender was a factor as well in self-ratings regarding confidence levels for language and use of technology. Regarding degree levels, these played an important role in influencing learners’ choice about participating in a course, their expectations about the course and their motivation for participating.

When looking at the dashboard, we found that many of the demographics and student feedback changed from course to course. This view can provide important insights to instructors to help them better understand their learners.

In conclusion, our findings showcased how diverse our learner community is and how different their needs can be. Nationality, ethnicity, gender, educational background and prior experience etc., all contribute to their expectations of, and experience during, the course. Thus, we hope our visualization can help course designers understand who their audience is, what they want, as well as how to make their content more accessible and eventually create a more targeted, personalized and engaged experience for every learner around the globe.

Faculty who are interested in learning more about MOOC survey data may reach out to the Office of Academic Innovation directly at academicinnovation@umich.edu

LSA and the March for Science

Reflections on the 2017 Coursera Partners Conference

James Park, Project Manager, Digital Learning Initiatives

Michael Skib, Instructional Media Specialist

James Park and Michael Skib were part of Academic Innovation’s contingent that attended the 2017 Coursera Partners Conference, March 29-31, hosted by the University of Colorado on the CU Boulder Campus. Below, they offer some reflections on their experiences at the conference.


James ParkReactions from James Park, Project Manager, Digital Learning Initiatives:

In collaboration with Stephanie Haley, Rebecca Quintana, and Steve Welsh, I created a poster, titled “How do MOOCs fit within the broader landscape of academic innovations at the University of Michigan?,” which I delivered at the poster session. It illustrates how MOOCs not only illuminate the work of the Digital Education and Innovation Lab (being the primary lab behind the creation of U-M’s MOOCs) but also emphasize the greater interconnectedness of Academic Innovation’s three labs and offer a window into the much broader, more diverse activities undertaken by Academic Innovation as a whole. That opportunity to share ideas and receive feedback, to ask and answer questions, and to learn from other attendees probably defined my conference experience. It also left me with some distinct impressions:

  • James park speaking to two individuals in front of a posterWe at Academic Innovation are deeply engaged in the work of digital teaching and learning not only at its core but, importantly, also at its (ever permeable) boundaries. Our collaboration with Coursera is emblematic of the MOOC work “of the present,” and we are also experimenting with ways that MOOC-type projects and non-MOOCs alike can be further harnessed inside and outside of university walls.
  • We are part of a community whose practices we are learning from as well as influencing.
  • We are extremely fortunate to have strong institutional support, generous resources, and our university leadership’s firm belief in our mission.

Though it was brief, the 2017 Coursera Partners Conference revealed some of the potential pathways forward for Coursera, U-M, and Academic Innovation, and it provided an opportunity for our team to reflect on what we’ve accomplished both on the Coursera platform and in the wider field of digital teaching and learning. And thankfully, a few of us even found a small window of time to enjoy Boulder’s beautiful mountains!


Michael SkibReactions from Michael Skib, Instructional Media Specialist:

As much as anything, I will remember the gorgeous CU Boulder campus. Flanked by the Flatirons in the foothills of the Rockies, the campus was built in the so-called “Tuscan Vernacular Revival” style. Seemingly frozen in time, it radiates a disarming Mediterranean warmth that paired well with Boulder’s expansive, cloudless sky. Perhaps that had something to do with why everybody seemed so happy to be there.

Selfie photo of James Park, Michael Skib and Noni Korf in front of a mountainThe atmosphere throughout the conference was warm and hospitable. Friends and colleagues reunited. Many of the people in this community already know each other — I felt somewhat like an outsider out at first, as my circle of professional acquaintances is more or less limited to the U-M community. Yet people seemed genuinely curious about my work and what I hoped to gain from attending this conference.

I was immediately taken aback by the size of this community. Stuck inside my U-M bubble, it is easy to forget how vast and multifaceted this community is; in attendance were people from more than 100 institutions, from 27 different countries. It seemed that Coursera’s entire staff was there as well, and many were eager to engage in conversation and hear about how we’ve been using their platform, and the things that we hope to see in the future. Despite the overwhelming number of attendees at the conference, I was struck by how much alignment I found between everyone I spoke to; we were all there out of a desire to make education work better, for all.

The first day of the conference featured a keynote presentation by Dr. Daphne Koller, co-founder and former president of Coursera. She took some time to celebrate a few significant milestones; Coursera now hosts more than 2000 courses, and more than 180 specializations. By their count, they’ve reached more than 25 million learners. They also host four fully online degree programs. Despite these successes, they are not content to stay the course, which isn’t entirely surprising. As influential and successful as Coursera has been, the online education space is becoming increasingly competitive. Throughout the conference, Coursera touted several major platform updates that will facilitate curriculum personalization and building career relevant skills. The general mismatch between the skills our workforce has and the skills the job market desires is a problem for many job seekers and for many growing industries. Though this is just one of several challenges higher education faces, it is one I believe Coursera can help to solve.

Later that day, after a blizzard of smaller, topical group sessions, I went to a presentation of posters created by about 20 different Coursera partners. Posters were hung in a densely packed atrium along with trays of vanishing hors d’oeuvres. This is when my initial feeling of being outsider began to disappear and give way to curiosity. It was fascinating to catch a glimpse of how other institutions view the digital education space, to converse with people from Stanford, Geneva, Alberta, Hong Kong, and elsewhere about things as broad as what it’s like to work with faculty, and as specific as the technical setup behind the Penn Modern Poetry MOOC’s live streaming office hours. I felt at ease and at home. I felt galvanized by the understanding of how immensely this field has grown in the three years I’ve worked for Academic Innovation. I felt gratitude toward Coursera for bringing us all together, to share and exchange knowledge; one institution operating in isolation cannot bring about the changes we hope to see in higher education.