Now You See Me, Now You Don’t: Making Searchability A Priority

Adam Levick, Data Scientist
@adlevick

At Academic Innovation, we build at scale. Whether we are building MOOCs, digital tools, or learning analytics, a big part of success is being able to sustainably bring our innovations to great numbers of people. This means that having a team devoted to understanding the value of what we do, and then being able to broadly promote that value is key to our success. That team is known as “Marketing.” This is the first of a series of posts that will describe the kind of work the Marketing team does to help ensure the success of our initiatives.

Approaching Solutions at Scale

What often comes as an early surprise for those that work with us, is that building for scale requires very different approaches than those used in the residential setting. These differences are especially pronounced in how promotion is approached. This can best be defined as a need to shift the balance from “readability” to “searchability” as the primary consideration for titling and describing our innovations. We define something with high “readability” as something that is enjoyable to read, whereas something with high “searchability” is something that has the ability to be found in the first place.

Finding Massive Online Open Courses

In the residential setting, when students are in the process of choosing courses they generally find those courses in one centralized system where they are able to easily browse every course offered in a subject area. Courses with titles and descriptions that have high “readability” for the student (based on their interests) will drive them to enroll in that course. This is extremely different from an online setting, where the number of potential options are amazingly more diverse, complex, and distributed. What may be an engaging and readable title in a residential listing may not connect to the types of search terms people are most commonly using to find related content (whether that be a Wikipedia page, new articles, or other online courses). For this reason, when we create MOOCs, we approach titles and descriptions with the primary goal of better matching the words and phrases people are searching for when looking for related content. While “readability” is important and still a major consideration, in the online setting, sacrificing “searchability” means the learner may never find the content in the first place!

Getting In Touch With Potential Learners

A screenshot of Google Trends resultsAfter identifying a course that could potentially benefit from better aligning title/description with search habits, we do a close reading of that course’s proposal to identify the different types of language used to describe content and to ascertain the key values learners gain from interacting with that content. Once we have an understanding of the kinds of words and word pairs used to describe the course, we use an online tool, Google Trends, to identify which alternatives are most aligned with the searches people make on Google. This allows us to build a pool of high value words that can then be put together to form potential titles and descriptions. We can then take 3 or 4 of these titles, and test them in an online survey with 200-300 potential learners to identify whether interest in specific titling is definitively more interesting to all or different segments (gender, age, or location) of learners. The end result is a simple recommendation on which title will produce the best “searchability” for your course, as well as a pool of high value words to use in descriptions and for promotion in the future.

“Go Gameful” this Summer at the Gameful Course Design Summer Institute

Rachel Niemer, Director of the Gameful Learning Lab
@rkniemer

The tradition began sometime in graduate school: every year some well-meaning aunt or uncle (or student or grandparent or random person I met at a party) would assume that I have the summer off, since I study/teach/work at a University. Certainly, I am not the only one to experience this phenomenon. For a time, the questions grated on me; “I wish,” I’d say.

Even within University of Michigan, I hear an individual from one department ask someone from another unit, “Is summer slower for you?” Responses generally sound like, “It’s different, but we’re still busy.” Of course both of those clauses are true, but I have learned, over time, to relish the different kinds of work I get to do during the summer.

As an instructor (and as a learning experience designer), the focus during the academic year has to be on supporting your students (or clients) through their learning journeys: observing their efforts, providing feedback, offering alternative explanations, helping them identify next steps, keeping the momentum going in the right direction. It is deeply rewarding work – I feel my successes and stumbles acutely in my chest during the academic year: A warm glow when a student/client crosses a threshold of understanding, an ache when I can’t figure out how to reframe a topic so that they “get it.”

The summer work, though, feels different. If the academic year is felt in my chest, summer feels like butterflies in my stomach. Summer work is all about digging deep, designing new things and building anticipation for the big reveal of what we’ve built. For some academics, energy is spent designing new hypotheses and experiments, others are working on new arguments and manuscripts. For those of us in the Gameful Learning Lab, this summer is about digging into a new initiative that will give instructors the space, guidance and foundation to (re)design a gameful learning experience for their students: a course design summer institute called “Going Gameful: Levelup your Teaching.”

On July 24 and 25, 2017, instructors from higher education and K-12 are invited to Ann Arbor for the opportunity to dive into gameful course design. What’s gameful course design, you ask? Gameful course design is an approach to creating learning experiences that draws from theories of motivation, the learning sciences and the principles of game design. Every gameful course has a unique design, but they all have some traits in common:

  • Students are offered choices in how they approach and demonstrate their learning,
  • The grading scheme frame is flipped from the traditional percentage-based system to a points-based system that builds from 0, and
  • One particular course grade distribution is not enforced, i.e. grades aren’t given on a “curve.”

We have planned a highly-interactive set of workshops for participants to apply gameful principles to their own course designs. Attendees will hear from, and talk to, students who have taken gameful courses, the experts in the Gameful Learning Lab and the burgeoning community of instructors shaping our understanding of, and experimentation with, gameful course designs. Dr. Barry Fishman, Alfred F. Thurnau Professor at the University of Michigan School of Information and School of Education, will provide the opening keynote on the first day of the institute. Paul Darvasi, educator, writer and game designer from Royal St. George’s College and York University, will describe his “ward simulation,” where his high school English students role-play the characters from One Flew Over the Cuckoo’s Nest for an extended period of time.

GradeCraftIn addition to giving plenty of time for participants to dig deep into their gameful course designs, and to reflect on how gameful principles align with their own teaching philosophy, workshop leaders will expose participants to GradeCraft, a learning management system built to facilitate the transparency and unique grading structures of gameful courses. This summer, GradeCraft launches as a product that is available to any instructor to manage their courses.

Our hope is that the experience of the Institute reflects what it feels like to be an academic/educator during the summer: more focused and more playful than during the academic year. One of the components of the institute that particularly gives me “butterflies” (both butterflies of excitement and of nervousness) is our evening activity: a scavenger hunt around U-M and Ann Arbor. It is a new kind of event for me to implement, but I’ve really enjoyed similar ones at other conferences. Plus, I’m excited to show off Ann Arbor!

Registration for the institute closes on July 10. We have a limited number of scholarships available for U-M graduate students who would like to participate. The scholarship application is due by June 30. Go gameful, and join us on July 24th and 25th!

Sowing Seeds with U-M Faculty and Staff: DIG at Enriching Scholarship

Amy Homkes-Hayes, Lead Innovation Advocate
@amynhayes

In Spring of every year as we bid adieu to students for their exciting summer plans, I look forward to the month of May for several reasons including the generally better Michigan weather (freak cold snaps withstanding), and, of course, Enriching Scholarship. Having worked at Michigan since 2008, I’ve had the opportunity to participate in and lead several Enriching Scholarship sessions, and I can say with certainty that the Digital Innovation Greenhouse (DIG) sessions I participated in this May were some of my favorites.

Filling with Soil

In DIG, we have several digital education tools that U-M faculty and staff have varying familiarity of across campus. We use opportunities like Enriching Scholarship to increase interest in, and understanding of, our tools, and equally the approach we take in DIG on how we design and implement education software. Putting it another way, we want faculty and staff to be both knowledgeable of the rich soil we use to grow DIG seeds (projects), and the seeds themselves. During a DIG-wide panel where faculty and staff from the College of Engineering and College of Literature, Science, and the Arts talked about their experiences using DIG tools like ECoach and Gradecraft, the conversation spanned several topics including why faculty and staff decided to use DIG tools, the differences they’ve observed in their classes and with their students since using our tools, and their experiences with DIG staff.

Sprinkling Seeds

robbie routenbergSo, what did faculty and staff have to say about their adoption of DIG projects? Well, it turns out quite a lot. When discussing her experience with ECoach, College of Engineering Professor Dr. Mary Lou Dorf discussed her early adoption of the tool in her EECS 183: Elementary Programming Concepts course. She highlighted the benefits of ECoach making information more transparent for her students, and valuing the motivational and personalized messages delivered to her students at key times in the semester (such as after an exam). robbie routenberg, Director of the Global Scholars Program in the College of Literature, Science, and the Arts explained that of the several reasons why they use Gradecraft it has helped, “students feel more in control of their grades.” Indeed, given that courses that use Gradecraft offer students more assignment choices than traditional classes, students have more autonomy over how and when they earn points. College of Literature, Science, and the Arts Professor Dr. Mika LaVaque-Manty said students using Gradecraft are more apt to take risks when making assignment decisions because no matter what, “they earn something and they learn something.” In another Enriching Scholarship session, Ford School of Public Policy Professor Dr. Elisabeth R. Gerber showcased the role-playing simulation tool Policymaker, and talked about how her students increased their engaged learning by embracing aspects of the tool like the Newsfeed (a Twitter-esque like feature where students write and respond to statements throughout the simulation as the character they are playing, and which are broadcast to the rest of the group for their consumption or response).

Watering Seedlings

Laura Alford and Mika LaVaque-MantyOf course, we also used our Enriching Scholarship sessions to hear from faculty and staff who work with the Office of Academic Innovation to share how well we are nurturing the soil. In other words, what kind of gains do faculty and staff experience when working with DIG? The theme that stood out to me the most was echoed by Dr. LaVaque-Manty when he said, “we are seeing a lot of interest in these tools because (DIG) makes everything a lot easier.” His words ring true for me as a member of the DIG team striving to aid faculty to seamlessly integrate DIG tools in their classes.

Enriching Scholarship presented a ripe opportunity for the DIG team and our faculty and staff partners to talk about their experiences, and while I appreciate everything that was shared the comments that stuck with me the most were College of Engineering Professor Dr. Laura Alford’s remark that, “ECoach language makes so much sense to students” followed by Dr. Dorf’s statement that , “The students are happy. They love ECoach!” This excellent feedback will continue to inspire us as we work on our tools in anticipation of students returning in September.

Expanding the University of Michigan Teach-Out Series

This article was originally posted on 5/31/2017 on the edX Partner Portal

James DeVaney, Associate Vice Provost for Academic Innovation
@devaneygoblue

In September 2016, University of Michigan’s President Mark Schlissel charged the Office of Academic Innovation and the U-M community to launch a set of rich and interconnected experiments to explore the future of education at the University of Michigan. One of many ideas that surfaced through the President’s Academic Innovation Initiative was the U-M Teach-Out Series which was conceptualized in January and February and launched in March 2017.

University of Michigan Teach-Out SeriesWith the Teach-Out, we combine the global reach of MOOCs with a model designed to explore new approaches to just-in-time teaching and learning. Yet the Teach-Out is very much connected to the ongoing Michigan saga. Teach-outs are modeled after the historic U-M teach-ins, which started fifty-two years earlier in March of 1965 in response to military action in Vietnam. Faculty considering how to best respond to President Lyndon Johnson’s escalation of troops into the country created a marathon educational event designed to activate public concern and elevate public discourse. Within a year, teach-ins were conducted at 35 other college campuses, and a few years later the model inspired the first Earth Day.

Throughout U-M’s 200-year history, we’ve leveraged academic innovation to expand our community and realize our public mission. MOOCs have provided new ways to think about how to best disseminate our broad portfolio of scholarly work with the world and how to connect our intellectual power with lifelong learners and decision-makers across society. MOOCs have helped to reframe several important conversations around teaching and learning, knowledge dissemination, openness, and inclusive learning environments, to name a few. We see significant room for further innovation as we continue to embrace compassion, openness, personalization, and inclusivity in higher education.

We believe the Teach-Out will accelerate our ability to bring new individuals and communities into unprecedentedly open and inclusive learning environments and offers a concrete contribution to the design of the compassionate public square for the information age. Through the Teach-Out model we will continue to explore opportunities to make the great public research university even more open in the future. We will unbundle our expertise from the disciplines and rebundle around problems that demand our attention. We are now seeing the MOOC evolve beyond minimum viable product and can point to a sizeable wave of second order experiments that move us closer to a future where anyone committed to lifelong learning and listening can fully participate.

We held our first four Teach-Outs between March 31st and May 14th and have released a new call for proposals to solicit ideas from instructional teams to create opportunities for learners around the world to come together with our campus community in conversation on topics of widespread interest. We have seen significant support for this model across disciplines and expect the next wave to reflect an even broader range of academic expertise and experience with learning technologies.

As with all pilots, we’re thinking actively about measures of success. We’re looking at relatively simple metrics like reach and participation, learner satisfaction, and change in understanding. We’re collecting different kinds of information as well. We are exploring the extent to which different Teach-Out approaches help us to effectively share our broad portfolio of scholarly of work. We’re looking at faculty involvement and the benefits and challenges of team instruction and multidisciplinary teaching teams. We’re thinking about different ways to measure our ability to connect U-M’s broad intellectual power to the problems most import to society. We’re looking at engagement from other institutions and our ability to attract new communities of learners. We’re exploring ways to capture whether learners are exposed to new ideas and perspectives. We’re interested in the relevance of the Teach-Out model to decision-makers at all levels of society. And we’re capturing the different ways that these global community learning events can become resources for different learners in different learning environments.

Micro-Credentials and College Admissions: Enhancing Access and Supporting Learning

This article was originally posted on 5/24/2017 on Inside Higher Ed

James DeVaney, Associate Vice Provost for Academic Innovation
@devaneygoblue

Given the frequency with which I am asked about the summertime pace on university campuses, I suspect there are many who imagine the gears of innovation grinding to a halt. Sorry to disappoint. There is just a bit of a break from the norm but summertime in Ann Arbor is anything but monotonous.

As we imagine again the future model of teaching and learning at a public research university we often create opportunities to collaborate with innovators from other institutions, relax constraints to encourage new ideas, and work together to design a better future. This summer we’ll support collaborative and open research when hosting the Learning Analytics Summer Institute and we’ll go gameful in launching the first Gameful Course Design Summer Institute.

But first up this summer was an interactive workshop focused on micro-credentials and college admissions. Barry Fishman (@BarryFishman) is Professor of Information and Professor of Education at the University of Michigan. Stephanie Teasley (@stephteasley) is Research Professor at the School of Information at the University of Michigan. Last week, Barry and Stephanie hosted a workshop in Ann Arbor that pulled together leaders from more than a dozen institutions and organizations that are actively exploring the potential of micro-credentials.

Barry and Stephanie graciously agreed to answer my questions about the direction of micro-credentials, aka badges, in college admissions as we work toward an elegant design that effectively considers access, participation, and learning.

Question 1: Last week you hosted a workshop at the University of Michigan with support from the National Science Foundation (NSF) that focused on micro-credentials and college readiness. Can you give us the backstory? What was your goal for the workshop? What problems are you trying to address?

This workshop was born out of an interest in exploring new pathways to and through higher education. Our work over the past decade has explored various ways to expand participation and success in higher education, through a range of means. Micro-credentials — also known as digital badges — struck us as a promising tool for recognizing student learning in ways that are not currently captured by more typical measures such as standardized tests and high school GPAs. We were aware of the excellent work being done by colleagues in the Chicago City of Learning and Mouse in NYC to support and record diverse learners’ engagement in a range of informal and formal learning activities using badges, and wanted to see how their work could be leveraged along with efforts at broadening access to higher education. Because of a general orientation towards academic innovation and diversity, equity, and inclusion, the University Michigan has already begun to explore new pathways into the university. The time seemed right to explore this topic further on our own campus.

Question 2: How would you characterize the micro-credentials and badging landscape today? What are some of the most important questions people are working through?

Many if not most of the required technical components are in place for a robust badging infrastructure… but we’re still lacking strong use cases and a coordinated strategy for implementation. This is a common challenge in education (and other fields), where people often focus on isolated elements of a challenging problem, but don’t take on the harder task of building workable solutions or the underlying social and technical infrastructures needed for complex real-world systems. There is a near-final second-generation standard for digital badges from IMS Global, along with a (small) number of groups and companies working on platforms and infrastructure for awarding and sharing badges. Groups like Mouse and Chicago City of Learning are developing practices for awarding and aggregating badges for K-12 learners. And several consortia are thinking about how badges can be used at the college level to indicate workforce readiness. But how these various efforts translate into coordinated broader application remains unknown.

Our workshop focused on the more limited – but still big – domain of college and university admissions. A real challenge for admissions is to develop trustworthy measures of student potential that can be reviewed in a relatively short period of time. Various issues remain challenging: measurement validity (How do we know what a badge measures?), endorsement (Who vouches for the validity of badges?) and scaling (Can badges help us evaluate thousands of applications?).

Question 3: To really explore the connections between micro-credentials and college readiness requires insight from individuals representing a range of roles and organizations. Can you tell us about the workshop participants how you thought about assembling the right group for this particular conversation?

Our goal was to bring together three key stakeholders: organizations that award badges to students, college admissions officers, and people with expertise in educational assessment and innovation. Badge awarding organizations were represented by Mouse and the Chicago City of Learning. College admissions was represented by the University of Michigan Office of Undergraduate Admission and the Penny Stamps School of Art and Design both at Michigan, and Kalamazoo College Admissions, as well as participants from the Parsons School of Design, Eastern Michigan University, the Stevens Institute of Technology, and others. The third group was made up of practitioners, university faculty, and innovation developers. We wanted to get people thinking about opportunities and challenges in both the college admissions space and the digital badge space. In the workshop, everybody was “surprised” by something – some new information or perspective – from a space outside of their normal work.

Question 4: Can you tell us about some of the key takeaways from the workshop? What do you hope to see happen next? Any bold predictions about the future?

The workshop was valuable for bringing important issues to the surface, and for informing different audiences about the nature of challenges faced by the people working in these areas. The goal of a workshop like this is not necessarily to find solutions to problems, but rather to help frame an agenda for making progress in the future. So one big takeaway from our workshop will be recommendations to the National Science Foundation (and others) about which problems in the badging-and-college-admission space need particular attention. Our workshop closed with the participants generating and prioritizing issues that need further focus. While we are still sorting through our notes, some issues that rose to the surface include: Developing an infrastructure that simultaneously meets the needs of college admissions organizations, badge awarding organizations, and learners; Creating systems for validating and endorsing learning from badges to support scalability and trustworthiness; and ensuring that newly created systems do not replicate or inadvertently amplify current inequities in the education system.

Question 5: How does this effort align with your own work as faculty at the University of Michigan? How does it connect to the President’s Academic Innovation Initiative?

We see micro-credentials — also known as digital badges — as a key component of the larger conversation about how we might re-think what teaching and learning looks like at a public research university. The past five years have seen explorations in many areas at Michigan, from MOOCs to learning analytics to personalized coaching, from rethinking the nature of the academic transcript to re-framing classroom assessment with gameful learning. A lot of these efforts are either explicitly or implicitly oriented towards broadening access, enhancing participation, and supporting student success. We believe that digital badges can play a role in all three of these areas. For example, badges can be used to recognize learning and accomplishment. They might also serve as evidence of a student’s readiness for study at a place like Michigan, adding information that goes beyond what we can learn from traditional measures like test scores or GPA. Digital badges can even be used to help identify students with particular interests earlier in their academic careers, so that those students might be steered towards activities that will open up further opportunities and help identify a possible set of higher ed institutions that may best serve their interests and aspirations.

Question 6: What advice do you have for people across the higher ed landscape who are just beginning to think about the potential for micro-credentials? How should they get started? What should they know about the conversation and experimentation already underway?

We encourage people to think broadly about their goals for enhancing access to and supporting learning across higher education. If you treat badges as an “add on” or purely extrinsic incentive or reward for some student behavior, it’s unlikely that they will make a real difference for learners… or for your organization. College admissions is built on an elaborate infrastructure that shapes not just the application process, but K-12 education and how students engage with learning, especially in secondary schools. But that infrastructure came from somewhere, and took time to develop into its’ current form. To make meaningful or lasting change, we need to engage with this infrastructure and design for uses of badges that have meaning and offer equitable access for everyone involved. As with most important challenges in education, there is no “one thing” that leads to real change or improvement. Those who seek to use digital badges to represent learning — at any educational level or for any purpose — will benefit most from partnering with others who are exploring in this domain, and working together to build a new infrastructure for learning supported by badges.

Time to Bust The Myth of the Slow University and the Fast Edtech Company

This article was originally posted on 5/9/2017 on Inside Higher Ed

James DeVaney, Associate Vice Provost for Academic Innovation
@devaneygoblue

Since Josh Kim stayed home this year, I feel obliged to share some thoughts from the ASU-GSV Summit in Salt Lake City. As the Summit rounds third and heads into its final day, I’m eager to attempt two things. I’m compelled to highlight and dispel a myth that I believe is limiting the collective potential of the wide range of actors seeking positive change in higher education: the myth that universities are slow to embrace change and edtech companies are always agile. I’m also inspired to share a quick and playful guide to establishing university-edtech partnerships that exceed expectations. To achieve the latter requires some digression about the difference between partnerships and vendor relationships.

Let me bring you to the conference for a moment and specifically to a panel session that I participated in on Monday morning. The panel centered on how different academic innovation leaders are responding to unprecedented levels of change and tech-driven disruption. The conference organizers provocatively titled our session, “Change Agents or Kamikaze Pilots? Can Higher Ed innovators survive the ongoing wave of market disruption?”

Like you, my first reaction to the title was to roll my eyes. Are we doing this again? Haven’t we sufficiently exhausted the disruption lexicon and all its awkward metaphors? Couldn’t we approach change, which is constant, with more level-headed observation. When the Green Bay Packers started the 2014 football season with a disappointing record of 1-2, there was panic across the Packer fan base. Quarterback Aaron Rodgers responded, “Five letters here just for everybody out there in Packer-land: R-E-L-A-X”. It might be time to share the same advice across the higher education sector. This doesn’t mean put your head down and ignore the world around you. I am suggesting that universities are well equipped to navigate change.

In a conference event with far more edtech company representatives and investors than higher education leaders, I suspect just about everyone in our audience heard change agent as the obvious positive choice and kamikaze, with all its World War II imagery, as the more dire and inevitable destiny of university leaders trying to keep up with Silicon Valley. To borrow from The Princess Bride and the great Inigo Montoya, “You keep using that word. I do not think it means what you think it means.”

It turns out that kamikaze translates to ‘divine-wind’. The Japanese survived multiple invasion attempts by Genghis Khan’s grandson in the thirteenth century. Both invasion attempts were thwarted by typhoons that the Japanese believed to be sent as protection by the gods. They called these winds kamikaze. I didn’t choose this metaphor for higher education but I can certainly find relevance in the idea that universities are creating new strategies that shift the current in more favorable directions. Afterall, strategic decision-making in higher education looks and feels far more like sailing a ship toward a destination than laying out a precise turn by turn driving (road)map.

The question of change agent or kamikaze, the way it was intended, reflects a harmful and pervasive belief that runs through this conference year over year, and across a higher education sector now populated with a diverse range of organizational actors and influencers. It’s time to dispel the myth of the slow university and the fast startup. At best it’s not accurate. More alarmingly, this perception is standing in the way of universities and edtech companies establishing the best kinds of relationships and fully realizing potential. This should matter equally to diverse university communities and edtech founders, employees, and investors. To paraphrase Mark Maples in his keynote comments at the Summit this morning, we are operating in world shaped by Moore’s Law and Metcalf’s Law. There is an opportunity to bring abundance to the world if we don’t get in the way of our own progress.

It turns out our understanding of each other matters a great deal if we seek to create a future where everyone can participate. Tapping into our collective exponential potential is only possible if we establish partnerships that meet or exceed our expectations. Good news, there’s a way to do this.

Given the traditional role of universities in our society, many are quick to assume that universities always operate slowly and that edtech startups always operate with great agility. “Always” is a funny word. Is one organization always slow and the other always fast or is it more complicated? To follow this stereotyping logic would be to assert that the university is always methodical and the startup always impulsive. This too is flawed and distorts a reality where we are all on a continuum of managing the explore-exploit tradeoff. Simply put, we explore by collecting information and exploit when we leverage that information to take action. Some think about this in terms of the speed with which we move from insight to application. Different organizations manage this tradeoff differently. What we often forget is that organizations also manage this tradeoff differently depending on the context.

When we gather at events like the ASU-GSV Summit I’m reminded of our flawed assumptions about each other. One entrepreneur gave a pitch and suggested that we are facing a global listening epidemic. This may be true in the current typhoon of university-edtech ecosystem as well. Research universities are communities bound together by a commitment to the discovery of what’s next. There is nothing slow about the incredible, and sustained, pace of innovation that has come from our great universities. At the same time, startups are introducing a breathtaking range of approaches to areas like AR, big data, blockchain, and adaptive learning, to name a few. Many of these organizations take great pride and care in their approaches to designing new products.

If we aren’t careful, we are subject to folly when assuming how the other handles the explore-exploit dilemma. We have more in common than we think and the stereotypes often do us a disservice. In reality, whether we know it or not, we are all participating in a massively distributed change management exercise to reshape the future of education. Change agent responsibilities are distributed and the collective action across this distributed network of actors is providing the divine wind that propels us forward.

New campus units like U-M’s Office of Academic Innovation, and others represented on the panel in Salt Lake City, are bringing this more complicated story to the surface. How do we optimize the explore-exploit tradeoff in order to drive academic R&D at the speed of Moore’s Law without deviating from an essentially evidenced-based culture fundamental to a vibrant research community. The best edtech players are leveraging information at breakneck speed without shortchanging proven methods for information collection. Together, we’re sharing complementary perspectives and strategies for navigating the explore-exploit tradeoff. When do we know enough to act? What should we observe when we do? Repeat.

The myth of the slow moving university and the fast start up is dangerously oversimplified and deeply flawed. We are assuming that we need to choose between ‘look then leap’ and the scientific method. I would argue that we are forcing an unnecessary compromise. A true partnership will bring organizational strengths together in a way that can’t be accomplished through more transaction oriented vendor relationships.

After the panel session and through the rest of the conference, talented individuals representing some of the world’s best edtech startups would ask me how they can establish a relationship with the University of Michigan and similar institutions. My playful but sincere reply: do you fancy yourself a partner or a vendor?

In my view, there are fundamental differences. If I were to establish an edtech startup or evaluate a new investment, this would be a primary inquiry in my due diligence.

Unfortunately, as with “luxury” apartments, and “gourmet” coffee, there is nothing stopping a vendor from euphemistically repositioning itself as a university “partner”. But ‘partnership’ should mean something. We’re in this together. We have a shared view of what ‘this’ is. Our interests and values are sufficiently aligned. We’re comfortable with the level of transparency. Data flows are seamless. There is real opportunity to co-create a roadmap. We’re in sync most of the time and have reasonable methods for thinking through disagreements. Risk is shared appropriately. We understand what each of us bring to the partnership. We have a shared view of how to manage tension in the explore-exploit tradeoff.

Anyone who has attended the ASU-GSV Summit this year or in the past knows that three days are filled with many rapid-fire conversations. With more time, I might have followed my playful question (partner or vendor), with a few questions to help determine what kind of university relationship an edtech company wants to have. To be clear, there is nothing wrong with establishing a vendor relationship. That said, there may be a lot wrong with establishing a vendor relationship when the edtech company and the institution think they are establishing a partnership.

Here are three non-exhaustive and playful questions for edtech companies to consider in looking at current relationships with universities and in exploring new relationships designed to meet or exceed expectations.

Question 1: Too Many Notes? Are our values and interests aligned?

Research universities are flush with expertise. Really deep expertise. In my humble opinion, these institutions are essential to a functioning democracy. In fairness to its critics, universities have work to do to make this rich expertise more readily accessible to the public and to reassert the vital importance of universities to society. Further, while universities are highly differentiated, you wouldn’t know it from our mission statements and campus tours. There is work to do to make complex adaptive systems more readily understood.

Some say universities are very good at understanding their supply side: what do we know and how to we exploit this knowledge to push the boundaries of discovery? And less expert in understanding the demand side: what do learners need now and next to thrive in a changing global economy? Enter a wide range of edtech problem solvers and market makers. In many cases, new entrants to the higher education space have brought urgency to questions about how universities can and should serve lifelong learners facing a rapidly changing future of work.

Here our values are aligned around developing learner centric models, transforming the future of higher education, and finding models to personalize learning at scale. We all want to build a future where everyone can participate whether our primary motivations hinge on learning outcomes or market inclusion. But the devil is in the details. Are we in sync on priorities, pace, and trade-offs between scale and quality? Is there sufficient understanding of a university’s intellectual diversity? Have we reconciled the long-term incentives of universities with the pressures felt by organizations with quarterly expectations?

Actor Todd Louiso’s character (Chad the nanny), in the movie Jerry Maguire provides a good illustration of the fears that are shared by more than a few in higher ed. In recommending a jazz cassette to a lovestruck Maguire, Chad the nanny couldn’t stifle a cynical digression, “This…is Miles Davis and John Coltrane. Stockholm. 1963. Two masters of freedom, playing in a time before their art was corrupted by a zillion cocktail lounge performers who destroyed the legacy of the only American art-form – jazz.”

Perhaps many of us in college towns find ourselves responding to rapid change with soapbox soliloquy and assuming the role of overprotective custodian of university tradition. It’s not quite right or fair to believe that the values of edtech companies are always at odds with our own, or to succumb to a fear of cocktail lounge lectures as our inevitable academic dystopia. But in fairness to our colleagues, many edtech companies have fallen short in developing a full understanding of the values and interests that drive university communities bound together by a commitment to discovery.

It is true that universities have incredible opportunities to expand reach and improve connection and that many opportunities are perishable. But the requisite experimentation must be in accordance with our values and strengths. Some institutions have to step forward before we can point to best practices. How else do best practices emerge?
Strong partnerships, unlike more transactional vendor relationships, require deep understanding of values. Just as universities seek to understand new kinds of edtech partners, new edtech partners should seek to understand the complexity of universities that are at one and the same time powerfully conservative embodiments of cultural heritage and massively disruptive creators of new ideas, technologies and forms of expression. When thinking about pace we can’t think in binary terms. Edtech partners aren’t always fast and universities slow. It depends and sometimes we reverse roles. But only in a strong partnership do we have the mechanisms to maintain focus in our collaboration and flexibility to allow our respective strengths come to the surface in the form of benefits to learners and society.

All too often universities find themselves in conflict with edtech companies who push for simplicity and scalability without fully understanding why universities hold tight to a certain way of doing things. For edtechs, this must feel like entrenched bureaucracy. For institutions, we hear “sirens of a zillion cocktail lounge performers.” Neither is quite right, but good partners find a way to understand different perspectives and approaches to managing the explore-exploit tradeoff and develop a shared view with shared benefits.

The ways in which this tends to go wrong is perhaps depicted best in one of my all-time favorite movie scenes. There are partnerships that will help universities to broaden their communities through creative pedagogical, technical, and programmatic innovation. Unfortunately, sometimes our negotiations with edtech companies feel more like Mozart’s exchange with the Emperor Joseph II in Amadeus after hearing Mozart’s new opera, The Abduction from the Seraglio. The prodigious Mozart, eager to share his latest work with the Emperor, is soon mystified by the seemingly thoughtless call for simplicity.

EMPEROR: You have shown us something quite new tonight.

MOZART: So then you like it? You really like it, Your Majesty?

EMPEROR: Of course I do. It’s very good. Of course now and then – just now and then – it gets a touch elaborate.

MOZART: What do you mean, Sire?

EMPEROR: Well, I mean occasionally it seems to have, how shall one say? [he stops in difficulty; turning to Orsini-Rosenberg] How shall one say, Director?

ORSINI-ROSENBERG: Too many notes, Your Majesty?

EMPEROR: Exactly. Very well put. Too many notes.

MOZART: I don’t understand. There are just as many notes, Majesty, as are required. Neither more nor less.

EMPEROR: My dear fellow, there are in fact only so many notes the ear can hear in the course of an evening. I think I’m right in saying that, aren’t I, Court Composer?

SALIERI: Yes! yes! er, on the whole, yes, Majesty.

MOZART: But this is absurd!

EMPEROR: My dear, young man, don’t take it too hard. Your work is ingenious. It’s quality work. And there are simply too many notes, that’s all. Cut a few and it will be perfect.

MOZART: Which few did you have in mind, Majesty?

EMPEROR: Well. There it is.

Well. There it is. Not much of a resolution when two parties are clearly not seeing eye to eye. One party seeking excellence while the other seeks simplicity.

Anyone familiar with the biopic understands that Mozart was brilliant and also that he struggled mightily to accept feedback from those around him. Sound familiar? It’s hard to live in the depths of a discipline and at the edge of discovery and feel good about feedback unsupported by significant evidence. “Cut a few and it will be perfect” is insulting to those who set the bar in their fields. Yet clearly institutions have room to improve in terms of taking in new kinds of feedback to enrich the ways we reach and meet the needs of increasingly diverse lifelong learning communities.

As edtech companies approach universities to establish new partnerships, they need to understand the transparency required to consider such relationships. Part of this transparency is demonstrated through active listening and developing fit-for-purpose mechanisms to share new insights with established institutions. There are likely opportunities where institutions could simplify models and programs in order to make learning experiences more accessible and inclusive. Edtech companies are tapping into new sources of data and have an opportunity to enrich our view of the world. Building trust around methods for data collection and exploitation of new information is essential to driving meaningful change.

Question 2: It’s all just the same notes, right? Does it matter who does what?

In higher education’s current mode of experimentation and disruption, it’s cliche to ding the lecture for it’s imperfections. As an advocate for active learning and with responsibility to foster a culture of innovation in learning at my own institution, I would hardly argue that lectures are either infallible or fit for every purpose. Yet, anyone privileged to learn from and with a great instructor has a mental repository of specific lessons, lectures, and labs that form many of life’s eureka moments.

One lecture filed away in my own repository was delivered by Professor Tim Fort who, at the time, was leading an ethics discussion at the Ross School of Business at the University of Michigan. A captivating storyteller, Fort was a graduate of the University of Notre Dame before joining the faculty at Michigan. Anyone reading the previous sentence with an ounce of interest in college sports just immediately shifted mentally to the gridiron. In order to illustrate an idea about culture and identity in organizations, one that has stuck with me ever since, Fort went to the playing field as well to make his point. “The Michigan-Notre Dame game is the best college football rivalry in the country,” he began.

My sport enthusiast readers are now distracted by equally subjective counter arguments to this claim. Stick with me for a bit. Conveniently, much of this story was captured in Fort’s book[1] written a few years after he delivered the lecture. After grabbing our attention with football (it was fall in Ann Arbor after all), he went on:

“…beyond the historical excellence of the programs’ traditions, there is little doubt that the schools have the two best college (fight) songs. It’s worth attending the game just to listen to the bands play all day long. The schools have a neat tradition of first playing the other’s song before playing its own. When they play the other’s song, they follow the first two characteristics of being an Honest Broker. They abide by the rules. They play the right notes, the right time signature, right rhythm – they do everything “right”. There is something to be said for that. They also build a sense of respect for politely playing the song. The fans whose song was just played aren’t going to boo their own song and the fans whose band just played aren’t going to boo their own band. So there is polite applause and there is something to be said for that too. But when they play their own song, it takes on an entirely different character because it is played with passion, pride, and identity. At that point, it transcends the particulars of the written music and takes on a profoundly inspiring transcendent character.”

In other words, Notre Dame’s marching band is perfectly capable of playing the greatest fight song of all, “The Victors”, but we don’t truly experience the music until Michigan’s band brings the performance to the next level. This has me thinking about who does what when edtech companies and universities seek to establish new partnerships. Do we take the time to answer this simple question: who does what? Moreover, does the implied division of responsibility and degree of collaboration yield the best outcomes for our learners and other university constituents?

This is not a commentary on the unbundling of the OPM model. That topic has been covered sufficiently elsewhere. Plus, it’s inevitable that it will unbundle and rebundle again (and repeat) in a continuous tug-of-war between the pull to better address university needs and the relentless quest for stickier ecosystems.

It is a question of who is best suited to do what in the context of mission-based activity. In making decisions on behalf of a university community, when and where is an institution willing to let go of some control? All of it? When and where should you hold on tight? Letting go of a technical platform seems reasonable as long as one feels comfortable with the flow and ownership of data and that the institution has a role in co-creating the roadmap. But what about the relationship with students? As we determine different and shared responsibilities for attracting learners and guiding their experiences, are institutions comfortable with an outside organization making the case to a prospective learner that your institution is the right choice? Back up a bit. What about helping the learner understand the choices he or she has the opportunity to make? Is the institution comfortable with the ways in which an outside organization will represent the institution’s values and interests? Is the institution comfortable with the ways in which the edtech company is empowered to make decisions on behalf of the institution?

This feels different to me and is an important consideration in a space that suffers from the tendency to bundle services. This is about passion, pride, and identity. We’re all playing the same notes to the same songs, right? There is a reason the University of Michigan engages current U-M students to facilitate walking tours on campus for new prospective students. It’s not about the script. Almost anyone could deliver the script with precision. Like Notre Dame’s marching band playing the Victors, an actor delivering the campus tour would receive polite applause. But only someone with passion and pride for U-M and an experience-based understanding of our institutional identity will be able to transcend the particulars of the script.

Why should this be any different when we move beyond campus tours to helping prospective learners around the globe to understand new digital and hybrid academic programs? Sharing responsibility for the development of enabling infrastructure is one thing. Designing a student experience and helping a learner navigate a set of choices is something entirely different all together. Who shapes and guides a lifelong learning pathway is worth serious discussion. At Michigan, as we unbundle our own curriculum from the traditional disciplines and rebundle around the problems, events, and phenomena most important to society, we dramatically increase student choice. We balance agency with new forms of guidance. It’s complicated. An agent with a script, and even armed with rich data sets, would struggle mightily to communicate the character of the institution and the totality of the opportunities it may provide to a learner with specific needs.

In fairness to a large number of edtech companies who continue to see opportunity when entering the higher education space, some of this is on us. Universities have been slow to recognize how little we have communicated the differences between institutions. The good news is that there is a great deal of differentiation across the higher education landscape. And universities are in the best position to share what it means to engage with a particular community of learners.

Question 3: Does this collection of notes match the scene? Are we able to select the notes we want to play?

On the same day as our panel session in Salt Lake City, the Golden State Warriors were visiting the Utah Jazz for game four of their NBA playoff series. Many grown adults turned into wide-eyed children when Steph Curry walked through the lobby of the conference venue which also provided lodgings for the visiting team. We’re all human at the end of the day. Well, except Steph Curry, apparently.

Now imagine for a moment that you are watching a professional basketball game in a modern day arena and instead of listening to the Final Countdown, Thunderstruck, or Jump Around during stoppages in play, the stadium DJ was serving up a nice dose of November Rain, Wild Horses, or In the Air Tonight. Each of these songs just triggered flashbacks for this author. Nonetheless, each of these songs would also be incredibly awkward in the context of a NBA basketball game.

I lost track of the number of conversations that I had with conference attendees where an edtech representative summarized a pitch by saying, “really, we’re the Uber for education,” “the Netflix for education,” or “the Airbnb for education.”

Gang, I get it. We want to make things simple. But I need you to follow a crossover claim with the following: “Importantly, when we think about successful business models from other industries, we know we can’t just force-fit these models into our complex educational system. There are elements of this model that give us new insights when thinking about change within a unique sector populated by institutions with intricate operating models. And we are keenly aware that there is a critical translational step. We want to design this innovation with you, not for you.”

When I hear “Uber of education” I again queue Inigo Montoya, “I do not think it means what you think it means.”

As edtech companies consider establishing partnerships versus vendor relationships they should consider whether they understand those aspects of higher education that will stand the test of time and those aspects of higher education that are ripe for change. Don’t impersonate the character Dennis Hope from the movie Almost Famous, a movie set in the early 1970s: “If you think Mick Jagger will still be out there trying to be a rock star at age fifty, then you are sadly, sadly mistaken.”

Universities are incredibly resilient. This doesn’t mean institutions shouldn’t take today’s market forces seriously. But it does often feel like edtech companies don’t see the value that comes from these complex adaptive systems. R-E-L-A-X.

When universities explore edtech partners we explore whether the model proposed is sufficiently flexible or is it fixed. Will the university have the ability to influence the platform roadmap, shape the learning experience, and extract all (not some) of the data with unfettered access? Institutions like Michigan have incredible intellectual diversity and many areas of strategic focus. This is often hard to digest from the outside. We want to select the notes we will play.

Institutions select vendors for particular services. We create flexible strategic partnerships to enable core strategies. A comprehensive research university needs partners that understand all of our strategic interests, even if they intend to partner around a relatively narrow set of activities. Sometimes we feel like Elwood in The Blues Brothers when speaking to potential edtech partners about how a model maps to our highly diverse mission-driven activities:

Elwood: What kind of music do you usually have here?

Claire: Oh, we got both kinds. We got country *and* western.

With our partners, we seek a greater understanding of our full catalogue and our broadest aspirations. We seek transparency. We seek harmony through clearly defined shared values and interests. We seek mechanisms to debate future direction and opportunities to leverage our respective strengths and experiences. After all, we are neither slow nor fast all of the time. What we are is focused. Mission-driven institutions have clear north stars. As we move forward together as communities we calibrate speed accordingly. We’re not afraid of a push from good partners but we do expect our partners to be pulled.

Farewell to Ripened Fellows in the Digital Innovation Greenhouse

Eric Joyce, Marketing Specialist
@ericmjoyce

Student Fellows serve as a conduit for energy and creativity supporting new innovation in our Digital Innovation Greenhouse (DIG). Through these experiences, students work one-on-one with developers, user experience designers, behavioral scientists and data scientists to assist in translating digital engagement tools from innovation to infrastructure.

Several Fellows shared highlights of their work with the DIG team during a special “Student Fellows Showcase” last month. In their own words, Fellows articulated their unique approaches to enhancing digital tools at DIG through design, development, data science and more while also reflecting upon their personal growth throughout their experience.

In celebration of their great work, we would like to highlight a few of our student Fellows who recently graduated or have otherwise ended their fellowship with the DIG team this spring or will end their fellowship at the end of this summer.


Monica ChenMonica Chen

User Experience Design/QA Fellow Since September 2016

Worked on the Following Digital Tools: ECoach, GradeCraft, MWrite, Problem Roulette

What Have You Learned During Your Time with the DIG Team?

“In general, I have always understood the importance of broad perspectives and multiple approaches to a problem, but my time with the DIG team truly exposed me to all of these facets in an optimal way through the simple physical layout of the office space as well as the overlapping nature of the tools DIG has taken on. I’ve gained confidence in the practice of articulating my ideas, presenting my work, and taking apart issues that come up. The people at DIG operate with a lot of transparency, and as a student Fellow I felt comfortable interacting with everyone. I felt empowered and free to explore any interests I developed, whether it was to ask for a task to strengthen a skill I was feeling weak in, or whether it was to occasionally assist with another tool so I could get a feel for other work that was being done. In addition, I have learned to keep consciously trying harder to detach myself from the initial versions of any work I do, because additional meaning and value are found through cycles of iteration. Constant feedback and open channels of communication within the DIG team helped me be more precise and, more importantly, take risks.”

What’s Next For You After Your Fellowship?

“This outstanding fellowship contributed to the extension of my time at Michigan to a full fifth year rather than only the first semester, and its impact continues to benefit me in positive ways. Therefore, when I discovered the summer internship opportunity offered at DIG, I was immediately interested in prolonging my time with the team. I will be staying on through Summer 2017, after which I am considering the possibility of graduate school or full-time work somewhere on the West Coast – where I am from.”


Dana DemskyDana Demsky

Graphic Design Fellow Since January 2016

Worked on the Following Digital Tools: Academic Reporting Tools (ART 2.0), GradeCraft, Policymaker, Student Explorer

What Have You Learned During Your Time with the DIG Team?

“Over the year and a half of my Fellowship, I learned so much about design that I would have never learned in a studio class. I got to collaborate with the other student Fellows and the full-time designers on web platforms that students use across campus. My work here had a real impact on not only DIG itself but the university at large – for example, it is so cool to think that the logo I designed for ART 2.0 is now seen by students everywhere at the University of Michigan.”

What’s Next For You After Your Fellowship?

“I feel confident in my design skills because of what I’ve learned at DIG. After graduation, I see myself as a successful graphic designer for the digital world, working at an innovative marketing agency. I can’t wait to see where the skills I’ve gained at DIG will take me!”


Yidi HongYidi Hong

User Experience Design Fellow Since September 2016

Worked on the Following Digital Tools: Policymaker

What Have You Learned During Your Time with the DIG Team?

“It was my first time to work closely with [Policymaker] and engineers and work on a real project where my design got to be built out. During the tons of meetings with my team, I learned how to present my ideas and justify my design decision, and most importantly, I gained confidence during the process. In terms of design, I learned to think more comprehensively about use cases and pay attention to details in design. I’m also glad that we have easy access to our users, students, and we do value students’ thoughts and feedback. I have the opportunity to talk to students and incorporate their thoughts into our design.”

What’s Next For You After Your Fellowship?

“I’m moving to the Bay Area after graduation. I’ll continue my passion in design, starting at a startup, Tile, as a UX designer.”


Rob TruexRob Truex

Data Science Fellow Since January 2017

Worked on the Following Digital Tools: Academic Reporting Tools (ART 2.0), Transcript of the Future

What Have You Learned During Your Time with the DIG Team?

“During my time at DIG I learned a variety of techniques for unpacking meaning from complex datasets, particularly through data visualization. It also provided me with an opportunity to expand my skills in Python and R through coding collaboratively with my peers. One unexpected lesson I was able to take from DIG is the importance of reaching outside of my comfort zone. I worked on design mockups for the Transcript of the Future project without having any background in UX design, and it was incredibly rewarding.”

What’s Next For You After Your Fellowship?

“I am currently looking for learning analytics positions, as my overarching career goal is to improve education through technology. In addition to my time at DIG, I have worked with digital libraries in order to work toward this goal. Specifically, I hope to use data analytics to improve student access to educational materials.”

*Rob recently accepted a position as a Data Scientist working on the Institute for Research on Innovation and Science (IRIS) project.


Marisa XhekaMarisa Xheka

Personalization/User Experience Design Fellow Since June 2015

Worked on the Following Digital Tools: ECoach

What Have You Learned During Your Time with the DIG Team?

“In my time at DIG I had the opportunity to work on different aspects of the ECoach project and learn what kind of career I wanted to build for myself. And importantly as I gained confidence in myself, I learned how to take ownership of my work and how to be able to advocate for my ideas. As I’m leaving DIG I know what part of the UX field I want to work in and I have the experiences to prove I can do it.”

What’s Next For You After Your Fellowship?

“I’m going to be working as a UX researcher. I’m currently looking for a full time position on the west coast.”


We would like to wish a fond farewell to all Fellows who have recently departed, or will soon depart, from the Digital Innovation Greenhouse, including the following:

  • Jaee Apte – User Experience Design Fellow
  • Jessa Bartley-Matthews – User Experience Design Fellow
  • Wake Coulter – Graphic Design Fellow
  • Mikaela Gonzales – Content Management Fellow
  • Jianming Sang – Software Development Fellow
  • Pavithra Vetriselvan – Software Development Fellow
  • Denny Walsh – Data Science Fellow
  • Jun Wang – Data Science Fellow

The Office of Academic Innovation offers a variety of fellowship and internship opportunities for undergraduates, graduate students and recent graduates at our Digital Innovation Greenhouse as well as our Digital Education and Innovation Lab. Please visit our Student Opportunities page to learn more and apply!

Enter the Compassionate Public Square for the Information Age

This article was originally posted on 5/1/2017 on Inside Higher Ed

James DeVaney, Associate Vice Provost for Academic Innovation
@devaneygoblue

Like many institutions, the University of Michigan was inspired by the early aspirations of the massive open online course (MOOC) movement to democratize education and, in our case, to advance our mission as a public research university. We thought better of a big toe dip, saw reason for a deeper commitment to experimentation, and dove in head first for a swim. Many pushed for swimmers to pick a stroke and stay in their lanes. We prefer the flexibility of freestyle and open swim.

As with any new space, there is no shortage of uncertainty in the world of MOOCs which has led many institutions, as well as those outside the academy with more casual interest, to move on to other things. We remain critically inspired in Ann Arbor for a number of reasons and here I will briefly offer two explanations. First, we chased mission from the beginning and not revenue. Second, when those inside and outside the academy sought to force a definition upon MOOCs, to assign us to a specific swim lane, we resisted, and persisted. Our values based decision-making and conviction around open experimentation are leading to rewards. We were right to dismiss the notion of the MOOC as a single monolithic. We were wise to pull MOOCs into our overarching approach to mission aligned academic R&D rather than pivot our academic strategy to MOOCs. And we will be careful not to overstate our early successes or forget that we remain in a mode of experimentation.

While I continue to encourage patience to those who care to listen and resist any temptation to prematurely write the history of the MOOC, I do believe we are now entering a period of enlightenment. We are now seeing the MOOC evolve beyond minimum viable product and can point to a sizeable wave of second order experiments that move us closer to a future where anyone committed to lifelong learning and listening can fully participate.

Let me provide a concrete example of today that foreshadows a sharp trajectory toward a new compassionate public square for the information age. While the idea of the compassionate public square should supply you with rich imagery of the open and natural flow of people through shared spaces, the future we are building together is anything but pedestrian.

Teach-Out SeriesLast month U-M launched a new Teach-Out Series. Our multiple motivations for the Series are captured in a recent conversation. While it is early in the life of this latest experiment, I’m confident we are onto something big. But let’s back up for a moment. Rarely does a movement roll out without a hitch. One unfortunate aspect of the rise of the MOOC was the acronym itself and, more precisely, the letter “C”. Framing matters.

There are good reasons to anchor something new in something we think we understand. But in so doing we may also inadvertently limit our collective imagination. By framing MOOCs as a new kind of “course”, we immediately pushed people to think in terms of semesters, credits, and the construct of the somewhat industrialized unit of knowledge exchange and interaction that we call courses.

Five years into the MOOC experiment, the Teach-Out may help us truly break free. We are allowing ourselves to think less about taking experiences online and more about what is only possible when we remove many constraints of time and space. Forget replicating the college campus online. It’s a losing proposition. What we have in front of us is an opportunity to do something completely different. This is what many of us imagined the MOOC would be in the first place. With the Teach-Out, we combine the global reach of MOOCs with just-in-time teaching and learning. Accidents in history and an urge to define a new product for the market tethered MOOCs to the credit hour economy. The credit hour economy may not be long for this world and I would argue, even more vigorously, that we are ready to break free from a single definition of the MOOC.

The Teach-Out speaks to our sensibilities at Michigan in that there is vast opportunity to turn our research mindset on ourselves and reimagine lifelong and lifewide learning, residential communities committed to discovery, and public engagement in an information age. We began to think in less constrained ways five years ago. Through experimentation, it is now more obvious that we can think boldly about the ways we engage with the world.

Reflecting on our history and in consideration of our institution’s future, U-M’s President, Mark Schlissel, sees in the Teach-Out series an opportunity for even more impactful public engagement, “Throughout our 200-year history, the University of Michigan has excelled in research and teaching, leading the academy with our broad portfolio of scholarly work. To extend our leadership during our third century, I want to help us disseminate our work and share our expertise in a more conspicuous, and public, manner. One opportunity to do so is through our new U-M Teach-Out Series, which leverages academic innovation to reimagine public engagement for the century ahead. Efforts like these will advance our mission as a public university by better connecting U-M’s broad intellectual power to areas of society where research and understanding can make a difference in lives and communities. There is no shortage of problems that demand our attention and our rigor. And the more we can use our work and expertise to influence decision-makers at all levels, the better our world will be.”

Institutions like ours have long been committed to public engagement. Town-gown success stories and challenges abound. Research universities are especially adept at supporting their campus communities bound together by a commitment to discovery. Over time, we’ve made important strides to make great universities more open to the world and to rethink what it means to be a part of a university’s community. As a result, our constituencies have grown. This growth has been important but by future standards it may appear relatively flat. With MOOCs and Teach-Outs we may be on the verge of a hockey stick moment. The sudden, sharp, and upward shift of data points will require entirely new measures of public engagement and societal impact.

The cacophony of voices currently questioning the impact and relevance of public universities may soon find harmony through massive public engagement that leads to massive societal impact. We think the creation of the Teach-Out model will ultimately be viewed as an inflection point in this important story.

Enter the compassionate public square for the information era. Town-gown relationships will remain critically important but it’s time to think bigger. Like physical world public spaces with flows of people, culture, and mercantile exchange, which are often neglected when trends shift, our virtual public squares with flows of ideas need constant attention.

Only with focused attention will we create a compassionate public square that is virtuous, vibrant, and vital. How do we reverse digital polarization? How do we support a free-flowing exchange of ideas? How to we accelerate the flow of insight to application? How do we ensure that our open spaces are in fact inclusive? What might we borrow from physical spaces that bring us together and the minds of Olmsted and Jacobs and the rest? How should universities participate in the sharing economy? How should we think about the ideas of selectivity and openness as we discover new opportunities to engage with the world? These are questions we seek to explore as we reimagine public engagement and our potential for societal impact.

A vigorous debate has emerged around safe spaces and speech on and around college campuses. There are important questions to address and I’m confident that strong institutions will explore them carefully. In parallel, we can continue to build the environment we know we need. A compassionate space is virtuous, vibrant, and vital. Because the public square we are building is part virtual, it can also be vast. The potential is immense if we build it carefully. To paraphrase Sara Armstrong, a very talented colleague from U-M’s Center for Research on Teaching and Learning (CRLT), to build a community carefully is not to walk gingerly past controversial topics, but rather to commit ourselves to build a community that is full of care.

It is difficult for me to know what your safe space looks like and for you to understand mine. To do this well is to embrace a massive and distributed listening exercise. And we should. But on the way to understanding each other in the long run we can immediately dial up compassion in the short run. Paul Bloom, professor of psychology and cognitive science at Yale University, tells us that empathy isn’t enough anymore. More specifically, the capacity for empathy that concerns Bloom is the capacity to put yourself in the shoes of others. In making his case against empathy, he makes a compelling argument for rational compassion.

My potentially dangerous oversimplification of this view in the context of the opportunities ahead for universities goes something like this. Compassion is more inclusive than empathy, more immediately actionable than a call for safe spaces, and is more uniformly applied toward alleviating suffering whereas empathy can be used for good or evil. Or in our university context, a compassionate space that is virtuous, vibrant, vital, and vast, can only serve to elevate public discourse, not distort.

The result is splendid. The compassionate public square creates conditions that demand understanding. The citizen who participates is not only engaged and informed, but equipped with compassion. In our moment in history where instantaneous global connections among people with differing views are commonplace – a moment that could be tragically short-lived if we do not create the conditions for pluralism to thrive – I challenge you to come up something more essential than a compassionate humanity.

The Teach-Out model presents new opportunity. It is born of the teach-ins of the 1960s and the MOOCs of the early 21st century. The time elapsed between origin story chapters is at once an eternity in Silicon Valley and a blink of an eye in the long view of the academy. A positive story of strange bedfellows will someday replace “this will never work” with “of course it did”.

We seek a future where everyone participates. Today everyone can’t. Our institutions have mastered models for discovery as evidenced by a broad and impressive portfolio of research and scholarly work. The fact that many still ask if selectivity and openness are mutually exclusive means there is work to do in maximizing the impact of our institutions on society. I suspect it is in our lifetimes when we cease to acknowledge the tensions between these terms as we move much closer to a future where everyone can participate.

The Teach-Out model is a concrete contribution to the compassionate public square for the information age. U-M will continue to expand this work by unbundling our expertise from the disciplines and rebundling around the problems, events, and phenomena most important to society. We will invite other great institutions to join us in this effort. And we will expand a community of engaged citizens that seeks to understand each other and the world around us.

The compassionate public square for the information age is anything but pedestrian.

A Picture is Worth a Thousand Words: Understanding Learners Through Visualization

Kara Foley, MOOC Assessment Fellow, University of Michigan Ross MBA and M.A. Educational Studies

Eejain Huang, Data Visualization Fellow, Combined Program in Education and Psychology, Statistics
@eejainh

Filip Jankovic, Research Assistant & Data Science Course Coordinator, MSE Industrial and Operations Engineering

Why Data Visualization?

In the virtual world, learners’ background and needs are assumed and indistinct, creating one of the biggest challenges for massive online courses: finding out who the learners are.

Here at the Digital Education and Innovation Lab (DEIL), we have devoted a lot of time and resources into developing effective initiatives to transform education. As an essential element in this process, collecting and interpreting learner feedback is at both the beginning and end of every MOOC course.

Screen shot of dataThree surveys were designed with iteration, assessment, research and marketing needs in mind. However, useful information is often hidden from decision makers because of the considerable amount of data gathered from these surveys.

To better explore this large amount of survey data, understand our learners and adapt courses according to their needs and feedback, DEIL created the DEIL Data Team to tackle these issues.

Data visualization was chosen as an ideal approach to interpret big data based on several reasons. First, data visualization can serve as a summarization, a guidance, a metaphor and eventually a gateway to understanding data. Second, data visualization can convey learning analytics in a digestible, actionable format for the audience. Like the old proverb says, “sometimes a picture is worth a thousand words.”

Who We Are and Our Journey

Our team has three members from diverse backgrounds who have contributed different expertise. Eejain Huang is a Ph.D. candidate working in Combined Program in Education and Psychology as well as a masters degree in statistics. Filip Jankovic is a U-M graduate with a Master of Science and Engineering degree in Industrial and Operations Engineering and is currently working with Dr. Christopher Brooks on the Applied Data Science with Python Coursera Specialization. Kara Foley is a recent graduate and has obtained her Master of Business Administration degree and Master of Arts degree in Educational Studies.

Eejain Huang, Filip Jankovic and Kara Foley in discussion while sitting around a table with laptops in front of large screen with a visualization of the globeKara was hired as a MOOC Assessment Fellow in January 2016 to evaluate which data visualization tool(s) would both be feasible to implement and highly automated. With input from additional faculty and staff across the University, we opted to reformat and clean the data using R and construct the visualizations in Tableau. Kara created initial prototypes of dashboards, and saw the challenges of data manipulation and advanced visualization design that needed to be addressed by a larger team with additional skills.

Eejain then began her full-time role in the summer of 2016 to clean the data and reshape it into a format that could be used to create Tableau visualizations. Much work had been spent on optimizing data structures to accelerate processing time, cleaning text responses with regular expressions and creating unique identifiers to link different surveys.

Finally, when the data was ready for creating and refining the visualizations using Tableau, Eejain created the Tableau dashboard. Filip joined our team in August of 2016 bringing with him his expertise in data science guiding best practices for visualization design. Filip helped ensure the visualizations represented the data accurately and effectively – an essential step when creating dashboards that allow decision makers to make well-informed decisions.

As dashboard revisions were made, we shared the visualizations with faculty and staff for their feedback. This involved hosting multiple meetings with different teams within the Office of Academic Innovation, and also conducting individual interviews with people who have specialized expertise in different steps of shaping a MOOC course including Dr. Christopher Brooks, Laura Elgas, Dr. Gautam Kaul, Noni Korf, David Lawrence-Lupton and Dr. Donald Peurach. Through this process we have learned about main priorities, key data points of interest and potential implications of this data.

What Did We Find and Some Insights

Currently, we are still experimenting on different ways to represent data and are seeking the best ways to distribute our findings. However, even at this transitioning stage, we think our findings may serve as an insightful guide for understanding learners and transforming MOOCs.

Illustration of a visualization

Although we recognize that our sample size is a small percentage of our learners, we discovered some trends that instructors may find useful for course design. While the majority of our respondents come from the United States and self-identified as Caucasian/ European / Russian, learners from developing countries constitute the next largest proportion of respondents. We noticed a difference in gender participation in our MOOCs, and in higher education degree attainment. Gender was a factor as well in self-ratings regarding confidence levels for language and use of technology. Regarding degree levels, these played an important role in influencing learners’ choice about participating in a course, their expectations about the course and their motivation for participating.

When looking at the dashboard, we found that many of the demographics and student feedback changed from course to course. This view can provide important insights to instructors to help them better understand their learners.

In conclusion, our findings showcased how diverse our learner community is and how different their needs can be. Nationality, ethnicity, gender, educational background and prior experience etc., all contribute to their expectations of, and experience during, the course. Thus, we hope our visualization can help course designers understand who their audience is, what they want, as well as how to make their content more accessible and eventually create a more targeted, personalized and engaged experience for every learner around the globe.


Faculty who are interested in learning more about MOOC survey data may reach out to the Office of Academic Innovation directly at academicinnovation@umich.edu