Jeremy Nelson, Director of the XR Initiative
2020 has started right where 2019 left off, as communities across campus and higher education explore new opportunities in a rapidly evolving extended reality (XR) space. We had a great response to our inaugural XR Innovation Fund call for proposals where we received 14 proposals from seven different University of Michigan schools and colleges. Our XR speaker series has been finalized, and we have a great lineup to build awareness of XR.
Building on this momentum, we are excited to formally announce Dr. Michael Nebeling, Assistant Professor of Information in the School of Information and Electrical and Computer Engineering in the College of Engineering, as our 2020 XR Faculty Innovator-In-Residence. Michael has been a great partner to Academic Innovation and has been instrumental in laying the groundwork for the XR Initiative. Through this partnership, Michael will help the XR initiative by providing strategic guidance, evaluating proposals for our XR Innovation Fund, providing input on research opportunities within our projects, and supporting students participating in the XR Initiative.
I sat down with Michael to learn more about his work and what excites him about the XR Initiative. We look forward to helping shape the future of XR in learning with Michael’s support.
What most excites you about the XR Initiative at the University of Michigan? What XR projects happening on campus now excite you the most?
I‘ve been part of precursors to the current XR initiative ever since I joined U-M in 2016. I consider our XR Initiative to be unique, at least in terms of scale, if not in the idea. There is no question about it: a lot of universities have realized the need to incorporate XR and invest heavily into new lab spaces. Over the years, I’ve seen many of those lab spaces and they all still kind of feel like the computer rooms when I was in 8th grade (when smartphones and laptops weren’t really a thing yet) — it was so special when our geography teacher took us to the computer room and had us do assignments and quizzes there, using much more primitive versions of software that you would now call “Maps.” While a very refreshing way of learning, it only enjoyed the support of less than a handful of our teachers at the time — those few who saw potential and were brave enough to go out of their standard way of teaching and “play with the new tech.” In my mind, the XR Initiative is a systematic effort to make XR both a method and a tool to enrich teaching and learning by providing services and support to our faculty, staff, and students to make XR easy and something worth adopting.
I’ve enjoyed most the interest and curiosity about XR technologies among members of the initiative and how they each think differently about its potential and challenges. It’s really been a diverse group of more than 50 faculty from more than 12 schools from across U-M. One of the most exciting projects on campus to me is iGYM — I got to work with my colleague and friend, Roland Graf, who was leading the project, and students I recruited from my new AR/VR courses. Together, we studied how iGYM as a novel interactive floor projection system can empower wheelchair users to compete with non-wheelchair users in a “life-size version of an air hockey game.” I reflected a lot on this project: I’ve really enjoyed working with the children who participated in our studies and with the parents who continue to be enthusiastic about the project. To me, it’s a great example of how XR can increase accessibility and inclusivity, and quite literally level the playing field.
What types of research are you working on now? What tools have you created and how would you like to see those used in the field?
While I’ve always been critical of new interactive technologies, which are really the subject of my studies, I’m overall excited about XR and am currently working along several directions. One stream of research is focused on making it easier, faster — and therefore cheaper — to create XR applications. Over the last three years, I’ve developed a set of rapid prototyping techniques that emphasize the physical and allow novice XR creators to easily transfer their knowledge from more traditional web and mobile application design. I’ve implemented each of the techniques in new tools to study their potential and limitations. It will be great to see those tools used outside my research lab and I’m looking forward to working with you and members of the XR Initiative to incorporate them into new research, design, and production workflows. Another major stream of my research is focused on creating heuristic techniques and metrics for evaluating XR experiences, both in terms of usability as well as higher research goals including effects on training and learning. We have developed an analytics toolkit and used it to design and evaluate an augmented reality version of a disaster simulation activity part of SI 537, Crisis Informatics, taught by Ed Happ.
What does the future of XR look like for you? Do you see convergence of AR/VR, or do you see a place for both? How do you see teaching and learning being affected by XR technologies?
I have this vision that it will be natural one day to use AR or VR depending on task, context, and individual preferences. AR and VR are quite different in how they look and feel, and while there are problems where either might provide a good solution, in the majority of cases, it is clearly only one of them, ruling out the other. So, no, I don’t think they will converge as media. But, yes, I do think there will be some convergence in terms of device support, although even there the technical requirements are so different that it is hard to imagine one kind of headset, or contact lenses if you will, that can do both well, without more significant advances in display and tracking technologies in the future.
I am a great supporter of embedding XR technologies in teaching. As you know, I’ve been teaching two fundamental AR/VR design and development courses on campus, SI 559 and SI 659. I’m often asked to give lectures on XR and provide my critical but overall positive perspective on the technologies. In my current roles with the Center for Academic Innovation and the XR Initiative, I will expand my efforts and help bring XR technologies in a variety of teaching and learning environments. I strongly believe that while it’s currently still really difficult to make good use of XR technologies in the “classroom,” many of us are constantly experimenting and, together, we will provide good and bad 🙂 examples from which we can all learn.
I heard you are creating an XR MOOC series, tell us more about that. What types of learners are you targeting? What do you hope they take away from your specialization?
It’s really an exciting project. I’ve done a Teach-Out on XR technologies a few years ago that has reached more than 2,400 online learners worldwide. The focus there was on making sure people understand the concepts and differences between AR and VR, and that we all start thinking about how these technologies may impact our daily lives, both personal and professional, in good or bad ways. The new series incorporates elements from this Teach-Out but will expand on it in a lot of different ways. We have been designing three new online courses to provide significant training in design and development of XR technologies, together with frameworks for ethical, social, privacy, and security aware design of AR and VR experiences, as well as guidelines and recommendations for creating new XR lab spaces for research and development as well as teaching and learning. I see a lot of the materials I am creating as blueprints of how to get started in XR for everybody, be it as a new researcher, teacher, designer, developer, or manager, and my hope is that what we are creating will be the number one go-to resource for many universities, schools, and libraries, including our own faculty, staff, and students here at U-M.
You have been involved in our inaugural XR Innovation Seed Fund review committee, what are your impressions of the proposals? What excites you?
Reading the proposals and discussing them with you and other members of the XR initiative has been a great experience. When looking at a set of proposals as diverse as the ones we have received, both in terms of discipline and prior experience of faculty, the huge variety of problems that could potentially benefit from new approaches using XR technologies is most impressive to me. In general, it was exciting to see how all faculty had great plans of incorporating their research with their teaching efforts on campus, which I think is also a major strength of our larger XR Initiative here at U-M.
What does the Faculty Innovator-In-Residence with Academic Innovation mean for you and your work?
In this official role with Academic Innovation, I consider it my task to help sustain and grow the efforts in XR. As I said, I’ve been a key partner in many XR efforts prior to the official launch of the XR Initiative in September 2019. So, it felt a little bit like a recognition of my continued effort. While the interest on campus has always been high, there are only a few of us who have the technical background and focused research interest to really push the envelope of how XR content is being designed and used. It’s an exciting opportunity for me to bring in my experience from working with XR over the past few years. I was able to demonstrate the benefits of using the techniques and tools developed in my research in user studies with what I would consider a broad audience. But it’ll be an interesting challenge to try and incorporate some of my solutions into the new research, design, and production workflows as we are creating the first examples of the efforts of our new XR Initiative and want to make sure we can scale and adapt them to a larger variety of disciplines and problems.
