Jeremy Nelson, Director of the XR Initiative

In this week’s MiXR Studios podcast, we explore how extended reality (XR) can be used in humanities to advance the University of Michigan’s liberal arts mission. Sara Blair, the Patricia S. Yeager Professor of English, discusses how she designed a course to frame the novel with a goal of immersion, and she used virtual reality (VR) as the object of comparison. She brought together a diverse group of students from across campus to examine this new media and develop a critical perspective of experiences created in XR.

Blair discusses the journey she took to develop her course, “The Novel and Virtual Realities.” We hear about the types of technical challenges she faced throughout the semester and the incredible support she received from U-M staff including Jan Stewart, the manager of emerging technologies within LSA, and LSA Technology Services staff. She intentionally designed her course to spend half the time in a conventional seminar classroom and the other half in a VR lab. This hybrid approach allowed the group to create a space for critically informed dialogue comparing literary texts and XR experiences. Blair said this enabled a powerful exploration of perspectives across diverse media.

We also talked about the challenges in evaluating and securing the XR titles that would be appropriate for the course. The class explored titles such as Dr. Courtney Cogburn’s, “1000 Cut Journey,” Gabo Arora and Barry Pousman’s “Clouds Over Sidra,” and Asad J. Malik’s “Terminal 3.” The class discussions about these titles were fascinating and created a unique learning experience for the students. Blair also shares her experience developing a short-form VR experience with the Emerging Technologies Group at the Duderstadt Center to explore multiple points of view from a pivotal scene in the novel “Uncle Tom’s Cabin.” The students were able to compare that scene from the book with a VR experience where you can embody the main character Eliza as she crosses the Ohio River. Additionally, you can experience that same scene from the point of view of a slave catcher chasing her and a bird’s eye view of the entire scene.

I enjoyed talking with professor Blair and learning how she constructed this course and thinks about XR as the object of study in her work. Please share with us what you would like to learn more about in the XR space at [email protected].

Subscribe on Apple Podcast | Spotify

Transcript: MiXR Studios, Episode 6

Jeremy Nelson (00:06):
Hello. I am Jeremy Nelson. And today we are talking with Sara Blair, who is the Patricia S Yeager collegiate professor of English language and literature, and the vice provost for academic and faculty affairs at the office of the provost at the University of Michigan. We are talking about her groundbreaking work in her course, the novel in virtual realities coming up next in our mixr podcast.

Jeremy Nelson (00:37):
Welcome Sara. Thank you for joining us today.

Sara Blair (00:40):
It’s great to be here. Thanks Jeremy.

Jeremy Nelson (00:42):
Yeah, I’m so excited to talk with you and share with our listeners the exciting work you’ve been doing, the pioneering work you’ve been doing in XR at Michigan, and yeah, I’d just love to start with sharing how you got into this. Like why did you pick VR and share your journey a little bit? That would be great.

Sara Blair (01:01):
I’m really happy to, and I should start by saying that I came into this work with, with VR with no, uh, expert knowledge, no experience as a gamer. I had never even worn a headset before I got interested in this kind of work. So, uh, as a faculty member, I took a really deep dive off a very high board last fall, 2019 that now it seems a million years ago, right? Um, I, I taught a new course, and a new kind of course for the first time it was called the novel in virtual realities. Um, it was hugely exciting to have the chance to plan and develop this course, which I got to do as part of a working group in the provost office, that was charged with thinking about how humanities faculty in particular could develop innovative ways of using XR technology, um, to advance Michigan’s liberal arts mission, um, how we could use those technologies to raise critical questions about culture, society, technology itself, the ways we’re shaped by it, not just as learners, but as citizens and as social beings.

Sara Blair (02:13):
So, as I said, I’m not a gamer or a VR maker, but I do have a long history in my research and teaching of working with photo based, indexical images and image culture, and VR had started to feel to me like, you know, kind of the next step. I got really interested in particular in the claims that were being made a bit about VR as a so-called empathy machine, a phrase that infamously now was coined by VR maker, Chris Milk, and which has come quickly to be viewed particularly among scholars and researchers and critical media studies with a lot of skepticism. Um, but even given that skepticism, we’re left with this really important question. Um, what about that promise of VR, of XR to transform us in some way through immersion, in the experiences of others and experiences we would otherwise never be able to have?

Sara Blair (03:08):
So I should say the hook for me, was I’m someone who has studied and taught the novel as a cultural form. And, um, from that perspective, I’m very familiar with these kinds of claims. They’ve been part of the history of the novel, the defense against charges that the novel was trivial, brain draining, feminine, worthless, you know, since the very beginning of the form. And what I wanted to do was design a class that would let students from all kinds of disciplines all over campus, think about the novel as a project, aiming for the creation of immersion and empathy, and to put it in dialogue with VR and kind of critical thinking about, about both of them together. And beyond that I wanted to help students understand immersive reality, not just as a technological project or even an aesthetic one, but as a deeply social one with all kinds of implications and assumptions, and really to think what can we learn from our desire to engage in these kinds of story worlds? So that’s, that’s really what got me into the course. What I could not have known on the front end was how challenging and complicated and rewarding and seat of the pants, the work of actually mounting the course was going to be.

Jeremy Nelson (04:24):
Oh, I’m sure. It sounds fascinating. I would, I would have loved to take in that course when I was an undergrad, uh, here at Michigan. Uh that’s yeah. Let’s, let’s hear a little bit more about that. Like how, what, what did you have in mind when you started, did you, did you anticipate a lot of these technical challenges? Did you think a lot of that had been solved already? Or how did that come about?

Sara Blair (04:43):
Well, I was very lucky. My greatest resource in this endeavor was probably humility. I mean, having have a really clear sense of how much, I didn’t know. And, um, I mean, even more than that, my greatest resource was the opportunity to collaborate with colleagues. So some faculty folks like Lisa Nakamura and Arthur Verhoot and Giovana, Mila, Chuck, and others who were trying to do this kind of work at the same time and who had a little bit of experience in, in thinking, uh, about bringing VR into classrooms. And then also with our extraordinary staff colleagues in Ellison, AIT, Blue Core, the Duderstadt all over campus. Um, so that I have to say, I couldn’t have done it without, uh, that space for conversation and collaboration was really critical. That said, this class was the first three credit course of its kind, um, in LS and A, um, certainly that I’m aware of and maybe, you know, among the first nationally, I’m not sure about that, that, you know, really was humanities focused.

Sara Blair (05:42):
Um, I knew from the outset that I wanted to design it to be a truly hybrid kind of dialectical experience in the sense that it, it meant halftime in a conventional seminar style classroom and half time in a, in a VR lab or, um, frankly the best we could do for VR lab on central campus, um, in the MLB, which is nobody’s favorite space. Um, we got a lot of mileage with students about, you know, thinking fantasizing about how, how different a VR lab space could be. Um, and it was designed although, you know, it was mounted as a 300 level English course, and certainly I was happy to have English majors, more advanced, um, students in, in my own department. Um, but it was designed to draw students from all over campus. And it really did. We had, you know, students representing eight different schools and colleges, all the different areas of LS and a, that was a fantastic resource and the structure of the course, the logic of it, the sort of work in the classroom on the ground was really meant to reflect the value of shifting perspectives from a, again, a more kind of conventional critical seminar perspective and conversation to practice, um, a lab based collaborative team-based experiential, deeply immersive in every sense of that word kind of focus and really to think about how each learning situation, each conversation situation, each practice situation, um, might help reshape our thinking about the other.

Sara Blair (07:20):
So we alternated between literary texts and VR, mostly VR, um, a couple of AR experiences as well. And what held all of this together, I think was the clarity of those central questions about how we engage with the experiences of others. And, um, my course in particular focused on what it means to aspire to travel, not just across historical boundaries or geographical boundaries, but across social boundaries of race, ethnicity, class, gender identity, uh, the likes. So what do we make of that, that promise whether it’s the promise of literary art or of XR, that we can know the inner lives, the histories, the struggles of others and strangers who we would never know in real life, is that a meaningful promise? Is it a dangerous one?

Jeremy Nelson (08:10):
Yeah. A lot of responsibility in trying to recreate that or imagine that like, well, so how did you, so I’m assuming you had to kind of curate some content or go about finding, uh, titles, right? Most of this was work that had already existed that we had to bring for students to experience.

Sara Blair (08:28):
Yes and no. And again, um, this is where ignorance was my friend. If I had known how challenging and difficult it is to find readily available VR content in particular for the kinds of purposes I had in mind, you know, I, I’m not sure I would have, um, I would have undertaken the course. I got a crash course over the summer last summer in trying to find and identify VR. Um, it’s even hard to know what to call them, in some contexts, commercial contexts, their titles in more, um, uh, exhibition museum, uh, kind of fight art contexts, they’re, they’re named as experiences in some cases they’re called films when the apparatus and the kind of perspective of the makers really comes more out of cinema, uh, conventional cinema. Um, so I, again, got a crash course in, you know, thinking about, do I use steam?

Sara Blair (09:24):
Do I, do I work with commercial catalogs of readily available VR titles? Do I look at gaming sites? Um, do I, do I, you know, hit up all my friends in kind of new media gallery world to ask them about, uh, current and recent exhibitions? Um, I mean, all of the above to find and curate an array of titles that was appropriate for what I had in mind with my students and that I could kind of sync with literary texts, novels that we were particularly interested in among those novels like Richard Wright’s, native son, Colson Whitehead’s Underground Railroad. We began the whole course with Harriet Beecher Stows’ Uncle Tom’s Cabin, um, which is in some ways a dreadful novel to read, um, but an incredibly important one and a historical example of claims to immersiveness really successful claims to immersiveness and, uh, forms of empathy that had huge political impact.

Sara Blair (10:22):
In fact, we’re still battling the impact in some ways of the, the kind of expectations that, that novel and its its long ongoing cultural reception has, has set up. So looking for titles that would sort of make sense to put in conversation meant, um, using not just some things that were commercially available, but also reaching out to other folks who were experimenting with VR production for research or for artistic purposes. And if it’s okay, I might take a minute just to point to a couple of those examples.

Jeremy Nelson (10:54):
That was my next question. So yeah. Yeah.

Sara Blair (10:57):
So a central experience, it turned out, uh, for our classwork and um, something that the students, again, who came from all over campus, some were computer scientists, kinesiologists, historians, lit majors, architects, I mean, you name it. We, we, we had a really wide representation of disciplines and kind of intellectual interests there.

Sara Blair (11:20):
Um, and I was maybe not surprised, but really interested to see that the one title that they all picked out, it was really strong consensus as being incredibly powerful, um, is in fact a work that’s still in development. I think this is really important as a work of VR, it’s uh, called The Thousand Cut Journey and it was produced by professor Courtney Cogburn of Columbia university, uh, in collaboration with Jeremy Bailenson and his HCI lab at Stanford. And this title is, uh, you know, we could sort of crudely describe it flyover view as an attempt to have people encounter ongoing and historical realities of, uh, racism and microaggression in a deeply embodied and immersive or, you know, um, in some ways immersive way. So users take up the embodied perspective of the, the first person protagonist, Michael Brown, uh, an African American male as he experiences these intimate and micro level and really crushing experiences of racial bias. Um, first as a very young boy in a classroom, um, then as a high school youth being profiled by police and, um, finally as a young man waiting for a high profile job interview, now my students and many other colleagues who experienced this VR, which we had to bring to campus along with Dr. Cogburn’s team to sort of provide the framework, the explanatory framework, and a kind of practice framework around using it in class.

Sara Blair (13:01):
They all found it extremely effective. Um, and not just because it’s based on deep research about individual experiences of racism and unconscious bias. Um, but also, and this is really important because the state of the art when this project was developed and also frankly, funding limitations, meant that it has a much lower level of what I call photo realism. There is not the attempt, there was deliberately not an attempt made to make the story world, the, the world of these experiences, the VR world look as realistic as possible. You know, now more advanced motion capture other, um, kind of aspects of VR technology, um, would, would really allow the filmmakers to push all that. But my students found that precisely because of the gaps and the glitches and the seams because of the, the kind of cartoon-like if you will quality of the visual world, they weren’t actually much likelier to lose themselves in the story.

Sara Blair (14:06):
And at the same time to get a kind of critical distance from it. And that turned out to be really, really important. Um, they did not come out of this experience feeling that they had become the avatar or that they could claim to know what it meant to cross racial or gender or other kinds of boundaries on the basis of 20 minutes in an Oculus headset. Um, so that kind of project, which was not produced commercially, but as a tool for research, I think has huge potential for work in liberal arts classrooms, especially around these core questions about empathy and about inhabiting other’s experiences.

Jeremy Nelson (14:47):
I recall, uh, that class, I sat in on that experience, and we brought Professor Cogburn to campus to talk about this. And it was just, it’s just such a fascinating piece that she built with the help of the folks at Stanford that I think almost every time somebody goes through it, they have a different experience and it’s a different experience watching somebody go through it from you going through it yourself. So there’s little nuance and little pieces that come up each time. And I think, yeah, it was, I think without that, I don’t know if you call it discussion or debrief or you have the conversation after you’ve gone through it. I think, you know, you could leave people with kind of raw feelings or, you know, kind of unsettled emotions on how to, how to handle that.

Sara Blair (15:36):
Absolutely. Absolutely. And, and that was true for users, whatever their own, um, uh, kind of claimed or socially given identities. The opportunity to, um, I think debrief is exactly the right verb, what it means to put these attempts to understand, to make sense of, to gain, uh, a kind of heightened awareness of, um, those are very different ways of putting the project than talking about immersion in an experience, right. Um, giving people an opportunity to, um, move toward a conversation and a critically informed conversation is absolutely part of the horizon of aspirations for a project like Professor Cogburn’s and, um, that made it a, you know, a really perfect, really, really valuable, uh, kind of, uh, resource for the work of the course that I was trying to do. On the other end of the spectrum, you know, I’d say kind of almost inverted in terms of logic, another title that turned out to be incredibly useful.

Speaker 2 (16:45):
Um, another VR that was incredibly useful for my course for my students, um, was actually the, now I’m making air quotes here, “classic” VR film, Clouds Over Sidra, um, which was actually produced in 2015 by Chris Milk, partnering with the United nations and Samsung. Um, it focuses on a day in the life of Syrian refugees in Jordan from the perspective of a 12 year old girl. Um, it made in its moment, a big splash at Sundance and Tribeca, uh, other film festivals, really high production values and completely familiar documentary conventions. Um, that, that VR makes it really easy for us as viewers to internalize the perspective we’re supposed to occupy understanding the refugee crisis, gaining a kind of, um, knowledge, uh, mastery, empathy, uh, a work like that really gave my students a chance to do critical examination of the assumptions of documentary more broadly in XR and print in any form who has the power to choose or manipulate perspective? What rights to a self-defined personhood do the subjects, you know, the, the people being filmed?

Sara Blair (18:00):
Um, do they actually have just feeling something for, or about victims of violence or injustice and especially people who are far away from our lives, does that transform something or does it just make us feel better about our own moral capacities? Can empathy ever really be about the subjects personhood or is it really just about our own? Yeah. So that was a great experience, um, you know, kind of in the opposite direction, if you like, because it really made it possible to, um, to understand the necessity of taking a kind of critical stance on the experience.

Jeremy Nelson (18:35):
Well, actually I just met this last week, one of the gentlemen that helped produce that film as an experience. Yeah. Barry Krausman. And, uh, he’s done some other work for the UN and they’ve built about six kind of VR projects. That’s, that’s fascinating. I have to share that with him, that you explored that in the course.

Sara Blair (18:55):
Yes. Although I, I’m not sure that the effect that we achieved is exactly what they might have been going for, but, um, I was really, you know, just so glad to have the opportunity to connect VR as a technology with a much longer standing set of problems. You know, what, what, what does it mean to think about the way we construct narratives about the lives of others, whether they’re fictional or documentary, or what have you, um, what, what does it mean that we’ve so internalized a set of habits and assumptions about perspective about which perspective is appropriate to take or good to take and, you know, who has the, the opportunity, uh, the means to create that perspective? Um, so it was, it was very valuable, but I think in ways that the filmmakers did not necessarily intend.

Jeremy Nelson (19:48):
Sure, sure. Yeah. Probably similar to what Professor Cogburn said, there were a lot of unintended outcomes or consequences from her work that she never thought about. Right. Well, I mean, these are great examples. This is fascinating, you know, course, you know, what kind of, in terms of kind of looking forward, I mean, I can see some potential concerns. I mean, what concerns do you have about, you know, the future of XR for teaching and learning and maybe what do you want to see Michigan do to help continue to enhance the education of students?

Sara Blair (20:20):
So of course I have to say that, you know, any answer I give or try to give to that question now is going to sound very different even to me than it might have, you know, three months ago, um, before COVID. Um, so I, I do have a longer standing concern, which is to try to make sure, uh, and here I’m just speaking from, um, and maybe for my discipline my perspective to make sure that a humanities centered approach is, is really integral to our efforts. And by that, I mean, an approach that’s led not just by incredible technical expertise and imagination. Um, and the deep respect I have for colleagues who make XR experiences more powerful, more visceral, more immersive, more realistic, um, extraordinary things they can do, but we need sustained critical engagement with the technology itself. And with its changing impacts, we need a stance that will keep us centered, not just on what VR can do, but on why we do it, why it matters, what kinds of values it promotes, how can we come to know ourselves better as users of technology as social beings as citizens?

Sara Blair (21:33):
Um, what does it for? So that’s my ambition. Um, both my concern and my ambition. Um, I think we have a real opportunity at Michigan to kind of lead in that project. You know, kind of looping back to COVID, um, especially as we are all, you know, necessarily so much more attuned to the realities of what it means to be distanced from one another. Um, what can we do using XR, um, to bring ourselves closer, to make us genuinely more connected with one another. And by that, I mean more responsive and more responsible to one another. Um, so again, that’s a pretty big aspiration and our friends and colleagues on the technical side, you know, the, the folks who actually make and study the making of, of these objects of this work, um, you know, we definitely need to be in conversation with them. So I guess that’s another part of it. I’d like to see a really robust partnership between folks with really different orientations, really different perspectives, um, as always, uh, you know, a diversity of experiences and viewpoints is really going to be critical to doing good things in this space.

Jeremy Nelson (22:50):
I couldn’t agree more that’s Oh, this has been great. This has been a fascinating discussion, you know, we’re, we’re always looking to continue the conversation, you know, who else do you think we should be talking to? What other areas would you like to learn more about as we continue our podcast?

Sara Blair (23:05):
Oh, wow. That’s a great question. Um, so I would love to hear from undergraduate students who are XR natives, who’ve grown up kind of with, or in XR one way or another, what’s still exciting to them, or what’s newly exciting to them about this medium about this space. Does it bring users with that orientation into a shared world? Does it bring them into connection with other people? Um, what, what’s the, the ongoing kind of allure there. And I’d also in a very different way, love just to hear more from people who have been working as you have, um, as you know, um, folks in the working group and, um, at XR I had been working to create spaces for conversation about this. Where can we best kind of convene to kind of mull over these questions together? So I guess those are my thought questions, looking ahead.

Jeremy Nelson (24:01):
Love it. That’s great. Yeah, no, that’s great. This is, this is wonderful. Thank you so much for your time today. I really appreciate all of your thoughts and the pioneering work you’ve done here that helps for XR, Michigan.

Sara Blair (24:12):
It’s a pleasure, Jeremy, very happy to be talking.

Jeremy Nelson (24:22):
Thank you for joining us today. Our vision for the XR initiative is to enable education at scale that is hyper contextualized using XR tools and experiences. Please subscribe to our podcast and check out more about our work at https://ai.umich.edu/xr.