Jeremy Nelson, Director of the XR Initiative

In this week’s MiXR Studios podcast, we explore diversity, race, ethnicity, and gender in terms of embodiment in VR. We talk with the esteemed Lisa Nakamura, the Gwendolyn Calvert Baker Collegiate Professor in the Department of American Culture and the director of the Digital Studies Institute at the University of Michigan. Lisa talked about her work in exploring identity tourism. We also talk with Chris Quintana, an associate professor of education at U-M to learn about how he and his colleagues in the School of Education think about XR assessment. Lisa and Chris talk about their XR experiences in the humanities and how it is possible to create an XR classroom to study affordances in society and culture.

Lisa talks about how deliberate she was in constructing the course in terms of balancing the time spent in VR, which titles to explore, and the physical constraints of the lab. She designed a space where students worked in teams with one student working as a spotter for the other while they were in VR. This created a more engaging environment for learning than Lisa had anticipated. 

Chris discussed his experience with students and how some could easily buy into inhabiting a person in VR while others couldn’t go there. He stressed people must think about VR as a tool and not as a one-size-fits-all solution. We discussed how the mission of the University of Michigan is to help students tolerate diverse ideas and how VR can be used to explore these experiences in a different way. While the technology doesn’t teach diversity in and of itself, it can be used as a launching point to create a dialogue about the experiences and have discussions to share different perspectives. It is important to structure your course where the students can develop relationships and connections prior to meeting in VR or connecting in social VR platforms. We also discuss the evolution of digital social platforms for learning from Animal Crossing to the current state of VR.

This was a fascinating discussion and such a unique approach to using VR to teach a course and how to evaluate where best to deploy these technologies for different schools and programs. Please share with us what you would like to learn more about in the XR space at [email protected].

Subscribe on Apple Podcast | Spotify 

Transcript: MiXR Studios, Episode 5

Jeremy Nelson (00:05):
Hello my name is Jeremy Nelson, and today we are talking with Lisa Nakamura who is a Gwendolyn Calvert Baker Collegiate Professor in the Department of American Culture, and the Director of the Digital Studies Institute at the University of Michigan and Chris Quintana an Associate Professor of Education at the University of Michigan School of Education. We are going to be talking about XR in the humanities and bringing a multi-disciplinary approach to teaching and learning. Coming up next in our XR podcast.

Jeremy Nelson (00:52):
Welcome Chris and Lisa, thank you for joining us today. We’re really excited to have you join our podcast.

Lisa Nakamura (00:59):
Hello?

Jeremy Nelson (01:01):
How are y’all doing.

Chris Quintana (01:02):
Good.

Lisa Nakamura (01:04):
Well, yeah. Sorry. There’s some interruption. There’s three of us. Thank you for asking us to, um, talk to people about the XR projects that we’re doing on campus.

Jeremy Nelson (01:14):
Yeah, for sure. No, I think, you know, I was really excited to have you both join. I think you’ve, you’ve been pioneers in XR at Michigan, and I think you bring a unique perspective both from, from your work and your backgrounds, and I’d just love to hear how this all got started, like your journey through your first courses and how you both work together. And maybe Lisa, if you wouldn’t mind starting, and then Chris, you can jump in as we go along.

Lisa Nakamura (01:39):
Sure. Um, in winter 2019, gosh, it feels like so long ago. I think it was winter 2019. Um, I taught a course that was just one credit called virtual reality and empathy. And it was processed with American culture, which is where I hold my faculty appointment, and digital studies where I’m the director of this new Institute that focuses on the impact on society and culture of the digital shift, uh, which is very relevant right now. Of course, since we’re all shifting whether we want to or not. Um, and this course was offered as a 300 level course, kind of an experimental course for students who were interested in learning more about virtual reality and whether it could in fact help people to understand what it feels like to be in other people’s shoes. And also to understand what teaching with an NBR can help us do and feel that we can’t do and feel in a normal or non VR classroom.

Lisa Nakamura (02:43):
So this was a pilot project, um, along with two other courses, one in classics that was looking at great monuments and sites of the ancient world taught by Arthur Verhoogt and another one taught by Sarah Blair on the novel and virtual reality. It was kind of on the world building, um, and, uh, mine focused on diversity, race, ethnicity, um, gender, to see if students could experience what it was like to be in a different kind of body. So this was really an idea testing experiment, students learn a little bit about some of the science around virtual reality, a little bit about the history on this with kind of a media class. And it was small because we were in a language lab that had been converted to a VR lab really fast, but I have to say I’ve never had more enthusiastic students. Um, they were so happy and felt very, very privileged to do this. Um, and now that we’re in a moment we’re exploring all kinds of remote teaching. Um, I’m really glad that we tried it and interested to see where it will go now that we’re all at home.

Jeremy Nelson (03:54):
That’s fascinating. Well, how did you come up with the idea or how did you, how did this start?

Lisa Nakamura (04:01):
Um, I’ve been writing about race and the internet since 1995. So, uh, in 1996, I published an article on chat rooms and how people were creating avatars that were offered a different race or gender from their own. So I coined a term called identity tourism to describe what happens when people take on another person’s body, their gender or race, and have a false sense of understanding about what it’s like to be in another person’s situation. So, um, I have a degree in English. I’m not a psychologist or a programmer, uh, but this research, um, really kind of opened my eyes to how people were interested in socializing online and how they wanted to create new bodies and new cells and new ways of interacting for themselves. And, and that’s remained consistent over the last 25 years since that research. Um, so, uh, I wanted to keep on pursuing that line of thinking to see whether we could increase, um, interest and understanding of diversity.

Lisa Nakamura (05:06):
I also teach in ethnic studies, so I’m always ready to use any kind of tool, you know, any medium, any TV show, any, you know, novel to help students engage with difference and diversity in a meaningful way. And also, I was just super curious to see what this was about. I had never used VR or very little before then. Um, and the university was willing to help create the situation to do it in. So I was thrilled and also Chris helps, um, and was involved in the class a lot, which made a big difference to me because it meant I wasn’t alone.

Jeremy Nelson (05:40):
Sure, sure. Yeah. Chris love to hear your, your thoughts, perspective on this course and what your involvement was.

Chris Quintana (05:47):
Yeah. So as, uh, as this group sort of convened that Lisa was describing, uh, and these pilot courses were being developed, some of us at the School of Education and then were invited to join the group to talk about the assessment part and sort of how, how do we think about assessment and evaluation of these types of tools in a learning environment? And so we’ve been working closely with everyone, as they’ve been developing and running their courses to observe the course, to talk to students, to think about the, the different questions about XR, that the different courses we’re covering. Lisa’s course in a sense you could think about hers as looking at VR as the object of study itself, where the students were talking about VR, what can we do with it and critiquing it and thinking about VR in terms of empathy and these other questions. Arthur’s course on the other hand, was looking at VR as a tool within the classroom where let’s use VR to go see some houses in ancient Rome, because he’s talking about ancient Rome in the class.

Chris Quintana (06:56):
And so they’re all sort of taking a slightly different spin with VR, and we were looking to get a sense of what the students think. What did instructors think? What were the, the challenges and affordances of VR in the classroom and all to sort of get a better sense of what role can VR play within a given learning environment? How can VR sort of coexist with the other resources that an instructor might pull together all in service of their ultimate learning goals in class? So it was a, it was a really interesting experience. My research is in learning technologies. And so I’ve looked at all sorts of technologies and I tend to start looking at technologies when they’re kind of becoming more popular and starting to emerge. And, um, we hadn’t really looked at, you know, VR has gone through several waves, the first wave in the nineties. And then, uh, I think the idea has outpaced the technology and now the technology has caught up and we’re back to looking at VR. So this is a chance for, for me and others at the School of Education to look at this type of technology in more detail.

Jeremy Nelson (08:12):
Yeah, no, I love that. I love, that’s kind of uniquely Michigan to have this cross, you know, discipline, you know, folks from digital studies and School of Education and lots of different areas exploring that. I think that’s, that’s great. Uh, Lisa, in terms of like your course, what were some of the interesting learnings or experiences, maybe both from your perspective and then from the students?

Lisa Nakamura (08:36):
Um, we had to come up with some kind of off the cuff collaborative rules about how we were going to use VR in our class. So, uh, I didn’t want to have the students be on the headset all the time, partly cause I wasn’t sure what the effects of that were going to be, but also because if we’re talking about a reading, it isn’t necessary to have a headset on, you know, having the face to face engagement is often better for that cause people can make eye contact. Um, but we had, uh, students in headsets who were moving around and possibly hitting each other or hitting desks. So we had to have everybody assign a spotter to themselves so that they would be safe while they were using it. And this had some unexpected consequences. Students became a lot more engaged with each other and how to engage in more trusting behavior with each other than they normally do in a seminar.

Lisa Nakamura (09:30):
So, my classes are, um, when their seminar’s often small and based on a lot of interaction between students. So rather than decreasing interaction, I think VR increased it, but in a way we did not expect. Um, and what I’m still trying to sort out is how much the students retain from the experience. I think they were just amazed and enchanted by say, marching in, um, a civil rights protest during the Martin Luther King historical period, which was one title we use called I am a man. They were just, you know, really engaged with it and wanted to talk about it. Um, but we have some students who were not from the U S who, who really didn’t know who Martin Luther King was. So kind of situating these things a little more, so everyone can have an equal understanding of what’s going on. Um, but if we were to ask the students today, you know, can you give me some details about what happened in Selma? I don’t know what they would say. I think they would remember vividly bending over to pick up signs because they did that with their body. Um, but it’s, it’s still very untested ground, I think. So the jury is still out. It was only a one unit course and, um, eight very intense weeks. But the, you know, the main feedback from the students was that they wish it had been longer.

Jeremy Nelson (10:48):
Did you teach a course like this before? And did you try to pick subjects that already had content exists? It sounds like. And were you able to compare the Martin Luther King? You know, that’s a unique experience. Were you able to compare this to previous ways you’ve taught that or readings or watching videos and,

Lisa Nakamura (11:04):
Oh, that’s a good question. Like Chris, I teach courses on video games mostly. So that was my entree to this as well. I already had some of the equipment. Um, and we, we talk a lot about race, gender identity, and video games. And you would be surprised how hard it is to get students to play video games if they’re being graded, they’re not being graded on how well they play, but you know, what they write or remember about the ideas of the class about video games. So, um, it wasn’t that big of a stretch for me. I think the novelty factor is huge. And I do think there was more, um, bodily engagement, than there was for video games.

Jeremy Nelson (11:46):
Sure, sure. Right. Yeah. You’re physically moving around, especially if they’ve developed the content to interact, Instead of more passive. You mentioned not a lot of research or understanding around, you know, the longterm retention or where effectively, you know, VR XR technologies can be used to enhance education. Chris, what are you learning through your, your study of these courses and other research you’ve done and where do you think we can lead in this space?

Chris Quintana (12:13):
Well, I think what was interesting in looking at Lisa’s class is how I think, as Lisa said, the students were all very excited and very engaged to be using the tools, but as they use the tools, we started to see just the differences in how some of the students were, were looking at and thinking about the VR. Um, certainly in the example that Lisa gave where they looked at, I Am a Man. I think some of the students kind of did feel like they were part of this experience that was in the VR, but others just sort of couldn’t buy in so to speak, because there’s a point in the, in the VR experience where you look at your own hands and you’re seeing the hands of the person whose body you’re inhabiting, or you look in the mirror and you see that other person.

Chris Quintana (13:04):
And some of the students said, you know, I just, that’s not me. You know, like some students can buy in to the experience and sort of play along, so to speak and say, okay, I will, I will be this person. And others still had a bit of a difficult time doing that. Um, but yeah, we had some other examples. There was another VR app. I think it was six by nine. We looked at it in your class, Lisa and in six by nine, it’s it, it takes you into a jail cell in VR, just so you can feel how it feels to be in this cell, which is six feet by nine feet. Hence the title of the experience. And I remember one of the students saying how much, how much better it gave her a sense of what it was like. And she said, you know, I’d even, I’d been to South Africa and seen Nelson Mandela’s cell, jail cell as a museum, but you can’t go into the cell.

Chris Quintana (13:57):
Right. So you can see it and you get a sense of what it’s like, but she says the VR experience really helped me feel the constraining nature of the jail cell in a way that I couldn’t, even though I’ve sort of seen this other physical jail cell. So we are still seeing these differences. And I think, um, we have to be careful to not think of VR as a one size fits all kind of tool, but there’s just like any other tool, people are gonna think about it differently, use it differently. And I guess what we’re trying to get at is, what are the different feelings students have about these tools and maybe start to test out why did they feel this way? Why are some students able to, you know, have this suspension of disbelief and others aren’t um, and, and what is it about these, these tools that, that can help some students feel immersed and some students still not, you know, they’re, they’re sort of leaving it at arms length.

Chris Quintana (15:00):
Um, so I think that those are some of the interesting things. That’s what was really interesting about Lisa’s class and exploring these notions of empathy, where I think the jury’s still out as to whether VR can really make someone feel empathetic and beyond that can the empathy feelings that might be generated last, you might feel some empathy in the moment, but will that feeling sort of continue on beyond the, the, the use of the VR? Uh, so I think these, these instances in, in how Lisa and others use VR are giving us these different takes into these types of tools that help us understand the tools in different ways than might be traditionally seen.

Jeremy Nelson (15:46):
Right. It gives that different perspective from, let’s say, a STEM focused field where it’s very, uh, targeted in terms of what, you know, learning a skill or an outcome. And, um, yeah, and I think that’s, you know, from the XR initiative standpoint, we’re very interested in learning, you know, where best can these tools be used to help supplement or enhance the learning for students teaching, right. It’s another technology, right? It’s another tool and the more we can understand how best to use it or how to support students, you know, from either preparing or onboarding or ramping up to that. Um, there’s, there’s still a lot to be learned there and explore it. I think Lisa, just in terms of the course itself, was it a, you know, you explored diversity equity inclusion, was it a very diverse group or was it, you know, how did the makeup of the class look in terms of the people that signed up?

Lisa Nakamura (16:40):
Well, because it was, it turned out to be such a popular class and there was a massively long waiting list. It was mostly seniors because they get their first pick and it was not especially diverse. And I think it was probably somewhat less diverse than the student body. Uh, but there were some intense conversations around what, um, diversity is, what racism is. And I wouldn’t say they were intense and uncomfortable, but they were conversations. Students were not having elsewhere. It was pretty clear. So I think given how divisive our society has become around these very issues, some people call it political correctness. Um, having students really think about how the digital sways, their opinion, just ways they’re thinking about what it’s okay to say or do in these environments, I think is really crucial. Um, you, might’ve heard about zoom bombing where people’s zoom meetings are getting, you know, attacked, controlled by people who just put out a bunch of racist and sexist crap and, you know, knowing how to be safe online is important. But also the mission of the university is to help students to tolerate diverse ideas and to really delve into them and to make their own decisions, um, based on scholarly work and, you know, research that we do here. So, um, uh, I also have written scholarly articles about VR and race in particular racism. Uh, so I think that the reason the students were engaged is that they really do want to understand the experiences of people very unlike themselves. And this is, this is one way to do it. Um,

Lisa Nakamura (18:23):
in the end it was still somewhat of an open question, I think, as Chris was saying, some people did buy in and thought, Oh, I really feel, you know, like I did this or like I now felt a little bit of what the racist experience was or what it’s like to be a different person. Um, and other people do not. But I think what was valuable was the conversation about that.

Jeremy Nelson (18:45):
It makes me think back to, you know, we had Courtney Cogburn come to campus earlier this year and, and talk about her work and the thousand cut journey. And, you know, I had seen some people go through it that didn’t have that chance to debrief and have a conversation. And it just kind of that whole experience is very impactful. And if you don’t have that component of the discussion and the interaction, I think it leaves, it can leave you to make certain assumptions or decisions that maybe you wouldn’t want the person to do if you created that experience.

Chris Quintana (19:18):
And I think that’s why we’re so interested in, when we look at VR, we ask, what role does the VR play within your learning environment? Because I don’t, I think we need to be careful sometimes that people look at VR as if it’s the thing that’s going to cause learning. And sure, there may be some instances where you’re looking at representations of some scientific phenomenon or something and where that might be more the case, but in this case, it’s really, as Lisa pointed out, the VR led to other conversations and other discussions in class. And so it’s those connections where the VR could be the launching point towards this rich discussion. That is the VR’s contributing to other things in the classroom that all seen together can lead to the learning. And so we were very careful to try and think about that, the role and the fit that the VR plays within this, this ecology that you’re creating for your classroom of other discussion points, other readings, other resources that you bring in, then the VR is part of that, but used effectively, it can really connect with everything else to create this richer learning experience.

Lisa Nakamura (20:38):
Yeah, I agree. I mean, we did it half in person in a seminar room and half in the lab with headsets on, and I think would be really a bad idea to just jump in and try to do everything with headsets on, um, because the students needed to know each other to use the space effectively as a group. So we had one class at MET in a multi-user VR space Altspace, which would be a really fascinating place to explore at teaching it because it’s, it’s way more embodied than being in a zoom. I think people are zoomed out and want some kind of alternative. Um, but there were, uh, so this is all to say, I don’t think that would’ve worked well if people didn’t know each other already, it would have been hard to find each other. It would have been hard to trust each other. I think having the experience and analyzing it in the moment is much more effective than trying to talk about it if we’ve all done it separately.

Jeremy Nelson (21:33):
Yeah. Yeah. It’s that balance. I think, as we explore these remote teaching options of how effectively do you do that? And, um, knowing that everybody has a headset and it just recently Microsoft did their, one of their big build conferences. Uh, you could participate in Altspace Microsoft on Altspace and, uh, but you could also participate on your desktop. And so it was, it was just kind of fascinating to see the level of engagement and they have limited number of avatars you can pick and representations. So there’s just some interesting choices people make to how to represent themselves to your point earlier.

Lisa Nakamura (22:09):
Yeah. I think the popularity of Animal Crossing right now has got a lot to do with it. People are feeling shut in, they make a little avatar, they visit each other. It’s the closest thing to being social that you can get. So, you know, VR, as Chris said, it was kind of brought back from the dead because in the nineties it was going to be everything. And it just wasn’t, it was too expensive. It was too heavy. It was too complicated. And, you know, no, no, um, big tech company was really investing in it and people didn’t really know what to do with it. And then it got cheaper and smaller and now everybody’s doing something with it. Uh, and now the question is what to pick, like how do we engage with it? Um, but I think, you know, it’s coming back because of games and because of media actually, which wasn’t the idea the first time around. But I think if we stick around for other reasons which have to do with social distancing and with needing new modalities for teaching.

Jeremy Nelson (23:03):
Yeah, no, I’ve just seen, I’ve seen lots of interesting use cases, trainings, you know, corporate trainings interview recently, you know, people training how to go back to the workplace and be properly socially distanced and how to work in a factory and how to do high risk, high stress situations where you have to have some distance. So there’s some interesting use cases there. Well, in terms of, you know, we’ve, we’ve been talking a lot about what we’re doing here at Michigan. What do you both want to see us do here at Michigan to enhance the student education and kind of furthering the work that you all started?

Lisa Nakamura (23:37):
That’s a good question, I want Chris to answer this cause he’s seen every course and I haven’t.

Chris Quintana (23:43):
I like what we did with this work group where it, it really was looking at the use of VR in the humanities and the humanities classroom, because I think you said earlier, Jeremy, you know, in science and engineering, we certainly have a lot of use cases for VR. Um, we’re starting to see on campus, you know, pockets of how are the humanities using it? How has dentistry using it, nursing the art school, the music school. I think continuing with that, to get us to just get this sense of what are the different potential uses of VR. Maybe we see some fields where we realize, you know, it might not be that useful in this field and that’s fine, but maybe we start to see some interesting pockets of activity that we just hadn’t thought of before. I think when we think about VR for education, there might be a stock approach that we have in mind and anything that can help us sort of shake that up a bit and think about these tools in different ways I think would be, um, really useful. So I like this direction of trying to see how different fields and different areas on campus might want to explore these types of tools and, and not just VR, but not to forget about augmented reality, all the different sort of flavors of XR, and to really get a better, a larger sense of what can we really do with this stuff.

Lisa Nakamura (25:16):
Yeah. Um, I teach in the humanities and our campus is divided into two halves, which are three miles apart, North campus, which has our vis lab, which is where a lot of VR equipment is and then central campus, which is where people actually like to spend a lot of time. There’s more to do and there’s more to eat and more to, you know, more campusy feel, I think than North campus. So what I would like to see happen is for the humanist and people who are not on North campus to have access to the same cool stuff that people up North have. So I think there’s a stereotype that humanists don’t like math and they don’t like technology and they don’t want to use it. But I think that’s absolutely not true. I mean, they’re just as entitled and should have just as much access to the great new gear that everybody has.

Lisa Nakamura (26:03):
We Michigan produces new ideas and new products all the time. Like that’s part of what we do as research on campus. And I’d like to see that kind of spirit that entrepreneurial and experimental spirit circulate a lot more, um, in parts of the university where it doesn’t distribute itself as much, but I should add the provost office supplied, you know, the support and the funding for this project. I certainly wouldn’t have been able to do it without that. So, um, you know, the university I think is, uh, has been smart and trying to move into unexpected directions. That’s what Michigan is also good at. I don’t think we follow so much as try to lead in these areas. Um, and as you know, as we were talking about before, I don’t think we should do this without some really good measurement and some assessment because you don’t really learn anything. So partnering with education was crucial. Otherwise we would have done a bunch of things and gone on with our lives. Like we wouldn’t have gathered data.

Jeremy Nelson (27:00):
Yeah, no, that’s great. I mean, that was, that was exciting to me to hear when I first started like, Oh, we’ve already got this partnership in place and we’re already thinking about these things. And that was, that was a great foundation. This has been great. I mean, as we’re continuing to expand this podcast, you know, who else do you think we should be talking to? Or what other topics should we be exploring that would be of interest to the listeners?

Chris Quintana (27:24):
Oh, goodness. So much.

Lisa Nakamura (27:28):
Who is your colleague, Chris, who came with you to some of the interviews?

Chris Quintana (27:33):
Our, our master student.

Lisa Nakamura (27:34):
I think it was a postdoctoral.

Chris Quintana (27:36):
Well, yeah, our postdoc Carolyn has joined us on a lot of this work, Carolyn Juro.

Lisa Nakamura (27:42):
Yeah. And I think you might have to interview, um, Stewart and Jaylen. Um, Jaylen Thomas, I believe Jaylen was the person who actually took care of the equipment. He was the one who wrote on the content end and fixed it when broke and sat in the room and held the students because I think they were the tech support, not support, but even like the, you know, the tech infrastructure for the class. So if people who are listening are interested in what the nuts and bolts are, um, what it would take to teach with this stuff and how much time you need to charge it and all that. I think they really have some great hands on experience.

Chris Quintana (28:20):
They were crucial because they have so many of the practical lessons learned that makes something like this work. Then we, we, we learned so much from them too, about just how to make sure all the equipment works. And how do you, how do we do things in these interesting physical spaces that we were in? And I think sometimes we just forget about the pragmatics of how to use these tools, but if we’re going to use them effectively in the classroom spaces, their perspective is also very important.

Jeremy Nelson (28:54):
To think about this at scale or a large classroom. You know, there’s a lot of technical challenges that need to be solved and addressed. And you know, we’re very interested in that, so that those are great. I will definitely reach out to them and get them scheduled for a future podcast. I think this has been a wonderful discussion. I think you both brought great perspectives. I really appreciate the time. Thank you so much. Thank you, Jeremy.

Jeremy Nelson (29:23):
Thank you for joining us today. Our vision for the XR initiative is to enable education at scale that is hyper contextualized using XR tools and experiences. Please subscribe to our podcast and check out more about our work at https://ai.umich.edu/xr.