Jeremy Nelson, Director of XR Initiative

In this week’s MiXR Studios podcast, we have a special opportunity to hear Professor Sara Blair interview filmmaker Barry Pousman, the COO and Lead Producer at LightShed.io about his transformative work in VR storytelling at the United Nations and Discovery Digital Networks. Professor Blair taught a course in the fall 2019 where students explored the perspectives of VR and the novel, and the course allowed her students to experience some of the work that Barry created. In this unique conversation, we hear from two experts in the space of digital storytelling and how that relates to bringing about social change.

In this episode, Blair, the Patricia S. Yaeger Collegiate professor of English and faculty associate of American Culture and Judaic Studies, explores Barry’s motivations and entry into immersive storytelling to bring about social awareness and change. Barry discusses how he got started in VR storytelling back in 2014 when he gained access to a 360 camera and began to push the boundaries of what you could do to bring audiences more into the story versus a traditional passive experience. More recently, Barry worked on a project with Magic Leap to create an AR experience to explore the eviction and homelessness crisis in the United States called These Sleepless Nights. They created an audio-first AR documentary that explored how spatial audio could tell a story in a new immersive way. The AR documentary premiered at the Venice Film Festival and was a mixed reality experience where the audience explored a large cube with Magic Leap AR glasses by walking around and interacting with the story.

Sara and Barry discuss how XR technologies promise to allow people to experience a level of immersiveness in the story that is rarely possible. Barry shares his thoughts on how that promise always seems to be greater than what the actual technology can deliver and what it means to take the headset off and come back to the real world. They continue to share their perspectives on the promise of VR being the ultimate “empathy machine” and how that really has to be contained to a specific slice of a person’s life. It is impossible to actually walk in another person’s shoes without living their whole life as that person. That requires the filmmaker or the VR producer to have a certain responsibility to not create a situation where the viewer walks away thinking they know everything about that other person.

In one of Barry’s early projects, Clouds Over Sidra, he talks about his vision for creating documentary VR experiences and how the restrictions of the medium really drive the story. The recommendation is to limit time in Virtual Reality to 20 minutes or less, and that leads many VR filmmakers to create 7-12 minute experiences. It is difficult to do much character development in that amount of time, and Barry prefers to avoid mediated moments in short films and not create any manipulated images in the edits. Barry shares his thoughts on how the filmmaking industry has changed over the last 20 years to move away from cluing the viewer into the fact that they are in a mediated experience. He really tries to focus on the story and the social situation they are trying to convey versus reminding people they are in a VR headset. The more you remind people they are in the headset, the less likely they are to empathize with the situation or characters.

Sara and Barry discuss some of the future of VR and how social interactions in virtual spaces can create a more immersive experience than the Zoom video conferencing we have today. They discuss how to bring new people into the XR space and with the current pandemic how the desire for immersion and face-to-face interaction could drive some interesting innovations. Sara discusses some of her early work in Second Life and how those lessons could be applied to the VR space and for audiences of today.

We conclude our conversation by discussing what aspiring students can do to lean into this type of documentary storytelling. Barry recommends that students explore all types of VR experiences and see what they like and what they don’t like. Then think critically about what you as a student are passionate about or like to do. If that is cinematography, coding, audio, script writing, graphic or UX design, then start there. These experiences take a village to create, so look for other students or other resources like the XR Initiative at the University of Michigan to connect with students and faculty working in this space to hone your craft.

It was my pleasure to facilitate this thought provoking and engaging conversation between two experts in their respective fields explore how VR can be used for the next generation of storytelling. Please share with us what you would like to learn more about in the XR space at [email protected].

Subscribe on Apple Podcast | Spotify

Transcript: MiXR Studios, Episode 24

Jeremy Nelson (00:10):
Hello, I’m Jeremy Nelson. And today we are talking with professor Sara Blair, who is the Patricia S Yaeger collegiate professor of English language and literature, and a vice provost for academic and faculty affairs for the office of the provost at the university of Michigan and Barry Pousman, the COO and lead producer at lightshed.io. We are talking about Barry’s work with creating VR for the United Nations and Professor Blair explores how he thinks about his craft in this new immersive storytelling, medium coming up next in our mixer podcast.

Jeremy Nelson (00:59):
Hello, Sara and Barry. Thank you for joining us today.

Barry Pousman (01:04):
Hey, Jeremy. Great to be here.

Jeremy Nelson (01:07):
Yeah. I’m excited to have the conversation today and connect the two of you and, and really explore, you know, the work you’ve both been doing. And I think it’s going to be exciting followup to your earlier podcast, Sara.

Sara Blair (01:22):
Well, I am delighted to have the opportunity to talk with Barry and to be a part of this conversation. And I guess I wanted to start by noting Barry that your, um, wow. Your portfolio was really impressive. Um, you know, everything from your creative work as a, as a maker of, um, VR films, your, uh, importance as a, uh, a kind of forecaster and, um, someone really thinking about the future of these media, um, new media and digital media, um, someone who helps develop and implement digital strategies, um, uh, someone who has consulted with a whole host of very different kinds of, um, organizations and groups, um, thinking about how to make change and how to make knowledge and, um, thinking about what this technology means. So it really is a pleasure to talk with you. And, um, I really want to start by asking, um, maybe this is an easy question. Maybe it’s a hard one. What your most transformative experience of working with VR or XR technologies has been? Is there a particular project or collaboration that, um, has really made you rethink, uh, the power and the limits of VR work?

Barry Pousman (02:39):
Sure. Well, thank you, Sara. I think I’m blushing under my beard here. You can’t quite see it in this podcast recording. Uh, but thank you. Uh, and yeah, I mean, in a lot of ways, of course, every project that I have had the opportunity to work on has really has expanded kind of my thinking and expanded, um, our, our opportunity, Uh, to impact the lives of others. And so, of course, when I first got into VR, it was 2014. I had access to this magical camera that could, um, take audiences instead of just kind of that one way shared that one way. Um, you know, from the big silver screen down to the seat, uh, conversation to a more shared conversation where the audience is fully wrapped around, um, it has a film wrapped around them. So, you know, that’s one sort of big moment, uh, where things changed for me. But also recently I got to work on an AR, uh, project. I produced, um, an experience called These Sleepless Nights, which premiered at the Venice film festival and was made, um, really inspired by the, uh, the book Evicted by Matthew Desmond, um, about the eviction crisis in the U S and, and of course now more timely than ever this topic.

Barry Pousman (03:56):
Uh, and so that in that experience, one thing we really wanted to focus on, and I think part of the reason that Magic Leap was so interested in supporting the experience was that we really wanted to go audio first. And we went with this sort of audio first documentary AR documentary plan. Um, and then we back-filled the visuals. Um, we did do some scans, uh, during the, uh, process, um, but we did a lot, a lot of spatial audio work and that in kind of a whole new way made me just realize that, you know, once upon a time we had this, the canvas of sound was this kind of plate, right? We had this surround sound plate, you know, of course it started with, uh, earlier than that, but let’s just start there, right? We get surround sound. So you get the 4.1, get that center channel, you know, you have a subwoofer and when you design for that environment of a plate of sound, um, then that’s very different than designing for what came next.

Barry Pousman (05:00):
That Dolby atmospheric sound that 9.1, right? All of a sudden you have a top left top, right. Bottom left bottom, right. And so when a plane flies over your head, or when you’re standing on a bridge and there’s a river running below you, you know, those sort of things, when there’s that bigger sound canvas, all of a sudden, it’s just that much more immersive. The trick of the immersion plays so much better, uh, on us as humans and, and really as a designer, as a creator, as a, as a maker of this content, I mean, we’ve just exponentially changed our, the size of our canvas. All of a sudden we’re designing for a much larger space. Um, at the same time, the, the AR experience, wasn’t just technically AR it’s really mixed reality, um, where you walk up to this cube and you’re kind of navigating this film, like thing through glasses, with sound coming from all sides, you know, above you, below you to your, Oh, just over your shoulder sometimes right in front of you, you know, and really honing in on those sort of geo-specific sound cues.

Barry Pousman (06:07):
And then moving your way around this larger cube with your hands, feeling it tactile pushing in all of a sudden, our film just became so much, uh, I don’t know how to explain it exactly but much bigger than a traditional like rectangle or even a 360 wrapped rectangle. Um, this became where you’re sort of on top of the film, you’re a part of the film. And, and so in that way, it’s, it really was an exciting light bulb that there’s just, it’s just so much more, every one of these projects, you know, it’s like each time we learn a new thing, we realize, wow, what can we do with this spatial sound? What can we do with this spatial experience? Um, it’s it just almost, uh, it has, it has the daunting effect of, you know, Oh gosh, I gotta learn a whole new thing next time, but it also has the empowering effect of how much, how much further can we take this? How much more can we get out of these new mediums as they emerge?

Sara Blair (07:04):
So that’s a really fascinating place to start because it makes me want to hone in on a word that you offered up as kind of a keyword, uh, immersiveness. Um, the trick of the immersion you said is, you know, exponentially more powerful. So I wonder if we could talk about that for a little bit. I mean, immersiveness is the medium, specific quality of mixed reality technologies we could say, and very differently for VR than AR. Um, and, uh, how do you, how would you, um, describe your relationship as a maker of, of these films to immersiveness as a goal, as an aim? Is it the end of your experimentation in these technological forms? Is it a means toward a different kind of end; what’s the value of immersiveness in and of itself? Do you think, um, when you approach a project?

Barry Pousman (08:00):
Yeah. I think of sort of that let’s call it the scale of immersion, um, in the media, uh, or in the let’s, maybe it’s bigger than media, maybe it’s the project. Cause sometimes it can be blended experience where there’s some physical things happening, maybe, you know, a talk or something, and that surrounds the media part of it. Um, but yeah, I think that scale of immersion is I would consider it one of the levers for creating or fostering building, growing empathy, really for me, um, that immersion is a tool. One of the tools we have within VR within this immersive tech, um, it’s a tool to create deeper connection really, uh, that is so I don’t really see it as the end goal. I think it’s part of the equation that we’re always trying to work out, at least in my work, which is really about, um, fostering empathy and leveraging that empathy into action. So, um, so rather than thinking of it as like, Oh, I want to create the holodeck. That’s not really what, that’s not my driving force. I don’t necessarily want to create the holodeck. I want to leverage the holodeck for what it’s going to be for how it’s going to help humanity, how it’s gonna improve the lives of those that, um, that need improving.

Sara Blair (09:22):
So one of the, um, one of the, um, kind of histories of thought in the, the, on the scholarly side, on the, on the critical side, which is where, where I live, um, around empathy and the production of empathy is the argument or the belief that, uh, in order for empathy to become, um, realized to become actionable, to become present, um, in, in the social world, it has to be accompanied by or informed by some kind of critical distance, some kind of imaginative distance from the object or the problem that the empathy is intended to address. Do you see the increasing power, the full body, uh, kind of power of the forms of immersion that you and others are so remarkably experimenting with in creating, um, as at some level, does it, does it threaten to overtake the critical engagement or the, the takeaway is the thrill, the pleasure of that extraordinary immersiveness of a kind that most of us have perhaps not experienced before it’s still so technologically new, um, kind of appearing and happening right before our very eyes. Um, is there a danger that, that overpowers the, the result that you’re really looking to produce, which is a return to an actual social world, where there are very real and urgent needs and problems and inequities and so on that need our energy and our commitment to address.

Barry Pousman (11:03):
Yeah. Um, so I, I think of VR as in two different states, there’s sort of the current state of VR and the, probably more, I would think more sort of realistic trajectory of VR and, and just immersive tech in general. And I’m saying VR, but I guess, I mean, all immersive technologies. Um, but then there’s also this other state, this ideal future state and I I’m of the belief that rarely do we actually meet the expectations of our ideal future states for almost anything, you know, uh, whether it’s a spaceship going to, uh, you know, planets that are far-flung we have yet still to even, um, you know, end up in Mar on Mars where we’ve always predicted, we would be, uh, since the forties, fifties, probably in cartoons, you know, so, and we, and, and the same applies for VR with you know, it’s it, the, the promise of VR is, uh, is quite exceptional, but I believe that always we will have these physical limitations, the reality of VR will always be a little bit underwhelming compared to the promise of VR.

Barry Pousman (12:12):
And that’s not to say that VR is doomed to fail. I think that’s just the nature of all technology advancements, that the promise typically, um, lives higher than the practice. And, um, with that in mind, I don’t have a strong fear that we will not get that sense, that time for reflection that, you know, currently you have to take the headset off, right? Like the headset only can stay on your face for so long. So the sheer weight of it, the heat of it, the stimulation of it, I mean, there’s so much going on, um, that eventually you do take that headset off. And as you take that headset off, you reenter your own shoes, right? You step away from the shoes that you’ve fallen into in this story, or in this experience. And as you take that headset off, you step back into your own shoes and the hope is, and I think that, you know, you’re totally right in, in highlighting that, that time of reflection, that time of all I could think of it as applied perspective.

Barry Pousman (13:12):
So, you know, you take out, you’ve gained some new perspectives by doing the VR experience or the immersive experience you take that headset off. And all of a sudden you have a few minutes, a short time span the small window to really think about how you would apply that in your, in your current shoes, um, with the new knowledge that you have, the new insights you have of the life in someone else’s shoes. So I think I totally agree that that’s really important. And actually I kind of alluded to it earlier, but a lot of our work doesn’t just begin and end at the VR experience. Um, it really is a blended experience. It’s hard to, you know, help someone really critically think during an experience where they’re often passive. Maybe there’s some interactivity, maybe there’s some lean forwardness going on, but at the end of the day, you know, you’re kind of being rained on.

Barry Pousman (14:08):
And, um, it’s, it does require that kind of like conversation, those thought starters, those moments for deep thinking, um, that often, you know, things like books, we often forget that like really books are the ultimate empathy machine. Uh, the novel of course can put us into anyone’s shoes, um, if, if crafted correctly or well, and that, uh, it’s that does allow us to kind of take a break. We can back up from a paragraph and say, Hey, let me think about how I would take that into my own life, you know, and we get to have this more natural kind of step in step out of perspectives when reading a book, because you don’t have to take the full headset off the experience. Doesn’t have to provide a permission to exit in order for you to sort of think through how you would then address the issue.

Barry Pousman (14:56):
So I think that, you know, we could learn a lot from other mediums probably as we think about the future of VR and its ability to both immerse us, but then allow us to take those insights it back into the real world and okay. Yeah, 20 years from now, 40 years from now, we might have contact lenses with AR in them, but are people gonna need to take those contact lenses out? I think so, I think rarely will we find ourselves in a future where we’re just living in the Matrix and people have forgotten that they, uh, you know, have real clothes and real feet and real pains and, and, uh, and all the other things that come with being a human

Sara Blair (15:32):
I’m so glad you went to the novel. Um, uh, in part, because my own experience with VR began with my work as a scholar and a teacher of literature. And the course I taught in which VR, um, featured centrally, uh, was actually called the novel and virtual realities. And it was intended to be a kind of comparative historical study of both the novel and VR as technologies for immersion, for the production of empathy. And I just know that my students were actually, um, who were not all English majors by any means they were spread out kind of all across disciplines and, um, parts of the university and, uh, for a surprising majority of them, the most important and interesting things that happened when they were wearing headsets happened because there was a fail or a glitch. They really were fascinated by those moments when, as you say, the, the real state of the technology, um, maybe didn’t do what its makers had intended.

Sara Blair (16:29):
Um, and they were forced into a kind of awareness in the moment of, um, what the, uh, what the experience was, was like a kind of double consciousness almost, um, being in the moment and observing it as well. And, uh, I think that’s fascinating. I wonder whether at any point in your production of, uh, VR films or AR or mixed reality experiences, have you ever considered building in, or have you ever built in, um, and I’m sorry, I’m not remembering this from, you know, the work of yours that I know, um, have you built in a kind of, uh, self-referential, um, gesture toward, you know, the, the medium itself in order to produce some kind of critical awareness? Is that part of your set of strategies as a, as a filmmaker?

Barry Pousman (17:15):
It’s a good question. So I, what, it makes me think about, is this, uh, let’s call it a trope, a production trope within documentary filmmaking, traditional documentary filmmaking, which in the eighties and nineties was a very popular trope, which has since sort of moved on, we’ve kind of moved to a new aesthetic in documentaries. And that trope was a very self-referential trope. What it was doing was, um, you know, they had the person being interviewed would look at the director off camera and the camera would be next to the director. So as you’re watching an interview in a documentary in the eighties, I would say 99 out of a hundred, the person that’s being interviewed is looking off camera. They’re looking at that director. And that was a decision, of course, that was being made around, um, revealing that this is a produced, this is a production, you are watching something that has a director, it’s kind of a hat tip to, um, exposing what’s behind the curtain, exposing the workings of the theater, um, that are, that are being presented.

Barry Pousman (18:23):
And so, um, in the two thousands though, as, as YouTube became more popular, as social media in general became more popular, where all of a sudden there was this new sort of style, this new idea where instead of looking off camera to acknowledge that this is a mediated story, um, the subject, the person who’s, the subject being interviewed would actually be looking directly into the lens. And so for the past, let’s say 10 years, um, we have sort of seen this big shift for a way from the, uh, more exposed mediated, sort of the self-aware mediated experience to a understood it’s like, you know, now that we’re in this new generation, now that everyone can make a movie on their phone, now that everyone understands how webcams work in America, you know, all of a sudden we, we don’t need necessarily to be hat tipped that this is a mediated experience.

Barry Pousman (19:23):
We already know it is we know that there’s a camera in the room. We know that what that camera does that camera blinks blinks red, right? Like we’re all aware that that camera, that there’s no way that person is having this conversation without aware, being aware that there that there’s a production happening. And so it feels like there’s kind of been this transition of media savviness that, um, has permeated the landscape, the audience, the creator landscape, and that has just changed kind of how we do things. Now, I’ll say that I come from a digital kind of like shared media. How do you create the most organic, um, shareable content? You can, that’s really where I started my career and in media and, uh, that I often pull from what I’ve seen work in that space, which is, you know, those sort of like don’t mention, don’t mention the, the production involved, what I try to do.

Barry Pousman (20:17):
And, and you could, uh, plenty of VR films have after the film and the credits are rolling. You know, you get some exposure to see, Oh, here’s the film crew, making the things and behind the scenes. But really what we’re trying my work is trying to do is, uh, to make all that disappear. I want that to fall away in order for people to have a deeper immersion and a deeper sense of, um, connection with the subject matter and to, and, and have the focus be less, much less about fact that we’re in VR. The more that I think we remind people that they’re in a headset, um, the, the harder it is for them to then empathize with this situation going on in Indonesia or a situation that, you know, wherever it is, they might not be exposed to otherwise. So I try to separate those two things.

Barry Pousman (21:04):
And in some cases you really want to, you know, it’s, maybe it’s a prototype for an academic setting where we want to lean into, um, kind of exposing the values of VR in ways that aren’t just, uh, immersion that are interactivity that are, you know, all these other sort of things that it offers. Um, but in our work for, you know, like documentary work in VR and in, in AR, um, it’s since, and okay, here’s another sort of element in the equation. One element is the amount of time, the runtime of these experiences, you know, we’re all used to two hour long movies, hour and a half long documentaries. Well, all of a sudden, now these documentaries are 10 minutes long, sometimes seven minutes long. Um, you know, at most they’re 20 minutes long, I mean, Oculus guidelines even suggest please don’t make experiences that require sessions that would go on beyond 20 minutes.

Barry Pousman (21:54):
So, um, that, that is a diff a new constraint also being added to this media versus traditional filmmaking and, and definitely versus novels, uh, where, um, they could take a week to read the whole novel, you know, or longer. And so that, uh, that time constraint, I also think is just a factor where you can have less characters going on, you know, how much empathy can you have for five different characters in 10 minutes, you only got two minutes per character, right? So all of a sudden the kind of the opportunity for what to highlight becomes constrained. And so then you have to make choices and we’ve made choices that move away from kind of highlighting the fact that there’s a production, um, in our work.

Sara Blair (22:41):
So that’s fascinating to me and forgive me for wanting to go back to your, you know, really first the, that, um, kind of founding experience. You, you referenced a few minutes ago, uh, with Clouds over Sidra, um, in 2015 because, uh, I I’ve watched it many, uh, viewed it, you know, been in it as a, as a VR many times with students as well. And it struck me that, um, it was almost certainly technical limitations or just experimenting with, uh, VR filmmaking that led to the very short sequences, the, the scenes I’m doing air quotes here that you can’t see, um, are very, very short in duration. Um, and then the, the kind of complex orchestration of the fades and the segues between them, um, that felt to me as if I, I’m not sure whether it was intended or, um, maybe an unexpected byproduct of, of working in this, you know, very, very new medium, but that felt like an attempt to balance and awareness of the storytelling, which is after all VR storytelling, um, with the, of immersiveness or a sense of the engagement through the complexity of this young girl’s life and the lives of the other refugees in this, um, uh, Syrian refugee camp in Jordan, um, precisely by avoiding a conventional linear A to Z kind of narrative structure.

Sara Blair (24:05):
Maybe I’m over reading. I wonder if you could respond to that?

Barry Pousman (24:07):
No, I think, I think that’s dead on, I mean, uh, so I, I believe heavily strongly in, in the idea that we should be reflecting our story in our, in the craft. And so I do, I that’s slightly different than when I was saying, which is just that I tried to kind of help people forget that they’re in VR, but, and that they’re watching a movie rather than not a movie, but there are moments in Clouds over Sidra in particular, for instance, when the kids circle the camera and everyone is in slow motion, you know, that’s a moment that wouldn’t be a realistic moment. And actually, um, you know, these things are made by a village and, uh, I didn’t have the final say on every cut as, so, that I at the outset, I probably would have, uh, voted not to include a slowed down, a manipulated image for fear of revealing that this is a mediated experience.

Barry Pousman (25:10):
Um, however, what you’re describing is a bit more about sort of speaking to the larger story itself and kind of, um, you know, the, the kind of the chaos of the, of the edit kind of a non-linear onus of the edit and the chaos of this person’s situation, I believe there is a, a mirroring, a reflection that’s happening there. And, um, so I always try, we try to kind of find where those moments of, I don’t know what to call them idiosyncrasies, but they’re kind of like these moments of that would feel abnormal to most audiences in the West. You know, how do we kind of bring those moments in, into the edit? I think that’s actually critical probably for communicating in on multiple levels. What’s what the situation is. Um, so I think that is probably that is a yes, you know, kind of playing with that aspect of it.

Barry Pousman (26:04):
But as far as like the production exposure, I personally, my personal opinion is to where possible to avoid those mediated moments in a short film, in a new medium, where the plot is where the point of this project is to convince people that you stepped in someone else’s shoes. So there’s like a few things at play. And if it, if the goal was a different goal than I think, make different decisions for sure. Um, but I do always try and at least in some way reflect the, the tone of the situation with the tone of the, of the final piece.

Sara Blair (26:38):
So that leads to a question I’m really interested exploring with you, which has less to do with immersiveness or the medium specific properties of VR or AR or MR. Uh, then with those underlying assumptions and aspirations that have attached to documentary work, uh, whatever it’s medium for much longer than, than VR has been available. Um, and those assumptions about, um, using representation more broadly, and I’m especially interested in what you said about soundscapes and audio laying down audio first, because I think the assumption has most deeply been held with respect to visual representation, right? If we can see the experience of others, if it can be exposed, if it can be made visible to us, um, which is to say, um, Americans who are in a position to do the seeing and the observing and the regarding, if we regard the experience of others to borrow from Susan Sontag’s title, and we can know it and feel things about it that either will lead to transformation or that themselves, um, in the argument of some, represent some kind of transformation themselves.

Sara Blair (27:54):
And so I wonder whether, I mean, obviously your commitments to, uh, transformation and, um, creating the conditions for making real and invaluable change is very deep and longstanding. Um, how would you respond to people who might say that VR’s assumptions about documentary, about the power of documentary, uh, reproduce or extend some, um, beliefs or habits that may themselves be problematic, um, idea that we can know things about other people’s lives, you know, maybe it’s reading a novel that takes a week, or maybe it’s watching a, you know, a two hour film or maybe it’s putting on a headset for, for nine minutes. Um, what would you say to people who want to challenge the premise that we can in a meaningful way, come to understand something about, or feel something about the experience of, um, uh, of, of others, um, other to us that we would not otherwise know?

Barry Pousman (29:01):
Yeah. Um, well, I, I mean, this is, uh, uh, uh, critique, let’s say that I have heard often, of course, um, I sort of, I, I, this is how I look at it. I think that the idea of empathizing with others, the idea of empathy in general is actually rarely a destination. Maybe it’s never a destination. You don’t actually end up where you’d like, check the box and say, I’m empathetic to everyone now, you know, so because we realize that it’s a journey, um, that it’s a journey that, and it’s, you know, it’s muscle memory, it’s a way to, you know, you have to kind of reinforce that over time and, and sometimes you kind of lose a little empathy. And so as you gain more empathy, and so kind of thinking of it as more of a growth opportunity rather than a destination really helps frame, because I think that, you know, the headline that VR is the ultimate empathy machine.

Barry Pousman (30:01):
Well, that kind of is signaling that there’s some kind of like end goal, like, Oh, I’ve reached empathy state, you know, that’s now I have galaxy brain. That’s not exactly how empathy works, right. We really it’s a journey. And so VR can be a way to get us a little bit further down that journey, but of course we could never, truly, even in a novel, even in a series of novels, we can never experience what someone else’s experience really is. You know, that true standing in someone else’s shoes, true, gaining their perspective. That is impossible. That is just, you just have to be them. Right. And so if that’s the, if that’s what like success looks like, then we’ve set ourselves up to fail because success can never truly be on empathy. Can never truly be that we’re actually going to fully understand the pressures, the dilemmas, the challenges that happy, the joys that people, that other people feel.

Barry Pousman (31:00):
We just can’t really capture that. Now we can do our best. We can try hard. We can say, I want to take this slice. And if my goal is to, I don’t know, let’s say it’s to encourage a federal, um, you know, a federal law that allows for people that are being evicted to have, uh, a right to counsel, currently people that get evicted don’t have, um, right to, uh, you know, state council in the same way that people that are convicted of crimes do. Um, so that’s, uh, an interesting, weird kind of gap in the, in the legal system where it’s like, okay, we see that evictions are going crazy. Uh, you know, on the rise in a huge way, you know, there’s these, um, there’s a few States that are kind of experimenting with, um, new policies around that. And, you know, we think that if every state did it, that then all of a sudden that would help reduce this sort of, you know, spiral into homelessness, which we’re currently facing.

Barry Pousman (31:54):
And so, um, you know, if we go after it with that kind of end goal in mind and then work backwards to get there, I think that is, you know, a way to, so when as we work backwards, we’re like we’re using the idea of fostering as a small amount of empathy around a particular thing, because those people, you know, all these issues are intersectional. It’s not just a mom, she’s a black mom. She’s not just a black mom. She’s a black mom that has, I don’t know, alopecia doesn’t have any hair because she was a vet and now she’s doesn’t have hair. So she has so many things going on in her life that are all stacked on top of each other, that how could we possibly communicate those complexities in 10 minutes or in the length of a book? Um, you know, whereas we want to say, okay, here’s a slice.

Barry Pousman (32:44):
Here’s one way in that in as the creator with a goal in mind, I’m my goal is not to have audiences better understand that mother. My goal is to have audiences come away and think I need to take steps to make policy change. I need to take steps to reach out to my lawmakers, you know? And so like, I need to find a way to help these people in this particular slice of their life. And with that focus with that kind of honing of focus, I think there is an argument to say there’s value here, there’s society value, whereas, um, societal value, whereas like the, the notion, I mean, and famously famous famously documentaries went through phases. Um, I even want to say the first documentary ever, I think Nanook of the North, uh, is really more of an ethnographic study. You know, it’s just, these are the lives of people and here’s what they’re doing.

Barry Pousman (33:41):
And, um, and there’s been documentaries that are many hours long where it’s just kind of a slow pan for 10 minutes across the hillside of how those people see the world now, is that get, I gained more empathy through that experience or less empathy, or am I better connecting or, you know, I don’t know. I think that if we really think about how audiences internalize, um, and, and, you know, take that input and then, um, and then assess it today, audiences, aren’t going to sit around for a 10 minute pan of a countryside Hill. You know, I’ve already moved on, I’m checking my emails. I’ve gotten 10 texts since then. Like, I’m worried about what’s next, what’s on what’s for dinner, what are my kid’s doing. I got too many things to worry about that I’m not going to actually fall into a six hour long personal point of view perspective movie.

Barry Pousman (34:30):
So if I want to get my job done, which is moving audiences to action, I have to make critical choices. And I got to kill some babies. You know, we got to like, say, look, I can’t do this. Cause it got to do that. And I think, you know, all creators face that dilemma. That’s the sort of the crisis of choice that we’re stuck in. And if you can’t make those choices quickly, you end up being a painter. Who’s never painted anything. You know, you painted a lot of stuff, but it never ended, you didn’t finish it. And so those sort of like those moments where you decided this painting is going to be finished, I have to just finish this painting because you can always go further. You can always add more. You can always make it deeper.

Sara Blair (35:07):
Yeah, absolutely. I really appreciate your, um, uh, deducing, you know, sort of, um, articulating the complexities, um, uh, around all of this. And I, I have a different view of that, of course, as, um, someone, uh, who’s mainly doing this work in classrooms right? In, in University, College classrooms. Um, and that notion of a journey it’s that, that metaphor, that idea is foundational to, um, the way most of their students of our students, their own educational project. And part of my job is to have them step back and ask, um, a question that I think is beautifully summed up by James Baldwin and his, um, very widely read, uh, collection of 1985 collection of essays, which he titled the Price of the Ticket. So what happens when, um, uh, a given individual or communities, lifelong experiences, historical legacy experiences, um, become the kind of material or the starting point for my journey, you know, what’s the price of that ticket to others.

Sara Blair (36:15):
So there’s all that further work that the experience of VR enables us to do. And I would hope at some point that the, um, the, the kind of critical study end in the production end would, um, be able to, um, engage more as we’re doing here, to think about what that looks like from our very different perspectives. Um, since I’ve, since I’ve mentioned students, I, and I’m aware that I want to give Jeremy a chance to jump in as well, but I, I can’t help, but ask a question about how you would respond not to critics of VR, but, but to students who are really interested in the medium, maybe not as filmmakers or people with technical expertise or, uh, coding, or, you know, uh, experience making and producing, uh, XR objects, but students who are just interested in studying, using these new technologies to ask questions about this world they’ve inherited, which has so many opportunities for transformative action at this point, how would you encourage young people, you know, students to think about VR and AR, what would you want them to be curious about and what would you want them to be critical about?

Barry Pousman (37:29):
Yeah, I typically, um, so I speak with a lot of students and, um, and actually I speak with a lot of groups that are outside sort of tangentially connected, um, to VR that aren’t typically, you know, the kind of core VR audience. And it’s, it’s really, in my opinion, it’s about doing your research. It’s about, if you want to go write a book, if you want to be a book writer, well, you better read a lot of books. You better decide what you like about certain books, what you don’t like about certain books, be a critical consumer of the media that you want to create. Um, often I have had some crazy experiences where I’ve met with people that are, that have started VR startups that have organized VR communities, immersive tech communities that are famous names, you know, like literally people that, Oh, yeah, I know that person.

Barry Pousman (38:29):
I know that person. Yeah, yeah, yeah. They I’ve been in situations where, Oh, that person has never experienced a VR film and a VR experience at all ever in their whole lives. So I think we have this like very strange, because it’s so new. Uh, and because each time it’s the canvas, as I mentioned is just getting bigger. It’s like, Oh, I saw that 360 film, but I’ve never done anything interactive. Oh, I’ve done some interactive stuff, but I’ve never done anything beyond beat saber, you know, or I’ve, you know, and so like, there’s sort of still this, um, I don’t know, nascent quality of audiences that is a double edged sword, one, the positive side being, uh, easy to impress because it’s, uh, a wide open field. So as a creator, there’s a lot of opportunity because it hasn’t really been set in stone the way that let’s say cinema language, um, uh, can be a little more cemented.

Barry Pousman (39:25):
Um, but on the other end of that, double-edged sword is that your audiences don’t really know, you know, what, how to, how to understand your experiments. You kind of have to, in order to realize kind of where that convergence is, where, where that meet between what I want to say and what the audience will, will understand or hear, or take away with them. Like you have to do research to find out where those, where those convergence points are in your own self. And so that’s really, I think my biggest piece of recommendation would just say, do a lot of VR do go to a museum in VR, go to a talk in VR, go watch a concert in VR, play a thousand games of VR, watch a bunch of documentaries in VR, you know, like really get a sense of what, where the layers are, where the different strata are, Oh, this is an entry-level experience.

Barry Pousman (40:18):
This is a more advanced experience. So I could say for, uh, for example, like Clouds over Sidra, you know, that was our first experiment, but it was also designed pacing wise. It’s really designed for an audience that has never experienced VR before. Right? So we wanted to show it at the World Economic Forum, we wanted to show it at the UN to these world leaders. These are people that have never tried VR before in 2015. And so we it’s really crafted kind of slow you get, you get plenty of time to really realize you can look all around. You get acquainted with the technology, as you’re slowly hearing the story seep in and over time, are you in VR body of work change? The pacing changed because that same audience was seeing this new video, new video, new video, new video, and at the end. So all of a sudden they’re demanding more, they’re demanding a different type of thing.

Barry Pousman (41:11):
And so they’re demanding fast. They get that. They can look all around, they get already that they can, you know, whatever it is. So we don’t, we want to, all of a sudden, we’re kind of like meeting our audience in the middle. One way I think about this stuff, is it as a parallel to IMAX, um, where, you know, at first only movies like the Serengeti or sharks or whatever, and we’re done in IMAX and presented in IMAX, um, at science museums and eventually, um, we did start to see real movies. You know, the Avengers is an IMAX and all these other great big movies are in IMAX, uh, full length movies. Well, that took a few years too, for two reasons, both on the production end, it was hard to get the technology, right. It costs a lot. It was, you know, kind of fickle on that side, but then also the audience wasn’t ready, you know?

Barry Pousman (41:59):
And so it took time. It took time for the audience to realize they could see an IMAX movie that was going to be two hours instead of 20 minutes, that they could sit in a theater with so much, uh, stimuli. Right. And I think that there’s something going on in VR where we’re in that transition, where rules aren’t submitted for storytelling, where even just kind of the, the UI of it all isn’t cemented for audiences. So we are slowly kind of finding that path. And that requires research that requires critical thinking. And I think students are, they have the most time to do that research. Um, and so they’re kind of poised to be of course, the next experts, but really the, the people that are saying, I re I’m realizing the value here. I see why this is problematic. And that is an opportunity.

Barry Pousman (42:49):
Let me go see if I can double down on that opportunity and try it a few different ways and experiment and prep, rope, rapid prototype. And, you know, just let me just try one more. What if I tweak this line and, you know, using software, like, I’m just gonna say Sumerian as one, easy free one, um, which is an Amazon product, which just allows avatars to be, you know, you can voice them. So, Oh, let me just try it this other way. Let me try it this other way. All of a sudden it’s a rapid prototyping VR creator, you know, I’ve got this avatar, I can have them say this. I can have them say that we could say it this way or that way we can make them bigger or smaller. And really just like quickly play with the things that are working and not working. I think that’s where I would, you know, kind of suggest that students spend spend time.

Sara Blair (43:28):
That’s really interesting. Um, and I, if we had a lot more time, I had, um, invite us all to meditate on what it means. Uh, you talked about audiences needing to be ready and how we work with, we sort of teach ourselves the affordances and the possibilities of these, um, these new technologies as we kind of train or invite audiences into that, that same kind of work. And I think we’re in a moment where people are craving both immersiveness and face-to-face, and it’ll be really interesting to see how VR or XR technologies speak to those usually, um, uh, opposed, but maybe now newly related kinds of, um, I’m not sure, you know, I, I’m not sure I have any creative imagination about that, but I think it is a really important moment for that.

Barry Pousman (44:24):
Well, we have been, I mean, just to that point, we have been seeing more and more, you know, like a zoom plug-in into VR headsets and things like that so you can imagine, and the not so far out future where we do start to really have both live and this immersive world building going on at the same time. Um, you know, with, of course the constraints of now you’re in a rectangle Zoom. No, it’s not exactly like being a real human, uh, but, uh, but closer and closer, I believe, um, everyday here.

Sara Blair (44:53):
Right. And of course, as a historian of culture, um, among other things, uh, you know, second life, right? I mean, there are, there are media archeologists to do around this experiments that we tried that, um, might have taught us, um, some things about how to do this more intentionally, um, given the, the needs all around us. Um, Jeremy, did you have any questions you wanted to throw in here after hearing all of this,

Jeremy Nelson (45:19):
Uh, enjoying the conversation? I think, uh, Barry, you started to address it, but maybe just one last kind of thought for, for students that are wanting to get into the production side of this type of work, you know, obviously the research and trying it out and experiencing it and having a more critical mind about it is important, but like from the types of skills, whether it’s more traditional video production, audio production, what advice would you give?

Barry Pousman (45:47):
Sure, Sure. I think the easiest way to think about it is it’s, it takes a village to create VR and really to come at it with what you, the student, um, are most passionate about. If you’re a student who cares deeply about story, you know, then that is a great place to get started. If you’re a person that cares deeply about scripts or audio, or, you know, live action video, like, like camera work and cinematography, then that’s a great place to get started. If you are a person that wants to be a coder, if you want to, you know, you’ve already dabbled a little bit, you made a mobile game for your friends, uh, you know, how to use Unity enough, it’s free also by the way. Uh, so you can download Unity and just start playing with it. And, you know, if that’s the kind of person you already want to just start getting, getting coding, getting there, um, on the technical side, that’s another great place to get started.

Barry Pousman (46:36):
So I think it’s really a pretty wide open net if it’s like, if you want to get into movies, you know, there’s a lot of roles to play in making a movie. And so I think leading with your strengths, leading with your interests, um, will keep you engaged enough to get it to the next level. And maybe you realize, Oh, I don’t know how to do that coding stuff. And I don’t, and I don’t want to know how, and I’ll find another student who can help me get there, or I’ll find a faculty advisor, or I’ll find a grad student who I can work with, you know, whatever it is to kind of like keep moving that ball forward, but to come at it first with their passion and then say, I’ve gotten it to a good place. I’ve kind of, I have a clear vision and now I can onboard others to help, uh, see that vision come to light.

Barry Pousman (47:18):
Um, I think that would be my suggestion. So, um, you know, whether you’re a coder, whether you’re a filmmaker, whether you’re a writer, uh, whether you’re even a UI designer, maybe you’re just a graphic designer. And you’re like, I really want to think about the user journey, the buttons people press, uh, the, you know, the colors schemes that this needs to be like, that’s what I care about. So maybe it’s about finding other students that have already gotten, you know, that have already kind of got the story going, and I’ve already maybe filmed something or created something that you’re going to then, Hey, I saw you made this project. I wanted to reach out because I’m really interested in the user interactivity in VR. And so I’d love to collaborate with you, that kind of thing, um, to find out where your passions can be a value to a bigger project.

Jeremy Nelson (48:03):
That’s great. I love it. Find their passion and build from it. Well, thank you so much for your conversation today. I found it very enlightening and fascinating. So I appreciate it. Thank you.

Barry Pousman (48:15):
Thanks so much, Barry. What a pleasure.

Barry Pousman (48:17):
You got it. Sarah and Jeremy. Great to talk.

Jeremy Nelson (48:20):
All right. Take care.

Barry Pousman (48:21):
Yep.

Sara Blair (48:22):
Take care. Everyone.

Barry Pousman (48:22):
Have a good day.

Jeremy Nelson (48:36):
Thank you for joining us today. Our vision for the XR initiative is to enable education at scale that is hyper contextualized using XR tools and experiences. Please subscribe to our podcast and check out more about our work at ai.umich.edu/xr.