virtual reality authority • professor • author
Jeremy Bailenson is founding director of Stanford University’s Virtual Human Interaction Lab, Thomas More Storke Professor in the Department of Communication, Professor (by courtesy) of Education, Professor (by courtesy) Program in Symbolic Systems, a Senior Fellow at the Woods Institute for the Environment, and a Faculty Leader at Stanford’s Center for Longevity. He earned a B.A. cum laude from the University of Michigan in 1994 and a Ph.D. in cognitive psychology from Northwestern University in 1999. He spent four years at the University of California, Santa Barbara as a Post-Doctoral Fellow and then an Assistant Research Professor.
Bailenson studies the psychology of Virtual Reality (VR), in particular how virtual experiences lead to changes in perceptions of self and others. His lab builds and studies systems that allow people to meet in virtual space, and explores the changes in the nature of social interaction. His most recent research focuses on how VR can transform education, environmental conservation, empathy, and health.
He has published more than 100 academic papers, in interdisciplinary journals such as Science and PLoS One, as well domain-specific journals in the fields of communication, computer science, education, environmental science, law, marketing, medicine, political science, and psychology. His work has been continuously funded by the National Science Foundation for 15 years, and he receives grants from the National Institutes of Health, the Office of Naval Research, DARPA, various nonprofit foundations, and corporations based in Silicon Valley and abroad. He is the recipient of the Dean’s Award for Distinguished Teaching at Stanford.
Bailenson consults pro bono on VR policy for government agencies including the State Department, the US Senate, Congress, the California Supreme Court, the Federal Communication Committee, the U.S. Army, Navy, and Air Force, the Department of Defense, the Department of Energy, the National Research Council, and the National Institutes of Health. His book Infinite Reality, co-authored with Jim Blascovich, was quoted by the U.S. Supreme Court outlining the effects of immersive media.
He has written opinion pieces for The Washington Post, PBS NewsHour, National Geographic, Slate, The San Francisco Chronicle, and The Chronicle of Higher Education, and has produced three VR documentary experiences which were official selections at the Tribeca Film Festival in 2016 and 2017. His new book, “Experience on Demand” was published by Norton in January.
Drew Kugler: Thank you Jeremy for coming in today.
Jeremy Bailenson: Well, thank you Drew. It's a great to have you here at Stanford University. You are in the Department of Communication. We're sitting here in my outer office surrounded by these plants I've had for about a decade that actually need some trimming as I look around. And here in the Department of Communication we study how people use media. How media are great for us, how media may not be so great for us, what are the ways media interact with the brain, and in particular I run a Virtual Reality Lab. I've been studying virtual reality since the late 1990s. And what I do is I build VR. I test to see how it changes the mind, and I examine applications that leverage what makes VR special and and look at the use cases that can help people train and communicate and be entertained.
DK: OK. So somewhere there in the late 90s you came upon this this concept of VR; virtual reality. You have now written a book; currently out, or about to be out. Timing on that?
JB: Book came out on Tuesday [January 20, 2018].
DK: Tuesday, so really good timing around that. The book is called Experience on Demand. We'll get into a little bit of that as we as we go. And what I what I wanted to get into right away was, having read a little bit about your bio which you can fill in, VR was not always your thing, you said. You hopped into it in the late 90s. What was right before that that exposed you to this idea?
JB: So my Ph.D. Is in Cognitive Science from Northwestern University, in particular, Cognitive Psychology. And my dissertation was using mathematical models to understand how the mind works in terms of reasoning and forming categories and listening to arguments. I'd basically run experiments on people to see how some mental process worked and then we try to build a computer program that could simulate it. And I was doing that between '94 and '99, and, you know, two things kind of struck me about my role there. The first is I wasn't that good at it. There's a lot of people who are much better better at it than I was, and I just, you know—I was fine, but I—really my work there—wasn't unique or special.
The second thing is I just wasn't in love with it. I was working eighty hour weeks, as you have to when you get your Ph.D. And, you know, I liked the research process and I liked being in this kind of technology area; but I just hadn't found my passion. And so by a really circuitous route, the third kind of my third line of research at Northwestern, I was studying how people made decisions while looking at maps. So when why did why is it that we take a different route from A to B than we do from B to A. It turns out all of us have these ratty symmetries in our lives and we're trying to understand that work. I published a couple papers in that decision making arena. And because I published in this general area of spatial reasoning, in '99 I applied to do a post-doc, which is what professors—that before you're a professor—after you're a grad student—kind of a purgatory land—to use virtual reality to understand how the visual system and spatial updating worked.
And so I got an interview and I went to the University of California, Santa Barbara. And what became really clear to me is that this job was way, way over my head in terms of what they were looking for. They really wanted somebody that knew how the retina worked and that was really deep on the the inner ear system. It was really someone who was more biological in some ways, and to use VR to study how the brain works. But when I was there, at U.C.S.B. interviewing for this job that I wasn't going to get—which became very clear—I was having dinner with who eventually became my mentor, this guy Jim Blascovich. And we were out at a Chinese restaurant and drinking a beer, and he was a social psychologist which is very different from the cognitive land that I came from.
And, you know, we just hit it off and had a really great conversation. And ended up talking about all the ways VR could just change the notion of the self, could revolutionize the way social interaction works; because you can do all sorts of things we'll talk about later on. And we just hit it off and, you know, I had a really good time talking to him. And I got a call about a week or two later. I'm sitting in my room in Chicago in Wrigleyville, and I'm waiting for this phone call to come, because I want to get a job. And you know it's hard—academia—as many job markets as there are. And I get this phone call and it's from Santa Barbara. But instead of being from you know the guy I interviewed with, it was with from Jim, from Jim Blascovich. He said, “Jeremy I've got bad news for you, and good news for you. The bad news is you didn't get the job you applied for.” But he said, “How'd you like to become a social psychologist?"
And it took me about twenty seconds—kind of looking around—and I said, “OK. Let's do it.” You know, I was kind of at my ropes and where I was in my program and I wanted to change, and going to live in Santa Barbara on the ocean, learning about virtual reality sounded pretty good to me.
DK: Yeah. Yeah. So was that really was the—as you were sitting there in that twenty seconds—and I always—in this work—I always talk to people about the conversation in their head. You remember literally saying to yourself, “Well, it doesn't sound so bad?” Or what was the thing that picked you up 2,000 miles or whatever it was? And was it the virtual reality piece though that really grabbed you?
JB: You know I actually have a flashbulb memory of sitting against the radiator and looking out the window in my room, in Chicago, down at the courtyard—this inner courtyard—and I would literally remember being on the call. And you know, for me, in the world of psychology, for better or worse, there's kind of this cognitive psychologists who think that they're the cream of the crop. And I'm sure social psychologists believe the same thing. And so to me, what I had to think about was, do I want to leave at that time the status of being a Cog Psy person and go into a field that that I didn't know.
But Drew—I have to be honest with you—for me at where I was at the time, if you would have gone back to Northwestern University and polled all my colleagues, all the other grad students, “Who is the least likely guy to ever become a Stanford professor?” that would have been me at the top of their list there. I mean at that point in time—I figured that before I leave academia entirely, let's give a try. The coolest job building. VR in the most amazing place in Santa Barbara. And I said, “Let's just, let's do that, give it a shot. Let's do it.”
DK: All right. So what is then—if we begin to fast forward to where we're sitting—what was maybe a flash bulb memory or a conversation that got you to Stanford from Santa Barbara?
JB: So the book is dedicated to a guy named Clifford Nass. And when you come to Stanford as an assistant professor, they sign what's called a faculty mentor, and mine was Cliff. He passed away a few years back. And Cliff, if you haven't met him, so he's famous for a book you recall The Media Equation he co-authored with Byron Reeves. And it's this book about how people interact with computers and how computers are a lot more social than we thought. And Cliff—I dedicated it to him— he's that the kindest genius I've ever met. He was—in any room—the smartest person. But in any room he was just the sweetest person—just kind of falling, falling all over himself in order to help you. Just, just, just super special. And the conversation when I was a post-doc at U.C. Santa Barbara in the year 2002, we decided we were going to try to replicate one of Cliff's studies.
So Ciff does these studies on the politeness. So when you are in the same room as a computer, you'll actually behave politely towards it. His classic study is—if you do an evaluation on it on a computer is if you do a learning tutorial on a computer, meaning that computer basically teaches you something, online on the computer; if you then are asked to evaluate how good that program was, how good of a teacher it was, when you answer that survey on the computer you took the test from, you're actually more polite to it. You give it a higher rating than when you take it on another computer. And this weird, mindless template we have of being social actually carries out to computers. And that's one of close most famous studies.
We decided we're going to replicate that in virtual reality. And I also remember being in the room at U.C. Santa Barbara. I remember of the the conference call that we were on; it was myself and Andy Biel and a guy named (unintellgible), and we were designing this test. And Cliff Nass's a big, famous Stanford professor, and we'd cold-called him and asked him to help consult, replicate, a study using virtual reality. And to our surprise, he answered us and he showed up to this call. And I still remember in that conversation just how thankful he was and how just delighted he was that we were doing this, and how helpful and sharing anything—that he was just this amazing—just display of being generous. You know, he's again, he's a very famous scholar in the in the social sciences, and so he was—it was a very special conversation from from my standpoint.
And then; 2003 a job opens up at Stanford where they're looking for someone to do VR. And I said, “Huh. I know a guy there.” And that conversation helped me funnel the way. I thought about the application and it was it was it was one of the steps to getting it.
DK: Yeah, so I can't help but note at this juncture of the story, that your breaks, if you will, or your directions were largely determined by a conversation, or set of conversations with Jim. And then a set of these conversations and a experience with this heralded scholar; not in how smart he was, but in how he treated you. So if there's one thing that stays consistent in my work, is the best of the executives and professionals often lose track of that, and that's where they get in their own way. So small editorial on that, based on the story you've just told.
JB: So when I talk about this—sometimes I'll tell that story when I get rambly—I lecture sometimes for undergrads—and I teach a professional issues class here at Stanford where I help the graduate students think about how to be a professional, not just a student. And how to think about the job market. And one of the lessons is just, “Don't be an asshole.” It's just goes so far if you can just be nice to people. I can't stress enough that had I written off Jim Blascovich, because he was in a different field or because that's not what I was there to think about—I would—my trajectory be very different today. It's just you know—just be nice to people.
DK: Yeah, excellent; we're going to definitely keep that in for effect. So let's get then into this world that you have—at least according to one of the blurbs on the back of the book from Kevin Kelly—you you are quite the expert—one of the truly knowledgeable people in the world on this. My link to it, as we have said before, is the connection and the curiosity more than anything, of how is virtual—how does virtual reality—which you can of course I'm sure briefly explain how it works—how is it going to affect relationships in the future? That really is where I see my clients, with both curiosity and having to face that reality, so tell me that. How, how is what your discovering out there affecting relationships at work in personal life, etc?
JB: So Virtual Reality is an experience where basically instead of watching a movie or playing a video game, you're inside a simulation. In response to your body movements, you're perceptually surrounded; so wherever you turn your head you see things, sounds are spatialized. There's a virtual touch where if you touch an object. You get vibrations in your hand. For those that haven't tried it, it just feels like you're inside of a virtual place. It feels really compelling. So that when we do demos in our lab people they come in saying, “What is this thing? Is it like a computer?” And then they try it out.
And what they realize—the book is called Experience on Demand—because the brain treats it like a real experience. When you go in VR, there's no pixels and there's no field of view, there's no, there's no jargon; you're just doing something—and it feels like it's happening to you. How that relates to conversations?—I mean I could talk for hours and hours about it. So Chapter Seven of the book is called “Bring in the Social Back to the Network" and if you think about one way that media is affecting relationships right now is that the way that we're communicating with people's not nonverbally, it's not natural. Where we've gone back to this very antiquated notion of typing in very small telegram-like bursts the via text. And and one of the things that the VR is going to do is to allow people to be together and to maintain eye contact. And things that you don't get now over video conference because you don't have that intimacy, that connection; VR gets you there. It feels like you're with someone. You can literally reach out and touch someone—feel that that virtual touch.
And you know one of the quests I've been on for quite some time is—when you look back thirty years from now, and maybe even ten years, who knows as my hope and dream. Imagine that you're watching a movie of the 1970s, and there's a doctor consulting a patient. The doctor's got a lit cigarette in his mouth, he's blowing onto the patient that's there to be treated for something. We look at that and laugh. Not laugh, but we just what else can you do? Can you believe we used to do that?
Now my father works in the Bronx. He lives up in Northern Westchester. He gets in a box every single day for an hour— drives right behind a bunch of other people in boxes—arrives at work and basically talks on the phone and types on a computer for most of the day. And what I don't want to do, Drew; I don't want to get rid of what you and I are doing right now. Like, this is different than what we had on the phone or we're getting to know each other. I don't want to replace social interaction. I do believe that there's a subset of work that is just—there's a subset of travel that just doesn't need to be there. Maybe we commute to work three days a week instead of five. Maybe we don't fly across the world. You know I've literally—so I just—we're going to see some irony here—I flew all the way to Malaysia to talk about how you can use VR to combat climate change and improve conservation. And think of the fossil fuels that I spent to go all the way over there to give a one hour talk that you know just between us and the listeners, I guess I was very similar to talk it already given on You Tube that they could have just played. And obviously having me there really added something, but the VR is going to give you that best of all. It's feels like you're there and that person is there, but we're just going to not have to have to travel.
DK: Good. Well explain then, a little bit more, about the experience that the lab is creating and how that hopefully, apparently, is going to get out into the world. So I was here two weeks ago to meet you and tour the lab and got to do the experience; and it certainly as you say was a fascinating thing. What are you hoping to take out of the lab—of all those different kinds of experiences? What's the piece that you hope gets out to the—we'll call the average person?
JB: So let me talk about a few instances that have scaled up and then we'll get to the ones that I'm really hoping get out there. And so some nice examples are around training, not just procedural skills, but how to talk to people. And so one of the projects that we have worked on in the lab for quite some time is about empathy. If you go into VR and you look down and you see “whoa, my skin color is changed.” Or wow; I've become somebody who's sixty years old and I'm only twenty. Or if I go from becoming a male to a female; we call this body transfer. Psychologically it really feels like you become someone else.
The next stage in our experiments is that you walk around VR. There's a second person that gets networked into VR with you, and that person treats you poorly based on your race or your gender. And says things that are inappropriate or hurtful or even threatening. And so what we've been doing since 2003, is publishing papers that show that when you have this intense experience in VR of becoming someone else—walking a mile in their shoes—that in general, that tends to change your behavior later on. It tends to make you more helpful to others. It tends to change your attitudes. We published a lot of work on that.
A few examples of those academic findings now going out into the world. So one is with the company Fidelity. Fidelity has built, with full disclosure, I've co-founded a company called Striver. Striver's a company that uses VR for training. So this is Stiver's work. Fidelity—when you are going to work in a call center—you've got to be trained. And you know working in a call center is tough work. And that the decisions that the people on the other end of the calls are making—which is where do we put their money—are really important ones. And so what we've built for Fidelity is taking that empathy research and when you're in a call center training, you get to basically hit a button and magically get transported into the living room of the person the you're talking to.
So maybe as you now are asking questions of this person, you get to see how she's on crutches and she can't actually get into a car to go to the bank. Or there's a huge stack of bills on the table. And by getting a better understanding of the client, you then get to ask different questions. And so you iterate and get to repeat this conversation, kind of like Groundhog Day. And what we're doing is we're using a very low-level AI to basically guide the questions that you're asking based on the experiences you have when you show connection to the other person.
DK: Got it. Wow. And you also—the piece that I really do remember and has obviously been translated out into the real world—is helping with sports training. You've got to—I was noticing Joe Montana on the back of the book talking about the the big difference that VR Is making. And that tell your favorite anecdote. I'm sure there are many of where sports has been impacted by VR.
JB: Well. Probably this season—let's stay topical and relevant. Case Keenum the quarterback of the Minnesota Vikings, who you know had had some rough seasons before this season. He had been cut from a couple teams and some people call him a quote journeyman unquote. He took over 2,600 virtual repetitions this season. Meaning he put on the helmet, and what he would do is he would get to see pre-snap before the ball snapped. He'd get to see the defense look around try to recognize the defense practice, where to look, decide whether or not he was going to change the call, and then communicate that so he would basically get to relive extra virtual practice. And he took—to put that in perspective—he played a lot of games and took a lot of snaps, but he took twice as many snaps in VR as he did in the physical world. And by all expectations he had a stunning, stunning season that defied most people's predictions and expectations. Now look, he did it because of his brilliance, his talent, and his hard work. VR was a tool that helped a little bit.
DK: That's great. Great. Well, there is one two-part question I have for us to begin to conclude. I know you've got a lot to do and I appreciate all the time you've taken today, but let me split the question this way. When you think about what you see, what you've experienced in the building of this, the first question I'm wondering about it is what has been the most inspiring part of it to you? What has excited you the most? And then, what worries you about VR? You obviously have many depths of understanding of it. So what inspires you and what worries you?
JB: So let me start with the worry part, because this has been a sobering week for me. I've been doing a lot of media appearances about the book, and the book got reviewed by The New York Times and featured on N.P.R., and reviewed in a lot of places. And if there's a theme that's coming out across these reviews, and kind of I get a sense for zeitgeist through the people who are interviewing me, the book is getting critiqued for being too optimistic. I think there's a there's a growing concern about the role social media is playing in our lives and I—in the New York Times review for example—of the you know—Kathy O'Neill says it's great, she enjoyed it. But then spends five or six paragraphs saying you really need to stop being naive and think about all the abuse that's going to happen in VR.
And so now chapter two of my book is called “You Are What You Eat”, and it's all about the down sides of VR. And let's spend a few minutes talking about what those are. The first one that concerns me the most is distraction, which is that when you're wearing VR goggles you can't see the world around you. You're going to step on the cat. You're going to walk out into traffic, smash into walls. Sadly, tragically, there is a man in Russia, in Moscow, who died in December because while playing a VR game, he fell through plate glass table and bled to death. So they're similar to smartphones, which cause many deaths each year due to distraction. Distracted driving; VR is going to cause some problems there.
The second is addiction. If you think about the VR as the best party have ever been at. Every social media occurrence is going to be the best party you've ever been at. Pornography will feel like sex. Online gambling will feel like Las Vegas—it's we're going to have some issues about how to manage what you do in the world when VR gets that good.
The third is simulator sickness. So if you've ever used VR for a long enough time you kind of feel a little bit wonky. And that's not going away anytime soon, which is one of the reasons why I advocate you know about a twenty minute rule to take it off after twenty minutes, and that that's about enough. There's learning bad habits, so I'm very proud to have spent a lot of my time and effort helping our soldiers get better at their jobs. And I want our soldiers to be the best in the world at combat, but I don't want citizens to have access to the most amazing training simulations that actually give you the muscle memory, and also give you the proper skills to succeed at violence. So it's a different kind of thing when these intense experiences are in VR.
So we go on. they're are down sides to VR. Where I fall now as I come in and I'm coming to guidelines for how to think about VR. And you know today I'm going to—I won't say to the name of the company—but I'm going to of the three players in the VR space that are the big tech companies. I'm going to spend the entire afternoon basically telling them what I'm about to tell you, and I'm also doing the other two companies, and then in the next few weeks. So they are listening to this message because of the downsides.
Don't use VR for everything. Save it for things that, if you were to do an experience in the real world, it would hit one of four standards and I'll go over each of them. Impossible. Dangerous. Counterproductive, or rare and expensive. So let's start with impossible. Changing your skin color— that's impossible. It's hard to do that in the real world. These empathy transformations are something that work in VR, because it can't do him in the real world.
The second is dangerous. Where do we get we get VR? From flight simulators in the late 1920s, Link decided that I want to learn how to fly, but he didn't want to do it from a pamphlet. And so he invents the flight simulator. And that's one of the greatest—you know—lives are expensive. Planes are expensive. Let's learn when it's safe. And that certainly applies to all of our athletes. If Case Keenum in taking all those extra reps physically, he would be fatigued and maybe get hurt. So that's an example there.
Counterproductive. We do a lot of work on climate change. So if I'm going to show you say the Arctic or if I'm going to show you a reef that's been devastated by ocean acidification—if I were a fly every person from here to that spot to show them the devastation that would be a counterproductive way to teach climate change. But VR gives you the best of both worlds. It feels like you're there, but you don't have to actually destroy things to get there.
And then the final one is probably the bucket that most people think about, which is just super expensive things. It's expensive to fly to go see The Great Wall of China, but in VR you can do it more cheaply, and there are lots of applications there. You know you and I talked about—a few weeks back—we talked about high school debate and one of the the home run areas of VR that before the consumer revolution that you know even twenty years ago people were succeeding and making money is public speaking. And so it's expensive to fill a room with twenty or thirty actors to speak. In front of VR they are on demand. Any time you hit a button and you've got a crowd that you can dial up. How nice and how not nice they are.
DK: Got It. Wow. Plenty of applications. So is that what ends up being your inspiration that carries you through all the warnings that you put out? I mean how do you how do you continue to have the hope and the drive that you obviously do?
JB: So, first of all, I actually think the companies are trying to do the right thing here. I know that's a unpopular view right now, and maybe I'm naive, but I actually of the downsides I talked about you know I'm going to talk to them today and later this week. And I don't think the companies want to leverage addicted people, and I don't think they want to teach people how to do combat. Maybe that's naive, but I'm hoping that they embrace the good sides of this, while minimizing—I'm not totally naive—minimizing the economic forces that perpetuate the others.
DK: Yeah, interesting. Well Jeremy, it is always the hope on the show that interesting people share interesting share interesting interpretations of this whole notion of simple conversation, and you have exceeded by far any hope I have. Thank you for doing that. The book is called Experience on Demand by Jeremy Bailenson, and I was fortunate enough to sit and talk to you today. So—
JB: I really appreciate it. Thanks both for doing this and for saying that. Thank you.
DK: You're welcome. Thanks.