EP026 - Augmented Reality and Virtual Reality Deep Dive
EP026 - Augmented Reality and Virtual Reality Deep Dive
We have a new facebook page!
Augmented Reality and Virtual Reality Deep Dive:
A. Background - A look at what is driving AR/VR, what the differences between AR/VR are
B. The hardware players
- Microsoft Hololens
- Magic Leap
- Oculus Rift
- Samsung Gear VR
- HTC Vive
- Sony Playstation VR
- Google Cardboard
- Apple
- Amazon
C. Our take on the players and experiences
D. AR/VR and retail
- Vendors
- Use Cases
- Store Design - TMobile
- VR In-Store / Out of Home
- Google Project Tango / Aisle411 / Walgreens (Video)
- Modiface - Sephora
- Modsy - Home visualization
- AR In Home
- AR & VR Branding/Marketing
- VR In Home
- VR In-Store
Misc Links:
Episode 26 of the Jason & Scot show was recorded on Thursday, May 11th, 2016.
Join your hosts Jason "Retailgeek" Goldberg, SVP Commerce & Content at Razorfish, and Scot Wingo, Founder and Executive Chairman of Channel Advisor as they discuss the latest news and trends in the world of e-commerce and digital shopper marketing.
0:00 Welcome to the Jason and Scot Show, your source for the latest news and trends in the e-commerce industry. Featuring hosts Jason "Retail Geek" Goldberg, SVP of Commerce at Razorfish, and Scot Wingo, founder and executive chairman of ChannelAdvisor. Here is Jason and Scot Welcome to the Jason and Scot Show. This is episode 26, being recorded on Thursday, May 11th, 2016. I'm your host, Jason "Retail Geek" Goldberg, and as always, I'm here with your co-host, Scot Wingo. Hey, Jason. I hope everything is well with you and yours, and I wanna be the first one to congratulate you on making a half year of podcasting.
0:45 Hey, thanks very much, Scott. I feel like it's even a bigger accomplishment that you put up with me for half a year, so congratulations to you as well. Thanks. It's been a struggle, but I've, uh, persevered, and it's kinda proving my grit. I know our listeners are grateful. Yes. Uh, and they have… We appreciate their grit as well for hanging in there with us. Um, so to celebrate the half year anniversary, we have set up a Jason and Scot Facebook page. So for all of you that are on Facebook, it's, uh, facebook.com, and then you go and you search for Jason and Scot Show. Scott has one T. That can, uh, that can make us a little hard to find, unfortunately. Uh, my, I, my father apologizes for that. Uh, or if that doesn't work for you, you can use the direct URL which would be facebook.com/jasonandscott.
1:31 So it's all spelled out, Jason, and then the word "and," A-N-D, and then Scott, S-C-O-T- And don't forget to like us while, while you're there. And, uh, the other thing is tonight we are going to… We've been watching the stats of who's listening to the podcast, and what we have found is our d- deep dive episodes and interviews are very popular. So this week we are going to do a deep dive. Uh, and it's a topic that both Jason and I are really, really passionate about. Uh, in fact, this is a bit of a teaser show because Jason's gonna be covering this at Shop Talk next week in even more detail with a panel. Jason, you wanna tell everyone what we're gonna talk about? Absolutely, Scott. And I, I wanna start by dispelling the rumor that I tricked you into doing AR/VR this week so that you would help me do my homework to prepare for the panel next week.
2:20 That's absolutely not true. Hmm. It's a little suspicious, the timing, but, but I'm good with it. We are gonna talk this week about all things augmented reality and virtual reality. Yeah. It's, uh, it's one of my favorite topics. I think it's gonna be pretty interesting, and I look forward to hearing your thoughts. We've never actually… We've both spoken about it, but I don't think we've ever spoken to each other about it, so it's gonna be… I, I look forward to hearing what, what about this excites you so much. Um, so to keep it short, uh, instead of saying augmented reality and virtual reality every time, uh, generally in the industry you say AR, AR/VR. Uh, and here's kind of a high level of what we're gonna cover. So first we're gonna give, uh, everyone some background to make sure we're all on the same page.
3:02 What, what's driving AR/VR? What's the difference between those things? Um, and, uh, you know, why, why now? Why has this suddenly become kind of a hot topic? Then the second piece we're gonna cover is go through all the hardware players and just kind of make sure everyone's oriented on what's going on there. There's a lot of news to share there. Um, then we're gonna kind of talk about the, the different hardware players and some of the experiences that they offer. Uh, and then finally we're gonna land the plane with the podcast. We talk about e-commerce and retail, and we're gonna talk about how will AR, VR in retail, you know, intersect and, and what will happen there, and what are some of the popular potential use cases that, that you and I foresee happening?
3:45 So to kick it off, why, why do you like, uh, this topic so much, Jason? What's, what fascinates you so much? Yeah. Well, Scott, it excites me for a couple of reasons, both sort of long term and short term. In the short term, uh, particularly for retail and commerce, I'm actually a lot more excited about AR than I am VR. And so I suspect that may be a slight bone of contention between you and I. We'll have to find out as we go through the topic. But I am very bullish on a lot of the experiences that are available today or in the very near future around augmented reality to help improve shopping experiences, and I think there's, there's lots of opportunities for that that have me excited.
4:25 And then I am very excited about VR, but I'm much more excited about VR in the longer term. So, you know, I'm not sure retail or commerce is really gonna be one of the early popular use cases for VR. I'm a lot more bullish on, on it as an entertainment vehicle, but I'm absolutely convinced in the long term that it's gonna be the dominant entertainment vehicle, and so we're all gonna have to learn how to adjust all of our marketing tactics to talk to people that are primarily consuming media via this VR format Yeah, we'll, we'll all be locked in our basements just kind of receiving all the events and stimulus we could possibly need through our, our virtual reality sets. So it'll, it'll be nice. We'll never have to interact with humans and, and/or leave the house.
5:07 So it's a, it's a dystopian, exciting kind of future for those geeks in the room. Exactly, and that's why we are the absolute perfect two people to take you on a tour, given that that's pretty much how we live now. Yes, you're in the right place, folks. We're gonna, we're gonna take you right along here. Um, the, the thing I love about this is the, how fast fiction has become reality. The, the… One of my favorite movies, as you know, I'm a big movie nerd, so, uh, one of my top 20 movies is Minority Report, and in there, Tom Cruise does this kind of mind-boggling thing where he has a, a glove and he kind of pulls up a user interface in the sky and in the air and starts doing things. Um, that, that's kind of one of the first times you really saw this, this kind of conceptualized, and then, uh, The Matrix was kind of took it a step further, where not only are you in virtual reality, but our entire reality is a virtual reality.
5:59 And then even more recently, there's been some really cool special effects around the Iron Man movies with Tony Stark, uh, where he does a lot of really cool things with, with augmented reality and virtual reality. So that was kind of my first sample of it. I even remember playing, um, back in the '80s, there was these bigger arcades, kind of like a, a, a GameStop, uh, or a, a Frankie's, and they… Or do you guys have, uh, what's the one that's like two people's names? Buster and Dave's? Do you guys have those? Dave & Buster's? Dave & Buster's, yes. Um, uh, it used to be there was this one virtual reality game where you would, you would put on this giant headset, and it was so heavy it had to have like a coil spring so it wouldn't crush your neck, and you would shoot this pterodactyl, and it was, it was kind of amazing for the time.
6:43 But, uh, you know, that was kind of like the VR. So what I think's pretty amazing about this is how fast those, those experiences, that really basic kind of Space Invaders experience and then what we saw in the movies that was fiction have become reality. Um, and I don't think a lot of people, when I, when I demo this with people, I don't think they realize how good it is. I think they're… Everyone I demo it with that, that hasn't spent a lot of time with video games and some of that latest technology, they're pretty blown away by this stuff. It's kind of the only thing I've seen that kind of makes people scratch their heads even more than this is 3D printing, but we don't… That's, that's a topic for another show. So that's what I think is really cool about it, is it really is interesting to watch how mind-boggling this is to people when they try it out.
7:27 So let's jump right in, and maybe we should start by giving a little background and talking about why we think now is the right time to really talk about AR and VR. Yeah. Why, why do you think we're seeing this kind of explosion happen now? Well, I think that the big thing is that the price point and the technology h- uh, the power in the technology has finally hit that sweet spot where we can really create good VR experiences. And in my mind, what's really driven that is that a lot of this VR technology is really just a remix of the latest and greatest mobile phone components. And so the huge volumes that are happening in mobile phones are enabling the price points to really drop on some amazingly powerful hardware, and I think the smart engineers have been able to remix a lot of those components to enable VR Yeah, yeah.
8:19 To do VR, and, and going back to my pterodactyl example, this thing was also connected to the equivalent of a, you know, it wasn't a supercomputer, but it also was, like, not a single PC. It was something like eight Amigas that were kind of networked together just to give it enough juice to give you a, a very pixely pterodactyl flying around your head. So, so to make this work from a hardware standpoint, you need lots of CPU and GPU, so GPU being graphics processing. Today's phones have, you know, 1980s supercomputers times 10 in them, which is pretty amazing. Um, and then you also need a really high-quality display, and when you get to the kind of that retina quality of display and, and you put it pretty close to your face, which is what all these do, then you can't see pixels, which really kind of helps with that immersion.
9:03 The other thing is there's a lot of these components that used to be crazy expensive. So for example, the head tracking, uh, requires a, a gyroscope and a couple other components, and those used to be really, really expensive. But you, you, you know, you know that now these are all on your phone just so that when you pick up your phone and say, "Hey, Siri," that it reacts to you or, um, or even if for step counting and those kinds of things. So, so this kind of Moore's law has happened that has brought the prices of these components down to very reasonable kind of sub, sub $1,000 kind of prices for very high-quality components that didn't exist in even the last five years. Yeah, it's amazing. And Scott, in your mind, how fast do these things have to be to really catch on?
9:47 Are we there yet? Or, you know, when do you think we, we get there? One of my favorite charts I talk a lot about is, um, th- and there's several flavors of this that I'm sure you've seen several of them, but, but generally, if you look at the adoption of new technology over a broad period of time, like the last 50 years, what-what's happening is, and, and as humans, we're very linear, right? And, um, we wake up every day, we have our X number of hours, we go to bed, and that just has happened our whole lives. So when you live a linear life, one of the things that happens is you miss some of these things that are logarithmic. And what you start to see happening when you chart this out is if you look at things like the adoption of the microwave or the color TV, those took 20 years to get to kind of s- above 80, 90% penetration into homes.
10:31 And then what's interesting is when you look at, uh, as, as we started to get into cell phones, just general kind of dumb cell phones, not smartphones, then suddenly it went from k- to kind of 15 years. And then with, uh, the advent of the iPhone and smartphones, then you're looking at 10 years. And then these, these things have kind of caused even further acceleration. And then you start to look at things like, uh, Facebook and, and social media. Uh, and when you really clock that when it started coming out of college, now you're kind of getting into sub 10 years. And then if you just look at things like Uber- So, so, so you have these new platforms out there like, like smartphones, and then social media on top of that, and that's causing an even further acceleration.
11:16 And you look at things like Uber. Uber is really like three or four years old, and now there's more Uber drivers than taxis in New York, and an Uber vehicle crosses every road in Manhattan every hour of the day. So just the scale of these things is happening so fast, but as linear creatures, it doesn't feel that fast. Um, and I really think that AR/VR is one of these things, and I kind of put it out there with drones and 3D printing, that is gonna happen way faster than people are giving it credit for because, uh, I don't think our minds have caught up to how fast this kind of pace of adoption is happening. And we see this-- You and I talk to a lot of retailers, and these are retailers who have been around for 150 years, 150 years, and these consumer changes that are happening because of all these, these new technologies, uh, they're-- they have this logarithmic problem times 10 because they're used to these very, you know, decade-long patterns, and now we're looking at two-year o- patterns.
12:11 So, so I actually think, uh, the forecast I've seen, um, show… And, and as we go through the hardware players, you're gonna see if it's a multi-billion dollar company, uh, in the, in the world of software and hardware, they're gonna be a player in this field, and that's a lot of investment in R&D dollars going into this. So the, the forecast I've seen show it being as large as twenty-five million headsets by 2020. Um, and, and I tend to believe that. I, I think what we don't know and what's hard to predict is the use cases, so I look forward to hearing kind of your thoughts on that. Um, and then just generally, what do you think about the adoption? I, I'm obviously pretty bullish. I tend to agree. I, I think there's two things that make that twenty-five million feel easily attainable or surpassable to me.
12:55 I do think the entertainment use cases for this are going to be pretty big, and you look at the number of users on the most popular PC-based video games today, and, you know, it really would take only a small fraction of those… And I, I recognize there's a significant financial investment, but it would only take a small fraction of those guys to surpass twenty-five million on the high end. And then when you lump in that there's about a lot of really inexpensive VR, and I'm thinking particularly of, like, the Google Cardboard-type VR that leverage a mobile phone, and if you include those in there, it's, it's almost impossible that we won't surpass that number. I mean, I think The New York Times distributed three hundred thousand Google Cardboards in one issue of the publication.
13:39 Yeah. Yeah. It's pretty, pretty interesting and, um, yeah, so that's-- And then they, they also gave a bunch to schools, I think. Isn't that right? Yeah, I mean, there's, there's a ton of projects, and so-- And, you know, those costs are going down and down. I think McDonald's did a, a-- This might have been in Europe, not in the US, but I think McDonald's did a super clever Happy Meal where the actual Happy Meal box refolded into a Google Cardboard. Oh, cool. Yep, so lots of, uh, opportunities for big distribution. I think it's definitely the future, and that's why it's worth talking about now. But before we get too far, maybe we should just clarify for our listeners what we mean when we're talking about AR and VR.
14:21 Yeah. So, so augmented reality generally means that you are looking through a device. Usually these are, um, they're headsets. And, and I should back up. So, so there's kind of two models I, I like to think through. There's kind of version one of, of augmented reality, whereas you're usually looking through your phone. So you hold your phone up, and it uses the front camera. You see the front camera, and then as the image flows through your phone, it augments it by putting some kind of graphical element or doing something with what you're looking at. And I think of that as V1 because it's really here today. It's been here for the last three or four years. Uh, and since, like, the iPhone 5, you've had the compute capability to do that. Um, and that's largely AR. Then the next generation of, uh, kind of V2, let's call it, is augmented reality moves to more of a headset.
15:08 Google Glass was kind of the first kind of- instance of this, v- you know, I hesitate to call it that, but, you know, you could kind of look through the little lens and see the reality and maybe a little overlay or something. Um, but it, it was kind of maybe V1.5. It wasn't near where a lot of these things are going. Most augmented reality, what you're doing is you're looking through some kind of a headset, and you're looking through a lens, um, and you can see reality, and then it, it paints on top of it. So the s- the smartphone for the ones that we'll talk about comes out of the equation. VR is where you are in-- You- your, your vision is totally occluded. You can't see the real world, and usually there's not even a camera piping the real world through to you. Um, there are some hybrids, but just in general, this is, this is what we're looking at, and then now you're wearing a headset, and you're looking… Everything you look at is in the virtual world.
15:57 There is no existing reality. If you're in a room, you don't see that room anymore. All you see is the world that's being piped down to you, uh, and I think of that, those things as kind of V2. Okay, I think that makes pretty good sense. Another distinction that we hear talked about a lot is on the VR, uh, folks are starting to talk about what they call mobile VR versus desktop VR. So desktop VR would be all those experiences that have to be tethered to a computer to generate the, the graphics, and so, like, I think Oculus or VIVE would be great examples of desktop VR, and I know we'll talk more about those in a bit. And then mobile VR would be all the VR experiences that leverage your phone as the video display, and so the Samsung, uh, Gear VR or Google Cardboard would be good examples of mobile VR.
16:49 Yeah. Yeah, and it's pretty interesting that if, you know, to need a desktop today, it's gonna be a, a, some pretty serious compute power that you're gonna need, and, um, the two you mentioned definitely are, are kind of tethered to a desktop, uh, to some degree. They're, they're-- You can get Oculus off the desktop, but, uh, it, you know, it is pretty interesting to kind of think about that 'cause obviously you're not nearly as mobile if you're kind of tugging along a desktop with you. Mm-hmm. Yeah, I mean, uh, to some perspective on how much power is required, you know, there are a lot of Apple users that are really disgruntled that most of these, uh, experiences are not compatible with Apple hardware today. And one of the, one of the Oculus developers, like, flat out came out and said, "Hey, we'll totally support Apple as soon as they make some decent hardware that can support this."
17:35 Yeah, you mean like the, the MacBooks can't keep up, yeah. Exactly. Yeah. And a lot of it is, um, also being driven by the, the graphic processing folks, like NVIDIA is really big into this, and, and they, they've started to-- The latest chipsets are really kind of becoming optimized for these kind of applications. Um, and, and, uh, I know… So, so I'm a computer engineering person, and, and what's amazing to me is the amount of complexity in these things. So, so think about one of these, uh, and I'll just pick an augmented reality one. So, so let's say you're looking at your, your wall, and you wanna just kind of paint a scene on that wall. Um, even the V1 stuff is pretty amazing. What, what you have to do is in real time, you have to be able to map the room that someone's looking at, and that, that's a non-trivial episode, you know, problem, because frequently you're, you're getting just two-dimensional data from that scene.
18:25 So you have to figure out how do you get kind of either approximate 3D data or, or, you know, or actually get 3D data from the physical room you're in. Then you've gotta figure out in real time, what am I gonna paint where? How do I get the size right? Um, and then many of these things are animated, almost kind of Pixar-like. And what's amazing about that is, you know, as re- recently as eight Maybe five years ago, Pixar would kind of do these renders, and they would say, "All right, I'm gonna move this character five inches from point A to point B," and then the computer would just sit there, you know, massive server farms would sit there and render this kind of realistic animation for days and generate terabytes worth of data.
19:07 Uh, now what we're seeing is compute power and the algorithms have gotten to the point where when you look through these AR and, and VR scenarios, these animations are happening in real time and interacting with you and things like that. It's, it's, it's pretty amazing the compute power, the, um, the, the algorithms to be able to kinda get the 3D world created and then have the animations and things that you're looking at happen inside of that, know where you are, know where you're looking. Um, and then another area that's really fascinating to me, and I, I've seen several talks on this, is the 3D sound is pretty amazing because to, to pull you into these immersive worlds, they-- to fake, to fool the human brain, one of the best things to do is to, um, have these sound sources that, that are much more realistic so that if you walk by a waterfall or something, as you walk by, you get that kind of Doppler effect that, you know, let's say you're looking at it and it wa- goes by your right, you really wanna kinda hear that happen, and it's part of this immersion that happens.
20:01 And there's, there's tons of companies and technology just in, in figuring that out. Um, and in, in fact, there's, there's a dearth of experts on some of these things because, um, you know, there's, there's just hasn't been a lot of need for people that are really into 3D sound. So a lot of them come from the gaming world, uh, and they, they-- there's not a ton of people that are expert at solving some of these problems. Yeah, so now that we've got all the hardware power we need to really make this stuff happen, let's jump into some of the key hardware that folks should know about that's enabling those experiences right now. Yeah, and it, it probably makes sense to start with augmented reality because there's, um, there's more players in virtual reality, so we can kinda knock these out. Um, so, so again, using this kind of metaphor of V2, uh, this kind of next gen side, uh, the, the first one… And this surprised a lot of people, and, uh, the first indication they were up to something was this company acquired Minecraft.
20:56 And I know you have a, a very young son. Um, I have a nine-year-old daughter, and for anyone listening that has someone in the kind of age, mm, six to 12, they probably know all about Minecraft. It's kind of this little 3D Lego world, and it's a simulator, and you can build stuff. It's, it's, it's awesome for kids, and, and, and, uh, we could probably spend a whole show on it. Uh, but this company bought them for $2.5 billion. This is a company that, that they effectively had, I think, $100 million of revenue. Um, so that was a really nice outcome for them. Um, and everyone was kind of scratching their head, "Why did they do this?" And then they announced, uh, and it really blew everyone's mind that they were working on this. No one had any idea. They announced that they were working on an augmented reality headset, and this one used, um, holographs.
21:41 And, uh, most people probably haven't guessed the company now unless they're, they're pretty geeky about this. It's Microsoft. So this, this totally surprised everyone because Microsoft missed the phone world, um, you know, almost entirely with mobile phones. Um, they had kind of a, a bad acquisition with Nokia they had to unwind to some degree, and no one really realized they would be working on this. And, and a lot of people were skeptical. But as, as HoloLens has come out, um, w- it has really blown folks away. So, so this is, um… it's still very early days for this, and they just put out their developer edition. It is relatively expensive. It's about $3,000 Um, but the way this one works is you, you wear a headset, and it looks kinda like a very heavy pair of, um… Well, it wraps around your head.
22:26 It has, uh, sound in it, um, and it, it looks like pretty thick glasses. Um, and it has effectively a Kinect mounted on the front. And in the computing piece I was talking about, how do you get the 3D world? And that's what they do. They kind of… You have a Kinect effectively on your forehead, and you bounce beams out, and that's how they get the 3D world. And you manipulate it very mino- minority report by waving your hand in front. And then the aha moment is in one of the first demos, they kinda said, "Look, you can play Minecraft." And, and these things are holograms, which means effectively, um, as you look at a room through the AR, you can put a hologram on the wall, and then as you walk through the room, that stays there, and you can walk around the hologram and look at it from all three dimensions.
23:07 So in this demo, they, they take Minecraft, and they create a little TV, and they put it on the wall, and they're watching Minecraft, another player play Minecraft. Then they say, "Build a world," and it actually takes that model and puts it right into the AR scenario. So, so pretty impressive, and, uh, Microsoft has the HoloLens product. The other one that, that I find really fascinating is Magic Leap. Um, and, um, I'll let you n- kind of go through that one. Magic Leap is an interesting one to describe. First of all, I would certainly call it augmented reality. They don't like to use any of the traditional labels, and I think from a marketing perspective, they're trying to just create their own branded thing. So they like to call it mixed reality, and essentially, the technology that they're developing is a technology that takes light and projects that light into the iris of your eye.
24:00 And, uh, they're, they're able to blend the light that your eye gets from the real scene in front of you with the artificial light that they're generating so accurately that what your eye perceives are both things from the real world and things that are generated from their display system. And so it's this sort of mixed reality of computer images and real-world images that you see at the same time. And like a lot of VR tech, it becomes really hard to show people what the experience is without having them wear or try on the experience. So early on, they released these PR photos that would show things like a baby elephant inside of a little girl's hand, and the implication was that a little girl was looking at her own hands, but the baby elephant was this, this computer-generated image.
24:52 And later they showed, like- Flying elephants and, and whales jumping out of scenes and things like that. And those images were all faked because, again, there's no way in a, on a, on a website or in a PowerPoint presentation to truly have someone experience the live, uh, VR from their particular technology. But the reason that these guys are on everyone's radar is because they've never done a public demo yet, and yet they've raised a phenomenal amount of money. So they've now raised something like $1.4 billion, including $500 million from Google. I think they have an investment from Alibaba.
25:32 Um, I think a number of other, uh, notable folks have, have invested in the company. They have like a $4.5 billion valuation, so they're a, a bonafide unicorn, and no-- uh, the public has never actually seen the, the technology, and so most of us are having to take the company's word for how cool this stuff is. Now, I can tell you, I have talked to some people that have visited their lab and actually been able to experience it, and what they've told me is in a, a set of equipment the size of a room with very posise- precise positioning of where you have to stand and where you have to look, the experience they're able to achieve is, is utterly amazing.
26:16 And so, you know, at this point, what sounds like the, the hurdle that Magic Leap has to get over is to take that room full of equipment and make it small enough or wearable enough that you could have this, this experience more at a consumer level. And of course, as you described earlier, you know, things that used to took- take a room full of computing power just a few years ago, you know, now fit in the, the palm of our hands in our smartphones. So it's totally believable that they'll be able to get there. But that's basically, uh, the Magic Leap story, is this sort of mixed reality by focusing light into your eye Yeah, and they have, um, and, uh, now you mentioned it, I don't know if it's real or not, they have this mock-up of a game that's pretty amazing where, you know, a guy is in a room, and aliens start dropping out of the ceiling, and he picks up kind of-- he holds out his hand, and it creates a virtual gun, and he starts shooting the aliens and running around the scene.
27:09 Um, and then not to be outdone, shortly after that came out, uh, Microsoft, uh, you know, and they had done, already done the Minecraft one I was talking about. They came out with a demo that showed, uh, a game called, I think it's Robot War or something, Robot Attack, uh, and very similar where, you know, the, the person is looking at a normal room that they'd be sitting in, and they can just put aliens in there that shoot at you, and you have to dodge around and shoot them and stuff. So, so it's, it's pretty interesting to think about the implications of when you can layer this very thick digital brush over reality, what can you create? Absolutely. Um, so we're gonna talk about some of the use cases later, but let's change gears and talk about the VR hardware for a second.
27:51 And I feel like the VR hardware is, you know, what's getting the m-majority of the, the mind share in the, the media and press right now. So maybe, uh, Scott, what's the, the first VR hardware we should talk about? Yeah, the, the, the one that really was kind of the shot that heard around the world that kicked all this off was Oculus. Um, and Oculus is the product, Oculus Rift is the company, and it was all started by a Kickstarter campaign, um, to my knowledge, the most popular, uh, Kickstarter campaign ever. Uh, and this was a-an entrepreneur that decided, just like we discussed earlier, that, hey, component costs for these displays and head tracking and whatnot, you, you could actually hack together something, um, for under $1,000 now and really, you know, bring the cost of this down.
28:37 Uh, that quickly caught the attention of some famous game developers, like one of the founders of id Software, John Carmack, I think is his name. Um, he joined the team, and, and they quickly assembled a-- after this Kickstarter campaign, kind of a dream team, um, to put this together. And, uh Af- after the Kickstarter, what they did is they shipped the first version, which was called Developer Kit 1 or DK1 in the industry. Um, and that was in 2013. So this is one of those Kickstarters that actually kind of delivered and, and did amazing. And then shortly on the heels of that, they did Developer Kit 2 in 2014. Uh, and then the urban legend goes that, you know, Mark Zuckerberg, who's the, the founder of Facebook, saw a demo of this and said, "I'd, you know, I'd really like to buy it."
29:20 And they're like, "It'll be a couple years before we're out with a consumer edition." He's like, "No, I'd like to buy the whole company." So so, you know, he, he, um-- And then, um, you know, this is a company with literally near zero revenue, maybe 20 million from the Kickstarter. I, I don't remember what it ended up at, but, you know, very small amount. Um, they ended up, uh, you know, he was so convinced that this off, off of the demo that this was gonna be the future platform that Facebook acquired Oculus for two billion dollars. Uh, and if you read articles about it, you know, he basically said to the founders, "Look, you're gonna have a blank check to spend whatever you need to make this work in commercial for as long as you need it."
30:00 So, uh, it's nice to have funding like that. So, so since then, they seem to have accelerated and, uh, the consumer edition is actually shipping now. Uh, and you and I both ordered them, and I, I got my ship notification today, so I'm really excited. I have not played with a, a consumer edition. Um, I have played with the other developer kits. Um, the cost on this is the retail price is $599 Now, that does require a Oculus-compatible PC. Uh, it won't work with Macintosh's Mac- MacBooks or any of the iMacs or any of that kind of stuff. Um, and, um, so, so be careful. I, I know a lot of people may not… If you don't get a Oculus-certified PC, I think it will degrade the experience.
30:41 Um, and if you go to Oculus, they'll, they'll walk you through the, the ones that are certified. The key is getting this latest generation GPU from Nvidia. Uh, and there's a couple. You can kind of, uh, you're looking at an entry level probably of about 1,500 here on a PC. Um, and then you can spend up to 3K if you really wanna get the fancy, very high-end graphic cards. Um, and this is one of those few things, the more you spend, probably the better the experience will be, because this does require quite a bit of compute to do what they're doing. Um, in addition to the headset that you get, um, and it has the display and the head tracking, all that, uh, they do have these hand controllers, um, that you use. Uh, and the way it works is these things, um, they have tracking in them so that when you're in the virtual world, you can see where your hands are, and you can manipulate stuff, and, um, it's, it's an important part of the whole virtual reality experience to be able to do that.
31:31 So that is gonna be included. So 599, uh, is the Oculus, and they're shipping now, and then you're gonna need at least 1,500 bucks, so you're looking at probably $2,000 for, for this all in. Yep, and on a, a side note, is, I think it's also fair to say that that $2 billion investment from, from Facebook really sort of validated the whole industry, right? So that- Mm … that probably gave cover for a lot of other people to invest in the space and probably accelerated not just the Oculus development, but a lot of VR hardware and software development in the whole industry. Yeah, and n- not just cover. I think everyone was like, "Oh, crap, what if Facebook's right? You know, what if, what if smartphones-" You know, kind of is becomes, you know, what happened to desktop computers, smartphones did.
32:15 What if VR, AR, uh, happens to smartphones? So, so it's, it's not only giving them cover, it's giving them kind of the impetus to get off the dime and invest pretty heavily in this. And that's a perfect segue to the next piece of hardware we wanted to talk about, which is the Samsung Gear VR. When we talked about desktop VR versus mobile VR, the Samsung Gear VR would be a good example of mobile VR. So instead of using a computer with a really fancy graphics chip in it that is generating two discrete images, one that it sends to each eye, the Samsung Gear VR uses the display of your high-res Samsung mobile phone, and it splits that screen in half.
32:57 So half the pixels on the, the screen are dedicated to your right eye and half are dedicated to your left eye, and it uses some optics to refocus each half of the screen into the, the right shape for your eyes. And so all of the electronics you need to make, uh, the Gear VR is really in the phone, and then the actual Gear hardware ends up just being the mounting device that puts it on your face and in their case, some sort of accessory interface, so like headphones that send the audio to your ears and some stuff like that. And that allows them to… If you assume that the customer already owns the phone, that allows you to deliver these experiences at a much lower price point.
33:39 So the Samsung Gear VR costs about a hundred bucks, um, and it works with a variety of, of Samsung phones, and they've actually partnered with Oculus for the software. That's smart for a couple reasons. They leverage Oculus's expertise in figuring out how to author experiences and deliver content on the device, and I believe it's true that you can create content, uh, once that will play in both the latest Oculus consumer hardware and at a lower resolution play in the Samsung Gear. So publish once, use a couple places. Yeah, that's absolutely true. The, um, the next one we wanna talk about is the HTC VIVE.
34:21 And HTC is, uh, typically probably well known as, uh… They're either the number two or three Android phone maker in the US, probably number four if you start looking globally. So, so one of the, one of the big makers of handsets for Android. Um, and, uh, they surprised everyone, uh, in March of 2015 by saying they were gonna jump into this, and I think what surprised everyone was their partner. They've partnered with a company called Steam, and, um, if you're a desktop gamer, Steam is, is become the platform, kind of the app store for desktop gaming. Um, and it's a fascinating company we don't have time to dig into here, into it. But if you, if you're interested in company culture, you probably have read an article about Steam 'cause they don't have titles and no org chart or anything like that.
35:05 So it's very, very strange or, you know, unique company in how it's run. Um, but they already had this kind of engine for, um, distributing and buying, uh, desktop games. So, so it was interesting because you had this hardware maker kind of partnered with a game kind of distribution company, and, and they make games as well. And, you know, this one is squarely kind of going after the console gaming market. So it- it's very much more focused on not kind of a horizontal play, but more towards the gaming vertical. Because of that, it, it's a little bit higher end than the Oculus. So it's $799, and that does not include a PC. It also needs a high-end PC. They haven't really, um… They just give you a spec sheet, so they don't have kind of a, a, you know, a, a, a branding kind of thing like Oculus does.
35:52 But if you were gonna get a PC, the Oculus ones would be good to get with the VIVE 'cause it does require very similar kind of set of hardware. So, so you can almost view the Oculus Ready as HTC Ready if, if you wanna think about it. This is available now. Um, it is shipping. Uh, and they have their own technology that allows them to say they're room scale. And what that means is this one's designed for you to move around a bit. Now, you can't move, you know, you can't go running 100-yard dash or something with this on your head, but it does come with a, I think about a 50-foot cord. And the other thing that's interesting is you set up these two, uh, sensors, and they're, they're about the size of a Rubik's Cube, and you m- you have to mount them, um, at least six feet in the air and ang- aim them down.
36:35 What that does is it gives it a point of, a better point of reference for where you are in this 3D room space. Um, and then when you're gaming, when you're playing first-person shooters and stuff, you can move around within this kind of safe zone of a circle. Uh, and that, that enables you to have a couple more interesting experiences and not worry about hitting a wall. It also includes a, um, some really nice hand controls, um, that have sensors in them, and then you can manipulate your environment with it. So they're really big on this kind of branding around room scale as kind of their differentiator and a really big focus on gaming. And, you know, a- an interesting thing there, I had an opportunity, uh, about two years ago to visit this very cool place at Stanford University that they call the Virtual Human Interaction Lab, so VHIL is their sort of acronym, and it's run by this famous researcher in virtual reality, this g- uh, Dr.
37:27 Jeremy Bailenson, and he's showing off these cool toys that he's gotten investors at, and, you know, the Stanford endowment to invest in. He's got these VR headsets you put on that cost him about $45,000 to make, and he's got this room-scale room that you go in, and you conduct these experiments that can track you around the room, and the room costs, like, half a million bucks. And two years ago, he's talking about the Oculus Rift in development and this VIVE and saying that that hardware will absolutely match all of the hardware he has in his lab and essentially make his complete lab obsolete. And to his credit, he, he was more excited about that than dreading it because he felt like That they were making all kinds of interesting advances when they were one of the only people in the world that could play with this stuff, but they felt like when millions of smart people get access to this stuff that the pace of evolution will be dramatically faster.
38:22 The next piece of hardware we wanted to talk about is actually the Sony PlayStation VR. We've talked about how much graphics horsepower needs to be in these PCs to accomplish VR. The current popular game consoles actually don't have enough horsepower to, to truly do VR at the resolution of these other devices. And so Sony has really been the first game hardware manufacturer to step up to the plate and say, "Hey, we're gonna make new hardware that has the horsepower to deliver a VR experience." And typically in game consoles, they really like to get to attractive price points. So one of the things they've had to do is kind of strip down some of the nice-to-haves and some of the accessories on all these things and either not have them or sell them separately.
39:07 So, you know, while the VIVE and the Oculus are made with, like, very nice materials and s- you know, a lot of padding and leather and stuff, the Sony VR, PlayStation VR is, is likely gonna be a little more plasticky, and all of those hand controllers and things are gonna be accessories that you're gonna have to buy on top of the platform. But the Sony PlayStation VR is likely gonna be in that sort of when you put the, the necessary accessories in there, about a $600 all-in solution, which, you know, you compare that to these, like, six to $800 solutions that you then have to plug into a $1,500 computer, and that compares very favorably. So potentially, the Sony PlayStation VR will be one of the w- the ways that VR hardware gets democratized into a lot more homes and under a, a lot more families' Christmas trees.
39:56 And when, when I see pictures of folks using that one, that always reminds me of Cylons from Battlestar Galactica because the, the outside of the unit lights up and has some kind of a light animation on it. It's kind of totally unnecessary because they can't see it, but it looks good when they're demoing it, I guess. Yeah, and is it fair to say that most things remind you of things from Battlestar Galactica? Some things do remind me of KITT, so… Okay. This is, this is more Battlestar Galactica the way it's all set up. Got you. More Starbuck than, than, uh, David Hasselhoff. Got it. Yes, yes. So the next piece of hardware we wanted to talk about is almost a stretch to call hardware, and this is one that we alluded to earlier. This is another mobile VR experience, and this is Google Cardboard.
40:40 So we talked about the Samsung VR gear. Well, you know, if you dumb that down even more, if you say, "Hey, what's the bare bones h- uh, that I need to put a smartphone on my face and put a couple of plastic lenses in that to bend the light to each of your eyes?" You come up with this Google Cardboard product, and it's, it's literally just a mechanical thing with these two, uh, plastic lenses in it that allow you to put a variety of smartphones on your face and get a, a mobile VR experience. It doesn't sound exciting when you say, "Put a smartphone on your face." It's hard to imagine what could be more exciting than putting a smartphone on your face, but this is a super approachable way to let a large amount of people experience VR, and we've already seen some clever marketing tactics.
41:31 We, we talked about The New York Times bundling a, a Google Cardboard kit in an edition of the, the, their magazine, their Sunday magazine. Yeah. And, and on the software side, what you do is the cardboard's kind of the easy part, but on the software, um, there's a couple of things. You download an app, and, uh, it's a jumping-off point, and it's called Google Cardboard. And from there you can, um… They have a couple of kind of little App Store-like things, so you can download some content. But then also they have, uh… And all this is, is a link, uh, into YouTube, uh, and YouTube now has a VR section, uh, and there's a burgeoning amount of content there, uh, where you can kind of see YouTubes that have been shot with 360.
42:12 And what's cool about that is it's kind of, you know, there's… Whatever's being filmed is going on, and you can just look around that scene, um, using the Google Cardboard in a 360-degree view. It's pretty amazing. Some of the, the fun ones to do are, um, there's some guys that do wing walking on these kind of old biplanes. Um, there's, you know, tours of the Grand Canyon. And, um, you can kind of imagine that if you were, you know, you had, didn't get the opportunity to travel a lot or whatnot, you can, you can kind of go anywhere you want using this pretty inexpensive device. Yep. And Google Cardboard gave way to my favorite version of Google Cardboard, which is a, a version put out by Mattel. It's the latest incarnation of the View-Master.
42:55 So some of you that are Scott and I, I, my age might remember a toy from our childhood. Mattel had this thing called the stereoscopic viewer called the View-Master, and it… You bought a little disc that had images in it, and you put the disc in the thing, and you held it up to your eye, and you could see 3D pictures of different things. And so you could get, you know, a disc that had dinosaurs or Mount Rushmore or all these different things. And so Mattel cleverly said, "Hey, Google Cardboard is kind of the same thing. We're gonna make a new plastic View-Master that you put your phone in and that supports the Google Cardboard software." And so Mattel has sells this hardware, it's like a $30 kit, and they also have published a bunch of high-quality Google Cardba- Cardboard content that you can buy from Mattel.
43:43 And there's some, like, interesting, uh, learn geography, uh, content and some National Geographic licensed content and a bunch of cool things that you can get. And, you know, to me, it's a little bit of nostalgia from the, the View-Masters, uh, back in the day. It doesn't make that satisfying click and that, that kind of shuffle that it… They need to, they need to kind of add that back in. Yeah. At least the s- the sound effect that the View-Master used to make when you'd, you'd pull the trigger to change from one image to the next image to kind of spin that disc around. Per chick. Exactly. I remember that sound well. That would be cool. And then along those lines, we are-- we're seeing more and more rumors that Google likely intends to launch a more significant VR solution than Google Cardboard.
44:32 I think the, the rumors are calling this Android VR, which would be, uh, presumably some kind of mobile platform that, that leverages the Android OS. Yeah, what would be interesting there is Google has slowly lost control of Android, so, um, you know, their largest licensee is Samsung, who's pretty invested in this thing with their own device, uh, and partnered with Oculus. And then you also have HTC, which is another one of their largest licensees that's done the VIVE. So it kind of remains to be seen who's, you know… And I think this-- The rumors have pointed to this being kind of a mobile solution. So it's kind of like, all right, if Samsung punches out and HTC, you're kind of left with LG and some of the Chinese guys, um, and whatnot.
45:16 And, uh, the, the Chinese folks totally, you know, take Android down to the metal and build stuff on top of it. They don't, they don't do any of the standard Google stuff. Um, and then you have Amazon as a large licensee. So it, it, you know… So maybe it'll only be applicable to Nexus phones. So it's gonna be interesting to see, can Google kind of do much here and, and push this kind of in, in… They're a little late to the game when you think about it. These kind of really nice consumer editions are shipping, and here they are kind of, you know, maybe gonna put a developer kit out next week. We'll, we'll have to see. Maybe they'll surprise us and be pretty far along, but it's gonna be interesting to watch this one from a couple angles. Yeah. But that certainly will be one of the things we should, uh, pay attention to in the I/O conference coming up. Oh, yeah.
45:56 Along those lines, you can imagine all the other big technology giants very likely have to be investing in this technology, right? And one of the indications that we always get is, you know, you scour the job listings, and you see all these tech companies hiring people with VR expertise, and then, you know, eventually you see them starting to file patents. And so one company that hasn't announced anything in this space, but that's hard to imagine they're not doing something in that space, is Apple, and there are both some VR patents that we've seen Apple file and some hiring that we've seen Apple do. And of course, the easy speculation at Apple is that some sort of mobile solution would make a lot of sense given that that's currently their dominant product.
46:44 Although, who knows, maybe that's the next evolution of Apple TV for Apple. Yeah, the patent I saw was kind of a-- it was very kind of Gear VR-like, um, and it focused on how the phone detects it's about to go into VR mode and that kind of thing. Um, so yeah, that is logical. But you know, if I'm one of these companies and, and I think I just saw that Apple's R&D budget like tripled in the last year, and it was already, you know… It's, it's double-digit billions. They could be running… They could be doing all three. They could do-- be doing mobile VR, they could be doing desktop VR, and they could be working on AR, and it wouldn't surprise me if a company their scale isn't kind of taking a shot at all three.
47:24 The, the stakes are very, very high on this and, um, you know, I, I could definitely see them investing on, on kind of a all three approach to kind of make sure they don't lose out here. Yep. Along those lines, there's another company that we don't get to talk about on this podcast very much, but there's a bookseller in Seattle. Oh, wait, we do talk about them a lot, Amazon. And an interesting thing to speculate is what could Amazon be doing in VR? Like Apple, we've seen them hire some VR experts. We've seen them file some patents, and one of the interesting things about the patents that we've seen from them is they seem to have a different take on it. They're talking about building VR into a physical room, so literally like projectors or display technology that you install in your living room to enable a VR experience there.
48:15 And Thematically, that feels very similar to their approach on, uh, the intelligent agents, for example, where they put the Alexa and the Echo in the room as opposed to, um, it being something that you carry around in your mobile phone. So it wouldn't surprise me at all if that's the next thing that Amazon is thinking about conquesting is place-based VR as opposed to device-based VR, if you will. Yeah, that's the one that's most science fiction-y, and I, I really look forward to that. And, um, everyone poo-poos the Fire, uh, phone, which you and I are the only two people that own, I think. Um, but the 3D on that is actually pretty good, so you could almost see that some of the technology developed for that could very well be used in an AR or VR kind of a mode.
49:00 So we'll have to kind of wait for, for to see what, what Amazon springs on folks. Um, so, so this is a good time just to kind of summarize. So we have the eight hardware providers. In augmented reality, we have, um, Microsoft HoloLens and then Magic Leap, and then we have the six virtual reality hardware companies, Oculus, Samsung Gear VR, HTC VIVE, Sony's PlayStation VR, Google Cardboard, and maybe the Android, and then Apple and Amazon. So that, that's eight players, all of which are, are very large kind of, you know, multi-billion dollar companies. Um, which of these have you experienced, and what's your take on kind of the, the different hardware players we've talked about so far?
49:41 I have gotten to try on both the Oculus and the VIVE, and, you know, I haven't gotten to do a side-by-side comparison or spend a ton of time with them. From my perspective, the, the experience on both of them is similar. The, the huge distinction between those two to me is the room scale in the VIVE, which allows you to not just look around but physically move around the room, and I know that likely will be an option to add to the Oculus down the road. But those are the two I've tried on. I haven't personally tried on the HoloLens yet. I haven't gotten a chance to try the Magic Leap. But I'm gonna throw an odd plug in here that you probably weren't expecting. I'm excited about all of these technologies, and I think in the, in the short run they're all gonna be super interesting for entertainment content.
50:27 But in the short run, I actually think it's some of those boring V1 AR experiences that we talked about at the very beginning of the show that may have the earliest big commerce applications. So what… You know, we'll talk about that a little bit more when we talk about applications, but I just don't want folks to forget about them. Yeah, yeah, and it… They're definitely, uh, you know, pretty important to this whole thing and where a lot of the companies are, are… You can actually download this stuff and play with it. Um, so, so I've been fortunate and because When-whenever I tell people that I have this as a hobby, it helps a lot because I've-- it helps, it's helped me get on a lot of beta lists and stuff. So, so I started with Oculus. I have the developer kit one and two.
51:08 Um, really enjoyed those. Um, actually only had a Mac at the time to use them with, so they were a little bit kinda slow compared to other things. Um, and then here recently I've been very fortunate. I ordered a-- we submitted at ChannelAdvisor for a HoloLens, um, with a Vive, uh, just ordered a commercial Vive and an Oculus. And I, it-- The way I ordered them, I thought the Oculus would come first and then the Vive, and then who knows about the HoloLens. And actually, the HoloLens came first and then the Vive, and now I'm, should be getting the Oculus here any day now. Um, and I've, I've had a Gear VR for a while, and it's a really good entry into this if, if you're interested in it. Very low entry point. If you have any Samsung device, it's, it's a no-brainer, I think, to get… Spend the ninety-nine bucks.
51:48 It does make a big difference versus the cardboard. The problem with the cardboard is it does let a fair amount of light in, which kind of starts to erode the experience. Um, the other thing the Gear gets you is Oculus has spent a lot on this three hundred and sixty degree sound. So if you get the Gear VR and put some noise-canceling headphones on, it, it gets pretty immersive, maybe like eighty percent immersive, which is pretty cool. Um, and then, um, the, the one, uh, the Vive is really amazing. So, so I was able to set that up, and it, it's-- the setup on it's pretty sophisticated. It requires a high-end PC. Mounting these cubes right was not trivial. Uh, it took me a while, but it, it was well worth the, the payoff. The, um, the three-- The sound on it is amazing.
52:31 It is extremely immersive. It is, it is, it is very immersive and to the point where, um, when I, I and others have played it, uh, it's very easy to get quite nauseous with it, so you have to be kind of careful. They all have rollercoaster demos, and that's kind of one way to test if this is gonna make you nauseous or not, uh. Um, the one that's probably surprised me the most is we did get a HoloLens, uh, and it is really amazing. So you, you can-- You use hand gestures, so it's got that Minority Report kind of a UI, which is awesome, and you can see your hands and you're making these gestures. Um, but just the… It's got a whole Windows 10 PC in it and just the amount of stuff you can do. You can just kind of be sitting there and you can go stick a stock chart on the wall, open a browser, and then be playing a game.
53:17 And as you walk around the room looking at the normal world, those things are all kind of stuck there. So it, it is pretty amazing, and I definitely, definitely recommend. I think as they get quantities on these, they'll be putting them in the Microsoft stores, and I definitely, definitely recommend people try that. It does help a lot with the nausea problem. Um, I, I found more people have this kind of nausea thing than, than, than I do, uh, and, uh, it's pretty important. The other thing is, uh, I mentioned Minecraft earlier And they just launched Minecraft, um, through the Oculus platform for the Gear VR. And, uh, again, that's… It's, it's an awesome experience. And, uh, you know, if your kids like Minecraft, putting them in the world is… there's just second.
54:01 It, it requires a controller too, so it's about… You have about $130 investment, uh, if you already have a Samsung device. I'd strongly recommend people that are interested in learning more about this and want to spend a little bit more than cardboard, that's a really good investment to… and if you have kids. Um, the other thing I'll, I'll say, uh, before we go on is, um, a lot of adults really kind of struggle with the suspension of disbelief that you have to have in these things. Um, and a lot of people just put it on, they look around like a bird real fast to see how shaky it is, and then they kind of take it off. Kids just, like, flow right into this world, and, um, they, they… It's-- You have to kind of yank them out. It's so immersive. So you can definitely see that this next generation, this Generation Z, this is gonna be their smartphone, I think.
54:46 And, and they're gonna be used to these experiences where they can go plunge right into it in a, in a very immersive way. So, uh, those are my thoughts on, on those experiences. Interesting. I, I know one of the things they mi- they felt very strongly at the Stanford lab was that it's surprising what low fidelity you actually needed to be immersive. You know, when you just look at a single frame from these things or just, to your point, like do the super quick demo, like, your brain may not fully embrace that suspension of disbelief and you might not get emerged, uh, immersed in the experience. But that pretty low fidelity graphics after a few minutes in the environment was enough to trigger all these psychological impacts as if you were really experiencing these things.
55:32 So, so one of the things that they learned at Stanford was it didn't need to be photorealistic, for example, to trigger someone's phobia, uh, if they had a fear of heights and you, you showed them a VR scene that was high up or, you know, you did some other behavior. It just… It didn't need to be super realistic. And that these hardware, you know, greatly surpass that threshold. Yeah. The, the sound really makes a huge difference, so definitely recommend if, if you play with these things, go ahead and, and, you know, whip out some headphones and put them on. It, it, it makes a world of difference. Yeah. Now that we've given the listeners a really good overview into the technology that's available and the, the platforms that are on the market now, let's spend a little time talking about how these things might apply to commerce and retail.
56:20 Yeah. To start with, there's several vendors that, you know… Th- this stuff is Is tricky enough from a computer science standpoint. Most retailers aren't gonna have people on staff that can do this. You're, you're looking at people that are familiar with gaming engines like Unity and Epic and, and how to deal with this. So, um, the first thing is there's a, a couple of vendors out there that have kind of specialized in marrying, uh, these levels of experiences to the retail environment. And, um, I think the three we wanna talk about are Blippar, C Imagine, and MarkZent. And you know more about Blippar, so you take that one. Yeah. Blippar is actually gonna be one of the panelists in my shop talk next week.
57:00 And Blippar is a library of software that do two things. It does augmented reality, so it's one of these V1 augmented reality experiences you described. You look through your phone, your phone uses the camera to see the real scene in front of you, and then they augment that video stream with graphics on the screen. But it blends that AR with image recognition. So their sort of principal demo is that you can aim that smartphone at produce in a store, and it will recognize an orange in the video stream, for example, using image recognition, and then it'll augment your video display by showing you the nutrition information for that orange.
57:43 So it's kind of Augmenting what your eye can see on a shopping journey with all this supplemental information, uh, that helps you shop. And that was super interesting to me because my first experience with AR might have been the first AR application on the iPhone was back in 2010, and it was this product called Layar that you may remember. It let you load lots of different data sets in and then overlay them over video. So for me personally, that was a database of Starbucks, and you could actually hold your phone up in the real world and point the phone in any direction, and it would show you where the nearest Starbucks was in that direction.
58:23 And so you… It was almost like X-ray vision. You could see the Starbucks through the buildings and see how far it was away. And in my life, being able to find a Starbucks at any moment is a, a super important thing. And so it, it was kind of funny, when I first saw Layar, it immediately occurred to me, oh my God, this is a game changer for shopping. Like, if you think about the ability to do augmented reality on your phone and you add to that the, you know, red lasers and those things that were out at the time that had price comparison, and you add to that image recognition, which Amazon had just started playing with in terms of, uh, recognizing the, the video boxes. You say, man, you put image recognition, AR, and price comparison together, and we're gonna be able to walk into a store, hold our phone up, and the phone's gonna recognize every product on the shelf and overlay on that video display the competitive pricing information and the ratings and reviews and all the digital content that we're just getting used to shopping for.
59:20 Um, and so that was 2010. I actually wrote a blog post about it back then. And now, you know, we fast-forward seven years, and Blippar is really one of the first, first tool sets that's paying off on that possibility that w- that I saw back then. Cool. And on the other two, um, I always like to be very transparent. I have no relationship with these guys. Um, and what I found in this VR, this augmented reality shopping world is it's very secretive. So you'll find a retailer that's using this stuff, and they won't, they won't I know who the vendor is many times, but the, it's not publicly disclosed. So, um, so just to kind of throw it out there, you have… And I don't know the pros and cons of these, these guys, so everyone will have to kind of check them out.
1:00:01 But, but essentially these are companies that you go to them and you say, "Hey, I'm Retail Geek, and I want to build a widget into my mobile app that does a cool AR kind of a thing." Um, and the first one is Marxent, and it's marxentlabs.com, and the other one is C Imagine. They both have generic demos on their site, so they'll show you, you know, "Hey, here's a cool example widget where we take some furniture and change the color," or, um, "We take your living room and drop a desk into it, and the consumer can twirl it around and stuff." Um, because it's so secretive who's working with who, it is kind of a little bit hard to figure out. They don't have a lot of case studies, but, you know, I, I can tell you these companies are being used by a lot of the, the case studies that, that you and I will go through.
1:00:44 So if you are a retailer and you're interested in, in adding this kind of functionality, um, those are the three kind of vendors that, that we hear brought up quite a bit. So now let's transition to the most important part. Let's talk about some of the different case studies for AR and VR in a retail context. So the first use case I wanted to talk about is one that I believe is one of the earliest commercially viable use cases for all this stuff, and it's one that's, that's out there and real today, and that's using VR to help design store environments before you actually Build the store. Building a store is super expensive. It's a shame when you build it and then bring some test customers in and find out that there were things that don't resonate well with those customers or that you should have changed.
1:01:32 Very often when you're building a store, you, you pay a lot of money to have these, like, scale models of the store built, but again, you can't put shoppers in those scale models. And so one of the cool things that VR has enabled is these days when you're designing a new store, you're almost certainly gonna do it in a 3D design package, and you're gonna be able to render it in a VR platform. And so not only can all your stakeholders, all your store management and your architects and, you know, all the folks that are involved in that store design actually get to go in and experience the store design via VR, but you can also bring customers in, and you can do focus groups on virtual stores and really, you know, get a good feel for what's gonna resonate and not resona- work well with your customers before you have to commit to the huge expense of building a store.
1:02:24 So I was involved recently in a new flagship store for T-Mobile, and we heavily leveraged virtual reality, and we were using the Oculus platform to visualize the various versions of the store before we ultimately built it, and it dramatically improved the end experience, but it also saved us a lot of money and time. Cool. Were consumers pretty open to seeing the store in that virtual kind of a way? Yeah, and the consumer part is not perfect, right? So obviously, you know, putting on those headsets is not the same, and again, the V- Oculus does not have, or at least the, the ones we had did not have the room scale. So they, they couldn't truly experience moving around the space.
1:03:05 You know, they, they used a keyboard to move forward and backwards and left and right. But we absolutely get good feedback and we were actually able to do eye tracking on top of that. So you could… When you were designing displays and you, you know, you're thinking about putting a really expensive video display 12 feet in the air, you could find out whether or not that was gonna draw visual attention or whether it wasn't gonna be noticed by those shoppers as they walked in and explored the store. Oh, I never thought of the eye tracking that kind of ties, uh, you know, a very digital experience to the, the, the brick-and-mortar world. That's … Seems like that's the only way to do it. So we did it with a slight kludge by having the, the shoppers wear two different products, an eye tracking product that didn't get in the way, that had to sit inside of the Oculus.
1:03:51 But you could imagine sort of building the eye tracking technology into a commercial version of the Oculus, and that, that could be a interesting opportunity for someone. Cool. So, uh, so store design is kind of one use case that you've seen. What's a another example of a use case? So the next one that I think is already upon us, at least at some level, is a lot of what you were calling the AR 1.0 experiences. So I'll say augmented reality that's intended to be used out of home, and very often, augmented reality experiences that are intended specifically to be used in store. So one of the coolest ones is not one that's on your phone. This is a collaboration of Walgreens with Google, and Google has a technology called Project Tango, which is a, a 3D r- uh, rendering technology that can render in, in, uh, VR.
1:04:43 And then they partnered with a third company called Aisle411, which is a geolocation te- uh, vendor that have the ability to, to accurately locate a shopper within a store. And so Walgreens launched this pilot where they put hardware on shopping carts, and you can get one of these shopping carts, uh, at the pilot stores here in Chicago. And you push the cart around the Walgreens, and on the video display on the handle of the cart, it's showing you a, a video screen of the aisle in front of you. But because it knows exactly where you are, it's augmenting that screen with product content and promotions that are digital, uh, that get overlaid.
1:05:25 So you can look for a particular product, and it'll highlight it on the shelf for you. If you're a member of, uh, Walgreens Affinity program, you can see the pricing and offers for the products you're walking by that are relevant just to you and your, you know, your specific ops, o- uh, offers and personalization from, from Walgreens. So really true realization of that sort of hypothetical use case that, that we talked about earlier from 2010. Um, so that seems like a very interesting use case. And then another really popular one right now is at Sephora. One of the services Sephora offers is you can walk into a Sephora store, and you can talk to a cosmetics consultant that will give you an evaluation of your skin and your coloring and recommend particular products and applications for you.
1:06:19 And they've now launched a, a self-service version of this using AR. So they've partnered with another vendor in this space that'll also be on my panel called Modiface. And they essentially let you take a picture of yourself without makeup and then enable an AR experience, and they accurately identify all the features of your face, and they can actually very photorealistically render different makeup styles on your face. And a, a lot of women that maybe aren't comfortable talking to a person and are somewhat embarrassed or, you know, don't, don't want that one-on-one consultant are now able to use this AR experience to try different looks in the privacy of their own home or on their own in the store, which is pretty cool.
1:07:07 Cool. Another really common use case is a lot of the home furnishings companies and furniture companies are using AR to help you visualize what furniture and art might look like in your home. So IKEA has famously AR-enabled their catalog so you can actually Install the IKEA app, bring your catalog home, and you can open the catalog to any page, put the catalog on the ground where the sofa it would be that you buy, and the phone will recognize that page in the catalog and render that sofa as AR in your living room. So you can see how different, different IKEA furniture looks. There's a lot of the art stores like, uh, art.com.
1:07:47 I think Wayfair has a version of this. I think a lot of the tile companies have versions where you can aim your AR app at your kitchen and see what different backsplashes will look like, or at your living room and see what different art over the, the sofa will look like. And, uh, one of the vendors that enables a lot of those home visualization experiences is the third company that's gonna be on my panel this week, and that's a company called Modsy. Yeah. The… I really like The Home Depot one where you can kind of, you know, take a picture of a room and, and, uh, or you hold the phone up and then you can change the paint color. That's always pretty cool to help you. I'm a very visual person, and it helps you kinda see what that new paint color's gonna look like. Absolutely.
1:08:28 So none of those are, like, technologically revolutionary. I mean, I guess I would argue that the, the Modface stuff starts getting pretty advanced because of the image recognition in those things. But those are all really practical experiences that solve real problems for retailers today. So, you know, we talk a lot about, hey, 91% of shopping still happens in stores, but the e-commerce experiences that we focus on have really changed customer expectations. So now we all expect ratings and reviews. We all expect competitive pricing. We all expect much richer product details than are gonna be available on the little fact tag in the store when you're shopping. And all of these augmented reality experiences on the shopping cart or on your mobile phone are great ways to take all that digital experience and content that's locked on the internet and unlock it from internet jail and make it available to the 91% of shoppers that are making their purchase decisions inside of a brick-and-mortar store.
1:09:27 So to me, I think we're just gonna see more and more of those applications. I think we're gonna… customers are gonna get more and more used to using them, and that is my sort of near-term bet about AR/VR experiences that are gonna affect Retail. I think some of the other things we're gonna talk about are exciting, but they're further away. Yeah, and n- as we transition to kind of more of the t- 2.0 experiences, what are some of the things that you think will happen there? So, so now we're talking about the Oculus, the HTC, and, and those kinds of things. Well, so one, one sort of easier thing than using those for actual shopping is using those for marketing experiences around shopping. And so, uh, one recent example that I saw just this week is TOMS Shoes.
1:10:10 And listeners may be familiar, TOMS has this, uh, buy one, give one program where you buy a pair of shoes from TOMS, and part of the proceeds from the, that pair of shoes is that they buy a second pair of shoes for a child that can't afford shoes. That's a great model. Like, there's all sorts of socially redeeming aspects, and it makes the shopper feel better about their purchases and their consumption. But one of the problems is making that experience real for the shopper. And so TOMS has recently released a Oculus VR experience, I think it primarily is probably viewed today on the, on the Samsung VR hardware, but that lets you see in virtual reality these kids receiving their, their shoes in these, you know, impoverished, uh, parts of countries.
1:10:58 And, you know, you really get to viscerally see how it changes these kids' life and, and the experiences it enables, and makes you feel a lot better about your overconsumption of TOMS shoes. Yeah. One, um, one that, that is one of the demos that a lot of these guys use for kind of thinking about- retail applications. Uh, you talked a lot about, like, I think last week you went to New York and you saw kind of one of these magic mirrors. So a lot of them have kind of replicated that in the VR. So you go, you look in a virtual mirror, and then you could throw different apparel on you or something. That-- This is kind of a little bit… This is probably a couple years out because there's, there's some algorithms that are kind of tricky there. Um, that's another kind of, uh, interesting kind of thing, thing to think through.
1:11:41 If, if you are deep in this virtual reality world, why not be able to, you know, look through products, scroll through them kind of Minority Report style, and then look at the, the pair of jeans kind of on you? And if a lot of them do start to have more of these 3D scanners, then you essentially have the pieces you would need to do a, a pretty much a full body scan. So that's one of the things that's kind of an interesting, a little bit further out, uh, example. One I saw that was interesting, one, um, is in London, there's a company called Topshop, and they, uh, they debuted a new line at the New York Fashion Show, and obviously they're in the UK. So for, um, what they did is they, they set up I think about 10 Oculus Rifts, and then they au- they, they had some kind of a raffle or an auction, uh, for charity.
1:12:28 And if you won that, you could actually go to the London store and watch the fashion show live so that someone there was, was, was at the fashion show live streaming using a 360-degree camera. Uh, and you could live watch the fashion show. And they recorded that experience, and then now other customers can come in and see the fashion show. And it's, it's hard. Until you experience it, it, it's, it's very immersive because you can look anywhere while it's happening. So you could look at the reaction of the person next to you, and you can-- There's no camera pointing you to what's going on. It's just recording a full kind of a sphere of information. Um, and it's, it's very immersive and it's like you're there. So I thought that was interesting, kind of a, hey, there's these events we participate in as companies.
1:13:11 Maybe you sponsor a sports event or maybe you have a NASCAR or maybe you have, uh, you know, an extreme sports person or maybe you do some kind of a fashion event. Why not let all of your fans and customers go experience that? So I, I think events is an interesting intersection with retail to think about. Yeah, ab-absolutely. And those are the kind of use cases that you see people talking about the most and demonstrating the most at trade shows. And similar ones, like Dior recently did a, a similar fashion show to the Topshop one that, that's actually uses the Oculus Rift, and they, they make that available in the s- the store to, uh, view the The fashion show, and I know Lowe's has a number of pilot stores where they let you use an Oculus to view, uh, custom kitchen cabinets.
1:13:58 So you design the cabinets, and then they let you visualize your new kitchen before you buy it. And while all of those sort of in-store VR experiences seem interesting and sexy, I'm actually not that bullish on them in the short term, 'cause I, I feel like they have two tangible problems. Number one, the ROI is pretty tough because it, this is expensive and frankly somewhat fragile hardware, and, you know, you're, you know, a super geek and you figured out how to get it up and running at your house where you're one of only a few users. But you try to run that stuff in a, a, a high-traffic retail environment, and, you know, stuff breaks, stuff gets misconfigured. Maintaining it is kind of tricky, and most of this hardware really isn't designed for industrial use yet.
1:14:42 It's kind of expensive. Each super expensive headset can only serve one customer at a time, um, so you're spending a lot of money to not reach, you know, the majority of customers that walk in your store. And then the big problem to me is a human behavior problem Nobody looks cool in these headsets, and shoppers are not super comfortable not looking cool in front of all their peers, right? Yeah. So the worst experience is to both feel self-conscious and not be able to see around you, and imagine that everyone around you in this store is looking at you and thinks you're a, a total nerd. And so I think it's, it's actually, even though they offer a really good experience, it's really hard to get a high percentage of people that walk in a store to be willing to try this stuff.
1:15:28 Yeah. Well, I think you sh- you know, speak for yourself when you say you don't look cool in these. I think I look pretty, pretty uber cool. You have one of the rare looks that looks just as cool with or without a headset. I will give you that. Thanks, man. I, I really appreciate that. I'm, I'm happy to give you that backhanded compliment. The ultimate experience that you see a lot of digital agencies, including mine, demo, I feel like is the most far-fetched, and this is shopping inside of the VR environment, right? And you know, I'm not excluding the fact that w- you, we will one day do that, but actually conducting a transaction in one of these devices is kind of silly right now. You're, you certainly can't pull out your credit card and see it. There's lots of security issues with visualizing the payment inside the device, and the, the user interfaces in all these things just… It's a little clunky and granular to be putting stuff in a cart, be, you know, entering shipping information and, you know, doing all the, the hard things that we do in e-commerce sites to conduct commerce.
1:16:28 And so, you know, most of the demos assume all those problems away, and they just go, "Oh, all your customers will have a mobile wallet, and all your customers will have their, their shipping information stored." And we know in the real world today that a lot of users don't like to do all those things. So could that be in the distant future that we're actually playing a game in our Oculus Rift, and there's a, a commercial in the game, and we use that commercial to actually purchase some, some real world product? Sure, it's, it's possible, but I, I certainly don't think that's gonna be one of the early use cases that, that helps retailers make money on any of this technology. Yeah, it's, it's, it- we're, we're probably three or four years from that at, at the minimum.
1:17:09 The, the one-- the exception I will say is, um, one of the pieces of content I find my kids really like is, is a tour of anything. Um, so there's a couple of these where you go through kind of a Japanese, uh, fish market or you can go see some kind of a, you know, maybe the pyramids or something like that. So you could imagine if I was an enthusiast of Saks and let's say I lived in South Carolina and never been to New York, uh, it would be kind of interesting to be able to do a virtual tour of the store, but it would be more of a captured camera thing than, than kind of true animated VR kind of an experience. Yep, and I, I still-- You know, I don't think the VR is necessarily better than being there in person.
1:17:49 So it's, it's unlikely that you're gonna say, "Hey, I live in the same town as a Starbucks, but it's too much effort to drive to Starbucks. So instead I'm gonna throw on my VR headset and simulate being in a, a…" I'm saying Starbucks, but I mean Nordstrom's. That I'm gonna simulate being in a Nordstrom and buy a bunch of goods from that simulated Nordstrom. I just don't think the technology is there that the VR experience is really gonna compete with the physical experience, except in the cases you mentioned where the physical experience isn't available to you because of proximity or geography or something like that. Yeah, and one of the areas it really falls down is, um, the sense of touch, and that's where, um, you know, there's this technology called haptic feedback that, that these, these things have very kind of blunt haptic feedback.
1:18:35 So the VIVE will vibrate at different kind of levels as you get, you know, near something, and there, there is a little bit of kind of feedback, um, but, you know, you're not gonna be able to feel the, uh, if a cloth is, feels like cotton or something like that. But, but I do think there, you know, again, there's so much investment and R&D going on here. I, I know a lot of people are trying to solve this kind of sense of touch problem too, and so it, it's gonna be very fascinating, and this stuff's gonna progress extremely quickly. So stay tuned. Absolutely. But I guess my closing thought would be, you know, when you get that email from your board of directors saying, "Hey, what's our virtual reality strategy?" I would implore you to resist the urge to run out and build some sort of super fancy Oculus Rift demo to put in your stores, right?
1:19:22 One day there, there could be some viable niche use cases for those kinds of experiences, but in the short run, I'd way rather see you add image recognition or some sort of practical version of, of, uh, augmented reality to the mobile app you undoubtedly built last year when you got the email about mobile apps from your board of directors. Yeah, and I, I think, I think every retailer should have a lab. You know, invest 5 to $10,000, get a high-end PC, get all the headsets, have this hardware, uh, and just make sure you and everyone in your company is aware of this and, and experiences it because, um, you know, there, there may be a day when you do need to utilize it, um, and either put an app out for it or something.
1:20:04 And the more you kind of embrace this technology, the more you'll be ready for that day. Absolutely. And your consumers will be using it for entertainment and content consumption, uh, long before they're using it as a dominant shopping mode. But that still is gonna fundamentally change consumer expectations, so you should absolutely be on the forefront of understanding what those experiences look like. Yeah. And hopefully this deep dive into AR and VR is, serves as a jumping-off point for you. Absolutely. And, you know, I am happy to announce that we have successfully set the record for the longest episode of The Jason and Scott Show, so we clearly are out of time. But I do hope that the, uh, you found this deep dive useful, and that it potentially helps focus some of your own thoughts about how VR and AR might impact your retail business.
1:20:53 And if you did, we'd love to hear about it. If not, we'd like to hear that too. So, you know, as always, we'd, we'd certainly encourage you to write a review on iTunes or don't forget to visit our Facebook page and leave us some feedback there. So until next week, which will likely be from the Shop Talk show in Las Vegas, I would love to wish everyone a happy commercing. Thanks, everybody. You've been listening to the Jason and Scot Show. For all the latest news and trends on e-commerce and shopper marketing, subscribe to us on iTunes and please leave a review
Overcast
PocketCasts
Amazon Music
Audible
Youtube Music
RSS Feed