The WILDEST Tech I’ve Ever Tried - Meta Orion

- I just tried one of the coolest pieces of tech I have ever seen. So Meta were very kind and invited me out to their campus ahead of Meta Connect this year to try one of the most interesting pieces of tech that I have ever gotten my eyes on. Now you're likely familiar with VR, whether it be the Meta Quest or realistically even something like the Apple Vision Pro.

It is a way to put yourself in a different environment. It's a very immersive experience, but there are a lot of downsides. It's big, it's heavy.

Battery life isn't great. And the passthrough that you see, is entirely based on cameras, which let's be honest are never as good as the real world. Now, on the other side of the equation, you have something like the Meta Ray-Bans.

So, I can easily capture a photo. But Orion is something different. So, the basic idea is that take something, like Meta Ray-Bans, and build an entire computer in them.

First and foremost, you have actual displays in the lenses, but they're not just like fully passive sort of displays. You have two little projectors in silicon carbide lenses. And what that means is that those projectors can display an image over the top of the real world.

And I will say that I used it in a room that was relatively well lit and it was good. Now, clearly not perfect. First of all, I need prescription lenses and I didn't- not have contacts or something, so everything was a little bit blurry.

So, I can't really judge just how sharp it was, but it looked quite good. And importantly it was very bright, it was very saturated, but I still had pretty much full passthrough, right? Like the actual level of like tint for the lens was incredibly low.

Like you really did feel like you're just wearing a regular pair of glasses just with a little bit of distortion from the fact that they do have two individual lenses to let all the light through. Now, on top of that, there was a bracelet. The bracelet was not a hand controller, but rather it picks up the electrical impulses of your fingers and sends that over to the headset.

If you've tried something like the Apple Vision. It has cameras facing down, so you can like use like little pinch gestures, kind of like at your lap or something. It works really well.

But with this, you can have your hands anywhere. You can have it behind your back. Essentially what it does is, paired with eye tracking, you can look at anything and just pinch your fingers.

You know, you can swipe. And on top of that, it actually has haptic feedback. So as you're tapping stuff, you're feeling it, not only with the actual fact that your finger is touching your other finger, but you're actually feeling the bracelet give you a little bit of a vibration.

Now, there is a separate little puck that has some of the computing in it, but it's entirely wireless. And honestly it's tiny, you could easily put it in a belt or in a pocket. It was actually quite high frame rate, as well.

So, it was running at 90 hertz on the headset, it's being up sampled from 45 via the actual wireless link. It's obviously not a complete thing, right? Like this is very much sort of gen one.

They are not selling this version. I've heard that if they were going to sell this thing it would be absurdly expensive, like maybe like 10 grand or something, right? But, they're clearly developing this technology and this is kind of like the end game for all of the VR and AR stuff that Meta have been building over the years.

Now, they're still making headsets. In fact, they actually announced, at Meta Connect, a new version of the Meta Quest. So there is a brand new Meta Quest, which is, even though it may be not as quite exciting, is really a major step forward.

So this is the Meta Quest 3S. This is a $300 version of the Meta Quest 3. Now, there are some minor sacrifices.

It does use similar lenses and a display from the Quest 2, but it does have pretty much everything else that is going to be a match for the Quest 3, including the same processor, the same full color passthrough. It actually even has some additional upgrades including an IR camera. And while the Quest 3 has got a price cut to $500, the Quest 3S is available for $299.

And I think that's a really strong value. But, the thing that's much more exciting, to me, really is Orion. Now, I was able to try a lot of demos, so I was able to jump on like a WhatsApp call.

I was able to scroll Instagram. First and foremost, the displays are really good. But there's a lot of individual pieces layered on top of that.

The tracking is really quite solid. So, much like if you've ever used a Meta Quest or an Apple Vision, you can put a window and it will stay there in space. On top of that, you can take advantage of looking around and tapping to actually navigate, versus something on the Quest today where you actually have to either use the controllers or use your hand.

So, definitely feels more advanced on that standpoint. You also have Meta AI built in. So, one of the demos I got to try was that there was a list of ingredients and I can look at it and just tell me like, "Hey, gimme a recipe.

" And not only did it take a look at everything and sort of create a recipe, but it actually have little tags floating over each item. It's a little bit transparent. So, like if I was talking to someone and I had a window over their face, I couldn't really see it too much.

The fact that it felt like there was no compromise, I was just wearing a regular pair of glasses. They just happened to let me walk around and be like, scrolling Instagram by myself. Like that's a crazy idea.

This is the first piece of tech I've seen in so long, where I tried it- I stepped outta the demo and the only thing I could say was, "Holy (quack). " And while, the demo I tried was certainly not perfect, this is a working version of it, right? This is a concept right now, but it seems very clear to me that this is going to be an actual shipping product at some point.

Hopefully in the not too distant future. I- this is the first time I've ever tried a product that feels like it's a smartphone killer. It really does.

And that sounds like a crazy thing to say. And maybe it won't fully replace a smartphone. It might not.

And you do have to wear the bracelet, which is going to actually get that super high fidelity data when you're moving your fingers. Like, it's not a super seamless as just literally putting on glasses. But it's not far off.

And I've come away with this thinking that all the billions and billions of dollars that Meta has poured into the Reality Labs project. All the years they've spent developing the Oculus and the Meta Quest and the Meta Ray-Bans. All these things are coming together in the product that we've all kind of hoped for.

And I'm incredibly excited to try Orion when it actually comes out. Immediately after my Orion demo, I had the opportunity to interview Boz, who is the chief technology officer at Meta and essentially the guy who's made all of this possible. First of all, thank you so much for taking the time.

I'm sure you're incredibly busy this week. If you look at what Reality Labs has done over the last, you know, several years, right? You look at how Quest has evolved.

You look at the Meta Ray-Bans, which I think have been a really fascinating, at least from the outside. What's your sort of overall feeling on kind of where, what you've built is right now and how that looks like it's going into the future? - The- an investment horizon that we've taken on with the Reality Labs is pretty uncommon.

And so you're looking at, okay, we knew when we started AR work, it was gonna be 10 years to the Orion that you just got to try. - Yeah. - You expect two things.

First of all, along the way you're gonna discover a bunch of things that you thought would work, that don't work. But what's been nice is now you're also finding things that you didn't anticipate when you started, but because you've been making the investment, you're within striking distance when the time is right. And the same has been true for for, yeah, for the Ray-Ban Meta product.

We've got this amazing product that we designed without even a thought to AI, at the time. - Really? - No!

The first time we built it, in fact, it wasn't until months before launch that we were like, "You know what? We should get our AI assistant in here and make these AI glasses. " It's taken a little discipline to remind our industry what it takes to actually innovate in hardware.

- Yeah. - And, and I think, you know, what's fun about the moment we're in now is you're starting to see where it's going - Really feels like it leads up to Orion, right? Like, so I tried it literally like five minutes ago.

I literally stepped from the demo, oh hey, I'm late. Hey, what's up? So where- how long have you wanted to make that product and how- I'm sure the answer forever.

- Yeah. - But like how, at what point did this start to come in focus, where you're like, oh, wait a minute, this is starting to come together. This is starting to come together.

Where you actually think like this is a viable thing? - I have two answers to this question. One of which is we've wanted this forever.

We've wanted this as kind of an industry as long as we can imagine it, right? Like, and it goes deep into the science fiction, you know, history. And also this product was, you know, 10 years from when the research started, five years from when officially the Orion program started, little over five years.

And when we started it, we thought there was a 10- I have all these documents I was reviewing recently. We have a, we thought we had a 10% chance, less than 10% chance, of actually being able to build one. You know, as recently as two years ago, we were just like, really not sure we could do it.

There was like a lot of- it was, it was challenges in the program. We needed some breakthroughs. And then a year ago, was the first time I put on just the display workings.

There was no software stack running yet, so you couldn't see the operating system. And it was magic. You were like, oh my God.

Like, we've at least done something here. (Austin laughing) But it wasn't until this summer that I got to try it with the system working that I kind of had that meaning- that feeling of like, holy (quack) (quack). It's from a consum- from a product standpoint, it's ready.

It's absolutely ready. It's not the price point that I think is ready for consumers yet. (Austin laughing) So that's like, that's the next big piece of work to do.

And this, what trade offs do you have to make to get there and how do you get there? This- there is a future beyond the smartphone. Like there is, I love the smartphone.

Smartphone's amazing. But there is actually a future beyond it. There is even a set of better things that we can go do.

- [Austin] I know people always like to go like, "Oh, it was like the first time you did like a pinch-to-zoom," or something. But it kind of felt like that to me of like, wow. - [Boz] Actually it's funny that you used pinch-to-zoom as an example.

I think the work that Apple pioneered on Multitouch and making that feel so natural to all of us is an analog we look to. I think we look back to Xerox Park and the work that were doing on mouse and keyboard interactions. And this is one thing, to your point earlier about how these things come together or they don't.

There's a tremendous technological underpinning that they share, which we're very excited about. And I think with EMG, we're just at the very beginning of what it's capable of. But then there's also, like, how you use them is it can be described in similar terms, but it's different in the same way that the phone and the laptop are different.

And so when you're out and about in the world, how you wanna interact with the digital world is just very different than how you do when you're, you know, on a plane, in your house, you know, someplace more stationary where you're gonna have a headset on. So, I think like they're both valuable. You can obviously do things in virtual and mixed reality that you can do no other way.

Not even with augmented reality glasses. So we also like, are trying not to overly homogenize them and let them be what they naturally want to be on their own terms. There they are, yeah, here we go.

- Oh my- that's what it looks like on the inside. - Yeah, we gotta do the wide shot for these ones, guys. (Boz chuckles) This is, these ones actually have a problem, which is the, the plastic that we had to use to make these clear is thermally not as conducive.

- Yeah. - So these ones, they work in the sense that like you can turn them on, but they actually will power down pretty quickly because the thermals aren't right. This version doesn't have what's called disparity correction.

So as your lenses flex relative to one another, we don't want the image, to like, go blurry on you. So we wanna keep them stacked. The next version that we're building does have disparity correction.

And so, this stack right here gets much thinner, actually about half as thick. - Oh, wow. - 'Cause the protector comes in from the back at that point.

What's crazy about this, is you see, this is the Steve Jobs test, right? Where he took the iPod and he threw it in the fish tank and was like, there's bubbles. That means there's air in there.

There's no bubbles. Like you throw this thing in a fish tank, you're not gonna see anything. - You're doing a fair bit of processing?

I know obviously you've got this which is where the majority of battery and processing is happening. - That's right. But you're still doing, my understanding, is some of the like actual detection when it comes to like the cameras with having to pass through.

- [Boz] That's right and the reprojection - Reprojection. - Because we couldn't deal with the latency of your head moving is too great to send it to the puck and back. - [Austin] I was gonna say, I did not really detect a whole lot of latency.

Like there's like the Pong game especially, or the game where you were shooting at the- - [Boz] Yeah, Stargazer. - Stargazer, yeah. I was just like playing that.

It's like you're like diving and ducking your head around. It's like it felt really quick. It reminds me of did you ever play the Vectrex?

- [Boz] Yeah, of course. - Yeah! That reminded me.

I was being like, oh, this feels like the Vectrex. Or obviously, like you know, it's like an Atari 2600, or something. - Totally.

- But in the best way, 'cause it's floating. It's like the whole thing is like fully 3D. - I always try to undersell and over deliver.

I just think it's the most advanced consumer electronics we've ever tempted as a species. It's crazy. Like, it's like- - That's cool thing to say.

- [Boz] How much stuff is in there? - What's the next step? Like what, what do you, what do you need to do to get this into people's hands?

Whether it's developers or beta testers or me or you know. - Totally. Absolutely.

Yeah, that's right. Next goal is to get it to be cost effective. We've gotta get it to a price point where people feel like, okay, this is something that I can, like, even though there- it's early and there's not a huge number of kind of software that for it yet.

Not a big developer system for it yet. Like I'm getting value of it. So, there's a bunch of trade offs that we will have to consider as we look at the next generation.

We are well on our way. We actually have, not just the one next generation, we have two next generation products that are already in development. Now, you know, with us, you've seen the rumors.

Some of them will make it to production, some of them will not. But we have many, many prototypes that were stacked following this one. - Actually, are there cameras on- (Boz chuckles) Why are there cameras on the the puck?

- This is where we're gonna get real exclusive content here. - Ooh, excellent. - In an earlier version of this, we were trying to play with, what else could we want this thing to do?

And so, yeah, you know, using it as like a camera where it's like, hey, it's you put this on the table, but we've actually been finding out that the most important thing we can do with this, is we can not worry about it. - Yeah, yeah. (Austin laughs) - It's like the number one goal that we have for this one is like, yep, have it somewhere nearby and just don't worry about it.

And then for the most part, focus on the interactions that you're having naturally with that. So, there's like one less fussy thing to deal with. - That's phenomenal.

Thank you so much for taking the time. - Thanks for having me. - If you have an extra Orion floating around, I volunteer as a beta tester.

- So kind of you, so generous. (Austin laughs) Yeah, absolutely. - I had some questions.

The version that I tried certainly was not final, in any way, whatsoever. I think the price is probably gonna be the big one. The lenses, while they're incredibly impressive, are also probably not gonna be what they're going to ship.

And there was a little bit, because there's two pieces of silicon carbide, you can see a little bit of like kind of reflections in between the two lenses. So, it is not a 100% match for just, you know, real glasses. And the holograms aren't like completely real, but like it's pretty close.

And honestly, I am incredibly excited to see what this thing looks like when they finally ship it.