Look, here’s the deal. I stumbled upon something kinda cool. It’s about the Meta Quest 3’s new toy, the “Passthrough Camera API.” Now this thing, got the green light to be part of apps in the Meta Horizon Store. Finally. Developers have been poking at it for, what, six weeks? It’s like they’ve been itching to push their stuff to folks, but they couldn’t until just now. Frustrating, huh?
Oh, v76 of the Meta XR Core SDK is out, and it’s like, “hey, guess what? Not an experiment anymore.” Developers can ship apps with it, after jumping some review hoops. Fun times.
So, what’s this passthrough thing? Headsets like the Quest 3 have cameras that, surprise, let you see the real world. But the catch? Only the system software had legit access. Third-party devs? They got, like, summaries… coordinates, 3D mesh of your room, stuff like that. They couldn’t use their cool vision models. Felt like the headset was kinda nerfed, ya know?
And if you’re thinking, “How do I let apps use this?” It’s easy. You just give them permission to your headset cameras, like you’re letting them use a mic. Then, boom, they get color camera access, metadata, and all that jazz to run their fancy computer vision models. What can they do with this magic? Well, think QR code scanning, game board detection — maybe even spicing up enterprise experiences with virtual guides. The sky’s the limit. Or, maybe, what the XR2 Gen 2 chipset can handle. But who’s counting?
Roberto Coviello popped up with his QuestCameraKit samples — gotta say, that’s a name that sticks. Did you know the cameras stream at a resolution of 1280×960 at 30FPS? Latency’s at 40-60 ms. Doesn’t cut it for fast stuff like whizzing objects or tiny text. But good enough for… let’s call it moderate action scenes?
Here’s the quirky bit: there’s no actual Meta Quest Camera Passthrough API. It’s more like MacGyvering with Android’s Camera2 API — and working some OpenXR magic. So, they’re sorta reusing Google’s tools. Handy, if you’re also looking at Samsung’s future gizmos.
Using Unity? Leveraging WebCamTexture API makes accessing these cameras a breeze. Though, heads up, it’s a one-camera-at-a-time gig with them. Not the best for dual-camera action, but hey, can’t win ’em all.
Wanna dive deeper into this Quest passthrough trickery? Check out the documentation for Unity or Native Android. Five Unity samples are floating around GitHub (thanks Meta!), and Roberto’s sharing some goodies too, like Object Detection and QR Code Tracking. Trust me, it’s an interesting rabbit hole.
