Meta's CTO explains why motion trackers aren't in the cards for Quest, despite competitor...
Meta's ambitious leap from VR to AR reveals significant technical challenges.
What is Orion by Meta?
Meta introduced a budget-friendly version of its Quest 3 virtual reality headset, making immersive VR experiences more accessible to a wider audience. In addition to this lower-cost VR option, Meta showcased its prototype Orion augmented reality (AR) glasses, giving a glimpse into the future of wearable tech that promises to bring AR experiences to consumers in entirely new ways. The unveiling of Orion underscores Meta’s dedication to advancing AR technology, with key features like a "wireless compute pack" that supports seamless processing without tethering the glasses to a phone or bulky hardware.
The Orion AR glasses integrate high-resolution cameras and transparent displays to create a "window" into augmented reality, blending digital objects into the user’s physical surroundings. Unlike traditional "ban glasses" that obstruct the wearer’s view, Orion maintains transparency, allowing users to stay aware of their environment while interacting with holographic or digital elements. This makes the AR experience more natural and suitable for real-world applications, from navigation to on-the-go access to information.
With Meta’s growing focus on artificial intelligence, the Orion glasses could harness AI to deliver smarter, contextually aware AR experiences. The glasses' advanced cameras and sensors enable object recognition, gesture tracking, and personalized AI-driven interactions, enhancing productivity, learning, and day-to-day convenience. Whether used for work tasks, social interactions, or navigation, Orion’s lightweight, high-resolution display system offers users a glimpse into a world where digital information is seamlessly integrated with reality.
Meta’s dual focus on expanding its VR and AR product lines aligns with its long-term goal of building a comprehensive ecosystem of immersive devices, from affordable VR options like the Quest 3 to next-gen AR glasses. This strategy, combined with the ongoing refinement of high-quality, unobtrusive AR experiences, positions Meta as a leader in mixed reality innovation. As the company moves forward, the Orion glasses are expected to become a stepping stone toward mass-market AR products, shaping the future of augmented reality by merging AI, advanced displays, and wireless capabilities in a wearable format.
The Distinctive Hardware Differences Between Quest and Orion
The hardware features of Meta’s Quest headsets and Orion AR glasses reveal distinct design priorities aligned with their different use cases and environments. Quest VR headsets are built to immerse users in a completely virtual world, providing a wide field of view, full-opacity displays, and precise hand-tracking capabilities. These mixed reality headsets are primarily intended for indoor use, ideal for activities like gaming, virtual meetings, and work sessions, where users are stationary.
In contrast, Orion AR glasses represent the next generation of wearable devices tailored for mobile, on-the-go use in the real world. These "reality headsets" are designed as sleek, spectacle-like frames that seamlessly integrate digital objects with the wearer’s physical environment. This requires Orion glasses to be lightweight, resembling high-tech spectacles or frames rather than traditional headsets. Unlike Quest’s VR technology, Orion prioritises translucency and a narrower field of view to ensure digital objects appear unobtrusive and complementary to the real-world.
With Orion, users can engage with an augmented reality assistant, AI-driven applications, and digital content without being cut off from their surroundings. This open, blended experience aligns with future brands focusing on mixed reality innovations that balance technology with the real world.
Overall, while Quest VR headsets immerse users in a fully digital experience, Orion AR glasses represent a new direction in wearable tech, providing a subtle, glance-friendly interface that enhances—not replaces—the physical world. The distinct hardware configurations of each product underscore Meta’s vision of providing tailored solutions for different facets of the digital world, from stationary immersive VR to mobile, reality-enhancing AR.Bridging the Gap: Why Horizon OS Isn't Suitable for Orion
One might wonder why Meta wouldn't simply use Horizon OS, the operating system for Quest headsets, for Orion AR glasses. Horizon OS is tailored for devices with wide field of view headsets that are capable of full opacity and extensive hand tracking, used mostly in indoor and stationary settings. These design considerations make Horizon OS unsuitable for the unique requirements of AR glasses.
Orion AR glasses are meant for a more dynamic, on-the-go experience, which involves different design constraints and user interactions. The operating system for Orion needs to be optimized for limited compute capabilities and must support a different set of applications and user interfaces. Therefore, Meta is developing a specialized OS for Orion to ensure it meets the specific needs of AR glasses users.
Understanding the Performance Constraints of AR Glasses
The performance envelope for Meta's Orion AR glasses is significantly constrained compared to the Quest headsets, with Meta's CTO, Andrew Bosworth, revealing that Orion’s compute capabilities are around 10 times smaller than those of Quest. This places Orion's hardware on par with portable gaming devices like the Gameboy Advance or Nintendo DS, rather than powerful gaming consoles such as the PS3 or PS2. Due to these hardware limitations, the Meta Orion AR glasses—part of Meta’s “true AR” vision—require a strong user interface and apps designed for efficiency.
To achieve this, Meta Orion’s operating system, display, and core applications are tailored to deliver visuals that match the constraints of AR displays on such smart glasses. Unlike the Quest headsets, which boast powerful, immersive visuals for VR, the Meta Orion AR glasses need to strike a balance between performance and battery life, using transparent lenses and lightweight projectors to overlay digital content onto the real world. These true AR glasses make use of clear, transparent lenses that allow users to view digital elements without obstructing their view of the physical environment.
Meta’s design choices, including the translucent lenses and low-power projectors integrated into the frames, support visual overlays without overwhelming the hardware’s limited capabilities. The addition of a wristband, connected to Orion for gesture control via electromyography sensors, expands the user interface without heavily taxing the compute resources within the glasses themselves. These components create a cohesive, efficient experience, despite the Orion prototype's compute envelope restrictions.
The Meta Orion AR glasses were previewed at Meta Connect 2024, showcasing the potential for “smart glasses” to incorporate sophisticated AR displays within a sleek design. While Apple and Meta are both exploring wearable AR devices, Meta Orion AR glasses distinguish themselves by focusing on a practical, lightweight design with features designed to enhance daily experiences. The Meta Orion AR glasses embody a vision of "true AR," blending practical, efficient technology with futuristic concepts, aiming to bring augmented reality to users in a way that feels natural.
Designing User Interfaces for AR: Challenges and Innovations
Designing user interfaces for AR glasses like Meta’s Orion introduces both challenges and exciting opportunities for innovative interaction models. Unlike VR headsets, which immerse users fully in virtual environments, AR glasses like Meta Orion require UI elements that work harmoniously within a limited field of view and blend naturally with real-world surroundings. The translucent, holographic displays on Orion allow digital content to coexist with physical objects, demanding a subtle yet functional interface. Meta’s recent video showcasing the Orion interface reveals developments with multiple app launcher main menus, suggesting an ongoing exploration of UI.
Creating an effective interface for Meta Orion glasses entails balancing usability and responsiveness with the device’s hardware limitations, such as the limited compute power and the reliance on silicon carbide lenses for transparent visuals. These constraints mean the interface must prioritise simplicity and intuitiveness, as users may interact with the device while moving in various environments. Unlike the more immersive and stationary nature of virtual reality on a Meta Quest, Orion’s AR interface needs to accommodate users’ real-world interactions and movements. This could possibly be done by integrating eye tracking and gesture controls through a wristband or wireless compute pack.
To achieve this, developers are exploring innovative ways to use eye tracking and hand gestures, providing users with seamless and precise control over holograms and digital content layered over their actual view. This creates opportunities to rethink traditional UI paradigms, focusing on minimalist, glanceable elements that users can easily interact with while wearing a lightweight pair of glasses. For instance, holographic displays and icons need to be placed strategically to avoid overwhelming the user’s field of view, ensuring important information is easily accessible yet unobtrusive.
Moreover, the interface design for Orion AR glasses must consider the interaction between the device’s Meta AI features and smartphones or other connected devices. Notifications, app integrations, and real-time information overlays need to be displayed on the screen in ways that feel natural and useful without requiring the user to disengage from their surroundings. By prioritising intuitive, accessible UI elements like floating menus and simple hand gestures, Orion's interface can bring the promise of augmented reality closer to everyday life.
Meta’s Orion glasses signal the evolution of augmented reality, where a well-designed interface on a pair of “actual glasses” allows users to interact with digital elements as naturally as they do with their smartphones.
Meta's Vision for the Future: What's Next After Orion?
Meta’s Orion AR glasses serve as a crucial prototype in the company’s broader mission to develop consumer-ready augmented reality glasses. While Orion itself won’t reach mass production, its design, hardware configurations, and user interface tests are informing Meta’s long-term vision for AR. Building on the insights and technological breakthroughs from Orion, Meta plans to release consumer AR glasses, tentatively scheduled for 2027, which will feature a refined and narrowed field of view optimized for everyday use.
Meta’s approach to these consumer AR glasses, likely under the “Meta Ray” or “Meta Glasses” branding, involves fine-tuning the user interface and interaction models to ensure seamless usability in real-world environments. The goal is to develop AR glasses that incorporate an AI assistant, leveraging natural and intuitive interactions with digital overlays without disrupting the user’s awareness of the physical environment. This AI assistant, paired with a wireless compute pack or "wireless puck" for low-latency processing, could enable smooth operation across apps and services with minimal hardware bulk on the glasses themselves. Gesture controls via wrist-based devices and voice commands will also likely play a role in delivering a responsive, hands-free experience.
While the final product’s price and specs are yet to be disclosed, Meta’s focus on affordability and accessibility for the general public suggests the consumer AR glasses will prioritise essential AR functionalities over high-end features. Meta’s commitment to keeping the hardware feeling lightweight and stylish also means the glasses will likely incorporate advanced, compact lenses. The minimal, energy-efficient components, ensure that they will feel more like a regular pair of augmented reality glasses than a heavy tech device.
By 2027, Meta aims to have refined AR interactions so that users can navigate various settings—from busy streets to workplaces—with fluid access to information, real-time guidance, and interactive digital elements. The envisioned "Meta Ray" glasses will provide these features in a streamlined, everyday wearable form that seamlessly integrates with the user’s lifestyle, complementing smartphones and other devices.
With ongoing advancements in AI and wireless connectivity, these consumer AR glasses will likely enable personalised experiences that adapt to individual preferences and contexts, whether for work, travel, or social interactions. Meta’s work on Orion lays the foundation for what's next to come. With its technical achievements setting the stage, there is hope for a future where augmented reality is accessible, functional, and woven into the fabric of daily life.
