Apple’s Augmented Reality Headset
A headset with an augmented reality (AR) display projects digital information over the physical world. It can be used in various applications, including enabling customers to see how products might look before purchasing them.
Apple’s AR headset will not need an iPhone and will be able to function on its own. It will use a pair of Mac-level M2 chips and feature a dedicated Image Signal Processor.
Apple’s Augmented Reality Headset
Apple’s entry into virtual reality and augmented reality could finally have the impact that other headset makers have struggled to achieve. Unlike most hardware, where consumers don’t always flock to first-generation products, Apple has the ability to create demand for new products through its hundreds of retail stores where customers can walk in and try on the headsets.
The latest rumors suggest that the headset will be called Reality Pro, though it might be known by other names based on trademark applications and recent patent filings. The operating system will be named xrOS, according to Bloomberg’s Mark Gurman, while the software framework for AR will be called RealityKit.
The headset is expected to be tethered to an iPhone, so it can access data from the phone’s camera and sensors. It is also expected to be able to recognize and read voice commands, which would make it much more convenient than previous AR headsets that require users to touch the screen or type on a keyboard. Initially, the headset will only work in VR mode and won’t support AR passthrough, like Google Glass, which shows your real environment through transparent lenses.
Design
The headset will feature a sleek design and a lightweight build. It will be made of aluminum, glass, and carbon fiber to minimize weight. Apple also plans to add a built-in battery to keep the headset cool and comfortable to wear for long periods of time.
A recent report by Mark Gurman indicated that the headset will be called “Reality Pro” and have a less expensive version to follow in 2024 or 2025. Rumors have also suggested that the headset will run a new operating system that is based on iOS. References to the headset’s operating system have been found in various code leaks and Apple trademark filings.
Gurman’s January report detailed a number of features, including advanced eye and hand tracking, the ability to double as a connected display for a Mac, and the ability to make FaceTime calls using virtual avatars (as opposed to just seeing the person you’re talking to). A patent filed by Apple earlier this year describes a method for automatically adjusting the lenses in a headset to match the user’s IPD.
Technology
Apple’s rumored headset could have its own high-end dedicated chip or may dovetail with advanced processors found in its smartphones, tablets and Macs. It’s said to use a custom image signal processor that augmented reality headset stitches together input from cameras around the user, and short-range and long-range LiDAR systems for creating a 3D map of the wearer’s surroundings.
It’s also rumored to use a technology called “tracking,” which will track the user’s movements and position and translate them into virtual reality. The headset could also have eye and face tracking that lets users control a virtual world using their fingers.
Unlike VR headsets like Microsoft HoloLens and Magic Leap 1, Apple’s AR headset is expected to be more compact and look more like a pair of glasses. Early designs of the headset suggest it will feature a slim visor that appears more like futuristic ski goggles than eyeglasses. It may also support prescription lens inserts. The headset will be able to run apps from the App Store and also offer a FaceTime experience for one-on-one video calls. It will reportedly have an iris scanner for biometric authentication, and it can also be used to authorize Apple Pay purchases or logins.
Content
The headset will run on a new operating system, called xrOS. It will include a special App Store with apps designed specifically for the headset. Apps will be able to track the wearer’s eyes and hands, which will enable the user to interact with the virtual environment. Other apps will include FaceTime, which will allow the user to create a digital avatar for one-on-one video chats. Apple will also release apps like Final Cut Pro and Logic Pro to allow audio and video content creators to edit and create in the headset.
MIT researchers have developed an AR headset that uses computer vision to give the wearer “X-ray vision.” The headset can find objects hidden in boxes or under piles of clutter by using wireless perception to locate RFID tags. Once it finds a tag, the headset will guide the wearer to retrieve it by displaying its location on the display.
The device will use cameras to track the wearer’s eyes and hand movements. It will also feature a head-tracking sensor augmented reality headset that can recognize facial expressions and body posture.
Price
If Apple is serious about launching AR glasses, it needs to figure out how people will interact with them. The problems are even more complex than the ones with VR headsets because it’s not clear how you’ll navigate through a virtual world or what kind of inputs will be supported.
Kuo predicts that the company will launch a pair of glasses after it has figured out how to make a proper mixed reality headset. This would probably come a few years after 2023.
It is expected to compete with the likes of the PlayStation VR 2, HTC Vive XR Elite and Meta Quest 2. The device will have two 4K Micro OLED displays that will boast a high pixel per inch density and brightness levels.
The headset will also reportedly feature a camera for FaceTime calls and other interaction with Memojis, its avatars. Other features include a thimble-like controller and the ability to track the user’s eyes. It is rumored to have built-in battery power and not require a separate processing unit, as most other headsets on the market do.
