The stable release of visionOS 2 is out now for Apple Vision Pro.
visionOS 2 was announced back at Apple’s WWDC24 event in June. Since then the company has shipped nine betas and a release candidate, which required specifically opting in. But the first stable release is out now, and should be available for every Apple Vision Pro owner.
visionOS 2 Changes & Improvements
visionOS 2 introduces new features and multiple improvements across Apple Vision Pro’s operating system.
Menu Hand Gestures
visionOS 2 adds two system hand gestures. Opening your palm and pinching your index finger to your thumb now brings up the app launcher (the same menu gesture as on Meta Quest headsets). And facing your palm down again will reveal a new hand-locked information bar with the time, volume, battery percentage, and connectivity status. Pinching it will open Control Center.
The update also allows you to rearrange apps on your home screen, which was previously forced to alphabetical sorting with no customization.
Turn Any Image Into A Spatial Photo
visionOS 2 can turn any 2D image into a Spatial Photo, making them 3D using „advanced machine learning“.
We’ve tried this out and found it works remarkably well.
Hand Tracking & Scene Understanding Improvements
visionOS 2 brings two improvements to hand tracking and two improvements to scene understanding.
On visionOS 1 the system provided developers with hand tracking poses at 30Hz. On visionOS 2 hand tracking poses are provided at the display refresh rate, typically 90Hz.
The other hand tracking improvement is prediction, which is optional for developers. Enabling prediction provides extrapolated hand poses representing where the hand should be by the time each frame is displayed. This results in noticeably lower effective latency, but with the tradeoff of some accuracy.
With visionOS 2 the headset now has Room Tracking. The system will provide developers with precise geometry of the walls and floor of each room, even supporting non-rectangular walls. When entering a new room, the system will inform the developer and start providing geometry only for the new room. Apple says a developer could use this to have a virtual pet greet you when you enter your bedroom, for example.
Beyond just walls and floors, visionOS 2 also brings improvements to plane detection. While visionOS 1 could only detect horizontal and vertical planes, visionOS 2 can detect slanted planes too.
Safari Gets Cinematic Mode & WebXR By Default
In visionOS 2 Safari supports playing fullscreen videos on a gigantic virtual screen in a virtual environment, including YouTube and Netflix.
WebXR is also now on by default, no longer hidden behind a feature flag users need to enable.
Vision Pro Is Now An AirPlay Receiver
With visionOS 2 Vision headsets are now AirPlay receivers, meaning you can cast your iPhone or iPad display to the headset.
Bringing your iPhone and iPad into visionOS is a significant ecosystem integration move from Apple.
New Virtual Environment: Bora Bora
Keyboard Breakthrough & Physical Mice
In visionOS 2 your physical Magic Keyboard now shows fully immersive environments, called Keyboard Breakthrough.
Further, physical mice are now supported, whereas previously only trackpads worked.
Guest Mode Upgrade
visionOS 2 allows you to save a guest user’s hand and eye setup for up to 30 days.
This will be useful for a spouse, housemate, or friend who frequently uses your headset, as previously you’d need to redo hand and eye setup each time.
Static 3D Object Tracking
visionOS 2 also adds the ability for developers to track 3D objects, an entirely new tracking capability.
The developer must provide tracked 3D objects as USDZ 3D models to the Create ML tool on their Mac before app compilation. Create ML will train a neural network for each object, which will be baked into the app.
Travel Mode For Trains
Travel Mode, currently designed for airplanes, is getting support for trains.
Live Captions
And as was announced in May, visionOS 2 adds a Live Captions option which will apply to live conversations via microphone input as well as audio from any app, including on FaceTime calls.
HealthKit & TableTopKit
HealthKit, used on other Apple devices, is available in visionOS 2, enabling health and fitness apps to store and read data to and from your Apple Health profile.
And a new TabletopKit makes it easier for developers to „quickly build shared and collaborative app experiences centered around a table, like board games or a manufacturing workstation“.
Raw Camera Access For Enterprise
With visionOS 2, enterprise companies can access Vision Pro’s passthrough cameras to run custom computer vision models.
Apple makes clear that these features can only be used in apps „for use in a business setting only“, and apps using them can only be distributed as proprietary in-house apps or custom apps made for a specific business using Apple Business Manager, not on the App Store. A managed entitlement is required, alongside a license file tied to your Apple developer account.
Ultrawide Mac Virtual Display (Later This Year)
The Mac Virtual Display feature is set to get major improvements, expanding the display into a virtual ultrawide equivalent to two 4K monitors together. Apple says this will be powered by eye-tracked dynamic foveation.
This feature will require a future version of visionOS. It isn’t yet available.
Quelle:
https://www.uploadvr.com/vision-os-2-out-now-apple-vision-pro/
