Apple’s virtual-reality (VR) headset, the Vision Pro, was released only two weeks ago — but scientists are already grappling with some of the research implications. Researchers say the headset could be a watershed in the path towards widespread VR use, and its high precision could aid research tasks and analogue activities such as surgery. All have implications for human behaviour, opening up a torrent of research questions for scientists.

“This is insane for me,” says Ken Pfeuffer, who studies human–computer interaction at Aarhus University in Denmark. In 2017, Pfeuffer designed a ‘gaze and pinch’ feature that is similar to the one Apple Vision Pro uses for virtual navigation. When he gets hold of a headset, Pfeuffer plans to study how the headset uses this feature.

Since its US release on 2 February, the US$3,499 Vision Pro has won accolades as the most advanced VR and augmented reality (AR) headset ever made — although Apple calls it a spatial computer. Sam Altman, chief executive of OpenAI, which created ChatGPT, called it “the second most impressive” technology since the iPhone.

Tech giants including Microsoft and Meta have previously released VR and AR headsets. But the Vision Pro’s high-performance features, including its incredibly realistic ‘passthrough’ — its near-real-time display of the real world — make it unique, say experts. Eight outward-facing cameras project the outside world onto its 23-million-pixel interior screen with a delay of just 12 milliseconds. Four interior cameras track the wearer’s eyes, allowing a user to project screens and items into their virtual world, navigate to them with their eyes, akin to how they’d use a computer mouse, and ‘click’ or interact by pinching their fingers.

The Vision Pro’s popularity and slick performance could portend a future of smaller devices where humans interact with virtual overlays on the real world. “A wearable computer instead of a handheld computer [a mobile phone] could be the future,” says Dima Damen, who studies computer vision at the University of Bristol, UK.

Medical precision

Pfeuffer says that people almost laughed at him for being so focused on developing the gaze-and-pinch virtual navigation technique, but its use by Apple has made him an overnight celebrity. “Before Apple, when I tried to propose my ‘gaze and pinch’ ideas, everyone said ‘who needs that?’,” he says. Now, “It’s just clear. Now more students are working with me and it’s also easier to get funding.”

He says the feature has several applications beyond the Vision Pro. “You don’t need to point anywhere, you can just look at things,” says Pfeuffer, perhaps allowing new ways for people with disabilities to use computers and access information. “Maybe you’re injured and not able to move your hand,” says Pfeuffer. “Now you don’t have to.”

The headset’s high performance also shows promise for use in medical tasks. Jan Egger and Jens Kleesiek, medical researchers at Essen University Hospital in Germany are looking at ways the headset might be useful for healthcare². They say that the Vision Pro’s advanced eye-tracking technology might be capable of picking up conditions such as vertigo or even early signs of a stroke or dementia. The pair plans to study this when they get one of the headsets. “You have different eye movements and these can be detected using such a device,” says Kleesiek. “The quality of the sensor readings is so high that you can actually utilize it for such medical tasks.”

The Vision Pro weighs about 600 grams, with a 350-gram battery that must be attached to it during use. That can make it uncomfortable for prolonged periods of use, but future versions of the headset are expected to be smaller — perhaps one day mimicking a pair of glasses, says Kleesiek.

In such a scenario it could be possible, for example, for a surgeon to wear the headset and see an overlay of information during an operation. “You’d look at a patient, and it would show you a hologram and guide you through the sensitive vascular structures,” says Egger.

Vision of the future

If spatial computing headsets become widespread like smartphones, it could affect human behaviour, says Damen. Apple chief-executive Tim Cook said that the Vision Pro is the “beginning of a new era for computing” in which the digital and physical world are blended together.

“It will change our society in the same way previous technologies have done [like] mobile phones,” says Damen. “This type of technology might actually give us shortcuts to making things better.”

One example could be a better way to share information and knowledge, says Damen. She cites a project in which she and her colleagues collected footage of everyday activities using wearable cameras. “We asked people to record everything they were doing in their kitchens for three days,” she says. “I remember my PhD student walking into my office and telling me, ‘You don’t know how to cut onions.’ Currently we seek improvements when we know we need skill development, but there are many things we might not think about seeking improvements in.”

Behaviour and even the wiring of our brains could change as a result — vision is one obvious human capability that VR headsets could affect. “It’s changing your feeling towards something that might be foreign initially,” says Damen. “This is a 2D projection of your 3D world. How will this impact our actual 3D vision?”

The constant exposure to a virtual image of the world raises some concerns. “That projection can be manipulated in any way,” says Damen, perhaps through generative artificial intelligence, which is developing apace alongside VR and AR. “You might think you’re seeing the world but you’re not.”