Facebook's new patents reveal vision for hyperrealistic metaverse

Facebook's vision for a metaverse includes an array of body-worn sensors and devices that track and predict movement to create a hyperrealistic digital world with little privacy or user safety built in, according to an Insider review of hundreds of patents recently granted to the social media giant.Facebook CEO Mark Zuckerberg says this interactive realm will take at least a decade to scale, but the company's engineers have already been busy designing the technical ingredients — and patenting their inventions."Delivering a sense of presence, like you're right there with another person, that's the holy grail of online social experiences," Zuckerberg told analysts late last year. Facebook wants these virtual objects to be rendered in real time and with the change in your eye gaze, whether they are copies from reality or creations, according to several patents covering their representation.This push toward realism is the central goal for companies looking to build 3D worlds, according to Jeremy Bailenson, founder of Stanford University's Virtual Human Interaction Lab. He's been working with VR for 20 years and also cofounded Strivr, a company that does large scale trainings through VR."Every company that's building any AR/VR technology is trying to build photorealistic avatars that can interact and that also have low latency and are rendered quickly," Bailenson said.Not one company is there yet, he noted, nor has one been able to build a system that can host even 10 or 15 photorealistic avatars at once and have them interact. "It's mind blowing what they will have."Several patents granted to Facebook in recent months cover the minute tracking of eye, face and body movements through additional "wearables." There's an apparatus for "acoustic sensing" that would allow for the creation of sound from virtual objects when they are touched, a "magnetic sensor system" to be worn around the torso for "body pose tracking," and a system, for gloves and other devices, that simulates touch.There are also a number of apparent updates to waveguide technology, core to AR and VR capabilities, that more extensively track eye and facial movements.