The way virtual and augmented reality work is to trick us into thinking we are in a virtual world or that there are virtual objects in the real world. This is achieved by fooling our senses. Currently, most VR headsets out there are much better at fooling our sight than our hearing. When it comes to augmenting or improving our reality with sound there’s even less done in that area.
FRL(Facebook Reality Labs) Research realized this and brought together an interdisciplinary audio team. The team was tasked with improving audio presence in our virtual experiences and redefining human hearing. This article focuses on the second task, but you can read more about both of them here. The team, after working on the second problem for a while, has come up with several prototypes that will supposedly give you perceptual superpowers.
You’ve probably been at a party or a concert, or any noisy environment for that matter, and tried having a conversation with someone. Half the words that were spoken by the person you’re conversing with is probably going to be lost to the din around you. Using off-the-shelf hardware and software developed by the team, the team designed a pair of glasses that can reduce background noises and enhance sounds you are interested in.
The glasses have an array of microphones on the frame that listens to the sounds around you. It can pinpoint sounds you’re interested in using the movement of your eye and head.
When you make phone calls in a noisy environment, you usually have to yell into the mic before the person at the other end hears you clearly. While we have gotten used to this, it’s obviously not a pleasant experience. To solve this problem, the FRL audio team designed a pair of glasses with a set of microphones placed in specific locations around the frame.
Using a technology called near-field beamforming the glasses can isolate your voice from the noise around you, such that the person you’re speaking with over the phone can hear you clearly without you having to raise your voice. This will also work if you are recording a voice note or using your AI assistant.
The third prototype the team is currently working on is an in-ear monitor(IEM) that has active noise cancellation. When used in a noisy environment, loud noises would be significantly reduced, allowing one to hear other things clearly. This would also protect the hearing of the wearer, as prolonged exposure to loud noises (> 85dB) could lead to deafness.
Another feature the IEMs have is something the team called perceptually transparent hear-through. What that simply means is that you can hear the outside world as if there is nothing in your ear.
Sometime in the future, we will get a pair of AR glasses that will have all these technologies. But currently, these prototypes are far from being finished products. So if you were already imagining holding a meeting at a concert, you’d have to wait a while for that to happen. I have no idea why you would want to have a meeting at a concert though.
For more information, please contact us at [email protected]
Find us on Instagram:
Visit our website:
Find us on Facebook:
Tweet to us:
Image courtesy Facebook