PONS XR Interpreter
Overview
PONS XR Interpreter is an augmented reality proof of concept for communication and real-time translation. Built for Magic Leap, XR Interpreter lets professionals work together across distances and languages.
Helping Professionals Collaborate across Languages
Two people, one in Japan, the other in Brazil, talk about the latest electrical engine their company will launch in a few months. Each engineer speaks in his native language. With augmented reality glasses on their heads, they put the engine apart with swift hand gestures to see the newest changes in the design in a high-quality 3D model. Their conversation is translated in real-time, keeping productivity levels high.
Augmented Reality Facilitates Real-Time Collaboration
Remote work has bloomed in recent years — various sources estimate that a quarter of the global workforce will work remotely in 2022. This stands in stark contrast to the view business owners had just a few years ago when companies feared remote meant less productivity. Currently, workers tend to be even more productive in a remote setting than in the office.
The evolution of technology and tools are the primary facilitator of remote work. PONS XR Interpreter is an augmented reality proof of concept created in partnership with PONS Publishing, a leading language learning publisher, to experiment with cross-language communication.
By leveraging augmented reality, we created a way for professionals to communicate seamlessly.
XR Interpreter:
- Facilitates real-time translation
- Delivers an immersive viewing experience for 3D models
- Lets people maintain productivity
How We Built PONS XR Interpreter
XR Interpreter is a Magic Leap app written in C++ using Lumin SDK. When an interaction happens, Magic Leap connects to the C++ server, which directs the audio stream from the device to a cloud-based app on Microsoft Azure.
An interpreting service in Microsoft Azure then translates the audio stream and returns the translated text to the server. The server returns the information to view it on the Magic Leap through a web socket.
XR Interpreter use cases
Remote work in engineering
While the global demand for skilled professionals is growing, there’s a worrisome scarcity of them available. Remote communication and collaboration tools can alleviate the lack of a skilled workforce. Solutions for remote collaboration are becoming critical to maintaining the uninterrupted flow of operations.
XR Interpreter makes it possible for two people who speak different languages to communicate effectively in real time to discuss high-quality 3D visual assets. This way, the transfer of knowledge between two people is easier, more immersive.
The solution can be used to simplify communication in industries traditionally relying on the physical presence of an engineer to explain the intricacies of a specific machine.
Remote training
Training makes up a large chunk of time during employee onboarding. To be able to operate a machine effectively, especially in industrial and manufacturing contexts, machinists usually need at least a few weeks to become competent enough to produce outputs profitable to the manufacturer.
Having a remote collaboration tool like XR Interpreter cuts training time for new employees and helps reduce mistakes during the early employment period. This can potentially create annual savings for manufacturers who rely on specialists with lengthy onboarding timeframes.
Remote medicine
Before complex and rare surgeries, surgeons often consult with other colleagues about how to approach a specific case. XR Interpreter can be used to upload 3D reconstructions of organ scans and consult surgical plans.
Over distance and across languages, surgeons and other medical professionals can use XR Interpreter to deconstruct the 3D models and walk through specific steps in the surgery plan. With real-time collaboration, specialists on the other side of the world can provide detailed advice on critical aspects of the surgery.
