Augmented reality: between fascination and challenges
CT or MRT images that are perfectly positioned on the patient’s body? Or step-by-step prompts that guide you through the installation and maintenance process of machines? From medicine to manufacturing or everyday activities, augmented reality can make our work and our lives more efficient, fascinating and safer. But the new technology also entails challenges. Michael Burmester explains what they are.
Are you literate in the world of augmented reality? If not, we are here to help by introducing the most important Terms. As early as in 1994, Paul Milgram introduced the term Mixed Reality (MR) as a generic term, denoting all hybrid and transitional forms between plain reality and virtuality – the reality-virtuality continuum. Virtual Reality (VR) is the counterpart of reality, taking the user into an artificially generated world by means of head-mounted displays and headphones.
Augmented Reality (AR), in contrast, expands the user’s reality through virtual contents such as texts, 3D graphics, animations and videos. That way, for example, a virtual production machinery can be created in a real, empty factory hall, allowing the user to interact with these virtual objects. In Augmented Virtuality (AV), virtual space is expanded by overlays from reality. Collaborative VR, for example, displays a video stream of real people in a virtual environment.
How does AR work?
Augmented reality always requires in- and output technology that adds further objects or contents to reality. There are two different systems:
Video See-Through
Usually, reality is recorded using a smartphone or tablet, for example, and then expanded by virtual information or objects.
Optical See-Through
The user wears a head-mounted display and sees the reality through a half mirror. Virtual objects are shown in this mirror via a display. Microsoft HoloLens is one such system that captures the room using depth cameras. The users can then position virtual objects in the room, selecting them with their eyes and interacting with them via voice and gesture control.
AR is booming
There are numerous usage scenarios for AR. In the late 90s, UID already examined how AR can be used in industrial environments in its ARVIKA research project. Even then, there was a clear vision of how AR can be used in maintenance, diagnostics and training and what appropriate interactions could be. These scenarios differ only little from today’s scopes of use. However, the technology was not refined enough back then to realize usage scenarios outside of controlled settings. Today, AR is no longer a futuristic vision.
The technology is already in use in many fields, e. g. in entertainment, development, planning and construction, or in retailing: Ikea customers, for example, can use an app to visualize their new furniture in their home. And above all, AR is fun! It fascinates, stimulates, creates a special UX experience thanks to its novel character and triggers emotions and positive associations with regard to products or brands. On top of the fun factor, however, the technology also has very practical advantages.
AR provides information wherever it is needed
The advantages of AR are obvious: Visualizations foster imagination. Moreover, the user sees all relevant information exactly where it is needed. In the HoloMed research project UID develops an AR application for use in ventricular punctions.
In such an operation, the surgeon introduces a catheter into a patient’s head to drain off excess brain fluid. However, only in two thirds of medical interventions, the catheter is optimally placed, usually requiring several attempts. A head-mounted display projects virtual CT or MRT images on the real head of the patient, allowing the intervention to be performed with millimeter precision at the right position in the right angle and depth. The added value is enormous: more safety and precision.
Challenges and opportunities
Since ARVIKA, augmented reality has made a giant technological leap and now provides realistic solutions. As always, the main challenges lie in the details of design and operation. Here are some examples.
Intuitive interaction solutions
Currently, AR lacks good interaction solutions as the 3D-GUIde project, which is funded by the German Ministry of Economics and Technology, found. In this project, the Stuttgart Media University (HdM) and its partners develop 3D interaction patterns that are checked for usability. This includes draft patterns which provide descriptions for designing intuitive interactions. The project focuses, for example, on points of interest (POI), which show where additional virtual information on real objects is available, providing possibilities for interaction:
In a car dealer’s showroom, a potential buyer can get information on a car’s engine, screens, etc. by looking at a tablet. The number of challenges that need to be overcome in developing such a seemingly simple AR object is astounding. What object in the scene connects to the POI? What happens if there are several overlapping POIs in a room? How do you know where the POIs are located whenever they are not displayed on the tablet because the user is only viewing a screen section?
Usage limitations
In case of the HoloLens, for example, the user generally interacts with the device via gesture and voice control. The gesture control is limited to two hand movements which can trigger an action. But not only Microsoft is currently working on further intuitive forms of interaction. Another shortcoming is the wearing comfort of the heavy, uncomfortable head-mounted displays, which currently can hardly be worn continuously for several hours.
Tracking and hygiene
Where human life is concerned, as is the case with HoloMed, the real and the virtual world have to be perfectly synchronized. An extremely powerful tracking system is needed to capture the position in the room and to bring the virtual and the real world exactly in line. Another challenge in the medical environment is hygiene. How can hygiene requirements be met in an operating room? The steam cleaner that needs to be used in accordance with the regulations would quickly destroy the devices.
Impression of AR on others
What impression does augmented reality have on others? This question currently occupies the Stuttgart Media University. Gestures or voice inputs by the user are not necessarily comprehensible for bystanders and may seem strange or disturbing–this can be another challenge for AR.
Conclusion
AR technology evolves around many conflicting aspects. On the one hand, effective and fascinating projects are created. On the other hand, several issues requiring meticulous work when it comes to optimizing the operability need to be addressed. Moreover, AR should be more than just a nice-to-have—it should always offer added value. But one thing is clear: AR has created a similar hype as the iPhone has after its release. It remains to be seen in how far it will shape our life in the future.
The author
Michael Burmester, professor at the Stuttgart Media University HdM and UX expert, writes for UID.
More
- News: Study: augmented reality has great potential in medicine
- UX Buzzword Jungle: Home, Smart Home – theory and practice of the connected home
- UX Buzzword Jungle: Gamification – a playful way to positive user experiences?
- UX Buzzword Jungle: Intuitive operation – a positive user experience without instructions
- UX Buzzword Jungle: User Experience = Usability plus X?
- UX Buzzword Jungle: Design Thinking – new old creativity