Maintaining and servicing modern aircraft is a complex task.
However, researchers at the Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE have developed an exciting solution using augmented reality (AR) smart glasses.
By visualizing 3D information, these glasses make repair and maintenance work easier for aircraft mechanics.
Through gesture, voice, and gaze control, mechanics can interact with virtual elements like instructions, displays, and technical tools. This innovation has been applied to develop concepts and solutions for maintaining the Airbus A400M.
Aircraft, such as the Airbus A400M, are equipped with numerous mechanical and electronic components.
For instance, the engine alone consists of over 10,000 individual parts. This complexity makes aircraft maintenance and servicing demanding tasks. Traditionally, aircraft mechanics have relied on printed operating manuals for guidance.
However, these manuals can be cumbersome to use, particularly when multiple manuals need to be referenced simultaneously.
In tight spaces like the cockpit, juggling documents can lead to clutter and confusion.
Additionally, the two-dimensional representations of complex assembly tasks in the manuals are not always self-explanatory and can be misleading, potentially leading to maintenance errors.
To overcome these challenges, virtual 3D guides overlaid in the mechanic’s field of vision using AR smart glasses can provide a long-term solution, replacing two-dimensional maintenance instructions.
The “Ariel” project, conducted by researchers at Fraunhofer FKIE’s “Human-Machine Systems” department, evaluated how augmented reality can assist aircraft mechanics in maintenance work. They focused on two use cases: “Installation of a display unit in the cockpit” and “Maintenance of a battery in the workshop.”
The researchers tested prototype concepts using two types of AR glasses—the Microsoft HoloLens 2 and Epson Moverio BT-300.
They considered the design of suitable 3D information visualizations and interaction techniques like gesture, gaze, and voice control. The tests involved five aircraft mechanics and addressed usability, user experience, and comfort.
To cater to the different AR glasses, each pair was provided with a dedicated interaction concept.
This involved analyzing the workplace approaches of the mechanics, checking their personal gear’s compatibility with the glasses, and developing unique interaction concepts based on use, organization, and design requirements.
The concepts were tailored to the distinct environments of the cockpit and the workshop, considering factors like limited gestural interaction in the cockpit and increased noise levels affecting speech recognition.
The team developed 3D representations of existing regulations and supplemented them with animations to illustrate specific steps directly on the virtual components. The mechanics could perform tasks such as measuring battery resistance and follow special notations on the cockpit model.
Multimodal input techniques allowed the mechanics to interact independently, utilizing gaze and gesture control.
To avoid unintended input, a gaze control method was implemented, requiring users to briefly focus on an interaction element.
Additionally, a learning environment was developed to teach basic gestures and interaction techniques using animations.
Overall, the mechanics found the interaction to be intuitive, particularly appreciating gaze control over gesture or voice control.
The overlay of relevant information during battery maintenance helped identify specific components, and animations for assembly tasks received positive feedback.
However, there were some uncertainties with gesture control, prompting the research team to develop additional learning materials.
The researchers plan to digitize documentation and enable remote maintenance tasks, including collaboration with experts. The application of augmented reality in maintenance and repair work extends beyond aircraft, offering benefits in other sectors as well.
For example, it can assist less-experienced technicians with tasks like installing heat pumps through three-dimensional step-by-step instructions.
Augmented reality has the potential to compensate for the shortage of skilled workers in various industries, providing effective guidance for maintenance tasks.
Follow us on Twitter for more articles about this topic.
Copyright © 2023 Knowridge Science Report. All rights reserved.
