In line with Industry 4.0 objectives, many manufacturers strive to streamline their processes to quickly produce consistently superior products. Smart manufacturing relies on the cooperation of humans with leading technologies, and this collaboration flourishes as user experience improves. Augmented Reality (AR) supplements a user interface that represents real-life situations and data with helpful information in the form of visual, audio, or haptic signals. 

Real-time information provided through augmented reality technologies can help manufacturers increase quality, efficiency, and safety. AR brings in all the benefits of automated inspection—time savings, accuracy improvement, and nondestructive defect detection—while still including human expertise and judgment at each turn. Step-by-step guidance tailored to each employee and product keeps schedules running on time. Visual, audio, and haptic alerts keep employees aware of their surroundings and prompt alerts to promote safety. Benefits of AR for inspection and quality control are especially pronounced for training new, inexperienced employees, but AR still provides significant utility to experienced employees. 

Research has produced three use cases that serve as proof-of-concept to introduce the benefits of Augmented Reality to Industry 4.0: printed circuit board assembly inspection, quality assessment of polished surfaces, and cognitive support for assembly operations. 


A motherboard is a common example of a sophisticated PCBA. This image shows the immense amount of detail PCBAs can pack into a tiny space. This research team has shown how VR can zoom in on specific areas for side-by-side comparison of each PCBA component with expected standards. Source: Pexels

Case 1: Printed Circuit Board Assembly Inspection

Printed circuit board assemblies (PCBAs) are used in nearly all modern electronics, and rely on highly precise construction of tiny components. The density of components can make fully automated detection challenging, and PCBA manufacturers often require human expertise to maintain quality. This study embraces the flexibility of a markerless, automated inspection that guides a user through checks of specific measurements on a PCBA. 

The research team used wearable sensors—specifically, an optical see-through head-mounted (OST-HMD) system—and motion detection to establish reference points and reorient the user. Processing is handled via wireless communication to a remote automatic optical inspection (AOI) station. Overlaid 3D CAD-based arrows indicate the measurements of specific components in real time to the user, and detected defects are brightly highlighted as they appear on the board. Current progress is tracked on a status bar, and missing steps are called out with specific auditory cues. This ensures that all required measurements are satisfactory before moving on to inspect the next PCBA.

A few key features make this study stand out:

  • Operators can compare sections side-by-side with reference images, or they can zoom in on intricate areas to check production down to the finest detail. This feature proved invaluable, as operators did not need to shift their gaze between reference materials and the PCBA under inspection.
  • Users in this study agreed that the head-mounted display was highly advantageous over a handheld video see-through device, like a tablet or smartphone.
  • This system showed novelty in the successful application of markerless inspection—many similar systems require existing markings to ground displays, but the algorithms developed in this study were self-orienting.

For robustness, researchers tested this setup against PCBAs of three different dimensions and configurations, with both a tabletop and handheld inspection method supported for each. Future work in this domain would incorporate wireless charging to avoid disrupting the operators, and additional configurations of reference images to maximize their efficiency and utility. Additional verification of the algorithm and its applicability to a broader range of PCBAs would further strengthen this study. 


man wearing AR goggles

Researchers utilized real-time environment enrichment using AR to inspect polishing quality using headsets similar to those in the image above. Source: Pixabay

Use Case 2: Quality Assessment Of Polished Surfaces 

Polishing is used in nearly every sector to achieve optimal surface qualities and perfect material thickness. While most phases can be done automatically, highly skilled operators are best suited to achieve peak quality in the final polishing steps. In an effort to reduce overall time and cost, this study introduces AR to quickly evaluate surface quality in the finishing steps of manufacturing processes. 

Using a robot, these researchers scanned the surface in question to gather relevant metrology data. The robot’s sensors moved along predefined paths along the surface and inspected depth consistency and texture irregularities. Through an AR device, these measurements were projected directly onto the surface for operators to determine next steps in context. If the surface quality was deemed acceptable, the operator could move on; otherwise, suggestions were offered for particular metrics in need of improvement. 

The users supporting this research team found resounding success with this tool and were pleased with the accuracy and efficiency it provided. The projection of problem areas and objective associated measurements simplified and expedited decision-making processes. This highly targeted approach isolated only areas that needed improvement, and it avoided accidental disturbances of areas that had already met acceptable measurement standards. These factors massively reduced the time, resources, costs, and uncertainty associated with the final steps of polishing processes. 

Supporting research might further investigate the alignment of displayed graphics with the real-time images to ensure accuracy. Opportunities for additional automation include spot-checking by an operator, with actual polishing work conducted by a robot to further reduce polishing time. 


measuring small parts

The second task focused on quality control in measuring small parts, using a similar technique with calipers as shown in the image above. Source: Pixabay

Use Case 3: Cognitive Support For Assembly Operations 

Operators are increasingly put under physical and mental stress and experience a split-attention effect as they shift between reading instructions and completing tasks. As this is very likely to drive up error rates and slow down assembly times, AR can help operators focus only on information relevant to their current task. This study explains the advantages of AR over other instruction methods, including reduced operator stress, faster training times, reduced error rates, and an overall increase in production efficiency. 

Operators were presented with three tasks, which took four to twelve minutes each: 

  • Assembly of a basic light fixture. Operators placed a sticker on a specific location, connected specific wires using springs and screws, and applying glue to defined points.
  • Quality control of an aluminum part. Using a caliper, operators measured various dimensions of a small part, and separated parts that fell outside of the provided tolerances.
  • Assembly of a support wheel. First operators attached axes, positioned wheels, added separator bushings and supporting hardware. This task concluded with an inspection of the assembly.

While the prior two use cases required highly skilled operators, this study focuses on operators with cognitive or motor disabilities. This study used projection to maximize user comfort, enable hands-free operation, reduce the split-attention effect, and avoid the use of markers or batteries. AR proved to be the least stressful method for novice operators, and AR reduced errors across all levels of expertise. This enables flexibility in assembly line operations, as employees can be assigned quickly to new tasks as needed with minimal interruption.

To bolster applicability of this study, additional research might explore the personalization of instructions to the expertise level of the assembler. Customized instruction could reduce redundant explanations for experienced operators, or it could minimize the learning curve and discomfort of unseasoned operators.


Conclusion

These three use cases span a wide range of industry applications, from electrical design to operational strategies. Each study uses a unique combination of input data and provided output cues, but these barely scratch the surface of future AR applications. While there are many outstanding technical challenges, the largest barrier to bring AR into industry will be organizational change. These use cases serve as proof-of-concept for both the immense technical and quality advantages AR can bring to Industry 4.0, and they provide a preview to the operational adjustments that AR will require.