I recently recorded a podcast with Stuart Thompson, President of Electrification Service Division at ABB, to discuss how the company is using augmented reality (AR) and artificial intelligence (AI) for its field service operations.

We've been talking about using AR and/or virtual reality (VR) for service for a while now, but until the COVID-19 pandemic take-up of the technology seemed slow. While it seemed there were some viable use cases for field service, factors such as connectivity, cost, and change management kept adoption minimal. Things changed when the pandemic forced everyone to look for some type of remote technology for just about everything, including troubleshooting and training. A lot of companies quickly began experimenting with AR and VR for service and other industrial applications, and four years later we are starting to see the fruits of that experimentation.

When you consider some of the AR-related items in the news, I think they tie in well with that pandemic-inspired push to use AR/VR, as well as what ABB had to say about it.

While we normally think about AR/VR for service, it can be used in a lot of other applications that touch service, even peripherally, including computer aided design (CAD). A lot of manufacturers are trying to tie their service data back into the design process so insights from the field can help improve future product designs. One of the companies leading that charge has been Siemens, which (among other things) sells CAD software. The company just demonstrated a VR-based system developed with Sony that will allow engineers and designers to actually do some of their design work using AR and VR.

The other story was about a hydraulic pump manufacturer called Permco that faced a lot of challenges during the pandemic. The company's pumps require hand assembly at the factory, and new hires typically went through a lengthy in-person training process to learn how to build the pumps. The pandemic made that type of training impossible, but a software developer working for the company came up with a fix – a VR solution that lets trainees learn to assemble pumps using a headset, internally developed software, and data from their CAD models. New employees can assemble pumps hundreds of times in VR before they ever touch the physical pump. They’re even adding haptic gloves, so trainees can “feel” the parts they are working with.

AR Aids in Labor Challenges

While that's an assembly application, you can pretty easily see how that type of system could help under-staffed service companies make repair training faster and less labor-intensive. Even if using AR/VR at a job site might not be a workable solution for service techs, virtual repair training could be a big boost for an industry that doesn't have enough seasoned technicians to go around when it comes to on-the-job training/mentoring.

Which brings me back to ABB. Like Permco, the company turned to augmented reality because COVID made some in-person activities impossible. According to Stuart, a large project in China that required complex remote training was what finally pushed ABB to deploy AR into the field. After looking for some wearable devices that were practical for technicians to use, they shipped the units to China to help with training.

“[The] Chinese engineers were already trained in basic electrical infrastructure and systems. However, they might not have had the deep domain expertise on the particular piece of equipment they were working on,” he said. “We could then through wearable devices project onto the equipment and we could have engineers sitting in the U.S. … guiding them verbally and visually on the priorities and what to do and what not to do. [T]hey were able to communicate in English and then the tools were translated into Chinese for the field engineers.”

A short time later, they were able to do the same thing by connecting technicians in Germany with their counterparts working in a mine in Chile. “People were open to using it because they had no other choice at that point in time,” he said. “And we usually find in times of crisis like this, technology can move extremely quickly and people become much more open and adapt to using it.”

For service, of course, deploying AR for training technicians or helping remote techs work through a problem can represent a major change in workflow for organizations. But if it means saving several days of travel and/or struggling with translators when every minute of downtime counts, the investment is going to be worth it for a lot of companies.

We covered a lot of ground during our conversation (including how ABB is using artificial intelligence along with AR), so you should check out the entire podcast.

Have you found any good, practical uses for AR/VR in service? I would love to hear about them.