D-Keine/iStock/ Getty Images Plus
Unlike augmented reality headgear, some virtual reality goggles restrict the wearer’s view of physical surroundings.
Unlike augmented reality headgear, some virtual reality goggles restrict the wearer’s view of physical surroundings.
Unlike augmented reality headgear, some virtual reality goggles restrict the wearer’s view of physical surroundings.
Unlike augmented reality headgear, some virtual reality goggles restrict the wearer’s view of physical surroundings.
Unlike augmented reality headgear, some virtual reality goggles restrict the wearer’s view of physical surroundings.

Extended Reality Spreads Its Wings

March 20, 2019
As augmented and virtual reality technologies mature, industrial operations and maintenance applications stand to benefit, including some specific electrical scenarios.

Reality — at least at the level it’s perceived and stipulated to by humans — may no longer be a state sufficient enough on its own to engage with elements of the physical world. That’s a “deep” notion on a philosophical level, perhaps something for students of metaphysics and human consciousness to ponder. But on a practical level, it speaks to a very tangible revolution taking place in how the experienced world is being transformed.

Through advanced computing, communications, modeling, and imaging technologies, reality, in a sense, can now be transformed. Something present in the real world is capable of being digitally recreated or reimagined, and interacted with virtually, courtesy of virtual reality (VR) hardware and software tools. And, through augmented reality (AR) technologies, what’s seen or otherwise experienced in real time can be supplemented or enhanced in any number of ways.

Experiences along what might be called the extended/assisted reality spectrum, which include a blend of AR/VR dubbed mixed reality, can be achieved with varying degrees of complexity and innovation. Sophisticated computer simulation headgear that envelops the wearer produces an immersive, interactive VR experience. And “smart glasses” or headset devices that render heads-up, line-of-sight visuals, text, data, mounted-camera views, video and live voice communication, as well as app-enabled and connected phones, tablets or other mobile devices, can deliver a range experiences that fall on the AR continuum.

Extended reality has cut its teeth on development of next-generation entertainment, but it’s making inroads into more serious and consequential applications in fields ranging from science and medicine to transportation and construction. Its potential value lies in mimicking, simulating, improving, or enhancing physical world interactions, leading to better outcomes across many scenarios.

One that’s fast emerging is how operations and maintenance (O&M) personnel might begin to engage with assets and infrastructure deployed in industrial environments. Extended reality platforms are opening new ways for staff tasked with keeping physical plants of all sorts up and running to do their work. From worker training and real-time operations assistance to troubleshooting and preventive maintenance, reality simulation and enrichment technologies could empower O&M teams to boost productivity, learn more quickly, reduce errors, and work more safely.

O&M applications are slowly starting to transition from pilot and test phases now, and they span a range of industries and use cases. Developers of both software and devices that create and deliver content to the user are partnering with transformative technology-hungry companies in sectors like manufacturing, petrochemicals, and utilities. Myriad elements of the physical plant in each of those areas demand constant oversight, and lean-in companies see AR/VR as a vehicle for improving the process and addressing emerging industrial O&M challenges and opportunities.

Electrical Prospects

While there’s little evidence that extended reality technologies have penetrated electrical systems in industrial O&M settings, trials that have been conducted suggest routine tasks would be a natural fit. From electrical motor maintenance and repair to electrical distribution system troubleshooting to lockout/tagout procedures and other precautionary safety tasks, AR/VR could prove a powerful tool for helping technicians safely learn and perfect critical skills in training, allowing them to venture into the field with higher levels of knowledge, confidence, and support.

A 2016 research paper by Forrester Research on prospects for using smart glasses for workforce enablement, targeted at infrastructure and operations professionals, ranked electrical-related work high on the list. It foresaw maintenance and repair workers, electricians, electro-mechanical technicians, and commercial/industrial electrical and electronics repairers being among the biggest users of that presently core AR delivery device by 2025. By then, the report projected, some 14 million workers across many industries — 8% of the workforce — could be using smart glasses, and therefore AR, in their jobs. The predictions were based partly on interviews with about a dozen companies using or developing smart glasses technology at the time.

Since then, AR/VR sampling in the broadly defined industrial sector has continued, leading some companies currently active in the space to predict that industrial broadly and O&M particularly could become major beneficiaries.

Acensium, LLC, a Fort Myers, Fla., company that develops laser scanning and 3-D modeling technology central to more advanced assisted reality content creation, sees a long runway ahead for industrial facilities unpacking both AR and VR. Cale Collier, vice president of scan and model for the company, says reality support technologies ultimately could propel the workforce in these environments to unseen levels of productivity.

“The learning curve of newly on-boarded operators and maintenance personnel could be drastically shortened,” Collier says. “Initial training can occur completely in VR, and ongoing training, compliance checking, and in-field support would be handled through AR tools.”

Since O&M work is so dependent on training, procedures, and documentation, the ability to bring that digitally to workers in any mode they’re in is a key value proposition for enhanced reality scenarios, says Raj Raheja, cofounder/CEO of Heartwood, Inc., a San Mateo, Calif., company that puts detailed 3-D industrial equipment views into applications accessible in assisted reality sessions.

“The vision is to give every worker the ability to learn by doing by working with simulations in a training mode and then to be able to take that into the field and access it with devices they already have,” he says. “The information has to be visual, interactive, and portable.”
O&M workers, he says, are in a continuous cycle of learning, training, and performing field work, and technology that seamlessly connects those functions can pay dividends.

Helping Productivity and Safety

The need to work safely, without sacrificing competency, will be another factor driving adoption of the reality assist technology in industrial O&M, says Sanjay Jhawar, cofounder and president of Vancouver, Wash.-based RealWear, Inc., developer of a head-mounted wearable AR device — the HMT-1 — that delivers hands-free voice- and head motion-activated navigation of content provided through an Android-class tablet computer, viewable at the perimeter of the field of view. On-demand information relevant to tasks and connectivity, in the moment, will be key to elevating worker productivity and safety — two goals frequently in conflict in O&M work. AR delivered in a format that provides visuals in the line of sight and doesn’t require any keyboard inputs to activate it is the solution, he says.

RealWear’s device has attracted interest from outfits in various sectors, including utilities, port operations, and telecom services, Jhawar says. It’s been used by an oil refinery company to aid in the installation and wiring of electrically powered valve control equipment and connection to PLCs; by a wind farm-turbine servicing company looking to extend technicians’ ability to readily service different makes of equipment; and by an electrical contractor assessing its value in guiding workers on construction projects.

In May 2018, industrial technology provider Honeywell announced it was teaming up with RealWear to market a wearable AR solution targeting industrial customers looking to take field service worker productivity and safety to new levels. The HMT-1 delivers content generated by Honeywell custom software, giving wearers speedy access to a range of work aids — from live data, documents, and work procedures to health and safety guides and real-time connection to remote experts.

“This is something that plays into a big trend of more investment in industrial assets that are more sophisticated and connected, and the emergence of the Industrial Internet of Things and growing interest in predictive maintenance,” Jhawar says.

Honeywell appeared to double down on extended reality’s promise in the industrial sector last year with the announcement of a mixed reality solution for training customer plant personnel to service and maintain one of its controllers. In February 2018, it said it was introducing a combination AR/VR product that would allow trainees to view cloud-based holographic 3-D simulations of various O&M scenarios involving the controller, including primary failure and switchovers and cable and power supply failure. The simulations would be viewable within

Microsoft’s wearable holographic mixed reality device (HoloLens) and Windows Mixed Reality Headsets in tandem.

Donning the devices, workers would be able to interact with holographic models of the controller and intake detailed information on procedures they’d need to deploy in the field. Honeywell characterized the product as a “flight simulator”-like approach to training and testing that could double the amount of skill retention over traditional methods and cut training time by up to 150%. In the announcement, Honeywell said the initiative was a response to a growing demand for more creative and effective industrial training that could lead to “improved plant performance, uptime, reliability, and safety.”

For its part, Microsoft’s partnering with Honeywell looks to be part of a broader strategy to extend HoloLens’ reach into the industrial space and empower what it calls “firstline” — hands-on — workers. Introduced in 2016, HoloLens, more goggle-like and enveloping than smart glasses products, has seen action in industrial field technician service, training, and production realms. Field technicians with Germany-based elevator company thyssenkrup AG use it to understand potential maintenance scenarios ahead of a call and access technical and expert help on site; Chevron has substituted HoloLens for certain in-person equipment inspections requiring overseas travel; and Mercedes-Benz Global Training has 100 of the units deployed to conduct what Microsoft calls “faster, more engaging training sessions.”

To deepen HoloLens’ penetration into the first line worker world, the company unveiled its Dynamics 365 Remote Assist app last fall just for HoloLens. Enabling deeper collaboration using video calling, image sharing, and mixed reality annotations, Microsoft said in a statement, is a primary feature of the app, pre-configured for integration with Microsoft’s Dynamics 365 Field Service software.
“Workers share what they see and collaborate with experts remotely,” the statement read. “Workers and experts annotate their shared view with mixed reality ink and arrows or insert images into their view to pinpoint and solve problems efficiently.”

Wearables Challenges

Interest in extended reality applications has been animated by evolution of devices like the HoloLens, the RealWear and Oculus Rift VR headsets, the DAQRI Smart Glasses for AR, and the pioneering, but mothballed Google Glass, but hardware adoption comes with learning and comfort challenges. Early VR products have been plagued by incidents of user motion sickness, while some field-worn AR headgear, despite promises of bringing a net improvement in safety, is clouded by concerns that it will cause distraction and discomfort for workers.

With lingering questions surrounding the real-world availability and practicality of wearables, Raheja says, the focus in extended reality solutions should still be the content more than the device. What matters for Heartwood, he says, is the quality of the 3-D renderings and the interactive simulations they produce, which can be readily accessed and experienced on desktop or mobile devices. Workers can interactively engage with content in both dedicated training environments and in on-the-job training and field work without having to step into the more alien sensory world produced with wearables.

“Our vision is to give every worker the ability to learn by doing and to take that content with them whether they have a phone or a tablet as they work, on the web when they’re at home or at some point with a AR/VR headset,” he says.

Heartwood has developed procedural training products for operating, servicing, and maintaining equipment in a broad range of industries. Lockout/tagout (LOTO) to OSHA standards is among the tasks that have been put into “learn-by-doing” interactive simulations for customers. It demonstrates, with 3-D renderings that can be manipulated by the trainee, how to move through a step-by-step LOTO checklist procedure using queries that test the user’s knowledge, which is assessed in a follow-up exam.

On the Heartwood website, a safety training coordinator for a manufacturer customer is quoted as saying the training, because of its interactivity, helped workers who had been “grappling with concepts” understand LOTO’s importance and how to do it properly. “Now we use the simulation to onboard new hires,” the coordinator said.

Another proponent of making extended reality technology more device-agnostic, and therefore more accessible, is Upskill, a Vienna, Va.-based provider of enterprise AR software. It recently commercialized a smartphone- and tablet-optimized version of its Skylight AR platform, complementing one designed for HoloLens and other smart glasses.

Jason Kim, chief strategy officer, says hands-free access to mixed reality experiences via wearables is important for some tasks, but not all. In many, the ability to consult a mobile-device screen is sufficient. In still other situations, a blending of the two is optimal.
“Smart glasses and their unique ability to enable hands-free, heads-up interaction with task-specific information deliver huge value, but an expanded choice of devices is critical to bring the digital enterprise to every hands-on worker,” he says.

Yet it’s hard to imagine that the envelope won’t continue to be pushed on extended reality immersion. If the ultimate goal is to transform the work experience so errors are reduced, tasks are completed more quickly and worker safety is maximized, the pursuit of advanced wearable technology that deposits workers into assisted environments that are navigable and knowledge-rich will surely continue. At some point, capabilities and functionality will merge.

“In industry, AR platforms must be hands-free and need to allow as much interaction with the environment as possible,” says Collier. “Danger lurks at every turn, and wearers need to be able to process inputs and have full mobility.”

And VR, he predicts, will continue to be perfected and become a core part of the training function, especially in O&M scenarios that pose danger.

“Extremely high-value simulations that train for activities that could lead to catastrophic loss of life or assets will incorporate gloves, vests, and other wearables that provide feedback, and training environments could recreate extreme temperatures or other
stimulation,” he says.

Many questions remain unanswered with extended reality technology’s shape and path forward, but one near certainty is that it’s right for the times. A driving factor, most observers agree, is that tomorrow’s workers will be receptive to — or even demand — training delivered digitally, visually, and interactively, and that they might come to savor technology-aided reality assistance as they work. It could even be an efficient conduit for moving the knowledge, experience, and skills of an expected mass of retiring skilled workers to younger replacements.

“Extended reality offers many forms of knowledge transfer, whether its cloud-accessible operations manuals, recorded and narrated procedures, photos and video clips of daily operational tasks,” says Jhawar. “In the bigger picture, it can become the tip of the spear in a company’s digital transformation initiative, improving the interactions that humans have with machines.”

Zind is a freelance writer based in Lees Summit, Mo. He can be reached at [email protected].

Sidebar: Research Points to Growing AR/VR Industrial Interest

Extended reality technology is still in its formative stages, but it’s not so early that collected research data can’t shed some light on what’s happening with augmented reality (AR) and virtual reality (VR). Research companies are busy assessing trends and surveying current/prospective users, and the numbers they produce offer snippets of insight. A few that catch the eye include:

39%: Share of U.S. online adults interested in using VR training for specific job duties, according to a 2019 Forrester Research report on extended reality in manufacturing.

31% and 30%: Share of executives at 700 industrial companies testing/using AR/VR whose firms make use of it for viewing reference videos and digital manuals and for visualizing specific components and functions behind physical barriers, respectively, according to Capgemini Research Institute’s “2018 Augmented and Virtual Reality Survey.”

$1.54 billion: Amount “discrete manufacturing” companies will spend on AR/VR products in 2019, according to the IDC Worldwide Semiannual “Augmented and Virtual Reality Spending Guide.”

45% and 55%: Share of industrial companies deploying AR who are conducting proof of concepts/pilots, and conducting small/large scale implementations, respectively, according to Capgemini.

7%: Share of global information workers using a smartphone for work who use VR or AR apps on it each week as part of their job, according to Forrester Research.

$20.4 billion: Forecast worldwide spending on AR/VR in 2019, according to IDC.

$413 million and 119%: Amount forecast to be spent in 2019 on industrial maintenance AR/VR, and the projected five-year compound annual growth rate through 2022 for such spending, resulting in total AR/VR spending that could eclipse that for AR gaming by 2022, according to IDC.

About the Author

Tom Zind | Freelance Writer

Zind is a freelance writer based in Lee’s Summit, Mo. He can be reached at [email protected].

Voice your opinion!

To join the conversation, and become an exclusive member of EC&M, create an account today!

Sponsored Recommendations

Electrical Conduit Comparison Chart

CHAMPION FIBERGLASS electrical conduit is a lightweight, durable option that provides lasting savings when compared to other materials. Compare electrical conduit types including...

Fiberglass Electrical Conduit Chemical Resistance Chart

This information is provided solely as a guide since it is impossible to anticipate all individual site conditions. For specific applications which are not covered in this guide...

Considerations for Direct Burial Conduit

Installation type plays a key role in the type of conduit selected for electrical systems in industrial construction projects. Above ground, below ground, direct buried, encased...

How to Calculate Labor Costs

Most important to accurately estimating labor costs is knowing the approximate hours required for project completion. Learn how to calculate electrical labor cost.