Augmented Reality becomes mainstream in Manufacturing, changes the face of the industry

As it’s often the case with new technologies, Augmented Reality (and its more evolved sibling – Mixed Reality)  have attracted a lot of excitement in the tech world and sparked the imaginations of many entrepreneurs. This excitement is easy to understand. This is, after all, a technology that lets us blend the digital with the physical and enhance our senses with super-human capabilities.

However, the value of AR/MR was not immediately obvious to everyone in Manufacturing (we are a skeptical bunch). Early examples of AR/MR applications did not help its case: they showed a group of engineers gathered around a 3D model, waving their hands in the air to virtually pull parts apart and mark things up with “please review this part” virtual notes. The problem with this was that these examples were essentially the same scenarios already beaten to death by VR, with the only apparent difference that engineers used see-through glasses instead of opaque head-mounted devices. So, naturally, people wondered what the big whoop around AR/MR was.

But soon enough folks in the industry realized that the real value of AR/MR was in augmenting the real world with digital information, and not just projecting 3D models in physical space. In other words, providing additional context around the thing you were looking at. That’s when AR/MR “clicked” and more innovative and useful applications started to emerge, now attracting the attention of even the most skeptical and crusty manufacturers.

These AR/MR applications are changing the face of manufacturing: in the plant floor, the field, and in design departments. It’s rapidly becoming the new user interface for factory personnel, service technicians, and engineers.

We can group these emerging applications into two broad categories:

1. Applications that overlay digital information on top of physical things. Information such as:

• Operation, maintenance, and quality control instructions: to guide machine operators, service technicians, and quality assurance staff through the steps needed to perform a task (assemble a component, service a machine, inspect a part). One popular application of this – in an industry hungry for solutions to address an acute skills gap – is training.

• Real-time IoT data: which can help plant workers to quickly identify issues with a machine or process in the plant floor, or a service technician to better service a piece of equipment. For example, thyssenkrupp’s service technicians use Microsoft HoloLens to visualize real-time data about the health of their elevators, which allows them to make better informed decisions on how to best service the units.

• 3D models: of assets and equipment, to preview how they will fit in the building (without the heavy lifting). Microsoft recently released an MR application called Microsoft Layout to do just that. ZF Group, a German car-parts manufacturer, is using Microsoft Layout to help them layout equipment on its factory floor.

2. Applications that enable remote collaboration. For example:

• Remote expert assistance: this is in my opinion one of the killer applications of AR/MR. Here, a technician can collaborate remotely with an expert with heads-up, hands-free video calling, image sharing, and mixed reality annotations. Microsoft also recently released an application to enable this scenario, called Microsoft Remote Assist. ZF Group, mentioned earlier, is also using Remote Assist to connect with remote experts to help them troubleshoot and fix problems with their production machines in hours instead of days that would be required if they had to fly someone in.

• Design reviews/immersive design: AR/MR is a better option than VR for design in those cases where it is important to design the product in context of its physical environment. For example, thyssenkrupp designs custom-built stair lift home solutions. With Microsoft HoloLens and Azure, sales technicians share measurements with remote design teams, and show customers how the stair lift will look, all in one visit.

Getting started with Microsoft AR/MR

You can get started developing exciting AR/MR applications with Microsoft right now! I recommend the following steps:

1. Read the general overview of Mixed Reality. 
2. The Mixed Reality Development Launchpad is your starting point for development. Start with the “Get started” section. 
3. Complete a few tutorials from the Mixed Reality Academy. 
4. If you are interested in Microsoft’s Mixed Reality apps, check out Microsoft Layout and Microsoft Remote Assist. 
5. Learn more about how thyssenkrupp is using HoloLens and Azure IoT Suite.