The ways that we interact with computers are poised to change nearly as dramatically in the coming decades as computers themselves have over the past several decades. We’re in the early stages of an evolution from highly complex, techno-centric approaches to much more intuitive, anthropo-centric interfaces. The coming changes will almost certainly impact the way companies do everything from operating industrial equipment in their own production facilities to designing the products that they sell to customers. Here we have sought to outline our view of the current landscape and emerging technologies, relevant companies and the use cases that they’re enabling.

The term Human-Machine Interface (“HMI”) describes the methods in which people interact with computers, and is a subset of the broader User Interface (“UI”) field. This includes independent components and as well as unified systems which encompass:

• Input technologies such as gesture recognition, eye tracking and voice control
• Output technologies such as ultrasound, electro-tactile stimulation, and vibrotactile stimulation
• Analytical techniques such as Machine Learning & Natural Language Processing
• Unified systems which combine those technologies such as Augmented Reality (“AR”) & Virtual Reality (“VR”). Note: we have chosen to focus on AR technologies here. Many of the component technologies we’re discussing have applications in both AR & VR, but they are distinct ecosystems that are seeing interest from very different user bases. Even though “AR/VR” are commonly lumped together as a single topic, we try to avoid making a similar comparison.

The QWERTY keyboard, first developed in the 1870’s, was one of the first attempts to mechanize input from an individual user into a medium that could be stored & shared by many in the future (a letter or book). Designed to avoid routine collisions between keys which slowed typing speeds, it demonstrates the type of technological barriers that were a focus in early HMI solutions. These early approaches required the user to adapt their behavior to fit the machine in ways that were unnatural at best, and highly complex at worst. From punch cards to keyboards, the use of these technologies was reserved either for specialists or for those that specifically acquired a specific skillset (remember typing classes?).

The second wave of HMI kicked off in the early 1990’s, at which point GUIs and display technology had developed sufficiently to allow touch screens to be introduced. This was the first time that the medium through which humans communicated with machines became transparent: the combination of input & output mediums into a single interface gave users the impression that computers could understand human touch & motion the same way other humans do, without requiring a foreign object to mediate the process.

The dramatic decline in the cost of computing driven by consistent innovation in semi-conductors and followed by the smart phone revolution has led to the proliferation of small yet high-powered computers with high-bandwidth connections nearly everywhere that they can be useful to humans today. The lack of space that accompanies a traditional stationary computer (i.e. your desk!) and the capability of the latest computers have simultaneously required and enabled new, intuitive user interfaces to be developed.

Why does this matter?

The aforementioned changes have not been lost on those with large pools of capital to invest, and both venture capitalists and large tech corporations have not been shy about placing bets in the HMI space. VCs have invested more than $1bn per year into HMI technologies for each of the past three years. Following a brief investment spurt into 2D display companies in 2011, excitement about AR (and VR, although to a lesser extent) has accounted for the clear majority of inflows into the space.

Major tech corporations have also been reasonably active making acquisitions, albeit with a preference for earlier stage companies to bolster their technology portfolios and talent pools. Intel (Omek and Indisys), Apple (VocalIQ), Google (Flutter), Freescale Semicondutor (Cognivue), and Facebook-owned Oculus (Pebbles Interfaces) have all targeted HMI startups in sub-$100m transactions.

Applications of HMI in Industry 4.0

Lots of attention has been focused on the application of HMI technologies in consumer use cases like virtual assistants. Instead of repeating that message, we will focus on a few specific applications which are impacting industrial companies.

Industrial machine control is an obvious area of focus. The machinery utilized in industries ranging from oil & gas to healthcare is often plagued with archaic user interfaces which can reduce productivity on the machines which are commonly amongst the most expensive in the facilities where they’re located. Integrating state of the art HMI into these assets can create new efficiencies on many levels:

· Machine operators benefit from receiving physical feedback from haptic technologies, or as wearables or AR glasses free their hands and attention to focus on their primary task. New levels of safety are often achieved, which is far more important than efficiency ever will be.

· Maintenance professionals can quickly visualize machine status in real-time, enabling them to prioritize their workload and anticipate tooling and material requirements for individual tasks. Remote collaboration also allows off-site specialists to consult or guide local technicians through tasks that would require travel otherwise.

· Managers are empowered to quickly survey the status of all operations inside a given plant or facility and receive immediate notification when potential bottlenecks arise.

Across these categories, deployment of advanced HMI technology can help attract young people to join an aging workforce (a trend which is consistent with all Industry 4.0 tech).

The automotive sector has also become a rapid adopter of new HMI technology. Automakers all strive to deliver a differentiated UX that breeds demand for a given brand/model. New HMI technologies are most commonly found in the infotainment systems of high-end models, but the redesign of mundane features like door openers has also demonstrated their applicability.

Touch screens and gesture recognition technology have emerged as favorites for automakers. You’ve likely noticed that the touch screen in your car isn’t as sleek as the one on your smart phone. This is because automakers have favored resistive displays over the capacitive screens found on most phones. The long design cycles (7–10 years), the need for longevity (12 years, 200k miles), and the intense focus on cost all factor into a slow rate of adoption of new technology. However, suppliers like Continental are working to commercialize infrared screens. Infrared sensors are also being evaluated to monitor driver attentiveness in Advanced Driver-Assistance Systems.

Basic gesture recognition sensors are also being applied for tasks outside the car. Drivers approaching their parked Ford Escape with an armful of groceries can now open the tailgate by passing their foot underneath the rear bumper instead of setting down what they’re carrying.

Advertisers are also adopting new gesture recognition and eye-tracking technologies in Digital Out of Home (“DOOH”) advertising. Digital displays are more expensive to deploy, but have been shown to drive 2.5x more revenues per incremental dollar spent. The most advanced ad displays enable a wall-mounted advertisement to become a point of sale. In other cases, they can allow a viewer to access additional content related to the ad via an app download or launching a website from a QR code. Higher engagement combined with new analytics which enable advertisers to target precise groups of users have resulted in new revenue for both media companies and their customers alike.

Conclusion

Many of the use cases identified here may not feel like they’re on the cutting edge of technology in today’s internet age. However, the computerization of industry and manufacturing is one of the key theses underpinning the Industry 4.0 trend. As such, the integration of technologies being utilized in consumer electronics would drive a significant investment and productivity boom, and that’s to say nothing about the potential of state-of-the-art technologies currently in development.

The design and replacement cycles for the types of large physical assets we’ve covered are certainly longer than those in the consumer electronics industries. However, the highly capital-intensive nature of the industries that are being transformed means that dramatic improvements in cost structure and new revenue generated through operating efficiencies are possible.