When everyone is a knowledge worker, UIs need to shift

I talk a lot about how we’re embedding computing into everything, and I try to explain what that means for everyday people. In the home, it can mean we get smarter ovens that make it easier to cook a delicious meal. In the workforce, it may mean that we stop spending time checking on machines and more time having to make decisions about how to optimize them.

In short, we’re letting computers take on more of the grunt work of everyday life, presumably leaving us free to think more and be more creative. This began happening decades ago, when computers invaded offices and in the process, fundamentally changed the nature of white-collar jobs. But as computing moves to farming, industrial manufacturing, and service technician roles, the practical implication is that everyone will become a knowledge worker.

But the trappings of today’s knowledge workers — generally a laptop with a keyboard — make little sense for people who work on sales floors or on manufacturing lines. For them, the user interface for computing needs to change. Voice is one element; heads-up displays and devices that can track gestures are two others.

This thinking came to a head for me after a conversation with this week’s podcast guest, Mark Webster from Adobe, who discussed ways to bring voice into the enterprise, and a demonstration of Microsoft’s HoloLens 2 augmented reality glasses. Together they made me realize that, while thanks to the internet of things and AI we are able to deliver to workers actionable insights, for those workers to take full advantage of such insights we need to deliver those insights directly to them.

In other words, don’t make a manufacturing worker walk over to a tablet on a machine to read the information it’s displaying regarding the health of that machine. Give them the alert on their glasses, or on a wearable that lets them address the problem in situ. And don’t ping the phone of a salesclerk on the floor of a clothing store with a notification about a customer’s likes when an audio alert in her ear will allow her to continue her conversation with said customer and seamlessly suggest to her some dress options.

Broadly speaking, we can think of AI in the workplace as a type of digital assistant helping each employee do his or her job, complete with the intelligence needed to choose the interaction medium that makes the most sense for each employee’s individual situation. Right now, most people think of digital assistants as a voice assistant, but that’s far too limited of a view.

To make workers truly productive, enterprise and manufacturing software will have to be designed with different devices and user interfaces in mind. In trying on the HoloLens 2, I was impressed that I could interact with it using voice, gesture, and even my gaze. In a loud factory environment, voice may not be the best way to communicate, but a gesture or a gaze could be. If someone’s hands are busy trying to wrangle a machine, then gaze might make the most sense. It might also come in handy in a cubicle farm.  A voice prompt might notify the worker of an incoming alert, then show it on a heads-up display or a watch. A quick flick of the worker’s gaze or a tap of their finger might be enough to act on the information — all without ever disturbing their colleagues.

Of course, voice could still win out on a loud floor as a form of input and output. The folks at Microsoft might not think of voice as a good interaction for loud environments, but Realware, a startup that recently raised $80 million for its rugged, voice-enabled head-mounted display, has managed to make voice work in manufacturing and has the clients to prove it.

The point here is that the worker of the future — whether she’s on a manufacturing line or striding through an orchard — will be a knowledge worker, just as an accountant or lawyer is. So we need to develop the tools, software, and design paradigms that will enable that worker to get the information she needs in those environments and act on it.