Where does generative AI fit in the IoT?

Major publications are really worried about Alexa, Siri, and Google’s digital assistant these days. That’s because, so far, these products aren’t using generative AI, a set of machine learning models that are trained to guesstimate the next set of words or the right image to create based on a prompt.

DALL-E, Stable Diffusion, and Midjourney are popular on the image generation side, and came out back in early 2021. But the latest hype is around large language models, specifically ChatGPT created by OpenAI. When someone puts in a prompt and a style suggestion, the results can make easy reading and give actual information or the illusion of information.

All of these tools are exciting and will have a significant impact on how work gets done, how people create content, and how we build businesses. And yes, there are concerns over accuracy, the ability to use generative AI for easy propaganda, scams, and other malicious acts that we still must grapple with rather than simply blindly accepting that generative AI is the messiah technology that will Change The World.

That said, right now, we’re at the stage where technologists are creating blind belief in generative AI, including by putting billions of investment dollars into companies experimenting with use cases and new models. We’re also at the stage where members of the media spend hours trying to trick these models into behaving badly or try to prove that the AI is sentient and may have hostile feelings toward us.

I’m not really here to talk about that. I want to focus on areas where generative AI will have a significant impact on the way the Internet of Things gets deployed and used. For example, where can we use it to improve user experiences? What types of jobs might it aid or take over? And words and images aside, what other generative AI models might help with the IoT?

I’ll start in the smart home, because that is where my heart is. Rather than confuse Amazon Alexa with what is essentially a super advanced chatbot, and predict that Alexa will “lose the AI race” or have its thunder stolen, it’s likely that it and other digital assistants will continue to use natural language processing (NLP) for “understanding,” then taking action on, various task-based requests like “Turn on the lights,” or “Good morning” to start a wake-up routine, while also adding in a GPT-style chatbot to handle requests that require more in-depth communication.

A good digital assistant isn’t going to have one or two models, but be comprised of whatever version of models provides the most utility for the user. There are also economic concerns. Calling out to a chatbot might incur charges that require a different business model, and not everyone will want to pay a subscription to get Siri to tell them a story or share with them the best recipes based on the food in their fridge.

Plus, sooner rather than later we will see chatbot-style generative AI models in use in the home. I recently had Paulus Schoutsen, the founder of Home Assistant, demo how to use a HomePod to access a GPT-style chatbot to tell his kid a story. And I think being able to combine the NLP already in use inside a smart speaker to translate my spoken words into written prompts for a GPT-style model could make for an easy way for me to describe a routine for Siri and then write the code or connect the integrations needed to implement it.

Indeed, I think the utility of using both the NLP that’s already part of digital assistants in combination with a generative AI model is clear to SoundHound, which is introducing a platform that combines a voice assistant with a generative AI. So ChatGPT won’t kill Alexa, but it will probably end up becoming part of Alexa, with Alexa as the interface and ChatGPT just one of many services it provides.

Other areas in the smart home where ChatGPT or generative AI models will have an impact include children’s toys, fitness services (have the model deliver a custom workout), recipes, or suggestions of things to do. That’s because generative AI is really just another reason to add connectivity and sensing to everyday objects, either to provide personalized training data or act as a conduit to such services.

On the enterprise side, there’s the obvious utility of using generative AI to help business people implement digital solutions without coding. One example is how Software AG has combined its webMethods cloud-to-cloud integration platform with a generative AI model to help employees figure out how to link data and various digital services. Eventually, as we connect more things in buildings, manufacturing lines, commercial kitchens, etc., using plain written language to tell our connected devices how to work with our connected business software will help managers become more efficient and capable.

And in industrial environments, the promise of ChatGPT comes with compelling use cases and caveats. Several people have championed using generative AI for things like predictive maintenance. Generative AI models work by training on massive quantities of data and then generating the most likely next element. So in large language models, the generative AI model is training on huge swaths of text and generating what the model thinks is the most likely next word or phrase.

Presumably, with enough machine data, a model could decide what’s supposed to be next and send an alert if the expected result isn’t right. But frankly this feels like overkill, since traditional anomaly detection works fine for predictive maintenance and is much less cost-intensive. Where generative AI might get interesting is by taking process data and suggesting alternative workflows, or by using written language to describe workflows and having an AI code it for someone.

But there are caveats. These models are only as good as their training data and in some cases can generate wrong answers, but can be written so well that it’s hard to determine if they’re wrong.

“If you ask the technology to provide possible answers to technical questions, our experience is that, without proper context setting and filters, 80% of the answers are not accurate — potentially even harmful,” said Erik Udstuen, CEO of TwinThread via email. “But with proper context setting and filters/guardrails you can get very high accuracy.”

TwinThread uses a variety of different AI models to provide customers with services based on digital twins of their physical operations, including using generative AI to provide vast quantities of information to frontline workers and managers when they need it. Think of asking the AI why a specific batch of chemicals is too acidic or some other question related to a specialized process.

Udstuen believes the challenges around accuracy are a short-term issue that vendors will figure out, and that a bigger challenge with generative AI is related to intellectual property and the perceptions around how generative models are trained. “The last thing that anyone wants is to provide proprietary context or information that comes to be part of the public domain,” he told me.

He added that some customers are so worried about IP leaking they ask to turn any sort of ability to ask a chatbot about their specific processes off. While it’s not the case that their questions or proprietary data is shared, it’s such a big perceived risk for customers they don’t want to risk it.

Given the IP fights around generative AI, this last concern “feels” like it would be a problem. But in reality it is relatively simple to set limits on where training data actually comes from — or even if a model built on proprietary data gets deployed outside of the intended plant or business.

Time and education about how generative AI models are created and how they work will address some of the IP concerns. And since we’re only a few months into this hype cycle, I have faith that we’re going to see generative AI become as important as computer vision and NLP, and as accepted.

We’re also going to see some interesting new use cases for the IoT, so feel free to share what y’all are doing and thinking.