Imagine using a single app to control everything in your smart home. I don’t mean the apps of today which show your devices in groups or rooms. I’m talking about a context-aware interface that intelligently surfaces a specific device, and its controls when you point your phone at it. That’s the future of the smart home and one iOS developer is using ultrawideband (UWB) technology to demonstrate it.
I saw a demonstration of this future in Bastian Andelefski’s prototype application called Point (as in point and click). Andelefski creates iOS applications and he’s worked on a smart remote like this since 2015. Unfortunately, indoor device location technology wasn’t up to snuff back then. It’s only been recently that wireless radio tech such as UWB and wireless sensing solutions have provided the information needs for a contextually aware smart home experience.
Essentially, using the UWB chip in an iPhone along with Apple’s ARKit framework for augmented reality, Andelefski’s app shows where each device is. Technically, it only shows the area where he places certain devices. That’s because most connected devices in the home don’t have a UWB radio. If they did, his app could identify the specific device and not just the expected location.
Andelefski’s software then acts like an AirTag finder of sorts, automatically identifying where a smart device is as he points his iPhone around a room. And once the app “sees” a device, the interface changes to match the capabilities of that device.
If it sees connected colored lights, for example, the app shows controls relevant to them: on, off, a color palette, etc… Likewise, if you point your phone at a thermostat, you’ll see an interface to change the temperature or turn the heat on.
To truly appreciate this experience, take a look at this video where Andelefski demonstrates his prototype.
The first thing I noticed is how much easier this type of smart home experience is. You point your phone at a device and see only what you need to see.
Why clutter up a phone display with bunches of connected devices when you can simply show the specific one you want? And why have that extra tap to choose a device to view its specific capabilities and options? Adding devices to the Point prototype app is easy too, which he shows off in the video.
So, why don’t we have this smart home interface today? It comes down to timing.
We’re just getting started with the mainstream adoption of the technologies needed to make this all work. UWB is in all new iPhones and some Android handsets, but it’s still regarded as a feature for the high-end of the market. And although I’m sure there’s one or two out there, I’m not aware of any bulbs, cameras, locks, or other connected devices that have a UWB radio in them. That’s why Andelefski is limited to creating small areas around his home for Point; his app controls devices associated with those areas.
Another limitation is from the platform providers as well. As it stands now, Andelefski says he can’t use the UWB chip in Apple’s smart speakers because Apple doesn’t yet allow third parties to do so. It’s possible that will change in the future, although even then, Apple will surely maintain rigid control over what developers can and can’t do with an in-device UWB radio.
And then there’s the added cost if hardware makers even wanted to integrate a UWB chip inside their devices. Apple’s AirTag is a prime example of this at $30 per tag; the most expensive component is surely the UWB and Bluetooth radios inside.
While UWB is the solution Andelefski chose, it doesn’t necessarily have to be the only future option. A less expensive approach would use Bluetooth for device location. In theory, it would provide a similar experience as the range of modern Bluetooth radios can easily handle device location from across a room. Precision data from Bluetooth may not be quite as good, but close enough works in this case.
No matter how this experience could be architected, I want it. I’m tired of scrolling through rooms, groups, and lists of devices to control my smart home from my phone and watch. It’s why I turn to voice control in many cases, even when it’s not an ideal situation or when my smart speaker doesn’t quite understand my commands. This approach could even enable scenes or routines when you walk into a room; like the iPhone, my Apple Watch has a UWB chip inside that could be used to determine my location with respect to my connected devices.
It’s time to elevate smart home control with a more intuitive and contextual experience. Something like the Point prototype combined with next-generation wireless technologies could deliver it.
This has been done before with a different technology. There was a startup, SevenHugs, that back in 2018 had a universal controller where the display changed as you pointed it at different devices in the room. It sold for a couple of years for about $300, then they tried a cheaper model, then eventually it just died. There were complaints about various power issues with it as well.
(BTW, From an engineering standpoint if you want three-dimensional space precise location, UWB is way better than Bluetooth, which is why AirTags use it. Bluetooth gives you accuracy within 5 m; UWB brings that accuracy down to within 50 cm, a really significant improvement. UWB is also less subject to interference in a typical home.)
Of course this kind of device is useless for people like me: my iPad is in a caddy on my wheelchair and I can’t hold a phone to point it. But I recognize that’s an edge case and there are lots of people who would like a hold and point remote of this type if there was a good one.
It will be interesting to see what becomes available in the future in this device class particularly, as you note, if UWB becomes more widely adopted.
There are some other potential technology options as well, though. In the last two years, Apple has filed at least six patents on different kinds of microlocation for home automation, using a lot of different information that your phone already has. We will just have to see if any of them end up having practical use.
If anyone is interested, here’s a good technical article on what’s currently practically available in position location technologies. It’s aimed at those looking for industrial solutions, but the same comparisons between, for example, Bluetooth and UWB, apply to home automation.
https://www.inpixon.com/blog/chirp-uwb-ble-location-tracking-techniques
Good stuff as always, JD! In terms of UWB accuracy, I think it’s currently far better than 50cm. When I use the FindMy app with AirTags, for example, it resolves down to just a few inches. And I agree, UWB is a far better current solution than Bluetooth. I only mentioned the latter because the Bluetooth 5.1 standard added support for signal angle of arrival (AoA) and angle of departure (AoD): The promise was “centimeter level of accuracy”, although I haven’t yet seen an implementation providing that. Also, I wouldn’t be surprised to see UWB included in future iPads, so there’s hope yet. 🙂 Cheers!
There’s a lot of interesting work being done in both microlocation (finding something) and micropresence (knowing exactly who is in a room and then triggering events off of that).
Have you seen the RoomMe system from Intellithings? These are sensors that detect who is in a room based on their cell phones. (It would be better if it was smart watches or other wearables, but they aren’t there yet.)
It’s been available for a couple of years and the most unusual thing about it is its logic rules are based on giving some users priority over others. So if dad is in the room, his temperature settings for the thermostat take priority over kid one’s. Or whoever you make the “master“ of the room.
The technology is Bluetooth, but they have a patent on their method of creating zones which limits it to one room (an issue with many Bluetooth-based micro presence systems since Bluetooth will detect through walls).
Cost is about $200 dollars per sensor, but you need a minimum of two sensors even for just one room because it Has to detect coming and going.
The company suggests a lot of use cases for reporting when someone has NOT entered a space as expected. So child is late getting home from school or senior adult has not come into the kitchen at all today.
RoomMe has integrations available for IFTTT, Homebridge, Crestron and Control4, so there are quite a few options.
It’s clever, the technology looks good, I like that they have a patent, and reviews are quite good as long as it fits your budget. And of course it’s hands-free as long as you have your phone with you. And you can buy it today at Amazon.
Whether it’s this type of approach or Point or one of Apple‘s new patents or heartbeat detection or a mix of all of the above, I do think that within the next two or three years we are going to see context-sensitive micropresence.
There have been some IBeacon based micropresence systems available for several years and I use some in my own home, but they just never took off, I think because no one‘s really solved the “rooms are defined by walls“ issue. But I am excited to see what we’ll get in the future as far as context-aware home automation!
Yup, I took the RoomMe devices for a spin in 2019. I saw potential but not consistency. https://staceyoniot.com/roomme-brings-personalized-presence-controls-to-the-smart-home-but-consistency-is-an-issue/
There you are: ahead of the game! 🙂
I wonder if they’ve improved any in the two years since you tried them? For me, the barrier to trying RoomMe has always been cost.
Check out the app Homerise. It has this functionality, PLUS you can draw a floor plan of your home and place your devices inside. Super cool!! I think it’s way more intuitive than just seeing a list of your devices or having to page through different room lists. We should all support the developer so they can keep supporting and add more features! https://homerise.app/
I thought the whole point was to get away from having hand held remote controls for things? Alexa, tell google to ask siri to leave that UWB garbage outside.
I think choice is good in these cases. I participate in an online group for wheelchair users and one point that comes up frequently is that while most of us rely heavily on voice assistants, some people don’t have the ability to vocalize. Or it may just be late at night, you’ve got your headphones on, and you don’t want to wake up the rest of the house with voice commands.
I don’t have any issue with people getting excited about handheld controls as long as that’s not the only option. 😉
Reminds me of the old X-10 IR remote mini controller.