There’s already an optional gesture for answering a phone call on Apple Watch, but Apple wants you to control everything, anywhere, at once
AppleInsider may earn an affiliate commission for purchases made through links on our site.
In the future, you might be able to hail your HomePod to turn up the volume, swipe to close your smart curtains with HomeKit, or just raise your hand to pause everything.
There’s a lot to be said for being deep in the Apple ecosystem, but it’s very easy to be surrounded by Macs, iPhones, iPads, Apple TV 4K, and more. It’s not likely you’ll be playing audio or video on all devices at the same time, but pretty much every device is listening for you to say “Hey Siri.”
A recently unveiled patent filing shows Apple thinking about its entire ecosystem and how to make it easier for you to use. ‘Multi-Device Gesture Control’ describes a situation where you can indeed wave your arm to silence everything.
But it really describes doing this by waving an arm with an Apple Watch on it.
Detail from the patent application showing the interconnectedness of all things
“The electronic device can efficiently provide gesture control for multiple other devices by mapping a finite set of user gestures to a specific set of gesture controls for each of the multiple other devices,” the patent filing says. “In this way, a single gesture can be detected to potentially control different functions of different devices.”
It sounds like it might take a while to set this up, but you might have that when you raise your hand, every device reacts differently. Walk into a room, raise your hand and the smart blinds roll down, the smart lights switch to a romantic scene and Barry White starts playing on the HomePod.
You made one gesture, but separately the blinds can be set to interpret it as an instruction to open or close. The lights switch to a certain position.
And maybe a device performs a shortcut to select a particular Apple Music playlist.
Apple’s goal is not just “gesture control”, but specifically “a finite set of gestures detectable on a first electronic device” that can activate whatever a user wants.
That “first electronic device” is key. It’s not that every Apple device needs to have a camera to recognize gestures, it’s that at least one device needs to detect them – and pass them along correctly.
As always, Apple’s patent filings are about how something can be achieved, rather than specifically what purpose it can be used next.
So most of this patent filing is about how that “first device” — “like a smartwatch or other wearable device, a smartphone or something like that” — can detect a gesture.
Presumably you could hold your hand upside down and pretend to be Spider-Man throwing a web at these devices. Presumably.
The answer is by “using an ultra-wide band (UWB) sensor [or] a Bluetooth Low Energy (BLE) sensor.” One of your devices may be watching you, so the system may use “an image sensor paired to…another device.”
There’s also an issue that the user might want more fine control, so the Apple Watch or other wearable device needs to pull out some extra options. But the main thrust of the patent application is motion sensing, which Apple says can also be done through “inertial measurement unit (IMU) sensors… an accelerometer, a gyroscope, and a magnetometer.”
Apple is also betting on “additional sensors such as electromyography (EMG) sensors)” to make this patent application as broad as possible.
The patent application is attributed to four inventors, including Ali Moin, many of whom previously worked on machine learning technology for classifying biosignals.