PROBLEM
Alexa AI is responsive but not contextual or pre-emptive. Customers always need to say "Alexa..." to get Alexa to do something for them. This approach is not human-like and feels unnatural, leading to one-way interactions and forgettable, unintuitive voice commands.
OVERVIEW
I spearheaded a groundbreaking project at Amazon focused on redefining Alexa interaction and proactivity. The project involved decentralizing sensors (inputs and outputs) across multiple environments and improving cost efficiency. I assembled and guided a skilled team, rigorously proving the concept's viability over six months to set a new standard in ambient computing without the need for any on-person devices such as headsets or watches.
SENSORS
I proposed concealing the sensors in recessed can lights on the ceiling. These lights are widely used and can be easily installed in existing homes. They have enough space to accommodate various types of sensors, such as microphones, speakers, and radar sensors. Positioned up high, they are unobtrusive and unobstructed by people or furniture, making them the ideal location.
PERFECT SENORS FOR THE SPACE
These wireframes show how customers can use their smartphone's camera to scan their room, find upgrades points like ceiling lights or TVs, and measure the room's size. Our AI technology then selects the perfect sensors types and amount for that specific space.
BUY THE UPGRADES
Customers can purchase upgrades for their chosen rooms. Prime delivers the upgrades the next day in separate boxes that are easy to open and cause no frustration, one box per room.
INSTALLATION
Confidently upgrade your home starting in the living room by connecting the Alexa TV box. The home upgrade experience begins instantly, guiding you through the process and showing you which sensor to install first. With the TV as your guide, you'll feel like your home is truly upgrading and reacting to each step, rather than just following another app on your phone. Start by unscrewing the recommended light bulb.
Screw in the new sensor bulb, and the home reacts instantly. Repeat for each chosen room, with changing light colors to confirm room completion and on-screen tutorial.
THE BRAINS
By utilizing existing home power lines and dedicated wireless communications, the Hub is capable of provisioning account details, a setup free connection to Alexa sensors all while ensuring that they are not exposed to your home WiFi network. This makes for a secure and reliable setup, as your Amazon account is seamlessly integrated with the Hub. On device Alexa AI processes quickly and securely only sending data to the cloud as a last resort.
By utilizing existing home power lines and dedicated wireless communications, the Hub is capable of provisioning account details, a setup free connection to Alexa sensors all while ensuring that they are not exposed to your home WiFi network. This makes for a secure and reliable setup, as your Amazon account is seamlessly integrated with the Hub. On device Alexa AI processes quickly and securely only sending data to the cloud as a last resort.
EXAMPLE UX INTERACTION
Husband and wife on video call in the living room.
The wife walks away from the TV, and as she does, she is still talking to her family on the call. Alexa takes this natural cue that she wants to continue in the conversation
As the wife moves through the home, still talking, the edge sensors connect to her based on which are nearest to allow her to still hear and speak on the call.
In the kitchen making a cup of tea and still chatting and connected
She looks down at the screen in the kitchen. Her gaze causes the video portion of the call to come up. She can see her husband in the living room and her family across the country.
All this has happened proactively without ever having to say the "Alexa…" wake word and command. The call continued uninterrupted and went from a single to a split call.
OUTCOME
The project highlighted the importance of the design team devoting resources to future-oriented design planning. This resulted in myself leading the design of Amazon’s first robotic device, the Echo Show 10, which enabled customers to move freely around a room while engaging in activities like video calling or cook along at the same time. A step towards an untethered future presented here.
Through this project, I was able to demonstrate to Amazon leadership the importance of anticipating the future of homes and cities with distributed edge sensors. In order to create powerful mixed and extended reality scenarios without the need for glasses or headsets, it is essential to have hyper-contextual person location and identification capabilities with appropriate inputs and outputs within space.
To top it off the project reinforced the need for virtual reality prototyping in all new projects, in order to create hyper-contextual experiences that account for specific environments, humanistic interactions, and emotional states. This level of nuance cannot be conveyed through wireframes, motion graphics, or presentations alone, making virtual reality an essential tool for our team’s design process.
DAILY TEAM
I was the product design leader setting the vision for Ambient Intelligence.
Two UX prototypers.
Director of UX and VP of Design.
SMART MOTION AI & DYNAMIC UI
CONVERSATIONAL HEALTHCARE AI