PROBLEM
Customers are limited in using Alexa devices because they are tethered to one spot, preventing them from fully utilizing features like video calling and cooking assistance, which often require moving around the room, multiple people, or multitasking to be successful.
OVERVIEW
As the design leader for Smart Motion and Dynamic UI for Echo Show 10, I effectively led a team of UX designers, visual designers, motion designers, design technologists, researchers, and content designers to success.
Project length: 18 months
Amazon.com Product Rating: 4.5 ⭐ 49,720 reviews
ARCHITECTURE
I needed to develop a new interaction architecture for this new type of device. I had to consider how the Smart Motion would respond when orienting to individual people, multiple people, as well as safety events such as a collision or someone picking the device up.
IN-HOME PLACEMENT RESEARCH
DIFFERENT LOCATION
Most households prefer to place this new device type in a location different from their existing Echos. A frequent placement was on a kitchen island or peninsula to allow screen usage from multiple spaces, such as the kitchen and the family room.
PRIMARY USES
For most customers, we found their primary use cases were video calling and being a cooking companion in the kitchen. This was not surprising, given the nature of the moving device and its preferred placement in the kitchen.
WHERE
Power outlets are important and if a kitchen island had no power then the device is paced typically near a power outlet causing a wall(s) to be behind or to the side of the device.
AESTHETICS
Home layout, room size, and aesthetics were also important. Not having the power cord trailing over surfaces was a common ideal.
NEW WAY TO DESIGN
When designing the Smart Motion interactions, I had to think outside the box. Traditional design methods such as wireframing, on-screen prototypes, and presentation decks were insufficient to convey human-robot interactivity. So, I collaborated with software and design engineers to create VR prototypes of the device in various household settings. In the video below, you can see the device tracking an individual person and ignoring the second as they walk past, for example.
NEW WAY TO TEST
Using actual Amazon employees, I was able to test my designs and assumptions in real-time within the VR environment. This was live interactions with the virtual device, not preprogrammed tasks or wizard of oz.
TESTING HIGHLIGHTS
FOLLOW SPEED
I assumed that people would like a responsive screen with no follow-lag as they move around. However, testing proved otherwise. Testers preferred the screen to move slightly behind them. "It feels creepy moving perfectly with me, like I am being constantly watched." I found that about 0.5s lag was enough for the movement to get outside peripheral vision, and this re-tested very well. "Much better now; I move and don't see it following me, but when I look over, the screen is there waiting."
ROTATE ON "ALEXA"
Testers loved the way the device screen rotated to face them when they asked Alexa something, just like a person rotates their head when their name is called. "It behaves just like a person's head, very humanistic."
COMMUNAL TO PERSONAL
The way the screen faced the person asking Alexa a question made Alexa feel more personal and less communal, unlike prior Echo devices. "I love how it shows me my calendar, not the whole room."
DISTANCE IS IMPORTANT TOO
Now that customers can move freely with Smart Motion, the next step was to consider Dynamic UI to allow the screen to be useful at different distances. It is especially important for the information feed that makes up the ambient home screen and in rich experiences like cooking assistant.
TAKE ONE, UI IS RUNNING AWAY…
Partnering with a motion designer to bring the designs to life. This first version of Dynamic UI had potential however, it felt like the UI was running away from you as you approached. This was not the desired effect.
TAKE TWO, SIMPLIFY AND LAYER IN…
I have set up three different distance triggers to change the UI density, similar to responsive web breakpoints. When the user is 12 feet+ away from the screen, the UI will have a very low density, making it easily glanceable from across the room. When the user is 8 feet away, additional metadata is added to the UI, and at a distance of 3 feet, additional touch targets are displayed for more interactive touch or voice input.
KITCHEN EXTRAS
I proposed a line of robotic devices to enhance the cooking companion software. For example, below is the Robotic Pot Stirrer. Advanced AI and CV can monitor the pot for boil-overs, fires, etc., freeing the customer to do additional simultaneous cooking tasks!
"Alexa, start stirring for 5 minutes"
The extra starts rotating back and forth
"Alexa, watch the pot for me"
Echo Show rotates and watches the pot for any issues
PERSONALITY
With a device that can move and Alexa already has a voice personality, I pushed hard for smart motion to extend Alexa's personality with minimal but delightful movements. Working with a motion designer, here is a video example.
OUTCOME
The device has been very well-received by customers, with over 49,000 reviews and an average rating of 4.5 stars. This new Echo device represents the first step towards untethering the customer, as proposed in the Invisible Computer project, opened up new possibilities for future developments. The use of VR prototyping was a game-changer, saving Amazon millions of dollars by eliminating the need to produce early physical hardware prototypes. I am proud to have been part of this revolutionary project and the first steps to a bigger future for Alexa.
DAILY TEAM
I was the product design leader setting the vision for Smart Motion and Dynamic UI.
Three UX designers worked with me to go deep on all screens, redlines, components and shipping.
Three product managers collaborating with across verticals.
Hundreds of software engineers.
AMBIENT INTELLIGENCE POWERED AI
CONVERSATIONAL HEALTHCARE AI