Smart Motion AI & Dynamic UI

01 PROBLEM

Alexa devices were tethered to one spot, and that broke the exact moments people buy a smart display for. Video calls, cooking along with a recipe, multitasking with family in the room: all of them involve moving around, and the screen stayed put while you did.

As design lead for Smart Motion and Dynamic UI on Echo Show 10, I directed a 12-person team of UX designers, visual designers, motion designers, design technologists, researchers, and content designers across the full 18 months, from the first interaction architecture to the device on shelves. It shipped to a 4.5-star rating across 49,720 Amazon reviews.

02 APPROACH

ARCHITECTURE

A screen that physically moves had no interaction precedent at Amazon, so I developed the interaction architecture from scratch. I defined how Smart Motion orients to a single person, how it arbitrates when multiple people are in the room, and how it responds to safety events like a collision or someone picking the device up mid-rotation. This architecture became the shared contract the design, CV, and motor-control teams all built against.

In-Home Placement Research

DIFFERENT LOCATION

I worked with research on in-home studies before locking the design. Most households wanted this device somewhere different from their existing Echos, most often a kitchen island or peninsula, where one screen serves both the kitchen and the family room. That single finding shaped the rotation range and viewing-angle decisions that followed.

PRIMARY USES

The research confirmed the two primary uses I designed around: video calling and cooking companionship in the kitchen. That fit the nature of a moving device and its preferred kitchen placement, and it told me where the motion experience had to be flawless first.

WHERE

Power dictated placement more than preference did. When a kitchen island had no outlet, the device ended up near a wall instead, with walls behind or beside it. I fed that constraint directly into how far the screen needed to rotate and where motion should stop.

AESTHETICS

Home layout, room size, and aesthetics mattered too. Almost everyone wanted the power cord hidden rather than trailing across a counter, a small detail that kept coming up and that I made sure the industrial design conversation heard.

Designing & Testing the Motion

Wireframes, on-screen prototypes, and decks all failed at the same thing: none of them could convey what it feels like when a device physically turns to look at you. So I collaborated with software and design engineers to build live VR prototypes of the device inside simulated household settings, and we iterated on motion behavior there instead of on paper. In the video below, the device tracks one person and deliberately ignores a second walking past.

I recruited Amazon employees as test participants and put them in the VR environment with the virtual device, live. These were real-time interactions, not preprogrammed tasks or Wizard of Oz sessions, which meant every flinch, hesitation, and delighted reaction was a genuine response to the motion design.

FOLLOW SPEED

I assumed people would want a perfectly responsive screen with zero follow-lag. Testing proved me wrong. Instant tracking read as surveillance: "It feels creepy moving perfectly with me, like I am being constantly watched." I experimented with the timing and found that roughly half a second of lag pushed the movement outside peripheral vision, and when I retested, the reaction flipped entirely: "Much better now; I move and don't see it following me, but when I look over, the screen is there waiting." That half-second became a core spec of the shipped motion behavior.

ROTATE ON "ALEXA"

I designed the device to rotate toward you when you say "Alexa," the way a person turns their head when their name is called. Testers loved it without needing it explained: "It behaves just like a person's head, very humanistic."

COMMUNAL TO PERSONAL

A side effect I hadn't fully anticipated: facing the person who asked the question made Alexa feel personal rather than communal, a real break from prior Echo devices. "I love how it shows me my calendar, not the whole room."

Smart Motion freed people to move, which created the next design problem: a screen that follows you across the room is useless if its UI is only legible from three feet away. So I turned to Dynamic UI, making the screen useful at every distance, especially for the ambient home screen's information feed and rich experiences like the cooking assistant.

TAKE ONE, UI IS RUNNING AWAY…

I partnered with a motion designer to bring the first version to life. It had potential, but in motion the UI felt like it was running away from you as you approached, the exact opposite of the welcoming effect I was after. Watching it fail in video form was the fastest way to know it had to change.

TAKE TWO, SIMPLIFY AND LAYER IN…

For the second version, I flipped the model: instead of animating elements away, the UI layers information in as you get closer. I defined three distance breakpoints, working like responsive web breakpoints in physical space. Beyond 12 feet the UI runs at very low density, glanceable from across the room. At 8 feet, supporting metadata appears. Inside 3 feet, touch targets surface for direct interaction alongside voice.

Kitchen Extras & Personality

Beyond the shipping scope, I proposed a line of robotic kitchen accessories to extend the cooking companion. One example below: the Robotic Pot Stirrer, where AI and computer vision watch the pot for boil-overs and fires while the customer gets on with the rest of the meal.

Alexa already had a voice personality, and now the device could move. I pushed hard for Smart Motion to extend that personality physically, through minimal but delightful movements rather than gimmicks, and worked with a motion designer to define the vocabulary. Here is a video example.

03 OUTCOME

Echo Show 10 shipped and has been embraced by customers, holding a 4.5-star average across more than 49,000 Amazon reviews. It was the first step toward untethering the customer, the future I had proposed in the Ambient Intelligence Powered AI project, and it opened the door to what came next for Alexa. The VR prototyping approach I pioneered also outlived the project: it saved Amazon millions by replacing rounds of early physical hardware prototypes, and it proved that human-robot interaction can be designed and validated before a single motor is built.

DAILY TEAM

I set the vision and led product design for Smart Motion and Dynamic UI.

Three UX designers went deep with me on every screen, redline, and component through to shipping.

Three product managers collaborated with us across verticals.

Hundreds of software engineers brought it to production.