Gemini Automation
Letting people hand a task to the OS and still feel in control
Gemini Automation lets you ask for something like "reorder my usual coffee" and watch the agent do it across apps. I defined the Samsung side of the interaction framework: how a delegated task launches, how you watch it, how it asks when it's unsure, and how you take over for the final step. I also designed how it adapts to the Flip cover screen and Fold multitasking.
Delegation is a trust problem
Asking an agent to act on your behalf is different from asking it a question. People need to know what it's doing, when it's stuck, and that the last word is still theirs. The model's capability was not the hard part. The hard part was making delegation feel safe.
Four moments of a delegated task
Launch without losing your place
The task opens in a floating window, so the app you were in stays put. I worked with framework engineering on a launch-as-floating-window behavior so automation never hijacks the full screen.
Watch it work
Live View shows the agent moving through the app in real time. You see the steps, not just a spinner.
Ask when unsure
When the request is ambiguous, like two coffee shops with the same name, Live View pauses and asks. The agent never silently picks for you.
Hand back the final step
For confirming a payment or sending a message, the agent stops and hands control to you. Irreversible actions stay human.
Designing for every form factor
Flip cover screen
A glanceable view of task progress on the cover screen, so you can delegate and keep the phone closed.
Fold multitasking
A floating "view progress" surface lets the task run beside your other apps, matching how people already use the large screen.
North Star: automation that offers itself
Beyond launch, I proposed where automation should go next: stop waiting to be asked. These concepts were my own.
- Ambient widget that suggests a task you run at this time, in this place.
- Rerun drawer that brings back tasks you've delegated before, one tap away.
- Home screen shortcuts that turn a repeated request into an icon.
This connects Gemini Automation to Now Nudge: the OS notices the intent, then offers to carry it out.
Working across two companies
I represented Samsung UX in design alignment with Google, covering how Gemini surfaces render inside One UI, visual consistency, and where Samsung-specific behavior belongs. The job was to keep one coherent experience while two design systems met in the same screen.
Lesson learned
People will hand over a task when they can see it, stop it, and finish it themselves.