Tovi Grossman is a Senior Principal Research Scientist for Autodesk Research. He recently shared some good news with me. The paper, "Duet: Exploring Joint Interactions on a Smart Phone and a Smart Watch" won a Best Paper Award at the ACM CHI Conference (Top 1% of all submissions), and the presentation at the conference also won the Best Talk Award. Check it out.

DESIGNING A DUET OF INTERACTION BETWEEN YOUR SMART PHONE AND SMART WATCH

There is no doubt that in future we will carry not just one but many smart devices with us. To realize this vision requires more than a critical mass of smart device innovation but more importantly, at a higher level, a new design paradigm that makes these devices not just individually smart, but also collectively intelligent so that they will work together as a unified platform and create new interaction experience for the users.

A team of research scientists at Autodesk Research has been exploring new ways of designing interaction that spans multiple smart devices. In collaboration with research intern Xiang 'Anthony' Chen and University of Toronto Professor Daniel Wigdor, research scientists Tovi Grossman and George Fitzmaurice designed and built the Duet system — a joint interactive platform on a smart phone and a smart watch (a). Their research paper was recently published at the ACM CHI Conference on Human Factors in Computing Systems, the premier international conference of Human-Computer Interaction. The paper received a Best Paper Award, which is awarded to the top 1% of all submissions.

Duet is more than simply binding two smart devices; it enables new ways of using these two devices as a unified platform that go well beyond their capabilities as individuals. For example, the watch can be used as a tool palette when annotating text on the phone (b). Before a meeting starts, a simultaneous pinch-to-close swipe gesture on both devices mutes their notifications (c). Designers can also create new ways of touch interaction: the watch's orientation indicates which hand part causes a touch, thus enabling a seamless transition between touch modes: for example, writing with the pad of the finger (d), scrolling with side of the finger (e), and text selection with the knuckle (f).

These are just a few examples of what Duet is capable of. The central idea behind all these multi-device interaction techniques is how designers should consider the interactional relationship between the smart devices when designing their applications. In particular, the idea is to distinguish whether a given device is interacted with in the foreground or background of the user's attention.

Consider current commercial smart watches. Almost every smart watch is used as a viewport or remote control (foreground) when the phone is less available (background). Duet demonstrates alternate design possibilities by developing the watch's interaction potential even when the phone is in the foreground. Thinking in this way, a designer can transform the watch into an active element that enhances a wide range of phone-based interactive tasks.

Musical duets allow individual artists to combine their talents and instruments to create new forms of artistic expression. The Duet system similarly shows the possibility of designing interactions across multiple smart devices, so that their individual capabilities are leveraged to collaboratively create new interaction experiences.

Thanks Tovi.

Symbiotic pairing is alive in the lab.