Jaime Ruiz, Yang Li, Edward Lank. "User-Defined Motion Gestures for Mobile Interaction". UIST '11 Proceedings of the 2011 annual conference on Human factors in computing systems. ACM New York, NY, USA ©2011.
Author Bios
Jaime Ruiz is a fifth-year doctoral student in the HCI Lab at the University of Waterloo. His main research interests is further understanding users to augment the level of human computer interaction. For this paper he was a research intern at Google Research.
Yang Li is a senior research scientist at Google Research. Before joining Google, he was a research associate in the Computer Science and Engineering at the University of Washington, where he helped found the DUB. He received his Ph.D. in Computer Science from the Chinese Academy of Sciences, and conducted postdoctoral research in EECS at the University of California at Berkeley.
Edward Lank is an assistant professor at the University of Waterloo. Before he joined Waterloo, he was an assistant professor at San Francisco State University and conducted postdoctoral research at Xerox. He received his Ph.D. in Computer Science from Queen's University.
- Hypothesis - Conducting an end-user design test will result in best practices for designing gestural interfaces.
- Method/Content - This paper focused on a user study involving motion gestures using mobile phones. Twenty participants were involved. Because learning about a new device would have cause people to be less receptive to new concepts, they required that the participants used a smartphone as their primary mobile device. Nineteen tasks were given to the participants. For each task, the participant was asked to come up with an easy and comfortable gesture to conduct the desired action. For example, to come up with a gesture for answering a call. Because they did not not want users to focus needlessly on recognizer issues or the current level of technology, they asked participants to treat the device as a "magic brick", capable of automatically understanding and recognizing anything they might throw at it (figuratively, of course). The participants were asked to conduct the study while thinking aloud. In other words, they wanted the users to explain what they were doing, what they may be emulating, and why for each gesture. Once the participant had decided on a unique gesture for each of the 19 tasks, they were asked to perform each one five times. While they conducted each gesture the phone recorded data from the accelerometer and its other motion-detecting hardware and sent to another computer. Once they were done, the users were asked to rate their own gestures on a Likert scale based on its ease of use and whether or not they would use it often.
- Results -The results were that the closer a gesture was to emulating a real world object, the more of a consensus among participants was reached. For example, when answering a call, easily the most common gesture was to simply raise their phone to your ear. This makes sense, as for a normal call, this would be the first thing you do (usually after you answer it). An example of emulation that does not happen regularly is the act of hanging up the phone. The gesture users found the most consensus on this one was to emulate an "old-fashioned" phone; that is, they turned the phone so that its screen was parallel to the ground. Another thought that many users had is that some of the actions should have the same gesture; for example, going to the next photo, contact, or search results, it should all be a flick to the right. Users generally agreed that although in different contexts, the end result was basically the same, so the same gesture should be used. For actions that did the opposite of another, users generally performed the same gesture, only in the opposite direction. For example, going to the previous item in a list required a flick to the left. Another example is zooming in and out on a map; zooming in was to bring it closer to your face, zooming pushing it farther away.
Another curious result is that, in contrast to surface gestures, users generally wanted to move the window as opposed to the object. This means that when dealing with a map or image, the users panned left by moving the phone to the left, whereas with a surface gesture it is moving to the right. This was explained by the fact that when touching the screen itself, the user was in effect moving the object on the screen. However, when moving the phone, the user was moving the screen around the object, and expected it to react as such.
This paper seemed like it is a bit late in coming, although I enjoyed it. It seems that we have had the technology for a long time, and it took us this long to start looking for motion gestures? I remember back when the Gameboy Color was popular; Kirby Tilt 'n' Tumble was one of my favorite games. That used an accelerometer, which is one of the main tools used in motion gestures. It just surprised me that it took us this long to make the transition. However, I think that this technology/concept has a large range of practical applications. Things like conducting presentations or interacting with colleagues in a design room would be great places to use a device for this. However, this could also raise the problem of using a phone in the car, which we already have enough issues with. As seen in Why We Make Mistakes, making the phone accessible without looking at it would not solve the problem. Humans simply cannot multitask that well.
