Project/Multitouch Interaction

From Drexel Smart House

Jump to: navigation, search

Moore's Law states that the number of transistors on an integrated circuit chip approximately doubles every two years; in other words, our computers significantly speed up every year. Personal computers are becoming more readily available, extending to even the much less fortunate through acts such as the `One Laptop Per Child Initiative'. New technologies to access computers have been developed, including voice recognition, light pens, touchscreens, and gesture recognition. Since the days of Windows 3.1 and the Apple II, graphical interfaces have developed into Jeff Han's `interface without an interface,' the ever-growing `zooming user interface,' the emulation of real world interactions in interfaces such as `BumpTop,' and developments made in the Web 2.0 blogosphere (e.g., folksonomies). However, the common personal computer still relies on the keyboard and mouse for input, and graphical interfaces remain based upon the `desktop metaphor' and utilize the `window, icon, menu, and pointing device (WIMP) paradigm'. The purpose of this project is to explore the combination of multitouch technology and a natural user interface. In particular, notions from many of these metaphors will be synthesized for optimal multipoint input.

This project will be addressing the way in which people interact with technology by improving existing and creating new methods of interacting with computers and other technologies. This project was started as part of the class requirements for Special Topics in Smart Design - Fall 2007.

Contents

Introduction

The commercialization of technologies such as the iPhone and Microsoft Surface demonstrate the possibilities of multitouch interaction. For instance, consider walking up to a wall-sized touchscreen and controlling everything with your fingers; multiple users can even share large workspaces. However, as hardware support is becoming readily available, human computer interfaces have lagged behind. Most multitouch software today still uses the windows, icons, menus, and pointers interface introduced in the early 1970s. Specifically, this project is looking at using frustrated total internal reflection techniques to detect interaction on a rear projected polycarbonate surface. Additionally, we hope to develop new human computer interaction methods based upon psychological studies of human intuition. The primary objective is developing technology that the uninitiated of all ages can use intuitively, that behaves and responds as one would expect in real life situations.

Background

Previous Work

Jefferson Y. Han is a research scientist for New York University's (NYU) Courant Institute of Mathematical Sciences, and one of the main developers of an "interface-free" touch-driven computer screen. The screen is based on "multi-touch sensing," which is similar to existing touch-screen interfaces but able to recognize multiple points of contact.

He presented his multi-touch sensing work in February 2006 at the TED (Technology Entertainment Design) Conference in Monterey, California. TED released the video online 6 months later and it spread quickly on YouTube.

Han has founded a company called Perceptive Pixel to develop his touch screen technology further, and he has already shipped touch screens to parts of the military.[1]

BumpTop is a prototype graphical user interface, designed to enhance traditional computer desktop functionality by more closely supporting the normal behavior of a real world desk. It is aimed at stylus interaction, making it more suitable for tablet PCs and palmtops. It was created at the University of Toronto as Anand Agarawala's Masters Thesis.

In BumpTop, documents are described by three-dimensional boxes lying on a virtual desk. The user can position the boxes on the desk using the stylus or mouse. Extensive use of physics effects like bumping and tossing is applied to documents when they interact, for a more realistic experience. Boxes can be stacked with well-defined stylus gestures. Multiple selection is performed by means of a LassoMenu, which fluidly combines into a single stroke the act of lasso selection and action invocation via pie menus.

Design

There are three main components to this design: what the user experiences (the Human Computer Interface), and the hardware and software that enable it.

HCI

Applied psychological techniques will be used to develop an intuitive HCI conforming with Jeff Han's "interface-free" philosophy. This will dramatically decrease the learning curve for users by basing interactions upon "offline" experiences. This is where the BumpTop concept excels by emulating a real-world desktop, complete with a physics rendering engine.

Hardware

We'll be using frustrated total internal reflection techniques to detect interaction on a rear projected polycarbonate surface. You can read a brief introduction to this by Jeff Han.

Software

There are four main pieces to the soft components of the multitouch; they are the Multitouch framework, the FTIR-MPX drivers, the multipoint X Server, and the Desktop environment. Up until the later, these technologies are available and abundant on the internet. Most of the innovation will come in the development of an effective deskop environement. However, an understanding of all the components is necessary for a scalable and well designed desktop environement; thus, the following is a summary of the overall system with details about each soft component.

Multitouch Framework

The multi-touch framework provides services for capturing and interpretting data from cameras. It should have the ability to interface with webcams, CCD cameras, and any other video capture devices. This data is then interpretted into "blobs." The blobs are the digital representation of finger touches the video capture device has picked up, and serve as the main source of functional data. Thus, the multi-touch framework must then also be ablo to track appearing blobs as well as interpret the movement of the blob as a gesture (ie finger move down, two fingers forming a line). These commands inferred by the framework form the main source of commands useful to the end-user to interact with the desktop environment. Finally, after capturing and interpretting these blobs, the information about the blobs are packaged to be sent out to the multipoint X Server.

Some current multi-touch frameworks are as follows: blah blah blah

The Multipoint X Server

Timeline

Budget

College of Engineering Grant
ASME Action Diversity Grant

Personnel

Principal Investigators
Interior Design: Dr. Eugenia Ellis
Digital Media: Dr. Paul Diefenbach
Electrical & Computer Engineer: Dr. Youngmoo Kim

Design Course Team
Project Leader: Cody Ray
Technical Expertise: Peter Thai
Design Expertise: Jenna Banyas

Senior Design Team
Digital Media
Electrical & Computer Engineering

Downloadable Materials


Bibliography

Multitouch Bibliography

References

External Links

Retrieved from "http://www.drexelsmarthouse.com/wiki/Project/Multitouch_Interaction"
Personal tools