The Invisible User Interface

‘The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.’ That’s what computer scientist Mark Weiser wrote in his famous article The Computer for the 21st Century, originally published in 1991.

He predicted the emergence of ubiquitous computing — omnipresent information technology that fades into the background of the human environment. According to Weiser, this would allow people to ignore the technology and focus on new objectives. The user interface is a major stepping stone on the path toward computer invisibility.

'If you look at traditional computer interfaces, such as the keyboard and mouse, you see that people act as a cog in the process. Users have to learn to work with these interfaces because that is the only way to interact with the computer. The trend we see now is that people are looking more and more at the human aspect and how people communicate intuitively. What we are aiming for is to teach machines how to understand this human form of communication.'

(TIII) project. The objective of the TIII project is to close the gap between the digital and physical worlds, particularly with regard to the user interface. Until recently the keyboard was the only way to get access to the digital world, but an upsurge of new interfaces has started to emerge. The breakthrough of touchscreen display on the Apple iPhone and the popularity of motion sensing game controllers, such as the Kinect and the Wii Remote, show that there is a market for them. However, interface development has primarily been a game for the big players in the technology sector. The TIII platform takes the approach that the current situation in the electronics world makes it possible for relatively small companies to participate in interface development. The platform aims to promote this by means of support and cooperation.

 
Usability

There is a strong trend toward usability in industrial design and in human-computer interaction. That primarily revolves around efficiency, effectiveness, safety and ease of use. 'This means that people are regarded as part of the machine,' according to Saldien. “You focus on how people can perform their tasks as successfully and as efficiently as possible. This means that you look at speed, at how many mistakes they make, and how fast they can learn to click the right things in order to complete their tasks. All of that is part of usability. Ergonomics is also a major factor with products. For example, we look for the shape that best fits the hands of 90 percent of the population. People are generalized into a mathematical and psychological model with standard dimensions and reaction times. Then we try to build our systems so that everything goes as smoothly as possible with people as part of the system."

User experience

The TIII project shifts the focus from usability to the user experience, which involves looking at all aspects of the interaction between users and their products. How is the product perceived? How do users learn how to use it? And how does their use of the product change over time? Another factor that is examined is who the user is: where does the user live, what does the user do, and what does the user want? This is a shift from efficiency to experience, and it is an emerging trend in the interface world. Saldien: 'In the gaming industry they make interfaces with a focus on entertaining and surprising effects. Sometimes the interface is intentionally made more difficult to make the environment more challenging. If you constantly make the interface easier, simpler and more efficient, users may start to feel stupid. It’s like an operating system that tries to protect users to the point of annoyance by asking: Do you want to delete this file? Are you really sure?

'The shift toward user experience is also visible in areas outside the high-tech sector. Car makers such as BMW and fashion designers such as Armani or Dolce & Gabbana are catering to user experience much more than before. They focus on aspects such as fun, surprise, image, beauty and esthetics, because these aspects have a major effect on how people experience a product or a fashion line.'

Emotions

Jelle Saldien became interested in interfaces from his background in human-robot interaction. 'At the Free University of Brussels we developed the social robot Probo for autistic children. I have also done a lot of research on how to express emotions with a robot and how we can create social communication between people and machines. A lot of the interactions between people are non-verbal – we communicate with our tone of voice, our facial expressions, our body language and so on. That’s something you learn to understand emotionally as a person. Autistic children have difficulty with this.'

'With the social robot, we can express emotions in a controlled and gradual manner. For example, the robot can always laugh in the same way, or it can change its laugh bit by bit, very gradually. The robot expresses these emotions according to recognizable social scenarios that are explained to the children. This is a specific form of therapy, called social story telling. It is certainly not intended as a substitute for a human therapist – which I would like to emphasize, since that would not be possible — but the robot acts as an intermediary or a doll that can mimic social interactions for children.

'By studying this, we discovered that there is still a lot to be done in the area of human-machine interaction. How people communicate with each other is very different from how people interact with machines. This can be seen with e-mail messages, where misunderstandings can arise very quickly. There we miss the social interactions, such as eye contact and emotions, which help us communicate the right meaning. This is why people came up with emoticons, such as the smiley and the sad face. Even though keyboards were never intended to be used to express emotions, we humans quickly devise ways to convey emotional content in our messages.

'Now we are looking more at how we do this as people and how machines could pay more attention to human factors. Especially in the academic world, people are already looking at how we can let machines simulate emotions, and in the other direction, how machines can detect human emotions. For example, if my computer detects that I am angry, it can adapt the interface accordingly. People are also working on this in the car industry. It would be handy if a car could notice that the driver is drowsy and warn the driver. Or if the driver gets aggressive, the car could adapt by slowing its response to driver input.'

Limits to what is possible

Saldien thinks that if you are sufficiently creative, there aren’t any technological limits to what is possible with the interface. This is partly due to the impact of the Arduino platform on interfacing. 'Since this platform is so easily accessible, it allows artists, hackers and hobbyists to start tinkering and playing with technology,” he notes. “The software programs for Arduino are called sketches. You’re actually sketching with software code. You copy and paste bits of code and try things out. The focus is much more on interaction — what you can get the product to do — than on software engineering. This is really prototyping with electronics. That makes it possible to actually start prototyping, shaping, testing and exploring the interface between people and the product, which is very important.'

Education

'Learning to play and be creative with technology is also what we want to emphasize in the TIII project. As an educational institution, we believe that education should be more oriented toward this. A large part of innovation consists of creativity and daring, but there is no room at all for these traits in traditional education. Conventional education is mostly about solving problems. You are presented with questions that have only one right answer. The answers are in the back of the book, but you aren't supposed to look. You also aren't allowed to talk with your fellow students, since that's called cheating – but in normal life we call it cooperation. That has to change.

'As a teacher, I notice that freedom is very important for students. You have to give them a good environment and the right tools, so they can get started quickly. But you must also help them get in contact with companies and users quickly. The social aspect is important; you have to get users fully involved in the development process. Talk with them, do tests with them, make something, solder something together, put it on your head, put it on and take it out on the street. Don’t keep sitting in front of the computer making models and running simulations to see how it could be, because that's a dead end.'

Ubiquitous computing

'The underlying idea of ubiquitous computing is that in the near future we will have computers all around us without noticing them. This ties in with the advent of the Internet of Things (IoT). All of us are already virtually present on the Internet. Each of us has an e-mail address, an IP number and an online profile. With the IoT, every product will also have a virtual presence on the Internet. Everything will be linked together then. The interface vision is a very important aspect of this, since you can’t use a keyboard to interact will all those products. If the products in your surroundings can respond to the input they obtain from your gestures or facial expressions, the boundary between the digital and physical worlds disappears.

'This semester we will start having our students integrate sensor for electronics into fabrics. This is an important step in interfacing with electronics. Now we carry our smartphones in our pockets, but we think that in future electronics and interfaces will be incorporated in the products we already have and like, such as clothing, wristwatches, jewelry, tables, chairs, windows and mirrors.

'With a bit of luck the future will be nicer, with fewer boxes and cables, fewer new devices, and more integration into existing products. And hopefully with more cooperation between designers, engineers and social scientists. That’s a good recipe for making nice, attractive and pleasant products.'

Main image: The social robot Ono, an open source and DIY-friendly project. All images courtesy of Jelle Saldien and TIII.

This article first appeared in the May issue of Elektor.