A long time ago we, humanity, drew an animal on a cavewall and pointed and grunted at it. Somewhere down the line we invented the Alphabet and speech. But the last few decades, starting with the first Graphical User Interface, we are back to pointing and (sometimes) grunting again. We mainly use a mouse to command our machines and with the touchscreen devices we don’t even bother working with a mouse anymore. We just point at the screen all the time. In 1996 Jakob Nielsen and Don Gentner wrote an article (The Anti-Mac interface) about the Graphical User Interface in which they already point (no pun intended) out this problem. Fourteen years! In light of that article we’ve really gone downhill. Devices are being reduced to one big screen where pointing is the only option.

There are some bright points though. The mobile phone OS Android by Google is already implementing voice recognition in a workable way. Which is hard to do because of the many different languages which all have many different accents. But we need to push it further, we need to re-introduce the command line!

For example: You have a folder with images but you want to move all the images which are bigger than a certain size (between 1MB and 2MB) moved to another folder. On top of that you want to remove all the images which are bigger than 2MB. How great would it be if you could just type (or ask) the following: “Remove all images bigger than 2MB but put the ones between 1MB and 2MB in a different folder named ‘Big pics’.”

Already there are some great examples of applications which understand our (complicated) human semantics. Such as Ubiquity, a Firefox Plugin which is can be controlled with text commands making it easier for the user to navigate. But the best of all is Google. Google understands human talk. A very good example (taken from this article in Wired) is the following query: “hot dog”. Google understands you are looking for the food product and it understands that you are not intersted in boiling puppies. Look at it this way: imagine Google was touch-only and you had to click your way to the right result. It would take hours for you to get the desired result, so why are we so easy to accept this with other devices/services?

Currently we are in an interface paradigm which is dominated by touch. The paradigm is kept alive because we are too lazy. We’ve taken a long time to get accustomed to this paradigm so why throw it all away? Hopefully we will come to our senses so that we can stop touching and start talking with our machines.

Just came accross Ubiquity.  It’s a plugin for Firefox which enables text driven commands. We’re stuck in an interface paradigm where point and click rules but some things can be easier done using language. Ubiquity enables users to use text driven commands to perform tasks such as:

• A wikipedia search,
• a google search,
• updating twitter,
• adding something to Google Calendar, and much more.

Check the demo below:

I’ve been using it for about an hour and I’m already loving it.

Last week Bill Gates introduced the touchwall. A glimpse of the future? We’ve seen this before. Tom Cruise as Chief John Anderton slides through vast amounts of data using his hands in the Minority Report. But do we really NEED this piece of technology to enhance our interface capabilities or do we WANT this because it looks so cool in Minority Report?

We WANT it!

In Foresight and Hindsight: The Case of the Telephone Ithiel del Sola Pool et al. take a closer look at the predictions made about the social effect of the telephone. Predictions made by inventors and big telephone companies had a big chance of becoming reality. They created a future, sold it to the public and they went on inventing this future.

This concept is called the self-fulfilling prophecy. We see Cruise flicking through all sorts of data using his hands and fingers. We think: “That’s great! I want that to be a wall in my living room!” and six years after the movie has been released we see this becoming reality. And we WANT it!

Do we NEED it?

But do we actually NEED this piece of technology? Is this going to break us free from the desktop interface? Jakob Nielsen and Don Gentner made a claim against the desktop interface back in 1996 in their article The Anti-mac Interface: “We seem to have settled on the WIMP (windows, icons, menus, pointer) model, and there is very little real innovation in interface design anymore.” (Gentner and Nielsen, 1996:1). One of their main arguments is that we ignore the value of language:

“The see-and-point principle states that users interact with the computer by pointing at the objects they can see on the screen. It’s as if we have thrown away a million years of evolution, lost our facility with expressive language, and been reduced to pointing at objects in the immediate environment. Mouse buttons and modifier keys give us a vocabulary equivalent to a few different grunts. We have lost all the power of language…” (Gentner and Nielsen, 1996:3)

Although the technique is very cool we still need to consider: Is this a step forward or is this a step backward?

Sources

Pool, I. de Sola et al. “Foresight and Hindsight: The Case of the Telephone” The Social Impact of the Telephone. Ed. Ithiel de Sola Pool. Cambridge, Massachusetts: The MIT Press, 1977.
Gentner, Don and Jakob Nielsen. “The Anti-Mac Interface”. 1996. Link to the article. (PDF)

Some people claim that games might break us free from the boring interface shaped as a desktop. But sometimes even games get it wrong. Even the successful games.

I will discuss three interface mistakes made by three popular XBOX360 games released this year. Halo3, Assassins Creed and Call of Duty 4.

1) Unable to skip cut scenes – culprit: Assassins Creed 
Assassins Creed is a fantastic interactive game. You are able to fly over the rooftops of Medieval cities and kill unsuspecting targets in various entertaining ways. But… you can’t skip the cut scenes. Nope. You have to sit through them. And they are boring, very boring and long. I understand the goal is to create an interactive story, but at least give the player an option to bail out!

2) Bad visual indicators – culprit: Halo3
The most hyped game of this year. And don’t get me wrong, it is a great game. But sometimes it’s so damn hard to see if you killed someone. Especially when you and your teammate are pumping bullets in the same enemy. Eventually your foe will drop to his knees and you have to check the scores to see who got the kill. In a game like Halo 3, where ranking is a big part of the fun, keeping track of your kills is very important. The visual indicator which states you killed someone is a small sentence which pops up on the left bottom corner of your screen. It is almost invisible.

3) Ugly menu’s – culprit: Call of Duty 4.
Speaking about visual indicators, this fourth installment of Call of Duty gets it right. An announcement in the centre of your screen informs you how much points you just earned with a kill or action. Great! Too bad the the menu’s in this game are just butt-ugly. So much care is put in the gameplay, the balancing of the weapons and the crafting of maps but the ball was dropped when they slapped on this Frankenstein of a menu. The font being used is getting on my nerves, it looks severely outdated. The colouring and shapes make you wonder if you accidentally popped in Call of Duty 2. The subtitle of the game is ‘Modern Warfare’ but the menu’s look rather ancient.

Although games might be the future of the interface some mistakes made are very annoying. Game developers should take as much care in the interface as they do in other elements of the game.