Presence in Virtual Worlds

There have been a lot of experiments conducted to establish how much people feel “present” in virtual words. Loosely, this is usually defined as how much a person reacts to the virtual objects as if they were real (this paper (pdf) gives a good sense of it in its first paragraph). One of my friends in a 3d virtual environment recently wanted to sit on a chair. He’d been so taken in by the quality of the illusion that for a moment he treated the virtual chair as if it actually could support his weight. This is presence, and it’s the zenith of what we aspire to.

Actually, I don’t think it is. I don’t have a nebulous feeling of ‘presence’ in VR, but then I don’t when I’m in a shop, or at work, and sometimes I specifically feel unpresent in the real world. If I’ve been reading a good book, I feel that I’m more in its world than in this one. I think that almost all of these experiments suffer from a way of thinking that almost negates what the experimenters themselves are hoping for. They all start from the assumption that the virtual world is not real, and there’s a difference between it and the real world.

In fact, when I’m in a new environment, real or virtual, I treat it the same. I explore, I try to work out the rules of this new space. I experiment. If I find that the chair in this environment is insubstantial and I can pass my hand right through it, I don’t think that the chair somehow isn’t real, I simply realise that it’s a new kind of real chair, a chair I can’t sit on. I’d be the same in reality if someone gave me an insubstantial chair.

Nobody reacts in a 3d environment as if they are simply being shown patterns of projected colours, just like nobody treats the TV they’re watching Schindlers List on as if it’s just a mesh of colours, or for that matter – nobody treats a book as a bunch of squiggles on sheets. Art is a virtual reality that we explore, experience and decode. In neither simulators nor the walking around world do we experience unfiltered reality. What we experience are the concepts that our environment suggests to us.

There’s no question in my mind that Crayola land, where some of my other friends taunted crudely drawn ducks, or the library, where I was briefly menaced by a ghost child while books flew around are real places. They are just spaces with different rules to spaces I’m used to, and spaces where the assumptions I’m used to don’t always hold. We spend a lot of our lives, particularly when we’re young, working out what is real and what isn’t, and it’s exhilarating to be in that place of possibilities. Reality is a puzzle, and we love to try to solve it.

One of the presence experiments that has been carried out many times is the Pit experiment (pdf). People are asked to conduct a simple task in the virtual environment, and then relatively suddenly are confronted with a huge hole in the floor. Physiological measures show that their heart rate goes up, and their skin conductance changes. Is it Presence? Have people responded to the ‘virtual pit’ in the same way as they would to a ‘real pit’?

The Pit Experiment

I don’t think so, even though when I was there, I nearly lost my balance stepping out over it. (Did me talking about it as if it was really ‘there’ bother you?) I responded to it exactly as I would respond to a real pit – that I knew I couldn’t fall into, but that’s because it was a real pit that I knew I couldn’t fall into. There’s a difference between a ‘real pit’, which contains the notion of something you can fall into and a real pit that you can’t fall into. I’ve jumped out of an airplane, strapped into a parachute, knowing I was completely safe, but that doesn’t mean my heart rate didn’t go up. I responded to it not as I would respond to a real fall, but as I would respond to a real fall where I was safe – a virtual fall.

Of course, it’s difficult to run the experiments to prove this – you’d have to have an insubstantial chair to give me (and how would you carry it?), or run an experiment with a pit that people couldn’t fall into (perhaps using wires?) and compare it with a pit that they really could fall into. It’s difficult to get ethics approval for truly potentially life threatening experiments, even to try to establish a baseline.

And here we find some of the more interesting brand of presence experiments in virtual worlds. Doing experiments virtually that are too unethical to do in the real world. There’s the famous Milgram experiment for example, difficult to get approval for these days, but the virtual version is fine (I think it should be a compulsory part of education to watch the video of some of the 35% of people refusing to give the shocks in the original experiment. It’s inspiring). I saw an experiment where a virtual woman chatted up men, some of whom found her attractive and felt guilty with who knows what effects on the long term health of their relationships. The Millgram experiment is not ethically challenging because of its effect on the actor who is pretending to be shocked, it’s ethically challenging because of its effect on the participants. If people do respond in the same way to the virtual one as they do to the real one, then it’s just as unethical. There is nothing more or less real in the Virtual Milgram experiment than in the “real” one.

In some ways it’s more disturbing. Just like the man in a relationship who wondered if he was a freak because he found himself fancying a virtual woman, it is challenging to our sense of what it means to be human to discover our emotions and cherished empathy responses can be tricked and played so easily. And perhaps this is where the most interesting work is, searching the psyche for what makes us tick, because after all, finding out how we respond to what is inside our own and other peoples heads is even more important than finding out how we respond to various forms of the outside world.

Keyboard Layouts

Bye Bye QWERTY

Everyone knows that QWERTY (urgh, that’s difficult to type) was originally designed mainly for mechanical concerns rather than ergonomic ones. Given that, its astonishing that people can use it so well, and also that it’s outlasted the concerns that led to it by such a long time.

Anyway, it seemed crazy to me, so I’ve just finished the goodtyping basic course but with a twist; I did all the lessons with my keyboard set to Capewell (0.9.1) (there is a newer version available from his main page). I’m writing this with it as well, and although it’s slow going compared to the speed I type on QWERTY, I’ve been quite surprised that it has made no discernable difference to my ability to type in QWERTY. I expect I’ll pick up speed as I keep using it.

Capewell Layout

Alternative layouts

  • My choice was a Capewell layout because he takes the very sensible point of view that you’ll still need to be able to hit control-C, control-V, control-X and control-Z easily. He also doesn’t have the bias against running letters in the same hand, and what could be better than using a keyboard layout that evolved?
  • Another evolved layout comes from Peter Klausler, although he doesn’t rate it as highly as Dvorak.
  • Arensito is an interesting layout, particularly because he’s put some thought into designing it for programmers, perhaps more sensible than using a keyboard layout designed for typists. He’s also taken a slightly more radical approach and moved the hands position from the QWERTY to give those thumbs more options. Nice.
  • The asset layout is a nice layout, designed to be easy for the QWERTY typist to learn, but with a much better home row.
  • Maltron, mentioned below have their own layout. Check out their comparison of the home row frequencies. Yes, QWERTY really is that bad.
  • Compare different layouts here. You’ll find that the Capewell layouts perform quite well.
  • Fitaly is an input system designed for one finger or pen input.
  • Create your own with microsoft keyboard layout creator (warning: Microsoft.com link).

Alternative input devices

See also Hongkiats blog post.

  • SHARK a very cool gesture based system for tablet input
  • Touchstream LP a brilliant looking touch board that supports gestures and is a keyboard and mouse at the same time. Sadly they don’t make them anymore and they’re still pretty expensive on eBay.
  • A proper, no gimmicks single handed chordic keyboard is the Infogrip BAT keyboard, but it looks plasticy and cheap. Don’t they realise that the future is all chrome and black and OLEDs?
  • No one could deny that the Data Hand looks very cool and it has Zero hand movement! I’m not surprised that NASA use them.
  • The Orbitouch is a cool idea, but aimed more at disabled people rather than geeks looking for the future of input.
  • The tactapad has cameras, two handed interaction (that’s really cool), and a tactile feedback system. Not available for sale yet, probably quite expensive, and not really a keyboard either, although I like their idea of putting one in the split of a split keyboard.
  • The alphagrip is a game controller like keyboard and mouse. I like this idea, since moving the hands between keyboard and mouse is a very annoying context switch. I also like the ability to lean back that it would give you. Sadly, people don’t seem to get enormous speeds with it, and it’s really designed for more mobile devices. Maybe good for a tablet pc when you need a keyboard.
  • Like the accordian? Perhaps you want a vertical keyboard.
  • The Twiddler2 (yes I’m thinking friends and Claw) is a really neat chordal single handed keyboard and mouse, perfect for adding to your wearable computer cyborg get up. Again, not really designed for replacing a normal keyboard and mouse though.
  • I really like the Ergodex DX, a moveable custom keys system but it really needs to suport chordal behaviour.
  • Maltron make a large number of cool different keyboards, including one handed ones, and hyper ergonomic ones. Not cheap though.

Ideas

Try to implement a chordal system using a normal keyboard and perhaps turning it sideways. Get a mouse with buttons on it that can be programmed to Cut, Copy, Paste, Escape, Return. Ideally include Undo, Redo, Space, Alt, Home, End, Arrow keys, Delete, PgUp, PgDown, Tab, actually, why not the whole keyboard? A chordic mouse-keyboard, marvellous.

Jumping off links

One handed keyboarding in general, aimed at disabled people, but one hand on the mouse and the other on the keyboard might be pretty good.

The pretty Optimus keyboard might be good for learning new layouts on (if it were available).

Cornell unversity page on keyboard design.

Rahel pointed out that I really should include the new microsoft chordal layout that’s been designed from the ground up to take account of the frequency of key usage. You may have seen it before, but I think it’s innovative enough to deserve a link.