Blind Volunteers Needed

[EDIT: 6th May 2008] This stage of the project is now complete, we're no longer looking for volunteers. Many thanks to those who participated.

---

For the next stage of our project we will conduct interviews with people who are blind or significantly visually impaired.

We'd like to take 30 minutes of your time for a voice chat to hear about your experiences of getting around in the real world, and any experiences you have of doing so in virtual worlds. The aim is to direct our further work developing interaction techniques for blind users in Second Life.

For more information please read the Explanatory Statement and Consent Form.

If you'd like to participate please send an email to Gareth White, G.White@sussex.ac.uk, confirming that you've read both of these documents and agree to the conditions listed in the consent form.

Haptics

Following up from an initial thought and a more extensive conversation on another post I've begun considering a multi-modal approach to this project, mixing spatial audio with haptic feedback from devices such as the Novint Falcon.

Briefly, the Falcon is a consumer 3D haptic device targeted at the mass gaming market. Bit-Tech recently published an in-depth review which is worth reading. The product is interesting for this project as a mobility device to complement spatial audio. Think of it as a super-sensitive long cane white stick that could allow SL residents to reach out from the physical world and realistically feel objects in the virtual world.

Here's an informative promotional demo, and a rather enthusiastic hands-on by CNET's Veronica Belmont from the CES 2007 show. Novint also gave an audio interview with Gamasutra and a presentation at the 2007 Game Developers Conference.

Optional hardware devices are usually ignored by game developers as the effort required to support them is not justified by the small number of users who own the devices, and as there are few games that support them it's unlikely that many gamers would buy such devices: Catch 22. However as pointed out by a reader of this blog, blind users will often spend thousands of dollars on specialist hardware such as Braille keyboards, so the $190 that the Falcon costs is a relatively small investment. Furthermore support for the device in SL and other open source or moddable software can be implemented by the community rather than relying on industry.

The Falcon compares favourably to other haptic devices which cost upwards of ten times the price (for example Sensable Technologies' Phantom range of haptic devices are in the range of multi-thousand pounds Sterling.), and there are already some other academic researchers investigating haptics in Second Life and the Falcon in particular.

Jeff VanDrimmelen, an Academic Computing Expert in the Office of Arts and Sciences Information Services, University of North Carolina at Chapel Hill publishes research on a site called Haptic Education - Adding the Tactile Sensation to Virtual Learning.


VanDrimmelen's team have focused on another virtual environment called Croquet, but have also considered Second Life and make some interesting observations,

The creators of Second Life actually started their project out with a large haptic device, but soon abandoned it for more financially appealing options.

In Second Life the only way to navigate with a mouse is to bring up an on screen navigation menu that you have to click to move the avatar. It works okay when the avatar is flying, but otherwise you just end up using the buttons on the handle to move around. However, just in case anyone wants to work with the script, here it is.

In Linden's default client movement is controlled using the keyboard, but in my own research I have recently been able to control by walking and flying using a force feedback joystick (Logitech Wingman Strike Force 3D). This was made possible by using a free 3rd party tool called GlovePIE which VanDrimmelen's team also employed. The tool works by intercepting output from the joystick and injecting the corresponding keyboard signals, such that by moving the joystick left and right the Second Life avatar turns left and right, and moving the joystick forward and backwards moves the avatar forward and back. The same technique is used by VanDrimmelen's team to use the Novint Falcon as input device for Croquet. This approach appears to offer a very quick and easy way to prototype haptics in Second Life. VanDrimmelen continues, however,

It should be noted that about the same time we found the GlovePIE software Novint announced they are working on drivers that will work with not only Second Life, but World of Warcraft as well.

Currently both of these drivers are "in exploration phase" with no estimated completion date. Also in their (busy!) release schedule Novint also describe another interesting product, "Feelin' It: Blind Games™":

Novint will release a number of games that can be played entirely without sight. For example, in a bowling game, you will be able to feel the extents of the lane, feel the weight of the ball as it is thrown, and hear the pins crash down. After throwing the ball and hitting the pins, the game will bring up a touchable representation of how the ball traveled down the lane to guide the user's muscle memory for future shots, and the user will be able to feel with a 3D cursor which pins are still standing. All the information needed to play the game and become a true master, will be available without any graphics.

Further haptic research in Second Life is being conducted by Maurizio de Pascale, Sara Mulatto, Domenico Prattichizzo from the Haptics Group of the Siena Robotics and Systems Lab, in the Dipartimento di Ingegneria Informatica at the University of Siena. In particular they have a paper called "Bringing Haptics to Second Life: A Haptics-enabled Second Life Viewer for Blind Users", which is due for publication at the "Haptic in Ambient Systems" conference, which takes place in Quebec City, Canada on February 11-14, 2008.

Judging from the screenshot, I would imagine that the Siena team are not using the Novint, but rather a different haptic device that has a stylus, perhaps one of SensAble Technology's Phantom range which seem popular in academic research.

Another research project that is of interest as inspiration for our Second Life work is the Haptic Torch from the Interactive Systems Research Group at the University of Reading.

"The unique design of the torch allows users to range from sighted individuals in low-light conditions to people who are both deaf and blind. The torch provides a method of alerting users to presence of potentiol hazards using non-contact measurement techniques. An subtle tactile (touch) interface conveys relevent information to the user while not interfering with other senses." [sic]

Whereas the Haptic Torch is only capable of signifying the presence of objects, the Falcon could be used to reach out and feel their shape, and this immediate physical stimuli will assist the users construction of a mental map of the virtual space.

Of the suppliers that Novint list as selling the Falcon, Fry's looks like the best for delivery outside the USA and Canada, although it should be noted that in the Gamasutra podcast mentioned earlier, Novint's CEO, Tom Anderson, mentioned that they are providing Falcons to game development studios for free and they have also used the Falcon as an interface for medical dental simulators with the Harvard School of Dental Medicine. With their aggressive PR policy perhaps they'd extend this generous offer to other academic research projects too?


Fry's
"Through DHL, we quickly ship international orders just about anywhere in the world for very reasonable rates."

SkyMall
"Many SkyMall products are available for delivery outside the United States."

TigerDirect
"For all international orders, export, and distribution please contact our sales force at: 800-800-8300"

CircuitCity
"Due to our manufacturer distribution agreements, we are not permitted to ship products to international addresses except APO/FPO or U.S. Territory addresses. Circuit City does not ship to Puerto Rico."

GoGamer
"GoGamer does not ship to International Destinations at this time."

JR
"At the present time, we only ship to the Continental U.S., Alaska, Hawaii, U.S. territories, Puerto Rico, and Canada. J&R proudly ships to our Armed Forces APO/FPO customers."

SLED Accessibility Threads

Following are excerpts from the SLED mailing list, which deal with accessibility and visual impairment,


SLeek for the vision impaired?

Sean FitzGerald (seanf at tig.com.au) writes,

But the really neat thing about SLeek that leads me to think it may be useful for the vision impaired (if screen readers work with it) is that it has a "Follow" function that lets you nominate to hitch your av to another av in range... a virtual version of a vision impaired person holding the arm of a guide. It works quite well. Then the guide just has to describe the environment.

Blind people in SL - Idle speculation

Mike Reddy (mike.reddy at newport.ac.uk) writes,

This would be easier with a client SDK that could trap text, use text to speech and allow keyboard macros, but given the existing client could we not have a HUD or head mounted scripted object that 'spoke' information. Location, people's names as they came and went, object IDs. Within the current system, these would probably have to be pre-recorded and linked to specific text, say in a notecard. Alternatively, objects in an 'accessible' area could be able to self report, say if someone approached them within a certain distance for a certain time. This area could be made the home location for participants. We could even run a competition to design accessible vending machines that used sound instead/as well as text.

To aid people with visual impairments - most people who are blind aren't actually 'blind' - it would be great to have control over field of view in the client, which could effectively allow mouse view of a small angle to be the equivalent of a magnified image, much as PC viewing software allows the whole screen to be enlarged. Sadly, this would not easily include text. However, if we had a HUD object repeating any 'heard' text in the mouselook view, then even this might be possible. This would require chat in the mouselook view...

Ah well, maybe when I have a PhD student to throw at it...

Jeff Hiles (jeffrey.hiles at wright.edu) wrote

As Danielle said, right now you would have to pair a blind student with another student or with an assistant who could navigate, read, and describe what's on the screen. That's not unique to people with visual disabilities, though.

The visually impaired could participate more directly, though, if the SL client was accessible to screen readers. I know blind people who have embraced instant messaging with clients that work with JAWS. So, in theory, it would be possible for people who can't see to carry on their own text conversations in Second Life. That degree of independence, I think, would make the experience more immediate and immersive.

However, the Second Life client doesn't currently give screen reader access to chat or IM text. In fact, you can't even read the menus with JAWS. If the client did have that most basic accessibility--chat, IM and menus--blind users would still need some assistance getting around.

Accessibility

Lisa Dawley (lisadawley at boisestate.edu) writes,

I was doing a presentation in our amphitheater one day. A gentleman in a wheel chair asked me if I could make the stadium accessible, because there wasn't a seat large enough for him to "land" his wheelchair and he had to float.

Second life for the visually impaired

Roome, Thomas C (thomas.roome at student.utdallas.edu ) writes,

In the near future the Internet will make a shift from web sites to a 3D environment spaces. The same information that is on a web site can be available to people in a 3D environment, but the question is how can a 3D environment be accessible for people with disabilities? The UTD Accessibility Island will be trying to find the answers to this question. One of the, island goal is to provide information on video game accessibility and general information on the different disabilities. Another goal is to create a conference center for people to discuss different topics around Accessibility. The last major goal of the island is to provide some land for research and development, and I want to form an in world research team of scripters, programmer, educators and people with disabilities. If you would like to become a research team member, then please contact Tom06 Castro or e-mail thomas.roome at student.utdallas.edu

Further thoughts on people with visual disabilities in Second Life

Jeff Hiles (jeffrey.hiles at wright.edu) writes,

When I work with JAWS users in real life, they sometimes ask me to give them my arm and guide them where they need to go. What if you could "give an arm" in Second Life and lead someone around? Better yet, what if you could do that invisibly so no one else in Second Life knew you were there? The key would be for you to be able to guide someone remotely, without having to be in the same room as the person you were guiding.

For example, as a guide, you would have the ability move your friend's avatar through Second Life, and to see what that avatar would see. But your friend would have control of chat and IM. From your computer, you would move the avatar through Second Life wherever your friend asked you to take it. The two of you would communicate by voice, say through Skype, and you would describe everything you saw.

Danielle Mirliss (dmirliss at yahoo.com, Danielle Damone in SL) also comments,

I also work closely with several students on my campus that are blind and they would be willing to give us feedback on the experience.

Disability in SL

The BBC disability website, Ouch! discuss some of the appeal of SL in Staying in is the New Going Out

A new nightlcub called Wheelies officially opens its doors this Friday, the 1st September, at 9pm UK time.

Owned by Simon Stevens, who has cerebral palsy, Wheelies aims to make guests feel comfortable about disability as well as dancing, drinking and just plain having a good time.

And in the comments, Kopilo Hallard quite rightly says,

The point is that he couldn't go out and socialise and SL gives him a platform so that he can meet his needs (ie socialisation) even in his current physical state.

This gives him an escape from reality, a breath from being physically unable to do things.

Besides that point, SL is a great way to network with people from all over the world. To gain perspectives which may not be abled to be gained in the geographical region due to culture, social or other conforms.

Also SL gives developing artists both music, graphic, programming, etc a way of having more exposure which they can not just gain in their day to day life, if you like in a similar way to myspace, except the music can be played live.

Additionally, on the SLED mailing list, Jeff Hiles (Farley Scarborough in SL) recommends,

In addition to the many articles on the Web about Simon Stevens and his
Wheelies night club, you may want to look at Fez Rutherford's blog,
"2nDisability." He has created avatar animations that simulate disabilities.

Also, Cubey Terra has made three very nice wheelchairs that are available free at the GNUbie Store at Indigo. They are down the ramp and to the left.

http://slurl.com/secondlife/Indigo/195/62/40

Mailing Lists & Fora

Generally accessibility issues seem to be dealt by Linden with John Lester, AKA Pathfinder Linden.

"Pathfinder Linden: well, I'm very interested in things we might be able to do from LL's perspective to make SL more accessible

Pathfinder Linden: so please hit me with recommendations :)"

The following resources might be useful,

• Second Life Researchers is a mailing list for researchers (obviously).
• Healthcare Support and Education is "for people interested in or currently using Second Life for Healthcare Support and Education".
• Second Life Educators [SLED] is a mailing list "for educators interested in or currently using Second Life".

SLED also has a forum with a thread called Supporting Visually Impaired Users, though it looks like it hasn't had any posts since January 2007.

From that forum Jonathon Richter has the following to say,

I concur that we ought to frame the problem in terms of the various affordances that Second Life as a medium allows its users - indeed, the benefits of SL over other types of learning media are precisely the selling points as to why we want accessibility to these incredible learning environments, yes? So, first - documenting the various affordances and the skills/inputs required to successfully navigate the media and receive said benefits is crucial.

And Jeff Farley has this to point out:

"There's a slogan in the disability-rights movement that goes 'Nothing about us without us.'"

Following are some extracts from the SLED mailing list that deal with visual impairment:

Different users of Second Life

Jeff Hiles writes,

But I think the most promising technological aide lies inways to make SL chat accessible, since the SL client isn't accessible to screen readers.

If I had to accommodate someone today, I'd approach it like a stage production and provide a skilled audio describer. The describer might join the blind person or communicate through VOIP. The person could then tell the describer where to move the avatar and what to do with it, while the describer summed up the scenes along the way. If it was done right, perhaps no one in SL would know the person was blind.

Ideally, the person's screen reader would have access to his avatar's chat so that communication would be direct, not through the describer. I'm not sure if that level of accessibility is possible yet.

SimTeach host the Education Wiki and also recently published a transcript of a meeting called Supporting Students With Disabilities", which was about

"discussing accessibility issues within Second Life, with a particular focus on how to best accommodate students with disabilities when SL is used for educational purposes"

This is the summary,

* The user interface and software of SL does not currently allow much freedom in regards to how it is manipulated (e.g., mouse versus keyboard). The UI is currently not JAWS- compliant as well. The use of XML-based user interfaces in future versions could provide great flexibility for tuning the software to a user's needs.

* The vast amount of visual information in SL is currently inaccessible to residents with visual disabilities. The addition of metadata (like the ALT and LONGDESC tags used for images in HTML) was suggested. While enforcing the inclusion of helpful metadata is tricky, it was agreed that educational builds at least should adhere to a standard.

* Regarding accommodations for a student with disabilities in SL, it was suggested that equivalent RL practices could be applied. A blind student might have a companion to assist him or her. This led to an interesting question regarding whether the companion or the student or both would have avatars in SL.

The following parts of the conversation have been cut from their context and reassembled without any intervening and off-topic posts,

Kate Spatula: have anyone of you had an instance where a person with a disability, say visual issues, was involved in a class using SL?
buridan Simon: /not that i know of

Ali Andrews: not yet

Gus Plisskin: Kate: Not visual issues, but I've build footpedals for those with carpal tunnel who can't use mouse buttons.

Kate Spatula: so that's one concern... the interface is very mouse-heavy on here, isn't it?

buridan Simon: /mouse heavy as compared to?

Ali Andrews: especially when building

Janor Slichter: more keyboard commands to drive menus and actions?

Gus Plisskin: yes, but SL needs mouse heavy. An alternative would be very tough

Janor Slichter: the way gestures work in chat?

buridan Simon: i dunno, i find that i use the arrow keys a fair amount

buridan Simon: /and the tab

otakup0pe Neumann: Hello everyone. I know that lots of builders do just that

otakup0pe Neumann: Rez a cube, and use tab/arrow keys / numpad for the specifics

buridan Simon: /what would be nice is better proximity detection for friends and colleagues with audio cues.... so a friend approaches and a sound could get louder....

Rubaiyat Shatner: I think a big issue with accessibility is to somehow expose the data so that it can be read if it is text and translated if not

Corwin Carillon: if the cleint was JAWS compliant you would get some of the with HUDs buridan

Janor Slichter: Kate, are you referring to being able to add special functions, like with add-ins, to the client, to accomodate certain needs?

Kate Spatula: that's one approach that could be taken, or providing hooks for external software to use (like JAWS requires), or these could all be optiosn built in to SL

otakup0pe Neumann: I sense this is a direction that LL wants to move in... but i really have no idea

Farley Scarborough: JAWS access and keyboard access are both very standard on Windows apps

Kate Spatula: so here's the difficult question, if you had a class where SL was a key facet, and one student was blind, what would you do?

Ellie Brewster: seems to me that you'd have to get them a companion

Ellie Brewster: just as you do in a rl class

buridan Simon: /All of the students that i've had that were visually impaird had companion assigned anyway

Kate Spatula: so would they have an avatar on here or just the companion or both

Farley Scarborough: There are profesional describers we use in RL

Krisjohn Twin: @Kate: I just walked into this room, sat down at a pre-defined spot and started typing. How hard could that be to script for someone who is blind? Most of the 3D interface in SL is wasted.

buridan Simon: /it is true the 3d doesn't matter as much as proximity

Krisjohn Twin: Heck, an IRC bridge to this room would probably be more than enough to participate in this discussion.

Ellie Brewster: you can use sound files as cues. Tie them to the scenery

Farley Scarborough: Ah, but the visual's... they aren't wasted on the blind.

otakup0pe Neumann: And scripting movement will get more interesting with libsl.

Farley Scarborough: Listen to an audio described movie

Kate Spatula: so let's consider this room. could we augment it to make it more accessible beyond just visual

Farley Scarborough: the visual description is very important

Gus Plisskin: For the person who's visually-impaired, rather than blind

Ellie Brewster: what about using a different channel for viz impaired?

Gus Plisskin: with description? that'd work

otakup0pe Neumann: Do you mean chat channel Ellie ?

Ellie Brewster: yes

buridan Simon: i think someone has an irc bridge

otakup0pe Neumann: there are several

otakup0pe Neumann: we have developed one (we being my company)

otakup0pe Neumann: and i knwo tehre is one with libsl

buridan Simon: /Actually i know irc, and im bridge

otakup0pe Neumann: and the #secondlife irc channel runs one

Kate Spatula: i'm loolking right now at pictures of some famous philosophers hanging on the walls. the environment could provide a list of tagged objects to the user

Kate Spatula: which would be useful to scripters as well

otakup0pe Neumann: my company is in the process of developing a "hidden" metadata system for SL object

otakup0pe Neumann: uhh. hidden is a bad word.

otakup0pe Neumann: ubiquitous ? heh.

buridan Simon: /hah good luck with that... tagging perhaps, but object standard metadata...

Kate Spatula: the challenge, as it is in web accessibility, is making sure the data is provided

buridan Simon: /cidoc is a bugger

otakup0pe Neumann: maybe metadata is also a poor word ;)

buridan Simon: /metadata is the word... it means data about data

otakup0pe Neumann: I know. There are many kinds of metadata.

otakup0pe Neumann: And as we just saw, only so much room in a script.

Kate Spatula: i'm sure i could force rubaiyat to tag Trotsky's, but what about *insert random place* here

otakup0pe Neumann: and kate, good point again. tagging the whole grid is a daunting task =O

otakup0pe Neumann: let alone both grids !

otakup0pe Neumann: and having them all work together.

Ali Andrews: but isn't it tagged already, in the edit window?

buridan Simon: /tagging is also an area where you will have a good number of people who vary and some who actively resistantly participate by tagging wrongly

otakup0pe Neumann: That's a idfferent kind of tagging Ali.

Ali Andrews: how is it different? It can list the name, discription... it just needs to be done consistantly as we do when we build web pages

buridan Simon: 'everything is a cube'

otakup0pe Neumann: This is true Ali

otakup0pe Neumann: Consistency is the key.

Kate Spatula: there is a difference, ali.

Ali Andrews: so at least for our educational builds we can start a standard

otakup0pe Neumann: it's metadata ,but not strictly descriptive

otakup0pe Neumann: i wonder how many "objects" are around here.

Kate Spatula: web pages have a structure that supports the use of those descriptions. however, accessing just names and descriptions on here is fairly unstructured

otakup0pe Neumann: lack of consistency....

buridan Simon points out that there are standards, and it is better to attempt to conform to a standard than to create one anew

Bryan Mnemonic: does linden tag any objects with metadata at all?

Pathfinder Linden: not really, not in the sense you're thinking about

otakup0pe Neumann: object name, description, groups, that is all metadata

otakup0pe Neumann: but yeah. not too "descriptive"

Kate Spatula: here's a related issue... avatakind of like all the image alt tags that say "image"

Kate Spatula: rs and disability. aside from pathfinder with his lack of a nose and rubaiyat's inability to store fat, none of our avatars really display signs of disability

Bryan Mnemonic: I wonder if that can be added to the "edit" build window so folks ban begin adding specific tags, or a limited number of them based on a drop down menu

Kate Spatula: going back to our blind student, would his avatar have a white cane an dsunglasses?

otakup0pe Neumann: kate : it's up to them

buridan Simon: /it would seem to me that identity is up to them

Bryan Mnemonic: for instance, we riright click on this couch, and when we click "more"perhaps there could be an additional tab

Bryan Mnemonic: with metadata options

buridan Simon: /metadata that can be filled with 'arse' and related words

Farley Scarborough: Ah, but of course the blind student couldn't see where to click, so wouldn't have a mouse

Kate Spatula: i'm not saying to enforce avatar appearances, but consider the importance of avatar image to reflecting disabilitiy

otakup0pe Neumann: do you think other avatars would requrie that visual cue ?

Kate Spatula: there are two sides ot the issue. one is whether other avtars need the cues, but then there's also the importance of digital identity and one's avatar image

Pathfinder Linden: there was a recent paper about social cues and avatars in SL recently...

Farley Scarborough: To expand Kate's question... Many people with disabilities you can see in RL can go through SL without anyone knowing. Could someone who cant' see?

otakup0pe Neumann: i think it should be the option of the avatar in question

Kate Spatula: that's the point... are the avatar options enough to reflect some of these needs

otakup0pe Neumann: both sides really. does the blind student want to advertise that they are blind ?

otakup0pe Neumann: and do other avatars want those cues ?

Pathfinder Linden: aha, here it is: http://www.stanford.edu/group/vhil/papers/second_life_nonverbal.pdf

Kate Spatula: actually, they might want to advertise that they're blind, or in a wheelchair, etc. it's part of their identity and it's an issue of whether that person wants to express that and if they can express that

otakup0pe Neumann: but it's up to them.

Kate Spatula: RL example: i once roleplayed with a person who insisted that his elf cleric had a wheelchair built by gnomes

Farley Scarborough: It's only up to them if they can get about without anyone knowing.

Kate Spatula: from a disability critical studies viewoint, SL has an extreme bias towards physical perfection and able-bodiness

buridan Simon: /metadata is always a social and organizational issue first

Kate Spatula: for me, i think one of the larger open questions is how do people with disabilities view SL... both in terms of its usability and its potential

Farley Scarborough: There's a slogan among disability rights advocates: "Nothing about us without us."

Krisjohn Twin: For me, a larger question is how do people who barely know how to turn on a PC cope with SL at all? I've got plenty of able-bodied staff that have no chance coping with this environment.

The Acoustic Ecology of the First Person Shooter

I posted about Mark Grimshaw's Phd thesis on another blog earlier this year, but as it's relevant to this project too I thought it might be useful to repeat the link, especially as he gave a cut down presentation at DiGRA '07 in Tokyo.

Audio RTS

It sounds like an unusual prospect, but Jean-Luc Pontico has created a convincing Sound RTS. It's cross platform (Windows, Linux, Mac) and localised (French, English, German, Italian, Spanish) and free!

I'm particularly impressed with the menu system and the speech samples used. Very clear and easy to use. Real time 3D audio works reasonably well to give the impression of units moving around, and continual reports of building progress are useful and analogous to visually observing the state of a build or progress bar.

Apparently it has mutliplayer features and allows the players to use their own maps as well. There's an active forum and blog.

This is an impressive game with high quality production values and a complete RTS implementation.