Wednesday, 16 January 2008


Following up from an initial thought and a more extensive conversation on another post I've begun considering a multi-modal approach to this project, mixing spatial audio with haptic feedback from devices such as the Novint Falcon.

Briefly, the Falcon is a consumer 3D haptic device targeted at the mass gaming market. Bit-Tech recently published an in-depth review which is worth reading. The product is interesting for this project as a mobility device to complement spatial audio. Think of it as a super-sensitive long cane white stick that could allow SL residents to reach out from the physical world and realistically feel objects in the virtual world.

Here's an informative promotional demo, and a rather enthusiastic hands-on by CNET's Veronica Belmont from the CES 2007 show. Novint also gave an audio interview with Gamasutra and a presentation at the 2007 Game Developers Conference.

Optional hardware devices are usually ignored by game developers as the effort required to support them is not justified by the small number of users who own the devices, and as there are few games that support them it's unlikely that many gamers would buy such devices: Catch 22. However as pointed out by a reader of this blog, blind users will often spend thousands of dollars on specialist hardware such as Braille keyboards, so the $190 that the Falcon costs is a relatively small investment. Furthermore support for the device in SL and other open source or moddable software can be implemented by the community rather than relying on industry.

The Falcon compares favourably to other haptic devices which cost upwards of ten times the price (for example Sensable Technologies' Phantom range of haptic devices are in the range of multi-thousand pounds Sterling.), and there are already some other academic researchers investigating haptics in Second Life and the Falcon in particular.

Jeff VanDrimmelen, an Academic Computing Expert in the Office of Arts and Sciences Information Services, University of North Carolina at Chapel Hill publishes research on a site called Haptic Education - Adding the Tactile Sensation to Virtual Learning.

VanDrimmelen's team have focused on another virtual environment called Croquet, but have also considered Second Life and make some interesting observations,
The creators of Second Life actually started their project out with a large haptic device, but soon abandoned it for more financially appealing options.

In Second Life the only way to navigate with a mouse is to bring up an on screen navigation menu that you have to click to move the avatar. It works okay when the avatar is flying, but otherwise you just end up using the buttons on the handle to move around. However, just in case anyone wants to work with the script, here it is.
In Linden's default client movement is controlled using the keyboard, but in my own research I have recently been able to control by walking and flying using a force feedback joystick (Logitech Wingman Strike Force 3D). This was made possible by using a free 3rd party tool called GlovePIE which VanDrimmelen's team also employed. The tool works by intercepting output from the joystick and injecting the corresponding keyboard signals, such that by moving the joystick left and right the Second Life avatar turns left and right, and moving the joystick forward and backwards moves the avatar forward and back. The same technique is used by VanDrimmelen's team to use the Novint Falcon as input device for Croquet. This approach appears to offer a very quick and easy way to prototype haptics in Second Life. VanDrimmelen continues, however,
It should be noted that about the same time we found the GlovePIE software Novint announced they are working on drivers that will work with not only Second Life, but World of Warcraft as well.
Currently both of these drivers are "in exploration phase" with no estimated completion date. Also in their (busy!) release schedule Novint also describe another interesting product, "Feelin' It: Blind Games™":
Novint will release a number of games that can be played entirely without sight. For example, in a bowling game, you will be able to feel the extents of the lane, feel the weight of the ball as it is thrown, and hear the pins crash down. After throwing the ball and hitting the pins, the game will bring up a touchable representation of how the ball traveled down the lane to guide the user's muscle memory for future shots, and the user will be able to feel with a 3D cursor which pins are still standing. All the information needed to play the game and become a true master, will be available without any graphics.
Further haptic research in Second Life is being conducted by Maurizio de Pascale, Sara Mulatto, Domenico Prattichizzo from the Haptics Group of the Siena Robotics and Systems Lab, in the Dipartimento di Ingegneria Informatica at the University of Siena. In particular they have a paper called "Bringing Haptics to Second Life: A Haptics-enabled Second Life Viewer for Blind Users", which is due for publication at the "Haptic in Ambient Systems" conference, which takes place in Quebec City, Canada on February 11-14, 2008.

Judging from the screenshot, I would imagine that the Siena team are not using the Novint, but rather a different haptic device that has a stylus, perhaps one of SensAble Technology's Phantom range which seem popular in academic research.

Another research project that is of interest as inspiration for our Second Life work is the Haptic Torch from the Interactive Systems Research Group at the University of Reading.

"The unique design of the torch allows users to range from sighted individuals in low-light conditions to people who are both deaf and blind. The torch provides a method of alerting users to presence of potentiol hazards using non-contact measurement techniques. An subtle tactile (touch) interface conveys relevent information to the user while not interfering with other senses." [sic]
Whereas the Haptic Torch is only capable of signifying the presence of objects, the Falcon could be used to reach out and feel their shape, and this immediate physical stimuli will assist the users construction of a mental map of the virtual space.

Of the suppliers that Novint list as selling the Falcon, Fry's looks like the best for delivery outside the USA and Canada, although it should be noted that in the Gamasutra podcast mentioned earlier, Novint's CEO, Tom Anderson, mentioned that they are providing Falcons to game development studios for free and they have also used the Falcon as an interface for medical dental simulators with the Harvard School of Dental Medicine. With their aggressive PR policy perhaps they'd extend this generous offer to other academic research projects too?

"Through DHL, we quickly ship international orders just about anywhere in the world for very reasonable rates."

"Many SkyMall products are available for delivery outside the United States."

"For all international orders, export, and distribution please contact our sales force at: 800-800-8300"

"Due to our manufacturer distribution agreements, we are not permitted to ship products to international addresses except APO/FPO or U.S. Territory addresses. Circuit City does not ship to Puerto Rico."

"GoGamer does not ship to International Destinations at this time."

"At the present time, we only ship to the Continental U.S., Alaska, Hawaii, U.S. territories, Puerto Rico, and Canada. J&R proudly ships to our Armed Forces APO/FPO customers."


Gareth R White said...

Gamasutra are reporting that the Falcon has received a "Good Design Award" from the Chicago Athenaeum: Museum of Architecture and Design.

Ad said...

Hi Gareth,

I came across your blog and as the inventor of the haptic torch I thought I'd drop you a quick line. I think your comparison of the Haptic Torch with the Novint Falcon is rather unfair. They are completely different devices: the haptic torch is a mobile, cordless, handheld device used by the visually impaired for exploring real world environments while walking. The novint falcon on the other hand is a desk based device used primarily for representing virtual environments. They would both fair rather badly at each others jobs. Though the Falcon can represent shapes to higher fidelity then the torch it requires a PC to work and a large fixed frame via which it provides a reaction force to the user. Its not designed for mobility.

I'm actually collaborating with some members of Sussex Dept. Informatics on a torch spin off project ( and should be around your campus in march. Leave a comment on the enactive torch website if you'd like to meet for a chat (or to try out a torch)

Ad Spiers

Gareth R White said...

Hi Ad.

Thanks for commenting.

You're absolutely right about the differences between the Falcon and the Torch; I didn't mean to suggest that the Falcon was superior, so I'm sorry for giving that impression. In fact I mentioned the Haptic Torch because it was an inspiration for the work I'm doing now. It's clearly a great device for the real world and I was trying to identify interaction techniques that I could borrow and translate into the virtual world.

I'd definitely like to meet you in March and try out a torch, so I'll repost this on your blog and talk further on email.


Anonymous said...

As a visually impaired person myself I'll note most of the equipment we use that "costs thousands" is provided for by government initiatives, in the UK being such things as the Disabled Students Allowance. While they did provide me with a screen reader I doubt they would be terribly interested in providing most people access to Second Life unless it was a course requirement.

I applaud your project though and will be in contact in due course regarding your request for volunteers.

Gareth R White said...

Thank's for your comment.

You're absolutely right that the DSA is very unlikely to buy you a device just for Second Life.

Our research takes a long term view, with the belief that online 3d virtual worlds may become a significant environment for business and social communication (e.g Second Life and Sony's Home), similar to the way the 2D world wide web has in the last decade. While there are accessibility standards for the physical world, desktop computing and the web in particular, there's nothing comparable for 3D environments. By starting to research this now we hope to be able to produce work that can ensure accessibility for everyone if 3D becomes a dominant interface.

I see Second Life not so much as the destination but rather as the first significant implementation of this interface. Personally I think it's a very flawed implementation, but I think it's only a matter of time before someone else creates a really successful and usable version. With our work hopefully we can contribute to it being accessible as well.

Second Life now is a bit like the Internet was back in 1993 when Mosaic was first released. The first graphical websites were quite horrible and it took a decade before things settled down into a sensible format. We had to go through various different browser wars (Netscape Navigator, Internet Explorer, Mozilla FireFox), versions of the HTML standard, and education amongst developers and users before the Web became the de facto for information, business and social communication.

Looking forward to speaking to you at greater length during the study interview!