Haptics

Following up from an initial thought and a more extensive conversation on another post I've begun considering a multi-modal approach to this project, mixing spatial audio with haptic feedback from devices such as the Novint Falcon.

Briefly, the Falcon is a consumer 3D haptic device targeted at the mass gaming market. Bit-Tech recently published an in-depth review which is worth reading. The product is interesting for this project as a mobility device to complement spatial audio. Think of it as a super-sensitive long cane white stick that could allow SL residents to reach out from the physical world and realistically feel objects in the virtual world.

Here's an informative promotional demo, and a rather enthusiastic hands-on by CNET's Veronica Belmont from the CES 2007 show. Novint also gave an audio interview with Gamasutra and a presentation at the 2007 Game Developers Conference.

Optional hardware devices are usually ignored by game developers as the effort required to support them is not justified by the small number of users who own the devices, and as there are few games that support them it's unlikely that many gamers would buy such devices: Catch 22. However as pointed out by a reader of this blog, blind users will often spend thousands of dollars on specialist hardware such as Braille keyboards, so the $190 that the Falcon costs is a relatively small investment. Furthermore support for the device in SL and other open source or moddable software can be implemented by the community rather than relying on industry.

The Falcon compares favourably to other haptic devices which cost upwards of ten times the price (for example Sensable Technologies' Phantom range of haptic devices are in the range of multi-thousand pounds Sterling.), and there are already some other academic researchers investigating haptics in Second Life and the Falcon in particular.

Jeff VanDrimmelen, an Academic Computing Expert in the Office of Arts and Sciences Information Services, University of North Carolina at Chapel Hill publishes research on a site called Haptic Education - Adding the Tactile Sensation to Virtual Learning.


VanDrimmelen's team have focused on another virtual environment called Croquet, but have also considered Second Life and make some interesting observations,

The creators of Second Life actually started their project out with a large haptic device, but soon abandoned it for more financially appealing options.

In Second Life the only way to navigate with a mouse is to bring up an on screen navigation menu that you have to click to move the avatar. It works okay when the avatar is flying, but otherwise you just end up using the buttons on the handle to move around. However, just in case anyone wants to work with the script, here it is.

In Linden's default client movement is controlled using the keyboard, but in my own research I have recently been able to control by walking and flying using a force feedback joystick (Logitech Wingman Strike Force 3D). This was made possible by using a free 3rd party tool called GlovePIE which VanDrimmelen's team also employed. The tool works by intercepting output from the joystick and injecting the corresponding keyboard signals, such that by moving the joystick left and right the Second Life avatar turns left and right, and moving the joystick forward and backwards moves the avatar forward and back. The same technique is used by VanDrimmelen's team to use the Novint Falcon as input device for Croquet. This approach appears to offer a very quick and easy way to prototype haptics in Second Life. VanDrimmelen continues, however,

It should be noted that about the same time we found the GlovePIE software Novint announced they are working on drivers that will work with not only Second Life, but World of Warcraft as well.

Currently both of these drivers are "in exploration phase" with no estimated completion date. Also in their (busy!) release schedule Novint also describe another interesting product, "Feelin' It: Blind Games™":

Novint will release a number of games that can be played entirely without sight. For example, in a bowling game, you will be able to feel the extents of the lane, feel the weight of the ball as it is thrown, and hear the pins crash down. After throwing the ball and hitting the pins, the game will bring up a touchable representation of how the ball traveled down the lane to guide the user's muscle memory for future shots, and the user will be able to feel with a 3D cursor which pins are still standing. All the information needed to play the game and become a true master, will be available without any graphics.

Further haptic research in Second Life is being conducted by Maurizio de Pascale, Sara Mulatto, Domenico Prattichizzo from the Haptics Group of the Siena Robotics and Systems Lab, in the Dipartimento di Ingegneria Informatica at the University of Siena. In particular they have a paper called "Bringing Haptics to Second Life: A Haptics-enabled Second Life Viewer for Blind Users", which is due for publication at the "Haptic in Ambient Systems" conference, which takes place in Quebec City, Canada on February 11-14, 2008.

Judging from the screenshot, I would imagine that the Siena team are not using the Novint, but rather a different haptic device that has a stylus, perhaps one of SensAble Technology's Phantom range which seem popular in academic research.

Another research project that is of interest as inspiration for our Second Life work is the Haptic Torch from the Interactive Systems Research Group at the University of Reading.

"The unique design of the torch allows users to range from sighted individuals in low-light conditions to people who are both deaf and blind. The torch provides a method of alerting users to presence of potentiol hazards using non-contact measurement techniques. An subtle tactile (touch) interface conveys relevent information to the user while not interfering with other senses." [sic]

Whereas the Haptic Torch is only capable of signifying the presence of objects, the Falcon could be used to reach out and feel their shape, and this immediate physical stimuli will assist the users construction of a mental map of the virtual space.

Of the suppliers that Novint list as selling the Falcon, Fry's looks like the best for delivery outside the USA and Canada, although it should be noted that in the Gamasutra podcast mentioned earlier, Novint's CEO, Tom Anderson, mentioned that they are providing Falcons to game development studios for free and they have also used the Falcon as an interface for medical dental simulators with the Harvard School of Dental Medicine. With their aggressive PR policy perhaps they'd extend this generous offer to other academic research projects too?


Fry's
"Through DHL, we quickly ship international orders just about anywhere in the world for very reasonable rates."

SkyMall
"Many SkyMall products are available for delivery outside the United States."

TigerDirect
"For all international orders, export, and distribution please contact our sales force at: 800-800-8300"

CircuitCity
"Due to our manufacturer distribution agreements, we are not permitted to ship products to international addresses except APO/FPO or U.S. Territory addresses. Circuit City does not ship to Puerto Rico."

GoGamer
"GoGamer does not ship to International Destinations at this time."

JR
"At the present time, we only ship to the Continental U.S., Alaska, Hawaii, U.S. territories, Puerto Rico, and Canada. J&R proudly ships to our Armed Forces APO/FPO customers."