Chapter One: The Mechanics Of Virtual Reality


'I considered the encounter between human and machine as the central drama of our time. The connection between them was a fundamental part of the human condition....it seemed that there were two ingredients: the computer and the person. One was evolving faster than any technology in history; the other was not evolving at all. Whereas the user interface had always been a veneer applied to the computer to make it slightly easier to use, it seemed obvious that the ultimate interface would be to the human body and the human senses.' (1)

As VR researcher Myron Kreuger implies above, the ultimate interface would be to all the senses, not just sound and vision, as in the VR game system I initially tried. One of many pioneering researchers of the period, he began work in the late 1960s with the emphasis on vision, using small cathode ray screens placed in front of each eye, onto which could be beamed the graphical representation of a room, creating a stereoscopic effect, but by 1972 he had drafted a manuscript, 'Artificial Reality', describing "data goggles, a data glove, and a data suit with force feedback" -- extending an environmental effect to actual physical interaction. In essence, the concept has existed and been explored for some time before the necessary technology has caught up. Now, there are tangible versions of VR readily available; developments in technology and the politics of its application have made it possible for these concepts to be implemented as a commercially available product.

To provide a truly convincing virtual reality, the five senses must ideally be confronted with detailed information about this environment. Similarly information about the participant must be effectively obtained.

Figure 2. (overleaf) illustrates the essential VR principle.

FIG.2 THE CYBERNETIC FEEDBACK LOOP(2)

The 'puppet' monitors the physical, 'real' world through an assortment of sensors, and acts on the physical world through various effectors; or output devices. Sensors might include tracking devices to monitor a persons spatial position and orientation, keyboards, joysticks, steering wheels, pressure gauges, voice recognisers, etc. - in fact any device capable of picking up information from/about its user. Conversely, the effectors transmit information back to the user - they include various forms of graphic and video display, sound generators, resistance controllers, motion platforms, force feedback devices, etc. The puppet gives the patron a virtual body in so-called cyberspace; and the patron gives the puppet a personality. The more sophisticated the arrangements of sensors and effectors, so the more sophisticated the interaction between the user and the virtual world.

My first hands-on experience of so-called Virtual Reality came during Summer of this year, when I tried the 'Virtuality' system installed in the Trocadero Centre, Piccadilly, vaunted as a national first. 'Virtuality' comes courtesy of a Loughborough based firm, W Industries, who have commercially developed what they claim is the world's first video arcade game based on VR....

The version I try is equipped with a car racing game. Along with 5 other players, I am helped into one of the £20 000 devices, that resembles one of the 'sit-in' arcade machines found at seaside amusement parks, etc. and don the headset. ( See Figs. 3,4 )

Padded with foam inside, the headset is connected to the machine by thick plastic cables, that feed me with sound and vision. The twin LCD screens give a respectable 3d representation of the cockpit I am sitting in, except that through the windscreen is a racetrack. Electromagnetic sensors in the helmet record the position of my head, so as I turn my head from side to side, my field of vision alters accordingly. I manually operate what are recognisably vehicle controls - steering wheel, accelerator pedal, etc. I am, in effect, driving the car.

Whilst there is undeniably a sense of environment - I find myself involuntarily flinching when I deliberately crash my virtual vehicle - I am unavoidably disappointed.

I have to ask myself what I actually wanted. Perhaps I have been seduced by all the hype, and the £2 fee into thinking I am about to have a mind-blowing experience.

- It is good, but not overwhelmingly so.
- The sound is primitive, and annoying in a home-computer sense
- The graphics are flickery and of relatively low resolution.
- There is no sensory feedback - when I crash, I feel no jolt.

I am very conscious of sitting with a foolish-looking helmet on rather than sitting in a speeding vehicle on a racetrack. Nevertheless, it is doubtless sublime within machine arcade criteria. I remove the rather clammy headset, and step out of the machine. I keep reminding myself that the shortcomings I have noted with this Virtuality system are those of sheer sophistication of hardware; it just needs faster processing, greater memory, better interfacing and improved graphic resolution, all of which will unquestionably improve with time as computer manufacturers begin to vie with each other over product performance. The concept is sound. Within its limitations it does actually work; I was able to function as though I was actually there, wherever 'there' might have been. The supervisor who helps players in and out of the machines told me the game was attracting considerable attention; certainly a lot of money looked to be made......
.....but if I were to be spending mine in future, I really would want to be immersed.

Despite the sublime electronics running the VR process, it is the physical interface hardware on which the success of VR will stand. How faithfully will the hardware be able to reproduce the experiences purporting to be reality?

Vision

From Victorian zoetropes, the animated image has evolved through various mediums: the moving series of celluloid images at 24 frames per second known as cinema; cathode ray-tube TV, next, thanks to the intriguing shapes and exotic behaviour of liquid crystals, and demand for portable 'lap-top' computers, has become a flat gas plasma/LCD screen small enough to be worn in front of the eyes to give a stereoptic effect, and still the R&D continues. I was not greatly enamoured of my experiences in commercially available head-mounted displays. Howard Rheingold discusses the latest research...

'...perhaps we could create a means of immersive 3d without putting a head-mounted display on. What we're seeing here is a 3d video telephone...(Fig. 5a) ..I mentioned that you need two images, one for the left eye and one for the right eye......what they do is to take those two images and cut them into very thin vertical slices, and they take those slices and they interleave them, then they put a lense over it...it's called a lenticular lense and it has half-cylinders, so when you look at it the left eye sees the left eye view; the right eye sees the right eye view, and they use an LCD liquid crystal display screen, the kind you see on tiny pocket televisions....very realistic. It's not immersive, you're not surrounded by it, but that's because the screen is tiny. What they're working on now are 3'x3' screens that they tile together, and they're building a room where all the walls, and the floor, are screens, so you could walk into this room, and, depending on what computer model, the Amazon jungle, the middle of a big city, the human body, or outer space...' (3)

Even from description, the idea of an entire room becoming screen becoming environment is appealling, especially when combined with the 3D effect. (Fig. 5b). I had wondered why there seemed to have been little interest in holographics; in fact I did not recall coming across any mention of the hologram during the course of my research, despite the obvious striving for realism in three dimensions VR is concerned with, until I saw product at CG91, the annual computer graphics trade show. Here, was evidence of such lenticular lenses, which offered a peculiar but superior effect to a conventional flat screen. Again, technology is doing its leapfrog act :

'....other attempts to create 3d imaging technology are just beginning at the University of Washington, in the Human Interface Technology lab.....a fellow who ran the USAF head-mounted display project for twenty years has moved to civilian applications, and he has a strong belief that they can create colour laser scanners that will scan a 3D image directly onto the human retinae from a distance, and create a 3D image. If they can do that, not damaging the eye of course, then they can put several 10s of millions of picture elements on the eye. At a certain point, when you have enough picture elements, it's going to appear real. Now nobody knows how many that is...a fellow well-known in the computer graphics world, Alvey Ray Smith, is quoted as saying 'reality is 50 000 000 polygons per second.'

Whatever that number is, if you think about the way cinema developed, if you take a series of still photographs and show them one after another, you get a series of still photographs, but when you show them at 24 frames per second, it looks like real motion, so the idea is as we improve this method there is some level of resolution at which, at least visually, you can fool people, it will look real. Nobody really knows yet what that level is or whether we can achieve it.' (4)

Sound

The Virtuality system's sound effects were disappointing. A shame; 'perfect' sound reproduction is a goal that audio researchers and manufacturers have been striving toward for many years, and are now capable of achieving impressive results. Quadraphonic hi-fi failed through practical rather than technical reasons - it was not successfully turned into a market - viable alternative to stereo, with insufficient recorded product available to play. Similarly, binaural, or dummy head recording, which as the name implies, utilises two microphones placed appropriately within the ear cavities of a facsimile human head to record sound within a studio, whilst more effective than normal stereo, stumbled over its being only effective when played back through headphones.

'Humans hear spatially....the separation and shape of the ears enables us to determine the location of a sound event from all directions... computer-generated spatial sound is feasible with current technology with some minor compromises.The computational expense of generating echo distance cues remains a limiting factor in 3D sound realism, however, this problem goes away as speeds increase and hardware becomes more accessible' (5)

The music hardware company Roland this year launched the RSS 3D Sound Processor (Fig.6). The technical details demand a dissertation in their own right, suffice to say that by exploiting effects that occur in nature - doppler distortion; the relative incidence of sound arriving at ears a calculable distance apart, and frequency fall-off, a sound system has been developed that operates feasibly with just two loudspeakers, though it currently commands a price of £25 000. Demonstrated on 'Tomorrow's World' with a simultaneous FM broadcast, despite less than ideal circumstances the 3D effect was clearly discernible, although listening to the recording now, 'phasing' is evident.

'......the best way I can think of describing the sensation is like sitting inside an acoustic hologram.....the effect is certainly dramatic.....I can see great opportunities in the areas of TV soundtracks for Nicam stereo and even video-cassette soundtracks..... the end user needs no additional equipment....' (6)

'With just virtual audio alone, you can feel breath on your face, your hair being pulled.... I've fooled myself. This morning, when I was checking the headphones, I heard a knock and the door opening. I turned round and it was shut. I made the tape, I should know. It fooled me '

- Chris Currell, LA musician with special interest in 3D sound (7)

(Fig. 7)

So it seems that Virtual Reality, limited to the sense of sound and vision, has been effectively realised, and will steadily improve as time goes by. Current major limitations reside with physical; tactile, sensation, and smell and taste...

References

(1) Kreuger, Myron, 'Artificial Reality : Past & Future', in 'Virtual Reality', Meckler Publ., 1991, p.19
(2) Walser, Randal, 'The Emerging Technology Of Cyberspace', in 'Virtual Reality', Meckler Publ., 1991, p.36
(3) Howard Rheingold; from transcript of conversation; 28th October 1991
(4) Howard Rheingold; from transcript of conversation; 28th October 1991
(5) Karr, B, 'Virtual Reality: Directions of Growth', Meckler Publ.., 1991 p.10
(6) White, Paul, 'Roland RSS', in 'Home & Studio Recording', May 1991, pp. pp.8-10
(7) Sirius, R.U, 'Virtual Audio', in MONDO 2000, issue 4,1991, p. 23