Cubby
Volume Number: 16
Issue Number: 9
Column Tag: QuickDraw 3D Tricks
Cubby: Multiscreen Desktop VR Part I
by Tom Djajadiningrat and Maarten Gribnau
Multiple views and mirroring images in QuickDraw 3D
Summary
In a series of three articles we describe how to implement the visualization part of a
Cubby, a desktop virtual reality system which uses three orthogonally placed
head-tracked screens. This series will give you an understanding of how this type of
three dimensional display works and show you how easy it is to implement using
off-the-shelf components. To facilitate implementation we use Apple's QuickDraw3D
API. Even if you are not interested in virtual reality display technology, you may still
be interested in the QuickDraw 3D techniques presented here. The most important ones
are multiple views on a single model and the mirroring of images without the use of
offscreen GWorlds.
Cubby is a desktop virtual reality system developed at Delft University of Technology
(Djajadiningrat et al, 1997; Djajadiningrat, 1998). Cubby uses three orthogonally
placed head-tracked screens which form a cubic display space (Figure 1). Through the
coupling of the perspectives on the screens to the head-movements of the user in
real-time, the illusion is created that a virtual scene stands inside the display space.
Figure 2 shows a user in front of Cubby. Figure 3 shows a chair inside Cubby's display
space from four perspectives as generated by four different head positions. Because of
the way the screens are placed, Cubby allows the virtual scene to be viewed from a
wide range of visual angles (see movie 'visualization.mov'). And since the virtual
scene appears in front of rather than behind the screens, the user can get at the
objects in the virtual scene with an instrument without the screens forming an
obstruction. This makes it possible to manipulate objects by means of an instrument at
the place where they appear (see movie 'manipulation.mov'). The 3D impression that
Cubby creates is based purely on head-tracking. It does not use stereo though of course
this could be added.
Figure 1. The Cubby setup.
Figure 2. A user in front of Cubby.
Figure 3. Four perspectives on a desk chair as generated by different head positions.
Technically, Cubby is similar to a CAVE (Cruz-Neira et al., 1993). A CAVE is a
virtual reality environment in which the walls and þoor of a room form
projection screens. A CAVE measures approximately 3x3x3 metres, while Cubby's
display space is only 0.2x0.2x0.2 metres. Thus from a technical point of view you can
think of Cubby as a miniature CAVE. From an application point of view, however,
Cubby and CAVE are quite different. With a CAVE, the user is inside the cubic space
while with a Cubby the user is outside it. This gives each system its pros and cons.
With a CAVE, the virtual scene is all around the user. It is therefore well-suited to
panoramic viewing and walkthrough or rollercoaster types of simulation. Because of
its large size it is difÞcult to realize accurate head and hand tracking with the
currently available tracking technology. With a Cubby the user looks upon the virtual
scene as if it were an object. Cubby's workspace is much smaller than CAVE's, but
because of this, it is possible to realize accurate tracking of head-position and
instruments. Cubby is therefore well-suited to precision tasks such as surgical
simulation and computer aided design. Other advantages of Cubby are that it consumes
little space, is relatively low-cost, and can be built using consumer grade
off-the-shelf technology.
This article describes how to implement the visualization part of a Cubby in
QuickDraw 3D. It will give you a grasp of the technology behind multiple screen
head-tracked displays such as Cubby and CAVE. Even if you are not interested in such
technology, you may still be interested in the QuickDraw 3D techniques presented
here. They are multiple views on a single model and the mirroring of images without
the use of offscreen GWorlds. We also include a section troubleshooting, to help you get
Cubby running, and a section Tidbits, with suggestions to further improve Cubby.
Cubby on Power Macintosh
What you should know
We assume that you are familiar with the basics of QuickDraw 3D programming. If you
have not dealt with QuickDraw 3D before we suggest that you have a look at the
introduction to QuickDraw 3D in Develop 22 (Thompson and Fernicola, 1995) or at
chapter nine 'QuickDraw 3D' of 'Tricks of the Mac Game Programming Gurus'
(Greenstone, 1995).
We also assume that you are familiar with Part I and II of 'Desktop VR using
QuickDraw 3D', two MacTech articles that appeared in July and August of 1998
(Djajadiningrat and Gribnau, 1998; Gribnau and Djajadiningrat, 1998).
Required hardware and software
To try out the QuickDraw 3D techniques in this article you need a PowerMacintosh
with an accelerated 3D graphics board and QuickDraw 3D 1.5.4 or better.
If you wish to build an actual Cubby you need additional electronic hardware such as a
head-tracker, three projectors and possibly extra graphics boards and scan
converters. As a head-tracker we use a Dynasight infra-red tracker by Origin
Instruments. With regard to projectors, graphics boards and scan converters, your
exact needs depend on which conÞguration you choose. We will discuss possible
conÞgurations in a minute. Of course, a Cubby consists of more than
electronics and computer hardware alone. You also need to build a physical setup to
position the projectors relative to the screens. For quick experimentation you can
build the display space using cardboard and tracing paper or drafting foil and mount
the projectors on tripods providing they are not too big. Figure 4 shows what our first
setup looked like. For a more permanent and robust setup you need to build the display
space from 4-5mm thick projection material (available from professional
photography labs) and mount the projectors on a table (Figure 5).
Figure 4. Our preliminary setup with a display space built from foamboard and
drafting foil. The projectors are mounted on tripods.
Figure 5. A more robust setup. This table makes it possible to accurately line up
projectors and screens.
Possible hardware configurations
Your Mac needs to generate three images, one per projection screen. You can these
images to the three projectors in several ways. You can either work with one graphics
board or three graphics boards and with either computer projectors or video
projectors (A computer projector is a projector that can directly accept the VGA
output of the graphics board of your Mac; a video projector is one which only accepts a
composite or S-video signal such as produced by your home video recorder. A
projector which accepts VGA usually accepts S-video too, but not all video projectors
accept VGA input). This leads to four possible conÞgurations (Figure 6).
Figure 6. A 2x2 matrix leading to four different configurations.
Figure 7. The four different configuration a-d (top to bottom)
The simplest conÞguration is to work with a single graphics board, use a
multiplier to split the signal of this board into three identical signals, and feed each of
these identical signals to a computer beamer (Figure 7a). Each projector is placed in
such a way relative to its screen that the relevant part of the image appears on the
screen, while the two remaining images are simply discarded by being projected off
screen.
The second way is similar to the Þrst, with the difference that we use a scan
converter to convert the signal of the graphics board into a video signal, which is then
split up by a video multiplier and fed to three video projectors (Figure 7b). While
this results in lesser image quality it is likely to be less expensive. The extra costs for
the scan converter are less than the cost you save by using consumer grade video
projectors instead of computer beamers.
The disadvantage of using a single graphics board is that you do not make full use of the
resolution of the projectors as approximately three quarters of the display area is
discarded. A way to overcome this is to use three graphics boards instead of just one.
This leads to a setup in which there is one graphics board per projector. Consequently,
no splitter box is necessary. (Figure 7c). Of course, this assumes that your Mac has
enough PCI slots free to add extra graphics boards.
The last conÞguration is similar to the third, the difference being that it uses
video projectors rather than computer projectors (Figure 7d).
As always, there are clever ways to cut costs. For example, you can avoid the extra
cost of one or more scan converters by using graphics boards which have both an
output for a conventional monitor and a S-video output. Some desktop Macs and
PowerBooks even have S-video outputs as part of their standard conÞguration.
A Look at the Cubby App
Before we dive into an explanation of the computer graphics behind Cubby, let's run
the application to see what we are aiming for. Don't worry about connecting a
head-tracker to the Mac, for the moment we run under mouse control. Start up the
application 'Cubby' and open the model 'espresso.3df'. On screen you see an L-shape
with an espresso pot (Figure 8). The L-shape comprises three QuickDraw 3D panes
which are placed within a single window which covers the whole screen except for the
menu bar (The advantage of using three panes within a single window instead of one
window per view is that that the single window acts as a black backdrop to the panes.
This prevents the Macintosh desktop from appearing at the outer edges of Cubby's
screens). While the espressopot was loaded from disk, the background planes with the
marbled texture form part of the Cubby application.
Now try moving the mouse. On the conventional 2D-monitor that you are using now,
the three perspective views in the L-shape look strangely distorted. When the views
are folded into a cube, as in Cubby's display space, and coupled to the user's
head-position the espresso pot appears to stand within Cubby's space.
Choose Mirrored from the Images menu. This mirrors each image horizontally (Figure
9). By looking at the frames per second counter in the statusbar and choosing Normal
and Mirrored repeatedly you can see how mirroring does not impose much of a speed
penalty.
Figure 8. A screen shot of the Cubby application (non-mirrored).
Figure 9. A screen shot of the Cubby application (mirrored).
Multiple Head-Tracked Displays
The easiest way to think of Cubby is as three head-tracked displays. Each of the
back-projection screens is a head-tracked display. For an extensive discussion of
head-tracked displays please refer to MacTech July 1998 (Djajadiningrat & Gribnau,
1998). In that issue we explained that for a head-tracked display an off-axis rather
than an on-axis camera is needed. In QuickDraw 3D speak this means a view plane
camera rather than a aspect ratio camera. For Cubby we use three QuickDraw 3D
views each of which has its own view plane camera. Each of the Figures 10 to 12 shows
one view plane camera, its coordinates, its viewing pyramid and its line of sight. The
point at which Cubby's three screens meet is placed at the origin of the world