JOURNAL OF DISPLAY TECHNOLOGY

Volumetric 3D Display for Radiation Therapy

Planning
Jason Geng, Senior Member, IEEE
(Invited Paper)

Index TermsInteractive visualization, medical imaging, radiation therapy planning, volumetric 3D display.

IE
W EE
eb P
Ve ro
o
rs f
ion

AbstractIn current clinical practice, radiation therapy planning (RTP) has often been treated as a two-dimensional (2D)
problem, mainly due to the limitations in visualization technology
available to date. The slice-by-slice display format makes it difficult to visualize the path of radiation beam not perpendicular
to the axis of the CT slices. This discourages consideration of
treatment plans that utilize radiation beam out of the transverse
plane. Human body anatomical structures are inherently three-dimensional (3D) objects, and tumors and tissues/organs involved in
the RTP are all of 3D shapes. A clear understanding of 3D spatial
relationships among these structures, as well as the anatomic
impact of 3D dose distributions, is essential for designing and
evaluating radiation therapy plans.
We have recently made an important breakthrough in the highresolution volumetric 3D display technology and have made an initial attempt to apply it to RTP applications. By volumetric 3D
display, we mean that each voxel in the displayed 3D images
) spatial position where it is
is located physically at the (
supposed to be, and emits light from that position to form real
3D images in the eyes of viewers. We have demonstrated the feasibility of our system design by building full-scale prototypes and
achieved a multi-color, large display volume, true volumetric 3D
display system with a high resolution of over 10 million voxels in
a portable design. This type of true 3D display system is able to
present a 3D image of a patients anatomy with transparent skin,
providing both physiological and psychological depth cues to oncologists in perceiving and manipulating radiation beam configuration in true 3D fashion, thus offering a unique visualization tool
to ensure the safety, effectiveness, and speed of the RTP process.
The volumetric 3D display technology holds promise to significantly enhance the accuracy, safety, and speed of RTP procedures.
Such an understanding at a glance capability is necessary to keep
the clinicians from becoming bogged down in details, as he/she
would be if provided only with conventional 2D display of CT slices
with overlaid isodose lines.
The main focus of this paper is to provide technical details on
the volumetric 3D display system we developed, and present some
initial results on its capability of displaying true 3D images. While
the system design framework of applying such technology into RTP
is introduced, its full scale clinical applications to RTP is still an
ongoing effort and will be reported later in other publications.
Manuscript received January 8, 2007; revised February 21, 2008. This work
was supported in part by the National Institutes of Health under Grant R44
CA80577-02A1, by the Department of Energy DE-FG02-98ER82588, by the
U.S. Air Force F08635-97-C-0034, by DARPA DAAH01-97-C-R169, BMDO
DASC60-98-C-0018, by the National Science Foundation DMI-0124322, and
by NASA NAS13-01039. The content of this document does not necessarily
reflect the position or the policy of the sponsors, and no official endorsement
should be inferred.
The author resides in Rockville, MD 20852 USA (e-mail: jason.geng@ieee.
org).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JDT.2008.922413

I. INTRODUCTION

HIS paper documents our theoretical study and experimental demonstration of a revolutionary volumetric threedimensional (3D) display technique. We also present a framework for applying this true 3D display technology to radiation
therapy planning (RTP). Although we are still in the early stage
of development, the ultimate goal of this investigation is to develop a clinically viable volumetric 3D display technology for
medical image visualization in general.
The proposed volumetric life-like 3D image display technique relies upon a display media that is a true 3D volume instead of a 2D flat screen. Each volume element (called voxel,
analogous to a pixel in a 2D image) in the displayed 3D im) spatial position where it
ages locates physically at the (
is supposed to be and emits light from that position to form real
3D images in the eyes of viewers.
The volumetric 3D display we developed is fundamentally
different from conventional 3D rendering visualization technique, where the object is displayed on a 2D flat screen with
3D rendering for depth perception. It is also different from 3D
stereo video or head-mounted display (HMD), where the 3D
perception is created with a pair of polarized glasses or display
screens. The volumetric 3D display technology projects 3D images directly into true 3D space that does not require special
3D glasses to view it. Viewers can walk around the 3D image
and look at it from all different directions with realistic depth
just as looking at the real physical object. Such 3D display provides both physiological and psychological depth cues to human
viewers for truthfully perceiving objects in 3D space.
Furthermore, with realistic 3D representations of medical images in many imaging modalities (CT, MRI, PET, Ultrasound,
etc), viewers can interact with the life-sized volumetric 3D images being displayed, via handheld pointer and/or other userinterface devices, as if the true 3D virtual patient were there with
a transparent skin and visible internal anatomic structures. The
unique capabilities of walk-around viewing and direct interaction with the displayed 3D images could greatly simplify our
understanding of the complexity of 3D objects and spatial relationship among them.
We have recently made an important technical breakthrough
in implementing the high-resolution volumetric 3D display.
Using the spatial light modulator (SLM), high power visible
lasers and precision fabrication of helical screen, we have

1551-319X/$25.00 2008 IEEE

JOURNAL OF DISPLAY TECHNOLOGY

moving parts. Major difficulties to produce a practically useful

3D display using this approach are its scale-up capability and
ability to display multiple colors.
B. Gas Medium Up-Conversion
Another 3D display based on the up-conversion concept employs the intersection of two laser beams in an atomic vapor,
and subsequent omnidirectional florescence from the intersection point (U.S. Patent 4 881 068 by Korevaar, 1989). Two lasers
are directed via mirrors and - scanners towards an enclosure
containing an appropriate gaseous species (rubidium vapor, for
example), where they intersect at 90 deg. By itself, either laser
causes no visible fluorescence. However, where both lasers are
incident on the same gas atoms, two step excitation results in
florescence at the intersecting point. By scanning the intersection point fast enough, a 3D image can be drawn in the vapor.
The eye cannot see changes faster than about 15 Hz, so that if
the image to be displayed, it is repeatedly drawn faster than this
rate; the image will appear to be steady, even though light may
be originating from any one point in the volume from only a
small fraction of the time.
The advantage of this 3D display concept is its scalability: It
can be built in almost any desirable size without significantly
increasing the complexity of the system. The technical difficulties in implementing this concept including the requirement of
vacuums chamber, requirement for maintaining certain temperature, limitation of number of voxels by the speed of the scanners, and eye-safe problem of laser beams.

IE
W EE
eb P
Ve ro
o
rs f
ion

achieved a multi-color, large display volume, true lifelike 3D

display system with a high resolution of over 10 million voxels
in a portable design. We have demonstrated our high-resolution
volumetric 3D display concept by building a full-scale prototype that can display complex 3D images.
The volumetric 3D display is a revolutionary concept for RTP
and medical image visualization in general. The major innovations of our approach are twofold.
1) We have developed a functional hardware platform and
a high-speed data interface that enables a high-resolution
volumetric 3D image with over 10 million voxels to be
displayed at a rate of 20 frames per second. We have designed, fabricated, and tested all hardware components and
software package associated with the display prototype
system. The dynamic volumetric 3D display capability not
only makes the interactive RTP possible, but also opens
doors to many new applications in medical image visualization arena.
2) We have developed a novel conceptual framework of
the Interactive RTP Environment, and built a set of
prototype hardware/software. The Interactive RTP Environment enables direct interactions between the displayed
volumetric 3D image of patient anatomic structure and
tumor, and the simulated treatment beam configuration.
The intuitive interactions help radiation therapy planners
determine suitable beam directions and parameters to
maximize the tumor coverage and minimize the exposure
of normal tissues during a planning session.
II. BRIEF SURVEY OF PRIOR ART ON VOLUMETRIC
3D DISPLAY TECHNIQUES

In this section, we provide a brief survey of a number of 3D

volumetric display techniques that have been intensively developed in the past.
A. Solid-State Up-Conversion

One of the fundamental requirements for a volumetric 3D display system is to have entire display volume filled with materials that can be selectively excited at any desired locations. To
achieve this goal, one can have two independently controlled radiation beams which activate a voxel only when they intersect.
While an electron beam cannot be used for such purpose, a laser
beam can, provided that a suitable material of display medium
can be found. A process known as two-photon up-conversion
can achieve this objective (U.S. Patent 4 041 476 by Swainson,
1977, U.S. Patent 5 684 621 by Downing, 1997). Briefly, this
process uses the energy of two infrared (IR) photons to pump
a material into an excited level, from which it can make a visible fluoresce transition to a low level. For this process to be
useful as a display medium it must exhibit two-photon absorption from two different wavelengths so that a voxel is turned
on only at the intersection of two independently scanned laser
sources. The materials of choice at the present time are the rare
earths doped into a glass host known as ZBLAN. ZBLAN is
a flurozirconate glass whose chemical name stands for ZrF4BaF2-LaF3-AlF3-NaF. The two-photon up-conversion concept
for 3D volumetric display is quite promising, since it requires no

C. Rotating Light Emitting Diodes (LEDs) Array

One of the earliest volumetric 3D displays was designed

by Schipper in 1963 (U.S. Patent 3 097 261). It consists of a
rotating electroluminenscent panel with embedded high-speed
light emitter array. By controlling the timing of - addressing
of the light emitter array and the rotation of the panel, 3D
images can be formed within the volume swept by the rotating
panel. In 1979, Berlin developed an innovative approach to
solving the high-bandwidth data transmission problem of this
design using optical link and replaced the light emitters with
high speed LED matrix (U.S. Patent 4 160 973 by Berlin, 1979).
This system uses LED arrays that are rotated to sweep out a 3D
volume. The resolution of this volume is a function of number
and density of LEDs mounted on the rotating planar array, the
speed of rotation and the rate at which the LED can be pulsed.
D. Cathode-Ray Sphere

The Cathode Ray Sphere (CRS) concept was originally developed by Ketchpel in 1960s (U.S. Patent 3 140 415 by Ketchpel,
1960) and recently implemented by researchers at New Zealand
(US Patent 5 703 606 by Blundell, 1997). The voxels are created
by addressing a rapidly rotating phosphor-coated target screen
in vacuum by electron beams synchronized to the screens rotation. The view of this rotating multi-planar surface depends
on the clarity of the glass enclosure and the translucency of the
rotating screen. Another image quality issue is the interaction
between the phosphor decay ray and the speed of the rotation of
the screen.

IE
W EE
eb P
Ve ro
o
rs f
ion

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

Fig. 1. A brief survey of various 3D display technologies.

E. Varifocal Mirror and High Speed Monitor

A very clever method of 3D display employs the strategy of

forming optical virtual 3D images in space in front of viewer
(U.S. Patent 4 130 832 by Sher, 1978). The varifocal mirror
system consists of a vibrating circular mirror along with a
high-speed monitor. The monitor is connected to a woofer
such that the woofer can be synchronized to the monitor. A
flexible, circular mirror is attached to the front of the woofer,
and the monitor is pointed toward the mirror. With the vibrations from the woofer, the mirror changes focal length and the
different points being displayed on the monitor seem to appear
at different physical locations in space, giving the appearance
of different depths to different objects in the scene being displayed. Variable mirror based 3D display systems are primarily
limited by the size of the mirror and updating rate of images,
since this mirror has to vibrate.
F. Laser Scanning Rotating Helix 3D Display

Extensive attempts have been made by researchers at Texas

Instruments Incorporated (US Patent 5 042 909, 5 162 787, by
Garcia, 1991) to develop a 3D display device based on laser
scanning and rotating (helical) surface. Lasers scanning 3D displays operate by deflecting a beam of coherent light generated
by a laser to a rotating helical surface. Timing modulation of the
laser beam controls the height of the light spot that is produced
by the laser on the rotating surface. The deflectors include devices such as polygonal mirrors, galvanometer, acousto- optics
modulated deflectors, and micro-deformable mirrors. There are
several problems with this 3D display mechanism that have prevented it from becoming commercially feasible.
The most serious problem is the limitation on the maximum
number of voxels that can be displayed. Due to the nature of
sequential (non-parallel) laser scanning, only one spot of light
can be displayed at any given moment. All the activated image
voxels have to be addressed, one by one, by the scanning of
single laser beam in time-multiplex fashion. The time needed
for steering the laser beam and to stay on the voxel position to
produce sufficient brightness poses an upper limit to how many
voxels it can display. To increase the number of voxels, multiple channel lasers and scanners could be used. However, many

attempts to increase the spatial resolution have hampered with

high cost and bulky hardware design.
Fig. 1 summarizes various research and development efforts
on 3D display. Recently, there is a surge of research activities
on volumetric 3D display that promise to bring high resolution
( 100 million voxels) display into reality [1][11].
III. VOLUMEVIEWER 3D DISPLAY CONCEPTP

In this section, we provide detailed technical discussions on

the VolumeViewer 3D display design and its implementation.
A. Principle of the Multi-Planar Volumetric 3D Display

Fig. 2 illustrates the principle of the Multi-planar volumetric 3D image display using a high-speed 2D image projector
and a moving screen. Suppose that a sweeping screen can be
controlled to move back and forth along the direction at a
frequency higher than 20 Hz. Within the time period of each
sweeping motion, frames of 2D image patterns are projected
by the high-speed 2D image projector. The moving screen intercepts 2D image projections at different positions along axis,
forming a stack of spatial image layers in true 3D space. If the
cycling speed of the moving screen is sufficiently high, and the
2D image projector can produce sufficient number of 2D image
sections during each pass, human observers are able to perceive
a true volumetric 3D image floating in the 3D space without
flicker, due to the residual effect of human eyes.
The multi-planar volumetric 3D display principle is by no
means a complex concept. However, implementation has been
difficult due to lack of suitable high-speed image projector,
clever mechanism to produce sweeping screen motion, and
high brightness light sources. There has been a number of
attempts been made to build such cumbersome system without
success. A physically flat screen sweeping at 20 Hz creates
serious problems of mechanical design, balance, vibration, and
noise. Conventional liquid crystal projector can only achieve
a switching rate of few hundred hertz, leading to a very low
spatial resolution. High power light source has been very
expensive and cumbersome. All these factors contribute to
a slow progress of volumetric 3D display techniques using
multi-planar principle.

JOURNAL OF DISPLAY TECHNOLOGY

IE
W EE
eb P
Ve ro
o
rs f
ion

Fig. 2. Principle of the multi-planar volumetric 3D display using fast 2D projection and a moving screen.

Fig. 3. Volumetric 3D display concept using a fast SLM and a rotating helix screen.

B. Concept of the VolumeViewer 3D Display

We propose a new generation of the multi-planar volumetric

3D display system, taking advantages of rapid advances in material, laser, and semiconductor fabrication technologies. With the
newly developed ferrorelectric liquid crystal spatial light modulator (SLM) technology, the switching speed of SLM reaches
over 3000 fps for a SLM of 256 by 256 pixels or higher resolutions. Such a fast SLM can be used as a high-speed image pattern generator to produce volumetric 3D pictures, providing a
powerful tool to revolutionize the state-of-the-art of 3D display.

Fig. 3 illustrates a design concept of our SLM/helix volumetric

3D display.
In Fig. 3, light rays produced by a light source projector ,
passing through filter and collimating lenses , impinge on a
polarizing beam splitter cube . Due to the polarization characteristics of the beam splitter, the polarized light rays are reflected by the beam splitter and projected onto a SLM . The
image data shown on the SLM is generated by a host computer
. The SLM is able to alternate image patterns at high frame
rate (i.e., over 3000 fps). When a pixel on SLM is turned ON,
the light will be reflected back to the beamsplitter cube, while

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

when the pixel is turned OFF, the projected light on this pixel
will be absorbed by the SLM and will not be reflected. The patterns on the SLM are therefore able to control the patterns of the
reflected light rays. The reflected light rays with encoded SLM
image patterns transmit through the beam splitter cube.
is employed to project
An optical projection lens system
the image patterns towards a spinning helix screen, marked as
. The light spots projected on the helix screen intersect the
helix surface at different heights depending on different rotating
angles of the helix, thus form 3D voxels in 3D space (the display
volume ). Each section of the helix surface is described by the
following mathematical equations:

Fig. 4. Prototype of the VolumeViewer 3D display.

IE
W EE
eb P
Ve ro
o
rs f
ion

TABLE I
PERFORMANCE OF THE VOLUMETRIC 3D DISPLAY PROTOTYPE.

If rotation is synchronized via a motor driving the helix

screen with the switching timing of the SLM , such that 3D
image patterns are shown in the 3D space with a high refresh
Hz), naked eyes perceive it as a 3D volumetric
rate (i.g.,
image. No special eyewear is required to view such 3D image
floating in true 3D space, just as a real object is placed there.

C. Advantages of the Proposed SLM/Helix 3D Display System

Inherent parallel architecture for voxel-addressing: Instead of using single laser beam to address all the voxels
(such as the NRaD scanning laser system) the SLM/Helix
system use 256 by 256 (or more) light rays to address
simultaneously voxels, thus overcomes the bottleneck
in producing high resolution 3D images encountered by
other approaches.
High Spatial Resolution: The maximum number of voxels
that can be generated by the SLM/Helix display depends
upon the spatial resolution of SLM and the spinning speed
of helix. With the currently available SLM technology, a
SLM with 1024 by 1024 pixel and 300 000 frames per
second switching speed is available. The resolution of proposed 3D display can take advantage of the rapid advances
of SLM technology.
Simple structure and easy to build: Other than the rotating
helix, there is no other scanning or moving part. The optical design and alignment are not difficult. The system can
be built using commercial off-the-shelf (COTS) products,
which leads to shorter development period and low cost.
No special viewing glasses or helmet are needed by
viewers: The volumetric images are displayed in true
3D space with almost 360 degree viewing angle, which
preserve all physiological and psychological depth cues of
human visual system. Viewers can walk freely around the
monitor to see the 3D images, just as if the real 3D object
were sitting there.
Implementation of full color display is straightforward:
Just use three SLMs for Red, Green, and Blue respectively,
and the color of voxels can be automatically controlled.
Another way to implement color display is even simpler:
use Red, Green, and Blue light projector, and synchronize
the timing of three projectors with a high speed SLM.

D. VolumeViewer 3D Display Prototype

Fig. 4 presents an overall system design configuration of

the newly designed and prototyped volumetric 3D display. We
dubbed this prototype system the VolumeViewer. Inside the
transparent hemispherical dome is a rotating helix forming a
3D image display volume of 7 height and 20 in diameter.
There are nine pieces of reflective mirrors with large dimensions of optic surfaces. If fabricated using conventional
thick glasses, these mirrors would be heavy-weighted, fragile,
and difficult to assembly with acceptable accuracy of optical
alignment. We adopted a state-of-the-art mirror fabrication
technology that forms large piece of flat mirror using framed
thin films with high reflectivity. These thin-film mirrors have
only 10% of weights as their glass mirror counterparts, and can
be built to fit various difficult geometric dimensions. Due to
their light weight and flexible dimensions, we can easily mount
them into optically aligned modules, thus saves us tremendous
effort in the final stage of the optical alignment in the system
integration. Table I lists major performance specification of the
VolumeViewer Prototype system.

JOURNAL OF DISPLAY TECHNOLOGY

Fig. 5. SLM structure.

cells arranged as a square of 256 by 256 array with total dimension of 5 5 mm approximately. The device achieves better
than 25% optical throughput when used with collimated laser
light and better than 100:1 contrast ratio when oriented for amplitude modulation. A better than 100:1 contrast ratio of SLM
provides a fairly good image quality. The device can be operated
as fast as 5 kHz with complete switching of the liquid crystal.

IE
W EE
eb P
Ve ro
o
rs f
ion

IV. DESIGN AND FABRICATION OF THE PCI INTERFACE BOARD

ALLOWING FOR DYNAMIC 3D IMAGE DISPLAY
A. Primary Objectives of the PCI Interface Board Design

Although our initial success in developing the original prototype system represented the state-of-the-art true volumetric 3D
display technology then in terms of achieving high spatial resolution, the updating rate of 3D images in original system was
still slow. To update a displayed 3D image into a new frame of
3D image, the host PC has to upload the data set of the new 3D
image to the SLM driver via a parallel port. This data transmission of a single frame of 3D image usually takes about 20 s, due
to the size of 3D dataset and the slow speed of the PC parallel
port. Such a low updating rate certainly prevents our current 3D
display system design from being used in many dynamic interactive 3D display applications, such as radiation therapy planning sessions.
Therefore, one of the main efforts of this investigation is to
design and fabricate a PCI interface board to eliminate the bottleneck of 3D image transmission between host PC and SLM
chip. Primary goals of this PCI interface board are:
1) to achieve 3D image updating at a rate up to 20 images per
second from host PC to SLM chip;
2) to increase the frame rate of 2D image displayed on the
SLM;
3) to allow for multiple color 3D display.
B. Spatial Light Modulator (SLM)

SLMs, devices that alter the temporal and spatial character of

a light beam, can be either optically or electrically addressed.
Optically addressed SLMs often require bulky support equipment and additional light sources. To obtain high frame rate of
image projection for volumetric 3D display, we propose to use
the electronically controllable, fast ferroelectric liquid crystal
(FLC) reflective spatial light modulator. The device is built atop
a planarized 0.6 m CMOS SRAM backplane with 15 m pixel
pitch and 87% fill factor. A thin layer of FLC material is sandwiched between a metal conductor and a glass window coated
with a transparent conductive layer such as indiumtinoxide
(ITO). When a voltage is applied across the FLC layer the fast
axis of the bi-refringent FLC material is forced into one of two
possible states: ON or OFF (the image on the SLM is binary).
The structure of a SLM is depicted in Fig. 5. It is a specially
designed integrated circuit housed in a 49-pin ceramic PGA
package. The effective area of the SLM consists of 65 536 FLC

C. Design the PCI Interface Board to Control the SLM Chip

Controlling the SLMs operation is very similar to addressing
a Static Random Access Memory (SRAM) chip. The interface
board contains an on-board microprocessor, memory for up to
512 frame 2D images, circuitry for controlling the SLM, and
circuitry for communicating with host PC computer. We use
C++ and VXD (a low-level assembly) software to manipulate
the image data and to transfer them into the image buffer on the
controller, which in turn sends the image sequence to the SLM
in a predetermined high frame rate. Fig. 6 illustrates the block
diagram of our Interface Board design.
1) Microprocessor: The size of 3D data sets is inherently
huge. In order to transfer huge amount of 3D data in higher
speed among the host computer, on-board image memory, and
the SLM chip, the microprocessor must have the high-speed
data transferring unit, such as DMA, Interrupt Unit etc. The microprocessor must also have a PCI interface and other control
units for connecting with PC and communicating with other
standard facilities. By careful design comparison, we selected
the Intel 80960RP as the CPU of the board. The 80960RP is a
PCI IO processor with 352 BGA pins. It has many units for data
communication. Its DMA Controller, Address Translation Unit
(ATU), Message Unit (MU), Memory Controller and other control units are suitable for the design, and it has an available and
completed embedded software system, so it fulfills the design
features.
2) Memory: We need a higher speed RAM on the board as a
buffer to store 3D images. In the design two memory groups are
used to display dynamic 3D images and make the other display
functions. By considering the speed, volume and stability, we
chose eight SRAM, MCM6246 chips, as the image memory on
the board and divided them into two groups, each has 2 Mbytes.
This image memory can implement all features described above.
3) FPGA (Field Programmable Gate Array): Due to complex operation of the PCI interface board, thousands of gates
and flip-flops are needed to fulfill the desired functions. Dozens
of buses with 32 bits have to switch each other, which is impossible to be laid on an area-limited printed circuit board directly.

IE
W EE
eb P
Ve ro
o
rs f
ion

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

Fig. 6. Block diagram of the PCI interface board.

D. Fabrication and Test the PCI Interface Board

Fig. 7. Block diagram of FPGA1.

Using the newly designed SLM device interface board, we

have achieved a maximum transmission rate of about 2200 fps
and the resolution of each frame of 2D images is 256 by 256
pixels. At the target 3D image refreshing rate of 7 images per
second, we are able to produce 157 frames of 2D images for
each 3D image cube. This enhanced speed of image transfer effectively increases the spatial resolution of our volumetric 3D
by
by
million voxels. In comdisplay to
parison to the use of original SLM drive unit, the maximum spaby
million voxels.
tial resolution is about by
Furthermore, the 3D image-refreshing rate was about 20 seconds per image versus the 20 images per second of current
system equipped with the new PCI interface board. The success of the PCI interface board allows us to perform dynamic
3D image display, and makes the application of our 3D display
technology to RTP practically possible.

E. Software Drive Development Using VXD Techniques

Fig. 8. Block diagram of FPGA2.

We employed advanced FPGA technology which allows for a

software programmable functionality on hardware chips. Two
Xilinx XCS40 (each with 40,000 gates) are used and their functions are illustrated in Figs. 7 and 8. Due to space limit, a total
of 24 complex circuit diagrams implemented by FPGAs cannot
be included. Fig. 9 gives an example of programmable functions
implemented by the FPGA1. Powerful FPGAs make it possible
for us to design a compact PCI board with the desired features.

The software for the board is divided into host computer programs and the 80960RP microprocessor programs. The host
computer programs include a Windows based work studio and a
VxD (Virtual Device Driver) program, as shown in Fig. 10. The
80960-based program includes the embedded programs stored
in PROM or the executable code downloaded from the host computer to the 80960RP program RAM. In fact the board can be
controlled from host computer and 80960RP.
When the host computer is turn on, BIOS of the computer
finds the PCI board, so that the Windows 95/98 can get the information from the Intel 80960RP. Then the model of (Operation
System) OS loads the GTI3DD.VxD into the computer memory.
During loading GTI3DD.VxD, OS communicates with the VxD
to decide the resources allocation. After all VxDs are loaded, OS
builds up a table to save the results.

JOURNAL OF DISPLAY TECHNOLOGY

IE
W EE
eb P
Ve ro
o
rs f
ion

Fig. 9. An example of detailed design diagram inside the FPGAs.

Fig. 10. Operation of the VxD for the PCI interface board.

V. SWITCHABLE DISPLAY VOLUMEA NEW OPTICAL DESIGN

ENABLING BOTH FULL AND HALF HELIX DISPLAY VOLUMES

Fig. 11. Comparison of previous and current 3D image projection schemes.

In our previous system design and experiments, we projected

the 3D images onto a small portion (less than one half of a helix,
see the left drawing in Fig. 11) of the sweeping volume produced
by the helix. A large portion of the useful volume produced by
the sweeping helix was wasted. In our latest effort, we have
significantly improved the system optical design to project the
3D images into the entire helix volume (see the right drawing
in Fig. 11), thus increasing the size of the display volume to
entire sweeping volume of the rotating helix 20 (508 mm) in
diameter and 7 (178 mm) in height.

A. Overall Optical Configuration Design and Tradeoff: Front

Projection Versus Rear Projection
There are two possible overall optic configurations for the
SLM/Helix system design: Front projection configuration
(FPC) versus rear projection configuration (RPC). Previous
3D display systems, such as NRaD 3D display, use the front
projection configuration. This means that laser beams are
projected onto the rotating helix surface in the same side as
viewers view the 3D images [Fig. 12(a)]. The front projection
configuration makes it easier to implement a driving system

IE
W EE
eb P
Ve ro
o
rs f
ion

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

Fig. 12. Comparison of overall optical configuration for the SLM/helix 3D display.

for the rotating helix. It also provides convenience in adjusting

optical system setups and modifying other components or
subsystems. However, the disadvantages of the front projection
configuration include the following.
(1) It leads naturally to an overhead projection configuration so a compact system design (as a portable desktop
display, for instance) is difficult to achieve.
(2) Viewers may possibly block the projection of laser
beams.
(3) Since system components are spread out, maintaining an
accurate optical alignment is difficult.
We have developed the rear projection system (RPS) design
for our SLM/Helix 3D display. In a RPS configuration, laser
beams are projected onto the rotating helix screen from below,
while viewers look 3D images from above [see Fig. 12(b)]. The
helix surface is made of semi-transparent material so it transmits
50% light and reflects 50% light. This rear-projection approach
eliminates the unwieldy overhead mirrors from the NRaDs design and allows the lasers, scanners, optics, and the helix to
be packaged together as a single compact mobile 3D display
unit. Since the helix surface transmits as well as reflects light in
omni-direction, the viewing angle of the voxels in a 3D image
is very large (almost true walk-around viewing angle and group
viewing capability).
A major advantage of using the RPS configuration is that it
is possible to achieve a compact system design. All the components of the SLM/Helix system can be packaged into a cabinet
with 3D display volume on the top. From the viewpoint of final
commercial product design, RPS is a much better system design
configuration for a volumetric 3D display device. Compact and
stylish desktop 3D display unit can be built.
B. Optical Layout of the SLM/Helix System With a Full Helix
3D Display Volume

Due to structural constraints of the Rear Projection Configuration, entire displayable volume of the helix cannot be fully

Fig. 13. Optical layout of switchable 3D display volume: When the electronically controlled swing mirror is On, the 3D image occupies the entire helix
volume, while when the swing mirror is Off, the 3D image occupies one half
of the helical display volume.

illuminated by an image projector via single light path. The motion control components (motors, encoders, etc.) would block
portion of images located close to the rotating axis of the helix.
To solve this problem, we invented a new optical layout that
employs split light paths. As shown in the Fig. 13(a), the image
projection coming out from the SLM projector is first reflected
by the electronically controllable swing mirror (labeled as M),
to a 45 mirror A towards upward. The image is then split in
half by a pair of mirrors B and B. The light path on the left
subsequently goes through mirrors C, D and E to illuminate the
left half of the helix volume. In a similar fashion, a symmetric
light path on the right goes through mirrors C, D and E to
illuminate the right half of the helix volume. Fig. 13(b) presents
a 3D view of this dual light path arrangement. The dual light
path optical layout bypasses the motion control unit (motor and
encoder) and is able to deliver the image projection that covers
entire displayable volume on the helix (except for the central
axis).
When the swing mirror is on the off position, the light projection coming out from the SLM projector is reflected by the
mirror , and the entire image ray will pass only the path
of A, B, C, D, E, towards to the one half of the helix volume.

10

JOURNAL OF DISPLAY TECHNOLOGY

Fig. 14. (a) 3D image data consisting of a voxel cube. (b) 3D image data is
sliced into helical slices conformal to the shape of helix screen at different
locations.

Fig. 15. Pictures of a volumetric 3D image (human head) displayed on our

VolumeViewer prototype system.

IE
W EE
eb P
Ve ro
o
rs f
ion

This display mode is often needed to offer viewers the flexibility of seeing 3D image in a higher voxel density and higher
image brightness. With the same projected light energy, smaller
the display volume, brighter the image.

C. Real-Time Dynamic 3D Image Data Generation

Although we have developed the dynamic 3D display

capability for our volumetric 3D display, fast 3D image data
preparation for real-time applications remains a challenging
task. The 3D image preparation for our display can be illustrated in Fig. 14. A set of 3D data is represented as a 3D data
cube, as shown in Fig. 14(a). The preparation tasks include
scaling and orientating the image, and slicing the 3D data
cube using helical surfaces, as shown in Fig. 14(b). The sliced
data set (a stack of 2D images) can then be sent directly to
SLM chip for projection. In general, the size of 3D image is
inherently huge (20 Mbytes each, for example) and most 3D
image processing tasks takes a long time to complete using
off-the-shelf PC computer.
In this project, we have discovered a new approach to greatly
increase the speed of the 3D data processing for displaying the
dose distribution. In RTP application, the locations of tumor and
critical organs are known via preprocessing. In the real-time display operation, we only need to adjust the color of voxels on
these objects to reflect the dose values on these voxels resulting
from real-time dose calculation. We call these voxels the active
voxels. Usually, the number of the active voxels is only a small
percentage (i.e., 5%) of all voxels. All the data corresponding to
other voxels remain unchanged. By processing only the active
voxels, significant amount of time can be saved, thus real-time
dynamic display of the changing 3D image is feasible.
We have preliminarily implemented this active voxels
approach on a prostate tumor visualization experiment. The
tumor can be tuned On and Off by viewer using a mouse at
a response time of about 0.1 s.

D. Example of Volumetric 3D Display Images

Fig. 15 shows an example of true 3D image displayed on the

VolumeViewer prototype. Note that due to the nature of true
3D image, it is very difficult to present the true 3D nature of
the display on flat media such as on a flat paper. However, the
observers who have had opportunity to see the true 3D display
all appreciate the unprecedented capability of providing both

Fig. 16. Radiation therapy planning: irradiate a tumor using multiple radiation
beams while sparing neighboring tissues from radiation damage.

physiological and psychological depth cues to human viewers

to truthfully perceive 3D objects in volumetric images.
VI. TRUE 3D DISPLAY FOR RADIOTHERAPY PLANNING

A. Basic Concept of Radiation Therapy Planning

The primary goal of a radiation therapy treatment is to deliver

a high and uniform dose to the tumor while keeping the dose to
the neighboring healthy tissues and radiation-sensitive organs as
low as possible. Fig. 16 schematically illustrates the basic concept and constraints in the radiation therapy planning. A cross
section of a body anatomy with a circular tumor is shown. If
the tumor is irradiated from only one direction with a cylinder
beam (labeled as beam1), all the healthy tissue along the beam
path are exposed to approximately the same dose as the tumor.
If, instead, we use multiple beams (the beam1 and beam2, for
example), the dose deposited on the tumor would be approximately several times of the dose exposed to the healthy tissue.
Using more beams in different directions can lead to further improvements of the dose distribution, and a very sharp dropoff
of the dose in the tissue surrounding the tumor region can be
achieved.
The planner(s) of a radiation therapy procedure should carefully select the beam configuration in order to achieve the best
treatment result. By beam configuration we mean a set of parameters including the number of beams, spatial orientation of
each beam, beam angels, intensity, beam weights and cross-section shape of each beam, etc. The best treatment result is
judged by the maximum dose distribution on the tumor and the
minimum dose distribution on surrounding healthy tissues. The
radiation therapy planning is an interactive process where the
planner has to produce, evaluate, modify, and compare several
alternative plans based on available information regarding the
patients anatomy, tumor characteristics and planners knowledge and clinical experience [12][21].

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

11

B. Why Use True Volumetric 3D Display in

Radiation Therapy Planning?

Fig. 17. Hardware setup of the interactive RTP environment.

IE
W EE
eb P
Ve ro
o
rs f
ion

1) Human Anatomy is Inherently 3D: The true volumetric

3D display technology offers unambiguous spatial relationship among the 3D structures allowing viewers to perceive
3D anatomical structure correctly and quickly. In the radiation therapy applications, the ability to visualize 3D internal
structures, as if the patient had transparent skin, allows the
oncologist to select beam angles, weights, and field shapes that
will minimize inclusion of radiosensitive organ/tissues with the
beam. More importantly, the volumetric 3D display capability
reveals the complex spatial relationship among these body parts
in a true 3D physical space, providing the planner a much more
effective way to comprehend the complex spatial relationships
between tumor and surrounding healthy organs, as well as the
dose distribution in 3D space.
Displaying dose coverage as color objects also has significant
advantages over existing technology. In existing practice, dose
distribution can either be viewed slice-by-slice on sectional images or as a 3D rendered color object displayed on a computer
screen. Therefore, the operators either view all slices to get the
dose coverage or perform 3D rendering at many viewing angles to get the complete picture of dose coverage. Either way,
it would take a long time or the picture has to be completed in
the operators mind since neither method can show the complete
picture at once. In contrary, when displayed with an interactive
lifelike 3D display monitor, the complete picture of dose coverage could be presented for the planner.
2) Limitation of Conventional Display Techniques: In conventional RTP practice, planning has often been treated as a
two-dimensional (2D) problem, mainly due to the limitations
in imaging/display technology and resources. Conventional
slice-by-slice display of CT or MRI data while providing detailed anatomic information imposes serious limitations on the
radiation treatment planning process. First, the slice-by-slice
display format makes it difficult to visualize the path of any
radiation beam not perpendicular to the axis of the CT slices.
This discourages the consideration of all treatment plans that
utilize radiation beam out of the transverse plane. Second, by
displaying the radiation iso-doses on each CT slice, the merits
of multiple competing treatment plans can be compared only
in a piecewise fashion. Experience has shown that under these
conditions, it is not always easy either to recognize the best
treatment plan or to suggest useful modifications. Finally, for
brachy therapy treatment, the conventional CT format may
offer ambiguous information as to the location of the implant.
It may be impossible to determine whether a radioactive seed
seen on one CT slice is the same as that seen on an adjacent
slice.
There have been rapid advances recently in 3D visualization
techniques (both software and hardware) to produce 3D effect
on 2D display screens. However, CT/MRI data is inherently of
3D nature, yet all conventional displays use flat 2D screens or
films (e.g., CRTs, LCDs, and slices) that lack important depth
cues. This fundamental restriction greatly limits the capability
of oncologist to perceive the complexity of the anatomy and
radiation beam configuration, therefore affects the safety, speed
and accuracy of the radiation treatment planning process.

3) Understanding at a Glance: We believe that the true

volumetric 3D display technique holds the potential to revolutionize current clinical practice of 3D treatment planning, and is
a logical evolutionary step to the fifth generation technology in
the history of radiation therapy treatment planning. The inherent
capability of displaying 3D data with true 3D cues allows clinicians to understand the spatial radiation dose distribution much
more quickly and easily. Such an understanding at a glance
is necessary to keep the clinician from becoming bogged down
in endless details, as he would be if provided only with conventional 2D display of CT slices with overlaid iso-dose lines.
The 3D RTP techniques have received broad clinical acceptance and has shown in improve clinical outcomes. It is evident
that improved visualization tools in RTP can make significant
improvements in patient care. The lifelike 3D display provides
significant advancement over the existing 3D rendering technique in that the inherent capability of displaying 3D data with
most true 3D cues allows clinicians to understand the spatial radiation dose distribution much more quickly and easily. Such an
understanding at a glance is necessary to keep the clinician
from becoming bogged down in endless details, as he would
be if provided only with conventional 2D display of CT slices
with overlaid iso-dose lines. We believe that the true volumetric
3D display technique holds the potential to revolutionize current
clinical practice of 3D treatment planning, and is a logical evolutionary step to the next generation visualization technology in
the history of radiation therapy treatment planning.

VII. FRAMEWORK OF THE INTERACTIVE RADIATION THERAPY

PLANNING ENVIRONMENT

We propose an interactive RTP environment framework that

takes full advantage of the true volumetric 3D display capability,
as shown in Fig. 17, which illustrates a prototype of the Interactive RTP Environment with gantry, beam simulator, and the
dynamic volumetric 3D display. Note that the gantry can rotate around the hemisphere display volume, and the beam simulator can adjust its angular position, thus realizing a two degree-of-freedom positioning capability, which is able to simulate typical beam positions used in RTP.
Fig. 18 provides a flowchart of the interactive RTP process
using our volumetric 3D display technology. Images of patients

JOURNAL OF DISPLAY TECHNOLOGY

IE
W EE
eb P
Ve ro
o
rs f
ion

12

Fig. 18. Interactive radiotherapy planning environment using volumetric 3D display.

anatomy and cancer/organs are acquired and processed to provide 3D digital models of anatomic structures and cancer organs. These data are sent to the volumetric 3D display for visualization. An oncologist/planner starts his/her planning process
by visualizing directly the true 3D images displayed on the volumetric 3D display monitor, just like he can view the patient with
transparent skins. The oncologist can specify the beam configuration by define beam parameters or by using the simulated
beam simulator hardware that shines a simulated radiation beam
directly on the anatomic structure and tumor location. The spatial position and orientation of the simulated beam can be totally controlled by the oncologist/planner so he/she has entire
3D freedom to place and adjust the beam configuration. Beam
Eyes View (BEV) and Room View can be provided for the visualization.
Once the planner selects the beam configuration, dose distribution corresponding to this set of beam configuration will
be calculated and the results will be sent to the volumetric 3D
display monitor for visualization. Should the planner decide to
modify the beam configuration based on the visualization results, he can go back to the beam configuration planning stage
and define the modified beams.
After the dose distribution of a plan meets the requirement,
the system automatically performs the collision avoidance verification, based on the kinematics relationship among the treatment machine, couch, and patients body shape. The collision
avoidance verification process can be animated and displayed
on the volumetric 3D display so the oncologist can visually confirm the collision-free treatment plan.

Finally, the system will formulate a final radiation therapy

plan and compute various quantitative figure of merits (FOM),
such as dose-volume histogram, dose statistics, normal tissue
complication probability (NTCP), and tumor control probability
(TCP), etc. These data can be displayed on the 2D/3D monitors
simultaneously.
We have performed experiments to demonstrate the feasibility of this novel Interactive RTP Environment concept.
Components of the Interactive RTP Environment are described
in the following paragraphs. Results of our initial experiments
are promising, as judged by a number of radiation physicists
who observed the experimental demonstrations in our prototype
system.
The 3D volumetric images of patients anatomic structure and
tumor site are displayed on the volumetric 3D display monitor,
and a simulated radiation beam mounted on a gantry structure
and controlled by the oncologist can illuminate directly the displayed tumor to observe the radiation effect. Such a beam simulator is able to duplicate the motion similar to that achieves by
the treatment machine and it has position tracking sensors that
record the motion of the beam head.
On the other hand, the configuration (orientation and position) of the displayed patient anatomic image can be controlled
by the planner to simulate the realistic patient setup configuration. The combination of the displayed volumetric 3D image and
the simulated radiation beam mechanism allows the oncologist
to adjust and select interactively the configuration and parameters of a beam (divergence, orientation, intensity, and shape) as
well as the patients setup position to achieve the best figure of

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

13

IE
W EE
eb P
Ve ro
o
rs f
ion

merits and to avoid the beam paths that could cause potential
damages of neighboring healthy tissues.
After beams are selected, computer will generate a treatment
plan, and the 3D dose distribution will be calculated. The 3D
display monitor then superimposes the dose distribution maps
with anatomical structure, allowing the radiation oncologists to
further review, modify, and approve the radiation therapy plan.
The entire planning process is highly intuitive and interactive
thus is very easy to learn and master, takes much less time from
oncologists to the RTP, and can achieve better quality of the
resulting plan.
We now describe individual components of the proposed
framework for the interactive RTP environment.
A. Volumetric 3D Display Monitor

Using a volumetric 3D display monitor in the proposed Interactive RTP Environment has unique advantages.
The 3D images of anatomy and tumor organ are floating in
the true 3D space, with the correct 3D spatial relationship
as true objects. The images can be viewed from all directions without needing any special eyeglasses, and independence from observers capability of stereo-vision. These
features offer the planner high degree of intuition and easiness to comprehend patients specific anatomic situation.
The volumetric 3D display is the only information display
media that allows for the true 3D interaction between the
displayed anatomy and tumor organ images and the simulated radiation beams. RTP Planner can interactively configure the patient position and beams configurations. The
interactive nature of the volumetric 3D display allows the
planner to modify geometric parameters while viewing directly at the 3D images of anatomic organs, as if the true
object is there. 3D images are not really useful unless the
viewer is able to interact with display in a convenient way.
The 3D images displayed on our volumetric 3D monitor possess the see-through feature. This means that
the foreground images of organs would not occlude the
background images. This transparency feature allows
viewers to see both the tumor and surrounding healthy
organ as well as the treatment beams simultaneously, thus
greatly increase the understanding of 3D spatial relationship among these elements.
B. Gantry Motion Fixture

In a single iso-center radiation treatment plan, all beams

intersect at the accelerators iso-center. To meet this requirement in our Interactive RTP Environment, we have designed
the system similar to that of gantry on the treatment machine,
with one degree-of-freedom motion fixture along the gantry (as
shown in Fig. 17) that hosts the simulated beam head. The beam
head can be moved freely around the patient by the planner
during the interactive planning session. The 3D image of the patients anatomy can be manipulated by the planner to simulate
the realistic setup position of the patient in the radiation therapy
planning session.
A position tracking sensor is installed on the gantry to track
the location of the beam head on the gantry. The sensor output
will be send to the central computer to calculate the beam configuration.

Fig. 19. Interactive visualization of anatomic structure/prostate and

beam configuration.

Fig. 19 shows a set of interactive visualization of anatomical structure of a prostate and treatment beam configuration.
Note that the simulated beam controlled by a planner is able
to directly interact with the life-size 3D anatomic structure of
a patient, and an optimal beam configuration can be selected
intuitively via interactions. The unique direct interaction capability offered by the volumetric 3D display makes it an ideal
tool for radiation therapy planning.
VIII. CONCLUSION

In this paper, we presented a novel design of true volumetric

three-dimensional display systems that is able to show true volumetric 3D images with high volumetric spatial resolution. We
documented some of our effort in designing, prototyping and
testing the volumetric 3D display systems, and our initial attempt to apply this unique 3D display technology as an augmented visualization tool to helping oncologists in selecting the
best radiation treatment plan. Although exciting progresses have
been made in terms of developing the volumetric 3D display
technology, we are still far away from achieving our ultimate
goal, which is to develop a clinically viable hardware and software that will provide unique capability of volumetric 3D visualization to aid oncologists in radiation therapy planning with
higher accuracy, effectiveness, convenience, and speed.
Good radiation treatment planning requires that the target
volume be treated with a high and uniform dose of radiation
while irradiating normal tissue as little as possible. Judging the

14

JOURNAL OF DISPLAY TECHNOLOGY

[5] J. Chu et al., 3D display of treatment planning and anatomy data:

Initial observation using a promising technical advance, in IFMBE
Proc.World Congress on Med. Physics and BioEng., 2006, vol. 14, pp.
17291732.
[6] J. Geng, Method and apparatus for high resolution three dimensional
display, US Patent 6 064 423.
[7] J. Geng, Method and apparatus for an interactive volumetric three dimensional display, U.S. Patent 7 098 872.
[8] D. L. Macfarlane, A volumetric three dimensional display, Appl.
Opt., vol. 33, no. 31, pp. 74537457, 1994.
[9] W. Matusik and H. Pfister, 3D TV: A scalable system for real-time
acquisition, transmission, and autostereoscopic display of dynamic
scenes, ACM Trans. Graphics, vol. 23, no. 3, pp. 814824, 2004.
[10] T. Okoshi, Three-Dimensional Imaging Techniques. San Diego, CA:
Academic, 1976.
[11] K. Perlin, S. Paxia, and J. S. Kollin, An autostereoscopic display, in
Proc. ACM SIGGRAPH, 2000, pp. 319326.
[12] G. C. Bentel, Radiation Therapy Planning, 2nd ed. New York: McGraw-Hill, 1996.
[13] W. Hendee and G. S. Gazelle, Biomedical imaging research opportunities workshop III: White paper, Ann. Biomed. Eng., vol. 34, no. 2,
pp. 188198, Feb. 2006.
[14] K. G. Vosburgh and F. A. Jolesz, The concept of image-guided
therapy, Acad. Radiol., vol. 10, pp. 176179, 2003.
[15] M. W. Vannier and J. L. Marsh, Three-dimensional imaging, surgical
planning, and image-guided therapy, Radiol. Clin. North Amer., vol.
34, pp. 545563, 1996.
[16] T. R. Mackie et al., Image guidance for precise conformal radiotherapy, Int. J. Radiat. Oncol. Biol. Phys., vol. 56, pp. 89105, 2003.
[17] D. Yan, D. Lockman, A. Martinez, J. Wong, D. Brabbins, F. Vicini, J.
Liang, and L. Kestin, Computed tomography guided management of
interfractional patient variation, Semin Radiat Oncol., vol. 15, no. 3,
pp. 16879, Jul. 2005.
[18] F. Laerum, Demand for a new main speciality in image-guided
therapy, Comput. Methods Programs Biomed., vol. 66, pp. 8185,
2001.
[19] D. M. Coldwell and P. E. Sewell, The expanding role of interventional
radiology in the supportive care of the oncology patient from diagnosis
to therapy, Semin. Oncol., vol. 32, pp. 169173, 2005.
[20] D. A. Jaffray, J. H. Siewerdsen, J. W. Wong, and A. A. Martinez,
Flat-panel cone-beam computed tomography for image-guided radiation therapy, Int. J. Radiat. Oncol. Biol. Phys., vol. 53, pp. 13371349,
2002.
[21] Zerhouni and E. Medicine, The NIH roadmap, Science, vol. 302, pp.
6372, 2003.

IE
W EE
eb P
Ve ro
o
rs f
ion

merits of a given treatment plan from the conventional 2D display screen can be difficult for radiation oncologists to select
the best of several alternative treatment plans. The problem becomes even more difficult if the entire spatial distribution of
the radiation dosage is to be considered, because of the enormous amount of 3D data that must be evaluated. We believe
that lack of suitable method to simultaneously display 3D dose
distribution superimposed on the relevant anatomy has greatly
contributed to the slow incorporation of 3D considerations into
routine radiation treatment planning.
The drawbacks of conventional CT or MRI displays can be
largely overcome by employing the true volumetric 3D display
technology. Such true 3D display system is able to provide both
physiological and psychological depth cues to oncologists in
perceiving and manipulating radiation beam configuration in a
true 3D fashion, thus providing unique visualization tool to ensure the safety, effectiveness, and speed of radiation treatment
planning process.
The main focus of this paper is to provide technical details
on the volumetric 3D display system we developed, and present
some initial results on its capability of displaying true 3D images. While the system design framework of applying such technology into RTP is introduced, its full scale clinical applications
to RTP is still an ongoing effort and will be reported later in other
publications.
The field of true 3D display technology is still quite young,
comparing to its 2D counterpart that has developed over several
decades with multi-billion dollar investments. It is our hope that
our preliminary work could provide some stimulations and attractions to more talented researchers from both technical and
clinical background to this fascinating field of research and development.
ACKNOWLEDGMENT

The authors would like to thank many collaborators and supporters who contributed in part to the success of this study,
among them Dr. J. Rogers, Dr. M. Freedman, Dr. T. DeWeese,
Dr. M. Vannier, Dr. S. Li, Dr. D. Frassica, Dr. J. Wong, J. Russell, M. Deis, Dr. P. Zhunag, Dr. Y. Feng, Dr. H. Li, Dr. J. Qiao,
Dr. G. Ying, Dr. S. Nerlove, Dr. J. Hennessey, Dr. R. Coryells,
Dr. K. Narayanan, Dr. P. Srivastava, Dr. L. Quatrano, Dr. H.
Baker, and Dr. B. Donoff.

REFERENCES

[1] B. Barry, Enhanced Visualization. Hoboken, NJ: Wiley-Interscience,

2007.
[2] A. Sullivan, 3 Deep, IEEE Spectr., vol. 42, no. 4, Apr. 2005.
[3] G. E. Favalora et al., 100 million-voxel volumetric display, in Proc.
SPIE Cockpit-Displays IX: Displays for Defense Appl, 2002, vol. 4712.
[4] X. Gong et al., Evaluation of volumetric display for radiation therapy
treatment planning, Med. Phys., vol. 33, no. 6, p. 2209, 2006.

Jason Geng (SM89) has over two decades of

experience in leading the research, development and
commercialization of advanced imaging technologies. He has over 80 technical papers and one book
published in the related fields. In 1995, he solely
founded and served as CEO of Genex Technologies
Inc, a Maryland-based U.S. company specialized
in advanced 3D/360-degree imaging and display
technologies and products. He has served on review
panels for National Science Foundation, National
Institutes of Health, and US Army Medical Research
Commands. He taught as adjunct professor in George Washington University,
Washington, DC, and New Jersey Institute of Technology, Newark. He is
inventor of 20 issued patents and over 20 patent applications.
Dr. Geng received the Rising Star award and ranked #291 by Deloitte &
Touch on the lists of Fast 500 Growing companies in US and Canada. He
also received prestigious national honors, including the Tibbetts Award from
US Government and was ranked #257 as INC magazines INC 500 company in
2002. He was honored by DARPA as one of the 200 top scientists in USA as
the Scientist helping America. He currently serves as the Vice President for
IEEE Intelligent Transportation Systems Society (ITSS) and is the chairman of
ITSS standard committee and ITSS publications committee.

JOURNAL OF DISPLAY TECHNOLOGY

Volumetric 3D Display for Radiation Therapy

Planning
Jason Geng, Senior Member, IEEE
(Invited Paper)

Index TermsInteractive visualization, medical imaging, radiation therapy planning, volumetric 3D display.

IE
Pr EE
int P
Ve ro
o
rs f
ion

AbstractIn current clinical practice, radiation therapy planning (RTP) has often been treated as a two-dimensional (2D)
problem, mainly due to the limitations in visualization technology
available to date. The slice-by-slice display format makes it difficult to visualize the path of radiation beam not perpendicular
to the axis of the CT slices. This discourages consideration of
treatment plans that utilize radiation beam out of the transverse
plane. Human body anatomical structures are inherently three-dimensional (3D) objects, and tumors and tissues/organs involved in
the RTP are all of 3D shapes. A clear understanding of 3D spatial
relationships among these structures, as well as the anatomic
impact of 3D dose distributions, is essential for designing and
evaluating radiation therapy plans.
We have recently made an important breakthrough in the highresolution volumetric 3D display technology and have made an initial attempt to apply it to RTP applications. By volumetric 3D
display, we mean that each voxel in the displayed 3D images
) spatial position where it is
is located physically at the (
supposed to be, and emits light from that position to form real
3D images in the eyes of viewers. We have demonstrated the feasibility of our system design by building full-scale prototypes and
achieved a multi-color, large display volume, true volumetric 3D
display system with a high resolution of over 10 million voxels in
a portable design. This type of true 3D display system is able to
present a 3D image of a patients anatomy with transparent skin,
providing both physiological and psychological depth cues to oncologists in perceiving and manipulating radiation beam configuration in true 3D fashion, thus offering a unique visualization tool
to ensure the safety, effectiveness, and speed of the RTP process.
The volumetric 3D display technology holds promise to significantly enhance the accuracy, safety, and speed of RTP procedures.
Such an understanding at a glance capability is necessary to keep
the clinicians from becoming bogged down in details, as he/she
would be if provided only with conventional 2D display of CT slices
with overlaid isodose lines.
The main focus of this paper is to provide technical details on
the volumetric 3D display system we developed, and present some
initial results on its capability of displaying true 3D images. While
the system design framework of applying such technology into RTP
is introduced, its full scale clinical applications to RTP is still an
ongoing effort and will be reported later in other publications.
Manuscript received January 8, 2007; revised February 21, 2008. This work
was supported in part by the National Institutes of Health under Grant R44
CA80577-02A1, by the Department of Energy DE-FG02-98ER82588, by the
U.S. Air Force F08635-97-C-0034, by DARPA DAAH01-97-C-R169, BMDO
DASC60-98-C-0018, by the National Science Foundation DMI-0124322, and
by NASA NAS13-01039. The content of this document does not necessarily
reflect the position or the policy of the sponsors, and no official endorsement
should be inferred.
The author resides in Rockville, MD 20852 USA (e-mail: jason.geng@ieee.
org).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JDT.2008.922413

I. INTRODUCTION

HIS paper documents our theoretical study and experimental demonstration of a revolutionary volumetric threedimensional (3D) display technique. We also present a framework for applying this true 3D display technology to radiation
therapy planning (RTP). Although we are still in the early stage
of development, the ultimate goal of this investigation is to develop a clinically viable volumetric 3D display technology for
medical image visualization in general.
The proposed volumetric life-like 3D image display technique relies upon a display media that is a true 3D volume instead of a 2D flat screen. Each volume element (called voxel,
analogous to a pixel in a 2D image) in the displayed 3D im) spatial position where it
ages locates physically at the (
is supposed to be and emits light from that position to form real
3D images in the eyes of viewers.
The volumetric 3D display we developed is fundamentally
different from conventional 3D rendering visualization technique, where the object is displayed on a 2D flat screen with
3D rendering for depth perception. It is also different from 3D
stereo video or head-mounted display (HMD), where the 3D
perception is created with a pair of polarized glasses or display
screens. The volumetric 3D display technology projects 3D images directly into true 3D space that does not require special
3D glasses to view it. Viewers can walk around the 3D image
and look at it from all different directions with realistic depth
just as looking at the real physical object. Such 3D display provides both physiological and psychological depth cues to human
viewers for truthfully perceiving objects in 3D space.
Furthermore, with realistic 3D representations of medical images in many imaging modalities (CT, MRI, PET, Ultrasound,
etc), viewers can interact with the life-sized volumetric 3D images being displayed, via handheld pointer and/or other userinterface devices, as if the true 3D virtual patient were there with
a transparent skin and visible internal anatomic structures. The
unique capabilities of walk-around viewing and direct interaction with the displayed 3D images could greatly simplify our
understanding of the complexity of 3D objects and spatial relationship among them.
We have recently made an important technical breakthrough
in implementing the high-resolution volumetric 3D display.
Using the spatial light modulator (SLM), high power visible
lasers and precision fabrication of helical screen, we have

1551-319X/$25.00 2008 IEEE

JOURNAL OF DISPLAY TECHNOLOGY

moving parts. Major difficulties to produce a practically useful

3D display using this approach are its scale-up capability and
ability to display multiple colors.
B. Gas Medium Up-Conversion
Another 3D display based on the up-conversion concept employs the intersection of two laser beams in an atomic vapor,
and subsequent omnidirectional florescence from the intersection point (U.S. Patent 4 881 068 by Korevaar, 1989). Two lasers
are directed via mirrors and - scanners towards an enclosure
containing an appropriate gaseous species (rubidium vapor, for
example), where they intersect at 90 deg. By itself, either laser
causes no visible fluorescence. However, where both lasers are
incident on the same gas atoms, two step excitation results in
florescence at the intersecting point. By scanning the intersection point fast enough, a 3D image can be drawn in the vapor.
The eye cannot see changes faster than about 15 Hz, so that if
the image to be displayed, it is repeatedly drawn faster than this
rate; the image will appear to be steady, even though light may
be originating from any one point in the volume from only a
small fraction of the time.
The advantage of this 3D display concept is its scalability: It
can be built in almost any desirable size without significantly
increasing the complexity of the system. The technical difficulties in implementing this concept including the requirement of
vacuums chamber, requirement for maintaining certain temperature, limitation of number of voxels by the speed of the scanners, and eye-safe problem of laser beams.

IE
Pr EE
int P
Ve ro
o
rs f
ion

achieved a multi-color, large display volume, true lifelike 3D

display system with a high resolution of over 10 million voxels
in a portable design. We have demonstrated our high-resolution
volumetric 3D display concept by building a full-scale prototype that can display complex 3D images.
The volumetric 3D display is a revolutionary concept for RTP
and medical image visualization in general. The major innovations of our approach are twofold.
1) We have developed a functional hardware platform and
a high-speed data interface that enables a high-resolution
volumetric 3D image with over 10 million voxels to be
displayed at a rate of 20 frames per second. We have designed, fabricated, and tested all hardware components and
software package associated with the display prototype
system. The dynamic volumetric 3D display capability not
only makes the interactive RTP possible, but also opens
doors to many new applications in medical image visualization arena.
2) We have developed a novel conceptual framework of
the Interactive RTP Environment, and built a set of
prototype hardware/software. The Interactive RTP Environment enables direct interactions between the displayed
volumetric 3D image of patient anatomic structure and
tumor, and the simulated treatment beam configuration.
The intuitive interactions help radiation therapy planners
determine suitable beam directions and parameters to
maximize the tumor coverage and minimize the exposure
of normal tissues during a planning session.
II. BRIEF SURVEY OF PRIOR ART ON VOLUMETRIC
3D DISPLAY TECHNIQUES

In this section, we provide a brief survey of a number of 3D

volumetric display techniques that have been intensively developed in the past.
A. Solid-State Up-Conversion

One of the fundamental requirements for a volumetric 3D display system is to have entire display volume filled with materials that can be selectively excited at any desired locations. To
achieve this goal, one can have two independently controlled radiation beams which activate a voxel only when they intersect.
While an electron beam cannot be used for such purpose, a laser
beam can, provided that a suitable material of display medium
can be found. A process known as two-photon up-conversion
can achieve this objective (U.S. Patent 4 041 476 by Swainson,
1977, U.S. Patent 5 684 621 by Downing, 1997). Briefly, this
process uses the energy of two infrared (IR) photons to pump
a material into an excited level, from which it can make a visible fluoresce transition to a low level. For this process to be
useful as a display medium it must exhibit two-photon absorption from two different wavelengths so that a voxel is turned
on only at the intersection of two independently scanned laser
sources. The materials of choice at the present time are the rare
earths doped into a glass host known as ZBLAN. ZBLAN is
a flurozirconate glass whose chemical name stands for ZrF4BaF2-LaF3-AlF3-NaF. The two-photon up-conversion concept
for 3D volumetric display is quite promising, since it requires no

C. Rotating Light Emitting Diodes (LEDs) Array

One of the earliest volumetric 3D displays was designed

by Schipper in 1963 (U.S. Patent 3 097 261). It consists of a
rotating electroluminenscent panel with embedded high-speed
light emitter array. By controlling the timing of - addressing
of the light emitter array and the rotation of the panel, 3D
images can be formed within the volume swept by the rotating
panel. In 1979, Berlin developed an innovative approach to
solving the high-bandwidth data transmission problem of this
design using optical link and replaced the light emitters with
high speed LED matrix (U.S. Patent 4 160 973 by Berlin, 1979).
This system uses LED arrays that are rotated to sweep out a 3D
volume. The resolution of this volume is a function of number
and density of LEDs mounted on the rotating planar array, the
speed of rotation and the rate at which the LED can be pulsed.
D. Cathode-Ray Sphere

The Cathode Ray Sphere (CRS) concept was originally developed by Ketchpel in 1960s (U.S. Patent 3 140 415 by Ketchpel,
1960) and recently implemented by researchers at New Zealand
(US Patent 5 703 606 by Blundell, 1997). The voxels are created
by addressing a rapidly rotating phosphor-coated target screen
in vacuum by electron beams synchronized to the screens rotation. The view of this rotating multi-planar surface depends
on the clarity of the glass enclosure and the translucency of the
rotating screen. Another image quality issue is the interaction
between the phosphor decay ray and the speed of the rotation of
the screen.

IE
Pr EE
int P
Ve ro
o
rs f
ion

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

Fig. 1. A brief survey of various 3D display technologies.

E. Varifocal Mirror and High Speed Monitor

A very clever method of 3D display employs the strategy of

forming optical virtual 3D images in space in front of viewer
(U.S. Patent 4 130 832 by Sher, 1978). The varifocal mirror
system consists of a vibrating circular mirror along with a
high-speed monitor. The monitor is connected to a woofer
such that the woofer can be synchronized to the monitor. A
flexible, circular mirror is attached to the front of the woofer,
and the monitor is pointed toward the mirror. With the vibrations from the woofer, the mirror changes focal length and the
different points being displayed on the monitor seem to appear
at different physical locations in space, giving the appearance
of different depths to different objects in the scene being displayed. Variable mirror based 3D display systems are primarily
limited by the size of the mirror and updating rate of images,
since this mirror has to vibrate.
F. Laser Scanning Rotating Helix 3D Display

Extensive attempts have been made by researchers at Texas

Instruments Incorporated (US Patent 5 042 909, 5 162 787, by
Garcia, 1991) to develop a 3D display device based on laser
scanning and rotating (helical) surface. Lasers scanning 3D displays operate by deflecting a beam of coherent light generated
by a laser to a rotating helical surface. Timing modulation of the
laser beam controls the height of the light spot that is produced
by the laser on the rotating surface. The deflectors include devices such as polygonal mirrors, galvanometer, acousto- optics
modulated deflectors, and micro-deformable mirrors. There are
several problems with this 3D display mechanism that have prevented it from becoming commercially feasible.
The most serious problem is the limitation on the maximum
number of voxels that can be displayed. Due to the nature of
sequential (non-parallel) laser scanning, only one spot of light
can be displayed at any given moment. All the activated image
voxels have to be addressed, one by one, by the scanning of
single laser beam in time-multiplex fashion. The time needed
for steering the laser beam and to stay on the voxel position to
produce sufficient brightness poses an upper limit to how many
voxels it can display. To increase the number of voxels, multiple channel lasers and scanners could be used. However, many

attempts to increase the spatial resolution have hampered with

high cost and bulky hardware design.
Fig. 1 summarizes various research and development efforts
on 3D display. Recently, there is a surge of research activities
on volumetric 3D display that promise to bring high resolution
( 100 million voxels) display into reality [1][11].
III. VOLUMEVIEWER 3D DISPLAY CONCEPTP

In this section, we provide detailed technical discussions on

the VolumeViewer 3D display design and its implementation.
A. Principle of the Multi-Planar Volumetric 3D Display

Fig. 2 illustrates the principle of the Multi-planar volumetric 3D image display using a high-speed 2D image projector
and a moving screen. Suppose that a sweeping screen can be
controlled to move back and forth along the direction at a
frequency higher than 20 Hz. Within the time period of each
sweeping motion, frames of 2D image patterns are projected
by the high-speed 2D image projector. The moving screen intercepts 2D image projections at different positions along axis,
forming a stack of spatial image layers in true 3D space. If the
cycling speed of the moving screen is sufficiently high, and the
2D image projector can produce sufficient number of 2D image
sections during each pass, human observers are able to perceive
a true volumetric 3D image floating in the 3D space without
flicker, due to the residual effect of human eyes.
The multi-planar volumetric 3D display principle is by no
means a complex concept. However, implementation has been
difficult due to lack of suitable high-speed image projector,
clever mechanism to produce sweeping screen motion, and
high brightness light sources. There has been a number of
attempts been made to build such cumbersome system without
success. A physically flat screen sweeping at 20 Hz creates
serious problems of mechanical design, balance, vibration, and
noise. Conventional liquid crystal projector can only achieve
a switching rate of few hundred hertz, leading to a very low
spatial resolution. High power light source has been very
expensive and cumbersome. All these factors contribute to
a slow progress of volumetric 3D display techniques using
multi-planar principle.

JOURNAL OF DISPLAY TECHNOLOGY

IE
Pr EE
int P
Ve ro
o
rs f
ion

Fig. 2. Principle of the multi-planar volumetric 3D display using fast 2D projection and a moving screen.

Fig. 3. Volumetric 3D display concept using a fast SLM and a rotating helix screen.

B. Concept of the VolumeViewer 3D Display

We propose a new generation of the multi-planar volumetric

3D display system, taking advantages of rapid advances in material, laser, and semiconductor fabrication technologies. With the
newly developed ferrorelectric liquid crystal spatial light modulator (SLM) technology, the switching speed of SLM reaches
over 3000 fps for a SLM of 256 by 256 pixels or higher resolutions. Such a fast SLM can be used as a high-speed image pattern generator to produce volumetric 3D pictures, providing a
powerful tool to revolutionize the state-of-the-art of 3D display.

Fig. 3 illustrates a design concept of our SLM/helix volumetric

3D display.
In Fig. 3, light rays produced by a light source projector ,
passing through filter and collimating lenses , impinge on a
polarizing beam splitter cube . Due to the polarization characteristics of the beam splitter, the polarized light rays are reflected by the beam splitter and projected onto a SLM . The
image data shown on the SLM is generated by a host computer
. The SLM is able to alternate image patterns at high frame
rate (i.e., over 3000 fps). When a pixel on SLM is turned ON,
the light will be reflected back to the beamsplitter cube, while

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

when the pixel is turned OFF, the projected light on this pixel
will be absorbed by the SLM and will not be reflected. The patterns on the SLM are therefore able to control the patterns of the
reflected light rays. The reflected light rays with encoded SLM
image patterns transmit through the beam splitter cube.
is employed to project
An optical projection lens system
the image patterns towards a spinning helix screen, marked as
. The light spots projected on the helix screen intersect the
helix surface at different heights depending on different rotating
angles of the helix, thus form 3D voxels in 3D space (the display
volume ). Each section of the helix surface is described by the
following mathematical equations:

Fig. 4. Prototype of the VolumeViewer 3D display.

IE
Pr EE
int P
Ve ro
o
rs f
ion

TABLE I
PERFORMANCE OF THE VOLUMETRIC 3D DISPLAY PROTOTYPE.

If rotation is synchronized via a motor driving the helix

screen with the switching timing of the SLM , such that 3D
image patterns are shown in the 3D space with a high refresh
Hz), naked eyes perceive it as a 3D volumetric
rate (i.g.,
image. No special eyewear is required to view such 3D image
floating in true 3D space, just as a real object is placed there.

C. Advantages of the Proposed SLM/Helix 3D Display System

Inherent parallel architecture for voxel-addressing: Instead of using single laser beam to address all the voxels
(such as the NRaD scanning laser system) the SLM/Helix
system use 256 by 256 (or more) light rays to address
simultaneously voxels, thus overcomes the bottleneck
in producing high resolution 3D images encountered by
other approaches.
High Spatial Resolution: The maximum number of voxels
that can be generated by the SLM/Helix display depends
upon the spatial resolution of SLM and the spinning speed
of helix. With the currently available SLM technology, a
SLM with 1024 by 1024 pixel and 300 000 frames per
second switching speed is available. The resolution of proposed 3D display can take advantage of the rapid advances
of SLM technology.
Simple structure and easy to build: Other than the rotating
helix, there is no other scanning or moving part. The optical design and alignment are not difficult. The system can
be built using commercial off-the-shelf (COTS) products,
which leads to shorter development period and low cost.
No special viewing glasses or helmet are needed by
viewers: The volumetric images are displayed in true
3D space with almost 360 degree viewing angle, which
preserve all physiological and psychological depth cues of
human visual system. Viewers can walk freely around the
monitor to see the 3D images, just as if the real 3D object
were sitting there.
Implementation of full color display is straightforward:
Just use three SLMs for Red, Green, and Blue respectively,
and the color of voxels can be automatically controlled.
Another way to implement color display is even simpler:
use Red, Green, and Blue light projector, and synchronize
the timing of three projectors with a high speed SLM.

D. VolumeViewer 3D Display Prototype

Fig. 4 presents an overall system design configuration of

the newly designed and prototyped volumetric 3D display. We
dubbed this prototype system the VolumeViewer. Inside the
transparent hemispherical dome is a rotating helix forming a
3D image display volume of 7 height and 20 in diameter.
There are nine pieces of reflective mirrors with large dimensions of optic surfaces. If fabricated using conventional
thick glasses, these mirrors would be heavy-weighted, fragile,
and difficult to assembly with acceptable accuracy of optical
alignment. We adopted a state-of-the-art mirror fabrication
technology that forms large piece of flat mirror using framed
thin films with high reflectivity. These thin-film mirrors have
only 10% of weights as their glass mirror counterparts, and can
be built to fit various difficult geometric dimensions. Due to
their light weight and flexible dimensions, we can easily mount
them into optically aligned modules, thus saves us tremendous
effort in the final stage of the optical alignment in the system
integration. Table I lists major performance specification of the
VolumeViewer Prototype system.

JOURNAL OF DISPLAY TECHNOLOGY

Fig. 5. SLM structure.

cells arranged as a square of 256 by 256 array with total dimension of 5 5 mm approximately. The device achieves better
than 25% optical throughput when used with collimated laser
light and better than 100:1 contrast ratio when oriented for amplitude modulation. A better than 100:1 contrast ratio of SLM
provides a fairly good image quality. The device can be operated
as fast as 5 kHz with complete switching of the liquid crystal.

IE
Pr EE
int P
Ve ro
o
rs f
ion

IV. DESIGN AND FABRICATION OF THE PCI INTERFACE BOARD

ALLOWING FOR DYNAMIC 3D IMAGE DISPLAY
A. Primary Objectives of the PCI Interface Board Design

Although our initial success in developing the original prototype system represented the state-of-the-art true volumetric 3D
display technology then in terms of achieving high spatial resolution, the updating rate of 3D images in original system was
still slow. To update a displayed 3D image into a new frame of
3D image, the host PC has to upload the data set of the new 3D
image to the SLM driver via a parallel port. This data transmission of a single frame of 3D image usually takes about 20 s, due
to the size of 3D dataset and the slow speed of the PC parallel
port. Such a low updating rate certainly prevents our current 3D
display system design from being used in many dynamic interactive 3D display applications, such as radiation therapy planning sessions.
Therefore, one of the main efforts of this investigation is to
design and fabricate a PCI interface board to eliminate the bottleneck of 3D image transmission between host PC and SLM
chip. Primary goals of this PCI interface board are:
1) to achieve 3D image updating at a rate up to 20 images per
second from host PC to SLM chip;
2) to increase the frame rate of 2D image displayed on the
SLM;
3) to allow for multiple color 3D display.
B. Spatial Light Modulator (SLM)

SLMs, devices that alter the temporal and spatial character of

a light beam, can be either optically or electrically addressed.
Optically addressed SLMs often require bulky support equipment and additional light sources. To obtain high frame rate of
image projection for volumetric 3D display, we propose to use
the electronically controllable, fast ferroelectric liquid crystal
(FLC) reflective spatial light modulator. The device is built atop
a planarized 0.6 m CMOS SRAM backplane with 15 m pixel
pitch and 87% fill factor. A thin layer of FLC material is sandwiched between a metal conductor and a glass window coated
with a transparent conductive layer such as indiumtinoxide
(ITO). When a voltage is applied across the FLC layer the fast
axis of the bi-refringent FLC material is forced into one of two
possible states: ON or OFF (the image on the SLM is binary).
The structure of a SLM is depicted in Fig. 5. It is a specially
designed integrated circuit housed in a 49-pin ceramic PGA
package. The effective area of the SLM consists of 65 536 FLC

C. Design the PCI Interface Board to Control the SLM Chip

Controlling the SLMs operation is very similar to addressing
a Static Random Access Memory (SRAM) chip. The interface
board contains an on-board microprocessor, memory for up to
512 frame 2D images, circuitry for controlling the SLM, and
circuitry for communicating with host PC computer. We use
C++ and VXD (a low-level assembly) software to manipulate
the image data and to transfer them into the image buffer on the
controller, which in turn sends the image sequence to the SLM
in a predetermined high frame rate. Fig. 6 illustrates the block
diagram of our Interface Board design.
1) Microprocessor: The size of 3D data sets is inherently
huge. In order to transfer huge amount of 3D data in higher
speed among the host computer, on-board image memory, and
the SLM chip, the microprocessor must have the high-speed
data transferring unit, such as DMA, Interrupt Unit etc. The microprocessor must also have a PCI interface and other control
units for connecting with PC and communicating with other
standard facilities. By careful design comparison, we selected
the Intel 80960RP as the CPU of the board. The 80960RP is a
PCI IO processor with 352 BGA pins. It has many units for data
communication. Its DMA Controller, Address Translation Unit
(ATU), Message Unit (MU), Memory Controller and other control units are suitable for the design, and it has an available and
completed embedded software system, so it fulfills the design
features.
2) Memory: We need a higher speed RAM on the board as a
buffer to store 3D images. In the design two memory groups are
used to display dynamic 3D images and make the other display
functions. By considering the speed, volume and stability, we
chose eight SRAM, MCM6246 chips, as the image memory on
the board and divided them into two groups, each has 2 Mbytes.
This image memory can implement all features described above.
3) FPGA (Field Programmable Gate Array): Due to complex operation of the PCI interface board, thousands of gates
and flip-flops are needed to fulfill the desired functions. Dozens
of buses with 32 bits have to switch each other, which is impossible to be laid on an area-limited printed circuit board directly.

IE
Pr EE
int P
Ve ro
o
rs f
ion

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

Fig. 6. Block diagram of the PCI interface board.

D. Fabrication and Test the PCI Interface Board

Fig. 7. Block diagram of FPGA1.

Using the newly designed SLM device interface board, we

have achieved a maximum transmission rate of about 2200 fps
and the resolution of each frame of 2D images is 256 by 256
pixels. At the target 3D image refreshing rate of 7 images per
second, we are able to produce 157 frames of 2D images for
each 3D image cube. This enhanced speed of image transfer effectively increases the spatial resolution of our volumetric 3D
by
by
million voxels. In comdisplay to
parison to the use of original SLM drive unit, the maximum spaby
million voxels.
tial resolution is about by
Furthermore, the 3D image-refreshing rate was about 20 seconds per image versus the 20 images per second of current
system equipped with the new PCI interface board. The success of the PCI interface board allows us to perform dynamic
3D image display, and makes the application of our 3D display
technology to RTP practically possible.

E. Software Drive Development Using VXD Techniques

Fig. 8. Block diagram of FPGA2.

We employed advanced FPGA technology which allows for a

software programmable functionality on hardware chips. Two
Xilinx XCS40 (each with 40,000 gates) are used and their functions are illustrated in Figs. 7 and 8. Due to space limit, a total
of 24 complex circuit diagrams implemented by FPGAs cannot
be included. Fig. 9 gives an example of programmable functions
implemented by the FPGA1. Powerful FPGAs make it possible
for us to design a compact PCI board with the desired features.

The software for the board is divided into host computer programs and the 80960RP microprocessor programs. The host
computer programs include a Windows based work studio and a
VxD (Virtual Device Driver) program, as shown in Fig. 10. The
80960-based program includes the embedded programs stored
in PROM or the executable code downloaded from the host computer to the 80960RP program RAM. In fact the board can be
controlled from host computer and 80960RP.
When the host computer is turn on, BIOS of the computer
finds the PCI board, so that the Windows 95/98 can get the information from the Intel 80960RP. Then the model of (Operation
System) OS loads the GTI3DD.VxD into the computer memory.
During loading GTI3DD.VxD, OS communicates with the VxD
to decide the resources allocation. After all VxDs are loaded, OS
builds up a table to save the results.

JOURNAL OF DISPLAY TECHNOLOGY

IE
Pr EE
int P
Ve ro
o
rs f
ion

Fig. 9. An example of detailed design diagram inside the FPGAs.

Fig. 10. Operation of the VxD for the PCI interface board.

V. SWITCHABLE DISPLAY VOLUMEA NEW OPTICAL DESIGN

ENABLING BOTH FULL AND HALF HELIX DISPLAY VOLUMES

Fig. 11. Comparison of previous and current 3D image projection schemes.

In our previous system design and experiments, we projected

the 3D images onto a small portion (less than one half of a helix,
see the left drawing in Fig. 11) of the sweeping volume produced
by the helix. A large portion of the useful volume produced by
the sweeping helix was wasted. In our latest effort, we have
significantly improved the system optical design to project the
3D images into the entire helix volume (see the right drawing
in Fig. 11), thus increasing the size of the display volume to
entire sweeping volume of the rotating helix 20 (508 mm) in
diameter and 7 (178 mm) in height.

A. Overall Optical Configuration Design and Tradeoff: Front

Projection Versus Rear Projection
There are two possible overall optic configurations for the
SLM/Helix system design: Front projection configuration
(FPC) versus rear projection configuration (RPC). Previous
3D display systems, such as NRaD 3D display, use the front
projection configuration. This means that laser beams are
projected onto the rotating helix surface in the same side as
viewers view the 3D images [Fig. 12(a)]. The front projection
configuration makes it easier to implement a driving system

IE
Pr EE
int P
Ve ro
o
rs f
ion

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

Fig. 12. Comparison of overall optical configuration for the SLM/helix 3D display.

for the rotating helix. It also provides convenience in adjusting

optical system setups and modifying other components or
subsystems. However, the disadvantages of the front projection
configuration include the following.
(1) It leads naturally to an overhead projection configuration so a compact system design (as a portable desktop
display, for instance) is difficult to achieve.
(2) Viewers may possibly block the projection of laser
beams.
(3) Since system components are spread out, maintaining an
accurate optical alignment is difficult.
We have developed the rear projection system (RPS) design
for our SLM/Helix 3D display. In a RPS configuration, laser
beams are projected onto the rotating helix screen from below,
while viewers look 3D images from above [see Fig. 12(b)]. The
helix surface is made of semi-transparent material so it transmits
50% light and reflects 50% light. This rear-projection approach
eliminates the unwieldy overhead mirrors from the NRaDs design and allows the lasers, scanners, optics, and the helix to
be packaged together as a single compact mobile 3D display
unit. Since the helix surface transmits as well as reflects light in
omni-direction, the viewing angle of the voxels in a 3D image
is very large (almost true walk-around viewing angle and group
viewing capability).
A major advantage of using the RPS configuration is that it
is possible to achieve a compact system design. All the components of the SLM/Helix system can be packaged into a cabinet
with 3D display volume on the top. From the viewpoint of final
commercial product design, RPS is a much better system design
configuration for a volumetric 3D display device. Compact and
stylish desktop 3D display unit can be built.
B. Optical Layout of the SLM/Helix System With a Full Helix
3D Display Volume

Due to structural constraints of the Rear Projection Configuration, entire displayable volume of the helix cannot be fully

Fig. 13. Optical layout of switchable 3D display volume: When the electronically controlled swing mirror is On, the 3D image occupies the entire helix
volume, while when the swing mirror is Off, the 3D image occupies one half
of the helical display volume.

illuminated by an image projector via single light path. The motion control components (motors, encoders, etc.) would block
portion of images located close to the rotating axis of the helix.
To solve this problem, we invented a new optical layout that
employs split light paths. As shown in the Fig. 13(a), the image
projection coming out from the SLM projector is first reflected
by the electronically controllable swing mirror (labeled as M),
to a 45 mirror A towards upward. The image is then split in
half by a pair of mirrors B and B. The light path on the left
subsequently goes through mirrors C, D and E to illuminate the
left half of the helix volume. In a similar fashion, a symmetric
light path on the right goes through mirrors C, D and E to
illuminate the right half of the helix volume. Fig. 13(b) presents
a 3D view of this dual light path arrangement. The dual light
path optical layout bypasses the motion control unit (motor and
encoder) and is able to deliver the image projection that covers
entire displayable volume on the helix (except for the central
axis).
When the swing mirror is on the off position, the light projection coming out from the SLM projector is reflected by the
mirror , and the entire image ray will pass only the path
of A, B, C, D, E, towards to the one half of the helix volume.

10

JOURNAL OF DISPLAY TECHNOLOGY

Fig. 14. (a) 3D image data consisting of a voxel cube. (b) 3D image data is
sliced into helical slices conformal to the shape of helix screen at different
locations.

Fig. 15. Pictures of a volumetric 3D image (human head) displayed on our

VolumeViewer prototype system.

IE
Pr EE
int P
Ve ro
o
rs f
ion

This display mode is often needed to offer viewers the flexibility of seeing 3D image in a higher voxel density and higher
image brightness. With the same projected light energy, smaller
the display volume, brighter the image.

C. Real-Time Dynamic 3D Image Data Generation

Although we have developed the dynamic 3D display

capability for our volumetric 3D display, fast 3D image data
preparation for real-time applications remains a challenging
task. The 3D image preparation for our display can be illustrated in Fig. 14. A set of 3D data is represented as a 3D data
cube, as shown in Fig. 14(a). The preparation tasks include
scaling and orientating the image, and slicing the 3D data
cube using helical surfaces, as shown in Fig. 14(b). The sliced
data set (a stack of 2D images) can then be sent directly to
SLM chip for projection. In general, the size of 3D image is
inherently huge (20 Mbytes each, for example) and most 3D
image processing tasks takes a long time to complete using
off-the-shelf PC computer.
In this project, we have discovered a new approach to greatly
increase the speed of the 3D data processing for displaying the
dose distribution. In RTP application, the locations of tumor and
critical organs are known via preprocessing. In the real-time display operation, we only need to adjust the color of voxels on
these objects to reflect the dose values on these voxels resulting
from real-time dose calculation. We call these voxels the active
voxels. Usually, the number of the active voxels is only a small
percentage (i.e., 5%) of all voxels. All the data corresponding to
other voxels remain unchanged. By processing only the active
voxels, significant amount of time can be saved, thus real-time
dynamic display of the changing 3D image is feasible.
We have preliminarily implemented this active voxels
approach on a prostate tumor visualization experiment. The
tumor can be tuned On and Off by viewer using a mouse at
a response time of about 0.1 s.

D. Example of Volumetric 3D Display Images

Fig. 15 shows an example of true 3D image displayed on the

VolumeViewer prototype. Note that due to the nature of true
3D image, it is very difficult to present the true 3D nature of
the display on flat media such as on a flat paper. However, the
observers who have had opportunity to see the true 3D display
all appreciate the unprecedented capability of providing both

Fig. 16. Radiation therapy planning: irradiate a tumor using multiple radiation
beams while sparing neighboring tissues from radiation damage.

physiological and psychological depth cues to human viewers

to truthfully perceive 3D objects in volumetric images.
VI. TRUE 3D DISPLAY FOR RADIOTHERAPY PLANNING

A. Basic Concept of Radiation Therapy Planning

The primary goal of a radiation therapy treatment is to deliver

a high and uniform dose to the tumor while keeping the dose to
the neighboring healthy tissues and radiation-sensitive organs as
low as possible. Fig. 16 schematically illustrates the basic concept and constraints in the radiation therapy planning. A cross
section of a body anatomy with a circular tumor is shown. If
the tumor is irradiated from only one direction with a cylinder
beam (labeled as beam1), all the healthy tissue along the beam
path are exposed to approximately the same dose as the tumor.
If, instead, we use multiple beams (the beam1 and beam2, for
example), the dose deposited on the tumor would be approximately several times of the dose exposed to the healthy tissue.
Using more beams in different directions can lead to further improvements of the dose distribution, and a very sharp dropoff
of the dose in the tissue surrounding the tumor region can be
achieved.
The planner(s) of a radiation therapy procedure should carefully select the beam configuration in order to achieve the best
treatment result. By beam configuration we mean a set of parameters including the number of beams, spatial orientation of
each beam, beam angels, intensity, beam weights and cross-section shape of each beam, etc. The best treatment result is
judged by the maximum dose distribution on the tumor and the
minimum dose distribution on surrounding healthy tissues. The
radiation therapy planning is an interactive process where the
planner has to produce, evaluate, modify, and compare several
alternative plans based on available information regarding the
patients anatomy, tumor characteristics and planners knowledge and clinical experience [12][21].

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

11

B. Why Use True Volumetric 3D Display in

Radiation Therapy Planning?

Fig. 17. Hardware setup of the interactive RTP environment.

IE
Pr EE
int P
Ve ro
o
rs f
ion

1) Human Anatomy is Inherently 3D: The true volumetric

3D display technology offers unambiguous spatial relationship among the 3D structures allowing viewers to perceive
3D anatomical structure correctly and quickly. In the radiation therapy applications, the ability to visualize 3D internal
structures, as if the patient had transparent skin, allows the
oncologist to select beam angles, weights, and field shapes that
will minimize inclusion of radiosensitive organ/tissues with the
beam. More importantly, the volumetric 3D display capability
reveals the complex spatial relationship among these body parts
in a true 3D physical space, providing the planner a much more
effective way to comprehend the complex spatial relationships
between tumor and surrounding healthy organs, as well as the
dose distribution in 3D space.
Displaying dose coverage as color objects also has significant
advantages over existing technology. In existing practice, dose
distribution can either be viewed slice-by-slice on sectional images or as a 3D rendered color object displayed on a computer
screen. Therefore, the operators either view all slices to get the
dose coverage or perform 3D rendering at many viewing angles to get the complete picture of dose coverage. Either way,
it would take a long time or the picture has to be completed in
the operators mind since neither method can show the complete
picture at once. In contrary, when displayed with an interactive
lifelike 3D display monitor, the complete picture of dose coverage could be presented for the planner.
2) Limitation of Conventional Display Techniques: In conventional RTP practice, planning has often been treated as a
two-dimensional (2D) problem, mainly due to the limitations
in imaging/display technology and resources. Conventional
slice-by-slice display of CT or MRI data while providing detailed anatomic information imposes serious limitations on the
radiation treatment planning process. First, the slice-by-slice
display format makes it difficult to visualize the path of any
radiation beam not perpendicular to the axis of the CT slices.
This discourages the consideration of all treatment plans that
utilize radiation beam out of the transverse plane. Second, by
displaying the radiation iso-doses on each CT slice, the merits
of multiple competing treatment plans can be compared only
in a piecewise fashion. Experience has shown that under these
conditions, it is not always easy either to recognize the best
treatment plan or to suggest useful modifications. Finally, for
brachy therapy treatment, the conventional CT format may
offer ambiguous information as to the location of the implant.
It may be impossible to determine whether a radioactive seed
seen on one CT slice is the same as that seen on an adjacent
slice.
There have been rapid advances recently in 3D visualization
techniques (both software and hardware) to produce 3D effect
on 2D display screens. However, CT/MRI data is inherently of
3D nature, yet all conventional displays use flat 2D screens or
films (e.g., CRTs, LCDs, and slices) that lack important depth
cues. This fundamental restriction greatly limits the capability
of oncologist to perceive the complexity of the anatomy and
radiation beam configuration, therefore affects the safety, speed
and accuracy of the radiation treatment planning process.

3) Understanding at a Glance: We believe that the true

volumetric 3D display technique holds the potential to revolutionize current clinical practice of 3D treatment planning, and is
a logical evolutionary step to the fifth generation technology in
the history of radiation therapy treatment planning. The inherent
capability of displaying 3D data with true 3D cues allows clinicians to understand the spatial radiation dose distribution much
more quickly and easily. Such an understanding at a glance
is necessary to keep the clinician from becoming bogged down
in endless details, as he would be if provided only with conventional 2D display of CT slices with overlaid iso-dose lines.
The 3D RTP techniques have received broad clinical acceptance and has shown in improve clinical outcomes. It is evident
that improved visualization tools in RTP can make significant
improvements in patient care. The lifelike 3D display provides
significant advancement over the existing 3D rendering technique in that the inherent capability of displaying 3D data with
most true 3D cues allows clinicians to understand the spatial radiation dose distribution much more quickly and easily. Such an
understanding at a glance is necessary to keep the clinician
from becoming bogged down in endless details, as he would
be if provided only with conventional 2D display of CT slices
with overlaid iso-dose lines. We believe that the true volumetric
3D display technique holds the potential to revolutionize current
clinical practice of 3D treatment planning, and is a logical evolutionary step to the next generation visualization technology in
the history of radiation therapy treatment planning.

VII. FRAMEWORK OF THE INTERACTIVE RADIATION THERAPY

PLANNING ENVIRONMENT

We propose an interactive RTP environment framework that

takes full advantage of the true volumetric 3D display capability,
as shown in Fig. 17, which illustrates a prototype of the Interactive RTP Environment with gantry, beam simulator, and the
dynamic volumetric 3D display. Note that the gantry can rotate around the hemisphere display volume, and the beam simulator can adjust its angular position, thus realizing a two degree-of-freedom positioning capability, which is able to simulate typical beam positions used in RTP.
Fig. 18 provides a flowchart of the interactive RTP process
using our volumetric 3D display technology. Images of patients

JOURNAL OF DISPLAY TECHNOLOGY

IE
Pr EE
int P
Ve ro
o
rs f
ion

12

Fig. 18. Interactive radiotherapy planning environment using volumetric 3D display.

anatomy and cancer/organs are acquired and processed to provide 3D digital models of anatomic structures and cancer organs. These data are sent to the volumetric 3D display for visualization. An oncologist/planner starts his/her planning process
by visualizing directly the true 3D images displayed on the volumetric 3D display monitor, just like he can view the patient with
transparent skins. The oncologist can specify the beam configuration by define beam parameters or by using the simulated
beam simulator hardware that shines a simulated radiation beam
directly on the anatomic structure and tumor location. The spatial position and orientation of the simulated beam can be totally controlled by the oncologist/planner so he/she has entire
3D freedom to place and adjust the beam configuration. Beam
Eyes View (BEV) and Room View can be provided for the visualization.
Once the planner selects the beam configuration, dose distribution corresponding to this set of beam configuration will
be calculated and the results will be sent to the volumetric 3D
display monitor for visualization. Should the planner decide to
modify the beam configuration based on the visualization results, he can go back to the beam configuration planning stage
and define the modified beams.
After the dose distribution of a plan meets the requirement,
the system automatically performs the collision avoidance verification, based on the kinematics relationship among the treatment machine, couch, and patients body shape. The collision
avoidance verification process can be animated and displayed
on the volumetric 3D display so the oncologist can visually confirm the collision-free treatment plan.

Finally, the system will formulate a final radiation therapy

plan and compute various quantitative figure of merits (FOM),
such as dose-volume histogram, dose statistics, normal tissue
complication probability (NTCP), and tumor control probability
(TCP), etc. These data can be displayed on the 2D/3D monitors
simultaneously.
We have performed experiments to demonstrate the feasibility of this novel Interactive RTP Environment concept.
Components of the Interactive RTP Environment are described
in the following paragraphs. Results of our initial experiments
are promising, as judged by a number of radiation physicists
who observed the experimental demonstrations in our prototype
system.
The 3D volumetric images of patients anatomic structure and
tumor site are displayed on the volumetric 3D display monitor,
and a simulated radiation beam mounted on a gantry structure
and controlled by the oncologist can illuminate directly the displayed tumor to observe the radiation effect. Such a beam simulator is able to duplicate the motion similar to that achieves by
the treatment machine and it has position tracking sensors that
record the motion of the beam head.
On the other hand, the configuration (orientation and position) of the displayed patient anatomic image can be controlled
by the planner to simulate the realistic patient setup configuration. The combination of the displayed volumetric 3D image and
the simulated radiation beam mechanism allows the oncologist
to adjust and select interactively the configuration and parameters of a beam (divergence, orientation, intensity, and shape) as
well as the patients setup position to achieve the best figure of

GENG: VOLUMETRIC 3D DISPLAY FOR RADIATION THERAPY PLANNING

13

IE
Pr EE
int P
Ve ro
o
rs f
ion

merits and to avoid the beam paths that could cause potential
damages of neighboring healthy tissues.
After beams are selected, computer will generate a treatment
plan, and the 3D dose distribution will be calculated. The 3D
display monitor then superimposes the dose distribution maps
with anatomical structure, allowing the radiation oncologists to
further review, modify, and approve the radiation therapy plan.
The entire planning process is highly intuitive and interactive
thus is very easy to learn and master, takes much less time from
oncologists to the RTP, and can achieve better quality of the
resulting plan.
We now describe individual components of the proposed
framework for the interactive RTP environment.
A. Volumetric 3D Display Monitor

Using a volumetric 3D display monitor in the proposed Interactive RTP Environment has unique advantages.
The 3D images of anatomy and tumor organ are floating in
the true 3D space, with the correct 3D spatial relationship
as true objects. The images can be viewed from all directions without needing any special eyeglasses, and independence from observers capability of stereo-vision. These
features offer the planner high degree of intuition and easiness to comprehend patients specific anatomic situation.
The volumetric 3D display is the only information display
media that allows for the true 3D interaction between the
displayed anatomy and tumor organ images and the simulated radiation beams. RTP Planner can interactively configure the patient position and beams configurations. The
interactive nature of the volumetric 3D display allows the
planner to modify geometric parameters while viewing directly at the 3D images of anatomic organs, as if the true
object is there. 3D images are not really useful unless the
viewer is able to interact with display in a convenient way.
The 3D images displayed on our volumetric 3D monitor possess the see-through feature. This means that
the foreground images of organs would not occlude the
background images. This transparency feature allows
viewers to see both the tumor and surrounding healthy
organ as well as the treatment beams simultaneously, thus
greatly increase the understanding of 3D spatial relationship among these elements.
B. Gantry Motion Fixture

In a single iso-center radiation treatment plan, all beams

intersect at the accelerators iso-center. To meet this requirement in our Interactive RTP Environment, we have designed
the system similar to that of gantry on the treatment machine,
with one degree-of-freedom motion fixture along the gantry (as
shown in Fig. 17) that hosts the simulated beam head. The beam
head can be moved freely around the patient by the planner
during the interactive planning session. The 3D image of the patients anatomy can be manipulated by the planner to simulate
the realistic setup position of the patient in the radiation therapy
planning session.
A position tracking sensor is installed on the gantry to track
the location of the beam head on the gantry. The sensor output
will be send to the central computer to calculate the beam configuration.

Fig. 19. Interactive visualization of anatomic structure/prostate and

beam configuration.

Fig. 19 shows a set of interactive visualization of anatomical structure of a prostate and treatment beam configuration.
Note that the simulated beam controlled by a planner is able
to directly interact with the life-size 3D anatomic structure of
a patient, and an optimal beam configuration can be selected
intuitively via interactions. The unique direct interaction capability offered by the volumetric 3D display makes it an ideal
tool for radiation therapy planning.
VIII. CONCLUSION

In this paper, we presented a novel design of true volumetric

three-dimensional display systems that is able to show true volumetric 3D images with high volumetric spatial resolution. We
documented some of our effort in designing, prototyping and
testing the volumetric 3D display systems, and our initial attempt to apply this unique 3D display technology as an augmented visualization tool to helping oncologists in selecting the
best radiation treatment plan. Although exciting progresses have
been made in terms of developing the volumetric 3D display
technology, we are still far away from achieving our ultimate
goal, which is to develop a clinically viable hardware and software that will provide unique capability of volumetric 3D visualization to aid oncologists in radiation therapy planning with
higher accuracy, effectiveness, convenience, and speed.
Good radiation treatment planning requires that the target
volume be treated with a high and uniform dose of radiation
while irradiating normal tissue as little as possible. Judging the

14

JOURNAL OF DISPLAY TECHNOLOGY

[5] J. Chu et al., 3D display of treatment planning and anatomy data:

Initial observation using a promising technical advance, in IFMBE
Proc.World Congress on Med. Physics and BioEng., 2006, vol. 14, pp.
17291732.
[6] J. Geng, Method and apparatus for high resolution three dimensional
display, US Patent 6 064 423.
[7] J. Geng, Method and apparatus for an interactive volumetric three dimensional display, U.S. Patent 7 098 872.
[8] D. L. Macfarlane, A volumetric three dimensional display, Appl.
Opt., vol. 33, no. 31, pp. 74537457, 1994.
[9] W. Matusik and H. Pfister, 3D TV: A scalable system for real-time
acquisition, transmission, and autostereoscopic display of dynamic
scenes, ACM Trans. Graphics, vol. 23, no. 3, pp. 814824, 2004.
[10] T. Okoshi, Three-Dimensional Imaging Techniques. San Diego, CA:
Academic, 1976.
[11] K. Perlin, S. Paxia, and J. S. Kollin, An autostereoscopic display, in
Proc. ACM SIGGRAPH, 2000, pp. 319326.
[12] G. C. Bentel, Radiation Therapy Planning, 2nd ed. New York: McGraw-Hill, 1996.
[13] W. Hendee and G. S. Gazelle, Biomedical imaging research opportunities workshop III: White paper, Ann. Biomed. Eng., vol. 34, no. 2,
pp. 188198, Feb. 2006.
[14] K. G. Vosburgh and F. A. Jolesz, The concept of image-guided
therapy, Acad. Radiol., vol. 10, pp. 176179, 2003.
[15] M. W. Vannier and J. L. Marsh, Three-dimensional imaging, surgical
planning, and image-guided therapy, Radiol. Clin. North Amer., vol.
34, pp. 545563, 1996.
[16] T. R. Mackie et al., Image guidance for precise conformal radiotherapy, Int. J. Radiat. Oncol. Biol. Phys., vol. 56, pp. 89105, 2003.
[17] D. Yan, D. Lockman, A. Martinez, J. Wong, D. Brabbins, F. Vicini, J.
Liang, and L. Kestin, Computed tomography guided management of
interfractional patient variation, Semin Radiat Oncol., vol. 15, no. 3,
pp. 16879, Jul. 2005.
[18] F. Laerum, Demand for a new main speciality in image-guided
therapy, Comput. Methods Programs Biomed., vol. 66, pp. 8185,
2001.
[19] D. M. Coldwell and P. E. Sewell, The expanding role of interventional
radiology in the supportive care of the oncology patient from diagnosis
to therapy, Semin. Oncol., vol. 32, pp. 169173, 2005.
[20] D. A. Jaffray, J. H. Siewerdsen, J. W. Wong, and A. A. Martinez,
Flat-panel cone-beam computed tomography for image-guided radiation therapy, Int. J. Radiat. Oncol. Biol. Phys., vol. 53, pp. 13371349,
2002.
[21] Zerhouni and E. Medicine, The NIH roadmap, Science, vol. 302, pp.
6372, 2003.

IE
Pr EE
int P
Ve ro
o
rs f
ion

merits of a given treatment plan from the conventional 2D display screen can be difficult for radiation oncologists to select
the best of several alternative treatment plans. The problem becomes even more difficult if the entire spatial distribution of
the radiation dosage is to be considered, because of the enormous amount of 3D data that must be evaluated. We believe
that lack of suitable method to simultaneously display 3D dose
distribution superimposed on the relevant anatomy has greatly
contributed to the slow incorporation of 3D considerations into
routine radiation treatment planning.
The drawbacks of conventional CT or MRI displays can be
largely overcome by employing the true volumetric 3D display
technology. Such true 3D display system is able to provide both
physiological and psychological depth cues to oncologists in
perceiving and manipulating radiation beam configuration in a
true 3D fashion, thus providing unique visualization tool to ensure the safety, effectiveness, and speed of radiation treatment
planning process.
The main focus of this paper is to provide technical details
on the volumetric 3D display system we developed, and present
some initial results on its capability of displaying true 3D images. While the system design framework of applying such technology into RTP is introduced, its full scale clinical applications
to RTP is still an ongoing effort and will be reported later in other
publications.
The field of true 3D display technology is still quite young,
comparing to its 2D counterpart that has developed over several
decades with multi-billion dollar investments. It is our hope that
our preliminary work could provide some stimulations and attractions to more talented researchers from both technical and
clinical background to this fascinating field of research and development.
ACKNOWLEDGMENT

The authors would like to thank many collaborators and supporters who contributed in part to the success of this study,
among them Dr. J. Rogers, Dr. M. Freedman, Dr. T. DeWeese,
Dr. M. Vannier, Dr. S. Li, Dr. D. Frassica, Dr. J. Wong, J. Russell, M. Deis, Dr. P. Zhunag, Dr. Y. Feng, Dr. H. Li, Dr. J. Qiao,
Dr. G. Ying, Dr. S. Nerlove, Dr. J. Hennessey, Dr. R. Coryells,
Dr. K. Narayanan, Dr. P. Srivastava, Dr. L. Quatrano, Dr. H.
Baker, and Dr. B. Donoff.

REFERENCES

[1] B. Barry, Enhanced Visualization. Hoboken, NJ: Wiley-Interscience,

2007.
[2] A. Sullivan, 3 Deep, IEEE Spectr., vol. 42, no. 4, Apr. 2005.
[3] G. E. Favalora et al., 100 million-voxel volumetric display, in Proc.
SPIE Cockpit-Displays IX: Displays for Defense Appl, 2002, vol. 4712.
[4] X. Gong et al., Evaluation of volumetric display for radiation therapy
treatment planning, Med. Phys., vol. 33, no. 6, p. 2209, 2006.

Jason Geng (SM89) has over two decades of

experience in leading the research, development and
commercialization of advanced imaging technologies. He has over 80 technical papers and one book
published in the related fields. In 1995, he solely
founded and served as CEO of Genex Technologies
Inc, a Maryland-based U.S. company specialized
in advanced 3D/360-degree imaging and display
technologies and products. He has served on review
panels for National Science Foundation, National
Institutes of Health, and US Army Medical Research
Commands. He taught as adjunct professor in George Washington University,
Washington, DC, and New Jersey Institute of Technology, Newark. He is
inventor of 20 issued patents and over 20 patent applications.
Dr. Geng received the Rising Star award and ranked #291 by Deloitte &
Touch on the lists of Fast 500 Growing companies in US and Canada. He
also received prestigious national honors, including the Tibbetts Award from
US Government and was ranked #257 as INC magazines INC 500 company in
2002. He was honored by DARPA as one of the 200 top scientists in USA as
the Scientist helping America. He currently serves as the Vice President for
IEEE Intelligent Transportation Systems Society (ITSS) and is the chairman of
ITSS standard committee and ITSS publications committee.