From:
 Dr. David K. Kahaner
 US Office of Naval Research Asia
 (From outside US):  23-17, 7-chome, Roppongi, Minato-ku, Tokyo 106 Japan
 (From within  US):  Unit 45002, APO AP 96337-0007
  Tel: +81 3 3401-8924, Fax: +81 3 3403-9670
  Email: kahaner@cs.titech.ac.jp
Re: 3D Displays
12/26/94 (MM/DD/YY)
This file is named "3d-displ.94"

ABSTRACT. Several examples of 3D display technology in Japan from Sanyo,
Matsushita, and Terumo.

I have been interested in three dimensional display technology for
several years, see for example "fujitsu.3d" 25 April 1990, "3d-2-92.1"
and "3d-2-92.2", 19 Feb 1992, and especially the second report for
background. Portions of that are reproduced below for completeness.

It is useful to make the distinction between 3D, biplano-stereoscopic,
and multiplano-stereoscopic images (the latter two are commonly called
"stereo"). True 3D images not only give the sensation of depth, but
allow observers to "look around" to their sides and perhaps even their
back.  Biplano-stereo images are produced from (only) two original
images. They also can give a very realistic sensation of depth, but have
no "look around" capability.  An observer moving his/her  head while
viewing a stereo image causes the image to shift slightly, but no
occluded visual information comes into view and the perspective remains
the same.  Multiplano images are composed of more than two original
images and do have some look around capability. Often, the distinction
between 3D and stereo images is ignored. Indeed, there are "3D" and
even "4D" workstations on sale but this almost always means three
dimensional images projected onto a two dimensional display device such
as a CRT.

Most stereo systems use a stereogram, left and right image pairs, to
recreate the depth sensation of binocular vision (stereopsis). The
majority of these systems require a device to be worn by the observer to
select the image to be viewed by each individual eye. Systems without
such an observer-worn selection device are called autostereoscopic, or
"glassless".

In viewing the real world it is known that the sense of depth is the
result of a ten or more different factors.  For example, overlapping or
occlusion, where one object obscures part of another, is a depth clue
that does not depend on having two eyes. Another monocular clue is the
image of road edges that we expect to be parallel. Similar monocular
clues are related to retinal image size, areal perspective, shading,
shadows, texture, etc. An important binocular clue to distance from the
observer is the difference in angle between the viewing axes of left and
right eyes when both are focusing on a point (convergence).  Adjustment
of the focal length of the crystalline lens (accommodation) is another
clue, although this is mostly monocular.  Binocular parallax is the most
important binocular clue, relating to the fact that each eye sees a
slightly shifted view of the image.  Individuals differ greatly in their
ability to use these clues either because of physical impairments,
training, or some processing difficulties. This is much like color
vision; people who lack it entirely discover so at an early age, others
whose abilities are below average may go through their entire lives
accommodating in other ways.

Images can be viewed on electronic displays such as TVs, CRTs, flat
panels, etc., or in hard copy form such as a photograph, plot image, and
so forth. Viewing images may or may not require the use of special
glasses. Anaglyph images require red/green glasses as selection devices,
and most people are familiar with these from the large number of
commercial motion pictures (in the 1950s and 60s) which required them,
but their use can be traced back to as early as 1858. Anaglyph
techniques can be used for viewing either still images on paper or
dynamic images such as films.  However, the current trend for films,
video tape or computer screen images has been toward polarized glasses
as selectors, or systems without glasses.

Current work seems to be primarily directed toward stereo vision,
although the main technique for full 3D imaging is holography, i.e. the
reconstruction of the object wavefront. The original principle involves
illuminating the object with a laser and simultaneously recording the
reflected (or diffused) light from the object and a reference beam from
the laser, creating an interference fringe pattern. The recorded pattern
can later be illuminated with the same laser to reproduce the image.
Work in holographic techniques has recently focused on using
conventional light rather than a laser, and the creation of holographic
stereograms, in which multiple images of an object are recorded by
ordinary cameras at different positions and a hologram of each image is
recorded sequentially.  Holograms can provide a very high resolution and
geometrically accurate image which can be viewed without glasses and in
principle are indistinguishable from the original object.  But initial
enthusiasm for holography notwithstanding, practical problems such as
reproducing color and providing dynamic displays have not yet been
effectively solved.  Also, many techniques for holographic imaging
produce images smaller than observers would like to see. As one
researcher commented, holograms provide too much information, i.e., it
isn't really necessary to completely reconstruct the wavefront to have
an effective image.  However, some of the most exciting developments in
this area are being carried out at MIT's media lab under the direction
of Prof Stephen Benton. Benton's early claim to fame was the invention
of the white light transmission (dubbed rainbow) hologram, and more
recently, the practical demonstration of holographic video.  The lab is
also working on holograms that are in full color, large size, animated,
and can be totally synthesized by computer.  Perhaps most importantly,
this work has re-energized the field and forced researchers to take a
more serious look at ongoing related work.

In my 1992 report I stated that "we are still years away from practical
holography in our homes such as holographic TV. For example, any
practical holographic display device relying on Benton's approach will
require time-bandwidth product far exceeding those available with single
channel acousto-optic modulators, and require other techniques such as
multichannel modulators, parallelism, etc." However, NTT has recently
announced the development of a prototype holographic movie system.
Quoting from the NTT Review, Vol 6 No. 6, Nov 1994, "NTT has realized a
system that utilizes 35mm holographic film already on the market to
continuously photograph 3D video pictures. The usual 3D movie has a
fixed viewing direction and is stereoscopic only on one side, which is
not realistic in the real world. However, holography makes it possible
to observe from any direction, and to photograph or reproduce a subject
in all directions. In also allows the eye to focus on any portion of the
image, which provides for a more natural three dimensional picture.
However, electronic display devices for holography have not yet reached
the level of practical use, so a practical electronic animated
holographic picture system has yet to be developed.

"To collect the various basic data required prior to practical use of an
animated holographic picture system, NTT has developed equipment capable
of holographic photography/reproduction in real time as experimental
equipment for simulation.

"The system developed at this time consists of a photographing mechanism
plus a laser light source. For holograms that record more than 1,000
interference fringes per 1mm interval, it [normally] takes a few minutes
to photograph each frame using a normal light source because the
photographic sensitivity is very low. This [new] equipment, however,
uses a strong light source to enable continuous photographing of 3D
video pictures on film in real time.

"This experiment confirms that the natural movement of difficult
substances, such as flowing liquid or trailing smoke, can be faithfully
photographed and reproduced. The new system enables existing movie
equipment to be used to holographically photograph and reproduce 3D
movies of long duration. It is expected to find a wide range of
applications in entertainment, medical use, etc., in the future."
[end of NTT remarks]

If right then left eye images are displayed sequentially from a source,
and a synchronized shutter system in front of the eyes allows the right
eye image to only enter the right eye, etc., then stereo vision can be
observed. The shutter can be mounted in glasses which are matched with a
display in which two constituent pictures are presented in alternation
instead of simultaneously. The glasses occlude one eye and then the
other in synchronism with the image presentation. This is often called
"field sequential". This method avoids the retinal rivalry caused by
anaglyph viewing but can introduce other discomfort such as the increase
of flicker (on 60 Hz displays), the introduction of time parallax
between the two images, or the possibility of "ghosting" between the
images due to phosphor persistence. On computer displays flicker can be
solved by increasing from 60 to 120 frame refreshes per second, although
this is accomplished by halving the number of pixels that are painted
per frame, perhaps leading to lower resolution.  Most glasses-based
shutter systems use LCDs which work with polarized light.  Currently,
glasses using LCDs can provide good switching speed and reasonable
extinction of the alternating lenses.  The electro-optical polarizing
shutters now in use transmit about 30% of the unpolarized input light
(rather than 50% for perfect polarizers) and this reduces the image
brightness a little, but in practice this does not appear to be a major
problem.  Some eye-glass shutters are connected by wires to the monitor
(tethered), others are controlled by infrared and are wireless. Another
system uses a polarizing shutter mounted on the display device and
eye-glasses with fixed (circularly) polarized lenses.  While this
reduces the complexity of the eye-glass system, the large
screen-covering shutter is expensive to produce and is fragile.

In 1982, C.Smith wrote that "future generations will be astonished that
for a few decades in the 20th century we were happy to accept these
small flat images as a representation of the real three-dimensional
world." It seems obvious that in robotics, photogrammetry, pattern
recognition, etc., three dimensional imaging would be a great help.

In Japan, work in stereo and 3D imaging spans the same broad subfields
as in the West except that there is a decided difference in emphasis.
The Japanese have been much more active in research concerning
autostereoscopic imaging. Early research work was mostly connected with
lenticular sheets and that continues, but today there is also a growing
Japanese interest in holography inspired by the impressive work at MIT.

The basic idea in this approach (lenticular sheets) is also not new. An
object is photographed with two cameras corresponding to left and right
eye. Then images are displayed on a sequence of narrow vertical stripes,
left eye image, right eye image, left, right, etc., a corduroy or
interdigitated display.  These days flat panel display devices are often
used for the displays.  Immediately in front is an array of half
cylindrical lenses roughly matching the pitch of the display with axis
of revolution from top to bottom of the screen. Out in front of all this
sits the observer, who can have an authentic stereo image if he/she is
positioned in exactly the right place. Little head movement is allowed
and the observer must be seated at exactly the right position.  Multiple
observers can view this kind of display at the same time although each
observer must be correctly positioned. Also it is possible for observers
to get a pseudostereoscopic image (right image to left eye, and left to
right).

All researchers would like to dispense with glasses, but most
(Westerners I spoke to) believe that practical systems will require them
for the remainder of this decade. At the moment, the main problem with
practical autostereoscopic systems (nonholographic) is that the viewing
position can be critical.  In 1992, one American remarked to me that he
didn't understand why there was so much Japanese interest as there
seemed to be major technical problems, and there might even be a wall
that could not be breached. Another commented that he found that the
difficulties encountered when moving from viewing lobe to viewing lobe
(i.e., head movement) in glassless lenticular systems to be far more
problematic than properly presented glasses approaches.  A third said
that he saw no Japanese systems that were anywhere near being
productizable, and some that were much more than a decade away.

In 1992 I stated that "my own view is somewhat different. The problem of
stereo or 3D imaging is old enough that many fundamental ideas have
already been proposed.  Some of these may have failed in the past
because the implementation technology was not up to the demands placed
upon it. But it may be appropriate to look more carefully again.  Even
in the case of restricted viewer position, there are obvious
applications, such as sitting in front of a computer monitor looking at
the image of a molecule."

Sanyo has now (1994) released several 3D autostereoscopic systems as
products including two large screen models (40 inch and 70 inch, selling
for US$100K and US$50K) using a pair of LCD rear mounted projectors and
lenticular screens, and three small models using self contained  LCD
displays (10, 6, and 4 inches, no selling price announced) using an
image splitter (parallax-barrier) technique. Some of this technology was
developed jointly with NHK (see my report mentioned above for further
discussions of NHK's research).  I viewed all the systems and found them
to be as bright and clear as more traditional displays.  Viewing
position is important, as expected. Sanyo feels that there are very
natural applications in game/entertainment/simulation/museum situations,
as well as in selected commercial demonstration fields. Sanyo also
believes the smaller systems can be installed in both car and aircraft
where head movement is constrained. One specific application mentioned
was to add 3D to the growing number of navigational map displays (in
autos).

In addition to the displays, Sanyo has also developed and is marketing
an add-on board that converts ordinary 2D video images to pseudo-stereo
3D by use of a modified time difference algorithm. The board digitizes a
linear sequence of image signals from conventional video software for
division into two image signals (L and R). The R signal is stored
briefly in a video memory and then reproduced on the display a short
time after L is displayed. The time lag yields parallax or position
shift in the image displayed to the left and right eyes, and gives the
appearance of stereo. Since this hardware (board plus 3D LCD display)
can be used with any video signal, Sanyo envisions customers viewing
their favorite TV shows, but now in "3D".

My hosts for the visit to Sanyo were

    Mr Kenji Oyamada
    Manager, Hypermedia Research Center
    Multimedia Systems Department
    Sanyo Electric Co., Ltd
    1-1 Dainichi Higashimachi
    Moriguchi City, Osaka 570 Japan
     Tel: +81 6 900-3519; Fax: +81 6 901-6844

and

    Mr Kenji Taima
    Sanyo Electric Co., Ltd
    3D Project
    1-1 Dainichi Higashimachi
    Moriguchi City, Osaka 570 Japan
     Tel: +81 6 900-3511; Fax: +81 6 900-3536
     Email: KENJI@IMAGE-LAB.OR.JP or KENJI@YDI-01.YD.HM.RD.SANYO.CO.JP


Another autostereoscopic display device was shown by the Terumo Corp, a
medical instrument maker. The company has been working in this field for
several years, using time interlacing, a large format convex lens in
front of the display and infrared lighting, combined for the stereo
effect, in collaboration with Nagoya University College of Medicine.
Stated commercial applications include a two camera laparoscope and
endoscope.

For information about Terumo, contact the following.

    Mr Tomohiko Hattori
at
    Medical Device Department II
    Terumo R&D Center
    Terumo Corp
    1500 Inokuchi, Nakai-machi
    Ashigarakami-gun, Kanagawa-ken 259-01 Japan
     Tel: +81 465 81 4155; Fax: +81 465 81 4158