Introduction
Computers have been used to mediate the generation of sound and images
for nearly half a century now. The field of
Computer Graphics came into it's
own during the late seventies through the eighties with huge advances
in technology and widespread application in many fields from computer
interfaces, to entertainment to scientific and engineering design and
analysis, to cartography. In the late eighties, there emerged the new
term,
Visualization which was
primarily used to depict physical quantities and processes in what is
known as
Scientific Visualization
. By the mid nineties, the term
Information
Visualization had been coined to cover these tools and
techniques for more abstract, informational objects and domains.
The application of computer generated visuals and sound from a
first-person perspective probably first occurred in flight
simulators. The term
Virtual
Reality emerged in the late eighties, replacing an earlier term
from artificial intelligence,
Artificial
Reality. Virtual Reality implied an immersive perceptual
environment that the user was embedded in through the use of advanced
computer mediated graphics and sound and possibly haptic, olfactory and
other perceptual stimulae.
Immersion
An important precept of our work is that the task of understanding
abstract information demands even more of the human than scientific
visualization and that one key to increasing our ability to handle the
quantity, complexity and subtlety is to use varying techniques to
create a sense of presence or
Immersion.
Firstly, the use of multiple senses , the use of large field-of-view
visual presentations (Powerwalls, CAVEs, Head Mounted Displays, etc),
real-time rendering and animation speed, 3D perspective, complex
lighting models, texturing, etc. naturally affords us a much
larger encoding space. Secondly, the use of higher-level
metaphorical devices than simple geometry (colored points, regions,
even polygons and symbols) offers a wider range of subtlety. And
thirdly, the sense of presence seems to enhance the coupling of higher
level perceptual functions and lower level cognitive functions.
It is not hard to believe that humans, and in fact primates, mammals,
vertebrates and even the lowest order of creatures with sense organs
are evolved to use the huge range of cues provided by the real world,
integrating multiple sensory streams and creating higher orders of
information, knowledge and perhaps even wisdom through the application
of complex, layered models of the world around them.
Flatland
For most of our 3D immersive visualization work, we use the Flatland
development environment from the University of New Mexico High
Performance Visualization center and the Homonculous project run by Dr.
Thomas P. Caudell of the Electrical and Computer Engineering
department.
On top of this development environment we have added another layer of
our own known as Flux (not a true acronym) which is a data flow
environment to support the development and coordination of many
independent information visualization components in a single
application.
Examples
The following are a few very early examples of our attempts to apply
these principles to several complex and subtle but relatively abstract
problems. They reflect the work of many people ranging from those
who provided the problem data (or simulation) to those providing the
underlying technology and those who participated in developing the
actual metaphors, encodings and representations.
Motivation
We believe that the human perceptual and cognitive systems are
currently extremely underutilized by visualization systems.
The human sensory organs and nervous system were evolved over a huge
period of time under a wide variety of survival pressures, living in a
fully dynamic, 3 dimensional world. Not only do we see in stereo,
but our hearing is simultaneously capable of omnidirectional listening
and highly directional discrimination. We are able to intuitively
predict a wide variety of phenomena in the real world, ranging from
simple geometric tasks (e.g. recognizing lines, areas, volumes) to more
physics based tasks (e.g. predicting ballistic trajectories) to complex
social tasks (e.g. recognizing, even manipulating pack, herd, or mob
behavior in prey or in fellow humans). Our cognitive systems
also have a very strong abstract symbolic component, developed
originally perhaps for language, but adapted for the symbolic, abstract
thinking characterized by mathematics.
We believe that
the limited application of immersive environments to information
visualization reflects a number of things, but perhaps most
importantly, the lack of experience. Over approximately 25
years as a computer graphics practitioner, I have watched new tools,
techniques and even paradigms arrive and be ignored and dismissed by
many while the groundwork is laid and experience is gained by those
willing to experiment and to make mistakes. It should also be
noted that we are on the cusp of widespread availability of inexpensive
but high quality equipment for limited immersion. The
entertainment industry in particular is driving his technology hard and
3D gaming, animation and interactive storytelling will continue to
drive the technology and the acceptance of these modes of interaction
with computers. The limited window provided here through
text, low resolution, still images without sound does not begin to
suggest the experience that goes with using these tools in a full
or semi-immersive environment.
FROTH
(Force-directed Representation Of Tree Hierarchies)
The
FROTH algorithm uses forces derived from the Lennard-Jones potential, a
stylized force often used to model the forces between atoms and
molecules and tends toward an organization of hexagonal closest
packing. Since we are laying out trees, we are able to do a
recursive, divide-and-conquer layout in order N to the power of
((M+1)/M) where N is the total number of nodes and M is the average
number of branches at each level in the tree. The
Lennard-Jones potential essentially balances two high powers(6 and
12) of 1/r in a single force. Inspired by atomic
physics simulations, this has lead us to consider a wider range
of forces similar to this where a single force has an equilibrium
point. The particular rendering seen here on the right, we refer to as
"the
bubble tree" with a variant above laid out on a
hemisphere. The actual behavior of these bubble trees, as
they lay out dynamically is very reminiscent of cell
fissioning. A new node is placed randomly into the point in
the tree where it belongs and then through recursive application of a
force derived from the lennard-jones potential, all of the nodes in the
tree adjust to each other until equilibrium is reached again.
H Fat Tree
The following visualization is of 6144 (6K) Quadrics interconnect
switches arranged in a Quad Fat tree, connecting 1024 4-processor
machines. These 4096 (4K) processors comprise 1/3 of the 12K
processor
Q machine developed for the ASCI project at Los Alamos National
Laboratory. The challenge
in this project was to compactly layout the 6K switches in a way that
allowed us to unambiguously see the communications between processors
and switches while maintaining a sense of the implied latency of
messages through switches. The inspiration for this layout
are the
H-arranged waveguides for phased array radar where the distance from
every antenna in the array to the receiver point is identical.
This layout was generated by a recursive function, yielding a fractal
dimension of 2 (therefore not technically fractal).
The message traffic shown here comes from the Alacarte simulator, which
exploits the deterministic rules in the Quadrics switches to anticipate
the route a given message must take given the total state of the switch
fabric and a source, destination pair.
CIP/DSS graph layout
The Critical Infrastructure Protection/Decision Support System
(CIP/DSS) project is part of the DHS portfolio of projects doing
R&D in infrastructure protection for Homeland Security.
This
project involves work from several national laboratories, with Los
Alamos, Sandia and Argonne national laboratories providing most of the
modeling effort.
There are 14 major infrastructures modeled, such as Energy, Finance,
Environment, Transportation, Communications, Water, etc. These
Infrastructures are then broken down further into subsectors such as
(within energy) Electrical Power, Natural Gas Distribution, etc.
On the right, you will see each of the 32 subsectors laid out in
parallel layers, ordered arbitrarily by the names they are
given. In the source representation, each "subsector" is
built independently of the others with the interdependencies between
their variables being external references. The layout within each
subsector was defined explicitly by the developer of each model.
The layout on the left shows the same 32 subsectors, comprised of about
5000 coupled ordinary differential equations, laid out in a circular
array to help deconflict the intradependencies from the
interdependencies.
A commercial modeling tool, Vensims, is used to develop, run and
analyze this huge, complex model. This particular model is
designed to represent an urban scale set of infrastructures but there
area similar models being built to describe regional and national scale
infrastructures which in turn may also be coupled.
As you can see, the complexity, scale and subtlety of this problem is
quite significant. We plan to not only help with
understanding the topology of these networks but also the dynamics of
their execution.
The visualization shown here involves the application of
force-directed layout algorithms similar to the FROTH project described
elsewhere. For general graph-layout in 2 and 3 D we use a
modified spring model.
As you may be able to see in this limited snapshot, the elements with
the least interdependencies move to the outside edge of the graph while
those with the most interdependencies move to the inside.
We currently treat all nodes and edges identically, but intend to
distinguish how we map spring constants and coulomb forces to the nodes
and edges to help group and separate subsectors and infrastructures
more clearly.
NID (Network Intrusion Detection)
Here we model the internet and our open internal network using what we
call the "Space Defense" metaphor. The internal protected
space is the small circular (annular) region in the center, the inner
transparent hemisphere is our firewall or gateway router and the
region between the two external hemispheres is where the internet
at-large resides.
This work is being done for the Los Alamos National Laboratory security
team, funded by NNSA.
Flows (such as TCP connections) are indicated by rays between the
source and the destination host. The color of the ray represents
the destination port or loosely the service being requested or
provided. The duration of the ray's existence and some features
of it's animation encode the duration of the flow's existence.
The height and coloration of the surface representing the internal
protected space encodes (logarithmically) the number of simultaneous
connections to the host at that location.
The internal space encodes the last two octets of the IP address as R
and Theta. Each subnet is therefore a "ring" in the annulus with
256 positions around the circumference.
The external space similarly encodes the first 3 octets of the IP
address as Phi (elevation), Theta (azimuth) and R (position between
hemispherical shells). Each subnet in the internet at large,
therefore is represented by a unique location in the hemispherical
shell.
In the foreground you see the H Fat tree representation of a
supercomputer switching fabric and messaging flux. This shows how
multiple visualization applications can be run in the same virtual
environment.
Credits and Acknowledgments
The bulk of the above work represents the combined efforts of several
institutions and numerous individuals. My own team in the
Decision Applications division at Los Alamos National Laboratory;
Paul Weber, David Hite, Robert Gislason, Steve Linger, Kriste Henson,
Chris Davis, Jeremy Martinez. The UNM team lead by Dr. Thomas P.
Caudell: Dr. Kenneth Summers, Paniotis, Tim Eyring, Chris Davis,
Donner Holten, Satyam Kothapelly, Lisong Sun, Victor Vergera, Moad
Mowafi, Hong Gi, Takeshi Hamata, et alia. LANL Collaborators: The
Alacarte supercomputer simulation team (Brian Bush, Kei Davis, Kathy
Berkbigler, Nick Moss), the Network Security Research team (Mike Fisk,
Eugene Gavrilov, Ben Uphoff, et alia), The CIP team (Brian Bush, Kevin
Saeger, Mark Witkowski, David Thompson, et alia), External
collaborators at LBNL (Wes Bethel, John Shalf, et alia), George Mason
University (Dan Carr), UC Davis (Kwan Liu Ma), USC (Scott Fisher,
et alia), FakeSpace Laboratories (Mark Bolas, Ian McDowell, Dan Corr)
and other Visualization teams at LANL in CCN-8, CCS-1, NIS-4, ESA,
EES. Our funding sponsors: NNSA security, the ASCI program, and
DHS.