Exploring a new Visualization Infrastructure
January 19, 2005
Mountain View, CA

EnvisionO5 Workshop

Steve Smith
January 21, 2005



On January 19th, 2005, a group of 25 experts in Information Visualization, Geospatial Information Systems, Collaborative Interfaces, Immersive Interfaces, Scientific Visualization and related disciplines convened at the Silicon Graphics (SGI) headquarters at Mountain View, CA to explore the possibilities implied by the convergence of the technology and general literacy in these fields.   I found it somewhat satisfying that this group has settled on the same dimensions of Visualization that I have and in general were aligned with the way I view the future of visualization for exploration, discovery and analysis.

Organizers Katy Borner and Bonnie DeVarco simultaneously herded approximately 25 hyperactive cats (some would say badgers) through a day of presentations and discussions while documenting the day quite thoroughly with the help of an AV person and a professional illustrator who listened to our discussions and drew images and words that captured much of what was discussed.

As an invited guest, I was very impressed by the quality, not only of the presentations and the organization but the discussion in general.  It was a model I would like to see employed for other workshops of this type. 

At the end of the meeting one participant challenged the group as a whole to continue the work "virtually" after the meeting, implying that if we cannot employ the tools we are promoting for our own work, then perhaps they are not ready for prime time, or we are not fit to be the ones to promote such tools.   The group was guardedly receptive, both supporting the idea that a continued conversation was desireable and probably quite doable with the tools at hand, but also that face-to-face communication such as we were experiencing that day was still the best possible way to do the kinds of brainstorming we did.

To that end, I have suggested to several that a Wiki site might be an excellent way to continue this.  Others suggested other tools for asynchronous telecollaboration and meeting.  Whatever we may collectively choose, I will submit this report as an integral part.  In the spirit of rich media and sharing, I have chosen HTML as my format for this document.

Katy and Bonnie have promised to deliver a straw-man white-paper to the group to add to, and I will link that into this document as it is made available and evolves.  Here are some of the pre-presentation slides compiled before the workshop.


Despite arriving late (having automatically driven to the pre-crash facilities of SGI first and having to get directions up the road to SGI's more modest yet still anything but humble new digs), I was able to see all of the presentations and engage in all but the opening discussions.

SGI Reality Center

As the host of this meeting, SGI was given rich opportunity to preach to the choir about high quality visualization, immersive interfaces, etc.   They hosted this session in their ever-popular Reality Center which they use as a prototype and demonstration environment for the kinds of centers they install worldwide for corporate, academic and government clients.   The center stage of this incarnation of the Reality Center is a large (10' x 30') curved (dual axis, toroidal section) screen front-projected by 3 high resolution projectors (make and resolution unknown to me) and driven (surely) by a very hot SGI box.    SGI is known for their large, shared memory, multi-processor, multi-pipe equipment.  The LANL VIEWs system consists of several of the largest examples of these machines with hundreds of video outputs.

One of the more senior SGI staff from the UK (David Hughes) gave us a walk down memory lane, focusing on 3D graphics and SGI's play in it.  Aside from having developed all of the early commercia 3D pipeline transformers and renderers and workstations, SGI also was the incubator and sometimes partner for/in most of their competition in the commodity graphics world (ATI, nVidia, etc) and in the game system world starting with Nintendo XX in 19XX..

Hughes then segued from reminiscing and reviewing the huge changes in technology, application and culture over the years to one of their more interesting products, the Media Fusion Engine which he is intimately involved in the development of.  My perspective of the product, which appears to be both a commercial product and an ongoing research project, is that it provides one aspect of approximately what I've been hoping we could achieve with Flatland... a single, virtual 3D workspace where many tools outside the environment could be integrated or "fused" into a single working space.   Active signals/visualizations from other platforms could be fed into the enviroment which would mediate where those were placed in the environment and how they were interacted with.  This product/concept seems to juxtapose nicely with another technology shown later in the day they call Visual Area Networking.   Only 3 simple organizational metaphors were shown for the workspace, but with a true 3D environment many more suggest themselves.   While 2D desktops tend to be limited to "stacking" and "docking", this 3D environment easily handled arrangements more reminiscent of variously familiar items like a rolodex, carousel, and multiple complex stacks.   Conversations with Hughes offline, after the fact suggest that they are very eager to collaborate with anyone such as ourselves who have existing experience with such environments (viz. Flatland).

Geospatial Visualization


The state of Geospatial Visualization was presented by Eric Frost of GeoFusion and GEON(a Grid Infrastructure for the Geosciences).   Eric covered not only the familiar world of ESRI and it's suite of products and tools, but also opened up the big world of open-source and Grid-enabled tools being used by the non-profit, academic and government organizations around the world who need open, collaborative solutions.   This talk primarily set the stage for a presentation later in the session from  (John Graham, SDSU) and others who demonstrated a fully collaborative remote visualization system built primarily (exclusively?) of public domain or widely available tools, supporting hundreds of workers from different organizations and countries currently involved in the Tsunami Relief effort.   On a completely different planet, they showed the same technology applied to the Mars Landers  . The story, as told and demonstrated was very convincing.   I am now (more than ever) a believer in the real value and utility of distance collaboration in these circumstances, and in the advantage of commodity and open source software and data.  It appears that geospatial systems are no longer for those who can afford several thousands of dollars per seat for highly private, closed systems.   This section is missing several links and references that I hope I will get from any resulting white paper, etc.

Mike Liebhold presented work on geo-registered augmented reality where GPS technology is combined with geospatial databases and real-time image capture, processing and display to provide heads-down and eventually heads-up overlays and annotations for everyone from first-responders in crisis situations to museum or gallery patrons.  Jaron Lanier, who was slated to speak during dinner weighed in heavily on many topics related to practical geospatial registration at that scale and encouraged all participants to provide for resolutions on the order of .1mm or better.  Jaron also had a lot to offer on the current state of the art in practical geolocation including orientation (x,y,z,roll,pitch,yaw) suitable for heads-up display augmented reality.

Information Visualization using the Cartographic Metaphor

Katy Borner, workshop organizer from Indiana University and Kevin Boyack of Sandia National Laboratory presented their work using VxInsight to map knowledge domains.  Katy also showed the work she did with her students which won the http://www.cs.umd.edu/hcil/InfovisRepository/contest-2004/contest in October 2004 and made the cover of the PNAS issue on Mapping Knowledge Domains in April 2004.

Andre Skupin spoke on his own work in applying cartographic and geographic metaphors using self-organizing-maps (SOM) on knowledge domain visualization.  He introduced the "first principle of geography" that "proximity implies similarity".  Simple but subtle and powerful and very central to geospatial metaphors of information visualization.

Ramana Rao, a former Xerox PARC researcher,  started with a cartographic metaphor of terrain maps then moved us further afield into information visualization, holding only onto the metaphor of "navigation" in introducing his work in "Navigating heirarchies, wide widgets" embodied in his InXight(tm) software.   Rao also showed his contribution to the highly acclaimed book and website Understanding USA compiled by Richard Saul Wurman.  Rao is credited with two fundamental information visualization techniques, hyperbolic tree exploration and table lenses.

Mike Liebhold of the IFTF then spoke on place centered computing and Geoweb and the Evolution of Deep Place, of Spatial Awareness and Spatial Thinking.  The main thrust of this talk was about the use of GPS and other location technology along with augmented reality technology which supports the visual (and other sensory) overlay of synthetic perceptual informatin onto the direct perceptions.  This talk was a bit of a divergence from the others, having a very 1st person approach that assumes ubiquitous computing, ubiquitous location technology and non-invasive visual and auditory augmentation.  By coincidence, I believe that it was our colleague, Dr. Tom Caudell who coined the term "augmented reality" while at Boeing, developing the use of "heads-up displays" for overlaying schematics and other diagrams for workers deep inside the wings or fuselage of airplanes as they were being built. 

Implications for "us"

This report does not replace the need for a visionary "white paper" on these topics but is perhaps the seed of such a paper.   I felt very inspired during this meeting and deeply satisfied that many of the ideas I'd been working with for years had hit a critical mass of people who had managed to find eachother.

Infrastructure Analysis

Infrastructures have a confounding dual nature, being both relational/logical entities and physical entities at the same time.   The geospatial registration of infrastructures is relatively obvious if challenging.  The locational overlay of energy grids, transportation systems, financial networks, communication networks, etc.  is a very natural and important view of all things "infrastructural".  The geospatial correlation between infrastructural elements and "field" effects such as weather, natural disasters, even politics is obvious and important.  At the same time, many of these infrastructures are highly structured as "networks" where the topological properties are in their own way more important than their spatial relationships.   These two "views" of the domain are equally valid and neither should exclude the other.  Similarly, there are also "implicit" relationships across these high dimensional data sets where the connections are either obscured or unknown but can be discovered through careful data analysis.  Visual analytics and visual data mining tools come to play in seeking partial correlation between items or sets which are not connected explicitely either by geospatial proximity, nor by graph or network structure.  The rich interplay between these different views of the same data and systems is going to require the coordinated application of techniques from spatial data analysis, high dimensional data analysis and graph analysis.

Network Monitoring and Intrusion Detection

Computer networks are a very specialized but in some ways representative example of infrastructures in general.  All of the issues that arise in digital computer networks are likely to be seen in other systems as well as coupling with the other systems.  There may be a useful organism metaphor to be applied here with digital communication networks mimicing the function of the neurological systems in higher organisms, while the vascular system mirrors the transportation network and the immunological system mirrors the emergency response system, etc.

Digital networks are somewhat unique and interesting because they tend to be systems that can be explored from the "inside"... rather than observeing them from the outside (by flying over them, mapping them, imaging them in some way), they are explored more by passively monitoring the data (including timestamps, routing info, etc) itself or actively by deliberately routing information and observing it's behaviour.

It is important for many reasons (reliability and security being the most obvious) to understand computer networks from a geographic point of view.  "What country does that host that hacked into us reside?", "Will that network link go down if there is power failure at X location?", etc.

Information Visualization, Perceptualization, Visual Analytics

All of these terms apply to related problems and technological approaches.   They all are based on the assumption that the human perceptual system somehow enhances cognition or cognitive coupling with abstract systems, data sets, problems.   This workshop was most interesting to me because it had a wide range of people presenting and in attendance who understand this implicitely and are pursuing aspects of these problems, each in their own way, for their own purposes.

Those who see the world primarily through it's geographic registration are still quite interested in what is hidden from them by that approach.  Those who traditionally might not think of geolocation as being particularly relevant are working with larger data sets or with the geographically distributed (and therefore relevant?) systems they represent are becoming more interested in using and understanding the geographic knowledge that they have.  

The juxtaposition of these two somewhat different views of the world helps to enhance an awareness of the value of "place" and "location" and "orientation" and to validate to a small extent what those of us who have pushed for immersive interfaces have believed for a long time, that a "sense of presence" and/or "shared place" with data can be important and valuable and useful in exploration and analysis.


There are no specific, "aha!" type conclusions for me to report.  The experience for me was somewhat more like "coming home" than anything else.  In this case, I came "home" to a familiar place which is now populated by a lot more people who see things in a richer, more synthetic way than is common in the "real world".  I believe that this fusion of multiple ways of percieving, thinking about, and presenting information is the future. 

There is so much work left to be done in each of these fields (geospatial visualization, information visualization, scientific visualization, geolocation, augmented reality, immersive interfaces, etc) that they will continue on quite validly without neccesarily needing eachother for their independent progress.  On the other hand, there is great opportunity for those of us who do see the crossover to exploit it and to contribute across multiple fields simultaneously while drawing from each of them.

I am reinforced in my belief in using vision and other perceptions to enhance our access to data, cognition and collaboration.  I am reinforced in my belief that shared collaborative and telecollaborative environments which are rich in media can be very supportive of exploration, discovery, analysis and communication in all fields of endeavor.  Physical science and technological innovation being the most obvious and relevant to my own work, but sociology, political science and even art can equally benefit.