Check out the Deep Sky poster in the Berkeley Lab booth (#723):     
Deep Sky Provides A Data Portal into the Data Universe

Image of the Adromeda Galaxy
created by co-adding 423 images.
Peter Nugent, Berkeley Lab

Deep Sky Project Update


Deep Sky demo on the
globe in Berkeley Lab's
booth (#723) at SC09


Deep Sky is a collaboration between the Computational Cosmology Center at the Lawrence Berkeley National Laboratory, the National Energy Research Scientific Computing Center (NERSC), and many universities and other research institutions. The overarching goal of Deep Sky is to develop an astronomical image database of unprecedented depth, temporal breadth, and sky coverage, starting with the images collected over the seven-year span of the Palomar-Quest and Near-Earth Astroid Tracking (NEAT) transient surveys. This set of images covers approximately 20,000 square degrees on the sky with ten to 100 pointings at any given set of sky coordinates (see the image below). We currently have approximately thirteen million images stored on the NERSC Global Filesystem.

All thirteen million images have now been processed, and production of the deep reference images is underway. How the images are being processed and are co-added to produce 'deep reference' images is described in a self-guided demo written for SC07. See the self-guided demo written for SC08 that describes the Deep Sky infrastructure.

Sky coverage of images from the alomar-Quest and NEAT transient surveys. The gray-white
coloring represents about 30 pointings, blue on the order of 70 pointings, green about 100, red about 200.

Update: Palomar Transient Factory

The combined software and hardware infrastructure of the Deep Sky Project has been tapped to provide real-time transient detection for the Palomar Transient Factory (PTF). The PTF survey camera system saw first light in December 2008; commissioning data were collected through June.

Deep reference images are generated when a sufficient number of processed images for a region in the sky become available. Images taken at the Palomar Observatory for the PTF transient survey are being transferred to NERSC nightly, and if there is a deep reference image corresponding to a new incoming image, then a subtraction image is made. Each subtraction image generates a list of possible transients, i.e., candidate objects in the new image that are not in the deep reference image. These candidates may be transients such as asteroids, variable stars, active galactic nuclei, novae or supernovae, but are most often spurious. On the average, the software finds 400 candidates per image with only about 20 of them being real.

Tapping into a wealth of experience in transient surveys around the world, we constructed a Groupthink Web site in which fifteen astronomers visually inspected a wide variety of randomly selected candidates in an effort to classify their quality based upon over 30 parameters recorded in our database on each of them. The candidates were selected based on physical inspection of all candidates discovered over a two-week period early in the commissioning phase of the survey. The results of the Groupthink effort were passed to the Weka 3 machine learning code to classify the nature of the candidates as either real or bogus. Between the beginning of March and the end of June, over 2,000 optical transients were detected and followed-up using this commissioning data (this does not include the discovery/re-discovery of over 30,000 asteroids). PTF's first confirmed supernova, SN2009av, is shown below.

These early supernova discoveries were made possible because the Deep Sky infrastructure already was in place and is flexible in design. We were able to quickly extend the processing algorithms and database design to accommodate the specific requirements of the PTF Project. We added tables to support detection and classification of candidate transient objects and locating candidates within the footprint of nearby galaxies.

Discovery of Supernova SN2009av. The two images on the left show the portrait of a distant
galaxy taken on different days in late February.  The two images on the right are subtraction
images that result from subtracting newly taken images (February 28 and March 2) from deep
ref images. The rightmost image shows a brightening of the supernova over the two days shown.

Links to More Information

For more information about Deep Sky and the Palomar Transient Factory:
         Peter Nugent's Deep Sky Web site
         NERSC's Deep Sky Project Provides a Portal into Data Universe
         NERSC Helps Discover Cosmic Transients
         The Palomar Transient Factory: System Overview, Performance and First Results
         Exploring the Optical Transient Sky with the Palomar Transient Factory