[an error occurred while processing this directive]
The DPSS is a network-accessible, "block data" device. Network-accessible means that the DPSS, as a data source, is located "somewhere out on the network." "Block data" device means that the DPSS is intended to be used for large, block data transfers. According to its designers, the DPSS performs best when data is moved in 16MB blocks per request. The API for the DPSS looks like a low-level UNIX file library, so it has routines for things like "file open," "file write," "file read" and "file seek." One of the DPSS claims to fame is that it is an economically scalable storage system - consider a federation of Linux-based PC's, each with a large IDE disk as the DPSS back end, and a front end (a master) that can propogate client data requests from the client to the DPSS workers.
Which DPSS should you use? That's a good question, and there's no easy answer. As of the time of this writing, there are only a couple of experimental DPSS systems in existence. Since these are experimental, there are no claims of production level quality, with features such as security or reliability. This situation may change in time, but for now, you have to hang out with the right people to know which DPSS to use.
Given that the DPSS is a block-level device, the real challenge in making most efficient use of this resource is to collect large amounts of data into large homogeneous blocks, and to transfer as much as possible in as few calls as possible. To this end, there is a "DPSS data format" used by Visapult. This data format is certainly not the most advanced design, but only serves to make the most efficient use of the DPSS by fostering a small number of large payload transfers.
The two components of a Visapult DPSS data set are a "config file" and the accompanying "large data" files. When you run the Visapult back end, and request DPSS as a data source, you will provide the fully-qualified name of this config file (more on running Visapult will come later in this document).
The config file is a plain old ASCII file that contains information about the size of the problem, what variables are present along with their names, and other information. The format of this file is subject to change without notice. In contrast, the large data files are raw binary IEEE big-endian floats that contain ALL the data for a given species for ALL time steps. More information about these files will follow.
Here's a sample config file, which we'll call "explosion-config":
explosion 640 256 256 10 1 density 1.34088 2.64286
The Visapult Back end will read the config file at start up time, and from the config file, construct a file name of the form:
problemName-variableName when it goes looking for data. From the example above, the Visapult back end will be expecting to find a file called explosion-density on the same DPSS as the config file.