Previous versions of AVS/Express had limitations on array sizes and field sizes on 64-bit platforms. For example, the number of nodes in a field was represented by a 32-bit integer and was thus limited to about 2.3 billion (2^31 - 1, to be exact). There were similar limitations on array sizes - the total number of elements in an array could not be larger than 2.3 billion.
In AVS/Express 7.0, the core Object Manager, the OM APU, the field data structure and FLD API, and visualization modules have been rewritten to remove these restrictions. Quantities like the number of nodes in a field and array dimensions are now represented by 64-bit integers on 64-bit platforms. For practical purposes, the sizes of fields and arrays are now limited only by the amount of memory and processing power of the computer running AVS/Express 7.0.
New OM API and FLD API function calls are available so that customer written code can take advantage of the new functionality. Some OM and FLD calls are now available in two versions - a "narrow" version for backward compatibility or a "wide" version for customer code that needs to handle very large datasets. Customers can choose which API they would rather use.
On a 64-bit platform, the OM now always runs in "wide mode". The underlying OM data structures and field data structures defined in fld.v are the same no matter which API you use. The backward compatible "narrow" API is implemented as a wrapper around the "wide" API that translates, as necessary, between 32-bit and 64-bit values. There are no restrictions on mixing code that uses the different API's, thus converting to the "wide" API can be done on a one module at a time, or even one file at a time. Customers can incrementally convert existing code, starting with only the code that needs to be able handle very large datasets.
Some data file formats have been expanded to handle fields with more than 2.3 billion nodes. The field file format (.fld) and HDF5 (.h5) format were expanded without any compatibility issues. The UCD (.inp), geom (.geo), and GFA file formats were expanded, but do have compatibility issues. For those three formats, you now have to explicitly choose whether to use a 32-bit or 64-bit versions of the format.
Developers migrating applications to AVS/Express 7.0 have several different choices for porting their source code. Code that only needs to run on 32-bit platforms need not be altered at all. Code that runs on 64-bit platforms probably will have to be altered but developers can choose whether to do a minimal conversion or a full conversion. Full conversion is only required for code that will handle fields with more that 2.3 billion nodes or handle arrays with more than 2.3 billion elements. However, there are some situations where there is a performance loss associated with doing only a minimal conversion.
The standard AVS/Express header file avs/port.h defines a new C type - "xp_long" -
that is the key to the porting process. It is defined to be a standard 32-bit integer on 32-bit platforms
and a 64-bit integer on 64-bit platforms. Additionally the new type "long" has been introduced
into the V language. It is the counterpart to the C type "xp_long" - it is a 32-bit integer on
32-bit platforms an a 64-bit integer on 64-bit platforms.
The new V "long" type is used in fld.v to widen several parts of the field data structure. The number of nodes in a field and the number of cells in a cell set are now defined as V as longs. Additionally, a structured field's dimensions array and a cell set's node connect list and poly connect list are now defined in V as arrays of long.
Most of the porting process involves changing variable declarations in C/C++ code from "int" to "xp_long". Since these types are actually exactly the same on a 32-bit platform, no porting is needed for code that runs only on 32-bit platforms. A minimal conversion means to only alter code that accesses the V field data structures that have been widened. A full conversion means to also use the new types to handle all array dimensions.
No changes are needed.
Most of the code can work without modifications by using the backward-compatible "narrow" versions of the OM and FLD APIs. However, there are a few situations that require modifications. Some of the underlying field data structures have changed in a manner that can not be completely hidden by the backward-compatible APIs. The situations to watch out for are direct access to V data structures that have been widened, most notably a field's dims array and a cell set's node_connect_list or poly_connect_list array.
A characteristic of how the OM handles array type conversion needs to be understood. The OM has always had an ability to do "on-the-fly" type conversions as needed. If a certain V variable is declared as a float in V and you read the value by calling OMget_int_val(), then the OM returns the value after truncation to an integer. The same holds for arrays, but there is an important asymmetry when using OMget_array(). Type conversion works when accessing an array in read-only mode, but not in read-write mode. When you ask for an array in read-only mode and the requested type does not match the actual type, then the OM will generate a temporary array of the requested type and copy the array values into the temporary array, and finally return the temporary array. This is similar to what happens when you request an array in read-copy mode. However, an analogous procedure is *not* available when accessing an array in read-write mode. The reason is that ARRfree() does not have to ability to convert the type of the requested array back to the type of the OM's copy of the array. If this was implemented, there would be potentially be a lot of overhead because the array would be copied and converted twice over. It is possible to evade these restrictions by getting the array in read-copy mode and then calling OMset_array(), but usually it will be easier to adjust your code for the widened type of a few key arrays in the field data structure.
The most common scenario where changes are required involves C++ code. The following code is now incorrect on 64-bit platforms because the dims array is now an array of 64-bit integers.
int *dims = (int *)field.dims.ret_array_ptr();
...
ARRfree(dims);
The following will work correctly on both 32-bit and 64-bit platforms.
xp_long *dims = (xp_long *)field.dims.ret_array_ptr();
...
ARRfree(dims);
A cell set's node_connect_list and poly_connect_list should be handled in a similar manner.
int cs = 0;
xp_long *conn_list = (xp_long *)field.cell_set[cs].node_connect_list.ret_array_ptr();
...
ARRfree(conn_list);
From C code, a cell set's node_connect_list and poly_connect_list are usually accessed using FLD API calls. The following is the original code in read-only mode. This code does not have to be converted, because in read-only mode the OM will translate the long array to an int array. For a large field, there can be a performance loss from the translation.
int *node_conn = NULL;
int size;
int mode = OM_GET_ARRAY_RD;
FLDget_node_connect(cell_set, &node_conn, &size, mode)
...
ARRfree(node_conn);
The following is the original code in read-write mode. This code will *not* work, because the OM will not translate the array in read-write mode.
int *node_conn = NULL;
int size;
int mode = OM_GET_ARRAY_RW;
FLDget_node_connect(cell_set, &node_conn, &size, mode);
...
ARRfree(node_conn);
There is a problem here, because there appears to no way to get the connection array in read-write mode. Calling FLDget_node_connect in read-only mode and then calling FLDset_node_connect, would get around the problem, but there is a possibility of a heavy performance penalty for large fields. For the node_connect_list and poly_connect_list, there are special "wide" versions that are accessible when using the backwards-compatible API.
xp_long *node_conn = NULL;
xp_long size;
int mode = OM_GET_ARRAY_RW;
FLDget_node_connect_w(cell_set, &node_conn, &size, mode);
...
ARRfree(node_conn);
There is no need for a FLDget_dims_w function. FLDget_dims does not allow the caller to choose an access mode - the dims array is always accessed in read-only mode. Furthermore, FLDset_dims must be used to alter the dims array. When using the backwards-compatible API, the dims array is translated back and forth between an long array and an int array, but as the dims array is always very small, the performance loss is negligible.
A less common scenario involves direct access to the array from C code using OM calls. The following code will work correctly when getting the "dims" array in read-only mode, because the OM will automatically translate the type of the array. However, the OM will not automatically translate the type of the array when getting the array in read-write mode.
int dims[OM_ARRAY_MAXDIM];
int type = OM_TYPE_INT;
int ndim = 0;
int mode = OM_GET_ARRAY_RW;
int *fld_dims;
elem_id = OMfind_subobj(field, OMstr_to_name("dims"), OM_OBJ_RW);
OMget_array(elem_id, &type, (char **)&fld_dims, &ndim, dims, mode);
The following is the correct version.
int dims[OM_ARRAY_MAXDIM];
int type = OM_TYPE_LONG;
int ndim = 0;
int mode = OM_GET_ARRAY_RW;
xp_long *fld_dims;
elem_id = OMfind_subobj(field, OMstr_to_name("dims"), OM_OBJ_RW);
OMget_array(elem_id, &type, (char **)&fld_dims, &ndim, dims, mode);
C code that accesses these arrays using OMret_typed_array_ptr() should be handled in the same manner as above.
C code that accesses these arrays using calls to OMret_array_ptr() and OMret_name_array_ptr() is similar, but slightly different in that there is no ability to specify a new type in read-only mode. The following does *not* work as intended because OMret_array_ptr does not look at your value of "type". It gives you an array of type OM_TYPE_LONG and then sets "type" to match.
int *size;
int type = OM_TYPE_INT; /* Oops - this is ignored! */
int mode = OM_GET_ARRAY_RD;
int *fld_dims;
elem_id = OMfind_subobj(field, OMstr_to_name("dims"), OM_OBJ_RW);
fld_dims = OMret_array_ptr(elem_id, mode, &size, &type);
/* now, type == OM_TYPE_LONG */
The following is the correct version.
int *size;
int type = OM_TYPE_UNDEF;
int mode = OM_GET_ARRAY_RD;
xp_long *fld_dims;
elem_id = OMfind_subobj(field, OMstr_to_name("dims"), OM_OBJ_RW);
fld_dims = OMret_array_ptr(elem_id, mode, &size, &type);
/* now, type == OM_TYPE_LONG */
Again, when doing a minimal conversion, these changes are only necessary for the handful of key arrays in fld.v that have been widened.
Code that fits in this category needs more extensive modifications. Most of the work involves changing variable declarations from "int" to "xp_long". The variables that need to be changed are variables that are used to represent:
Conversion is done on a file by file basis. The underlying OM data structures and field data structures defined in fld.v are the same no matter which API you use. A call to FLDget_nnodes(OMobj_id, int *) and a call to FLDget_nnodes(OMobj_id, xp_long *) end up going to the same underlying OM routines and accessing the exact same value. The only difference is that the backwards-compatible version that takes a "int *" will return a incorrect result if the relevant field has more that 2.3 billion nodes. You need to remember that the backwards compatible API is the default. You start the process of switching to the new wide API by defining the pre-processor symbol XP_WIDE_API *before* any AVS/Express header files are included.
#include <stdio.h>
#define XP_WIDE_API /* Use the full 64-bit API */
#include <avs/om.h>
Most of the changes involve calls that access arrays. It is very important is distinguish between accessing the handful of key arrays in fld.v that have been widened versus accessing arrays that have not been widened.
The first example show an array that has been defined by the application, not AVS/Express. Consider an array defined in the application's V code as "int my_array[]". Its type has not changed, although the call used to access it has changed.
int my_dims[OM_ARRAY_MAXDIM];
int type = OM_TYPE_INT;
int ndim = 0;
int mode = OM_GET_ARRAY_RW;
int *my_array;
elem_id = OMfind_subobj(field, OMstr_to_name("my_array"), OM_OBJ_RW);
OMget_array(elem_id, &type, (char **)&my_array, &ndim, my_dims, mode);
The following is the converted version.
xp_long my_dims[OM_ARRAY_MAXDIM]; /* changed */
int type = OM_TYPE_INT; /* NOT changed - still an int */
int ndim = 0;
int mode = OM_GET_ARRAY_RW;
int *my_array; /* NOT changed - still an int */
elem_id = OMfind_subobj(field, OMstr_to_name("my_array"), OM_OBJ_RW);
OMget_array(elem_id, &type, (char **)&my_array, &ndim, my_dims, mode);
The only difference with AVS/Express 7.0 is that there is no 32-bit limit on the array size, thus the "my_dims" array needs to be wider on 64-bit platforms.
The following example looks similar, but with an important difference. The type of the array itself has changed.
int my_dims[OM_ARRAY_MAXDIM];
int type = OM_TYPE_INT;
int ndim = 0;
int mode = OM_GET_ARRAY_RW;
int *fld_dims;
elem_id = OMfind_subobj(field, OMstr_to_name("my_array"), OM_OBJ_RW);
OMget_array(elem_id, &type, (char **)&my_array, &ndim, my_dims, mode);
The following is the correct version.
xp_long my_dims[OM_ARRAY_MAXDIM]; /* changed */
int type = OM_TYPE_LONG; /* changed */
int ndim = 0;
int mode = OM_GET_ARRAY_RW;
xp_long *fld_dims; /* changed */
elem_id = OMfind_subobj(field, OMstr_to_name("my_array"), OM_OBJ_RW);
OMget_array(elem_id, &type, (char **)&my_array, &ndim, my_dims, mode);
In C code, it is more common to access the FLD.dims array using the FLD API. The resulting call is simpler, because the FLD API can make some assumptions about the type and size of the FLD.dims array.
Original code.
int *dims;
int ndim;
FLDget_dims(field, &dims, &ndim);
Converted version.
xp_long *dims; /* changed */
int ndim;
FLDget_dims(field, &dims, &ndim);
Here is another example of a converted FLD call. The type of the node data array has not changed, but the maximum allowed size is no longer constrained to be a 32-bit integer.
Original code.
int comp = 0;
int type = OM_TYPE_FLOAT;
float *node_data = NULL;
int size;
FLDget_node_data(field, comp, &type, &node_data, &size, OM_GET_ARRAY_RD);
...
ARRfree(node_data);
Converted version.
int comp = 0;
int type = OM_TYPE_FLOAT; /* still float */
float *node_data = NULL;
xp_long size; /* changed */
FLDget_node_data(field, comp, &type, &node_data, &size, OM_GET_ARRAY_RD);
...
ARRfree(node_data);
Here is another example that shows more extensive changes. Both the type of the underlying array (node_connect_list) *and* the maximum possible size of the array have changed.
Original code.
int *node_conn = NULL;
int size;
int mode = OM_GET_ARRAY_RD;
FLDget_node_connect(cell_set, &node_conn, &size, mode);
...
ARRfree(node_conn);
Converted version.
xp_long *node_conn = NULL; /* changed */
xp_long size; /* changed */
int mode = OM_GET_ARRAY_RD;
FLDget_node_connect(cell_set, &node_conn, &size, mode);
...
ARRfree(node_conn);
The key type "xp_long" is defined by AVS/Express version 6.3. This means that code altered to use the new "wide" API should work with AVS/Express version 6.3 without changes. In version 6.3, "xp_long" is just defined to be a regular 32-bit integer, so the restrictions on field size and array size still apply.
Note that there is no object code compatibility on 64-bit platforms. When switching between AVS/Express 6.x and AVS/Express 7.x, you must make sure that all source code that make OM or FLD calls is recompiled.
For source code compatibility with versions of AVS/Express older than 6.3, add the following code *after* the AVS/Express header files.
#ifndef XP_HAS_LONG
#define XP_HAS_LONG
typedef int xp_long;
typedef unsigned int xp_ulong;
#endif
Although the following is not a complete source file, it shows a fairly representative set of header files and defined to use in source code that uses the new 64-bit API and additionally is still source code compatible with older versions of AVS/Express.
#include <stdio.h>
#include <stdlib.h>
#define XP_WIDE_API /* Use the full 64-bit API in AVS/Express 7 */
#include <avs/om.h>
#include <avs/fld.h>
/* Other AVS/Express headers files */
#ifndef XP_HAS_LONG /* Skip in AVS/Express 6.3 and later */
#define XP_HAS_LONG /* Module can be compiled in AVS/Express 6.2 and earlier */
typedef int xp_long;
typedef unsigned int xp_ulong;
#endif
The following OM API calls are new.
The following OM API calls have been widened.
The following FLD API calls are new.
The following FLD API calls have been widened.