Converting and Porting SPICE Binary Data Files

Revisions

January 13, 2002

The contents of this guide were extensively revised to reflect the inclusion of the DAF run-time binary file format translation capability into the SPICE toolkit.

Introduction

This document describes issues related to moving SPICE binary data files from one computing environment to another. Often a SPICE data file (or kernel) is generated within one particular environment for use in another. NAIF has developed software that allows users to read binary SPICE data files generated on most supported platforms by simply transferring the file directly. In a few cases, ``transfer format'' software, available in the SPICE Toolkit, must be used. The remainder of this document discusses when binary kernels may be exchanged directly, what restrictions exist in using files from other computing environments, and the mechanisms for exchanging data with users of environments that do not support this capability.

Binary File Format

SPICE binary data files are Fortran direct access unformatted files, i.e., they contain data in a raw binary form. The specifics of this form, henceforth referred to as binary file format, are determined by characteristics of the computing environment used to generate the file. Factors such as word size and floating point representation control exactly how raw data are placed into the file at the byte level. Computing environments that share the same binary file format may freely exchange SPICE binary data files as though the original files were produced in their environment. Files whose binary file format agrees with the local computing environment are referred to as native format files, or just native files. Similiarly those whose format disagrees, are commonly referred to as non-native.

The collection of computing environments for which SPICE toolkits exist (often called supported environments) utilize four distinct binary file formats: Big Endian IEEE, Little Endian IEEE, Vax D-Floating, and Vax G-Floating. The breakdown by computing environment is as follows:

 
   Big Endian IEEE:
   ---------------
      HP:
         HP Fortran, HP C
      Macintosh:
         Absoft Fortran, Metrowerks Codewarrior C
      SGI:
         N32/O32 Fortran, MIPS C (N32/O32 ABIs)
      Sun:
         Solaris Fortran, Solaris Native C, Solaris GCC
 
   Little Endian IEEE:
   ------------------
      PC:
         Linux G77, Compaq Visual Fortran, Lahey Fortran 95,
         Linux GCC, Microsoft Visual C++
      Alpha (Digital Unix):
         Digital Fortran, Digital C
 
   Vax D-Floating:
   --------------
      Alpha (VMS):
         Digital Fortran
      VAX:
         Digital Fortran
 
   Vax G-Floating:
   --------------
      Alpha (VMS):
         Digital Fortran
 

Reading Non-native Binary Data Files

The preceding section implies that users of computing environments utilizing dissimiliar binary file formats may not exchange binary files directly. In general this is true, however, the Toolkit software on most supported computing environments includes a run-time binary file format translation, or binary translation, capability. This enables these computing environments to read some non-native binary files, with some restrictions noted in the section that follows.

The diagram below illustrates which computing environments are capable of reading non-native files. The arrows in the diagram indicate the direction in which the files may be exchanged.

 
    ----------              ----------
   | Big      |----------->| Little   |
   | Endian   |            | Endian   |
   | IEEE     |<-----------| IEEE     |
    ----------              ----------
 
 
    ----------              ----------
   | Vax      |            | Vax      |
   | D-       |            | G-       |
   | Floating |            | Floating |
    ----------              ----------
 

However, users on these systems may not read binaries produced in either Vax D-Floating or Vax G-Floating environments. For file exchange between these environments the SPICE Toolkit provides the mechanisms discussed in the following chapter, using the ``SPICE Transfer format.''

Restrictions on Using Non-native Binary Data Files

The binary translation capability provides access to non-native binary files on most supported environments. However, there are several restrictions that require noting:

-- As of Toolkit Version N0052 this capability is implemented only for DAF based files (SPK, CK, and binary PCK). Users of E-kernels and other DAS based kernels will need to await a future release of the toolkit before this capability is extended to these files.

-- The binary translation capability provides ``read-only'' access to non-native binary files. Attempts to add, alter, or remove comments from non-native binary files (using tools such as COMMNT) will fail. Also, appending or altering the data contents of non-native binary files (using MKSPK or other mission specific tools) will also fail.

-- Using non-native files may impact the performance of SPICE based applications. The impact is highly dependent on the particular computing environment and the usage scenario. Typically the decrease is somewhere between 5 and 10 percent, but may be as large as a factor of 2 or 3 for highly read intensive scenarios.

First convert the non-native binary to transfer format using TOXFR:

    > toxfr non-native.bsp transfer.xsp

    > tobin transfer.xsp native.bsp

Utility Programs for Porting SPICE Files.

If it is the case that one of the computing environments generating or using files is not supported for binary translation, several tools are delivered with the SPICE Toolkit to address this issue.

In these situations, the portability of SPICE binary data files is achieved by using an intermediate file called a ``transfer'' file. A transfer file contains an environment independent representation of the data from a binary SPICE file that consists only of ASCII characters, and hence is easily moved from one computing environment to another. There are three steps involved in moving the information in a SPICE binary data file from a host, or originating, computing environment to a target, or destination, computing environment.

1. Convert the SPICE binary data file into the environment independent transfer file in the host computing environment. The utility programs described later in this document provide this capability.

2. Move the environment independent transfer file to the target computing environment. (Typically using ASCII mode of FTP).

3. Convert the environment independent transfer file into a SPICE binary data file in the target computing environment. The utility programs described later in this document provide this capability.

The Two Conversion Methods

There are two methods for converting SPICE data files from one of their formats to the other; an interactive method and a command line, ``or batch,'' method. The interactive method makes use of the SPICE Toolkit utility program SPACIT, and the command line method makes use of the SPICE Toolkit utility programs TOXFR and TOBIN which convert files from the binary format to the transfer format and from the transfer format to the binary format, respectively. (Using TOXFR and TOBIN is usually faster than using SPACIT.)

TOBIN and TOXFR -- Command Line Based File Conversion.

The utility programs TOBIN and TOXFR provide a command line, or ``batch,'' oriented method for converting portable transfer files into SPICE binary kernel files and converting SPICE binary kernel files into portable transfer files, respectively. The name ``tobin'' is obviously derived from ``to binary,'' as in ``convert to binary''. The name ``toxfr'' is, possibly not so obviously, derived from ``to transfer,'' as in ``convert to transfer.''

TOBIN converts a SPICE transfer format file into it's binary format for use in a particular computing environment. TOBIN requires at least the name of the input transfer file, which is the first argument on the command line. Optionally, a name for the output binary file may be specified as the second argument on the command line. If a name for the output file is provided it overrides the automatic naming conventions used by the program.

If only the name of the input transfer file is provided on the command line, TOBIN will generate a name for the output binary kernel file that has the same base name as the input transfer file and an appropriate filename extension, based on the filename extension of the transfer file. If TOBIN does not recognize the filename extension of the input transfer file, or there is no filename extension on the input transfer file, a filename extension of '.bin' will be used as the filename extension of the output binary file. If a file having the same name as the output file already exists, TOBIN signals an error and stops. We assume that it is bad form to overwrite or replace an existing file.

TOBIN recognizes the transfer filename extensions ``.xc'', ``.xee'', ``.xpc'', and ``.xsp'' associated with transfer files for SPICE data products. These are converted to the SPICE binary kernel filename extensions ``.bc'', ``.bee'', ``.bpc'', and ``.bsp,'' respectively. Also recognized are the old transfer file extensions ``.tc'', ``.tee'', ``.tpc'', and ``.tsp,'' which are converted into the appropriate filename extensions for a binary kernel file.

TOXFR converts a SPICE binary kernel file into a transfer file format that is used to transfer, or port, the binary data to a different computing environment. TOXFR requires at least the name of the input binary file, which is the first argument on the command line. Optionally, a name for the output transfer file may be specified as the second argument on the command line. If a name for the output file is provided on the command line it overrides the automatic naming conventions used by the program.

If only the name of the input binary file is provided on the command line, TOXFR will generate a name for the output transfer file that has the same base name as the input binary file and an appropriate filename extension, based on the filename extension of the binary file. If TOXFR does not recognize the filename extension of the input binary file, or there is no filename extension on the input binary file, a filename extension of '.xfr' will be used as the filename extension of the output transfer file. If a file having the same name as the output file already exists, TOXFR signals an error and stops. Again, we assume that it is bad form to overwrite or replace an existing file.

TOXFR recognizes the filename extensions ``.bc'', ``.bee'', ``.bpc'', and ``.bsp'' which are associated with SPICE binary kernel files. These are converted to the transfer filename extensions ``.xc'', ``.xee'', ``.xpc'', and ``.xsp,'' respectively.

As mentioned previously, it my not be necessary to convert a SPICE binary kernel file into the transfer file format when moving the data from one computing environment to another. If the binary file formats in each of the computing environments are identical, the same binary file may be used without modification. The direct exchange of SPICE binary kernel files is known to work when moving the files among Sun Sparcstation, HP 9000, SGI, and Macintosh computing environments. This is possible because they all use standard UNIX file formats and IEEE representations for floating point numbers.

Examples Using TOBIN and TOXFR.

TOBIN and TOXFR each require the name of the file to be converted and, optionally, the name of the file to be produced, e.g.,

   prompt > tobin thisfile.xsp

   prompt > tobin thisfile.xsp thisfile.bsp

In the first example, the name for the binary file produced is derived from the name of the transfer file, using the full base name and replacing the letter `x' in the file name extension with the letter `b', to indicate that the file is a binary file. In the second example, the name for the binary file produced is taken directly from the command line.

-- Example 1: Convert the transfer file 'ephem.xsp' into a binary kernel file.

    > tobin ephem.xsp

This will create the binary kernel file 'ephem.bsp'.

-- Example 2: This is the same as Example 1, except we explicitly set the output filename on the command line.

    > tobin ephem.xsp myephem.bsp

This will create the binary kernel file 'myephem.bsp'.

-- Example 3: For this example, we provide a filename extension on the input transfer file that TOBIN does not recognize.

    > tobin mydata.ttt

This will create the binary file 'mydata.bin'.

-- Example 4: Convert the binary kernel file 'ephem.bsp' into a transfer file.

    > toxfr ephem.bsp

This will create the transfer file 'ephem.xsp'.

-- Example 5: This is the same as Example 1, except we explicitly set the output filename on the command line.

    > toxfr ephem.bsp myephem.xsp

This will create the transfer file 'myephem.bsp'.

-- Example 6: For this example, we provide a filename extension on the input binary file that TOXFR does not recognize.

    > toxfr mydata.bbb

This will create the transfer file 'mydata.xfr'.

SPACIT -- Interactive File Conversion.

The SPACIT utility program is an interactive, menu driven program that provides a file conversion capability, among other things. The SPACIT main menu has an option for converting a SPICE binary kernel file into a portable transfer file, and an option for converting a transfer file into a binary kernel file.

            SPACIT Options
 
      ( Q ) Quit.
      ( L ) Log SPACIT output to a file.
      ( T ) Convert transfer file to binary file.
      ( B ) Convert binary file to transfer file.
      ( S ) Summarize binary file.
      ( R ) Read comment area of binary file.
 
      Option:

Please see the SPACIT User's Guide for complete details on the use of this program.