Dr. Damien's Development - Object Based File I/O II - Requirements

[DFGray]

Last time, we looked at some representative file types. This time, we will look at requirements.

Interface

1. File objects should be as interchangeable as possible. One of the primary purposes of this exercise is to allow easy interchangeability of file types in the code.
2. API should be as flexible as reasonably possible. Given the choice between flexible and ease–of–use, flexibility wins. Ease–of–use can be gained by combining multiple, harder–to–use pieces together.
3. API will consist of several layers — expert, intermediate, and easy. The expert layer will be the focus of this exercise due to the emphasis on flexibility. However, the intermediate and easy levels will be considered during the design.
4. All layers will be usable together.
5. As the layers get easier, they will probably more resemble the current APIs for the particular file type, but it is expected that the interchangeability will drop. This drop in interchangeability will be avoided as much as possible.

Data Handling

1. All LabVIEW numeric data types will be supported.
2. Timestamps will be supported.
3. Waveforms of all flavors will be supported.
4. Strings will be supported.
5. Clusters may be supported. Their use can generally be supported through use of the base data types.
6. Objects may be supported through flattened strings.
7. Arrays and matrices of arbitrary dimensions of all supported data types (which support arrays and matrices) will be supported.
8. Compression will be supported.
9. Audio and video codecs will be supported.
10. Decimation for display will be supported.
11. Security will be supported as needed by the file types.
12. Arbitrary metadata will be supported. Metadata is ancillary data which is used in conjunction with the main data. For example, the timestamp and period are metadata to a waveform.
13. Arbitrary hierarchy in the file will be supported.
14. Fixed hierarchy in the file will be supported.
15. Compressed files will be supported (e.g. ZIP or bz2).

File Formats

1. ASCII file types, such as LVM, TDM, and XML, will be supported.
2. “Unstructured” binary will be supported.
3. Image file types, such as PNG and JPEG, will be supported.
4. Audio file types, such as WAV and MP3, will be supported.
5. Video file types, such as AVI and MPEG, will be supported.
6. Scientific file formats, such as FITS, DICOM, and HDF5 will be supported.
7. LabVIEW native file types, such as LVM, DataPlugin, and TDMS, will be supported.
8. Office productivity software file types, such as XLSX and ODC will be supported.

Note that support means that it can be relatively easily added to the framework, not that it will actually be written in the next few weeks. There are many man–years of work in the above list. Support is also contingent upon support in the underlying file type. I will be implementing a small subset of it, but hopefully a useful core that others can build upon. I spent a month exposing a fraction HDF5 in LabVIEW many years ago. It would probably take equal work to expose any of the complex file formats given above or in last week's list, such as FITS or TIFF. In addition, the easy availability of codecs and the proprietary nature of many of them will preclude easy implementation of many of the audio and video file types, although personal use of such frameworks as ffmpeg may make this much easier.

Previous Posts In This Series:

• Representative File Types with conversation on design

[Ben]

My comments and requests;

1) I Agree that we/you are just going to start us off in the right direction>>> not all of the above will be implemented.

2) LV 2009 and higher please.

3) A design document showing where the other version will fit into the puzzle.

4) A "tester" for each version of file that will verify it can operate in an exe (moving to a built version is something I need to learn more about since paths can change and dynamic loading of Classes needs to be taken into concideration)

5) Repeating (sorry) images of the Classes and how they will interact. UML and LV are different critters and translating from one to another is not always straight forward (where does THIS instance of the object live? in a SR in a Class in a SR etc)

Ben

[DFGray]

The next part in this series, top-level design, can be found here.