Suggest that LabVIEW support HDF5

[LL Rat]

Hello,

I would like to suggest that LabVIEW support HDF5 for data format, file storage and data transfer.   I would to suggest the support include maintenance to coincide with new and future versions of HDF5.

Thank you.

[LDBM]

Hi, 

I strongly support the development of a LabVIEW toolkit taking on from the one developped by Jason Sommerville.

I use HDF5 to store critical data ( several 100MBs) with no problems. I particularly find impressive how different data types ( array, image, txt etc...) can be saved together in a logical manner that make them easily retrievable. 

The availability of wrappers found in Matlab, Python, Java.. you name it... is also a crtical point  for HDF5 support.

I also used TDMS, mostly at the LV 2009 Level, and even though they have some nice features, I had corruption issues with large files. Supposedly the 2010 and 2011 implemnentations are better, however my doubts remain...

L.

[YongqingYe]

Hi BSI,

For the TDMS corruption issue you mentioned, did you have any post on ni.com with the details? If not, is it convenient for you to explain some details or post the reproduciable VI here?

Thank you.

[galbertini]

HDF5 support in LabVIEW would be a great idea, especially for people working in organizations that have decided to standardize on this file format.

At this time, the existing LabVIEW HDF5 support is good but minimalist. It doesn't provide 'advanced' capabilities, such as variable length data (vlen) and is not supoported 'natively' by NI.

This is a major concern for me and I have to develop my "data intensive" applications in C or Matlab at this time. This lack of HDF5 support is purely and simply keeping LabVIEW out of current developments.

File formats such as TDMS are interesting but have not been considered (considered as too NI-centric or even LabVIEW-centric).

HDF5 is an open standard that is now quite mature and is supported on a large set of platforms. Its rich API and extensive documentation make it a natural choice for large data manipulation.

As for me, I see HDF5 as the XML for binary data. It has the advantages of XML (self-decriptive, extensible, API, etc...) and goes beyond simplistic binary storage (packing bits for minimal footprint, compression)

As for me, I would strongly appreciate a full and native support of HDF5 in LabVIEW !

--Gérald

[LUDO_27]

Hi, I use HDF5 with this library http://sourceforge.net/projects/h5labview/?source=navbar . It is easy to use.

Ludo

[MartinMcD]

Hmm, just posted the same idea as this one didn't come up initally in my search.  I think it is a shame that in the 4 years since this idea was suggested nothing has been implemented by NI. I don't find the usual 'well the tdms file format is published' answer that NI typically give is particularly helpful.

[pavan_b]

Thanks LUDO_27. Any feedback to share about your experience? http://sourceforge.net/p/h5labview/discussion/general/

[X.]

The feedback is: H5labview is open source and Live HDF5 is not. The latter is from a NI Alliance Partner and UNINSTALLS the former when attempting to install it through VIPM. That is not playing nicely, to say the least. But, at least you have a choice.

H5labview works fine and support by its developper has been great so far.

[carlod80]

I would also greatly appreciate official NI support for HDF5!

[jrasco]

I know the idea is fairly long in the tooth but the NI devs seem to be interested in why choose HDF5 over TDMS.  I turned to HDF5 because of my frustration with being forced to use VBScript in DIAdem to access/manipulate my TDMS files.  Breaking out of the TDMS sandbox gives me access to a multitude of excellent, well maintained, open-sourced tools.

For LabView, h5labview is the way to go although it doesn't support the latest HDF5 release (1.10.1).  The devs have hinted at updating it in the future.

Regarding the benchmarks mentioned earlier in the thread, I have searched and cannot find the published results or code that created them.  The HDF Group has updated their code to address small random I/O accesses, which appears to be where the speed issues lie.  I suspect speed may not be a relevant argument any more.  I recommend you re-run the benchmark and publish the results.  A few graphs would go a long way.

HDF5 features that I'd like to see in the TDMS spec:

1. EDITING! (data, attributes, names... all of it)
2. Compression
3. Multi-dimensional array support
4. Compound data type support (aka clusters)
5. Unlimited (or at least expanded) hierarchy
6. Softlinks (aliasing)
7. Virtual Datasets

The last one is more concerned with how TDMS files are accessed, versus being a direct file format attribute.  There may be hooks in the file spec to allow this, which is why I added it to the list.

[carlod80]

I am quite surprised (or even astonished) by NI's silence on this topic: HDF5 is the de facto open standard for scientific data exchange, and LabVIEW is the same for scientific data acquisition (albeit not open)...