LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

U8 Data compression/decompression

Hello community :-),


I am struggling with the following issue:

I have a file which contains of a mix of ASCII and binary data.
I extracted the binary data successfully.
However, I noticed that my colleague had compressed the binary data. He mentioned that he had used a Windows ZIP algorithm (in C language).
I am going to implement a specific read out for these files in LabVIEW and I need some hint to overcome the compression.
In LabVIEW there are just VIs available to open, add and create ZIP files. But my file is a self made format file type.
Has any of you some ideas how I can (un)compress these U8 data but not the whole file?

Thanks in advance


Regards
D-Kid

0 Kudos
Message 1 of 11
(4,096 Views)

OpenG Zip library, download from VIPM.

 

mcduff

0 Kudos
Message 2 of 11
(4,065 Views)

Hi,
at first, thank you very much for your message, mcduff.

However, I am afraid that I cannot use these suggested VIs due to the fact that these VIs dealing with ZIP files only.
I do not want to generate or handle with ZIP files at all.

I extracted U8 data and stored them temporally in a binary text file. I did this just for programming reasons.
Further, I do not want to compress a file but just the data in it.
Or even better: if I have an U8 array in LabVIEW I do want to (un)compress the data without to generate an additional file.

It looks for me like I have to implement the ZIP algorithm by my own. What do you think?

Regards

DKid

 

 

0 Kudos
Message 3 of 11
(4,044 Views)

I think whoever thought up this method of sending data is a sadist.  😉  That's what I think.

Bill
CLD
(Mid-Level minion.)
My support system ensures that I don't look totally incompetent.
Proud to say that I've progressed beyond knowing just enough to be dangerous. I now know enough to know that I have no clue about anything at all.
Humble author of the CLAD Nugget.
0 Kudos
Message 4 of 11
(4,041 Views)

Hi D-Kid,

 

there are examples here in the forum explaining how to use .NET functions to compress/uncompress strings (or U8 arrays) in memory.

Search for them…

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
Message 5 of 11
(4,037 Views)

To add to Gerds answer, the .NET functions works on Streams, with is a string. So just extract the compressed part and unpack that. Though, why didn't your collegue just zip everything?

/Y

G# - Award winning reference based OOP for LV, for free! - Qestit VIPM GitHub

Qestit Systems
Certified-LabVIEW-Developer
0 Kudos
Message 6 of 11
(4,031 Views)

I searched for the .NET solution and at the moment I am struggling with some error but the main idea should work. thanks for that.

The current error 1172 does not occur in LabVIEW 32bit version but I am working in 64bit to use as much memory as possible. The files can be huge. Further, the error 1172 occurs just in the decompression vi.

The main thread can be found here, but I did not went through it completely yet.
https://forums.ni.com/t5/Example-Program-Drafts/GZip-Compression/ta-p/3534832

I will keep you up to date!

@Yamaeda:
The reason why the whole file is not ZIPPED completely is that the selfmade file type consists of two parts, ASCII and binary data.
The binary data represents the collected data from some hardware and just the combination of both, ASCII and binary data can be read only from our main software. So it has historical issues.
Maybe you are right and we should think about change.

0 Kudos
Message 7 of 11
(4,020 Views)

The OpenG ZIP tools do not only work for files. They also expose the compress and inflate routine from the ZLIB library that is used internally in the ZIP algorithmes. So it may still be a solution depending on how "sadistic" that programmer was?

 

That said, it sounds a lot like LabVIEW files. They compress most parts inside the LabVIEW files with the ZLIB compression algorithme too.

Rolf Kalbermatter
My Blog
0 Kudos
Message 8 of 11
(4,014 Views)

@D-Kid wrote:

Hi,
at first, thank you very much for your message, mcduff.

However, I am afraid that I cannot use these suggested VIs due to the fact that these VIs dealing with ZIP files only.
I do not want to generate or handle with ZIP files at all.

Regards

DKid

 


yes, you can:

0.png

0 Kudos
Message 9 of 11
(4,003 Views)

Hi everyone,

I want you firstly wish you a happy new year!

After some couple of weeks I am now able to take care about this problem again.

@alexderjuengere
Thank you for the showing 🙂
You are absolutly right. I could use it, but...


Regretly, this function gives me always an error 42. Which means that something went wrong during the unzip step. Further, it is also just available in a 32bit environment but I am going to use a 64bit environemnt due to the huge amount of data which can occur. Therefor I can"t use these functions for my purpose.

I also talked with my colleague again and he mentioned, that he had used the unzip/zip functions from a LiteZip.dll, a free source compressen/decompression library.
In my point of view, the functions contained in the LiteZip.dll are the quite same which are used in the ZLIB VIs.
In your point of view: is this right?
If so, any ideas except to create some VIs from the LiteZip.dll?

0 Kudos
Message 10 of 11
(3,921 Views)