How to count bit patterns in a file ?

[HansWerner]

Think of the following:

You have two files that you want to compare bit by bit.

A file has been send the other one has been received.

The two files can be of different size (Connection lost).

But you not only want to know how many bits are different you want also to know the run length of the different bits.

For example:

File 1:                      1010101011101010111101011001

File 2:                      1011101010001101010111010101

What is the result ?

Xor File1 and File2:   0001000001100111101010001100

Run Length 1:           3

Run Length 2:           2

Run Length 3:           0

Run Length 4:           1

I hope I didn't make a mistake.

Think also of using a blocksize for comparison.

Maybe you don't want to compare the files as a whole but want to know the number of errors for every block that is maybe 256 bit or 1024 bit. So read the files block by block, xor the blocks and count the bit errors by their run length.

So how can I get this result in LabView ?

Which data types do I have to take for comparison ? Which VIs do I have to use ?

Thanx for help and your time

[Davidek]

Hi Hans

I tried to make a simple example for you. What I did is that I read the files one byte at a time (in the form of a I8), convert the byte to a boolean array, do the Xor on the arrays and then append the arrays for each byte. Gives you something to start with. Hope it helps.

Best Regards

David

NISW

PS. The VI is in 8.6.1, if you are using a earlier version please let me know and I'll resave it for you. DS

Message Edited by Davidek on 08-07-2009 05:57 AM

[HansWerner]

Thanks for help Davidek.

I'm using LabView 8.6.1 (Developer Version).

As I know LabView 9 is on the way.

[Davidek]

Hi Hans

Ok, then you won't have any trouble opening the VI. LabVIEW 2009 (which is 9.0 in the old designation) has already been released, you can download a trial version here. Let me know if you have any questions regarding the example.

Best Regards

David

[HansWerner]

As I have understood the example I get an array of boolean.

This array has the size of the file.

Is this correct ?

So as next I have to set a blocksize to work only with a block of boolean.

Then the file has to be read "round up(filesize/blocksize) times".

The last block can be partially empty.

How to realize the search function ?

[HansWerner]

So, here is the result.

But it is still not correct.

What is wrong ?

[Alain_S]

Can you please explain this one?

---

HansWerner wrote:

Run Length 1:           3

Run Length 2:           2

Run Length 3:           0

Run Length 4:           1

---

The XOR that I understand but the "Run Length" stuff

Can you also provide two files to compare?

[HansWerner]

Hi Allen,

thanks for your posting.

I have now a working version.

For my as LabView newbee this was really hard work.

So how many bit patterns of "1" or "11" or "111" or "1111" are in the xor result of the two files ?

Or you can also ask: How long is the error (Burst error) ? And how often does it occur ?

Look for example for block error codes and this stuff.

Attached is the VI and two small text files (Both 10 Bytes).

I've tried a blocksize of 10 Bytes and also 5 Bytes.

The result is:

1     2     3      4

16   5     1      1

So there 16 errors of run length 1, 5 of run length 2 and so on.

Would be also nice to count special bit patterns like '1001101010111" or something like that.

But I think this is more easy.

Not the pattern is searched. The first 1 and the following 0 is searched.

The run length is used as index.

[Darin.K]

My new version of LV is in the mail, so I couldn't open your VIs.  Here is a little VI which will demonstrate a couple of tasks for you.  First, it will determine the run lengths in a boolean array.  You can then select either 1/0 (T/F) to find the distribution (since you are interested in runs of 1).  I took advantage of the built-in Histogram VI and set up bins centered on integer values.  This gives you the number of times a given run length occurred.  The second task is to find the locations of a pattern in a boolean array. (If this type of task isn't the number one use of shift-registers in a for-loop, it is in the top five).

I assume you have the file reading under control, my only advice is to avoid reading the files a few bytes at a time.  Unless you are dealing with very large files, I would read them entirely and then divide the data into whatever size blocks you want to analyze.  If you are dealing with very large files I would still read in large chunks at one time to minimize file read operations. 

Good luck. 

[HansWerner]

Thanx for your contribution.

I will analyze that.

So you couldn't open the file because you don't have LabView 8.6.1 ?

I will change the file reading in a later version.

At the moment the block which is read from the file and the block which is analyzed (analysed?) are identical.

See my next posting on an interpolation problem.