Delete Array Duplicate for only one column

[SimpleJack]

Hi all,

So I am trying to work out a problem here. I have searched the forums but I have not found anything that works for me. Here is my issue.....I have an array (converted from a .csv file) with many rows and columns. Each row is a logfile for a particular serial number. The amount of columns is fixed. But, I need to create a vi to go through the array and remove the duplicates below the original and create a new array.. For example, my array could look like this (I added a desc header for clarity):

Job, Tech, SN

1827, SJ, 23827

1825, SJ, 23827

1827, SJ, 47384

1827, SJ, 57483

1827, SJ, 37473

1825, SJ, 37473

But i want an appended array like this:

Job, Tech, SN

1827, SJ, 23827

1827, SJ, 47384

1827, SJ, 57483

1827, SJ, 37473

Can anyone help me out?

Thanks,

Ryan

[altenbach]

Since the rows are not true duplicates (e.g. the value in the first column are different), it is not clear which one of entries with duplicate serial numbers should be kept. Do you want to simply keep the first? Does it matter?

Are identical serial numbers always adjacent (as in your sample) or could they be scattered all over the table?

[Gak]

HI ryan,

Find attached example. Try this vi, let me know this is what you wanted.

I am sure this can be done in much more simpler and effective way, surely expert on this forum will help you.

In that vi, select "column" number from which you wish to remove duplicates.

[mikeporter]

How big can these arrays be in terms of rows. That is important because it will tell you how much time to put into making the code efficient.
Similarly, are all the rows equal in length as in the example? If so you might be able to take advantage of that fact.
In terms of the code itself, it could be as simple as looking at each element in turn and deleting it if it exists elsewhere in the array, but that can get inefficient if the array can get large because it doesn't scale very well.
You also might want to consider modifying how the data is represented by looking for ways to get rid of the strings as string manipulations can be inefficient due to the fact that strings can vary in size. For example you could replace the operator initials with an operator id number.
These are a few random thoughts. Search the forum for posts about manipulating arrays for some very good code and discussion.
Mike...

[altenbach]

Assuming that the identical serial numbers are adjacent as in your example, here's what I would do.

 

[myle00]

Here's my take. If the array size is large this doesn't grow the array in a loop. You select by which column you want to filter and only none dups of that column will be returned. But I'm sure it could be improved.

Matt

[altenbach]

Here's a version that does the bulk operation on smaller data structures:

 

[altenbach · Accepted solution]

Here's a version that will work if the duplicates are not necessarily adjacent. As a side effect, the output array will also be sorted by serial number.

 

[Gak]

@ Altenbach: Your solutions are too good...

I wish I had thought of one. 

[SimpleJack]

All,

Wow, thatnks for the all the responses....I really appriciate it. To answer a few of the questions:

1. No the duplicates are not always adjecent and probably will not be.Duplicates can occur more than one time depending how many times the operator retests. You see, I am going after first pass yield, so I only care about the first entry.

2. The actual amount of columns is always fixed, but much larger...about 55 columns. However the amount of rows will be variable depending on the amount tested.

I think Altenbach's solution may work out. I will try it later on tonight and let you know if I can come up with something. Something for me to do so easily in Perl is a challenge for me in Labview. I'm sure once i get more experience with arrays, it wont be so difficult.

Thanks,

Ryan