Dynamic Searching Large 1D String Array Painfully Slow.

[n2new]

I have a 1d string array Dynamic Data generator, fixed Size, 50K every second.  A large file with over 500k Lines is allocated in memory and is static. The generated data is searched in the large fixed Array. Everything works as expected but it takes 3 or 4 seconds, using loop parallelism helps but not much. If I decrease  the generated Array size speed increases. Conversely, If I use a smaller Fixed Array, it increases. This is not unexpected behavior however using double the size Array's in Other languages, takes milliseconds. Is labview not capable of fast searching or perhaps I am missing something?

Thanks in advance!

Tom

[Darren]

I think something else in your code is causing the long execution time... Search 1D Array is normally quite fast, despite being a simple linear search. In this code, I generated 500,000 large strings, and the Search 1D Array takes between 2 and 10 milliseconds to execute, depending on where in the array the searched string appears.

[RavensFan]

Let's back up a few steps and understand what you are trying to do.

What exactly do you mean by "Dynamic Data generator"?   Dynamic Data has a very specific meaning in LabVIEW as it is a very special datatype.

Where is this large data set coming from and what does it represent?

What are you searching for in this data set and why?  What do you do with the results once you found it?

You say a large file is allocated in memory and is static.  Files don't get allocated in memory, data does.  Are you saying you are reading a large file into memory?  And are you doing it just once at the beginning of the program?

I see you are writing some string data to a file called c:\match.txt, over and over again.  And it is overwriting whatever was previously in that file.  Is that what you want to do?  File operations are orders of magnitude slower than memory operations.  I suspect that is why your loop seems like it is too slow for you.  And it seems like a waste of time writing to that file when only the very last time that you write to it will the data stay in that file.

[n2new]

Wrong terminology I used. I just meant that is changes, size stays the same. 

[n2new]

You know, I tend to agree with you but there must be a ghost in the machine then! I have tried everything., producer loops, you name it.  Just to do a sanity check, I used to separate loops.
It takes 1 second to generate the Data array  and 8 seconds to search. I think you're right but the problem is, if I reduce the static array to 250k the speed doubles. The string  lengths are 45 to 50, could that be the issue? I wouldn't think that would matter much. 

[n2new]

By the way, Look closer.  That is a condition case. That only executes if a match if found. 

[rolfk]

---

@Darren wrote:

I think something else in your code is causing the long execution time... Search 1D Array is normally quite fast, despite being a simple linear search. In this code, I generated 500,000 large strings, and the Search 1D Array takes between 2 and 10 milliseconds to execute, depending on where in the array the searched string appears.

---

Well, except that the code of the OP does a search in 100k but 500k times every time. 500k * <1 ms per average is actually quite a bit

[n2new]

Also, I have plenty of CPU overhead when this is happening. 

[n2new]

Well son of a gun. It looks like you solved it sort of.  Is this unexpected behavior or improper programming?  If I disconnect the comparison from the case structure, it works. I guess this is a no no.  The plot thickens. It's a useless search. I can't wire the comparison. I am starting to think it's a bug. It's a fast search but I have no way of stopping the VI if it finds a match.  

[n2new]

Do something with the search result like stop the VI.  You will have time to get a cup of coffee.