08-23-2015 12:26 PM
Hello,
I am writing a program to concatenate several text files. However, I need to process each column individually before they are concatenated. I am using a for loop to normalze 6 columns witihin the larger concatenation loop. However, the shift registers are storing data from previous iterations of the larger loop. How can I clear the shift registers of the sub loop based on the iteration of the larger loop?
Thanks in advance!
Solved! Go to Solution.
08-23-2015 12:44 PM - edited 08-23-2015 12:46 PM
Please attach your actual VI instead if an image. If the inner shift regsiter should start fresh with each iteration of the larger loop, delete it (the shift register!) and just use autoindexing at the right boundary.
Overall, your code has very questionable constructs and you seem to constanty use the wrong tools. For example, don't use "delete from array" as "Index array". Index array is also resizeable.
You are slicing and dicing the array like a sushi chef, constantly resizing (growing/shrinking). This is very memory inefficient. Why not use index array/replace array subset to operate mostly in place?
08-23-2015 12:53 PM
Thanks. I understand this isn't the cleanest code. It started out very simple and as I've needed more processing I added bits and pieces which made it quite messy. My plan is to clean it up when it is complete and functional. Your input is appreciated though, thank you. Vi is attached.
08-23-2015 01:01 PM
Do you have a few typical datafile or at least a description on how the data is arranged in the files?
08-23-2015 01:08 PM
Unfortunately, I cannot share the data files as they are confidential. But they are all setup the same with 17 columns, each a different metric. The issue that I am having is with columns 8-14, which I need to extract, normalize to their max values, and then replace in the master array.
08-23-2015 01:21 PM - edited 08-23-2015 01:22 PM
OK, that's fine. are there any headers?
In any case, all you need to do is place the entire 2D array in the shift register of the inner loop, iterate over the relevant columns to be normaized (use "index array" to get the column, normalize it, put it back using "replace array subset". You probably want to initialize the shift register of the outer loop. then use "build array" to append or prepend the datasets from each file.
08-23-2015 07:17 PM
Can you share a file with the same format?
If you have any text, you can change the words. You can change all the numeric values. It's the format that's useful, not the data, when discussing how to work with it.
08-23-2015 11:28 PM - edited 08-23-2015 11:30 PM
@mcg123 wrote:
Thanks. I understand this isn't the cleanest code. It started out very simple and as I've needed more processing I added bits and pieces which made it quite messy. My plan is to clean it up when it is complete and functional. Your input is appreciated though, thank you. Vi is attached.
It's best to keep your living room clean to begin with instead of trying to cram everything into the closet when you have company over. Keep your code clean from the beginning - it's a BIG hassle to try to do it all at the end.
08-24-2015 03:06 AM
Altenbach expressed it well, the shift register is an unnecessary construct since you add a row each iteration a for loop with autoindexed out is the better solution. Less code, faster to write and run.
/Y
08-24-2015 10:49 AM
I assume that all files contain the same number of rows and columns. Since you can calculate the final 2D array size from the number of files, you can initialize a shift register with the final array size in the outer loop. Inside the loop you would read each file, normalize part of it, and write it into the allocated array (using "replace array subset"). You can uniquely determine the write position from the iteration value.
This is much more efficient than to grow the arrays as you go, constantly allocation new memory.