Opening and reading multiple non-sequential files

[jaslyn]

Hi Guys,

I show you a primitive file read code for combining multiple arrays and getting the average. I am VERY sure that there is a better way to do this, but I can't quite figure out how.

Problem with this:

Each file is opened separately using the file open dialog. With a large number of files (current number is 9. I'll need to go up to 10,000), human error might step in, opening the wrong file.

Opening files individually is also very time consuming.

If you get confused midway and cancel, Error 43 will result.

Also, it's not very practical to have to write a new programme for each time you need to add a different number of files.

Since timing is also very important, I technically need to add in a comparison of 2 arrays, one of 'Expected Tick Count' and one 'Actual Tick Count' that will only add the corrsponding file/array if Expected Tick Count=Actual Tick Count

I was thinking of a programme something along the lines of this:

but I'm getting confused... Also, I can't seem to convert the file number from string format back to number format so I can add '4' to it to build the next file path to open. (ok used 'Decimal string to number').

Any direction will be appreciated. Thanks.

Update:

Ok that previous file subsequently became this...

I was just trying to extract the original file number and add multiples of 4 to it. Then open these files and add it to an array if Expected Tick Count=Actual Tick Count.

More dazed and confused than ever... Excuse me while I go bang my head repeatedly against a wall.........

[altenbach]

Where does the "actual/expected tick count" arrays come from?

Shouldn't the expected count be a scalar and the actual count maybe depend on the number of rows read from the file?

Anyway, here are a few simple modifications that might give you some ideas. Modify as needed.

[jaslyn]

Thanks for replying, Altenbach!! 

Both actual and expected tick count will be read from a spreadsheet. Actual Count will be read/saved from 'Wait till next multiple' in the image acquire programme and Expected count will be.. Probably created using Excel based on the 'No of ms' set in 'Wait till next multiple' in the image acquire programme.

[NeilR]

Hi jaslyn

I think you're trying to achieve something like this ...

I've attached the code saved for LV 8.6.

The path is defaulted to ....\...- Total Output1.txt. The code strips out the '1' and creates a new path based on the loop count.

The file data is added (or not) based on the comparison of the tick counts. Probably best to use integers for the counts as comparing doubles can be dangerous

Good luck !

NeilR

[altenbach]

---

NeilR wrote:

I think you're trying to achieve something like this ... 

---

No, this will not work. You cannot add a 2D array to an empty 2D array (your SR initialization) and expect anything more than an empty array as output. In your case, ther result will always be an empty array, no matter how many files you read.

(Compare with my code which avoids the addition on the first iteration. (Yes, even my code will fail if the first file has a wrong count, but nothing that a little extra code could not fix. ;))

[NeilR]

  Ah yes, that old chestnut !  Nicely spotted Sir A.  You can of course set the constant array to 0 rather than an empty array to sort it.

Personally I really don't like having uninitialised shift registers, unless intentional. Should the code end up as a subVI and be called multiple times with different files for example, you could have problems.

[altenbach]

---

@NeilR wrote:

You can of course set the constant array to 0 rather than an empty array to sort it.

---

I don't quite understand what you are proposing.

If you initialize with a non-empty 2D array, it needs to be at least as large as the 2D array we are reading from the file. Sometime, the size might be known beforehand, but here we don't really have enough information.

[jaslyn]

Thanks so much for your codes, Sir A and NeilR.

N00b here but can't I just use the 'Initialize Array' function to solve the problem in NeilR's code..? In this case, yes the size of the array was known beforehand so I did initialize the array with a 'zero' to get that code to work like NeilR proposed.

However, I found that the output data was the same despite choosing different starting files. I found this really puzzling and am still thinking about it.

But yeah.. You guys are using functions I didn't even know existed. I do have aways to go.

Sir A, for your programme, I didn't understand why it kept on popping up a file selection dialog box. I got around it by changing the file path from a 'control' to a 'constant'. Also, I didn't quite understand why you connected the 'Iteration counter' to the case selector for your For loop..? Ohhhh I get it now.... That's to bypass the first iteration where the programme runs differently, right?

And I should modify the "1.txt" string and "1" I32 to match the name of the first file I feed in, right?

Would it be useful to add an 'Initialize Array' function to before the Double Shift Register?

Lastly, would it be useful to others if I were to post the modified code after I'm done with it?

Thanks so much for your help you both. You don't really need to answer my questions for my sake but perhaps other newbies could learn from them..

Best Regards,

Jaslyn

[NeilR]

Hey Jaslyn thanks for the kind words.

@altenbach

Assuming (not unreasonably) that the dimensions are consistent between files we can determine these from the 'base' file and initialise the array accordingly.

[altenbach]

---

NeilR wrote:

@altenbach

Assuming (not unreasonably) that the dimensions are consistent between files we can determine these from the 'base' file and initialise the array accordingly.

---

Yes, but here's the same with significantly less code. 😄

 

You would need to read one file outside the loop, though unless the size is always the same. That's significantly more effort than my iteration=zero case structure.