06-10-2016 05:12 PM - edited 06-10-2016 05:13 PM
I have a dataset made by successive scans, N scans will give me a N times a 2D array (XY values).
I would like to average all Y values having the same X value. It would be easy if all X were the same and ordered, but they are not. By that, I mean that sometimes a point is missed by the acquisition. Sometimes it is read twice.
These two are one scan each (actually line 2, Y data, is random but it's not important here).
As you can see, Set 1 does not have a X value at 564.000, and two at 564.001, while set 2 has no value at 564.001.
There is no way arround the problem, the instrument giving me the values is slow.
How can I average all Y values across all scans with a certain X coordinate?
Here is a bunch of code that reproduces the behavior of the system.
06-10-2016 07:30 PM - edited 06-10-2016 07:32 PM
It would be O(N^2), but why not simply iterate over each scan, building some sort of "Master Scan" and when you encounter another of some given X value, add it into your master scan. Then average as needed. Here's a rough idea:
Master Scan is an array of clusters:
Does that look like what you want? After running all "scans" through this VI, there will be a master scan with you can do you averaging and convert back to a normal 2D array for other uses.
06-11-2016 09:56 AM
Before worrying about averaging demonstrably bad data (because you are missing points, or have extra points), I would spend some serious effort to fixing the Acquisiition Program to give good data, where all of the points are sampled at the same rate, no time point is missing data, and no time point has duplicate data. Most acquisiition hardware has good clocks that tend to insure this behavior -- it typically goes awry in the software driving the hardware (can you say DAQ Assistant?). [Hmm -- should I compare this to putting lipstick on a pig? No ...]
06-13-2016 11:44 AM
That sure would be the best option, and I tried, but we run an experiment at 30 Hz, and I probe a laser over RS232 to ask for its wavelength. Problem is, it is a really old dye laser with a low response time. I can probe it at 8 Hz approximately, and to add to the fun, some digits are sometime missing, (instead of 564.651 it will return 564.65 or 564.6, or even 56) because of communication issues.
My data aquistion loop is able to reject those erronous points, but as a consequence I have to deal with different sized columns.
06-13-2016 06:51 PM
i just tried with your solution, but unfortunately, I might have done something wrong. At least I understood it wrong.
When the loop runs, all the data from master scan is zeroes, so the test always return 0 and by the end, the Master scan array is still empty.
You mind attaching it with some data so I could check it?