03-25-2011 09:38 AM
I am generating data from a ultrasonic sensor at 1K Hz, and its a lot of data (data points range from 0-10). However, in some cases when I know data should be around 7 (for example) I get outliers (about 9 and around 10). Is there a way to set a filter to obtain data in defined range.
I will be average the data obtained to get a mean value, and the outliers are spoiling it. In worst case, my outliers are 30-40% of the data generated. I created a filter to sort the data and taking data starting from lowest value. I stop the loop when data hits a higher value then 9. But this seems to take a lot of time (since the loop checks for each data point and there are 1000s of them).
Is there a better way to filter this data and set a pre-defined range for array to be collected?
I am attaching my filter.vi.. and a sample data set from my previous run. The data ranges from 10 -8, and would like to have range set to 7.5 to 8.5 to be considered. The sensor is recording voltage here, and the problem can be solved in installing a different type of sensor, but a if a filter in LabView can due it, the sensor we are using now is absolute great.
I am in kind of emergency, my design in stalled due to this problem, if anyone can find sometime to share few suggestions, I will be grateful.
Thanks in advance.
Solved! Go to Solution.
03-25-2011 10:42 AM
I did something years ago based on validating every value based on the slope.
1) Take derivative
2) Do statitsics to find standard deviation
3) Compare all samples and their slope and if in 2 standard deviations, keep if not toss (or do a linear fit between previous a following).
A the time it was real CPU killer but new mahcine should be better prepared to pull that plow.
Ben
03-25-2011 12:05 PM
Thank you Ben,
I would like to try that, but time is real constraint here. I have max 10 seconds to process the data, and I am running on the fastest possible machines.
03-25-2011 02:07 PM
See attached. I embedded the data you posted into the vi. It didn't seem like any data was less than 8.7 or so, so I modified the range so there would be some points to average. Some sets were completely out of range so the mean came back as NaN (not a number) because of a divide by zero.
03-25-2011 02:41 PM
Thank you Randall,
Your example through some light on how I can select data with in range.
I shall modify it further to fit into my program.
Thanks again!