general polynomial fit problems

[hasqual]

I am trying to use the general polynomial fit to fit 3 data points (using it to a degree of 2).  This seems silly I know, but I am working with instruments where I want to get a decent estimate of an optimum set up by taking three different measurements (where each measurement takes quite a bit of time) and then fitting the three datapoints to calculate where the maximum, or most optimum operating regime is.  Hence, keeping the number of measurements to a minimum is quite beneficial and why I want to fit three points and find the maximum of the resulting equation. 

That said, in the mean-time I am "faking" the whole system by loading in a normal distribution of numbers and using this data to simulate the real situation, which is where my problem comes in.  As can be seen in the attached vi, I read a text file into an array, process through the data to find the corresponding (x,y) values and load in three pairs of (x,y) values into the polynomial fit (load the y's together to the y terminal, and x's to x) and obtain coefficients.  The problem I am finding is that the coefficients seems totally bogus.  First off, I am guessing that the coefficients are reported like "a0 + a1*x + a2*x^2", rather than the inverse of "a0*x^2 + a1*x + a2", but I checked the data both ways and neither were anywhere near the ballpark they should be.  I tried plugging in data between the extreme points, as well as the three data points themselves, and instead of returning y values on the order of 2800-4000 or so (as should be returned) I get back very large negative numbers.  I checked the mse value, which was returned as 0 identically. 

I am unsure if I am totally missing something obvious here, or if there is some mathematical fundamental flaw in trying to fit normal data to a polynomial distribution, but if any of you can take a look at the attached vi and txt file and give me any pointers I'd be grateful, the test case I was working with input 40 degrees for the angle and 0.5 for AO (angle offset) .  Thanks!

~Jack 

[Gabi1]

Hei Hasqual,

your vi lacks a subvi it seems.

about the polynomial fit: look at the help (right click on the icon -> help).

you should plug the polynmomial order, if you want different than 2.
try the several boxing (algorithm) possibilities LV gives you for fitting. when using only 3 data points for poly order 2, this might be critical.
check using an indicator that indeed 3 params get into the subvi.
use an independant fit to check validity of your data (eg Excel or Origin).
wire the error output to see if it did not return any error.
display data and fit on graph.

good luck

[hasqual]

Yeah, I'm an idiot and forgot to attach it the sub-vi ... so here it is also.  I actually did try to replicate the fit in Excel and got "other" bogus data (I don't always trust Excel as I've run into problems with their fitting before).  I did try a couple other algorithms other than the basic one, but always used an order of 2 as it seems that it'd be good.  Actually, in the Excel fit, the fitted line looked great, but when I plugged the equation they reported into the spreadsheet, the results weren't at all what the graph "said" it should be.  If that makes sense ... I plotted the data, fit a line and the equation returned did not fit the line it was supposed to fit.  Anyways, if this can lend any insight, that'd be great, sorry for the hassle and thanks for the help!

~Jack

[altenbach]

You're doing a mindbogling amount of confusing wiring. Do you really read the same file with each subVI call? Why?

What are typical inputs?

Why don't you place indicators on the two arrays going to the polyfit VI, run the VI with typical data, turn the new indicators into constants and delete everything else before the polyfit call. Now we can test with real, typical data wthout having to deal with all the mess before that. 🙂

You should also wire the error out of the fit VI, maybe it wants to tell you something interesting. 😉

I doubt the problem is with the polynomial fit.

[hasqual]

I am going to clear up right now that I'm a VERY new Labview programmer.  I'm not good in any language, but proficient in Perl, Fortran and Matlab and just started toying with Labview about 2 weeks ago with a summer internship. 

I laughed quite hard when you said "You're doing a mindbogling amount of confusing wiring", as that's exactly what I think every time I look at my program but I am unsure how to make it much clearer.  Nah, I don't need to read the same file with each subVI call and I could jump that out to the beginning, I figured it made it a bit easier with only needing one input and one output in the sub-vi but perhaps that's dumb of me (if it makes my program run 10% slower but simpler it doesn't matter to me since the big time constraint I have is going to be on the instrument side ... plus I won't be reading files usually, this is just to fake up the scenario until the detector set-up I'll be using arrives). 

As an example of my problems though, I put the same data into Excel, fit an equation to it and then plugged the same X values into the equation and what Excel says is there isn't there!  It's interesting to me that the equations generated yield large negative numbers in both Excel and Labview though. 

So, I did the additions you mentioned, a very smart thing that I'll keep in mind for the future, so that it's very short but the same problem occurs.  I am amazed that the MSE value is 0, and the R-squared value in Excel is 1 identically.  There is virtually no difference between the different schemes, and I did wire the error connector, but to no avail.  Take a look at the attached code to see if it makes sense now, and at the Excel spreadsheet to see both the Excel fitted function and the equation generated in Labview and let me know if you see any glaring problems.  I don't.  Thanks again, and I'll take any pointers you've got!

~Jack

Message Edited by hasqual on 06-01-2007 12:07 PM

[altenbach]

Just show more significant digits and you'll see that the results are the same.

Message Edited by altenbach on 06-01-2007 10:21 AM

[hasqual]

Well this was a big lesson in not trusting computers.  In the end, it's not a LabView programming problem.  Both LabView and Excel return the same ... bogus ... result.  It has something to do with the fact that the numbers are so clustered together, but in any event I can't use the LabView built in polynomial fit.  Programming was ok (even if ugly), it's just a calculational deficiency in the mechanisms for computing the fit.  I can't go too much into detail about why this is the case, mainly because I don't fully understand quite yet, but I know it's not a problem I can fix.  Sorry for the headache over that, as it's not really a LabView programming issue, but a mathematics type problem manifest in LabView (and Excel).  When I get home I'm going to give it a try in Matlab and see if the same results are returned, or if Matlab handles it better.  If anyone cares I can give an update, if not ... I'll just learn the lesson myself. 

~Jack

[altenbach]

Fitting three points to a quadratic polynomial is mathematically trivial, and if the MSE returned is zero, there is no way to improve it! 🙂

The value for the offset depends on your x scaling. Maybe you want to e.g. subtract 40 from your x-values so you'll get values that are scaled better (they are not more or less correct!). The constant term is the y value for x=0, the further away from your data, the harder it will be to tell where it will be. 

Here are a few experiments with your data:

(1) Taking all your data and fitting with an 8th order polynomial works perfectly (mse=0.15).

(2) Here is a plot of the coefficients doing a 2nd order fit to 3 adjacent points, running trough all your data.

(3) Here is the same procedure if we subtract 40 from the x-values.

Message Edited by altenbach on 06-01-2007 02:30 PM