Levenberg-Marquardt, error

[Hobgoblin]

Hello all, I'm currently trying to implement a nonlinear curve-fit routine in LabView (Nonlinear Curve Fit.vi, Levenberg-Marquardt). I'm trying to find a best fit for a given exponential decay, and the fitting model is passed to the Nonlinear Curve Fit VI through a VI. However, I don't want to implement the model as simple function, but instead I want to describe the model with a Runge-Kutta ODE solver. I do that, so I can later on implement more complex mechanism and models that do not have an analytical solution. When i try to run the optimization the Levenberg-Marquardt algorithms runs a few times, and then returns "Error -20041 Analysis: The system of equations cannot be solved because the input matrix is singular." This error seems to occur quite often, but the other topics here couldn't solve my problem unfortunately. Any suggestions on how to solve the problem are highly appreciated 🙂

[LeopoldK-B]

Hi,

did you try the suggested steps in the following KB

http://digital.ni.com/public.nsf/allkb/36448BDBE67D99048625660C0000BDF0?OpenDocument

Best

[Hobgoblin]

Hi, thank you for your suggestion. The problem is, that my inital coefficients work perfectly fine. Only after a few function calls the algorithm crashes and I have no idea why it does so only after it already started. Best regards

[altenbach]

Since your model contains an iterative procedure you get accumulation of errors and jitter in the result. This means that your partial derivatives are most likely not good. (to calculate the partial derivatives, the function is evaluated with two microscopically different parameter values and that slope is most likely quite random.)

You can easily do the experiment and see how the partial derivative depend on the exact parameter value.

[Hobgoblin]

When I calculate the derivate of my function numerically and manually, the derivates seem to be 0. I'm calculating my function with 2 slightly different parameters and after my Runge-Kutta VI the two functions are the same. The Levenberg-Marquardt VI probably does the same which lead to the fact, that I can't optimize my function. Did I get you right about that point? Best regards

[altenbach · Accepted solution]

The partial derivatives are not always zero and if you had some way to calculate the partial derivatives in some analytical way, you could include that in the model, of course.

There are fitting algorithms that don't rely on partial derivatives, such as Nelder-Mead (downhill simplex). Curiously, we just had a related discussion a few days ago, which prompted me to release my nonlinear fit replacements that implements Nelder-Mead instead. As you can see, if you replace your current nonlinear curve fit with my version, the fit proceeds nicely. 😄

[Hobgoblin]

Hi altenbach,

thanks for your help. Your VI seems to work perfectly fine, even for some more complicated problems. Very nice!

Best regards

[Hobgoblin]

This is a LV2011 version of my example above for users who are not able to open my version.

[bharathp10]

hi hobgoblin

I think you uploaded the v15 again. I am unable to open on LV2011

[Hobgoblin]

Oh ok, sorry. 

Try this one

Best regards

[altenbach]

• You should not call it Levenberg-Marquardt if that has nothing to do with it.
• You need to specify that it depends on my example code, downloaded from elsewhere.

[Hobgoblin]

Excuse my sloppiness.

This version right here contains an example where Nonlinear Curve Fit  VI (Levenberg-Marquardt) doesn't work, as described above. However, using the Simplex (Nelder-Mead) method as suggested by altenbach, the fit works perfectly fine. The user can choose between the two algorithms. 

The original version of the Nelder-Mead VI can be found here: Nelder-Mead VI by altenbach

Best regards,

Hobgoblin