can you costrain General Linear Fit VI?

[patriziocandeloro]

Hi all,

i'm using the "General Linear Fit VI" of LabView2016 for fitting experimental data curve Y (from spectroscopy experiment) with a dataset of already known curves, namely B1, B2, B3, ..., Bn.

So i'm using the dataset of already known curves (B1, B2, B3, ..., Bn) as the H input of the VI, and calculating the coefficients a1, a2, a3, ..., an which best fit the experimental curve Y. Everything is working fine, but at the moment i get as a result both positive and negative values for the coefficients a1, a2, a3, ..., an.

But in our experimental conditions, these coefficients should be only positive (equal or larger than 0).

Is there any method to force, to costrain, the "General Linear Fit VI" to produce only positive resuts for the coefficients a1, a2, a3, ..., an?

Thanks in advance,

Pat   

[altenbach]

Can you tell us a bit more about your data?

You can always use the constraint nonlinear curve fit and implement the model yourself. You could also use the unconstraint version and fit for transformed parameters, e.g. the log of the actual values.

If your "parameters" represent a somewhat continuous curve, maybe non-negative Tikhonov regularization would work. I use this for example here.

[patriziocandeloro]

thanks for your answer altenbach,

unfortunately the experimental data used as basis for fitting, i.e. the H 2D-array input, cannot be replaced by continuous curves. They come from other experiments in which we measure pure substances.

Instead the Y array we want to fit comes from a mixture of these pure substances. So the "a1, a2, ..., an" coefficients should reflect somehow the concentrations of the pure substances in the mixture.

I saw your link about the Tikhonov regularization, but I do not know this kind approach.

By using Thikhonov approach, can I eliminate the negative coefficients by iterating the fitting step? not sure I have understood properly

pat

[altenbach · Accepted solution]

No, "plain" Tikhonov regularization does not eliminate negatives, you need an iterative procedure and there are quite a few algorithms out there. You can of course implement the entire objective function using non-linear fit (VI model) and give the H matrix via the "data" variant. The computation is basically just a Matrix x Vector multiplication. In your case of distinct components, the regularization parameter would be zero (i.e. no longer Tikhonov!), of course and you can do iterative plain linear algebra for a nonnegative solution.

Can you attach a simple VI and a typical dataset that falsely gives negative coefficients.

See also my recent answer here . Can you assume that all negative coefficients should actually be zero?

[patriziocandeloro]

Thanks again Altenbach,

I will try first the non-linear fit VI providing the H matrix by the variant data, as you suggested.

Hopefully it will work, I already used that VI but for other purposes and without variant data.

regards,

Pat 

[patriziocandeloro]

Sorry for not updating before,

but I could not find the time for testing non-linearFit.vi passing the basis of curves by variant data.

Finally today I could spend some time on it and everything is working fine. Thanks a lot Altenbach for advices and for providing some example VIs in former threads. They were very useful and made clear to me how to work with function model and variant data.

Best regards,

Patrizio 

[altenbach]

Good to hear!

Still I think a more direct method (e.g. my non-negative regularization with alpha=0) would be orders of magnitude faster. If you care to share some typical data, I could play around with it.

[patriziocandeloro]

Dear Altenback,

here i am again, and you were right: i would need a faster fitting procedure.

First i thought it was fast enough, but i need to fit autometically something like 1000-10000 spectral curves, and when trying to use the Non-Linear Fit.vi with data variant it becomes very slow... too much slow.

As you suggested i'm sharing some data in the attached file .rar. Once extracted you have:

1) coreFitting.vi, which is the fitting core i'm using in a larger software

2) subVI folder, in which there are some used subVIs just for loading data and also there you can find the model function named "combineSpectra_function.vi"

3) experData folder, in which you can find an example of spectrum to be fitted (namely "spectrum-to-fit_AvrgStDev.txt") and a basis of spectra in the subfolder "basisSpectra".

The fitting procedure produces as output parameters the coefficients to be used in a linear combination of the basis spectra in order to best fit the experimental spectrum "spectrum-to-fit_AvrgStDev.txt". In other words:

Y= c1*B1 + c2*B2 + ... + c_n*B_n

where Y is the experimental spectrum to be fitted,

(B1, B2, ..., B_n) are the spectra of the basis, and

c1, c2, ..., c_n are the coefficients to be found.

In order to have only positive coefficients, in the model function i'm using an exponential function, so that:

c1=exp(a1) ... c_n=exp(a_n)

where a1, ..., a_n are the parameters fitting.

I hope the statement of this problem is clear enough.

You suggested me to try non-negative regularization with alpha=0, but i do really not know enough of fitting to make this approach. Might you provide me with some other input to use the fitting procedure you suggested?

all the best,

Pat 

[altenbach]

Sorry, I am currently not on a computer with LabVIEW and my LabVIEW computer has windows 10.

I strongly recommend to always use zip (instead of e.g. rar), because it is natively supported by the OS. (Windows 11 very recently added native rar support , but that's a different discussion).

Yes, I have 7-zip (which is great), but other forum users might download questionable archive tools with malware! Searching for rar extractors is a minefield.

[patriziocandeloro]

sorry for the .rar archive.

Here you can find a .zip archive of the same files.

Kind regards,

pat