LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Automated background removal from spectra

This looks really good.  I would be very interested to look at your VIs and possibly implement within my software an automated option in addition to the manual method now available.  If I do this, I would give you full credit for that portion of the software.

 

Sincerely,

 

Don

0 Kudos
Message 11 of 39
(2,340 Views)

Sounds good. How should I get these vi's to you?

0 Kudos
Message 12 of 39
(2,326 Views)

Well you can post the llb here in case others are interested or you can send them to me at donald.j.roth@nasa.gov.  I am still using v8.5. 

 

I can post the overall background removal llb after testing and integrating your method. 

 

Sincerely,

 

Don

0 Kudos
Message 13 of 39
(2,323 Views)

Sounds like a plan to me. I should be able to post something this evening. BTW, what kind of spectra is this? Also, I would be interested in the Jim Nagle references you referred to earlier.

 

-Drew

0 Kudos
Message 14 of 39
(2,316 Views)

Here is the reference:

 

Investigation of Selected Baseline Removal Techniques as

Candidates for Automated Implementation

GEORG SCHULZE, ANDREW JIRASEK,
* MARCIA M. L. YU, ARNEL LIM, ROBIN

F. B. TURNER,and MICHAEL W. BLADES

 

Volume 59, Number 5, 2005  APPLIED SPECTROSCOPY

0 Kudos
Message 15 of 39
(2,309 Views)

Don,

 

Attached you will find the vi I used to process your example data set. It is similar to the SRM method in the reference you provided. The vi has an enum to choose the type of removal algorithm; none, linear, and baseline_filter. Linear is simply just a straight line drawn between the end points, although it does have the added control that you can specify an average of the endpoints in the case of noisy data. Anyways, baseline_filter will be the one you will want to try out. There is a cluster containing the four parameters needed for this method. f_sample, f_cutoff and order are filter parameters and as an example I think I used 100k for f_sample, 100 for f_cutoff and 4 for the order in processing your example data ASCII1.txt. The final parameter is # of iterations. Basically this vi just does these simple things,

  • low pass filter of the data
  • threshold the data based on the filtered data. If the "real" data is greated than the filtered data then replace the real data with the filtered data and repeat filtering process. This continues for the number of iterations specified.

This vi also includes a sub vi for extending the array of data to be processed in order to limit edge effects. I typically use the last point option instead of the symmetic option.

 

Finally, one easy way to see what is going on is to set the number of iterations to 0. This way you can see the first filtered pass to make sure it is behaving well. Add # of interations until the baseline looks correct. Once you have these 4 paramters set, as long as the spectral line widths don't change very much from shot to shot these setting should be reapplied and work quite nicely.

 

Let me know if you have any more questions about this and I'll thry to help as much as I can.

 

Thanks,

Drew

Download All
0 Kudos
Message 16 of 39
(2,298 Views)

Dave - thanks for this, I will get to in a few days or so.  By the way, you asked about the data.  It is from Raman Spectroscopy.

 

Sincerely,

 

Don

0 Kudos
Message 17 of 39
(2,283 Views)

Dave - when I process the data, I am getting extra points drawn as shown below that do not appear in the your prior post.

 

I am using the parameter values shown in the figure below, which were your default parameters  I am thinking we have to subset the final arrays in order to eliminate this effect.

 

 

 

 

 

 

Let me know if something needs to be changed.  How would you formally define the 'length_add_ends' - "the number of points to for edge effect removal algorithm?"

 

 

Thanks,

 

Don

 

Message Edited by DonRoth on 02-05-2009 02:24 PM
0 Kudos
Message 18 of 39
(2,261 Views)

Don,

 

The "extra" point is actually coming from your data file. Open the file with wordpad or notepad and scroll to the bottom and you should see an irregular character on the last line. Delete this line and save the file and your graph won't be showing this wrap around.

 

For the "length_add_ends" input, I don't know if there is a "formal" way to prescribe this value. Essentially all it is doing is adding a DC signal to the ends of the array so that no matter what the cutoff frequency is set to, the baseline gets pegged to the beginning and end of the "real" data. Obviously this is an issue if your data array starts or ends within a raman transition so I usually manually make sure that the start and end is a valid baseline point. I guess you could prescribe this value based on the user supplied f_cutoff and f_sample by adding the number of points that corresponds to f_cutoff in time. Say f_sample=100 kHz and f_cutoff=100 Hz then length_add_ends would be 100000/100=1000

 

Once you have the file fixed, you should run the code with the iterations set to 0 and then 1 and so on to get a feel for what is going on. Usually 15+ iterations and there isn't much change.

 

Hope this helps,

 

Drew "not Dave"

0 Kudos
Message 19 of 39
(2,253 Views)

I actually just remembered that I played with this a little bit a while ago. Here's my approach. I am not sure if it is sound, but it's quite fast:

 

Basically, I convolute the data with a gaussian of a certain width to smooth and broaden the data. Then I retain only points of the raw data where it is "close" to the the broadened data and the first derivative of the broadened data is below some settable threshold. Then I do a polynomial fit of the retained data for the baseline.

 

Just some ideas? 😉

 

 

 

 

 

 

 

Message Edited by altenbach on 02-05-2009 01:26 PM
0 Kudos
Message 20 of 39
(2,247 Views)