Baseline determination and subtraction

[MJHanagan]

Hi everyone - I need some help in figuring out how best to do a baseline determination then subtract it out leaving just the peaks.  My data is profilometry measurements across a substrate surface.  As you can see in the chart below the substrate surface is fairly straight but not lying flat on the scanner table.  I am looking for an easy method to identify the linear baseline then subtract it from the raw data thus leaving the four peaks with essentially a zero/horizontal)baseline.  Then I can use the Peak Detector to identify the peaks and their locations so I can make additional calculations (mostly peak area and average peak height).

Any suggestions on how to best go about this? 

[altenbach]

Your baseline does not look linear, you probably need at least quadratic polynomial (or higher).

So just select all the non-peak values and fit them to a polynomial, then subtract the polynomial. Can you attach some typical data file?

[Thoric]

Can you apply a High-Pass filter with a relatively low filter frequency to remove the nearly-linear baseline component. That ought to leave you with a nearly flat dataset.

[MJHanagan]

Spoiler

Attached is a sample scan of data and my first attempt at applying a filtering routine.  I don't think I'm usign this correctly becuase I cannot seem to flatten out the baseline without gross distortion of the peaks.

Can either suggest where I'm going wrong here?

[altenbach · Accepted solution]

A polynomial fit using "least absolute residual" is relatively insensitive to outliers and seems to work pretty well in this particular case.

See if this works for you.

[MJHanagan]

Wow, this is nothing short of awesome!  I have no idea what this small tight routine does, but it certainly seems to have solved my problem perfectly.  It's an elegant solution.  You must be a mathematician or a physicist.  Thank you!!

[altenbach]

Sorry, there was a bit of duplicate code. Some of the "complex to RE/IM" can be eliminated, of course.

[gas_chromatography_user]

This is an interesting topic to me, I would like to correct a baseline to zero for some data so that later I can determine the area of just the peak, and not the peak and its baseline.  I have entered the data into altenbach's block diagram (thanks!) well enough to see the raw data (scan data) but the front panel doesnt return anything for baseline or baseline corrected.  I will try to decipher what could be wrong on my end but am also submitting for anyone's consideration, thanks so much.  Pic attached.

[altenbach]

Instead of attaching an image, attach your actual VI and some data.

[gas_chromatography_user]

Yes, lets see if the servers like this size of a data file.  I had it as an .lvm but changed it to a txt file or it wouldnt upload...