FFT filtering on Spectroscopy Data

[beckham2k2]

Hi everyone, 

I want to create a vi that perform some data fitting with FFT filtering given by data that I have collected from another vi file (2 arrays- Intensity "y" vs Wavelength "x"). However, I have some trouble understanding how such concept is performed in labview. I played around with it, but I dont quite get the results I wanted (see the "After FilterFFT" graph in FFTtestDisplay.jpg), the closest I can get is when I put HighPass on, the Cut Off freq is the "Lower Cut-Off" Control in FFTtestDisplay.jpg

I had 3 picture attached to this question, 

FFTexample.jpg - This is an example of the FFT filtering that I want in the end.

FFTtest.jpg - This is the block diagram of what I attempted to make FFT filtering. (X Input is in Wavelength and Y is in Intensity for the XYGraph)

FFTtestDisplay.jpg - It is the front diagram for the FFTtest.jpg. 

Note that the Input for Wavelength and Intensity is very similar to the Data from the FFTExample.

If you guys can figure out the way to do it, or even some pointers on my approach, that will be much appreciated.

Thank you again.

[JustJohn]

Hi beckham2k2!

I think you have the basic concept down correctly in LabVIEW. I wanted to see if you could also post the initial signal before it is processed through the initial FFT. What does the filtered signal need to look like? The final product seems to be very clean and I was wondering what other questions you might have about it. Please respond with these items and we can go from there!

[beckham2k2]

Hi John

First, thank you for the reply. 

So essentially, my data is very similar to the Red line in the "FFT example.jpg" that I posted above. As you can see, the red curve has quite a few fringes (interference pattern like), so with the FFT filter smoothing, the end result should look like the black line in the same jpg file. Another words, the FFT filter should be a smoothing tool that can remove the fringes for the fitting. 

For my current progress, everytime I collected a data set (Intensity vs Wavelength such as the red line) from a vi that I made. I have to save it to a txt file and then open OriginLab and manually produce a FFT smoothing in order to get the "black line", thus I want to make a vi that can just do that in automation without going to another software which can enhance my work rate.

FFTOriginSample.jpg is the work I used through OriginLab, the red line is my data set I collected from my vi and Red is the FFT smooth curve that removes the fringes.

Again, Thanks for the help, everyone!

[JustJohn]

Hello beckham2k2!

I believe I understand what you want to do with this application, but is the current setup that you have not working properly? The filtered FFT looks very smooth on your picture from your previous post. Is that filtered signal not the correct signal? I think it would be good to see what you actually have as well as see in a screenshot the entirety of the code. 

[beckham2k2]

Hi John

Although the Filter FFT signal looks good, but it doesnt fit my original data. As you can see from my Initial Post, the FilterFFT goes to 40000 on intensity while my regular graph from Origin (FFTOriginSample.jpg) is only go to 80 on intensity. THAT is what I wanted to look like without going through Origin.

There really isnt any more coding on that .vi, it is just a draw file from my txt file and then Index array to separate the X (Wavelength) and the Y (Intensity)

[jamiva]

I have some experience in filtering, but I am just a little confused with your algorithm. What I see in your "FFTest.jpg" you are taking an FFT of some data, running it through a filter, and then taking another FFT of that data.

Question #1: Why are you ignoring phase in the first FFT?
Question #2: Why are you taking an FFT of an FFT. Shouldn't that be an inverse FFT?

[beckham2k2]

Jamiva

I am pretty new at this topic like I mentioned (since I know more labview than the FFT filter Smoothing concept) , so that's why I posted all my progress I have in that regards here to look for some pointers, but my goal is to use a Smoothing process (FFT filter type) to remove the fringes off the original data (see FFTOriginSample.jpg).

The link below is the Smoothing I used in Origin, and I tried to replicate this into a vi essentially

http://www.originlab.com/index.aspx?go=Products/Origin/DataAnalysis/SignalProcessing/SmoothingAndFit...

I hope that clears the air a little bit

[beckham2k2]

I am more familiar to Labview than the concepts of FFT, that's why I show all my progress here and seek for some pointers. So the block diagram is just me fiddling around the functions. 

In the end, my goal is to have a FFT filter Smoothing on the data that I have collected (see "FFTOriginSample.jpg" above where the red line is my data, and black line is the one I want) which removes all the fringes off my data.

(you can search "FFT Filter Smoothing" in OriginLab and see what it looks like there if you need more references)

[jamiva]

I never used the "FILTER" express VI and not really knowing what your data looks like, I'll just give you a few things to try out.

1) It looks like the express VI does the FFT and the inverse_FFT already for you. You don't need to do it. Delete your 2 FFTs and wire your data directly to "Filter". You probably still need to use the to/from DDT to convert your real data to dynamic data.

2) You probably want a linear phase FIR filter to eliminate any phase distortion. Double click the express FILTER, and in that window choose Finite Impulse Response. I am ASSUMING this VI creates a symmetric FFT since you can only choose an odd number of taps.

3) The biggest problem is that you will have a signal delay in your filtered output signal. But since you have a linear phase response FIR filter (see above), this delay will be a constant. I believe it is delta_T*(N-1)/2. Where N is the number of taps and delta_T for you would be delta_X. Just shift your filtered signal along the x-axis by that amount to the left and both curves should be centered around the same wavelength.

4) Finally, play around with the FIR filter settings until you get the desired output data.

One final thought: This will not work if your X values are randomly spaced, i.e., the delta in wavelength of all x-axis data needs to be the same, and sorted from lowest to highest wavelength

[jamiva]

Another thought:

If the express VI works like I think it does, the first N data points and the last N data points of the filtered signal will not be what you expect. The FIR filter is first initialized to all zeros. So until you run N data points through N number of taps, your filter is still using OLD data and the results could have significant error. So just ignore the data at the beginning and the end if it shows any unusual artifacts. It probably will not be an issue unless you pick a very large number of taps.