Issues reading in NIDAQ dual channel binary data

[musilmark25]

Summary

I am struggling to store data from a USB-6346 device in a usable format. 

Explanation

I am sampling the I and Q channels from a radar return. I am taking these samples using a National Instruments USB-6346 data acquisition device. What I believe is taking place is that the channels are being interleaved into the resultant binary file as 64 bit floating point data. That is what is suggested when I view the binary files in Visual Studio. Below is a screenshot of the first lines of such a file. 

I have written the following script in Python to attempt to extract the I and Q channel data. 

# Imports

import numpy as np
from matplotlib import pyplot as plt

# Globals

sample_frequency = 500e3
sampling_period = 1/sample_frequency
total_samples = 30e3
begin_time = 0
end_time = total_samples/sample_frequency
signal_frequency = 20e3
time = np.arange(begin_time, end_time, sampling_period)

# Read in the file

myFiles = {
    "Feb3-20kHz-30kSamples-500KspS": 'Ifeb3test.bin',
    }


with open(myFiles["Feb3-20kHz-30kSamples-500KspS"], 'rb') as fileHandle:
    _content = fileHandle.read()    

content = list(_content)



my_vec_of_64bit_words = []
for enum, elem in enumerate(content):
    if enum % 4 == 0:
        my_vec_of_64bit_words.append(content[enum:enum+4])

true_data = []

for word in my_vec_of_64bit_words:
    true_data.append(sum(word))

# Separate I from Q

I_channel = true_data[0:len(true_data):2]
Q_channel = true_data[1:len(true_data):2]

# Take its Fourier transform


amplitude = np.asarray(I_channel)
figure, axis = plt.subplots(2,1)

plt.subplots_adjust(hspace = 1)

chunk_of_data_ = 250

axis[0].set_title(f'Sample of acquisition')
axis[0].plot(time[:chunk_of_data_ ], amplitude[:chunk_of_data_ ])
axis[0].set_xlabel('Time')
axis[0].set_ylabel('Amplitude')


fourierTransform = np.fft.fft(amplitude)/len(amplitude)
fourierTransform = fourierTransform[range(int(len(amplitude)/2))]
tpCount = len(amplitude)
values = np.arange(int(tpCount/2))
timePeriod = tpCount/sample_frequency
frequencies = values/sampling_period

chunk_of_data = len(frequencies)

axis[1].set_title('Fourier Transfrom of the samples')
axis[1].plot(frequencies[0:chunk_of_data], abs(fourierTransform[0:chunk_of_data]))
axis[1].set_xlabel('Frequency')
axis[1].set_ylabel("Amplitude")

plt.show()

Such a script creates the following plot

This is nonsensical to me. There ought to be one spike at the signal frequency of 20kHz. What am I doing wrong here? How should I be reading in these data? 

Thanks!

[crossrulz]

It really concerns me that your plot is claiming a range of ~8GHz when your sampling rate is 500kHz.  I think your issue there is that your frequencies for the plot should be "values*sample_frequency/tpCount".  Each sample in the frequency domain is going to divide evenly the sample frequency.  So you have sample_frequency/tpCount to get the frequency step size.  You can multiply that by your values array to get the frequencies at each step.

That does not fix your bigger issue of your FFT plot in general.

Do you have the code you are using to save the data?  There might be some hints in there.

[musilmark25]

Here is a snapshot of the block diagram

[mcduff]

---

@musilmark25 wrote:

Here is a snapshot of the block diagram

---

You are saving Waveform data, not a 2d-array of I & Q. In addition, you are only saving the last part of your data when you press stop, not the whole data stream.

Waveform data contains t0, dt, an array of Y values, and possible Variant attributes. You can change your output to a 2d Array of Y-values for the DAQmx Read function. That may be easier to input into Python. Also on your save, you are including the length of the byte stream, you would need to account for that in your python script.

[musilmark25]

So I'm not aware of how to change that data into the form you have described. I am new to LabView and am unfamiliar with the correct way to perform certain functions. Would you mind pointing me to the appropriate documentation?  Thanks

[mcduff]

The DAQmx Read Function has a polymorphic selector, just change that. (You may/will have to modify other parts of the code to accept a 2D array rather than an array of waveforms.)

[Kevin_Price]

Here's another option to consider if you're more comfortable in Python than LabVIEW:

1. Consider using the TDMS file format and the DAQmx TDMS logging options.  Consult the LabVIEW shipping examples to see how easily you can at least *generate* the TDMS file.

    If you do this, I'd recommend you go back to reading a 1D array of waveforms because several bits of possibly useful info will be carried along with the waveforms and embedded into the TDMS file.  The sampling interval (= 1/sample_rate) is one of the key ones.  Then you can feel free to capture data at various sample rates because your reader will be able to discover what the sample rate was.  You won't have to guess or assume.

2. Find an appropriate TDMS reader for Python.  I don't do Python, but Google found me this as a starting point...

-Kevin P

[mcduff · Accepted solution]

---

@Kevin_Price wrote:

Here's another option to consider if you're more comfortable in Python than LabVIEW:

1. Consider using the TDMS file format and the DAQmx TDMS logging options.  Consult the LabVIEW shipping examples to see how easily you can at least *generate* the TDMS file.

    If you do this, I'd recommend you go back to reading a 1D array of waveforms because several bits of possibly useful info will be carried along with the waveforms and embedded into the TDMS file.  The sampling interval (= 1/sample_rate) is one of the key ones.  Then you can feel free to capture data at various sample rates because your reader will be able to discover what the sample rate was.  You won't have to guess or assume.

2. Find an appropriate TDMS reader for Python.  I don't do Python, but Google found me this as a starting point...

-Kevin P

---

Do not know if this applies to Python or not, but, the internal TDMS logging functions are a great idea. But they save RAW data and scaling information, along with a bunch of metadata. Your reader would need to interpret it correctly. In the past, some colleagues tried a Matlab reader, that would work for every TDMS logged file BUT the first file in the series. It was really odd. Not sure what the difference between the first and subsequent files were.