Tektronix TBS 2014B Waveform/Y-Value Decimal Precision Problem

[mhastings92]

Hello everyone,

I am currently working on a LabVIEW project (LV 2013) where I need to collect a data point from a Tektronix oscilloscope every 5.04 seconds and display it in real time over a period of 4.5 hours. At the end of that period I save the data to .csv file. The problem I am having is the data points have very low decimal resolution and seem to cut-off at 2 decimal places. This creates an issue when the data is processed as I need as great a decimal precision as possible. When I compare this method of data acquisition to the TEKXL Excel Add-in (Currently being used), I see the data precision can be much higher (EX: TEKXL Data point: 0.3395438194; LabVIEW Data point: 0.34). I am using the driver and VIs that come with them to acquire the waveform. The oscilloscope is communicating via USB. I have attached my stability test VI and the "Fetch Waveform.VI" that I am using to acquire the waveforms and data points, and a pdf to view. I was hoping to please get some insight on this issue. Thank you.

NOTE: After troubleshooting I have found that the "Fetch Waveform.VI" constructs a waveform with Y-Values already in that low precision, however I do not fully understand that VI. I suspect I would need to do some alterations in that VI. Also, I am a novice LabVIEW user and I know my VI has rookie mistakes and is not as clean as it could be but these are issues that I can fix and this decimal precision issue is the crutch of the entire purpose of this VI.

[Albert.Geven]

I had a short look at your code but saw too many things that could be cleaned up.

You probably 2 conversions from a floating point format value into a string.

The first you see selected is having 3 decimal places the second 6.

So I don't know where you get only 2 decimal places.

Or I may oversee another place where you write data.

[mhastings92]

Thank you for looking, I did put my code as reference, however the lack in precision of significant figures from the data output of the oscilloscope seems to come from the "Fetch Waveform.VI" as I have used just that VI in the "Continuous Acquisition.VI" example that came with the driver and noticed the lack in significant figure resolution in that scenario as well. Probing the data flow of "Fetch Waveform.vi", I noticed when the waveform is being constructed the amplitude value is only 2 decimal places. I realize in my code, I have conversions and decimal place values set, but since there are no significant figures coming from the data extraction of the oscilloscope it simply list all the decimal places beyond 2 as zeros.

[Albert.Geven]

Hi

You are mistaken by the representation on the screen. Try a right click on the indicator that shows only two digits after the decimal sign. Select properties. There you can change the precision to say 6 digits. That should be enough for a low resolution scope.

The resolution shown is by default 2 digits. But internally it has a binary representation and for a floating put double you only are limited by the instrument.

[mhastings92]

I believe I am not conveying the issue correctly. I do understand that you can change the decimal precision of indicators but that does not seem to be the issue. To try and convey this issue more specifically I have attached documents that go into detail. The "Fetch Waveform.vi" seems to be processing the raw data in some way to display as numeric values. As the "details.pdf" attached shows, I have probed the data flow and the array values entering into the Build Waveform VI are already low decimal resolution due to that equation shown in the image that is used to process the data. Therefore the waveforms themselves already have low decimal resolution values when I am collecting it after this step in my main VI (so changing decimal precision there will not help with this). I was hoping to get some insight on actually what the "Fetch Waveform.VI" is doing and what that "data out" equation is. This whole issue is very confusing to me and I don't understand why the values of "data out" come out this way. Thank you again for all your help, I will continue to troubleshoot and maybe reach out to Tektronix if I cannot find a resolution (or at least find out who created this specific "Fetch Waveform.VI" that I am using as it came with Tektronix driver).

[Albert.Geven · Accepted solution]

Hi

the resolution of the TDS 2014B is 8 bit !

So if excel has more resolution it is probably wrong.