TDX size increasing due to CFC Filter?

[TamerZero]

I have a script that runs through different processes on a dataset. Part of it is a user selected CFC filter.

The data is collected at 400kHz, there are 143 channels. When I select no filter the data saves to approximately 278KB/54MB for the TDM/TDX. When I do select a filter for the data the data saves to 282KB/188MB. While 188MB isn't out of control, it is 3.5 times larger than the unfiltered set. I noticed the filtered dataset carried a few more decimal places, so I truncated, but that didn't seem to make the filesize any smaller.

Any input on this? The only difference between the two datasets is these lines of code:

If FilterClass = 0 Then
   Call Data.Root.ChannelGroups(GroupNo).Channels(iLoopC).Properties.Add("Filter","Unfiltered",23)			
      Else
        Call ChnCFCFiltCalc("["&GroupNo&"]/[1]","["&GroupNo&"]/["&iLoopC&"]","["&GroupNo&"]/["&iLoopC&"]","CFC_Free",0,"EndPoints",FilterClass) '-- Filter
	Call Data.Root.ChannelGroups(GroupNo).Channels(iLoopC).Properties.Add("Filter","CFC " & FilterClass,23)
      End If

[AndreasK]

I assume you are measuring in 16 bit integer.

After using the filter your data become 64 bit double which causes the increase in size.

May this be the answer to the behavior?

[TamerZero]

That sounds logical, thanks Andreas. Is there anyway to convert back after the filtering?

[AndreasK · Accepted solution]

There is a method

Analyze->fx->Quantize

ChnQuantize

That can be used to force a bitness.

Another way is to use

Analyze->fx->Optimize Data Type

ChnOptimizeDataType

which may not work in your case because the filter is really making it float64, so you hav e to force it.

[TamerZero]

Thank you Andreas. This got my file size down to 50MB. As you stated, Optimize Data Type did not work for me (it was what I tried previously).

I wrote this script to quantize the entire dataset to 16 bit:

Dim iLoopC, GroupNo, ChnMin, ChnMax
GroupNo = 1
  For iLoopC = 1 to Data.Root.ChannelGroups(GroupNo).Channels.Count
		ChnMin = Data.Root.ChannelGroups(GroupNo).Channels(iLoopC).Minimum
		ChnMax = Data.Root.ChannelGroups(GroupNo).Channels(iLoopC).Maximum
		Call ChnQuantize("["&GroupNo&"]/["&iLoopC&"]","["&GroupNo&"]/["&iLoopC&"]",65536,ChnMin,ChnMax)
		Next