LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Idea to split my 1D-multiple-ramps alternatively?

 

 

 

Hi everyone,

 

I have here a VI  for splitting a .tdms file which may contain multiple ramps. The concept is pretty simple; for example I have 10 appended ramps at the input, each ramp is a simple 1-100 ramp function which looks like a sawtooth function.

The task of this VI is simply to split the ramps (take each element and put them into smaller files, in  .csv or .txt format).

 

1D array split.png

 

Also, if I set the parameter "File numbers" to 100,  "Word-length" to 1 and running this program, I would get 100 smaller files:

File 1.csv would have 10 elements of "1"

File 2.csv would have 10 elements of "2"

...

File 100.csv would have 10 elements of "100"

 

 

Alternatively, if I set the parameter "File numbers" to 10, "Word-length" to 10 and running the program, the results are:

File 1.csv would have 10x smaller ramps from 1-10

File 2.csv would have 10x smaller ramps from 11-20

...

File 10.csv would have 10x smaller ramps from 91-100

 

 

This VI runs okay if the source file has total elements less than 10 million elements. But it takes too much time to finish if I have for example 50 million elements.

I guess, there is an alternative way to split the ramps faster, but I cannot find a better way at the moment.

 

I would be glad if someone has an idea to split the ramps faster 🙂

 

 

Any helps are really appreciated,

Yan.

 

 

 

 

0 Kudos
Message 1 of 8
(2,521 Views)

Why do you need the source file? Why not just generate the data yourself on the fly? Once you do that, you won't have to deal with large amounts of data (with 50M DBLs you would need 400MB for each copy of the array).

 

Can you upload the code and a sample file?


___________________
Try to take over the world!
0 Kudos
Message 2 of 8
(2,503 Views)

hi tst,

 

The main purpose of the vi is to sort the arrays from the .tdms file.

You can test yourself to generate a simple ramp in .tdms file with the included vi.

 

I need to develop this vi so I can for example extract the ramps; take out 1, 101, 201, and build smaller ramp of 11 elements (2-12, 102-112, ...)

 

have a look at the vi's

 

regards,

yan

Download All
0 Kudos
Message 3 of 8
(2,491 Views)

Assuming that you really have to read the data from the TDMS file and that you will have such large pieces of data, I would suggest you try reading the data from the file in chunks. I have absolutely no experience with TDMS, so I can't really comment on this, but I see that the TDMS Read function does have inputs for setting the offset and count and that the advanced palette also has some functions which seems to support reading only parts of a file.

 

If that works, then you just need to calculate where the data you need will be in the file and I believe the code you have already does that. Basically, you will just need to move the reading part into the for loop and calculate the offset and count for each reading. Assuming that the TDMS functions know how to read only part of a file, this will prevent you from having a large array in memory and will allow you to do what you want.


___________________
Try to take over the world!
Message 4 of 8
(2,477 Views)

Tst is right about being able to read the TDMS files in chunks.  That is what the offset and count is there for.  I'm done similar things with binary files (try reading and processing a 2+GB file).


GCentral
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
0 Kudos
Message 5 of 8
(2,473 Views)

yes you are right,

I was too focused with complicated ideas, while actually the basic function (read in chunks) is available in the read vi.

 

thanks, I'll try what I can do with tdms read vi.

 

Yan.

 

0 Kudos
Message 6 of 8
(2,452 Views)

hi,

I've tried to modify my code, but i'm facing another problem.

I'd like to split my ramp which has 1309 elements (1+(109x12)) into 109 new files, and do it like this:

take out element 1,

put element 2 ~ element 13 into first file,

put element 14 ~ element 25 into second... until 109 files done

 

take out element 1309,

put next 12 elements into first file... until 109 files done

 

my problem with the recent code, i keep getting in first file: 

2,3,4,5,6,7,8,9,10,11,12,13,2,3,4,5,6,7,8,9,10,11,12,1309,2,3,4,5,6,7,8,9,10,11,1308,1309,...

instead of (what i would have):

2,3,4,5,6,7,8,9,10,11,12,13,2,3,4,5,6,7,8,9,10,11,12,13,... always the same order

 

I know i'm not programming the "take out" element not in the right way, any help would be really appreciated.

 

regards,

Yan.

 

code

0 Kudos
Message 7 of 8
(2,421 Views)

simple solution to my own question

 

code2

I'm changing the input of my (-1) function and seems to work fine

0 Kudos
Message 8 of 8
(2,415 Views)