Producer - Consumer/Producer - Consumer for improved performance?

[irPaul]

I think we're on the same page then. The attached snippet should describe what you say then.

Consumer 2 takes data directly whenever available. The data cluster has an Action (open, write or close, will be an enum), file path and reference, and the data of course. Within the Writing case, I create a data buffer - an array of data clusters for example.

I can use one typedef'ed cluster as described, no need to use variants for the few items in the cluster, right?

[_Y_]

1) Seems, loop 3 will always get the same data.

2) Variants. They are powerful but slow. If you have timing problems, avoid variants.

[Kevin_Price]

Re: variants

As poster "_Y_" cautioned, variants are slower than direct datatypes.  I've grown accustomed to using them for their flexibility and ability to support generic utility libraries, but they aren't the most CPU-efficient choice.  You're fighting a CPU usage problem so I'd agree with both of you that variants won't particularly help and there's no need to resort to them.  (For the record though, in many light-demand apps that might sling messages and data around to the tune of 10's of kB/sec, their inefficiency has been pretty indetectable.)

    The main benchmarking I've ever done with variants related to the use of variant attributes for a lookup table.  A lot of the discussions on the forum about the magical speed of variant attributes didn't appear to account for the overhead involved in converting the attribute back to its native datatype.   I included it in my testing and found that lookup tables based on matching pairs of arrays were faster until you got into the 100's of elements.  IIRC, other discussions seemed to suggest the crossover point was in the low 10's of elements.

-Kevin P

[irPaul]

I get the gist of using variants for their flexibility. It's a good trick I think, but based on your arguments I don't think I need it in this application. The actions are well-defined and the cluster is not unnecessarily big.  And indeed I'm fighting a CPU-battle here, all the small bits help.

Thanks for the input everyone! I will start rewriting this after the weekend.

[crossrulz]

---

@Kevin_Price wrote:

As poster "_Y_" cautioned, variants are slower than direct datatypes.  I've grown accustomed to using them for their flexibility and ability to support generic utility libraries, but they aren't the most CPU-efficient choice.  You're fighting a CPU usage problem so I'd agree with both of you that variants won't particularly help and there's no need to resort to them.  (For the record though, in many light-demand apps that might sling messages and data around to the tune of 10's of kB/sec, their inefficiency has been pretty indetectable.)

---

Personally, I like sending flattened strings as the data in my messages.  Most of my messaging was also over a network (PC to cRIO), so I had to flatten my data to do the TCP Write.  It is also useful if you have to communicate with non-LabVIEW applications.  I might have to put together a benchmark one of these days to see which is faster: To/From Variant or Flatten To/Unflatten From String.

[_Y_]

---

@crossrulz wrote:

---

---

 I might have to put together a benchmark one of these days to see which is faster: To/From Variant or Flatten To/Unflatten From String 

---

I did it some years ago and got unsignificant difference. Unfortunately I do not remember version of LabVIEW and which data types were tested.

[drjdpowell]

Variants to hold data aren't "slow".  Not at the 10fps the OP is talking about.  It's a false economy to avoid them.

[mcduff]

If I am passing data around, (this won't work in all cases), and I don't want to worry about variants slowing me down nor do I want to change my flexible variant data payload, I send a DVR reference instead. Read/Write the data only where needed and no need to pass a big data set around.

My 2 cents.

mcduff

[drjdpowell]

---

@mcduff wrote:

If I am passing data around, (this won't work in all cases), and I don't want to worry about variants slowing me down nor do I want to change my flexible variant data payload, I send a DVR reference instead. Read/Write the data only where needed and no need to pass a big data set around.

---

Not sure this helps the Variant overhead, as a variant is, I believe, a description of the data plus a pointer to the data.  There's no copying of large data required, so your DVR in a Variant is just as big as a giant array in a Variant.

Now, what does matter with big data is copies, which your DVR will probably prevent (though possibly at cost, due to the extra complexities of by-ref-data).

[wiebe@CARYA]

---

@mcduff wrote:

If I am passing data around, (this won't work in all cases), and I don't want to worry about variants slowing me down nor do I want to change my flexible variant data payload, I send a DVR reference instead. Read/Write the data only where needed and no need to pass a big data set around.

---

I often started with DVRs, only to refactor them out later on, without any performance penalty. When sending DVRs through events, preventing memory leaks can become tricky. And debugging with DVRs is always harder. DVRs are no golden ticket to performance. They can prevent data copies, but can also prevent parallel execution, slowing down the program. It's depends, but don't optimise if not needed...

Anyway, OO is helpul here. I made a data class, with a by wire child, and a by ref child. That made benchmarking much easier, and prevents switching back and forth.

OO is also helpful for the variant situation. Easy type would be a child... I doubt it would make any difference in performance. You can prevent casting to specific types, but only if you're in a good OO code base. It's hard to avoid this if it's the only OO peace of code.