Array of indexexes

[Rick Nelson]

ooh! In most cases like this, the end array will have to be the same
type of array that is coming in. So instead of initializing a new
array, you can just copy the original and do the whole replacing thing.
I got a 6% improvement. Unfortunately, my margin of error is very high,
and this would kill any chance of effective debugging (it's easier to
spot what goes wrong in an array of all 0's than an exact copy of the
original).

In this case, though, it's better to start a new array in this case
because having a U32 array is better than DBL.

I tried 🙂

Rick

[Greg_McKaskle2]

> If you compare Mike Porter's solution, you'll notice that it is at least 10x
> faster than the other versions discussed here. Apparently, growing an array
> (to an initially undetermined size) at the loop boundary is MUCH cheaper
> than doing the same with the built array tool as in B&C Duffey's version. I
> was not fully aware of that. I wonder if something could be more
> optimized...
>
....

> register with a worst-case (=size of input) array, use "replace array
> element" instead of "built array" inside the case, and, after the loop, you
> just strip it back to the final size with "array subset".
> This pre-allocates the array in one step and throws away the unused tail
> later.
>
> The speed difference is truly amazing! Could anyone c
omment on this...

I did a presentation at NIWeek that contains this exact
example along with other common performance problems.

Its examples should be on the ni.com web site with the
NIWeek files. This information is also in the user
manual in the performance chapter, used to be chapter 27.

As to why it takes longer to grow an array with build-array,
it comes down to the memory management that this causes.
An auto-indexing tunnel on a loop can figure out exactly how
many iterations a for loop will run, and it does some
estimates on while loops. When you place a build-array
inside of a loop and add an element, LV does them one at
a time. Someday, maybe LV will rewrite your diagram in an
equivalent, but more efficient manner, but for now, this
is something that is left to the diagram writer.

By the way, this was much slower before LV5. LV5 improved
inplaceness for build-array and some other nodes, and sped up
the build-array in a loop case by aobut 50-100X. This is still
slow co
mpared to the other techniques though.

Greg McKaskle

[Christian Altenbach]

Thanks Greg,
It was nice meeting you in Anaheim. I think it's impossible to come up with
something that you haven't thought up first. 😉 I always enjoy your input
to the mailing list and the newsgroup.

It is interesting, that the while loop (Mike's example) seems to tentatively
allocate the size of the largest input tunnel, because it always seems to
run extremely fast. This could be bad if your while loop requires only a few
iterations but has huge input arrays. Can you elaborate a little more about
the "estimates" that are taken? This will help us better decide which route
to take. I am dreaming that one day I will be able to simply look at code
and "see" which model is best in terms of speed or memory requirements ...
without benchmarking..

I had this idea for a "Houdini" tool. Basically, in certain iterations you
pipe your number through it and you get "something" of the same data type
that would NOT increment the output tunnel of the loop. This would solve
many problems similar to what we were discussing here.
Another option would be a little optional Boolean terminal on the output
tunnel where you can switch indexing on or off at will. (e.g. right click on
it would allow (disable indexing|enable indexing| enable conditional
indexing)).
There is probably something I am overlooking here.

Thanks
Christian


Greg McKaskle wrote in message
news:38260FF1.23A68BC6@austin.rr.com...
> I did a presentation at NIWeek that contains this exact
> example along with other common performance problems.
>
> Its examples should be on the ni.com web site with the
> NIWeek files. This information is also in the user
> manual in the performance chapter, used to be chapter 27.
>
> As to why it takes longer to grow an array with build-array,
> it comes down to the memory management that this causes.
> An auto-indexing tunnel on a loop can figure out exactly how
> many iterations a for loop will run, and it does some
> estimates on while loops. When you place a build-array
> inside of a loop and add an element, LV does them one at
> a time. Someday, maybe LV will rewrite your diagram in an
> equivalent, but more efficient manner, but for now, this
> is something that is left to the diagram writer.
>
> By the way, this was much slower before LV5. LV5 improved
> inplaceness for build-array and some other nodes, and sped up
> the build-array in a loop case by aobut 50-100X. This is still
> slow compared to the other techniques though.
>
> Greg McKaskle

[Greg_McKaskle2]

> It is interesting, that the while loop (Mike's example) seems to tentatively
> allocate the size of the largest input tunnel, because it always seems to
> run extremely fast. This could be bad if your while loop requires only a few
> iterations but has huge input arrays. Can you elaborate a little more about
> the "estimates" that are taken? This will help us better decide which route
> to take. I am dreaming that one day I will be able to simply look at code
> and "see" which model is best in terms of speed or memory requirements ...
> without benchmarking..
>

The For loop uses the numeric input at N, and any array
inputs that are being indexed. Before the execution of
the first iteration, the loop takes the smallest of
these
and sizes all of the outputs.

While loops don't know how many times they will execute,
so their first estimate is 16 iterations. If the loop
goes for more than 16, then it doubles to 32, and then
to 64. After that it adds 64 iterations whenever more
are needed.

This estimate works pretty well, and can be put inside
of any loop you like to make room for more items when
needed. In other words, instead of presizing the array
to the maximum size, it can be presized to 16 elements.
Then after each iteration, or after any element is replaced,
you can use resize to add element if necessary. When the
loop is over, use resize to shrink back to a tight fit.
This will use less memory, but run slightly slower than
estimates where you have additional sizing information
like an upper bound or a count estimate.

Greg McKaskle

[B & C Duffey]

Rick Nelson wrote in message <3821AA1C.EC334ED1@csciences.com>...
>I think that's what Duffy's solution is (not everyone can view PNG
>graphics). It's the most commmon way to do it, even though it might be
>slow on larger arrays (when it's better to alter the array instead of
>creating a new one).

My solution was not intended to be optimal - I created it in about 2 minutes
after reading the post - it took longer to attach the graphics files than it
did to write the vi. 🙂


>
>And, just to nitpick, you don't need anything hooked up to the "N", you
>can just hook up a small wire from the array to the FOR loop and it will
>auto-index.
> _____________
> |N|
> |
>Array =====|=====
> |=|
> |
>
>Just turn o
ff indexing for the first input, and on for the 2nd.


Yes this is true and I use this often, however since I elected not to use
the auto-indexing (if auto-indexing is disabled the loop cannot determine
the array size) feature I decided that the addition of a wire that
dead-ended into the loop edge might appear crytpic to a new LabVIEW user.

Chris

[Kevin_work]

B & C Duffey wrote:

> >
> >And, just to nitpick, you don't need anything hooked up to the "N", you
> >can just hook up a small wire from the array to the FOR loop and it will
> >auto-index.
> > _____________
> > |N|
> > |
> >Array =====|=====
> > |=|
> > |
> >
> >Just turn off indexing for the first input, and on for the 2nd.
>
> Yes this is true and I use this often, however since I elected not to use
> the auto-indexing (if auto-indexing is disabled the loop cannot determine
> the array size) feature I decided that the addition of a wire that
> dead-ended into the loop edge might appear crytpic to a new LabVIEW user.
>
> Chris

I thought I was a fairly decent LVer, but I never thought of usi
ng this.
I guess that is why I read this forum all the time. Never know what you might
learn !

I played with this a little just to see what was going on.
1) Using this approach saves about 0.6 KB (by the "Show VI Info.." window)

2) It is very slightly (sub millisecond) slower than the array size method.
(100 run average on 10,000 item array, 78.17 v 78.26 )

YMMV of course on these two.
I always look for ways to go faster and smaller.
3) This method will work on multi dimension arrays.
The catch here is that it only indexes the first dimension.
This may or may not be a problem depending on the intention
but it is something to be aware of.


A note to the original poster:
You may already know this but I have to say it anyway.
Doing comparisons (or almost any operation) on floating point numbers can be
tricky.
The precision displayed and that used for the operation are completely
different.
Normally you would multiply by the amount of percision you want, convert to an
intger
then do th
e calculations.

Kevin