> Thank you for understanding my question and suggesting to do something
> NI is expected to provide.
>
> Wouldn?t you say benchmarking of performance between LV versions is a
> valid and important technical spec for a version? Why then you expect
> a user to come up with it, given all inconveniencies of installing and
> uninstalling different LV versions on the same machine?
>
> It makes me suspect NI does not want to disclose that the addition of
> new bells and whistles to a new version usually comes up at a price.
> It slows LV down. Isn?t it the truth? And where are the benchmarks?
>
Keep in mind that I work for NI as one of the members of the development
team.
The way I look at it, there are three ways that people judge performance
depending on their personality.
1. They want to see something complicated, perhaps it needs to resemble
the app they want to write, but often it can be anything as long as it
shows that things are impressively fast.
2. They want to hear NI or another trusted source give them some
confirmation that performance is comparable. Often a tidbit such as
FFT's are 40% faster will be used as representative, even though it is
almost impossible to summarize the details in such a simple number.
3. People want to look at the raw data or measure it themself. They
want to stare at the plots, at the columns, and develop their own sense
of how things compare -- floating point is faster, but only when using
doubles, and it looks like some numeric formatting is slower, but by no
more than 10%.
#1 is best addressed by the games and examples on devzone and other
parts of the LV site. #2 is what the article comparing LV6 to LV6.1 was
doing, and since it didn't satisfy you, you probably want #3. That is
why I pointed out the VIs.
NI used to produce the log files for new releases to be used for
comparison, but to be honest it is so much data, that you cannot come up
with useful meta-numbers. Here are some reasons why.
People still use LV on 486, Pentium, P2, P3, and P4 processors. These
execute the same exact code in quite different ways depending on cache,
instruction ordering, and dual processor computers add still more
combinations. Next factor in OSes. LV still runs on Win95, 98, Me, NT,
2000, and XP. It would be nice if the OS didn't affect performance, but
the file I/O and memory management speeds will differ.
Scott listed a post to some benchmarks he did using the exact VIs I
pointed out on the ftp site. He summarizes that things in fact got
faster on PPC. His site also shows just how difficult it is to
interpret and summarize. Notice that he is measuring things on the
machines and OSes that he cares about, and his data probably doesn't
help you at all even though he shows lots of data to back up his summary.
So, your expectation seems to be that NI doesn't have a speedometer
attached to the new release, therefore we are embarrassed about our
performance. Here is my not-so-useful summary. Some things will be
faster, some slower, and overall, the performance is comparable given
the same OS and chip architecture. I know for a fact that the memory
manager is improved over 6.1, some code generation improvements were
added, single-point DAQ should be much faster, and while much new code
was added, most of it will not be executed when running your VI.
If you have some idea of what numbers are valid to compare, we will
consider doing the measurement. Or you can wait for someone to stumble
across the metric you want to see. Or you can get an eval and do the
measurements yourself using the VIs I mentioned or other VIs.
Greg McKaskle
