Statistics Probes

[Evan P.]

5/26/2011 Edit: Probes updated to have consistent context menu name and various UI updates for the double probe that were missed.

___________________________________________

Act I: Intro

In the same vein that led to me writing the timing analysis probes, I have a simple question.

How often do you find yourself creating this code off some nice, innocent array, somewhere in your code?

If you're like I was, you do it alot.

Let's say it takes 2 minutes to drop this code (and organize it, because of my OCD). Another minute to get the iteration graph scaled in a useful way with plot points instead of lines, etc.

I WANT MY 3 MINUTES BACK.

How about this:

1.

2.

Zero configuration (required) and you get some very decent preliminary statistical analysis on your data set.

In about 2 seconds.

WIN.

If that's all you need to know and you want it, go ahead and download it. If you're thinking "That's trivial, I don't need that or I'll just make my own", keep reading.

Act II: More details for the nerds like me

Included are 10 identical probes for almost every 1D numerical array representation (Single, Double, Unsigned and Signed 8/16/32/64 bit Integers).

The key feature for these probes is that they aim to require no configuration to give a good visual analysis of your data set, regardless of how normal the distribution is. They do this by:

1. Attempting to smartly auto-scale based on the distribution of your data set (as opposed to just the max and min of the data set)
2. Attempting to smartly choose the bin size for the histogram based on the distribution of your data set (as opposed to just the number of elements in and range of the array)

• Iterations Tab
  
  • A value vs. iterations plot (white points) overlaid with an average value plot (red dashed line)
    
  • The value plot type point size is determined by the number of points in the array for visual clarity
    • Large (shown) used with 1-100 values
    • Medium used with 101-699 values
    • Small (single pixel) used with 700+ values
  • The graph is scaled to clip the most out-lying 2% of the values if they take up more than 30% of the vertical range of the graph
    • This is done to make visual analysis of the trends easier for highly skewed data sets and is typically desirable
    • This is still done 'loose fit' style to give sane graph marker values
  • Y axis values are displayed with SI units and 4 significant digits
• DistributionTab
  
  • Bin size for the histogram is chosen based on the smaller of Sturge's formula and the square-root choice; it then further compensates for the last 2% of the outliers to handle skewed data.
    
  • X axis values are displayed with SI units and 4 significant digits
  • Y axis values are displayed as a percentage (of values within the total data set)
• General Statistics
  
  • The following statistical metrics are shown:
    • Mean
    • Standard Deviation (1 Sigma, ~68% of values fall within this range around the mean)
    • Skewness (variation of the Std. Dev., -/+ value indicates left/right tailed set, small or 0 value indicates a relatively even distribution about mean)
    • Minimum and Maximum values within the set
    • 2,3, and 6 sigma values (~95%, 99.73%, 99.99% of values fall within this range around the mean)
  • SI units with 5 significant digits are used here
• Optional Configuration Inputs
  • 'Value Multiplier to Desired Units' can be used to scale the data to make the graphed units more desirable.
    
    • Ex: If you array contains uSec but you wnat to see Sec as the graphed SI unit, set the Multiplier to 1M (1,000,000)
  • Histrogram 'Bin Width Scaling Factor' can be used to scale the size of the histogram bins
    • Has a default value of 1, which is the automatically calculated bin size mentioned above. A value of .5 would cut the bin size in half, resulting in twice as many bins.
  • These configuration inputs must be set before execution of the probe

Yes, in a different world I would have more time to make this an X control that had dynamic interface and a sexy look and feel. No, these aren't perfect and dont' scale flawlessly for every data set out there. But we use them all the time and just find them too useful to not share them before they are perfect.

Check back every once in a while to see if there is a new version (edit blurb will be at top of post for new versions). If you get data sets that make them barf, please post them by saving the data set as the default value for an array in a VI!

TLDR;

Attached are a set of LabVIEW probes (compiled for 2009 and later) that can be used to probe arrays to give you a quick but useful statistical analysis of the data in that array.

Downalod, unzip and drop them in your LV Probes directory (usually My Documents\LabVIEW Data\Probes) to get started probing.

[donkdonk]

nice idea!

Your timing probes are even more useful. Should've been in LabVIEW natively.

Gerrit van Donk, the Netherlands