Put your DAQmx data acquisition into a while loop. Put your limit_max and limit_min numerical controls into this while loop.
At every iteration of your while loop you create certain number of data (if you have multiple channels and you wanna get N numbers of data, go for the 2D array version of DAQmx Read).
Depending on the array size, if it is not too big, just check if all the numbers are inside the range (In range function). If there are numbers out of range, you can put info (channel number, number of outliers, number of detection, etc...) into an array of clusters (+shift register, and use build array-concatenate in a case structure).
You just have to put a condition into the while loop, and when you got 5 times outliers, you stop the while loop, and report the data from the cluster toward the user.
If you have a high speed DAQ with high number of samples, you may get problem with data analysis in the DAQ while loop. In this case use a producer/consumer structure with Queue.
yes, but for this you do not need a state machine. Just do a read, and analyze it in the while loop. Of course the story is different if the system needs to move to different states. But here we have only one condition: numbers are in range? If yes, go on with your while loop, if not, either you can stop your loop or give warning toward the user. Just use a simple pop-up window for this, the user can decide... KISS principle 🙂
edit: I guess you put AO DAQmx by mistake in your example, we deal here with AI