A couple things: First, regarding your instrument it seems strange for an instrument to make the type of error described--though heaven knows not impossible. One thing to check is the actual string being sent back. I did a little experiment. Assuming the string coming back is "2.90e-7" but the last character gets truncated, the string being converted is "2.90e-" which LV will dutifully convert to 2.90e0--hence your problem. First thing is check your serial io code to make sure that you are getting back the entire string every time. Where did you get the serial io drivers you're using? If you post them I will take a look at them for you. Also, is there a delimiter that the instrument sends at the end of every response, like a carraige return or a line-feed?
Bu
t my feeling is very much that the problem is with your serial io and that adding the filtering you mention is simply patching over it.
Mike...
PS: If tweaking the serial io doesn't help, I do have a median filter routine that should do what you are needing, and I'll send that to you.