high speed http GET help?

[wklove]

Hey I, I have a machine hooked up via RS232 to a standalone computer that has a C++ program that takes the machine information, converts it to xml, and posts it to the tcp/ip host.  I am trying to stream that data to network computer using labview and HTTP GET command and writing the values to binary file to be opened in excel.  My program works, but its only sampling twice a second and I assume that is because I have everything inside the loop and its taking too long to run through the program.  I was wondering how could I accomplish this faster because I need to sample like every 10-100ms.  Something in that range.  This program will be used for monitoring, but also for analysis so that is why the sampling speed needs to be faster.

I am completely new to programming and labview and I could use some real detailed help on this.  Ive attached my current program.  Please, any help or suggestions would be greatly appreciated.  I am working on a deadline and its taking me forever.  Thank you guys for your time.

[JackDunaway]

1. Try opening the TCP Connection only once before the loop starts, and close it once the loop finishes.
2. This is a classic candidate for what's called a Producer-Consumer design pattern. Two loops will run in parallel. The top loop will handle the TCP reads, relaying information via Queue to a second loop used for processing and file writing. (You could even have three loops in a stacked Producer-Consumer, where the top loop is TCP read, the middle loop is data processing, and the final loop is a file writer. In this scenario, you would have two queues). Research "Producer Consumer" and you'll find tons of resources
3. Typically, it's a goal to always have left-to-right data flow in wires, and also to contain your source code to one screen width/height
4. Consider creating a SubVI for each of the snippets that were copy-and-pasted in your Data processing (finding tags)
5. Use a Caption on your Front Panel String Indicators rather than another control in order give names to your outputs. Those unused Block Diagram terminals will then go away.

Try making all these changes, then post your VI and SubVIs back here and we can further optimize.

[wklove]

Hey, I have tried to make the changes you suggested.  Attached is the updated version.  It is still not working properly.  I am not sure if I did the producer consumer design correctly.  I am not sure I have the queue set up correctly or the correct things in the correct loops?  Could you or someone look at it and tell me if they see any glaring problems.

When I run it now, the xml response and pinpointed values are not posting.  The binary file is created, when I check that, it shows only one instance written.  When I put a numeric indicator on the loops, they still seem to be updating at that same slow speed.  

Can anyone provide some help or advice?  I need this program to update and write to the binary file much faster than what my previous program was doing (2 times a sec).  Preferrably at least every 100ms.  I apologize for being such a newbie at this stuff, but programming in general is not my strong suite. 

Sorry about the size of the block diagram, I tried to make it smaller so it wouldnt be so hard to follow.  I really need to get this working ASAP.  Thank you in advance for your help, it is greatly appreciated.

[wklove]

Whoops, here is the updated vi and the file written.

[JackDunaway]

Great job on the Producer-Consumer conversion! It looks like you put forth a good deal of effort and understand the general concept.

I mislead you when I said "Consider creating a SubVI for each of the snippets that were copy-and-pasted in your Data processing". What I meant was since each of the functions was so much alike, create ONE SubVI, then use multiple instances of that ONE SubVI. Practices such as creating reusable SubVI's promotes scalability and maintainability, yet Copy-and-Pasting code backs you into the corner of "technical debt" as the project scope increases. On inspection, it would just be better to process your data in a For Loop without needed a SubVI at all. Consider the following:

The efficiency of this code is not impressive, but it's maintains the spirit of the code you already have written, for the purpose of comparing what you have written to a more scalable solution.

Also, rather than having two 'Stop' buttons, you can stop the bottom loop by wiring the Error Out from the Dequeue directly into the Loop Stop Terminal. When the top loop exits from pressing the stop button, it will destroy the queue after exiting the loop, causing a (benign) error on the Dequeue in the bottom loop, causing that loop to cleanly exit. All of the File Write business should be in the No Error case of a case structure, so that it doesn't execute on this (benign and expected) error condition.

Finally, when you upload your top level VI, ensure to upload SubVI's as well. They show up as "?" when I open the main VI. I am especially interest in the VI you have named "TCP Read All.vi", because this is probably the culprit for your slow loop times.

[wklove]

Thanks again.  Is there anyway you could post the vi of that picture in a version of 8.5 or lower?  I am a little confused at what you did there.  Also, I have attached the subVis TCP Read All and its associated subvi's.  I did not write that part, found that from a search I did on this forum I believe and it seemed to work so you could be right on it slowing down my process.  I could see the glaring reason why it does, but maybe you could look at it and find a reason?

I apologize for being such a newbie,

[JackDunaway]

We need Check for All Content, and any SubVIs that may be on that diagram.

[wklove]

I think this is what you were looking for.

[JackDunaway]

Try increasing "Bytes to Read" significantly - how does this affect performance? Also, I am unable to post a snippet as a lower version, but perhaps you can recreate the code.

[wklove]

I was just thinking, it seems as though the tcp read function in the TCP Read All vi is set to immediate.  Could setting that to buffered help the speed?  The buffer size stated on the xml says 131072?  Should I just scrap that subvi and use basic tcp read function?  Its frustrating not knowing why its running slowly.  Thanks again for your help.