Serious Performance hit on RT (cFP 2120) using LV 8.5 compared to LV 8.2

[Ben]

Hi Geir,

When my engineer (mellowbuck) returns from his teaching trip I am sure he will be more than happy to supply all of those details.

Ben

[Ching_F.]

Hi All,

I would like to correct myself....I should probably check with the LabVIEW Real-Time team before I start saying things next time.   An increase in CPU usage is not expected with an upgrade in LabVIEW.  I'm currently working with the LabVIEW Real-Time Product Support Engineer (PSE) to determine what is causing this increase.  I will let you know what we find after he's had a chance to review the traces with me in more detail.

As for the compiling in Vista versus XP, that should not have any effect on your performance.  Not removing unused polymorphic VI instances etc. could explain some of the issues you are seeing.  You are correct that this option is no longer located under Advanced Settings, but you should be able to define what you would like to exclude under the Additional Exclusions section when you build your executable.  Please check that you are not including anything you do not need.

I'm currently gathering traces from a couple of upgraded programs to see what is causing the problem.  Again, once I get to review them with the Real-Time PSE one of us will reply to this forum with our findings.

[Ben]

It takes guts to correct yourself.

Thanks for clearing that up.

In this thread Jarrod mentions a performance hit with intergers to enums. Since this construct is used in the SDE, are there any performance hits by using the SDE?

Ben

[Ching_F.]

Hi Ben,

I don't know if using the SDE will cause the performance hits that you are seeing.  Speaking with the LabVIEW team, if you are going to see a performance degredation, it will be the similar on Windows as on an RT taget. 

[Riconquistiamola]

Hi Geir,

Ching informed me of this post and I found it necessary to comment after reading it. First, I apologize you have not received appropriate support for this issue. The Discussion Forums are a public space to discuss problems, and our limited involvement in this post is disappointing.

That having been said, Jimmie's involvement in troubleshooting this issue has been extremely helpful. He clearly stated, "If I see a performance hit when running the code using 8.5 compared to 8.2 then we know that it is due to the new RT version" and "What I wanted to achieve with the test though was to see if I could see similar behavior and a reasonable assumption is that this very issue you have encountered should be present on all targets running 8.5 and not just on cFP-2120." At this point, NI concluded this was not a general problem with FieldPoint targets running 8.5 (a relief, if I may say so myself).

At this point I think we failed to follow-up with the next obvious step: continued troubleshooting. This example may be the simplest piece of code you can show to reproduce the issue, but here are my immediate suggestions/observations regarding the code. I am curious how the code executes when you make these changes.

• Two sets of shared variable libraries. You mentioned you removed them from your application and the CPU stayed elevated.
• Removing parallel while loops. I assume if you removed the shared variable communication there is no longer a second while loop executing.
• Timed Sequence Structure around While Loop. I'm guessing the logic in your code mandates this, but I'm curious what happens when you replace both with a single Timed Loop.
• While Loop around For Loop. Same logic as above, but still a good test. I would try removing one, then the other.
• For Loop Shift Registers. Is the elevation reproduced without these?
• Reading all FP channels from IO Point. Is the elevation reproduced with single point IO?
  

You also made several references to someone at NI taking ownership of this issue. The NI Discussion Forums are loosely monitored by NI employees (being one of the few companies to do so), but the continued enthusiasm of Champions like Ben make posting to the forums such an effective way of getting problems solved. Have you spoken to your local sales or Applications Engineering office about creating a Service Request to formally investigate this issue? This is definitely your best course of action, considering the delays to your application release.

Cheers.

[geirove]

---

@Riconquistiamola wrote:

Hi Geir,

Ching informed me of this post and I found it necessary to comment after reading it.

.......
Cheers.

---

Hello Michael,

As I stated to Ching and Ben in this thread a few posts back: The clue here is my test code: I still do no have a clear understanding of the following:

a) Do NI engineers replicate the problems I am having using my test code?

b) If not, what is the difference betweeen my setup and NI's? We are both running cFP 2120, the same code why should this perform differently?  Are you compiling under Vista, because that is the only difference....

What you suggests is time consuming testing that will stall my current development, something the client will not tolreate and I cannot justify for the moment.

If NI is not able to locate the source of this problem, I can do more testing later in 2 - 3 weeks as it looks right now. I will get back to you on this.

But please, please, do your very best to reproduce the problem.

Geir Ove

[Ben]

Thanks for the reply Michael.

Speaking for the problem we (myself and mellowbuck) have seen.

The application running on the cFP 2120 runs OK if the app was deployed from LV 8.2.1.

It does not if deployed from LV 8.5.

Same app. same hardware!

The only diference is LV 8.5 vs LV 8.2.1.

By "runs OK" I mean the application keeps up with all of it tasks. When its is "not OK" one or more of the functions falls behind.

It sounds like the CPU is being starved.

mellowbuck has forwarded our code to support and should be available for your testing.

Trying HARD to make LV as great is it can be,

Ben

[MRedRaider]

Let me preface this post by saying that I don't have FP or RT, BUT:

I noticed a slow down in performance of FOR loops in 8.5 versus 8.2 and 8.2.1. The problem seemed to be VERY computer specific. The same application sometimes ran faster as a while loop than a for loop... something that seemed to baffle a few people at NI. We eventually decided that I was doing to much data manipulation inside the loop that the loop itself couldn't be blamed.... however, I have noticed this problem every so slightly appearing in other applications that I have built with FOR Loops regardless of hardwired N or conditional terminal based.

Link to the discussion:

https://www.blogger.com/comment.g?blogID=32599616&postID=1768227989695021259

Take from it what you will, but I thought this information needed to at least be mentioned.

[geirove]

Hello NI Support Team,

I am re-reading this thread, and must say that I am quite a bit dissapointed with the progress of this issue. It is now 14 days since my first post. In the 5. posting in this thread, I posted a very simple application (1 small VI !) that reproduces the problem.

Since then, 3 different NI engineers has been on the case, and the issue is still not resolved! Michael (from NI support) is now suggesting that I do a lot of different tests on the test app. We have gone into the effort to produce a VI that reproduces the problem on our side. Now I strongly feel that NI should do testing on this very small test app to find the source of the problem!

As I have said before: The simple questions are:

a)  Can or can't NI reproduce the problem?

b) If NI cannot reproduce the problem; what is the difference between NI's and our compilation setup?

I sincerely hope that LV 8.5 & cFP 2120's RT is stable so that problems may be reproduced.

We still have one client waiting for their final release that we wanted to do on LV 8.5 (and not on the slow to edit LV 8.2). Seems like we have to tell the client; Sorry, LV 8.5 cannot be used because it generates to slow code...

[Riconquistiamola]

Geir,

Thank you for your post. To immediately answer your simple question, Yes, we have reproduced the problem. Your questions seems to suggest we have not made any attempt to reproduce. This is not the case. Ching responded on 10-11-2007 01:54 PM that, "Using your example and your set-up I did see an increase in CPU usage, but I did not get the same results you were getting." I set up your example code myself using a cFP-2120 and both LVRT 8.2.1 & 8.5. I see the performance degredation you have mentioned.

In the same response from Ching, she asked if you could run an Execution Trace on your code. I have not seen you respond to this request, so I ran an Execution Trace myself. It appears the increased CPU usage is caused by decreased execution speed of the FieldPoint Read/Write VIs. This is the source of the problem.

When Ching first heard of your problem, she created an extremely simple VI (see below) to test CPU increase. This is VI also identifies the FieldPoint Read/Write VIs as the source of the problem. This VI is a more simplistic implementation of your code and has been the focus of my continued investigation. I am currently setting up tests with this VI to determine whether a linear increase in FieldPoint Read/Write VIs causes a linear increase in CPU usage. I expect results by the end of the day today. The results of this test could explain Ben & Tim's dramatic performance hit when upgrading to LVRT 8.5.

However, the larger issue with your post has been the inability to release your application to your client. It does not seem possible for you to test any troubleshooting steps I have recommended nor downgrade to LVRT 8.2. At this point we have done everything requested of you. How can we help you release your application?

P.S. -- You mention, "I am quite a bit dissapointed [sic] with the progress of this issue. It is now 14 days since my first post." First, it takes 24 hours for you to post and us to respond. Second, in several posts (and an email) Ching and I have asked you to contact your local office to gain the assistance of the Applications Engineering Department. This is the most efficient way to expedite resolving your issue. I cannot emphasize this enough.

Message Edited by Riconquistiamola on 10-23-2007 05:09 PM