NI Home
Cart Cart | Help
Hello Events Academic NI Developer Zone Support Solutions Products & Services Contact NI MyNI
You are here: 
NI Home > NI Developer Zone > NI Discussion Forums


Reply
Active Participant
SCXI_and_MS_2k3-VB.NET
Posts: 334
0 Kudos

Periodic and Substantial Increase in the Time Required for File Output

I am logging data using continuous asynchronous acquisition and writing this data (~100 columns) to a file in ASCII format at 1Hz (10Hz will eventally be needed).  I believe that the harddrive in this PXI RT system is a 4GB solid state drive.  In my loop I am logging the number of microseconds required to carry out certain operations, and file output shows some odd behavior.  A (not perfectly) periodic occurence is happening where the write takes almost 100ms when a normal write is about 2-4ms.  What could be causing this issue and how can I remedy it?  NOTE: A chart is attached.

 

The line of code is just a fputs(msg, DataFile);

 

 

_______________________________
Michael Chadwell
Department of Engine and Emissions Research
Southwest Research Institute
Member
Danceman
Posts: 90
0 Kudos

Re: Periodic and Substantial Increase in the Time Required for File Output

Hello,

   Looking at that graph it seems like every 100 seconds or so there is some lag in recording the data.  With 4 GB of Solid State storage the only thing I can think causing this bottleneck is something in either the programming or O/S.  Therefore I have some questions for you.
     Where are you marking the time between portion of the code?  I would try the following timing measurement to see where the delay is.
      For (i = 0; i < iterations; i++)  

         Tic();  // Start Timer
         //   Gather the data
         Gatherdata_Time[i] = Toc();  // Get time since Tic() 

         // Write Data to String format
         ProcessingString[i] = Toc();  // Get time since Tic() 

         // Write data to HDD
         Writetodisk_Time[i] = Toc();  // Get time since Tic()  
      End For loop 

Daniel de Gaston 

 

Active Participant
SCXI_and_MS_2k3-VB.NET
Posts: 334
0 Kudos

Re: Periodic and Substantial Increase in the Time Required for File Output

My devbox won't boot right now, but I can tell you that I have 4 timers using the GetMicroseconds (psuedonym) function.  I get the us JUST before and JUST after the file write call.  I also get the entire loop time, the time to build the string, and the time to send the string over TCP (though this time is not correct because TCP is threaded)

_______________________________
Michael Chadwell
Department of Engine and Emissions Research
Southwest Research Institute
Member
Danceman
Posts: 90
0 Kudos

Re: Periodic and Substantial Increase in the Time Required for File Output

Is the graph I see in your first post the micro-seconds for just the write call?   How much data are you writing?

Active Participant
SCXI_and_MS_2k3-VB.NET
Posts: 334
0 Kudos

Re: Periodic and Substantial Increase in the Time Required for File Output

Yes. About 600 characters.
_______________________________
Michael Chadwell
Department of Engine and Emissions Research
Southwest Research Institute
Member
Danceman
Posts: 90
0 Kudos

Re: Periodic and Substantial Increase in the Time Required for File Output

Could you try using the following lines of code.

      fprintf(File_ID, "%s" , msg);

      Determine timing.

      printf("%s" , msg);

      Determine timing. 

 

The point of this test is we can then determine if the following delay is found just with writing to the file or hard disk drive.   If the long delay is only found on the first fprintf then I would say we look into hard drive issues.  If it is found on both print statements,  I think the Operating system or system resources are being tied up by something else at the moment.

Active Participant
SCXI_and_MS_2k3-VB.NET
Posts: 334
0 Kudos

Re: Periodic and Substantial Increase in the Time Required for File Output

Will printf work in real time systems?
_______________________________
Michael Chadwell
Department of Engine and Emissions Research
Southwest Research Institute
Active Participant
SCXI_and_MS_2k3-VB.NET
Posts: 334
0 Kudos

Re: Periodic and Substantial Increase in the Time Required for File Output

See the file below.  I'll run it longer, but I was just able to bounce out of an all day training event for lunch.

_______________________________
Michael Chadwell
Department of Engine and Emissions Research
Southwest Research Institute
Active Participant
SCXI_and_MS_2k3-VB.NET
Posts: 334
0 Kudos

Re: Periodic and Substantial Increase in the Time Required for File Output

The columns for time are at the end and they are | Build Msg Time | FileIO Timer | PrintF Timer | 1Hz Loop Time

 

I used my method of getting the time GetTimeUS.  I hope that was OK.

_______________________________
Michael Chadwell
Department of Engine and Emissions Research
Southwest Research Institute
Member
Danceman
Posts: 90
0 Kudos

Re: Periodic and Substantial Increase in the Time Required for File Output

Hello,

    I feel like we should use the Real-Time execution trace in determining what is taking the most time.  Whether this is a speed issue with cache data or data transfer the following links should help us determine this.

                http://sine.ni.com/nips/cds/view/p/lang/en/nid/209041

                http://forums.ni.com/t5/SignalExpress/Flush-log-buffer-to-disk-while-running/m-p/701887 

 

Daniel

By using this web site, you accept the Terms of Use for this web site. Please read these Terms of Use carefully before using any part of this site. Please go here for information on ni.com's copyright infringement policy.
My Profile | Privacy | Legal | Contact NI © 2011 National Instruments Corporation. All rights reserved.    |    E-Mail this Page E-Mail this Page