export large MS Access database files to labview

[DSMarcinko]

I'm analyzing data contained in an "Networked" MSAccess database. The files are large 300-800MB. Is there a programmatic method to use ActiveX commands to export the database table in text format to the hard drive for processing. I'd like to stay away from user interaction and having to use the export commands manually. I have the database connectivity tools but there are very few examples in LabView. Anyone out there have any suggestions?

[.avi file]

Hello!

  That sure is a lot of data. The manual mentions about the capability of using the cursors and talking to networked databases. I am not sure about a specific activex command but I shall investigate further with R&D and see if there is a possibile method. Following are few links if you have not had a chance to read them.
 
   http://www.ni.com/pdf/manuals/321525c.pdf
  http://zone.ni.com/reference/en-XX/help/371361B-01/lvconcepts/using_activex_with_labview/
   http://zone.ni.com/devzone/conceptd.nsf/webmain/7C21F34FBB2BCDEA86256DD6007DBD91?opendocument&node=6429_US

Thanks for your patience
Avi

[DavidJCrawford]

Take a look at the example Read All Data.vi which demonstrates using DB Tools Select All Data.vi.

DB Tools Select All Data.vi will give you a 2 array which you can then convert into a spreadsheet string and write to a file

Hope this helps

David

Message Edited by David Crawford on 09-13-2006 08:17 AM

[nyc_(is_out_of_here)]

I'm not understanding why you want to export the entire Access database file for processing.

Do you need every single piece of information each and every time?

Can you use SQL to get a portion of the data that you need for processing?

[DSMarcinko]

David,

I tried this vi, the data file is too large and it errors out (ERROR -2147024882). It Works fine with smaller files 100K rows X 46 col,  but it can't handle the 1,000K X 46 file I'm attempting to access. I've had NI working on a solution. I could pull one col at a time and write to a file between each pull, but that takes a long time and I have to rebuild the file in spreadsheet format after.

Thank you for your suggestion

[DSMarcinko]

nyc

The data is message traffic accross a bus. The data is comprised in rows that are 46 col wide and anywhere from 50K to 4M rows. The intent is to read each row and process the message to the PC display. MSAccess on the network is very slow so I'm attempting to bring the file to the workstation as text. 2G Ram is not enough to load the table as an array so I must resort to reading each line from a file and then processing.

I'm open to any and all suggestions

Thanks for the reply

[.avi file]

Greetings!

     As you probably have seen the issue related to error code 2147024882 indicate that there is not enough storage available to complete the operation when accessing a large table in a large database. It appears to be more of a memory issue. I assume you are in contact with John Marsh at NI, is this right? I will work with him and get back with you on a resolution. Thank you for your patience.

Best regards
Avi Harjani

[DSMarcinko]

Avi,

Thank you for the reply,

Yes, I'm in contact with John and he has been working to find a solution, I have confidence in both your abilities.

Thanks

[.avi file]

Greetings!
      After speaking with Jon, it appears you were able to implement a work around by writing to text files. It will be great if you post what you tried to the forums.
Thanks
Avi

[DSMarcinko]

Avi,

I was able to make progress after manually exporting the database file as .txt out of access to my local HD.

I still have not been able to get the commands to allow LabView to programmatically tell access to export the file as text to my HD. John said he would work with the ActiveX commands to determine how I could tell MSAccess to export the file and save as text on a users local HD.

Does this make sense?

Dave