LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

How to link SQL server to Labview with Database Toolkit

Solved!
Go to solution

There you go. You posted before I figured out by myself. lol...

 

What I found is that the SQL server table name must starts with letter, not any NUMBER. I haven't try the space thing yet, but when my table contains only number, the vi gives error. But when I add a letter in front of the number, it works perfect. Just like the diagram shows. I will check the spacing thing and get back to you.

0 Kudos
Message 11 of 23
(5,884 Views)

Oh, to answer your question: 

 

Besides, why would you create a new table for each date? And why would you have 200 columns? What is your database schema? 

 

 We are doing a monitoring project in an advanced sustainable residential twin house, where we have more than 200 sensors installed in each house. We are tracking almost everything, every aspect related to the energy flow, water/air temp/flowrate, relative humidity, power consumption of each equipment, PV generation, HVAC system, etc... And I'm acquiring data every 1s and recording averaged data every 5s. I guess it's better (don't laugh if I underestimae the capability of SQL server) to create a table according to date, so that it's easy to handle the data?

 

I'm not sure about the size of the table for each day (few hundred MB?). If it's huge in a single day, it might be problematic in a year round? 

 

0 Kudos
Message 12 of 23
(5,883 Views)

Last reply before going to bed. It seems only eliminating space won't work. I can do the date change by adding a letter string in front of my data.

 

Thanks for all the help. 

0 Kudos
Message 13 of 23
(5,872 Views)

If you have identifiers (column names, table names) that contain characters that the SQL parser considers as syntax elements you have to tell the parser that these terms should be interpreted as identifier instead of trying to interpret them as syntax elements. For identifiers containing spaces you can do that for most SQL databases by enclosing the entire identifier into brackets, like [my spacy column].

 

Your SQL database may also allow to indicate columns having names starting with a number to be identifiers by enclosing them in the same brackets but it is generally a bad idea anyhow to name columns in such a way as it can limit the possibilities later on to do things like stored procedures or other similar stuff and some ODBC or other client type libraries my still not support that syntax. Most SQL parsers will however interpret any collection of characters starting with a number to be a literal number rather than a possible identifier for a table or column.

 

Rolf Kalbermatter

Message Edited by rolfk on 08-20-2009 10:11 AM
Rolf Kalbermatter  My Blog
DEMO, Electronic and Mechanical Support department, room 36.LB00.390
Message 14 of 23
(5,867 Views)

holyna wrote:

Oh, to answer your question: 

 

Besides, why would you create a new table for each date? And why would you have 200 columns? What is your database schema? 

 

 We are doing a monitoring project in an advanced sustainable residential twin house, where we have more than 200 sensors installed in each house. We are tracking almost everything, every aspect related to the energy flow, water/air temp/flowrate, relative humidity, power consumption of each equipment, PV generation, HVAC system, etc... And I'm acquiring data every 1s and recording averaged data every 5s. I guess it's better (don't laugh if I underestimae the capability of SQL server) to create a table according to date, so that it's easy to handle the data?

 

I'm not sure about the size of the table for each day (few hundred MB?). If it's huge in a single day, it might be problematic in a year round? 

 


My underlying assumption here has been that you're running SQL Server on a server, and not on a desktop. The full-blown version of SQL Server (which requires a purchased license) isn't really meant to be run on desktops. While it can, desktops don't usually have the necessary file, memory, and network I/O throughput that a server would have. You have previously mentioned that you're running SQL Server (apparently the full-blown version, and not the Express version) on your computer, which doesn't make much sense to me, given the volume of data that you intend to collect. 

 

As far as your database design, you seem to be approaching your database design as a "flat" design. From this respect, there's little difference between what you're planning to do, and simply saving your data as files, such as tab-delimited text files or even Excel spreadsheets. You're basically going to create a new table (spreadsheet), for each date. The power of SQL Server (and other similar database systems) is in being able to create N-tier databases, and establishing the relationships between tables.  I don't really see the point in creating a separate table based on date. By making the sensor count into columns you're also hard-coding the table design. What happens if you decide to add sensors, or remove sensors? If I were approaching this with a mind to create an N-tier database I'd have a table that defines sensors. Each sensor would have an ID. Then, the table that stores the measured data would have (this is a simplification) 3 columns: date, value, and sensor ID. The sensor ID would come from the sensor table I mentioned. This is an example of 2-tier database. Getting data from this table is relatively simple, as you can query based on data, sensor, value, whatever via SELECT statements.

 

My recommendation would be to spend some time reading some materials on database design. Given the volume of data that you're going to be collecting, and the project I think it would be time well spent. 

Message 15 of 23
(5,849 Views)

Yes. Our University has the full-blown version. The PC we are using, also as the server, is equipped with Intel Dual Core @ 3 GHz, and the ram is 3.25 GB. It has XP Pro. 2002 version on it. It is dedicated for this monitoring project. We will of course back up data from time to time.

 

As we don't have any database expert in our group, I assumed to use SQL as an excel layout. Do you mean this "simple-idea" design will cause difficulty later on in our analysis? If so, I think I should put everything into the 2-tier (3 columns) DB instead. I got your point how to design a database, but obviously I need to spend more time looking into that.

 

Now, I have a question on how to obtain the sensor ID. As I mentioned in the beginning of the post, I'm using Fieldpoint system. I think instead of "ID", it gives "Fieldpoint IO Point Out". I assume that corresponds to the channel ID ? If so, how can I convert that "Point out" as some kind of string as the ID for the sensor? I mean, in terms of VI, how can I achieve that.

 

Another basic question here, if we save all data into one table, can we backup the database without interrupting the vi programming? I think it should be ok.

 

Thanks again for your advice. 

 

 

0 Kudos
Message 16 of 23
(5,839 Views)

Rolf,

 

I realized my misunderstanding on the identifier issue. Although I may not need the table name by date in the future design, I appreciate your clarification. 

0 Kudos
Message 17 of 23
(5,835 Views)

Your Universtity would certainly have a full time IT staff that could be consulted on database design and most computer science students could give valuable advice as well.

 

I can't concieve of any design where a table name by date would even remotely be a good idea.

Message 18 of 23
(5,832 Views)

smercurio, one more question as I go further. In term of VI, I use Real-time project and have the deterministic loops and non-deterministic loops. The structure is that acquired data from multiple modules (cFP-AI-111, RTD-122, CTR-502, etc.) are pre-calculated (convert from the raw signal such as mA, mV, V, Pulse, Oum into whatever the physical signals such as Temp, flowrate, power, etc.) and sent to shared variables. Each shared variable corresponds to each module (I can probably use queue function to lump different types of signal together). There's a reference talking about how to use queue function to combine different types of signals, but I can't find it right now. I'm just wondering which one is better (faster) between shared variables and queue functions.

 

Getting back to the DB issue, I record my data into DB in the non-deterministic loop, where I have to specify which signal goes to which column (my initial design). If we do the 2-tier design, how should I link multiple shared variables to 3 columns in term of vi programming?

0 Kudos
Message 19 of 23
(5,828 Views)

holyna wrote:
 

Now, I have a question on how to obtain the sensor ID. As I mentioned in the beginning of the post, I'm using Fieldpoint system. I think instead of "ID", it gives "Fieldpoint IO Point Out". I assume that corresponds to the channel ID ? If so, how can I convert that "Point out" as some kind of string as the ID for the sensor? I mean, in terms of VI, how can I achieve that.


The ID can be anything you want. It does not need to be a specific "property" of the item. It can be something as simple as an increasing index. An example of how it's used is in the page that talks about JOIN statements on the w3schools site. Here you can see that each person is assigned an ID. In this case it's just an increasing index (which SQL Server can automatically create when you add new entries to a table). The Orders table links to the Persons table via the person ID, which is a column in the Orders table so you know who placed that order. 

 


Another basic question here, if we save all data into one table, can we backup the database without interrupting the vi programming? I think it should be ok.

The issue here isn't with the VI running and trying to add records, it's with being able to back up databases. While SQL Server is running the database files are open. Most backup utilities that don't cost oodles of money will not back up open files. Some have SQL Server plug-ins, but they cost money. Here at work I have a SQL Server Agent (a tool internal to SQL Server) task that runs right before our nightly backup. The SQL Server Agent simply backs up the database to a separate directory, so the normal backup backs up that file, without having to try to deal with open files.

 


There's a reference talking about how to use queue function to combine different types of signals, but I can't find it right now. I'm just wondering which one is better (faster) between shared variables and queue functions.


I haven't used shared variables so I can't really say which one is better or faster.

 


Getting back to the DB issue, I record my data into DB in the non-deterministic loop, where I have to specify which signal goes to which column (my initial design). If we do the 2-tier design, how should I link multiple shared variables to 3 columns in term of vi programming?

You would add a row for each reading from each sensor. In other words, instead of writing data horizontally (across 200 columns) you'd add data vertically using multiple records. If we continue with the example I provided, then you'd be adding records that would require a sensor ID. You'd first need to lookup the sensor ID based on the sensor that generated the data and use that ID as the value to insert into the column of the table that holds the data.

 

Let's take a concrete example. Let's say you use 2 tables. One table defines sensors, called "tblSensors" or something, that looks like this:




ID Module Channel RawMeas PhysMeas
1cFP-AI-1111VoltFlowrate
2RTD-1225OhmTemp

 

Now let's say you have the data table, let's called it "tblData" that looks something like this (with some dummy data):

 







ID Date RawMeas PhysMeas SensorId
108/16/2009 02:24:36 PM
0.35
2.31
208/16/2009 02:24:37 PM0.352.31
208/16/2009 02:24:38 PM0.362.41
208/16/2009 02:24:36 PM110027.12
208/16/2009 02:24:37 PM120027.52
208/16/2009 02:24:38 PM100026.82

 

The data here indicates three readings, once a second, from 2 sensors. 3 readings x 2 sensors = 6 records. When you want to add, say, a flowrate reading, then you would need to know the sensor ID that corresponds to the sensor that measures flowrate. In this case it's 1.  You can get this one of 2 ways: (1) from SQL Server using a SELECT statement on the tblSensors table; (2) have a lookup table that you just keep that's synchronized with the SQL Server table.

0 Kudos
Message 20 of 23
(5,811 Views)