How Generic are Your Modules?

[mbaudot]

Hi Ryan, 

I've been using DQMH for a while now and databases are something I deal with very often, so I hope I can give you some good hints.

Database is definitely a good candidate for a DQMH Module. Of course, you can use the Database Toolkit directly in the code where you need database access. However, enclosing this functionality in a DQMH Module will make your application more modular, and a lot easier to maintain if you want someday to use another database technology or driver.

What I've been doing in the past is a Generic Database DQMH Module, that is cloneable, and is responsible for:

- Open Database Connection

- Perform Database Query (Request and Wait for Reply)

- Close Database Connection

I don't really like the idea of broadcasting the database reference outside of the module. The reference should be kept private in the module since this module will be the only one to talk to the database.

This Module will be reusable as is in any of your application that requires Database access.

Then you would have other DQMH Modules that will be more specific and will call your Database Module to perform specific queries on specific databases and tables. An example could be a Database Logger Module with the following Requests:

- Start Module (open Database connection on Logging database during Module Initialize)

- Log data (call the Database Module to perform an Insert query in your DB)

- Stop Module (close Database connection).

Now, this module depends on the Database DQMH Generic module and can be reused in any application that needs this type of logging. 

And so on...

To answer your question on how many modules can you have in an application.

I'm working on applications that use around 100 DQMH Modules and this number is still increasing. More modules equal more decoupling, which means more modularity and maintainability. Of course, the design needs to be smart as well.

Some notes regarding the NI Database Toolkit. If you look at most of the Database VIs, they are not reentrant. This is because it's using ActiveX in the back, which is not reentrant by design. It doesn't mean you cannot use it in your Cloneable Database DQMH Module. However, your module will not be able to perform queries truly in parallel.

You might want to consider other database drivers, or reimplementing the functionality using pure TCP IP if you absolutely need simultaneous queries to be performed. If you are using MySQL in the backend, this code works pretty well and allows to perform truly reentrant queries to a MySQL DB

https://forums.ni.com/t5/Example-Program-Drafts/Native-LabVIEW-TCP-IP-Connector-for-mySQL-Database/t...

Hope this helps!

Matthias Baudot.

[joerg.hampel]

Great reply, Matthias! You beat me to the punch, and honestly you took the words out of my mouth.

We have a generic database DQMH module just as Matthias describes, with open, query, close requests. If needed, you could implement different database systems inside this generic module, to hide the details from users. I also wouldn't suggest broadcasting the reference if there isn't a really good reason for it.

I also agree with his comments regarding the Database Connectivity Toolkit, which we don't use. For Access, MSSQL or Oracle, we use an ADO DB driver, which also uses ActiveX, so all of Matthias' comments apply to it, as well. Most of the times, we actually use MySQL over TCP/IP and SQLite.

As to the number of modules, we're not in the 100s, yet. Anyhow, the number itself shouldn't affect your design decisions. Again, as Matthias already said, as long as your design has no flaws... 

One thing though where I agree with James is transactions. If you need those, there's no way around either a cloneable DQMH module or putting the database driver directly into the modules that need them. 

Again, thanks Matthias for your elaborate answer.

[drjdpowell]

Well, I still don't get it.  "Open", "Query" and "Close" could be three subVIs in a library/class.  Having any extra structure seems redundant.  Not having transactions is significant, and I don't know how you handle binding parameters (hope you aren't doing dangerous/ugly string formatting instead).

[mbaudot]

---

@drjdpowell  a écrit :

Well, I still don't get it.  "Open", "Query" and "Close" could be three subVIs in a library/class. 

---

You're right, it could be just SubVIs in a library or class. But the DQMH Module is a library designed to be reusable and easily testable. DQMH comes with an API Tester and Productivity Scripting Tools.

Now instead of just having a Library with my Methods and SubVIs, I have a library that I can test and validate with its own API Tester (ok you can write examples for your library and it's the same. But the API Tester comes for free with DQMH which saves a lot of time and more important : ensures that you actually have a tester!).

Adding new API methods (DQMH Requests) to my library becomes super fast and easy with the Scripting Tools. You can also save your DQMH Module as a Template if you want to reuse it and modify it for multiple different projects.

I guess that is why people who start using DQMH are then tempted to create a DQMH Module whenever they need a reusable API, instead of just creating a simple library. That could seem a little overkill in some cases but I believe this is a good design decision because at least you stick with a Framework that has been proven and tested.

There is nothing in the DQMH Module that would prevent you from using SQL transactions and bind parameters. The DQMH Module as a private data cluster that can take care of retaining and binding your parameters together.

Hope this clarifies a bit how DQMH can be useful and efficient in this situation.

[drjdpowell]

---

@mbaudot wrote:
But the DQMH Module is a library designed to be reusable and easily testable. DQMH comes with an API Tester and Productivity Scripting Tools.

---

Libraries of subVIs are reusable and easily testable.  DQMH makes the considerable achievement of making a asynchronous API using typed User Events something of comparable complexity, even though the equivalent of a simple subVI call involves two User Events with associated registration plumbing.  But it is not as simple or reusable as a class library.  

I often find it is best to mix both, with a simple class to act as a "driver" (low-level functions)  and a module (Messenger-Library "actor", in my case) that uses the driver.  Then the driver can be used in more than way in different Modules for different applications.

[ryan_sheppard]

---

@mbaudot wrote:

Hi Ryan, 

I've been using DQMH for a while now and databases are something I deal with very often, so I hope I can give you some good hints.

Database is definitely a good candidate for a DQMH Module. Of course, you can use the Database Toolkit directly in the code where you need database access. However, enclosing this functionality in a DQMH Module will make your application more modular, and a lot easier to maintain if you want someday to use another database technology or driver.

What I've been doing in the past is a Generic Database DQMH Module, that is cloneable, and is responsible for:

- Open Database Connection

- Perform Database Query (Request and Wait for Reply)

- Close Database Connection

I don't really like the idea of broadcasting the database reference outside of the module. The reference should be kept private in the module since this module will be the only one to talk to the database.

This Module will be reusable as is in any of your application that requires Database access.

Then you would have other DQMH Modules that will be more specific and will call your Database Module to perform specific queries on specific databases and tables. An example could be a Database Logger Module with the following Requests:

- Start Module (open Database connection on Logging database during Module Initialize)

- Log data (call the Database Module to perform an Insert query in your DB)

- Stop Module (close Database connection).

Now, this module depends on the Database DQMH Generic module and can be reused in any application that needs this type of logging. 

And so on...

To answer your question on how many modules can you have in an application.

I'm working on applications that use around 100 DQMH Modules and this number is still increasing. More modules equal more decoupling, which means more modularity and maintainability. Of course, the design needs to be smart as well.

Some notes regarding the NI Database Toolkit. If you look at most of the Database VIs, they are not reentrant. This is because it's using ActiveX in the back, which is not reentrant by design. It doesn't mean you cannot use it in your Cloneable Database DQMH Module. However, your module will not be able to perform queries truly in parallel.

You might want to consider other database drivers, or reimplementing the functionality using pure TCP IP if you absolutely need simultaneous queries to be performed. If you are using MySQL in the backend, this code works pretty well and allows to perform truly reentrant queries to a MySQL DB

https://forums.ni.com/t5/Example-Program-Drafts/Native-LabVIEW-TCP-IP-Connector-for-mySQL-Database/t...

Hope this helps!

Matthias Baudot.

---

Matthias,

I appreciate the detailed response. 

Your Generic Database Cloneable Module was exactly what I has in mind when I made this post. Although I was missing the Perform Database Query request, which is why I had thought about broadcasting the reference to another module. I see now why this would not be a good idea. I am fairly unfamiliar with databases in general, so as I learn more about databases I will be able to make better design decisions for with the modules I create for them.

When it comes to how many modules a project has, that was my idea behind this post. If I create modules that handle specific functions, and not complete solutions I would find they are reusable across many applications. My product owner really pushes this idea, so it is something I am really focusing on.

Again, thank you for your response.

Ryan Sheppard

[joerg.hampel]

After re-reading this thread, I'd just like to add that the combination of what Matthias says (cloneable DQMH module with generic API) and what James suggests (simple but specific "driver" used inside module/actor) is what our database module actually looks like.

Update: Actually, the first reply in this thread was Darren stating the exact same thing. Thanks, Fab!

So, thank you for your ideas and input, guys! Good to take a step back from time to time to verify that one's development is still on course 🙂

[FabiolaDelaCueva]

---

@joerg.hampel wrote:

After re-reading this thread, I'd just like to add that the combination of what Matthias says (cloneable DQMH module with generic API) and what James suggests (simple but specific "driver" used inside module/actor) is what our database module actually looks like.

So, thank you for your ideas and input, guys! Good to take a step back from time to time to verify that one's development is still on course 🙂

---

Also inline with what Darren suggested of using classes within the DQMH Module for the more specific calls.