At the 2016 CLA Summit, I watched Philippe's presentation on Caraya and it made me think about a deeper question - could Caraya help me do Test Driven Development (TDD) in LabVIEW? I started exploring this idea and quickly was able to create a strategy that I think lowers the barrier for TDD enough for me to give it a shot again.
The highest level view of this approach is:
Create the test in the VI that is being developed instead of as a separate test VI.
In its simplest form, here is an example of the test template:
The benefits of this approach are: simplicity and usability without a run-time performance penalty. The only downside I can see initially is that the test code itself can make it harder to read the actual functionality being tested - but I think this is not an unacceptable trade-off. If the scaffolding gets too complex, etc, the scaffolding can be converted to a separate test VI and you will fall back to what we currently are doing in the LabVIEW community - but with the benefit that we at least *started* with a test first approach.
I've documented the entire project template (which includes dynamic setting of the "TDD" conditional symbol, and running tests as a pre-build step of the executable build).
After getting some initial feedback from CLAs including: Russel Blake, Fabiola De la Cueva (and others that I can't think of right now) and demonstrating the idea to a few more CLAs I felt pretty excited so I cleaned it up even more. I've created a VI Package that creates a project template for doing TDD. A quick disclaimer - I haven't yet actually used this template for TDD LabVIEW development yet, but I intend to start trying right away. I want to know if anyone else has feedback on this idea so I created the tool on GitHub so others can contribute easily. The Github repo for all source code to create the package is here:
I hope others are as excited as I am! Let me know your thoughts.
This is an interesting idea but for me seems to have some serious scalability issues when I'm using TDD:
This is interesting to see though and I'm excited to see some potential templates based on Caraya. It seems to handle the test cases well but the workflow is a bit more open/you have to think for yourself (which can be a good thing!)
Regarding your points:
1 - Autodetect - I'm assuming you mean like the comment in the build hook example ("Future version should support dynamic discovery of all callers of "Caraya.Define Test" method"? I don't think that is too difficult to do but it wasn't my first priority for creating the initial project template.
2 - I agree that often you'll need multiple test cases - in which case here's the crazy part - you can just invert the code to go back to what you previously did - the inversion process is this:
Rename your VI to "Test ABCD" (or whatever you want - Caraya doesn't care) and then create a subVI of the code under test as ABCD.vi. Effectively you fall back into the traditional developement path - but the potential benefit is that you started from a test first position.
The main obstacle I found so far is that in most of my coding I'm working on adding functionality to something that already exists so I almost never start from New VI and instead start most of my VIs from "Create SubVI from selection" - so I'm planning to create a scripting plugin (TBD) that can help with that part.
I like really this idea. I find it similar to a python library I like a great deal, doctest https://docs.python.org/2.7/library/doctest.html. As you can see that the tests serve a triple duty of testing, documenting, and providing use examples in one place. Granted the set of test vectors could get unwieldy, but I think that the generat benefits outweigh that.