What Test Engineers (and their managers) Should Know About Software

[Elijah_K]

I just published an article in our quarterly Instrumentation Newsletter entitled 5 Things Test Engineers Should Know About Software.  This article is just a basic introduction to many of the topics that are often discussion on this community and in the ni.com/largeapps white papers.

To summarize my points in the article:

1. If you are not using source code control, you are playing with fire.
2. Development without requirements is prototyping.
3. You can measure the testability and quality of code.
4. You may think your code works, but can you prove it?
5. Reuse is not a myth, but it requires planning.

This is by no means an exhaustive list of things you should know, but it represents a short-list of things that are often overlooked, especially when people are using LabVIEW.

I also recently put together a list of mistakes I see made by many users, over and over again.  These are all common examples that can lead to the duct-tape and hot-glue block diagram that we've all seen before.  As engineers, we all appreciate the cost of cutting-corners or taking a short-cut… it may get us there sooner, but at what cost?  In the world of software, there’s a concept known as ‘Software Engineering Debt.’  If we cut corners, we take on more debt, and these are all examples of how we do that.  The more debt we take on, the more interest we accumulate over time, and as we’ve all seen in the recent years, that debt can quickly pile up and accumulate a mountain of interest, making it more exponentially more expensive.  If we wait to pay down that debt until after we release a version of the software to our customer, it’s going to painful and the bugs will be much more costly than if we’d caught them up front.

1. No source code control (or Project) – if you’re not using it, you’re playing with fire.  Even for a small project with one developer, these tools can save you a tremendous amount of time and heartache.
2. Flat file hierarchy – ever had trouble identifying the top-level VI in an application?  It’s not just an organizational problem, it indicates a lack of encapsulation and proper design.  Use folders, encapsulate your code and make APIs obvious from private subVIs
3. ‘Stop’ isn’t tested regularly – there are lots of ways to assess the health of an application, but this is an easy one that you should always be thinking about.  A new feature is no good if it impedes expected behavior and one key indicator is whether or not the application still stops instantly when asked to
4. Wait until the ‘end’ of a project to build an application – if you’re one of those that waited until you were ‘done’ to build an exe for the first time, you’re probably a long ways away from done.  The reality is that, as with any language, a lot of things change in the RTE, and you need to test this on a regular basis
5. Few specifications / documentation / requirements – this is perhaps the most common mistake and the easiest way to ensure that code is not reusable or maintainable
6. No ‘buddying’ or code reviews – sometimes the mere act of explaining your code to someone else finds major problems or bugs… do it, and do it often
7. Poor planning (Lack of consideration for SMoRES) – This acronym stands for ‘Scalable,’ ‘Modular,’ ‘Reusable,’ ‘Extensible,’ and ‘Simple.’
8. No test plans – you can’t test code if you don’t define how it should behave.
9. Poor error handling – does it stop? Does it tell the user an error? Does it log the error?  If you don’t plan for this early, a simple error can lead to even more problems
10. No consistent style = not readible, not maintainable
11. Tight coupling, poor cohesion – impedes reuse

A lot of us are under pressure to get things done quickly and to hit aggressive deadlines, but there are a few practices that if we invest in early, will pay dividends down the road.  If you go through this list and find that more than just a few of these apply to you, time to re-evaluate your development process!

[Taggart]

You're preaching to the choir.  What the title should really be is "What Test Engineering Managers should know about software."

[LuI]

Elijah,

your #2 is great said: 'Development without requirements is prototyping.'

But good requirements are develloped as well and are part of the whole devellopment process. In contrast what some managers _think_ to provide requirements are in fact 'wishes'. Reqs do contain the limits on how to proove they are fullfilled. So not 'must be cheap', 'white surface' or 'low power consumption', but give a price level to fall below, a reproducibly testable color range (RGB-values, pantone etc.) or a power limit. This holds true for SW as well.

Just my € 0.02!

Greetings from Germany!

[Elijah_K]

Very true.  A requirement isn't really a requirement unless you can test it.  "It has to be safe" is not a requirement.  A requirement is "It shall have an numerical indicator for the systolic blood pressure that displays the value to 2 decimal places"