Logo

MacroExpressions Unit Testing: Maestra

Home Products Services Download Contact us
General introduction to MacroExpressions ^
<You are here>
Consulting, products customization, custom development C-Slang, Unimal and Snob are available for evaluation Email, telephone and fax numbers

Click on a button below to go to a related page
 

Table of contents
What is Maestra?
Why yet another method?
Maestra approach
Getting Maestra


 
Credits and acknowledgments
bellsnwhistles.com
 
   

What is MAcroExpressions Test Application (Maestra)? ( Contents)

(Initial intro is here. For a brief executive summary click here).
Complete motivation and description is available as PDF.

[Updated!] This is a new updated version. Same rationale, better implementation.

Maestra is a reference implementation of a C/C++ unit test environment described in the article "C code unit testing on a shoestring" by Ark Khasin (and also discussed below).

It is free for use and modification; however it comes without any warranty whatsoever.

The name stands for MAcroExpressions Test Run Application; it is also an Italian term for a (female) elementary school teacher. That seems quite appropriate for a testing environment.

Why yet another unit test environment? ( Contents)

If your C (or C++) software project is safety-critical (or safety-related) and you plan to certify it according to a safety standard, you need to meet certain requirements. Invariably, one of them is to produce unit tests documentation proving code/branch coverage of your tests.

In embedded software, it may become a requirement to run the unit tests on the real target (or on a close emulator or simulator). That means that the footprint of the test must be reasonably small. That also means using the same compiler and with the same switches as in production code; there may also be other reasons for it.

These are the areas where the free and widely accepted test environments, CUnit and CppUnit, fall short. They provide no proof of test coverage and include test and test result management in the runtime environment thus increasing the footprint dramatically. They may also require compiler adaptation (port).

There are commercial tools that got it right:

  • They use test harness configured at compile time, so a unit test build is reasonably lean.
  • They produce plain-text output which is to be analyzed by a separate tool (definitely running on a host machine) to extract pass/fail and code coverage information.
  • Test set management is done by a host-based tool
  • Code/branch coverage is achieved by automatically instrumenting the code in the unit under test, i.e., replacing the code with another, functionally equivalent but talkative (announcing its execution trace).

So, what's wrong with using the commercial tools?

First, they are very expensive. Thus, they make entry threshold into safety-related product development rather high.

Second, they are complex software systems with complex configurations. That means a steep learning curve.

Third, they require adaptation to your compiler. This may not be readily available. If it is, it may have defects.

Fourth, code instrumentation depends on parsing the code. And if the code uses clever constructs, there are realistic chances that it won't be parsed correctly.

That said, there are (and will be) people who got over those issues or learned to live with them. However, there is a different approach possible.

Maestra approach ( Contents)

Acknowledging that correct code parsing is a difficult task, let's observe that you already have a parser that can successfully parse you code whatever language extensions your compiler provides: it's your compiler itself. If only it could be used for code instrumentation!

Well, as it turns out, it's possible to instrument substantial amounts of your code by means of your C compiler alone, and without changing your code's appearance. The trick is to abuse the C preprocessor, namely, to redefine the keywords of the language.

There will be a couple of language elements that escape instrumentation. They can be dealt with in two ways.

First, as the article proposes, a coding standard can adopt a certain style of using the constructs, so that instrumentation of these constructs could be achieved.

Second, a script can be written to convert a source file to a form lending itself to instrumentation. Note though that it is no small task: A general-purpose script should be an almost full-blown C (or C++, as the case may be) parser. A script good only to cover your coding policy is much easier to write, but since you are restricting the coding policy anyway, you can restrict it to making the first approach work.

Whichever of the two paths you take, you get code instrumentation for free. The rest of Maestra is a very lightweight and small-footprint test execution environment of the instrumented code that produces text output. In that respect, it follows the footsteps of the major commercial test tools, leaving test report analysis to the host tools.
(Actually, it does a bit more, namely, keeping track of test cases, test and test sets passed, accepted or failed. It can also produce HTML - or, for that matter, XML - output to assist in evaluating manual acceptance criteria.)

More reading is here.

Getting and working with Maestra ( Contents)

Maestra is mostly a do-it-yourself method. You can download a free reference implemntation from the download page to get started.

From time to time, updates may be posted here. If you'd like to be notified when an update is available, please contact info@macroexpressions.com via email.