 What is MAcroExpressions Test
Application (Maestra)? (
Contents)
(Initial intro is here. For a brief
executive summary click here).
Complete motivation and description is available
as
PDF.
[Updated!]
This is a new updated version. Same rationale, better
implementation.
Maestra is a reference
implementation of a C/C++ unit test environment
described in the article "C code unit testing
on a shoestring" by Ark Khasin (and also discussed
below).
It is free for use and
modification; however it comes without any warranty
whatsoever.
The name stands for MAcroExpressions
Test Run Application; it is also
an Italian term for a (female) elementary school
teacher. That seems quite appropriate for a testing
environment.
Why yet another unit test environment? ( Contents)
If your C (or C++)
software project is safety-critical (or safety-related)
and you plan to certify it according to a safety
standard, you need to meet certain requirements.
Invariably, one of them is to produce unit tests
documentation proving code/branch coverage of your
tests.
In embedded software, it
may become a requirement to run the unit tests on the
real target (or on a close emulator or simulator). That
means that the footprint of the test must be
reasonably small. That also means using the
same compiler and with the same switches as in
production code; there may also be other reasons
for it.
These are the areas where
the free and widely accepted test environments, CUnit
and CppUnit,
fall short. They provide no proof of test coverage and
include test and test result management in the runtime
environment thus increasing the footprint dramatically.
They may also require compiler adaptation (port).
There are commercial tools
that got it right:
- They use test harness
configured at compile time, so a unit test build is
reasonably lean.
- They produce plain-text
output which is to be analyzed by a separate tool
(definitely running on a host machine) to extract
pass/fail and code coverage information.
- Test set management is
done by a host-based tool
- Code/branch coverage is
achieved by automatically instrumenting the
code in the unit under test, i.e., replacing the code
with another, functionally equivalent but talkative
(announcing its execution trace).
So, what's wrong with
using the commercial tools?
First, they are very
expensive. Thus, they make entry threshold into
safety-related product development rather high.
Second, they are complex
software systems with complex configurations. That means
a steep learning curve.
Third, they require
adaptation to your compiler. This may not be readily
available. If it is, it may have defects.
Fourth, code
instrumentation depends on parsing the code. And if the
code uses clever constructs, there are realistic chances
that it won't be parsed correctly.
That said, there are (and
will be) people who got over those issues or learned to
live with them. However, there is a different approach
possible.
 Maestra approach (
Contents)
Acknowledging that correct
code parsing is a difficult task, let's observe that you
already have a parser that can successfully parse you
code whatever language extensions your compiler
provides: it's your compiler itself. If only it could be
used for code instrumentation!
Well, as it turns out,
it's possible to instrument substantial amounts of your
code by means of your C compiler alone, and without
changing your code's appearance. The trick is to abuse
the C preprocessor, namely, to redefine the keywords of
the language.
There will be a couple of
language elements that escape instrumentation. They can
be dealt with in two ways.
First, as the article
proposes, a coding standard can adopt a certain style of
using the constructs, so that instrumentation of these
constructs could be achieved.
Second, a script can be
written to convert a source file to a form lending
itself to instrumentation. Note though that it is no
small task: A general-purpose script should be an almost
full-blown C (or C++, as the case may be) parser. A
script good only to cover your coding policy is much
easier to write, but since you are restricting the
coding policy anyway, you can restrict it to making the
first approach work.
Whichever of the two paths
you take, you get code instrumentation for free. The
rest of Maestra is a very lightweight and
small-footprint test execution environment of the
instrumented code that produces text output. In that
respect, it follows the footsteps of the major
commercial test tools, leaving test report analysis to
the host tools.
(Actually, it does a bit more, namely, keeping track of
test cases, test and test sets passed, accepted or
failed. It can also produce HTML - or, for that matter,
XML - output to assist in evaluating manual acceptance
criteria.)
More reading is here.
 Getting and working with Maestra ( Contents)
Maestra is mostly a
do-it-yourself method. You can download a free reference
implemntation from the download
page to get started.
From time to time, updates
may be posted here. If you'd like to be notified when an
update is available, please contact info@macroexpressions.com
via email.
|
