OASIS Opens Public Review for Test Assertions v1.0 Specifications

Stephen Green and Dmitry Kostovarov (eds), OASIS Committee Drafts

Members of the OASIS Test Assertions Guidelines (TAG) Technical Committee
have released three approved Committee Draft specifications for public
review through May 09, 2010. This OASIS TC was chartered in April 2007
in to "facilitate the creation and usage of test assertions by any group
involved in designing a specification or standard of which software
implementations are expected to be developed, with a primary focus on
OASIS technical committees. The first step in achieving this is to
establish a common and reusable model, metadata, methodology and
representation for TAs (test assertions).

"Test Assertions Part 1 - Test Assertions Model Version 1.0" specifies
"mandatory and optional components of a test assertion model. A test
assertion is a testable or measurable expression for evaluating the
adherence of part of an implementation to a normative statement in a
specification. It describes the expected output or behavior for the
test assertion target within specific operation conditions, in a way
that can be measured or tested. A Test Assertion should not be confused
with a Conformance Clause, nor with a Test Case... Test assertions
may help provide a tighter specification: Any ambiguities,
contradictions and statements which require excessive resources for
testing can be noted as they become apparent during test assertion
creation. If there is still an opportunity to correct or improve the
specification, these notes can be the basis of comments to the
specification authors. If not developed by the specification authors,
test assertions should be reviewed and approved by them which will
improve both the quality and time-to-deployment of the specification...
Test assertions provide a starting point for writing a conformance
test suite or an interoperability test suite for a specification that
can be used during implementation. They simplify the distribution of
the test development effort between different organizations while
maintaining consistent test quality."

"Test Assertions Part 2 - Test Assertion Markup Language Version 1.0"
defines a markup for writing test assertions, where Section 3 presents
the corresponding XML Schema for the vocabulary. Key markup language
elements include testAssertion, normativeSource (precise specification
requirements or normative statements that the test assertion addresses),
target (the implementation or part of an implementation that is the
object of a test assertion or test case, categorizing an implementation
or a part of an implementation of the referred specification, whether
a specific item or a category of items), prerequisite, predicate,
prescription, description, tag, var, report. etc.

"Test Assertions Guidelines" provides "guidelines and best practices
for writing test assertions along with mandatory and optional components
of a test assertion model. Its purpose is to help the reader understand
what test assertions are, their benefits, and most importantly, how
they are created. As you will discover, test assertions can be an
important and useful tool in promoting the quality of specifications,
test suites and implementations of specifications. You will learn that
there are many ways to create test assertions. By following the
guidelines in this document, you will learn how to develop well-defined
test assertions that can have useful purposes and applications such as
the starting point for a conformance test suite for a specification.
Experiences in developing test assertions will be shared, along with
lessons learned, helpful tricks and tools, hazards to avoid, and other
knowledge that may be helpful in crafting test assertions. This document
is limited to the essentials of test assertions..."