Specialist Task Force 470:
Extension of Conformance tests for TTCN-3 tools
Who we are:
Team Leader: Kovacs, Andras / Broadbit Slovakia,
andras.kovacs@broadbit.com
Team Members: Jakó, Zoltán / Broadbit Slovakia,
zoltan.jako@broadbit.com
Urban, Tomas / Elvior,
tomas.urban@elvior.ee
Pintar, Bostjan / Sintesio,
pintar@sintesio.org
Pakulin, Nikolay / ISPRA, npak@ispra.ru
What we do:
The purpose of this work is to continue the implementation of a TTCN-3 tool
conformance test specification for TTCN-3 tools, previously done through STF
409/433/451, i.e., we specify ICS, test purposes and TTCN-3 tests for those
aspects of the TTCN-3 core language standard, which are considered to be the
most critical by the TTCN-3 community.
For more details, see our
Terms of Reference
Why we do it:
Over the last 10 years
TTCN-3 has become a significantly important testing technology with very high
deployment at various ETSI member companies as well as other organizations
internationally. With its uptake by 3GPP, OMA and the AUTOSAR (a consortium of
car makers and OEM suppliers), its role in standardization is further growing.
3GPP is using TTCN-3 as the test specification language e.g. for IMS call
control testing from Rel. 8 onward and. TC TISPAN is using TTCN-3 for NGN
supplementary service and interworking test specification. TC INT is using
TTCN-3 for automating IMS core network interoperability testing. TC ITS is using
TTCN-3 to specify all its test suites under the EC mandate M/453. Since 2004
ETSI organizes an annual, international user conference where it collects
feedback from the wide TTCN-3 user community. The first ETSI TTCN-3 tools
Plugtest has been organized in 2009.
How we do it:
Significant number of
TTCN-3 compilers are available on the market (at least 7 commercial tools and 2
internal tools of industrial ETSI members) that also indicate the high interest
and use of the language. The TTCN-3 standards which provide the foundation for
this testing technology, are however quite complex and encompass multiple
hundreds of pages. Part 1 of the TTCN-3 series, the TTCN-3 core language, alone
is estimated to contain on the order of 5,000 requirements. Over the past 10
years the TTCN-3 community has repeatedly requested by for some kind of
assurance that tools conform to TTCN-3 standards. A standardized testing
language such as TTCN-3 should lead by example, i.e., tools that check
conformance of systems to standards should also be tested for their compliance
to standard.
Based on these requests, TC MTS developed a first conformance test suite in STF
409 which lead to the discovery of 19 issues or ambiguities in the TTCN-3
standard version v4.2.1 when covering about 1/3 of the clauses in the main
standard with some tests. The test suite was extended, re-based to TTCN-3 v4.3.1
by STF433 and a new coverage analysis approach, reaching 38 % coverage of the
core language clauses at least with a few test cases. STF 454 has upgraded the
test suite to version v4.4.1 of the core standard and extended its clause
coverage to about 87%. TTCN-3 is, however, a living language, a new version is
published by ETSI each year, the latest published version is v4.5.1. Up to date
the TTCN-3 conformance test suite development is one release behind the language
development.
Deliverables:
The STF will produce the following deliverables (revision according to CR
Process), final drafts will be approved by correspondence during 2Q2014):
RTS/MTS-102950-1ed141 TTCN-3 Conformance Test Suite ; Part 1
: Implementation Conformance Statement
RTS/MTS-102950-2ed141 TTCN-3 Conformance Test Suite ; Part 2:
Test Suite Structure & Test Purposes
RTS/MTS-102950-3ed141 TTCN-3 Conformance Test Suite ; Part 3:
Abstract Test Suite & IXIT
Time plan:
The STF will produce the
deliverables according to the following time scale:
- TB adoption of WI May 2013
- Draft for approval May 2014
- TB approval (AbC)
June 2014
How to contact us:
On general inquiries,
please contact the team leader Andras Kovacs by e-mail
(andras.kovacs@broadbit.com). Otherwise, please use the e-mail addresses stated
above in the “What we do” section.
This information is based upon STF working assumptions.
The views expressed do not necessarily represent the position of ETSI in this
context.
Last updated: 2013-10-09 16:19:58