Testing Task Force T014:
TTCN-3 maintenance & evolution of language and conformance test
suites 2021
Who we are:
| Team leaders:
|
|
| Team Members:
|
- Gusztáv Adamis, Ericsson, gusztav.adamis@ericsson.com
- Ramon Barakat, Fraunhofer FOKUS, ramon.barakat@fokus.fraunhofer.de
- Zoltán Jakó, Broadbit, zoltan.jako@broadbit.net
- Lénárd Nagy, Ericsson, lenard.nagy@ericsson.com
- Axel Rennoch, Fraunhofer FOKUS, axel.rennoch@fokus.fraunhofer.de
- Tomáš Urban, Elvior, tomas.urban@elvior.ee
- Jacob Wieland, Spirent Communications, jacob.wieland@spirent.com
|
What we do
The Testing Task Force (TTF) T014 manages the evolution of TTCN-3
in 2021 and develops in parallel the corresponding TTCN-3 tool
conformance test suites. This includes the maintenance and further
development of the ES 201 873 series of standards and the eight
extension packages ES 202 781, ES 202 782, ES 202 784, ES 202 785, ES
202 786, ES 202 789, ES 203 022, and ES 203 790. The development of
the TTCN-3 tool conformance test suites includes the maintenance and
further development of the ETSI technical specifications TS 102 950-1,
TS 102 950-2, TS 102 950-3, TS 103 253, TS 103 254, TS 103 255, TS 103
663-1, TS 103 663-2, and TS 103 663-3.
For more details, see our Terms
of Reference
Why we do it
The TTCN-3 testing language has intensively been developed by ETSI
during the last almost 20 years. By today, TTCN-3 has become a
significantly important testing technology in different domains (see
more details at
http://www.ttcn-3.org/index.php/about/references/applicatio-domains).
It is used by standardization bodies as well as by EU research projects
and open source initiatives. TTCN-3 reached very high deployment at
various ETSI member companies. The language is also endorsed by ITU-T as
the Z.16x and Z.17x Recommendation series.
In standardization TTCN-3 is an enabler technology for several
conformance, end-to-end and interoperability test standards. 3GPP uses
it for several UE conformance test suites from Rel. 8 onward, for LTE,
VoLTE and lately for NB-IoT. ETSI TBs INT, ERM. In the C-ITS area
several TTCN-3 test suites have been developed and they are playing
important roles in ITS PlugtestsTM events, with automated C-ITS
interoperability testing being in progress. In 2016 oneM2M has started
using TTCN-3 for IoT/M2M conformance test development that has been
continued in ETSI smartM2M from 2017. Other bodies and alliances using
TTCN-3 are TCCA, EUROCONTROL, MOST, AUTOSAR and oneM2M (see more details
at
http://www.ttcn-3.org/index.php/about/references).
In research at least 12 big projects from different domains are known to
use TTCN-3, among them the EU projects MIDAS, IoT.EST, ARMOUR, PHANTOM,
5GTANGO and SMESEC (see details in at
http://www.ttcn-3.org/index.php/about/references/projects).
In the smart grid area CEAList has developed a model driven testing
solution, using TTCN-3 to implement the user domain of the solution. The
open source Eclipse project IoT-Testware is using TTCN-3 to develop
conformance, performance and security test suites for IoT protocols with
major contribution from Fraunhofer FOKUS and relayr (
https://projects.eclipse.org/projects/technology.iottestware).
The Osmocom project is an open source initiative implementing mobile
communication standards, including GSM, DECT, TETRA, 3G and others
(https://osmocom.org/) and intensively using TTCN-3 for functional and
regression testing. TTCN-3 plays an important role in the industry as
well. TTCN-3 is used by several ETSI member and non-member companies as
an essential test enabler language (e.g. Ericsson, Easy Global Market,
Software Radio Systems).
Especially industrial users want low time to market of their new
products. For this reason, they have introduced agile ways of working
with continuous integration (CI) and continuous delivery (CD)
machineries. Agile and CI/CD are heavily relying on automated testing
(AT), including TTCN-3 based AT solutions. Resolving new requirements
and user requests with short response time is important for user
satisfaction and for keeping time-to-market low.
Significant number of TTCN-3 test toolsets are available on the market.
At least five commercial tools, five free or open source tools and one
internal test tool of an industrial ETSI member are known to exist (
http://www.ttcn-3.org/index.php/tools).
This also indicates the high interest and use of the language. TTCN-3,
as THE standard test language, serving several domains and application
areas, is specified in very detail. For example, the TTCN-3 core
language alone is estimated to contain about 5,000 requirements. It is
of upmost importance for users of standard test suites as well as for
industrial users that the TTCN-3 tools conform to the TTCN-3 language
standards. This can be secured by TTCN-3 tool conformance test suites,
in a similar way as implementations of other ETSI standards (e.g.
protocol specifications) are checked by means of ETSI-developed
conformance test suites. In the past, the TTCN-3 tool conformance test
suite development process itself has led to several language standard
clarifications.
TC MTS is committed to keep the language powerful still easy-to-use,
up-to-date, well maintained and following the changing user needs. The
series of TTCN-3 standards consists of 24 ETSI standards, altogether
comprising almost 1800 pages and 3321 test cases in TTCN-3 code (145895
lines in 3702 modules) today. This commitment requires very high
expertise and experts knowing the standards in detail.
How we do it
The TTCN-3 language evolution part of the work comprises the following
tasks:
• Review and resolve change requests reporting
technical defects or requesting clarifications and new language features
for all existing TTCN-3 language standards.
• Develop proposals for language extensions requested
by ETSI TBs, 3GPP, oneM2M, ETSI members and the TTCN-3 community and
consent the solution with the contributor(s).
• Implement agreed solutions.
• Manage the change request (CR) process.
• Manage the interim versions of the standard
according to 3GPP needs (when requested), and the versions for approval.
• Present the TTCN-3 standards’ status and the work of
the TTF (previously STF) at the conference(s) associated with ETSI TB
MTS and at ETSI TC MTS meetings.
• Providing input for the updates of the TTCN-3
leaflet and the TTCN-3 web pages.
The TTCN-3 tool conformance tests part of the work comprises the
following tasks:
• Analysis of the latest published versions of the
relevant TTCN-3 standards and identifying new and changed requirements.
• Identifying affected existing test cases and define
new test cases for the new requirements.
• Implement changes and additions in the textual part
of the deliverables (PICS, TSS&TP, textual part of the ATS).
• Implement changes and additions in the code of the
ATS.
• Verification of the test cases with test tools.
• Discussion of validation outcomes with TTCN-3
experts, and raising CRs for clarification in case of discovered
ambiguity
• Analysis of the latest draft versions of the
relevant TTCN-3 standards and development of unvalidated new test cases
for the new requirements
Deliverables
The maintenance and further development of TTCN-3 is based on change
requests provided by 3GPP, OMA, ETSI members and the TTCN-3 community.
The TTF resolves the CRs in agreement with the contributor(s) and
produces new versions for those standards that are affected by
resolutions. Potentially, the TTF produces new versions of the ES 201
873 series of standards and the eight extension packages ES 202 781, ES
202 782, ES 202 784, ES 202 785, ES 202 786, ES 202 789, ES 203 022, and
ES 203 790.
The development of the TTCN-3 tool conformance test suites includes the
maintenance and further development of the ETSI technical specifications
TS 102 950-1, TS 102 950-2, TS 102 950-3, TS 103 253, TS 103 254, TS 103
255, TS 103 663-1, TS 103 663-2, and TS 103 663-3. We deliver TTCN-3
tool conformance test suites which allow TTCN-3 tool vendors to validate
the standards compliance of their tool, and to locate possible
discrepancies.
Time plan
For all deliverables produced by TTF T014 the following time schedule
is planned:
• Final Drafts of T1 and T2
deliverables
31-Jan-2022
• TB approval of T1 and T2 deliverables
31-Jan-2022
• Publication
31-May-2022
For further details, or if you wish to be involved in the work of the
TTF, please contact the TTF Management: Jens Grabowski or Andras
Kovacs