Testing, Interoperability and Technical Quality

Servicesbreadcrumb separatorCentre for Testing & Interoperabilitybreadcrumb separatorETSI Approachbreadcrumb separatorInteroperability Testing

Interoperability Testing

The purpose of interoperability testing is to prove that end-to-end functionality between (at least) two communicating systems is as required by the standard(s) on which those systems are based.

The important factors which characterize interoperability testing are:

  • the Equipment Under Test (EUT) and the Qualified Equipment (QE) together define the boundaries for testing;

    NOTE: In the most rigorous case the test architecture includes equipment that has already been proven to interoperate with similar equipment from other suppliers. Such items are referred to as the Qualified Equipment (QE). In the less rigorous case, both parties may be EUTs.

  • the EUT and QE come from different suppliers (or, at least, different product lines);
  • interoperability tests are performed at interfaces that offer only normal user control and observation (i.e. not at specialized interfaces introduced solely for testing purposes);
  • interoperability tests are based on functionality as experienced by a user (i.e. they are not specified at the protocol level). In this context a user may be human or a software application;
  • the tests are performed and observed at functional interfaces such as Man-Machine Interfaces (MMIs), protocol service interfaces and Application Programming Interfaces (APIs).

The fact that interoperability tests are performed at the end points and at functional interfaces means that interoperability test cases can only specify functional behaviour. They cannot explicitly cause or test protocol error behaviour.

Development of ETSI Interoperability Test Specifications

A methodology for developing interoperability test specifications is described in EG 202 237. This methodology is based extensively on ISO/IEC 9646 but with some modifications to make it suitable for interoperability testing.

In EG 202 237 the Interoperable Functions Statement (IFS) replaces the PICS and is a statement of which functions are supported by the EUT.

Test Purposes should be written for each selected functional requirement and one or more Test Descriptions specified for each TP.

Unlike the conformance process, the specification of Test Descriptions s should not be considered as optional. In many cases, these are the actual interoperability tests that will be performed manually. It is only in cases where suitable user interfaces are available that it is possible to develop Test Cases in a language such as TTCN-3.

A typical interoperability Test Description is illustrated below:

Identifier:

TC_SS_0001_01

Summary:

Supervised call transfer from User B to User A

Test Purpose:

ensure that {
  when {
    A call is established between User_C and User_B
  }
  then {
    User_B can transfer the call from User_B to User_A
    after User_B and User_A communicate
  }
}

TP Identifier:

TP_SS_0001

Configuration:

Test Architecture 2

Pre‑test conditions:
  • User A, User B and User C configured with Bearer Capability set to "Speech, 64 kbit/s"
  • User A configured to support the Call Transfer service

Step

Test sequence

Verdict

Pass

Fail

1

Initiate new call at User C to the address of User B

2

Accept call at User B

3

Activate the "recall" button (or equivalent) at User B's terminal

4

Is dial tone (or an equivalent indication) present at User B's terminal?

Yes

No

5

Initiate a new call from User B to the address of User A

6

Is User A's terminal alerting (visual or audible indication)?

Yes

No

7

Accept call at User A

8

Apply speech at User A

9

Can speech from User A be heard and understood at User B?

Yes

No

10

Can speech from User A be heard and understood at User C?

No

Yes

11

Apply speech at User B

12

Can speech from User B be heard and understood at User A?

Yes

No

13

Can speech from User B be heard and understood at User C?

No

Yes

14

Clear call at User B

15

Apply speech at User A

16

Can speech from User A be heard and understood at User C?

Yes

No

17

Apply speech at User C

18

Can speech from User C be heard and understood at User A?

Yes

No

19

Clear the call at User A

20

Clear the call at User C

Observations: