Specialist Task Force 575:
Methods for Objective assessment of Listening Effort based on
subjective test data bases
Who we are:
What we do
The work to be performed is :
- to create, at acoustical interfaces in the presence of background
noise, a set of:
- high quality reference speech samples
- test conditions which impair the reference items
- high quality recordings of the impaired reference
items (test sequences)
- to conduct a statistically significant number of auditory tests
(subjective tests) for objective model training (this task is not
included in the STF work plan) and model validation (this task is
included in the STF workplan)
- to produce the annex describing the whole work done by the STF.
Objectives of Task 1 – Creation of a set of test sequences at the
The objective of task 1 is to create recordings of speech samples with
various speakers in the presence of different types of acoustical
background noise. Three languages are selected: German, American English
and mandarin Chinese and the speech materials are based on contents
available in Recommendation IUT-T P.501. The types of background noise
cover a wide variety of different noises and the recordings must be at
least wideband, preferably super-wideband or fullband. These recordings
are provided in a quality suitable for the use in auditory tests. The
recording procedures comply with the corresponding clauses of TS 103
558. The background noise simulation in these experiments has to be
compliant with ETSI TS 103 224.
Objectives of Task 2 – Designing, conducting and analysing subjective
tests at the acoustic interface
The aim of Task 2 is to conduct subjective tests for the relevant
conditions as defined in Task 1. The results of the subjective tests
will be used to train the objective model (task not included in the
workplan) and will be compared to the output of the objective model for
the validation phase.
- One set of subjective tests is used to further train the objective
model in order to achieve a maximum correlation between the subjective
scores and the scores predicted by the objective model.
- The second set of subjective tests is used for model validation.
Task 3 – Overall coordination and project management
Tasks 1 to 2 will be performed by different labs (one lab per task) it
is important that there is an independent task defined which is
responsible to compile the various activities into the final
deliverables and to make sure that the requirement of the ToR are met in
full and in time.
For more details, see our Terms
Why we do it
The rolling standardization plan focuses on the need to improve speech
intelligibility, for normal or hearing-impaired people. To ensure that
there are speech intelligibility improvements, the long-term plan is to
define an ultimate objective model. This is a long-term objective, as a
lot of parameters have to be taken into account. So, we will progress on
two parameters: the impact of background noise and a parameter linked
with intelligibility, the listening effort (LE).
The full plan for such a project should be:
a) Design experiments to measure LE, using methods
that include all relevant aspects of accessibility for persons without
b) Collect a corpus of results over a wide range of
conditions relevant to contemporary telecommunications systems, with
initial focus on persons without hearing impairment
c) Development of an objective model for the
prediction of LE for persons without hearing impairment.
d) Explore relations between listening quality,
listening effort and intelligibility for users with normal hearing using
the new LE model. This step may require additional listening
e) Collect a corpus of results over the same
conditions as noted in b), but for persons with hearing
impairment. Due to the complexity of hearing impairments, this may
require extensive work,
f) Use the LE predictive model developed in step c)
with the additional data collected in step e) to extend the LE
predictive model for use with persons with hearing impairment.
The background noise to be taken into account depends on the interface
considered. The present STF focuses on the acoustical interface. An
additional work (outside the STF workplan) will consider the electrical
So, the present STF is intended to cover Steps a) and b) at the
The development of the LE predictive model Step c) will be developed
(and trained) outside the STF workplan. However, the STF is intended to
use a set of subjective test results to validate the model.
How we do it
The STF will produce:
- the recorded test sequences (mixing background noise and speech
- the test plans for subjective tests
- the results of the subjective tests (two sets of results). The first
set will be provided to the laboratory in charge of the model training.
It will be possible to communicate (in the STF deliverable) the
recordings and test results for the model training, but the results of
the tests for validation will be provided only after validation of the
model (the results shall be kept “secret” in the mean time).
The final deliverable will provide all the results obtained by the STF.
The main deliverable will be an annex to TS 103 558 (DTS/STQ-WI 264) for
the acoustical interface.
TS 103 558 is intended to develop methods for the objective prediction
of listening effort at the near-end side for the following two
1) at acoustical interfaces in the presence of background noise (in the
STF 575 workplan)
2) at electrical interfaces aiming at the effects of noise that
originates from the transmission chain or from the send side (outside
the STF 575 workplan)
The deliverable will detail the results of subjective tests for
listeners without hearing impairments.
The results of the STF575 will also be inputs to existing STQ documents.
|• Design of Test Plans for subjective test
• Test sequences for subjective testing
• Progress report to be approved at TC STQ#62
• Early Draft available
||• Conducting and
analysing subjective tests completed
• Progress Report to be approved at TC STQ#63
• Stable draft available
||Model for the validation available
||• Consolidation of tests results and
validation of the model
• Final Report and final draft to be approved
at TC STQ#64
||Deliverable published, STF closed
If additional information is needed, please mail to email@example.com
STF 575 will report on the progress at all the TC STQ Meetings :
STQ#61 Nuremberg(DE) 8-12 July
STQ#62 Boston (US)
23-27 September 2019
STQ#63 Bern (CH)
17-21 February 2020
STQ#64 Zagreb (CR)
29 June-3July 2020