STFbreadcrumb separatorSTFsbreadcrumb separatorSTF HomePagesbreadcrumb separatorSTF575

Specialist Task Force 575:

Methods for Objective assessment of Listening Effort based on subjective test data bases

Who we are:

 

Team leader:
Team Members:

What we do

The work to be performed is :

- to create, at acoustical interfaces in the presence of background noise, a set of:

    - high quality reference speech samples

    - test conditions which impair the reference items

    - high quality recordings of the impaired reference items (test sequences)



- to conduct a statistically significant number of auditory tests (subjective tests) for objective model training  (this task is not included in the STF work plan) and model validation (this task is included in the STF workplan)

- to produce the annex describing the whole work done by the STF.



Objectives of Task 1 – Creation of a set of test sequences at the acoustic interface



The objective of task 1 is to create recordings of speech samples with various speakers in the presence of different types of acoustical background noise. Three languages are selected: German, American English and mandarin Chinese and the speech materials are based on contents available in Recommendation IUT-T P.501. The types of background noise cover a wide variety of different noises and the recordings must be at least wideband, preferably super-wideband or fullband. These recordings are provided in a quality suitable for the use in auditory tests. The recording procedures comply with the corresponding clauses of TS 103 558. The background noise simulation in these experiments has to be compliant with ETSI TS 103 224.



Objectives of Task 2 – Designing, conducting and analysing subjective tests at the acoustic interface



The aim of Task 2 is to conduct subjective tests for the relevant conditions as defined in Task 1. The results of the subjective tests will be used to train the objective model (task not included in the workplan) and will be compared to the output of the objective model for the validation phase.

- One set of subjective tests is used to further train the objective model in order to achieve a maximum correlation between the subjective scores and the scores predicted by the objective model.

- The second set of subjective tests is used for model validation.



Task 3 – Overall coordination and project management



Tasks 1 to 2 will be performed by different labs (one lab per task) it is important that there is an independent task defined which is responsible to compile the various activities into the final deliverables and to make sure that the requirement of the ToR are met in full and in time.



For more details, see our Terms of Reference

Why we do it

The rolling standardization plan focuses on the need to improve speech intelligibility, for normal or hearing-impaired people. To ensure that there are speech intelligibility improvements, the long-term plan is to define an ultimate objective model. This is a long-term objective, as a lot of parameters have to be taken into account. So, we will progress on two parameters: the impact of background noise and a parameter linked with intelligibility, the listening effort (LE).



The full plan for such a project should be:

a)    Design experiments to measure LE, using methods that include all relevant aspects of accessibility for persons without hearing impairments.

b)    Collect a corpus of results over a wide range of conditions relevant to contemporary telecommunications systems, with initial focus on persons without hearing impairment

c)    Development of an objective model for the prediction of LE for persons without hearing impairment.

d)    Explore relations between listening quality, listening effort and intelligibility for users with normal hearing using the new LE model.  This step may require additional listening tests.

e)    Collect a corpus of results over the same conditions as noted in b), but for persons with hearing impairment.  Due to the complexity of hearing impairments, this may require extensive work,

f)    Use the LE predictive model developed in step c) with the additional data collected in step e) to extend the LE predictive model for use with persons with hearing impairment.



The background noise to be taken into account depends on the interface considered. The present STF focuses on the acoustical interface. An additional work (outside the STF workplan) will consider the electrical interface.



So, the present STF is intended to cover Steps a) and b) at the acoustical interface.



The development of the LE predictive model Step c) will be developed (and trained) outside the STF workplan. However, the STF is intended to use a set of subjective test results to validate the model.

How we do it

The STF will produce:

- the recorded test sequences (mixing background noise and speech sequences)

- the test plans for subjective tests

- the results of the subjective tests (two sets of results). The first set will be provided to the laboratory in charge of the model training. It will be possible to communicate (in the STF deliverable) the recordings and test results for the model training, but the results of the tests for validation will be provided only after validation of the model (the results shall be kept “secret” in the mean time).



The final deliverable will provide all the results obtained by the STF.

Deliverables

The main deliverable will be an annex to TS 103 558 (DTS/STQ-WI 264) for the acoustical interface.



TS 103 558 is intended to develop methods for the objective prediction of listening effort at the near-end side for the following two scenarios:

1) at acoustical interfaces in the presence of background noise (in the STF 575 workplan)

2) at electrical interfaces aiming at the effects of noise that originates from the transmission chain or from the send side (outside the STF 575 workplan)



The deliverable will detail the results of subjective tests for listeners without hearing impairments.



The results of the STF575 will also be inputs to existing STQ documents.

Time plan

 

Milestone

Destination         Description   Cut-off-Date
A

ETSI

•    Design of Test Plans for subjective test

•    Test sequences for subjective testing

•    Progress report to be approved at TC STQ#62

•    Early Draft available

27-Sep-2019

B ETSI •    Conducting and analysing subjective tests completed

•    Progress Report to be approved at TC STQ#63

•    Stable draft available
21-Feb-2020
C ETSI Model for the validation available 30-Apr-2020
D ETSI •    Consolidation of tests results and validation of the model

•    Final Report and final draft to be approved at TC STQ#64
 03-Jul-2020

E ETSI Deliverable published, STF closed 31-Jul-2020



 

How to contact us

If additional information is needed, please mail to jeanyves.monfort.6@orange.fr

STF 575 will report on the progress at all the TC STQ Meetings :

STQ#61    Nuremberg(DE)    8-12 July 2019

STQ#62    Boston (US)        23-27 September 2019

STQ#63    Bern (CH)        17-21 February 2020

STQ#64    Zagreb (CR)        29 June-3July 2020

This information is based upon STF working assumptions.

The views expressed do not necessarily represent the position of ETSI in this context.