Specialist Task Force 294:

Improving the quality of eEurope wideband speech applications by developing a standardised performance testing and evaluation methodology for background noise transmission

NEWS:

ETSI Workshop on Speech and Noise in Wideband Communication

 22nd & 23rd May 2007, at ETSI Headquarters in Sophia Antipolis, France

The workshop was organized by TC STQ, STF 294 and MESAQIN, under contract to ETSI

Find all information concerning this event on the specific webpage: http://portal.etsi.org/stq/workshop2007Home.asp 

Who we are: 

Team Leader Phase 1: Vincent Barriac
Team Leader Phase 2: Hans Wilhelm Gierlich

Team members : Javier Manuel Aguiar Perez
Noël Chateau
Laetitia Gros
Jan Holub
Frank Kettler
Isabel Ordas
Silvia Poschen

What we do :

The STF will provide the following ETSI deliverables (a multipart ETSI Guide):

• Revised DEG/STQ-00038-2 Speech Processing, Transmission and Quality aspects (STQ); Speech Quality performance in the presence of background noise; Part 2: Background noise transmission - Network simulation - Subjective test database and results

• Revised DEG/STQ-00038-3 Speech Processing, Transmission and Quality aspects (STQ); Speech Quality performance in the presence of background noise; Part 3: Background noise transmission – Objective test methods

Read our Terms of Reference

Why we do it:

Background Noise is impairing conversations mostly everywhere – in office and working environments and most often in mobile communications where mobile phones are used in a huge variety of different acoustic environments (see picture).

The aim of the work is to find methods for background noise transmission quality evaluation for real life network scenarios with speech present at the near end and at the far end as well as during periods where no speech is present but only background noise is transmitted.

Basic subjective tests carried out indicate that the background noise performance of terminals (and networks) contributes significantly to the perceived overall quality. The background noise performance can be considered as one of the dominating parameters for speech quality in modern telecommunication scenarios.

In order to evaluate the performance of this equipment objective testing procedures for the background noise transmission in the various situations as well as suitable background noise sound field simulation for laboratory evaluations are required. The evaluation methods needed should take into account the performance of the background noise transmission with near end speech, with far end speech and with no speech present. Furthermore the degradation introduced to the speech transmitted should be measurable. The current ITU-T Recommendations (P.862, P.561) are not applicable for the whole problem as described above.

Generic Workflow of the STF 294 Project

Tasks of Phase 1

Task 1: Transmission network simulation

• Evaluation of simulation methodologies and tools 
• Choice of the best simulation methodologies and tools 
• Setup of simulation environment in the laboratories of the STF experts 
• Validation of the simulation environment

 Task 2: Database for transmission networks and conditions evaluation/simulation

• Definition of the different network scenarios and conditions to be recorded 
• Preparation of the simulation recordings 
• Editing and preparing the data base 
• Description of the data base (Milestone A)

 Figure: Workflow of Task 2, Phase 1

Task 3: Subjective tests

• Preparation of the test material 
• Conducting experts tests 
• Evaluation of the experts test results 
• Preparing test material for naïve listeners tests 
• Conducting naïve listeners tests (2 languages : French and Czech) 
• Evaluation of the results 
• Statistical analysis 
• Description of the test results (Milestone B1)

 Phase 1 – Links to detailed presentations

• Phase I – Goals and Background

• Initial Recordings (based on STF 273)

• Noise Reduction

• IP transmission simulation

• Subjective Test Plan

• Phase I Results

 Tasks of Phase 2:

• Task 1: Expert review of listening test results and data selection

• Task 2: Development of new objective model for estimation of N-MOS, S-MOS and G-MOS, further analyses
  -The N-MOS and S-MOS calculations of the present objective model are based on the Relative Approach. Due to the time variant characteristic of speech and most of the background noise signals, the 3D Relative Approach spectrograph always shows a deviation between the expected and the current signal which is indicated by patterns in the time-variant signal.
  -The G-MOS is then calculated by applying a linear, quadratic regression algorithm to N-MOS and S-MOS.

• Task 3: Result Validation
   -The new objective model is in principle applicable for both databases
  -Different regression coefficient sets are needed in order to reproduce the different level strategies used in the two datasets and listening tests
  -As an example, objective vs. auditory G-MOS for French samples is depicted in the following figure:

Time plan for the work:

Part 1 has been published in December 2005.
Part 2 has been published in September 2006.
Part 3 is expected to be published in September 2007.

How to contact us:

If you would like more information, please contact the STF Leader:
Phase 1: vincent.barriac@rd.francetelecom.com
Phase 2: h.w.gierlich@head-acoustics.de

Note: this information is based upon STF working assumptions.
The views expressed do not necessarily represent the position of ETSI in this context.