Comparing methods of testing – objective metrics

There is a variety of approaches to carrying out sound quality testing. These can be divided into two main categories: Subjective testing and use of objective metrics.

Objective Metrics

There are a wide range of signal processing tools available on the market today advertising the ability to quantify sound quality objectively. Hardware and software choices range from standalone systems on special hardware (e.g. DSP boards) with no provision for integrating your own software/hardware to completely open systems in a standard programming language on standard hardware (PC or UNIX workstations). Closed systems have the advantage that they are easy to use requiring less knowledge of the operating the system, and open systems have the advantage of flexibility although more knowledge about the system is required for operation [1].

Companies who sell sound quality hardware and software include: 01dB (dBFA32), B&K (PULSE analyzer), and HEAD acoustics (Artemis and SQ labII). Sound quality systems such as these are widely used in the automobile industry and sophisticated techniques involving computer modelling of vibrations and airborne waves are also applied. These techniques have also been (tentatively) applied to domestic appliances [2, 3, 4]. Many of these packages also include signal manipulation and generation tools, and similar audio editing tools can also be found in software such as Adobe Audition.

Fig 1. Sound Quality Software

Disadvantages of these systems, include the fact that the cost of these systems may be prohibitive for SMEs. Also, the majority of sound quality metrics included in these tools are yet to be standardized and are in a continuing state of being updated so features of individual systems might change quickly as companies keep up, and are not necessarily comparable from system to system. Indeed, many of the sound quality metrics are not even properly documented.

Head and Torso Simulators (HATS) may be used for recording to give a more realistic spatial representation of the sound. However the metrics themselves are often based on monaural calculations so a correction relating to the head related transfer function (HRTF) of the dummy head must be incorporated into the calculation. The HATS themselves suffer from the drawback of inter-individuality: Because head and torso shapes vary from person to person the “binaural receiving characteristics differ to some extent between people.” [1] (A more time consuming alternative might be in ear microphones for recording to find the individual HRTF for each person taking part in the test). For this reason different companies may use different measurements to produce a ‘standard’ dummy head [5]. There are also different degrees of accuracy in representations; some HATS also model the ear canal for example. Despite these drawbacks, the spatialisation of sounds is reckoned to be better than stereo or mono presentation of sounds by a loudspeaker.

In theory a mathematical/scientific definition of metrics means that if sound quality can be related to a particular metric, communication of the attribute which influences sound quality becomes more definite. So metrics can improve communication of sound quality. This is a particular advantage when communicating across disciplines, something which effective sound quality development usually demands. Engineers often demonstrate an intuitive preference to a metrics which will calculate a numerical value. However there is a danger that the restrictions of metrics will be ignored. Preferences of instrumentation outputs over jury analysis may encroach upon the test validity, after all, the ability of a metric to predict sound quality is only as good as the jury test which defined the link between the sound quality and the given metric.

To conclude there are a wide variety of options available for instrument application to sound quality analysis and a careful selection of these must be made to ‘get it right’ and come up with a meaningful answer.

References

[1] ‘Instrumentation for Sound Quality Evaluation’, Acustica, Vol 83, P775-783, (1997)

[2] ‘Design of Combination Metrics for Two Household Appliances’, I Sobhi & P Ladegaard, Sixth ICSV (1999)

[3] ‘A comparative Study on the Sound Quality of Wet-and-Dry Type Vacuum Cleaners’, E. Altinsoy et al, Sixth ICSV, 1999

[4] ‘Practical application of sound quality technique in pump development’ K. B.Ginn, H. Haslev, J.B. Nielsen, Internoise (2004)

[5] ‘Towards a new Dummy Head?’ H. Fastl, Internoise (2004)