You are here: Reference Guide > Audio Rating Metric

Audio Rating Metric™

Frontline's Audio Rating Metric™ (ARM) is a process for rating audio impairments or differences resulting from problems in Bluetooth audio channels. In the Audio Expert System™ ARM operates only in the reference mode because it produces an objective score based on a set of deterministic measurements of captured reference audio as compared to the original reference audio.

ARM is a real-time process. Individual ARM scores are calculated for each sub-segment of the reference audio. Therefore, the scores are displayed as an overlay on the captured audio waveform providing visual traces of the audio channel quality. For most sub-segments, scores are computed when a change of frequency is detected, and for silent segments, for multi frequency segments, or for level changes. Supported test files can change frequency as often as every 50 milliseconds which equates to ARM's smallest real-time response. However, typical segments resulting from frequency change would occur every 100 milliseconds, as well as a few longer segments.

There are six components to the ARM segment score computations. These components are listed and described in See Audio Rating Metric Score Components. Each metric score is based on the percent change formula

Percent change formula

where A and B are the two component values being compared. This computation always results in values between 0.0 and 1.0.

Each component score is multiplied by an associated weighting factor. The exact weight value for each component depends on its relevance with respect to audible degradation, and the sum of the individual weight values must total exactly 1.0. Having prescribed a set of component weights, all audio measured is subjected to the same judgments, and the score outcome of a given audio sample is repeatable.

Audio Rating Metric Score Components
Score Component Comments
Frequency Resets

This component contributes sensitivity to lost continuity, unexpected changing frequency, or other forms of (abnormal) frequency deviation.

As each audio waveform peak occurring above Measurement Threshold is detected, the time between it and the previous peak is used to calculate the instantaneous peak-to-peak (p-p) frequency associated with the interval. These instantaneous frequencies and peak amplitudes are also used to generate running averages of frequency and amplitude. Whenever the instantaneous frequency deviates from the average by more than a configured percentage, a “Reset” is declared, and re-acquisition of average frequency begins. This event occurs naturally in reference audio, which includes numerous transitions between low and high frequencies; but normally a reset occurs only once per scripted transition. In the face of impairments such as lost data, events such as “Phase Change” and/or “Unexpected Duration” may be reported.

Frequency Measurement Restarts

This component is an indicator of significant frequency instability.

Following a Frequency Reset, the Audio Expert System™ begins tracking just the instantaneous p-p frequency, and comparing each measurement to the previous measurement. In all scripted reference audio, frequency transitions are abrupt and occur at 0 degrees. That is, the frequency changes instantaneously. However, various encoding/decoding operations, and in particular combinations of concatenated encoding/decoding operations, can result in a variety of unanticipated interactions. They can result in “cross fading” from one frequency to the next in some cases (a “fade out” of one frequency, and a “fade in” to the next). Also, some codecs employed in mobile networks do not convey instantaneous transitions from one frequency to another. The first cycles of a new frequency may be higher or lower than the transmitted frequency, “settling” to the transmitted frequency after some number of cycles, essentially exhibiting some degree of frequency deviation over the duration of the segment.

These cases result in p-p frequency measurements with continuously changing outcomes as a result of the “beating” of multiple frequencies, or frequency deviations.

Whenever a p-p change occurs during the reset state which is greater than a configured percentage, a “restart” of the acquisition process occurs, and this score component tracks the percent difference between the number restarts anticipated per segment, and the actual number encountered.

Frequency Difference

This score component contributes the percent difference between measured and scripted frequencies.

When each frequency acquisition process concludes, reaching a steady state average frequency, an internal “Frequency Change” event is declared. When this occurs script tracking advances to the next segment of the script, and a comparison of the measured to scripted frequency is performed.

Level Difference When the Audio Expert System™ detects a significant level change, it compares the new level to the scripted level, and this score component contributes the percent difference between the two.
Amplitude Modulation This score component contributes the maximum Amplitude Modulation measured over each segment. In effect, it provides a measure of amplitude stability.
Duration Difference When each new segment is encountered the duration of the previous segment is measured and compared to the scripted duration. This score component contributes the percent difference.

The maximum ARM score is 5.0 and descends to a minimum of 1.0, in a manner similar to the scoring common to most “Mean Opinion Scores” (MOS). “Errors” found by each of the detection mechanisms effectively “deduct” from the maximum. The implications of the score values with respect to hearer's perceived audio quality are indicated in the following See Audio Rating Metric Scale Interpretation.

Audio Rating Metric Scale Interpretation
Major Scale Point Quality Hearer's Perception of Difference
5 Excellent Imperceptible
4 Good Perceptible but not annoying
3 Fair Slightly annoying
2 Poor Annoying
1 Bad Very annoying

The ARM score computation implements a simple linear scaling formula.

Audio Rating Metric score computation

Where,

ARMfactor = a scaling factor to achieve an ARM range of 1.0 to 5.0.

ARMBaseScore = 5.0.

The percent change formula naturally produces an exponential-like curve for each of the component scores. The ARMfactor inverts it to look appear as in See Audio Rating Metric Score Chart.

Audio Rating Metric Score Chart

The resulting general behavior is that score reduction tends to follow a somewhat logarithmic behavior. Exactly what it takes for any particular component to reduce score from 4, to 3, to 2, etc., depends on the value being measured. But as one example, the number of Frequency Resets that should occur for perfect audio is exactly 1 per frequency change. But, if there are phases changes or other degradations Frequency Resets may increase. Imagining that frequency resets are the only component in the score, the resulting scores for increasing resets would be as shown in See Audio Rating Metric Example: Frequency Reset Impact.

Audio Rating Metric Example: Frequency Reset Impact
Resets 1 2 3 4 5 6 7 8 9 10
ARM Score 5.0 3.7 3.0 2.6 2.3 2.1 2.0 1.9 1.8 1.7

The effective behavior is that the score is fairly sensitive to any initial deduction (impairment), but becomes less sensitive as deductions (impairments) continue to occur. For example, as indicated above, 2 additional unscripted frequency resets (for a total of 3) drops the score by 2 full points to 3.0. But 3 times this, or 6 resets for a total of 7, are required to drop the score by another full point to 2.0.