Comparing MP3 Software

Two of PCAVTech's technical performance tests most clearly show the most significant performance differences between the Music Match Jukebox release 4.3.0.58 and Real Audio Jukebox release 1.04.36 products used to create and play MP3 files having a bitrate of 128 Kbps. One test is called "Complex Signal Signal-To-Noise Ratio" and the other is called "Frequency Response With The Right Channel 10 dB Down" .

Complex Signal Signal-To-Noise Ratio

"Complex Signal Signal-To-Noise Ratio" (CSSNR) is a test of the ability of Unit Under Test (UUT) to accurately handle a complex signal composed of 36 sine waves with arbitrarily-chosen frequencies, while adding minimal unwanted new signals at other frequencies. The left and right channel signals are recorded with the same signal. This complex test signal simulates multiple musical instruments playing at the same time, positioned at the center of the soundstage.

Figure 1 shows the frequency spectrum of the CSSNR signal under ideal performance conditions, those in which the output of the UUT is identical to its input. The key performance indicators are peaks with a consistent height, and valleys that are as deep as possible. This corresponds to flat frequency response and minimal generation of unwanted new signals at other frequencies. The peaks in the spectrum of the test signal are flat within 0.1 dB or better. The valleys are at least 80 dB below the peaks.

Figure 2 shows the frequency spectrum of the CSSNR signal after processing by the Music Match MP3 coder and decoder. The vertical scale has been expanded during analysis, as compared to that used to analyze the reference signal, for improved clarity. The peaks have reasonably consistent height, showing that frequency response is good. The valleys are about 50 dB below the peaks, showing slight degradation of the signal by the UUT. Response is sharply rolled-off after 15 KHz, which has a minimal audible effect.

Figure 3 shows the frequency spectrum of the CSSNR signal after processing by Real Audio MP3 coder and decoder. The vertical scale i has again been expanded as compared to the reference signal for improved clarity but is the same as that used for the test of the MusicMatch software. The peaks have reasonably consistent height except for a slight dip at 1 KHz. The valleys are as little as 18 dB below the peaks at 1 KHz, showing some degradation. There is a rather strong spurious response at 1200 Hz, about 20 dB below reference level, which can be expected to be audible under some circumstances. There is a mild roll-off above 15 KHz which can be expected to have minimal audible effect.

The Music Match MP3 coder and decoder performs better than the Real Audio coder and decoder in the CSSNR test primarily due to the frequency response and spurious signal situations found in the Real Audio coder and decoder.

Frequency Response with the Right Channel 10 dB down

"Frequency Response with the Right Channel 10 dB down" (FRR-10) is a test of the ability of Unit Under Test (UUT) to accurately handle a complex signal composed of 40 sine waves with arbitrarily-chosen frequencies. This test differs from "Complex Signal Signal-To-Noise Ratio" in that the left and right channel signals are recorded at substantially different levels. The right channel is attenuated by 10 dB as compared to the left channel. The choice of which channel is attenuated is arbitrary because when degradation in this test is observed, it is usually symmetrical between the channels.The degree of attenuation was chosen to be similar to what happens when a musical signal is presented quite a bit off the center of the soundstage, but not totally in one channel.

Figure 4 shows the frequency spectrum of the FRR-10 under ideal performance conditions, those in which the output of the UUT is identical to its input. The key performance indicators are peaks with consistent height for each channel and peaks for the right channel 10 dB below those for the left channel.

Figure 5 shows the frequency spectrum of the FRR-10 signal after processing by the Music Match MP3 coder and decoder. The peaks have reasonably consistent height, suggesting that frequency response is good in both channels when the left and right channels are the same signal, but the right channel is attenuated. The peaks in the right channel are 10 dB below those in the left channel, as desired.

Figure 6 shows the frequency spectrum of the FRR-10 signal after processing by the Real Audio MP3 coder and decoder. The peaks show a deep, broad depression in the 1 KHz range, suggesting that frequency response is degraded in both channels when the left and right channels are the same signal, but the right channel is attenuated by 10 dB. The peaks in the right channel are 10 dB below those in the left channel, as desired.

The Music Match MP3 coder and decoder performs better than the Real Audio coder and decoder in the FRR-10 test primarily because of the broad frequency response dip found in the Real Audio coder.

Conclusion and Additional Comments

In general, the tested version of MusicMatch Jukebox has better technical performance than the tested version of Real Audio Jukebox when handling MP3 files coded at 128 Kbps because it has better frequency response and generates far less spurious signals.

Many other performance tests are presented in detailed reports for the Music Match and Real Audio MP3 encoders and decoders which can be accessed though the hyperlinks (underlined text) in this sentence. Or, you may want to experience the entire PCAVTech web site starting at www.pcavtech.com.

You can perform Double Blind Listening Tests comparing musical signals processed by the Music Match and Real Audio MP3 encoders and decoders by downloading files from the PCABX web site at www.pcabx.com.

The PCABX web site has some special technical tones that can help you train yourself to hear the difference between these coders which are located in the "technical" sample folder including ftp://ftp.pcabx.com/pub/download/SAMPLES/technical/nl-2_mm430058_128.wav (MusicMatch) ftp://ftp.pcabx.com/pub/download/SAMPLES/technical/nl-2_ra10436_128.wav/ (Real Audio) and ftp://ftp.pcabx.com/pub/download/SAMPLES/technical/nl-2.wav/ (reference).

Please remember that if you wish to use your web browser to download sample files from the file hyperlinks above, or the sample library you will probably have to right-click the hyperlink text or file name and select the "Save Target As" (Internet Explorer) or "Save Link As" (Netscape Navigator).

Once you train yourself to hear the differences between the reference sample and the other samples in the PCABX "technical" sample folder, which will be quite easy, you may have an easier time hearing the differences using the musical samples.

Please run through ALL the steps contained in the "Getting Started" page at PCABX before you try to compare sample files for actual products.