Does nobody find it odd that pretty much every single measurement of the IE800 shows a pretty massive 10kHz spike? OK, great - this may or may not be a bad thing depending on taste - but are we really denying it's there? I'd bombard you with proof but I know how sensitive this site is over external links.
On one of these aforementioned external links was a remark that the 10kHz spike "goes away" with very shallow insertion (something about 6cm away from the plane of reference.... whatever that means in human terms), with measurements to back that up. Personally, I still felt it even with the largest tips and as shallow as an insertion as I could get, but perhaps that's just my head geometry.
But this got me to thinking: a stack of measurements show the IE800 with a 10kHz spike. Head-Fi says "nothing to see here". A stack of measurements show the MDR-Z1R with a 10kHz spike. Head-Fi says "nothing to see here".
This kind of reaction brought out a whole lot of "they're rigging the measurements to appease sponsors!!!" but maybe - just maybe - there's some convenient and co-incidental voodoo surrounding 10kHz spike resonances and the geometry of the G.R.A.S. system. Perhaps fitting the G.R.A.S. with earbuds puts them 6mm away from this "plane of reference", naturally (and legitimately) getting rid of this spike. Maybe this geometry applies to all headphones, and cancelled out the Z1R's spike.
This is of course just fun science fiction... but since Schiit's Gadget, why not?
Regarding headphone measurements you may have seen on the web, and the fact that many will show measurement characteristics in common: I think some of the measurement commonalities you'll see between different enthusiast measurements on the web is due to commonly shared opinions and recommendations on how to build and tune do-it-yourself measurement rigs. The gear we use to measure headphones here at our office, and the methods we use, are probably quite different than what was used to generate most of the measurements you're seeing on the web from do-it-yourself measurement rigs.
A couple of years ago, I made a post somewhat related to this (the discussion of different headphone measurement setups, and different results), which you can read at the following link:
Headphone Measurements: Different Setups, Different Results
If you search the forums and web you can find some information about, and photos of, the DIY measurement rigs from which many (perhaps even most) of the headphone measurements you've seen online have come from. Some of these can be seen below, with my post continuing after the photos:
As you can see, there are many things in common that these rigs have, probably guided by sharing of instructions and suggestions about how to make one's own rig. For example, to explain why almost all of them have a similarly sized piece of felt or foam around the microphone, we can probably look to this description by Marvey (aka purrin) about this:
Marvey (aka purrin) said:
...In the middle of the foam layer, a piece of felt about the size of an ear. Without this piece, I found measurements too ringy. Covered completely with felt, I found measurements too damped. The felt the size of an ear also serves another function. Proper measurement of supra-aurals that rest on the ear...
This would suggest that the intent of this piece of felt is to serve as a sort of pinna. I don't think a flat piece of felt or foam cut to about the two-dimensional area size of an ear could reasonably serve this purpose.
Microphone placement for most of these rigs tends to be flush with a flat plate. None of the rigs shown above are using ear simulators, nor are any using pinnae or canal assemblies. Given the length of the microphone that most are using, some of the rigs are quite wide, and that may also have an effect on headband tension and tightness of fit. For most, there is no support underneath the top of the headband. There are other things in common, too, obvious from the photos above.
Again, I think some of the measurement commonalities you'll see between different enthusiast measurements on the web is due to commonly shared opinions and recommendations on how to build and tune do-it-yourself measurement rigs.
Since we're talking about differences in measurements from others you've seen on the web, it's probably worthwhile to go over the current measurement systems we're using at Head-Fi's office, as they're quite different than the ones shown above. We've been putting together our measurement systems and techniques for nearly three years now, with a lot of help, knowledge, and feedback from industry mentors that include acoustical engineers and others who make their livings in/around audio measurements. There's always going to be much more learning ahead, no matter how much we do, how much we read, how much we're taught.
Anyway, here's what we're currently using at the office for audio measurements:
GRAS KEMAR, 45CA, Ear Simulators, and Pinnae / Canals
For headphone measurements, we're working with a head or fixture with more or less fixed dimensions (defined by international standards) representing average human dimensions, and those are the GRAS 45CA and the GRAS 45BB-12 KEMAR manikin.
In the photo (above left), KEMAR is face-forward in our measurement lab, inside a custom Herzan acoustic/vibration isolation enclosure (more information below).
The pinnae / ear canals we use on the GRAS 45BB-12 KEMAR are anthropometric, based on 260 three-dimensional scans of human ear canals. These pinnae include the first bend and the second bend of the canals, with "flesh" all the way to the mics. Because they're more anatomically representative than traditional measurement pinnae, they have (among other features) a more realistic, more oval-shaped entrance point. Here's a photo of our current GRAS 45CA ears (which currently use standard measurement pinnae):
Here are a photo showing the new anthropometric pinna / canal:
Another advantage of the new pinnae is their increased realism
externally. If you've ever felt measurement pinnae, they're typically stiffer than human pinnae and don't readily compress against the head, which is why many who measure headphones often have difficulty measuring supra-aural (on-the-ear) headphones with them. (They can also present problems with measuring shallow-cup circumaurals.) The new pinnae feel and move much more like human pinnae and compress against the head much more like (most) people's pinnae do. Here is a photo I took of the supra-aural Audeze Sine on the standard measurement pinnae on a GRAS 45CA:
Here's the same headphone on the new anthropometric pinnae on the GRAS 45BB-12 KEMAR:
Where in-ears are concerned, we've found the new pinnae/canals to help tremendously with more realistic and consistent placement, as the pinnae/canals are definitely more human-like now. With this improved realism we've found, for example, that characterizing the differences between eartip types via measurements (relative to our subjective experiences) is improved.
Additionally, I think the dimensional characteristics of the rig we're currently using (a GRAS KEMAR) might also contribute to some of the differences (especially versus the type of DIY rigs shown in the photos above). KEMAR has head shape characteristics that lead to more dimensional limitations on placement -- more human-like limitations, in my opinion. Whereas on a flat plate coupler you can place the headphone in any number of places and still maintain a seal (and thus maintain bass, the loss of which is one of the primary indicators that fit has gone wrong). On humans -- and on KEMAR -- if you go too far back, the curvature of our head can break the seal. Too far down on a human (and KEMAR) and you can also lose seal. In other words, the dimensional limitations of our anatomy -- and the larger the headphone, the more this may come into play -- play a role in guiding and limiting the placement range of the headphone over our ears in actual use. In the frequency ranges we're talking about (as the wavelengths get shorter), minor shifts in placement and dimensions can have substantial effects.
Unlike the DIY rigs shown in the photos that began this post, the measurement manikin (GRAS 45BB-12) and fixture (GRAS 45CA) use ear simulators to simulate the input and transfer impedance of a human ear. The GRAS 43BB ear simulators in this specific KEMAR configuration are quite different than standard 60318-4 simulators. While they still meet the IEC 60318-4 tolerances, the single high-Q resonance above 10 kHz is replaced by two more balanced, more damped resonances. The splitting of the one high-Q resonance into two low-Q resonances may present an advantage in decreasing the uncertainty in the measurements around the resonance (above 10 kHz). Also, the GRAS 43BB is highly sensitive, and
very low-noise, and extends the lower dynamic range
below the threshold of human hearing. Given its extremely low-noise nature, the 43BB can be used to measure and characterize things like the self-noise of an active headphone (both with and without active noise canceling), which is something we'll be increasingly interested in with the growing prominence of high-fidelity wireless headphones and earphones. It can also help in measuring low-level distortion in headphones and earphones. NOTE: One thing to consider with this low-noise simulator is that it's not suited to very-high-SPL measurements, with an upper limit of the dynamic range to about 110 dBSPL. This hasn't been an issue for us, though, as most of our measurements are set at 90 dBSPL (at 1 kHz).
Here's a whitepaper about the GRAS 43BB Low Noise Ear Simulator
A few weeks ago, GRAS announced still another ear simulator designed specifically for measuring high-resolution headphones. On Friday (two days ago) we took delivery of the new GRAS RA0401 High Resolution Ear Simulators, and we'll be installing them on our GRAS 45CA, along with the new anthropometric pinnae for the GRAS 45CA.
This is still another very exciting development for headphone measurements, as obtaining meaningful measurements above 8 kHz with most systems can be enormously challenging. This new GRAS RA0401 High Resolution Ear Simulator also meets the IEC 60318-4 tolerances, but GRAS was able to design it so that its performance from 10 kHz to 20 kHz is substantially improved, that range through which it has a tolerance of +/- 2.2 dB. Here is a graph showing the RA0401's response (including the IEC 60318-4 tolerances) compared to a standard 60318-4 ear simulator:
Again, meaningful headphone measurements above 8 kHz or 10 kHz have been a major pain point for decades, so I think these new GRAS High Resolution Ear Simulators may prove an important development in the world of headphone testing.
Here's a whitepaper about the GRAS High Resolution Ear Simulator
Unfortunately, we haven't had a chance to run our own measurements with the new RA0401 High Resolution Ear Simulators yet, as we shipped our measurement mic preamp power supply back to GRAS for a check-up and any necessary calibration (as it's now nearly three years old, and has been jostled around quite a bit). We should have that (GRAS 12AQ) back in the next few days, and we'll fire up these newest ear simulators just as soon as we do.
To help improve the quality of the measurements, we wanted to maximize environmental isolation. Though we obviously do not have the space (not to mention the budget) to build a full walk-in anechoic chamber, we still wanted to achieve as much acoustic and vibration isolation as reasonably possible. Skylar Gray (formerly of AudioQuest, now with Definitive Technology) recommended we contact Herzan. We worked with Herzan to carefully spec out a custom-built acoustic and vibration isolation enclosure. Our
Herzan enclosure has thick walls, made with 11 variable density layers of sound-damping material, and the interior of the enclosure is lined with acoustic sound absorption foam. This enclosure is (by design) fairly massive, weighing around 1200 pounds -- the more mass there is, the more energy it takes to excite the system. The enclosure has two cable ports, both covered with solid machined metal screw-down blocks that are damped, and also have soft gaskets that allow full sealing around the cables. (See photo below.)
The headphone measurement manikin or fixture being used at the time is placed on a Herzan Onyx-6M vibration isolation table, to further help isolate the system from vibrations caused by foot traffic, HVAC systems, vehicle traffic, etc. The Onyx-6M is essentially a steel tabletop supported by pneumatic isolators, and provides isolation beginning at 4.5 Hz.
(Above left) Closed left-side cable port on the Herzan acoustic/vibration isolation enclosure.
Even with the Herzan enclosure, we still make sure to keep it as quiet as we can in the office while doing measurements. Both of the office HVAC systems are switched off when we measure. You'll frequently hear the tongue-in-cheek cry,
"Measurement in 3...2...1...fire in the hole!" before we start a measurement, and everyone remains quiet until an all-clear is given. We're particularly careful about this when using the GRAS 43BB ear simulators because, again, their dynamic range extends below the threshold of human hearing -- so even the very faintest sounds you can hear can be heard by the 43BB's.
The Audio Analyzer: Audio Precision APx555 and APx1701
At the center of our audio measurements -- whether we're doing electronic measurements (another topic for another time) or headphone measurements -- is an audio analyzer. The audio analyzer generates a stimulus of known characteristics, and then analyzes the response. (Wikipedia has an entry for "audio analyzer" which you can see at the following link for more general information about them:
audio analyzer.)
We use the
Audio Precision APx555 audio analyzer. In terms of its analog performance, the APx555 has a typical residual THD+N of -120 dB and over 1 MHz bandwidth, which exceeds the analog performance of all other audio analyzers. It will also do FFTs of 1.2 million points and full 24-bit resolution. The Audio Precision APx555 is an incredible tool, whether we're measuring DACs and amps (again, another topic, another time) or doing electro-acoustic tests, which is obviously more relevant to this thread's topic.
For some reading about the importance of low test system noise floor, Dan Foley from Audio Precision (one of my audio measurement mentors for over two years) wrote an article for
audioXpress titled "Test and Measurement: 'I Can Hear It. Why Can't I Measure It?'" In this article, Dan uses headphones as a key example, as he discusses the human hearing threshold and how it relates to the noise floor of a sound card interface compared to an audio analyzer's noise floor. You can read this article at the following link:
"Test and Measurement: 'I Can Hear It. Why Can't I Measure It?'"
We've also added all of
Audio Precision's Electro-Acoustic Test Options to the APx555 through their Electro-Acoustic Research & Development option (APX-SW-SPK-RD), which is a suite of measurements intended for designers and engineers who develop electro-acoustic audio products. You'll be seeing some measurements specific to this module from us in the near future, along with explanations of them.
Earlier I mentioned testing wireless headphones. Another update we're making to our measurement lab very soon is the addition of
Audio Precision's new Bluetooth Duo Module to the Audio Precision APx555 audio analyzer. This new Bluetooth module can act as source and sink for AAC, aptX, aptX-HD, aptX-LL, and SBC. Yes, I said aptX-HD, and yes I'm
very excited about that. Last year we had two aptX-HD-enabled wireless headphones in our office. Now we have many more, and it will be exciting to be able to measure these headphones at their wireless best. We'll be covering this module more over time.
In the photo above, the Audio Precision APx555 Audio Analyzer (bottom) and Audio Precision APx1701 Transducer Test Interface.
For amplification (to drive the earphones and headphones), we use the
Audio Precision APx1701 Transducer Test Interface. The APx1701 integrates instrument-grade amplifiers and microphone power supplies, but we do not currently use its mic power supplies (we use GRAS's power modules). The APx1701 is made to drive both loudspeakers and headphones, includes current-sense resistors in the amplifier outputs for impedance measurements, has a signal-to-noise ratio of 134 dB, THD+N is ≤ -105 dB, output impedance is 0.13Ω, and it can drive up to 100W per channel into 8Ω. Given the APx1701's fixed gain of 20 dB, we will sometimes use the
Rupert Neve Designs RNHP headphone amplifier (set to unity gain) to drive more sensitive headphones and earphones.
Keeping all of the measurement equipment calibrated and maintained is obviously important. We do the calibrations we can do at the office. We use GRAS pistonphones to regularly calibrate the measurement microphones and systems. Ideally, we would do this before every measurement session -- we do it no more than once per month, corrected for temperature and ambient static pressure at the time of the calibration. We run Audio Precision's APx Self Test utility on the APx555 from time to time to confirm proper operation of its analog and digital sections. (We have not yet run that utility on the APx1701, which is still quite new.) We do occasionally measure the Neve RNHP on the APx555 just to confirm the consistency of its performance and to confirm unity gain. We intend to adhere to the manufacturers' maintenance and calibration schedules, and will send the gear to them for check up and calibration accordingly.
We're in the process of shooting
Head-Fi TV videos now to discuss the components of the measurement systems in Head-Fi's audio measurement lab, including some examples of them in actual use.
Getting back to some of the measurements you're discussing:
Does nobody find it odd that pretty much every single measurement of the IE800 shows a pretty massive 10kHz spike? OK, great - this may or may not be a bad thing depending on taste - but are we really denying it's there? I'd bombard you with proof but I know how sensitive this site is over external links.
On one of these aforementioned external links was a remark that the 10kHz spike "goes away" with very shallow insertion (something about 6cm away from the plane of reference.... whatever that means in human terms), with measurements to back that up. Personally, I still felt it even with the largest tips and as shallow as an insertion as I could get, but perhaps that's just my head geometry.
But this got me to thinking: a stack of measurements show the IE800 with a 10kHz spike. Head-Fi says "nothing to see here". A stack of measurements show the MDR-Z1R with a 10kHz spike. Head-Fi says "nothing to see here".
This kind of reaction brought out a whole lot of "they're rigging the measurements to appease sponsors!!!" but maybe - just maybe - there's some convenient and co-incidental voodoo surrounding 10kHz spike resonances and the geometry of the G.R.A.S. system. Perhaps fitting the G.R.A.S. with earbuds puts them 6mm away from this "plane of reference", naturally (and legitimately) getting rid of this spike. Maybe this geometry applies to all headphones, and cancelled out the Z1R's spike.
This is of course just fun science fiction... but since Schiit's Gadget, why not?
Looking back at some of the IE800S measurements that were shared by others in this thread:
The first two are perhaps characterized more by sizable scoops out of the frequency response moving from 1 kHz to around 6 kHz, with the first one being more severe. Coming out of this, the rise to 10 kHz looks more pronounced. We've gone over this already, but let's do it again. If you listen to a frequency sweep through that range, are you hearing a scoop of that magnitude? Again, some may, I think most will not. On the third graph, the rise to 10 kHz is about 12 or 13 dBFS above the average level leading to it. Again, sweep through this range. Do you hear it? Some may, I think most will not. (I do not.)
As for the Sony MDR-Z1R, here are some recent quotes from Tyll related to that discussion (comments that may also apply to the above):
Tyll Hertsens at InnerFidelity said:
You may remember the Sony MDR-Z1R measurement discrepancy between Jude's rig and mine. You'll recall the biggest discrepancy was at around 10kHz. Jude uses the G.R.A.S. KEMAR head and torso simulator with anthropometric pinnae that includes anatomically correct ear canals; the ear canals in my head are perfectly cylindrical. I've been thinking about it quite a bit, and I've got to think the anatomically correct ear canals have lower-Q (more damped) resonances than mine. We may not be used to seeing the curves as presented on Jude's rig, but they may actually be more representative of what we hear.
In a later post there he said:
Tyll Hertsens at InnerFidelity said:
Many readers will be familiar with the controversy regarding the Sony MDR-Z1R and the disparity between my measurements and Jude's, much of it surrounding the significant difference of the measured peak at around 10kHz. I think much of the curfuffle can be written off with the understanding that measurements from differing systems really can't be compared. I continue to believe these headphones have too much energy in that area, but if you look at all the InnerFidelity measurements you can see a clear tendency for my head to show a peak in that area...and I think that's a measurement accuracy error.
Is that the answer? I think it may be part of it, some of the details of which were discussed in the first half of this long post.
Simply put, I think it's primarily the differences in measurement systems that generate the different results (and the common results among like DIY systems), some of the details of which we've gone over in this post.
Again, a couple of years ago, I made a post somewhat related to this (the discussion of different headphone measurement setups, and different results), which you can read
at this link.