Saturday, 17 June 2017

MEASUREMENTS: Audioquest Dragonfly Black 1.5 - PART 1 (General)

Four USB mini-DAC's (back to front): Light Harmonic Geek Out V2, Audioquest Dragonfly Black, SMSL iDEA, AudioEngine D3.
As I showed a couple weeks ago, I have in my possession one of the Audioquest Dragonfly Black DACs for testing. I'm actually borrowing it from a friend so it's not going to be in my possession for long... Long enough for me to listen and run some objective testing to see what the "deal" is.

I think it's useful to show these test results on the Dragonfly because AudioQuest clearly has a healthy advertising budget and promotes it quite heavily on audiophile sites with ads in magazines as well; as such it's a bit of a "standard" even though a number of other alternatives exist. For example, a few weeks back, I showed the measurements for the SMSL iDEA which I thought performed objectively amazingly well for such a small device although I had some issues with seamless connectivity to Linux / Android. As you can see in the image above, I have tested a few others of these kinds of  DACs already including the AudioEngine D3 measured in 2014, Light Harmonic Geek Out V2 in 2015, and also the previous revision Dragonfly v1.2 in 2014 which I don't have on hand any more (it was given as a gift to a friend).

I. Intro

Like the Dragonfly v1.2 before, the new Dragonfly Black 1.5 (and I would assume the Dragonfly Red) feels like a well made product. There's a rubberized-metal solid feel to the device. There's some heft to it, unlike the more plasticky feel of the Geek Out V2's 3D printed case. The AudioEngine feels solid as well with its aluminum construction. The SMSL iDEA is comparatively lighter weight which is consistent with it's tiny frame.

No buttons or any hardware control on the device. There is of course the large dragonfly logo on top which changes color to show the sample rate: red = standby, green = 44kHz, blue = 48kHz, amber = 88kHz, magenta = 96kHz, and purple (darker magenta) = MQA rendering as of firmware 1.06.

Here's a handy comparison table that shows the difference between the Dragonfly models. As you can see, the new Black and Red models use a Microchip PIC32MX microcontroller. This is a low power device paired with the ES9010(K2M?) DAC in the Black, and the ES9016 DAC in the Red; both of which are of course part of the ESS Technology Sabre family with the ES9016 having better dynamic range and distortion measurements of the two. As a headphone amp, the Black is capable of lower output defined as 1.2V direct-coupled, while the Red can provide up to 2.1V. I did not see any specifications around total power for the headphone amp.

It has been said that in order to maintain simplicity, the Dragonfly devices are asynchronous USB Audio Class 1 interfaces only, capable of a maximum 24/96 PCM audio. I did not have any compatibility issues at all running this device on Windows 10, Mac OS X, Android or Linux. Totally plug-and-play.

The measurements I'm going to do is essentially the same as what I did for the SMSL iDEA DAC a few weeks back. Here's the set-up:
Device (eg. Surface Pro 3 laptop) --> Audioquest Dragonfly Black --> generic 6' phono-to-RCA cable --> Focusrite Forte --> 6' USB cable --> Win 10 measurement computer
Unless stated otherwise, the Dragonfly was connected to my Microsoft Surface Pro 3 (i5) Windows 10 Creators Update. Native WASAPI driver used.

I started measurements with firmware 1.03 and later on upgraded to the 1.06 version with MQA "rendering" capability. For today's "general" tests, there were no differences between the two versions. I will talk about MQA next time.

II. Digital Oscilloscope, Digital Filter, Impulse Response

Let's start with the microscopic evaluations... Here's the 1kHz 0dBFS 16/44 square wave as per the digital oscilloscope:


That looks good - Vmax of 1.85V or Vrms of 1.3V for this sample. Channel balance is excellent and much better than the Dragonfly v1.2. By the appearance of the post-ringing, the digital antialiasing filter is of the minimum phase variety.


Standard looking minimum phase impulse morphology. Absolute polarity maintained.

Based on the above, we would expect a reasonably "sharp" filter around Nyquist with likely good antialiasing ability...


There it is, the "Digital Filter Composite" based on the "Reis Test". Some 0dBFS overload as commonly seen in most DACs but I see the -4dBFS wideband noise curve isn't quite perfect either. In any event, good suppression of higher order harmonics; about -60dB between 20kHz to 40kHz.

Using a 20-ohm resistor at the output of the headphone jack, using a 1kHz sine wave, the calculated output impedance was 0.61-ohms (the Stereophile review says "<1 ohm"). The DAC is advertised as direct-coupled and this is consistent with my findings.

AudioQuest touts the USB power utilization as being very low with the new Dragonflies. Indeed, the Black sips very little power from the USB bus - only about 30mA @ 5V idle and up to 90mA with a 16-ohm JVC HA-FXC51B IEM at full volume while playing music! Remember in comparison the SMSL iDEA from a few weeks back measures at 110mA to 210mA, and the AudioEngine D3 160mA to 280mA. Significantly more with the Light Harmonic Geek Out V2.

III. RightMark Tests

16/44.1:
Starting at "standard" CD resolution, here's the summary chart...

We see in the lineup our 4 mini USB DACs - left to right are the Dragonfly Black, SMSL iDEA, Geek Out V2 (100mW into 16-ohm mode), Geek Out V2 (1000mW into 16-ohms), AudioEngine D3, and finally the TEAC UD-501 desktop DAC for comparison. If you're keeping track, the AudioEngine D3 is based on the AKM AK4396 DAC, and TEAC UD-501 uses dual-mono TI/BB PCM1795 while the others all use some variety of ESS Sabre DACs.

Generally, I would say that 16/44 is no big deal for modern hi-res DACs. But there's something fishy about the Dragonfly's distortion measurements. THD and IMD looks higher than the other devices. And crosstalk is found to be significantly lower as well compared to the others despite using the same set-up and cables.


This Dragonfly Black certainly "stands out" compared to the others in the stereo crosstalk and IMD+N graphs particularly! Interestingly, the old Dragonfly v1.2 also showed these similar irregularities back in 2014.

24/96:
Moving along then, let's have a look at high resolution performance.

Hmmmmmmm (extra long)... Clearly, the Dragonfly Black is not objectively impressive. Again, we're seeing relatively weak performance compared to the others in the field. Noise level is about on par to the Geek Out V2 in low-power output mode (100mW into 16-ohms). Like with the 16/44 results, we're seeing an unusually high amount of stereo crosstalk and comparatively poor distortion results. These anomalies can be easily appreciated in the graphs:


The Dragonfly's resolution is around 16.5-bits, slightly better than CD resolution. Similarly, Stereophile's measurements described the Black as "the less expensive Dragonfly offers 17 bits' worth of resolution" which I believe is generous as a resolution estimate.

IV. Jitter


The Dunn J-Test looks good with no unusual sidebands. This is as expected from a modern asynchronous USB device. Notice that because of the Dragonfly's relatively low resolution just clearing the 16-bit noise level, we can't see the 16-bit modulation signal poking through the noise floor as well. Look at the recent SMSL iDEA's 16-bit J-Test for comparison.

V. Subjective Evaluation

I have been listening to this device off and on over the last 2 weeks since I received it on loan. To make sure that I wasn't influenced by the testing, I actually did not construct the graphs or look deeply at the numbers coming out of the testing until I started to write this article with comparisons.

From the start, I thought the device sounded good. I was pleased that it didn't give me any trouble with my Android phone or when plugged into a Linux machine. As expected, Windows and Mac compatibility was trouble free. It played back without trouble on an old 2010 iPad 1 using the Apple Camera Connect Kit :-).

I have a few "standard" tracks that I know well. For example, Akon's album Konvicted (2006, explicit content warning) has prodigious bass that blasted through my Sennheiser HD800 well using the DF Black. The HD800's are high-impedance (300-ohm) headphones and the lower voltage from the DF Black didn't allow them to reach the same volume as the comparably priced SMSL iDEA.

A well recorded jazz standards album like Ben Webster's Gentle Ben (1973, 2012 Analogue Productions DSD64 converted to 16/48, DR16) sounded great. Very good detail, nice clarity, natural tonality. Webster's sax was reproduced with fantastic nuance. Compared to the old Dragonfly 1.2, the better channel balance was noticeably improved (this was a concern I had with the old Dragonfly easily demonstrated in the measurements).

Often, good quality gear shows us the imperfections in recordings. Recently, my kids wanted to have a listen to the recent Beauty and the Beast soundtrack which sounded fine with the little CD player (most of the music is of course a re-recording of the 1991 animated classic with a few new songs). However, with the CD's ripped, having a listen on the computer with the Dragonfly, one could easily hear the Auto-Tune "fakeness" in Emma Watson's vocals... Just have a listen to track 5 "Belle" and notice the weird tonal shifts and "plastic" sheen of the vocals from our heroine even in the first few lines! After all, the character's vocal ability should have reflected the natural beauty of "Belle". Instead, the Auto-Tuning made the voice sound like a modern "Top 40" pop diva - more of Katy Perry or Lady Gaga and less of the sweet inventor's daughter from the small provincial town :-(. The honesty of accurate reproduction should make these blemishes evident whether we like it or not.

By the way, there is that term "uncanny valley" to describe the eeriness of CGI at times, especially of facial movements, that don't quite capture reality. I got that sense watching Star Wars: Rogue One with the computer reconstruction of Peter Cushing as Grand Moff Takin or the young Princess Leia. Likewise, the human voice is said to be the most difficult to reproduce perfectly. These days we're used to Auto-Tune pop, but when it shows up in what should sound like the natural vocals of musical theatre, I find that it comes across disturbingly fake.

VI. Conclusions

Compared to the previous generation Dragonfly V1.2, the Dragonfly Black 1.5 is an improvement. Channel balance is clearly better and the current draw of <100mW @ 5V even with music at full volume into ~16-ohm IEMs is very impressive and means that this device can be used with many portable devices including phones and tablets. I was for example able to get it working with my old Google Nexus 7 tablet which didn't have enough juice to stably power the SMSL iDEA. This is also great for battery life.

Where the Dragonfly Black stumbles is in the objective measurements of sound quality. Noise level could clearly be better and distortion results seem strangely high compared to the others (similar to what I saw with the Dragonfly 1.2 in 2014). While it can handle >16-bits resolution, it's at best a sub-17-bit device which is surpassed by the SMSL iDEA, AudioEngine D3, and Geek Out V2 (in higher power mode). I suppose this is why AudioQuest offers the more expensive Dragonfly Red which provides higher resolution and more headphone amplification presumably at the expense of power draw from the USB port.

From a consumer perspective, I think the DF Black targets directly at the owners of laptops with poor built-in DACs and people like iPhone 7 users who want analogue output (so long as they don't mind adding a Lightning to USB cable as well). Remember that relatively recent cell phones like even the iPhone 6 and Samsung Note 5 already have quite good quality outputs already although this DAC can provide more power and has lower output impedance for better sounding low-impedance headphones like IEM's though not necessarily better resolution.

I can certainly understand AudioQuest's desire to keep things simple and compatible by sticking with USB Audio Class 1, thus the maximum of 24/96 input. Compared to most DACs today, that also means not having access to the higher sample rates and DSD. To be honest, I don't really have any albums worthy of 24/192 and DSD isn't exactly ubiquitous so I don't think this is a problem especially for day-to-day portable use.

Ahhhh... But the AudioQuest Dragonfly Black can "render" MQA you say! That surely must be a significant feature and worthy of consideration, right?

Indeed, it is worthy of consideration... And objective evaluation! That, my friends will be the topic for PART 2 next week :-).

--------------------------------

Seems like a bit of a dry spell in audiophile-land this week in the news. That's alright I suppose as we head towards warmer weather here in the northern hemisphere and summer holiday season for the kids.

I have seen a few web posts though on the documentary series "American Epic". Looks like an interesting series to check out this summer with great music and information about the history of early American music.

Over the years, I have talked about the importance of value when it comes to being wise consumers. Previously I posted on "The Value of High Resolution Audio" and I could not help but think of that when I saw some ads coming through that the music featured in American Epic was available at places like HDtracks in 24/96!

Wow! Do we need 1920's and 30's recordings made using old disc-cutting lathes be encoded in 24/96? No matter how much "restoration" was involved, is there any signal in those recordings that even comes close to using the "high resolution" capabilities, much less even CD quality? Obviously not, and it's grossly evident when you listen them! Furthermore, HDtracks is selling these albums in 24/96 for ~US$18.00 - same price as vinyl where there's actual printed material and an LP, or a CD for ~US$12 which again contains real material and can be sold used in the future if you want to recover a few bucks.

To me, here's a clear example of where "high-resolution" just does not compute at all because there is absolutely no value in paying more for literally nothing - heck, are there even pride of ownership bragging rights attached to hi-res very old recordings?! How odd...

Happy Father's Day to the dudes with kids out there.

Have a great week ahead and hope you're all enjoying the music!

Addendum (June 25, 2017):
Part 2 with an exploration of the Dragonfly's MQA Rendering now up.

17 comments:

  1. Could we get a subjective sound quality comparison between these DACs?

    ReplyDelete
    Replies
    1. Hello TL,
      Yes, a very important question. And at the same time a very difficult one to answer without context! I think you've seen that most web sites shy away from such comparison questions.

      My hope is that the objective analysis will allow you to evaluate your needs based on the data available. I believe that sound quality these days is almost uniformly good within the appropriate circumstance of use. Unless there's something very wrong, it would be hard to not enjoy the music from these little DACs. For example, if I wanted to use a DAC primarily for an iPhone, something like the LH Geek Out would obviously be out due to power consumption. Likewise, for very low impedance headphones, I'd avoid the AudioEngine D3 which I found had a 1kHz impedance of ~3-ohms.

      So, to try to answer the question somewhat, let's consider a few scenarios:

      1. Out walking with my phone and don't mind dragging out adaptor cable and a DAC with low impedance & sensitive IEM's:
      Who cares about nuances! You're walking, birds are singing, you'll hear bus and traffic noise. Dragonfly Black will do the job with low power consumption, good mobile phone compatibility. Great <1-ohm output impedance for headphone matching.

      2. Travelling with my Microsoft Surface 3 laptop plugged in, wanting to enjoy a quiet evening of music in the hotel... I've got my 63-ohm Sony MDR-V6 workhorse headphones with me - not too cheap but certainly not expensive such that if they broke on the trip, they're easily replaced. (I find them more comfortable than IEM's.)
      SMSL iDEA > LH Geek Out V2 > AudioEngine D3 > Dragonfly Black > Dragonfly V1.2
      On my laptop I have some DSD music along with various PCM tracks which the little iDEA can play. The V6 has great sensitivity with 106dB/mW so I don't need anything really powerful to drive it. The SMSL sounds clean, low noise floor, great tonality as one would expect from a recent ESS Sabre DAC, and can easily play back DSD with hardware volume control. It also doesn't demand much power from the laptop USB port. Just watch the issue I had when using on Android / Linux.

      3. At home with a workstation desktop. I've got my Sennheiser HD800 headphones with me. Let's say I don't want a headphone amp on my table top... These are 300-ohm impedance headphones with 102dB/mW sensitivity and I want the best sound with good dynamics and ability to play hi-res PCM and DSD.
      LH Geek Out V2 > SMSL iDEA > AudioEngine D3 > Dragonfly Black > Dragonfly V1.2
      The Geek Out V2's 1000mW into 16-ohm mode sounds fantastic with "audiophile" quality headphones. With 2.2Vrms output, <1-ohm output impedance, this thing matches quite well with most headphones including some of the lower sensitivity ones. I can click the button to put it into the cooler running 100mW/16-ohm mode when I don't need as much power. I've actually used this combo a number of times over the last couple years and have been impressed by the bass response, nuances extracted, and can listen for hours (most of this owing to the comfort from those oversized HD800 soft ear pads).

      Overall I was not impressed by the channel imbalance of the Dragonfly 1.2 (at least the sample tested) and other qualities have been superseded by the newer DACs. The AudioEngine D3 is the least expensive of them all here in Canada so from a feature perspective, it's the same as the Dragonfly but measures better with good peak voltage of 2Vrms. The main criticism is the higher output impedance and higher current draw.

      I know you asked about "subjective sound quality comparison". Clearly from what I have written, that would not be my main consideration and in fact CANNOT be used as the main differentiating factor in isolation from the rest of the system!

      Enjoy...

      Delete
    2. Thanks for your thorough reply!

      Delete
  2. Excellent write-up!

    I think you touched upon an important element in the audiophile hobby... that something can still sound very good even though it may measure poorly.

    I have been very impressed by my Black v1.5... I had intended to purchase a Red but upon demoing (H6 v1, HA-FX750, & SE535LTS-J) I found there wasn't a huge difference between the Red and Black. I actually ended up buying the Black (along with the JitterBug, I know, I know...) as I really did prefer the Black DragonFly's ever so slightly darker/softer (more mellow?) sound signature and lower volume control... it was much easier to fine tune and adjust to a lower volume than on the Red which was much, much louder due to its higher amp output.

    What was the most surprising to me while demoing that day was that I honestly couldn't pinpoint any major improvement over the DragonFly Black with either a Deckard or Benchmark DAC3! Slowly coming to the realization (I am a slow learner) that anything beyond the DragonFly is simply overkill and well past that diminishing point of noteable returns for these pair of ears.

    ReplyDelete
    Replies
    1. Thanks Ricochet,
      Appreciate the personal experience not only with the DF Black but also the general experiences you've had with trying out the Red and even "big boy" DACs like the Benchmark DAC3!

      I obviously don't know your personal circumstance, age, exposure to loud concerts in the past, etc :-). But what you're expressing is "par for the course" even if when we read audiophile magazines and all those writers seem to have golden ears and can tell differences between $5000 DACs as if differences were "obvious" or "massive".

      AS I'm sure you already know, the reality is that differences between DACs are very subtle even at quite disparate price points. Sometimes, we can hear differences between devices like say 16/44 music played on a PonoPlayer vs. a DAC with more orthodox frequency response using music with significant high frequency content. I find that most of this correlate with objective results and can be verified with A/B listening with variables controlled as much as possible.

      BTW: Any opinion about that Jitterbug?!

      Delete
    2. Still on the fence about the JitterBug. Maybe hear a slight difference when hooked up to my PC (quieter background as its main job is filtering the dirty usb 5V power as well as all the wireless crap I have hooked up to my PC!) but nothing detected/audible thus far via my smartphone... but for USD50 I have no complaints.

      I had some strange sound loss problems when the JitterBug was plugged into my usb hub... switched it around (usb hub now plugged into the JitterBug which is plugged into a rear usb 3.0 port) and I haven't experienced any sound loss since. So my audio signal path:

      PC > AQ JitterBug > UGREEN USB Hub > AQ DragonFly Black > Headphones

      Delete
    3. I do think the Dragonfly Black's sound is a bit lacking, despite being rather impressive. The greater finesse and detail of the Red seems to bring that last bit of "hi-fi-ness" out. IMO the Red is what the Black should've been in the first place. Anything better than the Red hasn't really caught my ear at this point.

      Delete
    4. @Ricochet:
      If you plug the Jitterbug like that, wouldn't you theoretically lose some of the benefits since you don't know how much "noise" the UGREEN hub would inject into the system from its switching power supply?

      Not to freak you out of course because I don't think it makes a difference :-). I've only been able to measure things like PHY microframe packet noise in the past through a very sensitive analogue system picking up the interference and never through the output from a USB DAC...
      http://archimago.blogspot.ca/2015/05/measurements-usb-hubs-and-8khz-phy.html

      Speaking of the Jitterbug and these kinds of USB devices, has anyone seen objective results or evidence of any improvement anywhere?! Questionable audio guru John Swenson of USB REGEN (1 port USB hub! LOL) and MICRORENDU fame I believe suggested objective results were forthcoming years ago. What evidence is there for the $175 REGEN and $325 ISO REGEN and potentially more if you want an expensive power supply? IMO this is all a scam that cannot survive open exploration into the claims.

      @Degru
      Yes, I assume the Red would measure better. Depending on where it's used and whether you need the capabilities like the stronger output for your headphones; the real-world benefits will depend on these factors.

      Would be great to know how much current the Red draws from the USB port.

      Delete
    5. Yeah, I have been a little worried about that but no choice but to put it there in my chain as when I plugged the JitterBug into the UGREEN USB HUB I was experiencing a daily loss of sound across the board. The JitterBug and hub just weren't playing nicely with each other in that configuration. I don't hear any difference though between the DragonFly direct into the JitterBug or via the UGREEN USB Hub.

      Delete
  3. I would love a similar review of the Dragonfly Red. I got the Dragonfly Black, and despite being quite impressed by it, something was a little.. off about it. I returned it and got the Red, and that's been my daily driver DAC ever since.

    ReplyDelete
  4. Off Topic ...
    If you are interested, there is people stating that the Ethernet cable makes an audible difference: https://community.roonlabs.com/t/ethernet-treatment/26874/216.

    ReplyDelete
    Replies
    1. No surprise Teodoro! As I've written before, it makes no sense at all but like all things in the audiophile world, people will have their own opinions and some will listen and incorporate testimony.

      Been there, done that in 2015 :-)
      http://archimago.blogspot.ca/2015/02/measurements-ethernet-cables-and-audio.html

      http://archimago.blogspot.ca/2015/02/measurements-intercontinental-internet.html

      and one more in late 2016:
      http://archimago.blogspot.ca/2016/11/measurements-on-value-for-ethernet.html

      Those guys might want to check the articles out and see if they can find a difference I didn't...

      Delete
  5. So it turns out that the Drangonfly black does indeed use a minimum phase filter as advertised unlike the 1.2 which had a linear phase filter. I know you don't believe that the pre-ring is audible, and it's likely hard to produce solid evidence for that. I believe my existing DAC, the dacmagic xs uses a linear phase filter simply through the sound characteristics, whereas my phone, the LG G3 uses a minimum phase filter. I find redbook to be dryer on both devices, and whenever I switch to 24/192, I smile a bit and can more easily locate the elements.

    Since the dragonfly 1.2 and black have been so well reviewed despite their less than stellar objective measurements, while the SMSL idea looks very good objectively, I wonder how well it would perform in real life. Guess I'll need to buy one to find out.



    ReplyDelete
    Replies
    1. Hi LY,
      Remember that the pre-ringing is only present in music with segments that are not properly low-passed below Nyquist for the samplerate you're using. You can test this out yourself with your own upsampling software that allows different filters to be used like iZotope RX.

      Also remember that minimum phase filtering has been used for *years* by devices like the iPhone and iPad! Did you ever hear of those sounding more "relaxed" or somehow better as audiophile devices?

      The Dragonfly 1.2 had an unfortunate channel imbalance in the copy I tested that I found noticeable and this alone disappointed me. I'm actually OK with the DF Black 1.5. The low power use is great if you want to connect this to a cell phone. But objectively it's clearly not the best I've heard or seen...

      Delete
    2. Hello

      Isn't is the case that all pcm music regardless of sample rate contains pre and post ringing hard coded as pcm requires a brick wall filter when sampling, but an ideal filter (sinc function) requires an infinite delay, so in practice there is some aliasing, phase distortion and ringing. On reconstruction of course, more ringing is added to the ringing already present.

      I'll have a look at the software you mentioned.
      I do have an ipad air 2 but not an iphone, I do find both my ipad air 2 and my phone sound more relaxed, but my dac has better soundstage and drive, and I think that's a result of the minimum phase filters smearing more in time. In fact in Foobar, I use the SSRC resampler to resample redbook to 176.4khz, and I prefer the linear phase filter for this purpose as it's more accurate in that I feel I get a better sense of space and separation, it still sounds dry and less natural though compared to my 24/192 music. I don't think bit-depth is as important as the sample rate as your objective tests show, few if any dac's can reproduce more than 120db of dynamic range.

      The low power use of the dragonfly black is really good especially on a smartphone, but I agree that the objective figures aren't the best.



      Delete
    3. Hi LY,
      Yeah, have a look at the software and play with it...

      As for PCM having pre- and post-ringing "hard coded". I suppose these can be found in some circumstances. For example, we can see this with synthetic music containing stuff like square waves. When resampled you'll see the post and/or pre ringing.

      However, if you're recording natural sounds through an ADC with appropriate low-pass antialiasing filtering this should not be an issue. Likewise when playing back on your DAC with the upsampling filter, there will be no issue.

      Would love to see any links you have suggesting some kind of ringing issue!

      Delete