PDA

View Full Version : EWQLSO 16-bit vs 24-bit


Sycraft
12-08-2009, 10:35 PM
Since people asked, here is a test of 24-bit Play samples vs 16-bit Play samples, specifically with Symphonic Orchestra. To try it out just download the archive (http://sycraft.org/24bit/SOTest.7z).

Inside, you’ll find 4 files. They are all WAVs of a simple MIDI rendition of Promenade from Pictures at an Exhibition. The files are labeled in the form of “SO test xx-bit xx-bit.wav”. The first bit indicates the size of the samples used for the creation, the second one indicates the size of the wave file itself. So you can hear both 24-bit and 16-bit sources in 24-bit and 16-bit destinations.

The idea here was to keep things very simple so as to remove any differences in the versions. This is not a masterpiece of audio engineering. All the instruments are just simple sustain versions, no switching of any kind, no processing other than dither/bit conversion. All Play settings with regards to the samples were left at defaults. There are no effects of any kind added. No SRC, no limiting, not even any normalization. So what you’ve got is just the stage mics and their sound, nothing else. The levels I made sure to keep precisely the same in both mixes. I did a realtime bounce, since EastWest doesn’t recommend fast bounce at this time. The VI tracks were mixed down to a 32-bit FP wave, which I then dithered to 16-bit and 24-bit final waves. Same dither is used in all cases.

So, this is as simple and as close as I can get two versions. There should be no differences that aren’t a result of the samples going in, and the bit size of the final waves.

The files are packed with 7-Zip (http://www.7-zip.org/download.html) so you’ll need that or something else that can expand the 7z format to get at them.

If you want a blind ABX test, Foobar (http://www.foobar2000.org/) has the ability to set that up.

As a note, there is likely very little audible difference at 16-bit, if any. Doing a difference on the 24-bit files gives an average level of around -93dB to -96dB. However it does have some spikes as high as -68dB. However, when I listened to the difference file, the noticeable thing was noise. With each additional instrument that kicked in, the noise level raised. What this means is a big thing you get with the 16-bit samples is, no surprise, higher noise floor.

How much that is going to matter is largely dependent on how many instruments you use. In this case I had 7 instances of Play, with one instrument each. Larger numbers of instruments would lead to a higher noise floor and thus more potential difference.

A.Leung
12-09-2009, 12:08 AM
And THAT my friends is the difference between 16 and 24. :)

Jose7822
12-09-2009, 12:23 AM
OMG, you're the man Sycraft!

Thanks so much for providing these samples. This has finally closed the case for me :-)


Take care!

A.Leung
12-09-2009, 12:51 AM
I wonder though how many people will dismiss the differences from this test. Just because you can't HEAR the difference doesn't mean your not gaining a lot in the mixing environment.

Sycraft
12-09-2009, 01:17 AM
All depends on what you are after. I believe that people should have the most information possible and make their decision.

This test suggests empirically that if you are using a simple (as in not many instruments) arrangement and targeting a 16-bit format, that 24-bit samples may gain you little. That doesn't mean they are useless, just that in some cases they may not be worth it.

Personally I like 24-bit samples for the same reason I like 64-bit mixing: Insurance against noise. The lower you can get the digital noise floor, the more natural your stuff will sound. Quantization noise sounds wrong to people.

Also if you were mixing for a movie, 24-bit samples could get more critical since your music might be 20-30dB below peak in the bitstream and thus closer to the noise floor. Of course if you are using more instruments (and probably more voices for the ones you do use) this is also true as those all add noise. An arrangement with 100 instruments would have significantly more noise than mine with 7.

However, to help people understand here's the diff files for the tracks (http://sycraft.org/24bit/diff.7z). This is taking the two versions, inverting one, and mixing them. The result is all the sound that is present in one but not in the other. As I said, the thing to note in the 24-bit version is how you can hear the noise increase as more instruments kick in. An important thing to remember those is that I normalized this to make it easily audible. The noise is actually down in the -90dB range. So while this is the difference, it is amplified.

My initial assessment would basically be that 24-bit is not critical to the final sound, in many cases you can have little to no difference in a 16-bit target. However it does give you more headroom, more leeway. Each time you add more instruments, more tracks, more voices, you are upping the noise, losing detail. When you cross the line that it becomes audible I'm sure is highly variable, but it can happen. 24-bit buys you a whole lot more.

noiseboyuk
12-09-2009, 01:30 AM
Thanks for the test, Sycraft. Well I think my sig applies! As you say, if even combined noise is in the -90db range.... well. Allen makes a fair point about mixing in a 24 bit environment, but as for samples...

Ironically, with the current deals it can be actually cheaper for a 16 bit Gold user to upgrade to 24 bit-only Platinum in a 2 for 1 than it is to upgrade to 16bit close and ambient Gold mics (effectively the same product but different bit rate). I'd honestly rather the latter... it's a third less strain on my system that I can use for other valuable things (always a killer argument against 24 bit). So maybe the price incentive is right?! Is 16 bit the new 24 bit?!!!

Jose7822
12-09-2009, 01:52 AM
Well, for me personally, the difference in sound quality between the two doesn't merit the CPU strain that my PC will endure. Not to mention the extra RAM needed in order to load all the mic positions with 24 bit files. Otherwise, I would have definitely gone for the Platinum upgrade going for $100 USD more than the extra 16 bit mic set, which I think is a very good deal. Choose what best suits you. In any case, thanks again, Sycraft, for providing these files.


Take care!

Ravich
12-09-2009, 01:57 AM
Tried comparing both samples of the 24bit destination files. I noticed the difference when starting both files at about 30 seconds in. The 24 bit samples seem to create a sense of space by comparison to the 16 bit seeming much more flat. The viola line just sort of jumped out for the 24 bit samples, so I guess that would mean a lot less "fighting" over which line can be heard prominently amidst the mix. I guess the detail of the 24 bit samples might be noticible for for stuff like the brass at the beginning. I think when the brass gets thicker harmony that it doesnt get quite as much of that... I dont know what it's called but the sort of modulating sound/effect that makes sampled brass sound fake. At least it seems to mask it slightly.

I think it'll be worth spending an extra 100$ to get Platinum Complete Plus instead of the gold upgrade (besides, then I can get SD2 PRO for free). Even if my system cant handle 24bit samples yet... I can see it'll be a worthwhile investment.



Thanks for posting this!

A.Leung
12-09-2009, 02:26 AM
The viola line just sort of jumped out for the 24 bit samples, so I guess that would mean a lot less "fighting" over which line can be heard prominently amidst the mix.

Absolutely. That is what I noticed when I did the test for myself a few years ago. 'Instrument definition'. And creation of 'percieved space' as well as reduced noise and added headroom. I think you have to either have 'engineer' ears or you have to train yourself to hear the difference.

It is sort of like some people can pick out either all of the instruments playing in a mix or only some of them.

OneThrow
12-09-2009, 03:28 AM
16 bit is good quality. 24bit adds an extra dimension. Mm that about sums it up. Never thought of it like that before, but it seems to fit.

Interesting discussion. ;)

Jose7822
12-09-2009, 08:40 AM
I think it'll be worth spending an extra 100$ to get Platinum Complete Plus instead of the gold upgrade (besides, then I can get SD2 PRO for free). Even if my system cant handle 24bit samples yet... I can see it'll be a worthwhile investment.


Actually, the extra $100 USD I mentioned earlier was to upgrade Gold Complete Plus to Platinum Standard. But for the Platinum Plus upgrade, it is another $100 USD from that, and a total of $399 USD from Gold Complete. Just thought I'd clarify that.

Ravich
12-09-2009, 09:48 AM
I'm still running kontakt powered Gold, so I believe that the difference is only 100$ for me, though I could be mistaken.

JCL
12-09-2009, 11:33 AM
Thanks for doing this, sycraft. Do you have a final wav file or mp3 for a quick audio comparison - I'm not good with zips, wrappers, etc.

Jose7822
12-09-2009, 07:31 PM
I'm still running kontakt powered Gold, so I believe that the difference is only 100$ for me, though I could be mistaken.

In that case, I believe you'll have to add yet another $100 USD to the $399 price I mentioned earlier if you want to upgrade NI Gold XP to Platinum Plus PLAY version. You can get the exact prices by choosing the upgrade path that applies to you and entering your serial number in the box here:

http://www.soundsonline.com/product.php?productid=EW-177

You have to select "Purchase/Upgrade" on the top right.


Take care!

Ravich
12-10-2009, 02:32 AM
Yep, I did that and had it in my cart. Selected that I was upgrading from a previous NI powered verison, selected Gold + XP, gave the serial number, selected did not pre pay for my play upgrade, and it was 399.

Sycraft
12-10-2009, 02:48 AM
Thanks for doing this, sycraft. Do you have a final wav file or mp3 for a quick audio comparison - I'm not good with zips, wrappers, etc.

Ya ok. Just go to the directory (http://sycraft.org/24bit/) and you'll see the waves in there along with the zipped versions.

Topo
12-11-2009, 12:57 PM
How might one access these files on a Mac? The download only produces a huge file of text characters.

Since people asked, here is a test of 24-bit Play samples vs 16-bit Play samples, specifically

with Symphonic Orchestra. To try it out just download the archive (http://sycraft.org/24bit/SOTest.7z).

JCL
12-11-2009, 01:26 PM
Got 'em, Sycraft! Thank you! Excellent demo on your part, and "Kiev" was an excellent choice for such a demo. I know you spent a lot of your time doing this; I hope it inspires others to do the same. I look forward to hearing your complete "Pictures". Thanks again

JCL
12-12-2009, 08:21 AM
Sorry, I should have said "Promenade" - not "Kiev".

Oldfield
12-14-2009, 02:24 AM
Stupid question, but this seems a sensible place to put it, since we're talking about different versions.
I just got the complete composer's collection, so I have SO Gold Complete - I just have the stage mics for each instrument @ 16 bit, right?

I'd quite like the other mic positions for some instruments. I guess that's just with the platinum something version.

What's the best upgrade for me?

interpolate
12-14-2009, 02:34 AM
If you don't care about 24-bit then go for the Gold Edition Plus upgrades which adds close and hall microphones.

A.Leung
12-14-2009, 02:35 AM
Just upgrade your Gold to the extra mics.

tarekmansur
12-14-2009, 04:28 AM
Hey guys, I think right now it is most worthwhile to buy Platinum Plus & a library for the money instead of putting them on a Gold updated with several mic positions when you can not have the offer buy 2 pay for the 1st.
But if I to be honest I noticed no difference in sound quality... I always render in 24 bit and then after converting to 16 bits.
Haha so in my case it becomes Platinum Plus and Silk!
And Sycraft thanks for the demo!

Love//Tarek

Oldfield
12-14-2009, 05:31 AM
Just upgrade your Gold to the extra mics.

http://www.soundsonline-europe.com/product.php?productid=EW-193U this be the one?

A.Leung
12-14-2009, 05:44 AM
That's it! :)

Sycraft
12-14-2009, 06:01 AM
Yes, you have the stage mics, 16-bit. So your upgrade choices presently:

1) Additonal mic position(s) with Gold Plus. (http://www.soundsonline.com/product.php?productid=EW-193U) This stays at 16-bit and gives you one or both additional mic positions. Currently it is $150 per mic position or $200 for both the close and hall mics.

2) Upgrade to Platinum. This gets you all three mic positions, and changes the bit depth to 24-bit. That is $300, presently. 24-bit does mean more disk and CPU load, though I've not found it to be problematic on my system.

3) Upgrade to Platinum Plus. This is all three mic positions, both 16 and 24-bit. More or less Gold Plus and Platinum combined. That is $400 presently.

So kinda depends on how much you want to spend. $150 will get you an additional position, so if you just want close mics, you can have those for $15. $200 gets you all the positions. $300 gets you 24-bit and $400 gets you everything.

Oldfield
12-14-2009, 07:29 AM
I guess I just want the extra mic positions, maybe both. I think I got the price to €138 or something, whatever that is in real money :cry:

If i feel the need to have 24 bit samples the I might upgrade again in the future. I don't really see the need for them right now...

gstitt
12-14-2009, 10:10 AM
As a note, there is likely very little audible difference at 16-bit, if any. Doing a difference on the 24-bit files gives an average level of around -93dB to -96dB. However it does have some spikes as high as -68dB. However, when I listened to the difference file, the noticeable thing was noise. With each additional instrument that kicked in, the noise level raised. What this means is a big thing you get with the 16-bit samples is, no surprise, higher noise floor.

Finally! A post about 24-bit vs 16-bit that doesn't make me want to jump off a building. I've been trying to make this point for years without much success. In fact, I posted a similar experiment a few years ago. I've heard countless claims of "night and day" audible differences between 16 bit and 24 bit. I can't imagine anyone claiming that a difference at -93 db, or even -68 dbf, is "night and day". I'd be willing to bet that I could add random sounds at -68 db to an existing track and practically nobody would be able to even notice. I still haven't seen a study that shows that people can distinguish between 16-bit and 24-bit for the same signal recorded at reasonable levels. However, I've seen several that suggest people cannot tell them apart in general. For extreme cases (e.g. HUGE dynamic range), obviously there is a difference, but I can't name a single piece of work that needs that much dynamic range.

As suggested by others, IMO the main advantage of 24-bit is being able to mix at reasonable levels with plenty of headroom without having to worry about noise.

Shenrei
12-14-2009, 10:19 AM
Thank you very much for posting these. I can definitely hear the difference, though I'm sure a lot of people won't. I agree with what was said earlier that it takes an engineer's ear to hear the difference. I'll add to that a good monitoring system that you know really well. The 24 bit version definitely sounds "lighter," "airy," and more "floaty," than the 16bit version, but the difference isn't as significant as I thought it would be, at least with those files. I'd garner a track with lots of layered instruments would widen that gap.

Sycraft
12-14-2009, 10:25 AM
As with most things digital at least, it is going to make more difference the more you do, as in the more tracks you have, the more calculations you do, etc, etc. The rounding errors slowly add up and you get more and more noise and so on. This mostly applies to doing lots of calculations which is why processing is generally done floating point and 32-bit or 64-bit. However in terms of samples or recordings, the more tracks you have, the most instruments, the more noise you'd get using 16-bit over 24-bit.

So like you say, it buys you more headroom, more leeway. It isn't about attaining some amazing new sound you couldn't have before, it is about making sure you don't have problems in certain situations.

I'm a big fan of using more precision than you think you need in production because I've seen the problems that can result if you don't in video. With video, you only have 8-bits per color channel on normal consumer displays. Yet you discover that you sometimes have to do 32-bit floating point calculations or you get visible colour banding in the final product. The minor errors of lower precision add up to visible artifacts over tons of calculations in material shaders and lighting.

gstitt
12-14-2009, 10:32 AM
I wonder though how many people will dismiss the differences from this test. Just because you can't HEAR the difference doesn't mean your not gaining a lot in the mixing environment.

Hopefully, nobody. What's funny is that I've been trying to make this point, but as soon as you say there is in general no audible difference, most people don't listen to what you say after that.

An arrangement with 100 instruments would have significantly more noise than mine with 7.

That's not necessarily true, is it? If I sum 100 signals with the same SNR, the resulting signal has the same SNR, doesn't it? Or, are you saying that with 100 tracks, you are more likely to have tracks with a higher SNR?

Sycraft
12-14-2009, 10:45 AM
The problem with quantization noise is that it's not random. So you end up getting more noise (and/or distortion depending on how things go) the more you have. You can hear it in the difference files I posted, as new instruments kick in, the noise level goes up.

More or less you can look at it as rounding errors. Taken alone, it is not problematic. However as more you have, the more it adds up.

gstitt
12-14-2009, 11:01 AM
So you end up getting more noise (and/or distortion depending on how things go) the more you have.

I agree that the noise will get louder, but so will the summed signal. In other words, the difference file will get louder, but I don't think this necessarily means that the SNR is worse.

By the way, don't take my questions as arguing. I just want to make sure that all these issues are perfectly clear. I certainly could be wrong here.

More or less you can look at it as rounding errors. Taken alone, it is not problematic. However as more you have, the more it adds up.

I have the same thoughts here. The sum of the quantization errors will certainly increase with more tracks, but I don't see why the SNR has to get significantly worse.

Jose7822
12-14-2009, 11:46 AM
gstitt,

The SNR gets worse when you start using compression/limiting, since you're reducing peaks and thus making the crest factor (difference between Peak and RSM levels) smaller. But not necesarily when adding more instruments, which I agree with you there. Unless one starts to compress the hell out of each track, I wouldn't worry about the noise floor rising much. Typically, this is not the case with the type of music you would be mixing these intruments in where light compression is used. Worst case scenario though, would be having to use an expander to lower the noise build-up.


Take care!

gstitt
12-14-2009, 12:31 PM
I can definitely hear the difference, though I'm sure a lot of people won't. I agree with what was said earlier that it takes an engineer's ear to hear the difference.

I used to think I could hear the difference, but now I'm not so sure. Here's a test you can do. Write a quick script that randomly outputs either 24 or 16 with equal probability. Next, export audio to different files, using either 24-bit or 16-bit as determined by the script. Wait a day or two so you forget the order, then listen to all of the audio files. Write down your guess of whether each file is 24 bit or 16 bit. I'd be surprised if the % of correct guesses wasn't around 50%. If there is truly a "night and day" difference, it should be around 100%.

To do this, you have to use audio that has a chance of working well in both 24-bit and 16-bit. So, avoid a huge dynamic range. There are likely examples where the difference is clear, but in general, I'm guessing the results will be around 50%, which means you can't tell a difference.

PBCliberal
12-14-2009, 02:19 PM
At my age and with the levels I've run studio speakers over the years, I probably would not be able to hear the difference between 16 and 24 bit. I can rarely hear artifacting in 384Kbps compressions.

But this isn't about that. CDs are quite satisfactory at 44.1kbps 16 bit for most everybody's listening, and if you're meticulous at every point in the process, 16 bit is great.

But consider the future. Billie Holiday did three recordings for Decca that used a newfangled device called the Ampex tape recorder. The tape speed was 30 ips, and the results were far beyond anything heard before. Of course, the only people who could hear it were the ones listening to the masters, because the playback was only used for creating the release format, which was 33, 45, and 78 rpm grooved disks.

When the CD came on the scene with the move to find original masters and reissue everything, the difference between the tape and even the glass masters of the period was stellar, and suddenly everybody had access to something close to the original.

While production formats that are the same as release formats will work admirably, you never know what's down the road. The public's collective ear gets better (people used to be unable to tell the difference between AM & FM), and new enhancements to reproduction may depend upon computer analysis of content that mere mortals are unable to hear.

So its not just about how it sounds now on today's speakers with today's listeners; its how it will sound in 50 or a hundred years to people and machines that may detect subtleties we're unaware of.

noiseboyuk
12-14-2009, 02:45 PM
It's an interesting argument PBCliberal, but I don't buy it. Our ears have specs, and there are finite limits to what we can resolve. I can't see us as a species evolving into bat-territory any time soon, or getting minus decibels in s/n tests. The specs of AM radio et al were clearly way below the limits of human hearing.

CD is pretty damn close to those limits. Transients -sure. Bad mastering and coding - sure. But unless something has gone horribly wrong, with today's tech we're in the realms of minutiae. And as my sig says...

It kinda makes me sad in a way. I grew up in the era of vinyl and cassettes, both flawed, and I'd obsess over making both as good as possible. But I've kinda lost interest now, there's nowhere useful left to go, all quality improvements from hereon in are strictly for the purist audiophiles. And as a market, I don't think they're growing... possibly the opposite (says he with no evidence to back it up!). I've lived through so many technological breakthroughs, but all the important stuff has happened before this decade. And I'll wager that's it (purely in terms of quality).

Oh well - another lousy day in paradise!

Shenrei
12-15-2009, 07:02 AM
I used to think I could hear the difference, but now I'm not so sure. Here's a test you can do. Write a quick script that randomly outputs either 24 or 16 with equal probability. Next, export audio to different files, using either 24-bit or 16-bit as determined by the script. Wait a day or two so you forget the order, then listen to all of the audio files. Write down your guess of whether each file is 24 bit or 16 bit. I'd be surprised if the % of correct guesses wasn't around 50%. If there is truly a "night and day" difference, it should be around 100%.

To do this, you have to use audio that has a chance of working well in both 24-bit and 16-bit. So, avoid a huge dynamic range. There are likely examples where the difference is clear, but in general, I'm guessing the results will be around 50%, which means you can't tell a difference.

That would be something else entirely. Exporting the audio in specified bit rates is irrelevant, as we're talking about the source bitrate, not the resulting bitrate. The difference will become more apparent when you listen to patches relative to other patches in the background.

gstitt
12-15-2009, 07:44 AM
It's different, but certainly not irrelevant. It's a reasonable test to see if you can tell the difference between 24-bit and 16-bit for exported audio. I was addressing the general claim of "night and day" differences between 24-bit and 16-bit audio, not EW samples.

Hopefully this will make you happy. Export 2 different versions of a reasonable-sized midi orchestration with 16-bit and 24-bit samples, always exporting to 24-bit. Write a script to copy each file an equal numbers of times, while randomly renaming the files. Do the same test I suggested before. I posted a similar test a couple years ago, but it was only with a few instruments.

Does anyone know of an easy way to stream wav files online? If so, I'd be happy to setup a true blind test.

Jose7822
12-15-2009, 12:08 PM
gstitt,

You don't have to go that far. Just make the wave files available with a link and suggest the use of Foobar2000, which has built-in ABX capabilities. OR, export the orchestral piece alternating between the 16 bit and 24 bit samples into a single wave file without telling us the interval or when the samples change until the end (in a "Don't Read Me Until You Listen to Me First" file :-P).

gstitt
12-15-2009, 02:02 PM
Not a bad idea. I had considered that, but there is still some room for cheating by pre-analyzing the files outside the test. Foobar would work great for personal use, but I'm actually to the point where I think the entire community would benefit from widespread semi-formal experiments. I'm actually considering creating a "benchmark suite" of audio files which can be transformed/processed in various ways, with an automatic way of having people choose the version they think is correct, with results automatically being sent somewhere to prevent people from lying. With thousands of participants in a true blind test, we might be able to start making general conclusions about numerous touchy subjects. It could potentially be generalized enough to test different bit depths, sampling rates, different plugins, etc. It wouldn't even be that hard to create, although making it cross platform is where it could get tricky.

Jose7822
12-15-2009, 09:15 PM
Hmmm, that's out of my league. This would actually be something if you could pull it off.

Please, let us know if you find a way to do it.

gstitt
12-16-2009, 08:30 AM
I have a few ways in mind, but this has the potential to open Pandora's box if I'm not careful. I tend to get carried away with small projects, basically turning them into full-blown products. I may have to limit it to Windows initially. I already have 0 free time.

Domenico
03-22-2010, 04:39 AM
Sadly, the files are not available anymore :( I would love to have a listen it seems very interesting.

Sycraft
03-22-2010, 10:06 AM
I took down the archive but the files are still up as individual files. You can find them here (http://sycraft.org/24bit/). The SO Test ones are the ones that I was talking about in my orignal post.

Domenico
03-24-2010, 07:18 AM
I took down the archive but the files are still up as individual files. You can find them here (http://sycraft.org/24bit/). The SO Test ones are the ones that I was talking about in my orignal post.

Thank you Sycraft,

I downloaded them and give them a listen on my monitors, right now I have only headphones available. I made a delta file in Wavelab and indeed when they cancel out each other the only thing that is left is noise that is building when more instruments are introduced. However, I had to add some 40-50 db of gain for this to be heard.

I suppose if I want to make a proper comparison I should hear side by side the 24-24 and 16-24 bit files right?

Sycraft
03-24-2010, 08:16 AM
Kinda depends on what information you are after. If you want to try and hear the difference between samples, with the highest resolution possible, you want 24-bit wave files and to vary the source. That would be the 24-24 and 16-24. Now if you want to hear the difference as it would apply to a CD, that would be 16-24 and 16-16.

juliancs
03-24-2010, 08:41 AM
All has to be relative. Writing for a shooting game a 16 bit sample isn't going to get noticed much! Thanks for the test, was interesting to see (hear?).

cnk
03-25-2010, 02:15 AM
At least Hollywood Strings will be 24 bit ;) And I think it will be my first 24-bit library set.