PDA

View Full Version : A Method for Distributing a Single QL Piano Over Multiple Hard Drives


BillyD1953
04-29-2008, 07:55 AM
This post provides users of QL Pianos (or any other very large VI sample library) with a method for distributing their sample files over multiple drives, even for an individual instrument in the collection, such as the Steinway D, for example. Other threads have mentioned that for optimum performance it is best to buy the fastest hard drive you can afford when using QL Pianos, which is the largest VI sample library ever released. Some users think RAIDed drives help, but many say that RAID is not useful in a situation where thousands of small files are being streamed rather than a few large files. My method allows you to distribute your samples over multiple drives without using RAID. My examples below use two hard drives, but the same principles apply to three or four or even more drives.

Let's take the simplest case first. Say, for example, you would like to run the Steinway D with the Player mic and the Room mic together, maybe even with some convolution reverb, but your system just doesn't have the horsepower for it. With my approach, you split up the instrument into two separate instruments and run them both together in PLAY standalone mode. In this case, you would set up the standard QL Pianos Library folder structure on two separate hard drives. One drive would contain the instrument file for the Player mic perspective and the other would contain the instrument file for the Room mic perspective. Then you simply add both instruments to PLAY and you're all set to run them simultaneously in the standalone mode as if they were a single instrument.

But what about a more complicated situation where you'd like to split up the samples for just a single mic perspective over multiple drives. This is a little trickier but it can be done. Again you want to use two separate instrument files, one on each hard drive. However, in this case, instead of defining one instrument for one mic perspective, you want both instruments to be set up identically but with each calling up only half of the samples from the library. This cannot be done programmatically, so you have to divide up the samples so that each instrument has access to half of the samples. However, QL Pianos won't load an instrument unless all of the expected sample folders and files are present, so simply splitting up the sample files onto multiple drives won't work. Instead what you do is create empty dummy files of the correct file name for each sample file. Then, on each drive you store half real samples and half dummy files. Of course you put complementary real files on each drive so that over both drives all of the real sample files are present. Then you add both instruments to PLAY standalone and you're all set to go.

But how should you divide up the real sample files on the two drives. Probably the best way is to divide them up by putting the samples for every other note on one drive and alternate notes' samples on the other drive. There is probably an optimum way to do this to avoid key dependencies in your performance: You don't want playing in C or G-flat to matter. So you might try putting the C and C# samples on one drive and D and D# samples on the other drive and so on. Whatever works best. This will improve your performance and effective horsepower by having different drives responsible for streaming different samples. Calling the dummy files does not cause PLAY any problems and since they’re empty they don’t waste time streaming any data from the disk (they may still use the same amount of seek time, however). When you use this dual identical instrument setup in PLAY it sounds like one instrument but should perform better with less overall computer horsepower required, and having multiple hard drives is now quite affordable, especially as compared with purchasing a new, more powerful computer.

Ideally, an updated version of QL Pianos may come out with the necessary features to accomplish this more simply, perhaps with the ability to programmatically assign notes to an instrument. In the meantime, I'd be interested to know what sort of luck people have with this approach. Enjoy.

chest
04-29-2008, 09:11 AM
... a more complicated situation ... you have to divide up the samples so that each instrument has access to half of the samples ... create empty dummy files of the correct file name for each sample file. Then, on each drive you store half real samples and half dummy files. Of course you put complementary real files on each drive so that over both drives all of the real sample files are present. Then you add both instruments to PLAY standalone and you're all set to go.

... Probably ... divide them up by putting the samples for every other note on one drive and alternate notes' samples on the other drive. ... you might try putting the C and C# samples on one drive and D and D# samples on the other drive and so on. ... When you use this dual identical instrument setup in PLAY it sounds like one instrument but should perform better with less overall computer horsepower required ...

... I'd be interested to know what sort of luck people have with this approach. Enjoy.
Hm. Yes, I expect we'd all be interested to see how SOMEONE ELSE gets on with this.

Have you acatually tried doing this, BiilyD153? Or are you just hoping to delegate an emormous amount of work to other people who'll try it out for you? ;)

Or was this meant as a suggestion for EW, rather than for individual users?

I've not used Play yet - can you actually make it look at two different drives for the two sample subsets, in the way you're suggesting?

As an alternnative approach, I wonder whether there might be a way of transforming the MIDI data so as to assign the notes to two different (complete) installations of the samples - if so, that would remove the need to construct the thousands of dummy files.

BillyD1953
04-29-2008, 09:26 AM
Well, I think the first thing to try is just the simple one where you divide up two mic perspectives. No need for dummy files there. My extra drives are due to arrive Thursday. I should really take the time to write a utility program to create all those dummy files--unless someone else is chomping at the bit to do it. ;-)
Your MIDI idea might work, but I think with two complete installations of samples wouldn't you still be unnecessarily loading both full sets into RAM? As far as the issue of whether PLAY will access the samples from two different drives, I'm pretty sure that it works according to the location of the instruments you load. So, if you load your two instruments from two separate drives, it will know to follow the folder structure of each of those instruments to find its respective samples. I could even go ahead and test that on one drive using separate folder structures for the two instruments. The only thing I've actually tested so far is that I can use dummy files successfully and play two different mic perspectives as two separate instruments loaded into PLAY together (from one folder structure on one drive). I'm hoping some enthusiastic young go-getters join our thread. It could become a team effort. ;-)

chest
04-29-2008, 11:09 AM
Your MIDI idea might work, but I think with two complete installations of samples wouldn't you still be unnecessarily loading both full sets into RAM?
Oh, yes, I forgot about that - just focussing on your idea of using two drives.

Are you actually finding that polyphony is restricted by the "bottle-neck" of a single drive, rather than by RAM?

BillyD1953
04-29-2008, 11:29 AM
That's an excellent question. My computer has a 3.2GHz single processor, running Windows XP32, with 4GB of RAM (3GB effective RAM), and a reasonably fast Western Digital 500GB hard drive. The hard drive is almost full, so that's not ideal, plus the system and programs are all on the same drive as the samples, which is also not ideal. If I run the Steinway D with one mic and reverb, it behaves pretty well until the polyphony builds up with the sustain pedal down. Then it crackles and drops out. This happens even more easily and more commonly if I try to use 2 mics, although I can get some use with 2 mics and reverb as long as I keep the polyphony down enough. According to the indicator in PLAY it seems to be my CPU that is maxing out when this happens. The reports from Forum members differ quite a bit. I'm not sure exactly what kind of luck people are having. One member reported crackles and dropouts even with 16GB RAM, 2 separate drives for samples and system files, and dual quad core CPUs. Another one said he did fine with only 4GB of RAM and a dual core processor, but I'm not sure he was trying to play in real time with a low latency or not. I thought I would try using separate drives with my current system before investing in a new machine. I should know within a few days or so (when the new drives arrive) whether that will work out for me. I kind of doubt that it will. If I have to have a new machine I'll go head and bite the bullet and get one, because I think QL Pianos is the most incredibly beautiful sounding virtual piano I've heard so far. I like some of its competitors, including NativeInstruments Akoustik Piano, which I own, but for me none of them match up to QL Pianos. I guess that's not too surprising considering the enormous sample library it has. The Bosi is 87GB just by itself. :-)
I mostly use the Steinway D. Garritan & Steinway have just come out with one, too, which sounds pretty darn good in the demos. I may spring for that one someday too. Not sure yet.
I'd be happy to hear from any QL Pianos users who can decisively say they can use 2 or 3 mics with reverb in real time with low latency and not have any issues with crackles or dropouts. It would be nice to know exactly what the minimal hardware is that is required to achieve this--what kind of CPU power, hard drives, and RAM combination is successful.

chest
04-29-2008, 12:44 PM
... it behaves pretty well until the polyphony builds up with the sustain pedal down.
Slightly off your topic: how does the QL Pianos s/w deal with repeated piano chords played with the sustain pedal on? Does it just build up more held notes, or are the pedal-sustained notes whose keys were previously released cancelled by the repeated ones? (ie by the s/w (in effect) generating a MIDI Note-Off just before the new Note-On)?

BillyD1953
04-29-2008, 03:39 PM
Slightly off your topic: how does the QL Pianos s/w deal with repeated piano chords played with the sustain pedal on? Does it just build up more held notes, or are the pedal-sustained notes whose keys were previously released cancelled by the repeated ones? (ie by the s/w (in effect) generating a MIDI Note-Off just before the new Note-On)?

It sustains the original chord and overlays the new key strikes of the same chord on top of the sustained samples. If you keep playing chords quickly (whether the same notes or different), it will start crackling as the polyphony builds up. If you play the chords repeatedly but somewhat more slowly it takes longer for the sustained polyphony to pile up, but it eventually does and then starts crackling as with the quickly repeated chords.

chest
04-30-2008, 03:46 AM
It sustains the original chord and overlays the new key strikes of the same chord on top of the sustained samples.
I wonder if there really isn't a better way of dealing with repeated chords played with the sustain pedal on, though - unless, of course, the sustain pedal is implemented through modelling rather than sustain samples - it's far from obvious what else could be done.

It wouldn't be right just to cancel the previous note (by generating a Note-Off) because that would let a soft repeat cancel the sustain of a loud note; and in any case even cancelling a soft note could make the overall sustain too quiet. And it's no good trying to compensate for a cancelled note by increasing the velocity of the new note, because that would give the wrong timbre (too bright) in the new note. And just, somehow, "turning up the volume" of the new note wouldn't work properly if its timbre was softer than the previously sustained note's.

I wonder how closely the sound of a real piano sustain actually is represented by the basic method of letting the samples pile up. I suppose if it's better than any methods that don't consume the available polyphony so fast, users just have to "grin and bear it" while they wait for faster computers.

Can modelling the sustain effect (eg by IRs) be made to work well? Or well enough to be acceptable in a trade-off between quality and polyphony? Or is the CPU load for modelling too great?

BillyD1953
04-30-2008, 10:39 AM
I'm pretty sure their intent with QL Pianos was to make it the most realistic piano VI they possibly could. I'm not very familiar with modelling technology, but I would guess it would be a CPU hog, plus it would have to be pretty sophisticated probably to recreate the decay harmonics, etc. of a sustained piano key or especially multiple sustains piling up. I'm just hoping to ultimately make full use of this software without taking out an equity loan on my house. ;-)