A couple of weeks back I posted a puzzle about an experience that I just had with Server 2008 R2 software RAID subsystem: somehow the speed of formatting a new very large RAID-5 array was highly unpredictable. It advanced by a mere 12% a day for the 4 days and then suddenly sped up 4x and finished the last 50% of the formatting in just one day.
Furthermore, I expected precisely this behavior. Why?
This weirdness occurs because of the typical Microsoft phenomenon that was best expressed by an acquaintance of mine that works in Windows Mobile. He thought that the biggest difference between Microsoft's and Apple's approaches to development are these:
Microsoft attacks problems horizontally: it builds core system, then builds layers on top of it. When it's time to ship and something needs to get cut, it's the top of the stack that goes first - and more often than not it's the user experience.
Apple develops software vertically - it enables a user experience, from top to bottom. When they need to cut, they cut the entire experiences (and also maybe the bugs, judging from the fact that my iPhone seems to crash considerably more often than most of Windows Mobile phones I owned in the past). So for example, iPhone would ship with Bluetooth support just for the mono phone headset, but no stereo profile. But the experiences that are left are implemented completely, to the maximum possible level of usability.
Back to our RAID problem. The reason the formatting is so slow in the very beginning is because two things happen at the same time: RAID resync and the formatting. Resync gets kicked off immediately after the RAID system is built, and it simply ensures that the parity volume (in the case of RAID-5) has correct checksums, or the mirrored volume (in the case of RAID-1) has a correct copy of the primary volume.
The other thing that happens once you create RAID volume from the UI is, of course, formatting. One can select quick format which only creates a file system. However, most people probably prefer to run full format when a new drive is added, just to make sure that it is not full of bad sectors.
So we have two write operations that are going on - first one is creating the redundant information, the other is filling the disk with zeroes.
Writing to the disk is very fast these days, but the seek time has barely improved in the last decade, and because the two writes happen in different places on the disk, the whole thing is completely dominated by the disk head moving from one track to another. And with a seek time of 15ms you can only have ~70 of these per second.
Of course, if the format is in progress, there is absolutely no sense to do a resync at the same time - whatever redundancy data resync creates is going to be instantly overwritten by the format.
But there was no Steve Jobs standing over the devs' shoulder, and getting the (filesystem) format in sync with (block device) RAID required two different teams to do something together so it probably got punted. I am sure it's in in a readme somewhere... and now in at least one blog :-).
So to avoid unnecesary delays, select quick format option when adding the RAID volume, wait for the resync to finish, THEN format it again with the full format.