[wplug] RAID-5/6 Considerations -- WAS: Thoughts & Considerations for email server

[Bryan J. Smith]

On Sun, 2007-08-19 at 08:02 -0400, Bryan J. Smith wrote:
> First off, I mirror everyone's statements that RAID-5 is _deadly_ for
> mail spooling.  Your /var/spool/mail (like anything under /var) should
> be on a RAID-10 (or at least RAID-1) volume.  RAID-5 is great for lots
> of little reads (and it acts almost like RAID-0 in that regard), but
> lots of little writes kill it (like most anything under /var).  99% of
> the benchmarks you see on RAID-5 are reads, _ignore_ their marketing.
>  ...  
> And because you can_not_ "guarantee" hot-swap with the Linux kernel (or
> any OS for that matter) right now (only select, proven SCSI -- not with
> any ATA with a SCSI driver, and _not_ with software RAID, I know people
> will argue with me on this, but trust me, I've dealt with _thousands_ of
> these "cases" ;), you should have true hardware RAID controller.  A true
> hardware RAID controller has on-board intelligence the kernel driver
> talks to, and _removes_ the direct access to the drives, so the "volume"
> is only presented, so hot-swap is "fool proof."  I recommend 3Ware/AMCC
> products.  Looks like you have an older HP DL/ML series (G4) with PCI-X,
> so consider a 3Ware/AMCC PCI-X product, like the 9550.

To throw a tangent discussion ...

A lot of people make a big deal about RAID-5 "efficiency" in disk usage.
In reality, you do _not_ need to choose RAID-5 for _all_ disks.  E.g.,
with 8 channels ...

- Use two (2) or four (4) disks for RAID-1 or 10 as "write-heavy"
- Use four (4) to six (6) disks for RAID-5 (or 6) as "read-centric"

The "write-heavy" disk should be for things like swap, /tmp, /var and
anything else with lots of writes that performs poorly on RAID-5.

The "read-centric" disk should be for things like user data, select,
largely read-only data, binaries, etc... that is either largely
read-only or has infrequent writes.

I personally use this strategy with disk controllers of eight (8) or
more channels.

As far as hardware RAID considerations, I still stand by 3Ware/AMCC
becuase:  
- AMCC is the PPC400 fabless semiconductor IP owner/designer
- 3Ware has a proven, solid Linux track record**
- 3Ware's volumes are upward compatible with newer products
- Newer DeviceMapper implementations can read 3Ware volumes
  (i.e., Linux software RAID can read the 3Ware hardware volumes)

[ **NOTE:  The only "negative" commentary I've seen on 3Ware has been
from people who don't use its hardware RAID, but _incorrectly_ assume
that software RAID can use 3Ware cards in JBOD mode and get hot-swap.
You can't, and it utterly ignores the real support details of
hot-swap. ]

I have used Areca products in the past as well, but I don't stand by
them as much anymore because ...
- Linux drivers are stock Intel, and tools are still lacking
- Intel considers the IOP33x/34x X-Scale products end-of-life**

[ **NOTE:  It's clear Intel is phasing out X-Scale and, likely, going
with an x86-based + added I/O Processor Engines (IPE) for its next-gen
product.  I don't know how that is going to affect upward volume
compatibility, but Intel has changed the format several times on its ICH
and IOP units in his history (much to my dislike). ]

Just some considerations.


-- 
Bryan J. Smith         Professional, Technical Annoyance
mailto:b.j.smith at ieee.org   http://thebs413.blogspot.com
--------------------------------------------------------
        Fission Power:  An Inconvenient Solution