[wplug] RAID performance

[Tom Rhodes]

On Fri, 02 Mar 2007 10:35:36 -0500
Patrick Wagstrom <pwagstro at andrew.cmu.edu> wrote:

> I'm in the process of speccing out a new machine for a research group 
> here at CMU.  Needs to be fairly beefy, handle about 1TB of MySQL 
> databases, plus room for other computations to take place.  Anyway, 
> we'll put aside all the issues of MySQL and their choice of software, 
> and instead focus on an interesting issue I noticed last night, and I'm 
> looking for some help with it, or someone to double check it.
> 
> We've got a machine right now with 4x400GB Hitachi SATA I drives (model 
> HDS724040KLSA80S) connected to a 3ware Escalade 9500S 4 port SATA RAID 
> controller (128MB of ram on the controller).  These drives are 
> structured in RAID 5.  And on the other hand, at home I've got 4x320GB 
> Seagate SATA II (model STS3320620AS) running software raid 5 on a MSI 
> K8N-Neo4 Platinum (8x SATA ports).  I went software RAID at home because 
> of cost and because performance isn't overly critical for the home 
> machine because it's just HDTV, which only needs about 2MB/s write speed 
> max.
> 
> Anyway, I did some admittedly synthetic benchmarks to compare 
> performance on the system because I wanted to get an idea how much the 
> hardware RAID made a difference, or if we'd be better off getting an 
> additional 2 or 3 500GB drives for the cost of the hardware RAID.  As a 
> final point of comparison, I included my IBM T43p laptop which as a SATA 
> harddrive inside and no RAID.  For read the speeds are averaged over 10 
> runs of hdparm.  For writes the speed is averaged over 3 consecutive 
> runs.  Here's what I found:
> 
> using hdparm -tT to get an idea of read speed:
> 
> 9500S Hardware: 52MB/s
> K8N Software: 178MB/s
> T43p No Raid: 40MB/s
> 
> Then I decided to find some large files and copy them from one location 
> on the drive to another.  This was the best I could do because I didn't 
> want external drives to be the bottleneck and the hardware raid machine 
> has all disks as part of the array:
> 
> 9500S Hardware (1.5G file): 24MB/s
> K8N Software (1.1G file): 42MB/s
> T43p No Raid (2.3G file): 11.5MB/s
> 
> So, as should be expected the laptop lags behind on just about 
> everything.  However, what I was surprised to find was that software 
> RAID 5 destroyed the hardware RAID 5 in terms of speed.  This leads me 
> to wonder about a few things and I'd like to get other peoples feedback.
> 
> Does anyone else have a 3ware 9500s running RAID 5 that they could 
> provide some useful benchmarks from?  Are there tuning parameters I 
> should enact on the 9500s to increase performance?  Should the switch 
> between SATAI and SATAII drives really make that much of difference?  My 
> impression is that while SATA II drives theoretically supported 3Gbps 
> they really come nowhere close -- which is what my results show. 
> However, going 3x as fast for reads and 2x as fast for writes on 
> software RAID was quite surprising.

In my testing, software RAID always beats hardware RAID in most
environments.  The amount of IO needed by the OS is mimimal
compared to the cost of passing control via a hardware interface
using ioctls, etc..

Unfortunately, I'm at home right now so I don't have the figures
from our testing, but we tested specifically Adeptec and Promise
in both a SCSI and SATA configurations.

Really neat: The other day, a several TB sata storage device was
on my desk when I arrived at the office.  We're going with the
software RAID solution on this guy, along with RedHat's GFS
software (which I still think isn't production-ready), so it
should be interesting.

-- 
Tom Rhodes