[wplug] Large Database

[Bill Moran]

In response to Michael Semcheski <mhsemcheski at gmail.com>:

> On Fri, Mar 6, 2009 at 11:12 AM, Bill Moran <wmoran at potentialtech.com> wrote:
> > SANs are notorious for crappy performance under database load.  Granted,
> > there are a few SAN systems that actually do well ... but there are far
> > more that crumble under the frequent random access of a DB.
> 
> Appreciate your input on this.  We're not looking at a SAN
> specifically for database performance, but some of the options I've
> looked at and been quoted are large flash-accelerator modules (e.g.
> Readzilla) that I would think would do very well for database access.

Will help a lot if your DB is primarily read-only.

There are lots of things to consider when buying storage.  One of the
big problems with a SAN is what happens to it when all the other servers
using it start accessing it as well.  and if this SAN is going to be
dedicated to just that server, why not spend less money for fast SCSI
storage.

Also, the cache will struggle if you've got a lot of data being written,
because the cache will still be limited by the speed that the SAN can
flush data to disk.  Of course, whether that's a problem depends on
the actual type of activity your database sees.

Anyone who promises that product X is the correct solution for all
problems, is obviously lying ;)

> As for blob performance, like I said that's based on a very small
> sample size and MSSQL... pulling 200MB blobs out ended up taking 30
> times longer than via filesystem.  Again, very small sample size.

200M per BLOB is large enough to justify PostgreSQL's large object
system.  For normal BLOB usage on Postgres, I wouldn't store anything
larger than about 10M, as the requirement to receive all 200M of data
in one shot is tough on the client app as well as the server.

-- 
Bill Moran
http://www.potentialtech.com
http://people.collaborativefusion.com/~wmoran/