[aklug] Re: Recommendations for DRBD + (GFS | Lustre)

From: Christopher E. Brown <cbrown@woods.net>
Date: Wed Jan 14 2009 - 20:14:04 AKST

On Wed, 14 Jan 2009, Jeremy Austin wrote:

> Looking for the wisdom of the crowd:
> I've been running DRBD on (small) clusters of servers for a few years. (DRBD
> is a distributed RAID block device, similar to md -- software raid, but with
> hard drives spread out over multiple servers rather than just one.)
> Until now I've been using the Primary/Secondary mode, where only one half of
> the DRBD can write to disk at any one time.
>
> I'd like to switch to a clustered filesystem, and I see that DRBD has
> support for GFS. Does anyone in the group have experience with GFS?
>
> Another filesystem alternative appears to be Lustre, which seems (at first
> glance) to provide very high performance, but be much more complicated.
>
> In general I don't need very high performance; I'm running a few web sites,
> file servers, and VMware machines, supporting users connected via DSL. I'm
> more interested in the high availability of DRBD (and that works very well),
> potentially coupled with the higher overall performance of a clustered
> filesystem.
>
> My goal is a 3-server cluster, with 3 arrays (one per 'pair' of servers.)
> Should I try DRBD+GFS, DRBD+Lustre, or stick with plain DRBD & ext3, as I've
> been doing?
>
> Thanks as always,
> Jeremy Austin
> IT Administrator
> Whitestone, Alaska

I would take a closer look if you are expecting higher performance. The
reducdancy gain comes at a cost, performance. A beefy cluster can improve
performance, but not as much as the same number of disks in a
non-redundant or non cluster config.

Async DRDB is pretty low cost performance wise. There is the additional
overhead of copying all the data to the second server, but the addition
write delays are hidden, and you are using capacity on the inactive server
to do the writes. Performance is pretty clost to that of a single system.

Sync DRDB in a master/master config is *SLOWER* than one server or the
async master/slave config. The writing node waits on the other node to
signal write complete before returning to the app. Using master/master
means using a multi-access safe filesystem. Many of these already have a
"cluster RAID" ability.

You can get higher performance (aggragate) out of a "network RAID" type
config, but it can be complex and the per client performance is usually
lower.

I seem to recall that dual async master/slave works. In this case you
would have say 4 drives, 2 per server. Each server runs as a master for
one and a slave for the other. Services are split across the 2 volumes.
This allows you to take advantage of the available CPU on each server and
split the read workload. Writes are the limiter though, each node must
write all data and there is DRDB overhead.

IIRC GFS allows safe multi access only, lustre is similar but does the
"network RAID" bit.

If you really want to play with it, GFS on top of master/master DRDB is
supposed to work but be touchy. With lustre, skip DRDB use the lustre
provided redundancy.

---------
To unsubscribe, send email to <aklug-request@aklug.org>
with 'unsubscribe' in the message body.
Received on Wed Jan 14 20:17:16 2009

This archive was generated by hypermail 2.1.8 : Wed Jan 14 2009 - 20:17:16 AKST