GFS2: Set of distributed preferences for rgrps
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 1/1 GFS2: Set of distributed preferences for rgrps
Hi,
This patch uses journal numbers to evenly distribute which node prefers which resource groups for block allocations. This is to help performance. The idea is to make each node in the cluster prefer to use a certain subset of resource groups for its block allocations. Other nodes use a different subset, thus minimizing inter-node DLM communications.
GFS1 has a similar scheme, but with GFS1, each node starts out using an initial section of the file system and works their way forward. While this works for the most part, simultaneous writes by different nodes can cause the heads to bounce on some devices. Also, nodes tend to stray from their initial locations. With this patch, the preferred resource groups are assigned in a round-robin fashion.
Tests prove that this patch can be a major performance boost for some applications. For example, using one particular application, I posted these run times before applying this patch:
Run 1 time: 2hr 49min 39sec
Run 2 time: 2hr 57min 59sec
Run 3 time: 3hr 1min 10sec
With the patch applied the times are improved by almost a third:
Run 1 time: 2hr 7min 31sec
Run 2 time: 2hr 5min 49sec
Run 3 time: 2hr 5min 1sec
Run 4 time: 2hr 4min 35sec
Run 5 time: 2hr 5min 55sec
Regards,
Bob Peterson
Red Hat File Systems
---
fs/gfs2/incore.h +2 -0
@@ -97,6 +97,7 @@ struct gfs2_rgrpd {#define GFS2_RDF_CHECK 0x10000000/* check for unlinked inodes */ #define GFS2_RDF_UPTODATE 0x20000000/* rg is up to date */ #define GFS2_RDF_ERROR 0x40000000/* error in rg */ +#define GFS2_RDF_PREFERRED 0x80000000/* This rgrp is preferred */ #define GFS2_RDF_MASK 0xf0000000/* mask for internal flags */ spinlock_t rd_rsspin;/* protects reservation related vars */ struct rb_root rd_rstree;/* multi-block reservation tree */
@@ -809,6 +810,7 @@ struct gfs2_sbd {char sd_table_name[GFS2_FSNAME_LEN ];char sd_proto_name[GFS2_FSNAME_LEN ]; +int sd_nodes;/* Debugging crud */ unsigned long sd_last_warning;
fs/gfs2/lock_dlm.c +2 -0
@@ -1113,6 +1113,8 @@ static void gdlm_recover_done(void *arg, struct dlm_slot *slots, int num_slots,struct gfs2_sbd *sdp = arg;struct lm_lockstruct *ls = &sdp->sd_lockstruct; +BUG_ON (num_slots ==0 ); + sdp->sd_nodes = num_slots;/* ensure the ls jid arrays are large enough */ set_recover_size (sdp, slots, num_slots);
fs/gfs2/ops_fstype.c +1 -0
@@ -134,6 +134,7 @@ static struct gfs2_sbd *init_sbd(struct super_block *sb)atomic_set (&sdp->sd_log_freeze,0 );atomic_set (&sdp->sd_frozen_root,0 );init_waitqueue_head (&sdp->sd_frozen_root_wait); + sdp->sd_nodes =1 ;return sdp; }
fs/gfs2/rgrp.c +61 -5
96 lines not on this page · raw message