glusterfs.git/xlators/cluster/dht/src/dht-selfheal.c, branch v3.5.7

gNFS: Incorrect NFS ACL encoding for XFS

2013-09-29T23:54:37+00:00

Problem:
Incorrect NFS ACL encoding causes "system.posix_acl_default"
setxattr failure on bricks on XFS file system. XFS (potentially
others?) doesn't understand when the 0x10 prefix is added to the
ACL type field for default ACLs (which the Linux NFS client adds)
which causes setfacl()->setxattr() to fail silently. NFS client
adds NFS_ACL_DEFAULT(0x1000) for default ACL.

FIX:
Mask the prefix (added by NFS client) OFF, so the setfacl is not
rejected when it hits the FS.

Original patch by: "Richard Wareing"

Change-Id: I17ad27d84f030cdea8396eb667ee031f0d41b396
BUG: 1009210
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/5980
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/dht: assign layout onto missing directories too

2013-09-09T21:58:09+00:00

The current self-healing algorithm is ignoring missing directories
for assigning new layout. When lookup() is racing against mkdir()
or when self-healing a half-done mkdir(), the layout assignment split
must happen based on the final number of directories, and not the
currently existing number of directories (because we finish mkdir()
of missing directories before hash layout assignment).

Without this fix, concurrent mkdir() and lookup() will step on
each others feet, create a messed up layout on disk, and end up
with different in-memory layouts.

Once two clients have different in-memory layouts, creation of
subdirectory will not arbitrate on the same hashed subvolume and will
result in GFID mismatch of the sub-directory.

Change-Id: Ia47acad67c265060405984c822b4d37512b9dbb3
BUG: 907072
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/5849
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Peter Portante 
Tested-by: Peter Portante

cluster/dht: Ignore decommissioned subvol in overlap optimization

2013-05-25T01:06:26+00:00

Change-Id: Ib727948c6e21b19fd509f258ff0aea1c5d1a84d1
BUG: 966845
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/5056
Reviewed-by: Amar Tumballi 
Reviewed-by: Jeff Darcy 
Tested-by: Gluster Build System

cluster/distribute: Ignore non-participating subvols for layout checks

2013-04-09T19:49:40+00:00

When subvols-per-directory is < available subvols, then there are layouts
which are not populated. This leads to incorrect identification of holes or
overlaps. We need to ignore layouts, which have err == 0, and start == stop.
In the current scenario (start == stop == 0).

Additionally, in layout-merge, treat missing xattrs as err = 0. In case of
missing layouts, anomalies will reset them.

For any other valid subvoles, err != 0 in case of layouts being zeroed out.

Also reverted back dht_selfheal_dir_xattr, which does layout calculation only
on subvols which have errors.

Change-Id: I9f57062722c9e8a26285e10675c31a78921115a1
BUG: 921408
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4668
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Jeff Darcy

dht: make DHT xattr names configurable

2013-03-21T22:04:31+00:00

This is necessary to support "DHT over DHT" configurations, so that the
upper and lower instances of DHT don't step all over each other.  Why
would we even consider such a thing?  Because it gives us the ability to
do data tiering and rack-aware placement, either by themselves or as
complements to other functionality such as erasure codes or
deduplication which save space but cost performance.  By setting up the
top-level DHT to place data into one of several lower-level DHT pools
based on policy instead of pure elastic hashing, we get better
performance for 90% of accesses and better storage efficiency for 90% of
data, all for relatively low effort.

Change-Id: I72e65c29edfc80babf39f7a2a00090f4588c4070
BUG: 924265
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/4694
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

dht: fix a typo

2013-03-18T18:34:24+00:00

Change-Id: Id6f156957e58aad06bf2602f880c7e4102b80fd1
BUG: 764890
Signed-off-by: JulesWang 
Reviewed-on: http://review.gluster.org/4679
Reviewed-by: Jeff Darcy 
Tested-by: Gluster Build System

cluster/distribute: Fix layout overlaps due to spread-count in selfheal path

2013-03-08T03:51:22+00:00

We needed to zero out the layout range, before we re-calculate the range.
When spread-count is issued, we would end up with stale ranges in the layout.

Replaced dht_selfheal_dir_xattr with dht_fix_dir_xattr, which correctly resets
the un-used (after re-cal) layouts.

Change-Id: I1a900d15df07335f59356bd23182ccec34381ab2
BUG: 884455
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4647
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

distribute: add hash-name-regex option

2013-02-19T04:50:48+00:00

This is to support the common "write to temp file then rename" idiom. In our
case this causes us to create a linkfile during the rename in (N-1)/N cases,
with a significant impact on performance e.g. for UFO which uses this idiom.
This patch allows the user to specify up to two regular expressions that
separate the permanent and transient parts of a temp-file name:

    rsync-hash-regex reimplements the existing RSYNC_FRIENDLY_NAME
    pattern where the temporary name for EXAMPLE is .EXAMPLE.suffix

    extra-hash-regex can be used for a second pattern, active concurrently,
    and can include alternatives via the POSIX extended-regex syntax

Change-Id: Ic1a6ed19324bc31fefe91ee34b8478877a9c5d2c
BUG: 912564
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/4116
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

dht: better layout-optimization algorithm

2013-02-07T16:27:40+00:00

This method deals with the case where swapping might gain a bigger overlap
for the xlator currently under consideration, but sacrifices even more from
the xlator we're swapping with. For example:

A = 0x00000000 - 0x44444443 (new 0x00000000 - 0x55555554)
B = 0x44444444 - 0x77777776 (new 0x55555555 - 0xaaaaaaa9)
C = 0x77777777 - 0xffffffff (new 0xaaaaaaaa - 0xffffffff)

Here, the new range for B has a bigger overlap with the old C than with the
old B (0x33333333 vs. 0x22222222 to be precise) so looking only at that
might lead us to swap. However, such a swap turns the new C's overlap from
0x55555556 (vs. old C) to *zero* (vs. old B).  In other words, we've gained
0x11111111 for B but lost 0x55555556 for C, so it's a bad idea.

The new algorithm accounts for all effects of the swap, so it not only avoids
bad swaps but can make some good ones that would have been missed previously.
For example, if swapping a range X with a later range Y would not increase the
overlap for X we would previously have skipped it even if the swap would
increase Y's overlap without affecting X's.  This is the normal case when we're
adding a new brick (which initially has zero overlap with any old range) so
finding more good swaps is probably even more important than avoiding bad ones.

Also, the logic in dht_overlap_calc was completely broken before, causing
integer overflows instead of providing correct values, so no matter what
higher-level algorithm was in place the GIGO effect would have resulted in
bad decisions.

Change-Id: If61ed513cfcb931916c6b51da293e3efbaaf385f
BUG: 853258
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/3908
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/distribute: get_layout should account only available subvols

2013-01-24T07:43:39+00:00

The earlier logic used to check if (layout-spread-count <= subvol_cnt -
decommissioned bricks). With this if a subvol was down, and layout-spread was >
upsubvols, a mkdir ended up creating holes in the layout.

The fix is to consider only the combination of subvols which are usable (not
down or not decommissioned).

Change-Id: I61ad3bcaf4589f5a75f7887cfa595c98311ae3bb
BUG: 902610
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4412
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati