cluster/afr: Prevent split-brain when bricks are brought off and on in cyclic order

        Backport of: http://review.gluster.org/15080

When the bricks are brought offline and then online in cyclic
order while writes are in progress on a file, thanks to inode
refresh in write txns, AFR will mostly fail the write attempt
when the only good copy is offline. However, there is still a
remote possibility that the file will run into split-brain if
the brick that has the lone good copy goes offline *after* the
inode refresh but *before* the write txn completes (I call it
in-flight split-brain in the patch for ease of reference),
requiring intervention from admin to resolve the split-brain
before the IO can resume normally on the file. To get around this,
the patch does the following things:
i) retains the dirty xattrs on the file
ii) avoids marking the last of the good copies as bad (or accused)
    in case it is the one to go down during the course of a write.
iii) fails that particular write with the appropriate errno.

This way, we still have one good copy left despite the split-brain situation
which when it is back online, will be chosen as source to do the heal.

> Change-Id: I9ca634b026ac830b172bac076437cc3bf1ae7d8a
> BUG: 1363721
> Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
> Reviewed-on: http://review.gluster.org/15080
> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
> Smoke: Gluster Build System <jenkins@build.gluster.org>
> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
> Reviewed-by: Ravishankar N <ravishankar@redhat.com>
> Reviewed-by: Oleksandr Natalenko <oleksandr@natalenko.name>
> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
(cherry picked from commit fcb5b70b1099d0379b40c81f35750df8bb9545a5)

Change-Id: I157f1025aebd6624fa3d412abc69a4ae6f2fe9e0
BUG: 1367272
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Signed-off-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Reviewed-on: http://review.gluster.org/15221
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>

2 files changed, 35 insertions, 0 deletions

diff --git a/libglusterfs/src/common-utils.c b/libglusterfs/src/common-utils.c
index 9a5f90b02f1..b62e69cf102 100644
--- a/libglusterfs/src/common-utils.c
+++ b/libglusterfs/src/common-utils.c
@@ -4494,3 +4494,27 @@ gf_zero_fill_stat (struct iatt *buf)
         buf->ia_nlink = 0;
         buf->ia_ctime = 0;
 }
+
+int
+gf_bits_count (uint64_t n)
+{
+        int val = 0;
+#ifdef _GNU_SOURCE
+        val = __builtin_popcountll (n);
+#else
+        n -= (n >> 1) & 0x5555555555555555ULL;
+        n = ((n >> 2) & 0x3333333333333333ULL) + (n & 0x3333333333333333ULL);
+        n = (n + (n >> 4)) & 0x0F0F0F0F0F0F0F0FULL;
+        n += n >> 8;
+        n += n >> 16;
+        n += n >> 32;
+        val = n & 0xFF;
+#endif
+        return val;
+}
+
+int
+gf_bits_index (uint64_t n)
+{
+    return ffsll(n) - 1;
+}
diff --git a/libglusterfs/src/common-utils.h b/libglusterfs/src/common-utils.h
index f1c26a2d0c5..93dee58b079 100644
--- a/libglusterfs/src/common-utils.h
+++ b/libglusterfs/src/common-utils.h
@@ -25,6 +25,10 @@
 #include <limits.h>
 #include <fnmatch.h>
 
+#ifndef ffsll
+#define ffsll(x) __builtin_ffsll(x)
+#endif
+
 void trap (void);
 
 #define GF_UNIVERSAL_ANSWER 42    /* :O */
@@ -835,4 +839,11 @@ is_virtual_xattr (const char *k);
 
 const char *
 gf_inode_type_to_str (ia_type_t type);
+
+int32_t
+gf_bits_count (uint64_t n);
+
+int32_t
+gf_bits_index (uint64_t n);
+
 #endif /* _COMMON_UTILS_H */