glusterd: Allow bumping down a peer's op-version during probev3.4.1qa1

  Backport of http://review.gluster.org/5715

Earlier, a peer running a higher op-version couldn't be probed into a
cluster running at a lower op-version. This created issues when trying
to expand an upgraded cluster. This patch changes this behaviour.

The cluster no longer rejects a peer being probed if its op-version is
higher than the cluster op-version. The peer will reduce its op-version
if it doesn't have any volumes. If the peer contains volumes and needs
to reduce its op-version, it fails the handshake and the probe fails.

BUG: 1005038
Change-Id: Iabe790a9f826a4ac63d379eeeba01efcfef01f4d
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5834
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>

2 files changed, 9 insertions, 28 deletions

diff --git a/doc/release-notes/3.4.0.md b/doc/release-notes/3.4.0.md
index 15aff92fadb..b906e6a94c0 100644
--- a/doc/release-notes/3.4.0.md
+++ b/doc/release-notes/3.4.0.md
@@ -165,16 +165,3 @@ More details can be found [here]
 
 - For Block Device translator based volumes open-behind translator at the
 client side needs to be disabled.
-
-- More information on this can be found in this mailing list
-  [post](http://www.gluster.org/pipermail/gluster-users/2013-July/036750.html)  
-  A freshly installed peer cannot be probed into a freshly upgraded cluster.
-  This is a limitation brought about by the introduction of op-versions.
-  There are two ways to workaround this issue,
-    * enable any new feature (for eg. open-behind) on a volume in the cluster,
-      and then perform the probe.  
-  Or, if you don't want to enable any new features  
-    * delete the 'operating-version' line from /var/lib/glusterd/glusterd.info
-      on the newly installed peer and restart glusterd on the peer.  
-      If the file doesn't exist (because of lazy uuid generation), try to probe
-      the peer from the cluster (which will fail) to get the file created.  
diff --git a/xlators/mgmt/glusterd/src/glusterd-handshake.c b/xlators/mgmt/glusterd/src/glusterd-handshake.c
index 9124c46ee21..faf05b4e272 100644
--- a/xlators/mgmt/glusterd/src/glusterd-handshake.c
+++ b/xlators/mgmt/glusterd/src/glusterd-handshake.c
@@ -408,12 +408,16 @@ gd_validate_cluster_op_version (xlator_t *this, int cluster_op_version,
                 goto out;
         }
 
-        if (cluster_op_version < conf->op_version) {
+        /* The peer can only reduce its op-version when it doesn't have any
+         * volumes. Reducing op-version when it already contains volumes can
+         * lead to inconsistencies in the cluster
+         */
+        if ((cluster_op_version < conf->op_version) &&
+            !list_empty (&conf->volumes)) {
                 gf_log (this->name, GF_LOG_ERROR,
-                        "operating version %d is less than the currently "
-                        "running version (%d) on the machine (as per peer "
-                        "request from %s)", cluster_op_version,
-                        conf->op_version, peerid);
+                        "cannot reduce operating version to %d from current "
+                        "version %d as volumes exist (as per peer request from "
+                        "%s)", cluster_op_version, conf->op_version, peerid);
                 goto out;
         }
 
@@ -734,16 +738,6 @@ gd_validate_peer_op_version (xlator_t *this, glusterd_peerinfo_t *peerinfo,
                 goto out;
         }
 
-        /* If peer is already operating at a higher op_version reject it.
-         * Cluster cannot be moved to higher op_version to accomodate a peer.
-         */
-        if (peer_op_version > conf->op_version) {
-                ret = gf_asprintf (errstr, "Peer %s is already at a higher "
-                                   "op-version", peerinfo->hostname);
-                ret = -1;
-                goto out;
-        }
-
         ret = 0;
 out:
         gf_log (this->name , GF_LOG_DEBUG, "Peer %s %s", peerinfo->hostname,