posix/ctime: Fix ctime upgrade issue

Problem:
On a EC volume, during upgrade from the older version where
ctime feature is not enabled(or not present) to the newer
version where the ctime feature is available (enabled default),
the self heal hangs and doesn't complete.

Cause:
The ctime feature has both client side code (utime) and
server side code (posix). The feature is driven from client.
Only if the client side sets the time in the frame, should
the server side sets the time attributes in xattr. But posix
setattr/fseattr was not doing that. When one of the server
nodes is updated, since ctime is enabled by default, it
starts setting xattr on setattr/fseattr on the updated node/brick.

On a EC volume the first two updated nodes(bricks) are not a
problem because there are 4 other bricks with consistent data.
However once the third brick is updated, the new attribute(mdata xattr)
will cause an inconsistency on metadata on 3 bricks, which
prevents the file to be repaired.

Fix:
Don't create mdata xattr with utimes/utimensat system call.
Only update if already present.

Change-Id: Ieacedecb8a738bb437283ef3e0f042fd49dc4c8c
fixes: bz#1720201
Signed-off-by: Kotresh HR <khiremat@redhat.com>

1 files changed, 21 insertions, 0 deletions

diff --git a/xlators/storage/posix/src/posix-metadata.c b/xlators/storage/posix/src/posix-metadata.c
index e96f222cff4..5a5e6cd744e 100644
--- a/xlators/storage/posix/src/posix-metadata.c
+++ b/xlators/storage/posix/src/posix-metadata.c
@@ -416,6 +416,22 @@ posix_set_mdata_xattr(xlator_t *this, const char *real_path, int fd,
                  * still fine as the times would get eventually
                  * accurate.
                  */
+
+                /* Don't create xattr with utimes/utimensat, only update if
+                 * present. This otherwise causes issues during inservice
+                 * upgrade. It causes inconsistent xattr values with in replica
+                 * set. The scenario happens during upgrade where clients are
+                 * older versions (without the ctime feature) and the server is
+                 * upgraded to the new version (with the ctime feature which
+                 * is enabled by default).
+                 */
+
+                if (update_utime) {
+                    UNLOCK(&inode->lock);
+                    GF_FREE(mdata);
+                    return 0;
+                }
+
                 mdata->version = 1;
                 mdata->flags = 0;
                 mdata->ctime.tv_sec = time->tv_sec;
@@ -527,6 +543,11 @@ posix_update_utime_in_mdata(xlator_t *this, const char *real_path, int fd,
 
     priv = this->private;
 
+    /* NOTE:
+     * This routine (utimes) is intentionally allowed for all internal and
+     * external clients even if ctime is not set. This is because AFR and
+     * WORM uses time attributes for it's internal operations
+     */
     if (inode && priv->ctime) {
         if ((valid & GF_SET_ATTR_ATIME) == GF_SET_ATTR_ATIME) {
             tv.tv_sec = stbuf->ia_atime;