dht: tackle thread race in dht_getxattr_cbk

problem:
 1. When two threads execute in parallel in dht_getxattr_cbk
it may so happen that, both may find local->xattr to be NULL. As
a result dht_aggregate_xattr may not get executed.

 2. In dht_getxattr_cbk,

             thread1                         thread2
T1      this_call_cnt = 2 -1
T2                                this_call_cnt = 1 - 1
T3                                fills local_xattr
T4                                DHT_STACK_UNWIND -> local_wipe
T5      tries to dereference local
        which is already freed,
        leading to crash.

Solution:
 for problem1: Execute critical section inside frame lock
to resolve race.

 for problem2: Calculate this_call_count just before out section.

Change-Id: I9827ac8fafebb0c733a4e4f3c710b752f1cd45fa
BUG: 1215592
Signed-off-by: Susant Palai <spalai@redhat.com>
Reviewed-on: http://review.gluster.org/10389
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Reviewed-by: N Balachandran <nbalacha@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Tested-by: NetBSD Build System
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>

1 files changed, 24 insertions, 17 deletions

diff --git a/xlators/cluster/dht/src/dht-common.c b/xlators/cluster/dht/src/dht-common.c
index 31270c8b8bc..12a283bd991 100644
--- a/xlators/cluster/dht/src/dht-common.c
+++ b/xlators/cluster/dht/src/dht-common.c
@@ -2767,29 +2767,36 @@ dht_getxattr_cbk (call_frame_t *frame, void *cookie, xlator_t *this,
         conf = this->private;
         local = frame->local;
 
-        this_call_cnt = dht_frame_return (frame);
+        LOCK (&frame->lock);
+        {
+                if (!xattr || (op_ret == -1)) {
+                        local->op_ret = op_ret;
+                        goto unlock;
+                }
 
-        if (!xattr || (op_ret == -1)) {
-                local->op_ret = op_ret;
-                goto out;
-        }
+                if (dict_get (xattr, conf->xattr_name)) {
+                        dict_del (xattr, conf->xattr_name);
+                }
 
-        if (dict_get (xattr, conf->xattr_name)) {
-                dict_del (xattr, conf->xattr_name);
-        }
+                if (frame->root->pid >= 0) {
+                        GF_REMOVE_INTERNAL_XATTR
+                                ("trusted.glusterfs.quota*", xattr);
+                        GF_REMOVE_INTERNAL_XATTR("trusted.pgfid*", xattr);
+                }
 
-        if (frame->root->pid >= 0 ) {
-                GF_REMOVE_INTERNAL_XATTR("trusted.glusterfs.quota*", xattr);
-                GF_REMOVE_INTERNAL_XATTR("trusted.pgfid*", xattr);
-        }
+                local->op_ret = 0;
 
-        local->op_ret = 0;
+                if (!local->xattr) {
+                        local->xattr = dict_copy_with_ref (xattr, NULL);
+                } else {
+                        dht_aggregate_xattr (local->xattr, xattr);
+                }
 
-        if (!local->xattr) {
-                local->xattr = dict_copy_with_ref (xattr, NULL);
-        } else {
-                dht_aggregate_xattr (local->xattr, xattr);
         }
+unlock:
+        UNLOCK (&frame->lock);
+
+        this_call_cnt = dht_frame_return (frame);
 out:
         if (is_last_call (this_call_cnt)) {
                 DHT_STACK_UNWIND (getxattr, frame, local->op_ret, op_errno,