glusterfs.git/xlators/cluster/dht/src/dht-selfheal.c, branch v4.0.2-1

core: fix some of the dict_{get,set} with proper APIs

2018-01-17T00:00:11+00:00

updates #220

Change-Id: I6e25dbb69b2c7021e00073e8f025d212db7de0be
Signed-off-by: Amar Tumballi

cluster/dht: Serialize mds update code path with lookup unwind in selfheal

2017-11-22T17:18:47+00:00

Problem: Sometime test case ./tests/bugs/bug-1371806_1.t is failing on
         centos due to race condition between fresh lookup and setxattr fop.

Solution: In selfheal code path we do save mds on inode_ctx, it was not
          serialize with lookup unwind. Due to this behavior after lookup
          unwind if mds is not saved on inode_ctx and if any subsequent
          setxattr fop call it has failed with ENOENT because
          no mds has found on inode ctx.To resolve it save mds on
          inode ctx has been serialize with lookup unwind.

BUG: 1498966
Change-Id: I8d4bb40a6cbf0cec35d181ec0095cc7142b02e29
Signed-off-by: Mohit Agrawal

cluster/dht: Check for NULL local before using it

2017-11-08T02:13:37+00:00

Coverity ID: 261
Coverity ID: 257

Check if local is NULL or not

Change-Id: I570f87f32e1347f891c6a02ee5641fcc5a262a96
BUG: 789278
Signed-off-by: Ashish Pandey

cluster/dht : User xattrs are not healed after brick stop/start

2017-10-04T09:55:35+00:00

Problem: In a distributed volume custom extended attribute value for a directory
         does not display correct value after stop/start or added newly brick.
         If any extended(acl) attribute value is set for a directory after stop/added
         the brick the attribute(user|acl|quota) value is not updated on brick
         after start the brick.

Solution: First store hashed subvol or subvol(has internal xattr) on inode ctx and
          consider it as a MDS subvol.At the time of update custom xattr
          (user,quota,acl, selinux) on directory first check the mds from
          inode ctx, if mds is not present on inode ctx then throw EINVAL error
          to application otherwise set xattr on MDS subvol with internal xattr
          value of -1 and then try to update the attribute on other non MDS
          volumes also.If mds subvol is down in that case throw an
          error "Transport endpoint is not connected". In dht_dir_lookup_cbk|
          dht_revalidate_cbk|dht_discover_complete call dht_call_dir_xattr_heal
          to heal custom extended attribute.
          In case of gnfs server if hashed subvol has not found based on
          loc then wind a call on all subvol to update xattr.

Fix:    1) Save MDS subvol on inode ctx
        2) Check if mds subvol is present on inode ctx
        3) If mds subvol is down then call unwind with error ENOTCONN and if it is up
           then set new xattr "GF_DHT_XATTR_MDS" to -1 and wind a call on other
           subvol.
        4) If setxattr fop is successful on non-mds subvol then increment the value of
           internal xattr to +1
        5) At the time of directory_lookup check the value of new xattr GF_DHT_XATTR_MDS
        6) If value is not 0 in dht_lookup_dir_cbk(other cbk) functions then call heal
           function to heal user xattr
        7) syncop_setxattr on hashed_subvol to reset the value of xattr to 0
           if heal is successful on all subvol.

Test : To reproduce the issue followed below steps
       1) Create a distributed volume and create mount point
       2) Create some directory from mount point mkdir tmp{1..5}
       3) Kill any one brick from the volume
       4) Set extended attribute from mount point on directory
          setfattr -n user.foo -v "abc" ./tmp{1..5}
          It will throw error " Transport End point is not connected "
          for those hashed subvol is down
       5) Start volume with force option to start brick process
       6) Execute getfattr command on mount point for directory
       7) Check extended attribute on brick
          getfattr -n user.foo /tmp{1..5}
          It shows correct value for directories for those
          xattr fop were executed successfully.

Note: The patch will resolve xattr healing problem only for fuse mount
      not for nfs mount.

BUG: 1371806
Signed-off-by: Mohit Agrawal 

Change-Id: I4eb137eace24a8cb796712b742f1d177a65343d5

features/dht: Initialize local hashed_subvol

2017-05-25T04:21:04+00:00

Self heal directory code path doesn't always
have local->hashed_subvol populated. Populating
the same which otherwise would fail the self
heal.

Change-Id: I03b64709fd7a68e28f9e7438243e817c53c6ef5d
BUG: 1455104
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17381
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: N Balachandran 
Reviewed-by: Raghavendra G

cluster/dht: Fix crash in dht_selfheal_dir_setattr

2017-05-19T19:44:51+00:00

Use a local variable to store the call cnt used in the
for loop for the STACK_WIND so as not to access local
which may be freed by STACK_UNWIND after all fops return.

Change-Id: I24f49b6dbd29a2b706e388e2f6d5196c0f80afc5
BUG: 1452102
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17343
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Shyamsundar Ranganathan 
CentOS-regression: Gluster Build System

feature/dht: Directory synchronization

2017-04-26T09:00:34+00:00

Design doc: https://review.gluster.org/16876

Directory creation is now synchronized with blocking inodelk of the
parent on the hashed subvolume followed by the entrylk on the hashed
subvolume between dht_mkdir, dht_rmdir, dht_rename_dir and lookup
selfheal mkdir.

To maintain internal consistency of directories across all subvols of
dht, we need locks. Specifically we are interested in:

 1. Consistency of layout of a directory. Only one writer should modify
    the layout at a time. A writer (layout setting during directory heal
    as part of lookup) shouldn't modify the layout while there are
    readers (all other fops like create, mkdir etc., which consume
    layout) and readers shouldn't read the layout while a writer is in
    progress. Readers can read the layout simultaneously. Writer takes
    a WRITE inodelk on the directory (whose layout is being modified)
    across ALL subvols. Reader takes a READ inodelk on the directory
    (whose layout is being read) on ANY subvol.

 2. Consistency of directory namespace across subvols. The path and
    associated gfid should be same on all subvols. A gfid should not be
    associated with more than one path on any subvol. All fops that can
    change directory names (mkdir, rmdir, renamedir, directory creation
    phase in lookup-heal) takes an entrylk on hashed subvol of the
    directory.

 NOTE1: In point 2 above, since dht takes entrylk on hashed subvol of a
        directory, the transaction itself is a consumer of layout on
        parent directory. So, the transaction is a reader of parent
        layout and does an inodelk on parent directory just like any
        other layout reader. So a mkdir (dir/subdir) would:

     > Acquire a READ inodelk on "dir" on any subvol.
     > Acquire an entrylk (dir, "subdir") on hashed subvol of "subdir".
     > creates directory on hashed subvol and possibly on non-hashed subvols.
     > UNLOCK (entrylk)
     > UNLOCK (inodelk)

 NOTE2: mkdir fop while setting the layout of the directory being created
        is considered as a reader, but NOT a writer. The reason is for
        a fop which can consume the layout of a directory to come either
        of the following conditions has to be true:

     > mkdir syscall from application has to complete. In this case no
       need of synchronization.
     > A lookup issued on the directory racing with mkdir has to complete.
       Since layout setting by a lookup is considered as a writer, only
       one of either mkdir or lookup will set the layout.

Code re-organization:
   All the lock related routines are moved to "dht-lock.c" file.
   New wrapper function is introduced to take blocking inodelk
   followed by entrylk 'dht_protect_namespace'

Updates #191
Change-Id: I01569094dfbe1852de6f586475be79c1ba965a31
Signed-off-by: Kotresh HR 
BUG: 1443373
Reviewed-on: https://review.gluster.org/15472
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System

Remove double free for disk_layout

2017-02-27T23:02:30+00:00

Since disk_layout is freed after the jump to err, we
do not need to free it again before the goto.

Found by coverity scan.

Change-Id: Ie0c0262f6b95c51c61a59faefbca70352bf1e604
BUG: 789278
Signed-off-by: Michael Scherer 
Reviewed-on: https://review.gluster.org/16720
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Tested-by: Michael Scherer 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/dht: Use int8 instead of string to pass DHT_IATT_IN_XDATA_KEY

2017-02-13T06:48:17+00:00

It is sufficient to pass an int value as opposed to a "yes" against the
DHT_IATT_IN_XDATA_KEY key since all posix cares about is whether the
key is present in the dict or not. Also note that this patch does not
violate backward compatibility since the handling of the key in posix
remains untouched.

Change-Id: I2f881494a257488709c8c1d2002f2d124ddcc089
BUG: 1390050
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16591
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

dht/cluster: add logs to fix-layout code path

2017-01-20T10:49:57+00:00

Currently there is no helpful log in fix-layout code path. Adding
the logs to be helpful for debugging fix-layout failures.

BUG: 1414782
Change-Id: I61c29ceedcaa2e235fa7be99866709d6ca6de3ae
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/16040
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G