glusterfs.git/xlators/cluster/dht/src, branch v3.13.0beta

xlator/tier: flood of -Wformat-truncation warnings with gcc-7.1

2017-11-01T13:31:13+00:00

Starting in Fedora 26 which has gcc-7.1.x, -Wformat-trunction is enabled
with -Wformat, resulting in a flood of new warnings. This many warnings
is a concern because it makes it hard(er) to see other warnings that
should be addressed.

An example is at
https://kojipkgs.fedoraproject.org//packages/glusterfs/3.12.0/1.fc28/data/logs/x86_64/build.log

For more info see https://review.gluster.org/#/c/18267/

Change-Id: Id7ef8e0dedd28ada55f72c03d91facbe1c9888bd
BUG: 1492849
Signed-off-by: Kaleb S. KEITHLEY

cluster/dht: fix crash when deleting directories

2017-10-16T10:33:13+00:00

In DHT, after locks on all subvolumes are acquired, it would perform the
following steps sequentially,
1. send remove dir on all other subvolumes except the hashed one in a loop;
2. wait for all pending rmdir to be done
3. remove dir on the hashed subvolume

The problem is that in step 1 there is a check to skip hashed subvolume
in the loop. If the last subvolume to check is actually the
hashed one, and step 3 is quickly done before the last and hashed
subvolume is checked, by accessing shared context data be destroyed in
step 3, would cause a crash.

Fix by saving shared data in a local variable to access later in the
loop.

Change-Id: I8db7cf7cb262d74efcb58eb00f02ea37df4be4e2
BUG: 1490642
Signed-off-by: Zhang Huan

dht: free memory allocated in dht_init() and dht_init_subvolumes()

2017-10-11T03:59:23+00:00

When glfs_fini() is called, DHT fails to free all memory allocations
which result in a considerable leak.

Change-Id: I37c6de5c93ca4516266dbe8288b4a416f5589901
BUG: 1443145
Signed-off-by: Niels de Vos

cluster/dht: Don't store the entire uuid for subvols

2017-10-10T08:58:39+00:00

Comparing the uuid string of the local node against that stored in the
local_subvol information is inefficient, especially as it is
done for every file to be migrated. The code has now been changed
to set the value of info to 1 if the nodeuuid is that of the node
making the comparison so this becomes an integer comparison.

Change-Id: I7491d59caad3b71dbf5facc94dcde0cd53962775
BUG: 1451434
Signed-off-by: N Balachandran

cluster/dht : User xattrs are not healed after brick stop/start

2017-10-04T09:55:35+00:00

Problem: In a distributed volume custom extended attribute value for a directory
         does not display correct value after stop/start or added newly brick.
         If any extended(acl) attribute value is set for a directory after stop/added
         the brick the attribute(user|acl|quota) value is not updated on brick
         after start the brick.

Solution: First store hashed subvol or subvol(has internal xattr) on inode ctx and
          consider it as a MDS subvol.At the time of update custom xattr
          (user,quota,acl, selinux) on directory first check the mds from
          inode ctx, if mds is not present on inode ctx then throw EINVAL error
          to application otherwise set xattr on MDS subvol with internal xattr
          value of -1 and then try to update the attribute on other non MDS
          volumes also.If mds subvol is down in that case throw an
          error "Transport endpoint is not connected". In dht_dir_lookup_cbk|
          dht_revalidate_cbk|dht_discover_complete call dht_call_dir_xattr_heal
          to heal custom extended attribute.
          In case of gnfs server if hashed subvol has not found based on
          loc then wind a call on all subvol to update xattr.

Fix:    1) Save MDS subvol on inode ctx
        2) Check if mds subvol is present on inode ctx
        3) If mds subvol is down then call unwind with error ENOTCONN and if it is up
           then set new xattr "GF_DHT_XATTR_MDS" to -1 and wind a call on other
           subvol.
        4) If setxattr fop is successful on non-mds subvol then increment the value of
           internal xattr to +1
        5) At the time of directory_lookup check the value of new xattr GF_DHT_XATTR_MDS
        6) If value is not 0 in dht_lookup_dir_cbk(other cbk) functions then call heal
           function to heal user xattr
        7) syncop_setxattr on hashed_subvol to reset the value of xattr to 0
           if heal is successful on all subvol.

Test : To reproduce the issue followed below steps
       1) Create a distributed volume and create mount point
       2) Create some directory from mount point mkdir tmp{1..5}
       3) Kill any one brick from the volume
       4) Set extended attribute from mount point on directory
          setfattr -n user.foo -v "abc" ./tmp{1..5}
          It will throw error " Transport End point is not connected "
          for those hashed subvol is down
       5) Start volume with force option to start brick process
       6) Execute getfattr command on mount point for directory
       7) Check extended attribute on brick
          getfattr -n user.foo /tmp{1..5}
          It shows correct value for directories for those
          xattr fop were executed successfully.

Note: The patch will resolve xattr healing problem only for fuse mount
      not for nfs mount.

BUG: 1371806
Signed-off-by: Mohit Agrawal 

Change-Id: I4eb137eace24a8cb796712b742f1d177a65343d5

dht: fix a coverity error of type - UNREACHABLE

2017-09-28T17:56:45+00:00

Problem: Unreachable assignment statement at dht-rebalance.c:1040

Fix: Delete line dht-rebalance.c:1040.

The goto statements at lines 1037 and 1031 are also deleted since
both branches of the if statement finally go to the same
immediately-following label anyway.

Change-Id: I5f47ea99244cae2a0a9f2aec7284faadf2ea286a
BUG: 789278
Signed-off-by: Kamal Mohanan

Fix a coverity error of checker type: CHECKED_RETURN

2017-09-26T09:48:31+00:00

Problem: dht_frame_return was being called without checking the
return value.

Solution: Typecast the value returned by the function to void.

Change-Id: Idfc6a7ed467d1c8f5f8d09ec26d9059f3d23b760
BUG: 789278
Signed-off-by: Kamal Mohanan

dht: add FOP check to dht_file_setattr_cbk

2017-09-07T02:17:04+00:00

Problem:
bug-797171.7 loaded error-gen xlator on the brick which sent EBADF for a
non fd-based fop, namely setattr. This caused
dht_check_and_open_fd_on_subvol_task() to crash as local->fd was NULL.

Fix:
Call dht_check_and_open_fd_on_subvol_task() from dht_file_setattr_cbk
only for dht_fsetattr and not dht_setattr or dht_setattr2

Signed-off-by: Ravishankar N 
Change-Id: Iab4999e213bf2065804f3f8237e470ad454e3c99
BUG: 1488399
Reviewed-on: https://review.gluster.org/18208
Smoke: Gluster Build System 
Reviewed-by: Susant Palai 
Reviewed-by: Amar Tumballi 
Reviewed-by: Raghavendra G 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System

cluster/dht: Log files skipped by rebalance

2017-08-30T05:56:12+00:00

There was no easy way to find out which files were
skipped during a rebalance.
Rebalance now logs a message for every skipped file
using msgid 109126, making it easier to find
all files that were skipped.

Change-Id: I2cac7db7285e2f82354251f3ea4094827b0daf3e
BUG: 1480445
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18021
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: hari gowtham 
Reviewed-by: Raghavendra G

cluster/dht: Aggregate xattrs only for dirs in dht_discover_cbk

2017-08-30T04:48:57+00:00

If dht_discover finds data files on more than one subvol,
racing calls to dht_discover_cbk could end up calling
dht_aggregate_xattr which could delete dictionary data
that is being accessed by higher layer translators.
Fixed to call dht_aggregate_xattr only for directories and
consider only the first file to be found.

Change-Id: I4f3d2a405ec735d4f1bb33a04b7255eb2d179f8a
BUG: 1484709
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18137
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G