glusterfs.git/xlators/cluster/dht/src, branch v3.12.3

cluster/dht: fix crash when deleting directories

2017-10-25T11:36:23+00:00

In DHT, after locks on all subvolumes are acquired, it would perform the
following steps sequentially,
1. send remove dir on all other subvolumes except the hashed one in a loop;
2. wait for all pending rmdir to be done
3. remove dir on the hashed subvolume

The problem is that in step 1 there is a check to skip hashed subvolume
in the loop. If the last subvolume to check is actually the
hashed one, and step 3 is quickly done before the last and hashed
subvolume is checked, by accessing shared context data be destroyed in
step 3, would cause a crash.

Fix by saving shared data in a local variable to access later in the
loop.

> BUG: 1490642
> Signed-off-by: Zhang Huan 
(cherry picked from commit 206120126d455417a81a48ae473d49be337e9463)
Change-Id: I8db7cf7cb262d74efcb58eb00f02ea37df4be4e2
BUG: 1505221
Signed-off-by: N Balachandran

cluster/dht: Don't store the entire uuid for subvols

2017-10-12T18:25:29+00:00

Comparing the uuid string of the local node against that stored in the
local_subvol information is inefficient, especially as it is
done for every file to be migrated. The code has now been changed
to set the value of info to 1 if the nodeuuid is that of the node
making the comparison so this becomes an integer comparison.

> BUG: 1451434
> Signed-off-by: N Balachandran 
> https://review.gluster.org/#/c/17851

(cherry picked from commit c4a608799a577a4f38139f6bb8a47da8efb0fec3)
Change-Id: I7491d59caad3b71dbf5facc94dcde0cd53962775
BUG: 1500472
Signed-off-by: N Balachandran

dht: add FOP check to dht_file_setattr_cbk

2017-09-10T18:32:02+00:00

Problem:
bug-797171.7 loaded error-gen xlator on the brick which sent EBADF for a
non fd-based fop, namely setattr. This caused
dht_check_and_open_fd_on_subvol_task() to crash as local->fd was NULL.

Fix:
Call dht_check_and_open_fd_on_subvol_task() from dht_file_setattr_cbk
only for dht_fsetattr and not dht_setattr or dht_setattr2

> Reviewed-on: https://review.gluster.org/18208
> Smoke: Gluster Build System 
> Reviewed-by: Susant Palai 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra G 
> Reviewed-by: N Balachandran 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 47188e9eac59de416a5c86c7ec7540ed6aaa1c98)

Signed-off-by: Ravishankar N 
Change-Id: Iab4999e213bf2065804f3f8237e470ad454e3c99
BUG: 1489260
Reviewed-on: https://review.gluster.org/18222
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran

cluster/dht: Log files skipped by rebalance

2017-09-07T07:09:20+00:00

There was no easy way to find out which files were
skipped during a rebalance.
Rebalance now logs a message for every skipped file
using msgid 109126, making it easier to find
all files that were skipped.

> BUG: 1480445
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18021
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: hari gowtham 
> Reviewed-by: Raghavendra G 

Change-Id: I2cac7db7285e2f82354251f3ea4094827b0daf3e
BUG: 1486557
(cherry picked from commit a4c43ba9374b8f75a48d38a032353a0c7d311a73)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18149
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

cluster/dht: Aggregate xattrs only for dirs in dht_discover_cbk

2017-09-07T07:07:55+00:00

If dht_discover finds data files on more than one subvol,
racing calls to dht_discover_cbk could end up calling
dht_aggregate_xattr which could delete dictionary data
that is being accessed by higher layer translators.
Fixed to call dht_aggregate_xattr only for directories and
consider only the first file to be found.

> BUG: 1484709
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18137
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: I4f3d2a405ec735d4f1bb33a04b7255eb2d179f8a
BUG: 1486538
(cherry picked from commit 9420022df0962b1fa4f3ea3774249be81bc945cc)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18146
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Amar Tumballi 
CentOS-regression: Gluster Build System

cluster/dht: Reorder dir operations in gf_defrag_fix_layout

2017-08-21T14:27:11+00:00

Earlier, rebalance performed a fix-layout on a directory
before healing its subdirectories. If there were a lot of
subdirs, it could take a while before all subdirs were
created on the newly added bricks. As dht_readdirp only lists
dirs from their hashed subvol, those dirs which hashed to
the newly added bricks but were not yet created on them were
not listed.

Now, the child dirs are listed and processed before the layout
of the parent is fixed. This introduces a change in behaviour
where files in subdirs are migrated before those in parent
directories.

Credit: Shyam 
Github issue: #239

> BUG: 1248393
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18045
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit 96b33b4b278391ca8a7755cf274931d4f1808cb5)
Change-Id: I8ae7f24a510754cd8d1b31e5d608bcf1928599e2
BUG: 1483402
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18071
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

cluster/dht: EBADF handling for fremovexattr and fsetxattr

2017-08-12T13:37:31+00:00

Add EBADF handling for dht_fremovexattr and dht_fsetxattr.

> BUG: 1476665
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17999
> Smoke: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit 747a08d34e2a1e94d7fce68a3577370288bb1955)
Change-Id: Ide0d5812dae79655d2565157e5baabcd753b4309
BUG: 1479303
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18012
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: Check for open fd only on EBADF

2017-08-09T11:29:50+00:00

DHT fd based fops used to check if the fd was open
on the cached subvol before winding the call. However,
this introduced a performance regression of about
30% for reads.

This check was introduced to handle cases where files
were migrated while IOs were happening. As this is not
the common case, dht will now check if the fd is
open on the cached subvol only if the call fails
with EBADF.

This will prevent a performance hit where a rebalance
is not running.

> BUG: 1476665
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17976
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Susant Palai 
> Reviewed-by: Raghavendra G 

(cherry picked from commit cdca1cb26a0aba390c6d8485c0d6d95e22ffc8bd)
Change-Id: I2035a858d63c3fcd22bb634055bbb0ad01686808
BUG: 1479303
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17995
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: Handle wrong rebalance status reporting

2017-08-03T14:13:07+00:00

> Change-Id: Id91ef35f890055cd42b9a94462f92297c77f1fff
> Bug: 1475282
> Signed-off-by: Susant Palai 
> Reviewed-on: https://review.gluster.org/17868
> Tested-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Smoke: Gluster Build System 
> Signed-off-by: Susant Palai 

Change-Id: Id91ef35f890055cd42b9a94462f92297c77f1fff
Bug: 1477152
Signed-off-by: Susant Palai 
Reviewed-on: https://review.gluster.org/17943
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: rebalance min-free-disk fix

2017-08-02T15:05:06+00:00

To calculate available space on a subvolume we used to do
the following in __dht_check_free_space.

post_availspace = (dst_statfs.f_bavail * dst_statfs.f_frsize) - stbuf->ia_size

Now to subtracting the file size from available space is tricky here.
Sometime available space will be lesser than the file size and since all the
participating members in calculation are unsigned int, the result is a large
number (integer overflow).

Solution: We do not need to subtract the file size from the space available,
since fallocate would have reserved file size space already.

> Change-Id: I4f724358c44b9911933742ff3ff8d55b3dfda1cb
> BUG: 1475282
> Signed-off-by: Susant Palai 
> Reviewed-on: https://review.gluster.org/17876
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Reviewed-by: N Balachandran 
> CentOS-regression: Gluster Build System 
> Signed-off-by: Susant Palai 

Change-Id: I4f724358c44b9911933742ff3ff8d55b3dfda1cb
BUG: 1477152
Signed-off-by: Susant Palai 
Reviewed-on: https://review.gluster.org/17942
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G