glusterfs.git/xlators/cluster, branch v3.10.4

cluster/dht: Additional checks for rebalance estimates

2017-07-04T14:39:20+00:00

The rebalance estimates calculation was not handling
calculations correctly when no files had been processed,
i.e., when rate_lookedup was 0.

Now, the estimated time is set to 0 in such scenarios as
there is no way for rebalance to figure out how long the
process will take to complete without knowing the rate at
which the files are being processed.

> BUG: 1457985
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17564
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra G 

Change-Id: I7b6378e297e1ba139852bcb2239adf2477336b5b
BUG: 1460914
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17599
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Raghavendra Talur 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/afr: Returning single and list of node uuids from AFR

2017-07-01T15:55:31+00:00

Problem:
The change in afr to return list of node uuids was causing problems
with geo-rep.

Fix:
This patch will allow to get the single node uuid as it was doing
before with the key "GF_XATTR_NODE_UUID_KEY", and will also allow
to get the list of node uuids by using a new key
"GF_XATTR_LIST_NODE_UUIDS_KEY". This will solve the problem with
geo-rep and any other feature which were depending on this.

> Change-Id: I09885dac6dfca127be94b708470c8c2941356f9a
> BUG: 1462790
> Signed-off-by: karthik-us 
> Reviewed-on: https://review.gluster.org/17576
> Smoke: Gluster Build System 
> Reviewed-by: Ravishankar N 
> Reviewed-by: Kotresh HR 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

(cherry picked from commit 475ec9928ef96b63a0bfa859a9ae68709275033c)

Change-Id: I5e741a48a426ee9a3cc69612051e0e9bcf33b500
BUG: 1464078
Signed-off-by: karthik-us 
Reviewed-on: https://review.gluster.org/17603
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

cluster:dht Fix crash in dht_rename_lock_cbk

2017-07-01T15:54:19+00:00

Use a local variable to store the call count
in the STACK_WIND for loop. Using frame->local
is dangerous as it could be freed while the loop
is still being processed

> BUG: 1466110
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17645
> Smoke: Gluster Build System 
> Tested-by: Nigel Babu 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Jeff Darcy 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit 56da27cf5dc6ef54c7fa5282dedd6700d35a0ab0)
Change-Id: Ie65cdcfb7868509b4a83bc2a5b5d6304eabfbc8e
BUG: 1466863
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17665
Smoke: Gluster Build System 
Reviewed-by: Jeff Darcy 
CentOS-regression: Gluster Build System

cluster/dht: Include dirs in rebalance estimates

2017-06-20T04:57:14+00:00

Empty directories were not being considered while
calculating rebalance estimates leading to negative
time-left values being displayed as part of the
rebalance status.

> BUG: 1457985
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17448
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra G 

Change-Id: I48d41d702e72db30af10e6b87b628baa605afa98
BUG: 1460914
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17530
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

afr: add errno to afr_inode_refresh_done()

2017-06-16T21:54:56+00:00

Backport of https://review.gluster.org/17413 and
https://review.gluster.org/17436

Problem:
When parellel `rm -rf`s were being done from cifs clients, opendir might
fail on some replicas with ENOENT. DHT ignores partial opendir failures
in dht_fd_cbk() and winds readdirs on those replicas. Afr inode refresh
(as a part of readdirp read_txn) sees in its fd context that the state
of the fds is *not* AFR_FD_OPENED and bails out to
afr_inode_refresh_done() without doing a refresh. When this happens, the
errno is set as EIO due to lack of readable subvols, logging split-brain
messages in the logs.

Fix:
Introduce an errno argument to afr_inode_refresh_do() to bail out with
the right error value when inode refresh is not performed.

Change-Id: Id88e604278abb8df47750d45258d9e6dde710600
BUG: 1457732
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/17516
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/afr: Return the list of node_uuids for the subvolume

2017-05-30T13:13:27+00:00

Problem:
AFR was returning the node uuid of the first node for every file if
the replica set was healthy, which was resulting in only one node
migrating all the files.

Fix:
With this patch AFR returns the list of node_uuids to the upper layer,
so that they can decide on which node to migrate which files, resulting
in improved performance. Ordering of node uuids will be maintained based
on the ordering of the bricks. If a brick is down, then the node uuid
for that will be set to all zeros.

> Reviewed-on: https://review.gluster.org/17084
> Reviewed-by: Pranith Kumar Karampuri 
> Tested-by: Pranith Kumar Karampuri 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 0a50167c0a8f950f5a1c76442b6c9abea466200d)

Change-Id: I73ee0f9898ae473584fdf487a2980d7a6db22f31
BUG: 1451561
Signed-off-by: karthik-us 
Reviewed-on: https://review.gluster.org/17337
Tested-by: Ravishankar N 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

cluster/dht: Fix crash in dht_selfheal_dir_setattr

2017-05-29T08:51:53+00:00

Use a local variable to store the call cnt used in the
for loop for the STACK_WIND so as not to access local
which may be freed by STACK_UNWIND after all fops return.

> BUG: 1452102
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17343
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 17784aaa311494e4538c616f02bf95477ae781bc)
Change-Id: I24f49b6dbd29a2b706e388e2f6d5196c0f80afc5
BUG: 1453056
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17349
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

afr: propagate correct errno for fop failures in arbiter

2017-05-18T08:08:18+00:00

Problem:
If quorum is not met in fop cbk, arbiter sends an ENOTCONN error to the
upper xlators. In a VM workload with sharding enabled, this was leading
to the VM pausing when replace-brick was performed as described in the BZ.

Fix:
Move the fop cbk arbitration logic to afr_handle_quorum() because in
normal replica volumes, that is the function that has the quorum and
errno checks in the fop cbk path before doing a post-op.

Thanks to Pranith for suggesting this approach.

> Reviewed-on: https://review.gluster.org/17235
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 93c850dd2a513fab75408df9634ad3c970a0e859)

Change-Id: Ie6315db30c5e36326b71b90a01da824109e86796
BUG: 1450934
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/17295
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Raghavendra Talur

cluster/dht: Rebalance on all nodes should migrate files

2017-05-18T08:06:59+00:00

Problem:
Rebalance compares the node-uuid of a file against its own
to and migrates a file only if they match. However, the
current behaviour in both AFR and EC is to return
the node-uuid of the first brick in a replica set for all
files. This means a single node ends up migrating all
the files if the first brick of every replica set is on the
same node.

Fix:
AFR and EC will return all node-uuids for the replica set.
The rebalance process will divide the files to be migrated
among all the nodes by hashing the gfid of the file and
using that value to select a node to perform the migration.
This patch makes the required DHT and tiering changes.

Some tests in rebal-all-nodes-migrate.t will need to be
uncommented once the AFR and EC changes are merged.

> BUG: 1366817
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17239
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Jeff Darcy 
> Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit b23bd3dbc2c153171d0bb1205e6804afe022a55f)

Change-Id: I5ce41600f5ba0e244ddfd986e2ba8fa23329ff0c
BUG: 1451561
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17311
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: Fix crash in dht rmdir

2017-05-18T08:06:43+00:00

Using local->call_cnt to check STACK_WINDs can
cause dht_rmdir_do to be called erroneously if
dht_rmdir_readdirp_cbk unwinds before we check if
local->call_cnt is zero in dht_rmdir_opendir_cbk.
This can cause frame corruptions and crashes.

Thanks to Shyam (srangana@redhat.com) for the
analysis.

> BUG: 1451083
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17305
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit 6f7d55c9d58797beaf8d5393c03a5a545bed8bec)
Change-Id: I5362cf78f97f21b3fade0b9e94d492002a8d4a11
BUG: 1451371
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17309
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan