glusterfs.git/xlators/cluster, branch v3.10.6

cluster/dht: EBADF handling for fremovexattr and fsetxattr

2017-10-03T12:26:48+00:00

Add EBADF handling for dht_fremovexattr and dht_fsetxattr.

> BUG: 1476665
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17999
> Smoke: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

(cherry picked from commit 747a08d34e2a1e94d7fce68a3577370288bb1955)
Change-Id: Ide0d5812dae79655d2565157e5baabcd753b4309
BUG: 1467010
Signed-off-by: N Balachandran

afr: auto-resolve split-brains for zero-byte files

2017-10-02T12:34:23+00:00

Backport of https://review.gluster.org/#/c/18283/

Problems:
As described in BZ 1491670, renaming hardlinks can result in data/mdata
split-brain of the DHT link-to files (T files) without any mismatch of
data and metadata.

As described in BZ 1486063, for a zero-byte file with only dirty bits
set, arbiter brick will likely be chosen as the source brick.

Fix:
For zero byte files in split-brain, pick first brick as
a) data source if file size is zero on all bricks.
b) metadata source if metadata is the same on all bricks

In arbiter case, if file size is zero on all bricks and there are no
pending afr xattrs, pick 1st brick as data source.

Change-Id: I0270a9a2f97c3b21087e280bb890159b43975e04
BUG: 1496321
Signed-off-by: Ravishankar N 
Reported-by: Rahul Hinduja 
Reported-by: Mabi

dht: add FOP check to dht_file_setattr_cbk

2017-09-29T09:01:07+00:00

Problem:
bug-797171.7 loaded error-gen xlator on the brick which sent EBADF for a
non fd-based fop, namely setattr. This caused
dht_check_and_open_fd_on_subvol_task() to crash as local->fd was NULL.

Fix:
Call dht_check_and_open_fd_on_subvol_task() from dht_file_setattr_cbk
only for dht_fsetattr and not dht_setattr or dht_setattr2

> Reviewed-on: https://review.gluster.org/18208
> Smoke: Gluster Build System 
> Reviewed-by: Susant Palai 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra G 
> Reviewed-by: N Balachandran 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 47188e9eac59de416a5c86c7ec7540ed6aaa1c98)


Signed-off-by: Ravishankar N 
Change-Id: Iab4999e213bf2065804f3f8237e470ad454e3c99
BUG: 1497122

afr: Prevent null gfids in self-heal entry re-creation

2017-09-17T13:38:22+00:00

> Reviewed-on: https://review.gluster.org/17981
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Karthik U S 
(cherry picked from commit bead74a6e085001225bc0704bad1a5db36dd75a1)

Change-Id: I5acb8bd0a19fc4e764d61e349bb690b5236ee610
BUG: 1491985
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18300
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

afr: heal metadata in discover code path

2017-09-17T13:37:32+00:00

******************************************************
Backport of: https://review.gluster.org/18202
Also added loc_is_nameless() to libglusterfs since the patch that
introduced it in master was not backported to release-3.10.

Note: 18202 is a squash of  17850 and 18187 in master.
******************************************************

During graph switch, if fuse sends nameless (gfid) lookups, afr takes
the discover code path to serve it. If there are pending metadata heals,
they do not happen unless an inode refresh happens as a part of
discover (which is not guaranteed to happen always).

This patch fixes it by attempting metadata heal as a part of discover,
just like how it is done in lookup code path.

Change-Id: I87c493045b9225741cad173bf3f645848697032e
BUG: 1492010
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18304
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

afr: check validity of afr_reply

2017-09-17T12:53:24+00:00

...in various self-heal code paths.

Originally found by Pranith in __afr_selfheal_name_impunge ()

Also change __afr_selfheal_assign_gfid() to send lookup only on those
bricks that don't have a gfid matching that of the source.

> Reviewed-on: https://review.gluster.org/18065
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit d594900dbca92c356152be65fce16f77c402117c)
Change-Id: I70a2ccd750a2af92c5fc36e0eefb2b6125404b4a
BUG: 1491995
Signed-off-by: Pranith Kumar K 
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18303
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: Check for open fd only on EBADF

2017-09-17T12:47:26+00:00

DHT fd based fops used to check if the fd was open
on the cached subvol before winding the call. However,
this introduced a performance regression of about
30% for reads.

This check was introduced to handle cases where files
were migrated while IOs were happening. As this is not
the common case, dht will now check if the fd is
open on the cached subvol only if the call fails
with EBADF.

This will prevent a performance hit where a rebalance
is not running.

> BUG: 1476665
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17976
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Susant Palai 
> Reviewed-by: Raghavendra G 

Change-Id: I2035a858d63c3fcd22bb634055bbb0ad01686808
BUG: 1467010
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18057
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

cluster/ec: Node uuid xattr support update for EC

2017-09-13T06:41:52+00:00

Problem:
The change in EC to return list of node uuids for
GF_XATTR_NODE_UUID_KEY was causing problems with
geo-rep.

Fix:
This patch will allow to get the single node uuid
as it was doing before with the key
"GF_XATTR_NODE_UUID_KEY", and will also allow to get
the list of node uuids by using a new key
"GF_XATTR_LIST_NODE_UUIDS_KEY". This will solve
the problem with geo-rep and any other features which
were depending on this.

>BUG: 1462790
>Change-Id: I2d9214a9658d4a41a3d6de08600884d2bda5f3eb
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/17594
>Smoke: Gluster Build System 
>Reviewed-by: Xavier Hernandez 
>Reviewed-by: Pranith Kumar Karampuri 
>CentOS-regression: Gluster Build System 
>Signed-off-by: Sunil Kumar Acharya 

BUG: 1487647
Change-Id: I2d9214a9658d4a41a3d6de08600884d2bda5f3eb
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/17667
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/ec: return all node uuids from all subvolumes

2017-09-01T07:44:52+00:00

EC was retuning the UUID of the brick with smaller value. This had
the side effect of not evenly balancing the load between bricks on
rebalance operations.

This patch modifies the common functions that combine multiple subvolume
values into a single result to take into account the subvolume order
and, optionally, other subvolumes that could be damaged.

This makes easier to add future features where brick order is important.
It also makes possible to easily identify the originating brick of each
answer, in case some brick will have an special meaning in the future.

>Change-Id: Iee0a4da710b41224a6dc8e13fa8dcddb36c73a2f
>BUG: 1366817
>Signed-off-by: Xavier Hernandez 
>Reviewed-on: https://review.gluster.org/17297
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Ashish Pandey 
>Reviewed-by: Pranith Kumar Karampuri 

BUG: 1487042
Change-Id: Iee0a4da710b41224a6dc8e13fa8dcddb36c73a2f
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/18148
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System

cluster/rebalance: Fix hardlink migration failures

2017-08-11T20:03:45+00:00

A brief about how hardlink migration works:
  - Different hardlinks (to the same file) may hash to different bricks,
but their cached subvol will be same. Rebalance picks up the first hardlink,
calculates it's  hash(call it TARGET) and set the hashed subvolume as an xattr
on the data file.
  - Now all the hardlinks those come after this will fetch that xattr and will
create linkto files on TARGET (all linkto files for the hardlinks will be hardlink
to each other on TARGET).
  - When number of hardlinks on source is equal to the number of hardlinks on
TARGET, the data migration will happen.

RACE:1
  Since rebalance is multi-threaded, the first lookup (which decides where the TARGET
subvol should be), can be called by two hardlink migration parallely and they may end
up creating linkto files on two different TARGET subvols. Hence, hardlinks won't be
migrated.

Fix: Rely on the xattr response of lookup inside gf_defrag_handle_hardlink since it
is executed under synclock.

RACE:2
  The linkto files on TARGET can be created by other clients also if they are doing
lookup on the hardlinks.  Consider a scenario where you have 100 hardlinks.  When
rebalance is migrating 99th hardlink, as a result of continuous lookups from other
client, linkcount on TARGET is equal to source linkcount. Rebalance will migrate data
on the 99th hardlink itself. On 100th hardlink migration, hardlink will have TARGET as
cached subvolume. If it's hash is also the same, then a migration will be triggered from
TARGET to TARGET leading to data loss.

Fix: Make sure before the final data migration, source is not same as destination.

RACE:3
  Since a hardlink can be migrating to a non-hashed subvolume, a lookup from other
client or even the rebalance it self, might delete the linkto file on TARGET leading
to hardlinks never getting migrated.

This will be addressed in a different patch in future.

> Change-Id: If0f6852f0e662384ee3875a2ac9d19ac4a6cea98
> BUG: 1469964
> Signed-off-by: Susant Palai 
> Reviewed-on: https://review.gluster.org/17755
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: N Balachandran 
> Reviewed-by: Raghavendra G 
> Signed-off-by: Susant Palai 

Change-Id: If0f6852f0e662384ee3875a2ac9d19ac4a6cea98
BUG: 1473141
Signed-off-by: Susant Palai 
Reviewed-on: https://review.gluster.org/17838
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan