glusterfs.git/xlators/cluster/dht/src/dht-helper.c, branch release-3.10

cluster/dht: Add migration checks to dht_(f)xattrop

2018-01-30T13:38:09+00:00

The dht_(f)xattrop implementation did not implement
migration phase1/phase2 checks which could cause issues
with rebalance on sharded volumes.
This does not solve the issue where fops may reach the target
out of order.

> Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c
> BUG: 1471031
> Signed-off-by: N Balachandran 

Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c
BUG: 1498081
Signed-off-by: N Balachandran

cluster/dht: EBADF handling for fremovexattr and fsetxattr

2017-10-03T12:26:48+00:00

Add EBADF handling for dht_fremovexattr and dht_fsetxattr.

> BUG: 1476665
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17999
> Smoke: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

(cherry picked from commit 747a08d34e2a1e94d7fce68a3577370288bb1955)
Change-Id: Ide0d5812dae79655d2565157e5baabcd753b4309
BUG: 1467010
Signed-off-by: N Balachandran

cluster/dht: Check for open fd only on EBADF

2017-09-17T12:47:26+00:00

DHT fd based fops used to check if the fd was open
on the cached subvol before winding the call. However,
this introduced a performance regression of about
30% for reads.

This check was introduced to handle cases where files
were migrated while IOs were happening. As this is not
the common case, dht will now check if the fd is
open on the cached subvol only if the call fails
with EBADF.

This will prevent a performance hit where a rebalance
is not running.

> BUG: 1476665
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17976
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Susant Palai 
> Reviewed-by: Raghavendra G 

Change-Id: I2035a858d63c3fcd22bb634055bbb0ad01686808
BUG: 1467010
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18057
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

refcount: typecast function for calling on free

2017-08-11T11:22:43+00:00

All of the functions called to free the refcounted structure are doing a
typecast from (void*) to their own type taht is being free'd. This
really is not needed and the refcount interface is made a little simpler
without the requirement of typecasting.

With this small improvement in the API, all callers are updated too.

Cherry picked from commit f2ca301bd741e3e3f076cd3f72fcd377bcef2a1a:
> Change-Id: I32473b6d1799f62861d4b2d78ea30c09e6c80ab1
> BUG: 1416889
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/16471
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Xavier Hernandez 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 

Backport note: This patch makes it easier to backport changes that use
               gf_refcount_t. There is no functional change.

Change-Id: I32473b6d1799f62861d4b2d78ea30c09e6c80ab1
BUG: 1471870
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17913
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: Fix fd check race

2017-07-20T09:33:20+00:00

There is a another race between the cached subvol
being updated in the inode_ctx and the fd being opened on
the target.

1. fop1 -> fd1 -> subvol0
2. file migrated from subvol0 to subvol1 and cached_subvol
   changed to subvol1 in inode_ctx
3. fop2 -> fd1 -> subvol1 [takes new cached subvol]
4. fop2 -> checks fd ctx (fd not open on subvol1) -> opens fd1 on subvol1
5. fop1 -> checks fd ctx (fd not open on subvol0)
   -> tries to open fd1 on subvol0 -> fails with "No such file on directory".

Fix:
If dht_fd_open_on_dst fails with ENOENT or ESTALE, wind to old subvol
and let the phase1/phase2 checks handle it.

Change-Id: I34f8011574a8b72e3bcfe03b0cc4f024b352f225
BUG: 1467010
> BUG: 1465075
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17731
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Amar Tumballi 
(cherry picked from commit f7a450c17fee7e43c544473366220887f0534ed7)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17829
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

cluster/dht: Check if fd is opened on dst subvol

2017-07-19T21:34:15+00:00

If an fd is opened on a file, the file is migrated
and the cached subvol is updated in the inode_ctx
before an fd based fop is sent, the fop is sent to
the dst subvol on which the fd is not opened.
This causes the FOP to fail with EBADF.

Now, every fd based fop will check to see that the fd
has been opened on the dst subvol before winding it down.

> BUG: 1465075
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17630
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Susant Palai 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 91db0d47ca267aecfc6124a3f337a4e2f2c9f1e2)
Change-Id: Id92ef5eb7a5b5226688e2d2868b15e383f5f240e
BUG: 1467010
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17752
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

cluster/dht: Rebalance on all nodes should migrate files

2017-05-18T08:06:59+00:00

Problem:
Rebalance compares the node-uuid of a file against its own
to and migrates a file only if they match. However, the
current behaviour in both AFR and EC is to return
the node-uuid of the first brick in a replica set for all
files. This means a single node ends up migrating all
the files if the first brick of every replica set is on the
same node.

Fix:
AFR and EC will return all node-uuids for the replica set.
The rebalance process will divide the files to be migrated
among all the nodes by hashing the gfid of the file and
using that value to select a node to perform the migration.
This patch makes the required DHT and tiering changes.

Some tests in rebal-all-nodes-migrate.t will need to be
uncommented once the AFR and EC changes are merged.

> BUG: 1366817
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17239
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Jeff Darcy 
> Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit b23bd3dbc2c153171d0bb1205e6804afe022a55f)

Change-Id: I5ce41600f5ba0e244ddfd986e2ba8fa23329ff0c
BUG: 1451561
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17311
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

feature/dht: undo partially successful dir rename

2017-01-11T15:40:25+00:00

As with dht, dirs are present on all subvolumes,
renaming them is a compound operation and thus a
partial success + partial failure scenario is
possible, resulting in an inconsistent state.

For purposes of reproduction, such a scenario can
easily be produced by stopping the volume, edit the
volfile of a certain subvolume to get at an
"option read-only on" setting, and then restart
the volume. Thus those operations that are to make change
on the affected subvolume will fail with EROFS.

To handle such scenarios, we introduce an in-memory cache
where we record the return values obtained from the
subvolumes. At the final stage of the dir rename operation
we check if it's a partial success/fail situation. If yes,
then we perform a reverse rename op on those subvolumes
where the operation succeeded.

Change-Id: I3d05f74f53932cb984a918d252a7309c1009a51d
BUG: 1412069
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/15739
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran

cluster/dht: Fix dict_leak in migration check tasks

2017-01-02T15:30:09+00:00

Fixed a memleak where dict was not being unrefed
in the dht_migration_complete_check_task and
dht_rebalance_inprogress_task functions.

Change-Id: I3d42e9a2e5c8596c985bf6431a68fd3905227383
BUG: 1409186
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/16308
Smoke: Gluster Build System 
Reviewed-by: MOHIT AGRAWAL 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: remove unnecessary struct member

2016-09-23T05:32:53+00:00

Removed a structure member that was not used
from dht_local_t. Skipped some unnecessary checks
in dht_filter_loc_subvol_key.

Change-Id: I81740b6528e063fb9cf5817e05865ff4d77aa748
BUG: 1378305
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15542
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G