glusterfs.git/xlators/cluster/dht/src, branch v3.10.0rc0

Readdir-ahead : Honour readdir-optimise option of dht

2017-01-27T14:01:15+00:00

>Change-Id: I9c5e65b32e316e6a2fc7e1f5c79fce79386b78e2
>BUG: 1401812
>Signed-off-by: Poornima G 
>Reviewed-on: https://review.gluster.org/16071
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Raghavendra G 

Change-Id: I9c5e65b32e316e6a2fc7e1f5c79fce79386b78e2
BUG: 1417027
Signed-off-by: Poornima G 
(cherry picked from commit 7c6538f6c8f9a015663b4fc57c640a7c451c87f7)
Reviewed-on: https://review.gluster.org/16461
Tested-by: Raghavendra G 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Shyamsundar Ranganathan

dht/rebalance Estimate time to complete rebalance

2017-01-24T13:22:35+00:00

The estimates will be logged to the rebalance log on running
gluster v rebalance  status

> Change-Id: I9d51b139cd4c8dfde1ff2c2050720ae606c13fc6
> BUG: 1396004
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/15893
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit 2edd75ec8de17da89004859375844f60890a4df0)
Signed-off-by: N Balachandran 

Change-Id: I58b0550210149443966798d9a26e73cb598eeb6a
BUG: 1415915
Reviewed-on: https://review.gluster.org/16458
Tested-by: N Balachandran 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

tier : Tier as a service

2017-01-17T04:49:47+00:00

tierd is implemented by separating from rebalance process.

The commands affected:

1) Attach tier will trigger this process instead of old one
2) tier start and tier start force will also trigger this process.
3) volume status [tier] will show tier daemon as a process instead
of task and normal tier status and tier detach status works.
4) tier stop implemented.
5) detach tier implemented separately along with new detach tier
status
6) volume tier volname status will work using the changes.
7) volume set works

This patch has separated the tier translator from the legacy
DHT rebalance code. It now sends the RPCs from the CLI
to glusterd separate to the DHT rebalance code.
The daemon is now a service, similar to the snapshot daemon,
and can be viewed using the volume status command.

The code for the validation and commit phase are the same
as the earlier tier validation code in DHT rebalance.

The “brickop” phase has been changed so that the status
command can use this framework.

The service management framework is now used.
DHT rebalance does not use this framework.

This service framework takes care of :

*) spawning the daemon, killing it and other such processes.
*) volume set options , which are written on the volfile.
*) restart and reconfigure functions. Restart is to restart
the daemon at two points
        1)after gluster goes down and comes up.
        2) to stop detach tier.
*) reconfigure is used to make immediate volfile changes.
By doing this, we don’t restart the daemon.
it has the code to rewrite the volfile for topological
changes too (which comes into place during add and remove brick).

With this patch the log, pid, and volfile are separated
and put into respective directories.

Change-Id: I3681d0d66894714b55aa02ca2a30ac000362a399
BUG: 1313838
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/13365
Smoke: Gluster Build System 
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Atin Mukherjee

feature/dht: undo partially successful dir rename

2017-01-11T15:40:25+00:00

As with dht, dirs are present on all subvolumes,
renaming them is a compound operation and thus a
partial success + partial failure scenario is
possible, resulting in an inconsistent state.

For purposes of reproduction, such a scenario can
easily be produced by stopping the volume, edit the
volfile of a certain subvolume to get at an
"option read-only on" setting, and then restart
the volume. Thus those operations that are to make change
on the affected subvolume will fail with EROFS.

To handle such scenarios, we introduce an in-memory cache
where we record the return values obtained from the
subvolumes. At the final stage of the dir rename operation
we check if it's a partial success/fail situation. If yes,
then we perform a reverse rename op on those subvolumes
where the operation succeeded.

Change-Id: I3d05f74f53932cb984a918d252a7309c1009a51d
BUG: 1412069
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/15739
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran

dht: At places needed use STACK_WIND_COOKIE

2017-01-09T18:37:02+00:00

Issue:
frame has a void * cookie pointer.
In case of STACK_WIND_COOKIE frame->cookie is assigned
to what is sent by the caller.
In case of STACK_WIND frame->cookie is assigned to point
point to the frame itself.

For ease of coding, at many places, the cookie in the cbk
is used to get the pointer to the next xl. This is
inconsistent when STACK_WIND_TAIL comes into picture.

Eg: dht_setxattr () {
    for (i = 0 ; i < conf->subvolume_cnt ; i++) {
       STACK_WIND (..dht_checking_pathinfo_cbk,
                   conf->subvolumes[i] ..);
    }

    dht_checking_pathinfo_cbk (...void *cookie...) {
        prev = cookie;
        ...
        for (i = 0; i < conf->subvolume_cnt; i++) {
            if (conf->subvolumes[i] == prev->this) {
                 ...
            }
        }
    }

    Consider the below graph:
    dht (parent)
    readdir-ahead  => Doesn't define setxattr and uses STACK_WIND_TAIL
    protocol-client

    With this graph, when dht_checking_pathinfo_cbk is called,
    cookie will have frame pointing to protocol-client.
    i.e. prev->this will be protocol-client. But dht was expecting
    it to be readdir-ahead as it has stored in conf->subvolumes[i]

Solution:
    Hence, as a thumb rule, if cbk is using cookie, then we explicitly
    call STACK_WIND_COOKIE.

Change-Id: I83aea1e24c809c5a91a0db7283e908e125471bd4
BUG: 1401812
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/16039
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/dht: Incorrect migration checks in fsync

2017-01-09T04:33:19+00:00

Fixed the order of the migration phase checks
in dht_fsync_cbk. Phase1 should never be hit if op_ret
is non zero.
Signed-off-by: N Balachandran 

Change-Id: I9222692e04868bffa93498059440f0aa553c83ec
BUG: 1410777
Reviewed-on: http://review.gluster.org/16350
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Tested-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

dht/rebalance: remove errno check for failure detection

2017-01-06T10:25:42+00:00

BUG: 1410355
Change-Id: I867419ca36a81ef7209e6911a46c1c2c898b8eab
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/16328
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

cluster/dht: Do rename cleanup as root

2017-01-05T14:07:17+00:00

Problem:
Rename linkfile cleanup is done as non-root which may not have priviliges to do
the rename so it fails with EACCESS. MKDIR on that name in future will start to
hole on this subvolume. It is not easy to hit on fuse mounts because vfs takes
care of the permission checks even before rename fop is wound. But with
nfs-ganesha mounts it happens.

Fix:
Do rename cleanup as root

BUG: 1409727
Change-Id: I414c1eb6dce76b4516a6c940557b249e6c3f22f4
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/16317
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: N Balachandran 
NetBSD-regression: NetBSD Build System

cluster/dht: Fix dict_leak in migration check tasks

2017-01-02T15:30:09+00:00

Fixed a memleak where dict was not being unrefed
in the dht_migration_complete_check_task and
dht_rebalance_inprogress_task functions.

Change-Id: I3d42e9a2e5c8596c985bf6431a68fd3905227383
BUG: 1409186
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/16308
Smoke: Gluster Build System 
Reviewed-by: MOHIT AGRAWAL 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Shyamsundar Ranganathan

dht/rebalance: reverify lookup failures

2016-12-22T14:28:03+00:00

race: readdirp has read one entry, and doing a lookup on
that entry, but user might have renamed/removed that entry just
after readdirp but before lookup.

Since remove-brick is a costly opertaion,will ingore any
ENOENT/ESTALE failures and move on.

Change-Id: I62c7fa93c0b9b7e764065ad1574b97acf51b5996
BUG: 1408115
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15846
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System