glusterfs.git/xlators/mgmt, branch v3.12dev

cluster/dht: Make rebalance throttle option tuned by number

2017-04-29T14:29:34+00:00

Current rebalance throttle options: lazy/normal/aggressive may not always be
sufficient for the purpose of throttling.  In our recent test, we observed for
certain setups, normal and aggressive modes behaved similarly consuming full
disk bandwidth. So in cases like this admin should be able to  tune it
down(or vice versa) depending on the need.

Along with old throttle configurations, thread counts are tuned based on number.
e.g. gluster v set vol-name cluster-rebal.throttle  5.

Admin can tune up/down between 0 and the number of cores available.

Note: For heterogenous servers, validation will fail on the old server if "number"
is given for throttle configuration.
The message looks something like this:
"volume set: failed: Staging failed on vm2. Error: cluster.rebal-throttle should be {lazy|normal|aggressive}"

Test: Manual test by logging active thread number after reconfiguring throttle option.
testcase: tests/basic/distribute/throttle-rebal.t

Change-Id: I46e3cde546900307831028b344ecf601fd9b02c3
BUG: 1438370
Signed-off-by: Susant Palai 
Reviewed-on: https://review.gluster.org/16980
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra G

glusterd: Fix removing pmap entry on rpc disconnect

2017-04-28T17:15:30+00:00

Problem:
The following line of code intended to remove pmap entry for the
connection during disconnects:

    pmap_registry_remove (this, 0, NULL, GF_PMAP_PORT_NONE, xprt);

However, no pmap entry will have it's type set to GF_PMAP_PORT_NONE
at any point in time. So a call to pmap_registry_search_by_xprt() in
pmap_registry_remove() will always fail to find a match.

Fix:
Optionally ignore pmap entry's type in pmap_registry_search_by_xprt().

BUG: 1193929
Change-Id: I705f101739ab1647ff52a92820d478354407264a
Signed-off-by: Prashanth Pai 
Reviewed-on: https://review.gluster.org/17129
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

build: conditionally build legacy gNFS server and associated sub-packaging

2017-04-28T14:08:41+00:00

Plus some additional logic in glusterd to ensure gnfs (glusterfs)
daemons are never started if server/nfs xlator is not installed.

As a service, nfs is still initialized. The glusterfs-gnfs RPM
may be installed or uninstalled independent of anything else,
including on a system where gluster is actively running, so the
existence of the xlator is always tested before trying to start
gnfs.

Change-Id: I56743ad1cb36a84917226d7d26cb9d015d441e66
BUG: 1326219
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16958
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

cluster/ec: Implement self-heal-window_size option

2017-04-25T06:36:27+00:00

Fix implements the heal window size option for
EC. This option control the maximum size of
read/write operation carried out in self-heal
process.

BUG: 1441491
Change-Id: I6c0ef65c9ca18b0828f91b319d4f52ac5b77d0d8
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/17098
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd/geo-rep: Fix snapshot create in geo-rep setup

2017-04-24T09:50:29+00:00

glusterd persists geo-rep sessions in glusterd
info file which is represented by dictionary
'volinfo->gsync_slaves' in memory. Glusterd also
maintains in memory active geo-rep sessions in
dictionary 'volinfo->gsync_active_slaves' whose key
is "::".

When glusterd is restarted while the geo-rep sessions
are active, it builds the 'volinfo->gsync_active_slaves'
from persisted glusterd info file. Since slave volume
uuid is added to "voinfo->gsync_slaves" with the commit
"http://review.gluster.org/13111", it builds it with key
":::" which is
wrong. So during snapshot pre-validation which checks
whether geo-rep is active or not, it always says it is
ACTIVE, as geo-rep stop would not deleted this key.
Fixed the same in this patch.

Change-Id: I185178910b4b8a62e66aba406d88d12fabc5c122
BUG: 1443977
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17093
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd: set conn->reconnect to null on timer cancellation

2017-04-20T15:14:17+00:00

Change-Id: Ic48e6652f431daeb0db027660f6c9de16d893f08
BUG: 1443896
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/17088
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

Implement negative lookup cache

2017-04-20T04:18:52+00:00

Before creating any file negative lookups(1 in Fuse, 4 in SMB etc.)
are sent to verify if the file already exists. By serving these
lookups from the cache when possible, increases the create
performance by multiple folds in SMB access and some percentage
in Fuse/NFS access.

Feature page: https://review.gluster.org/#/c/16436

Updates #82
Change-Id: Ib1c0e7ac7a386f943d84f6398c27f9a03665b2a4
BUG: 1442569
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16952
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

features/bit-rot-stub: bring in optional versioning

2017-04-19T03:29:46+00:00

* As of now bit-rot-stub does versioning always. This leads
  lots of getxattr calls being made in lookups. So make
  object versioning optional.

Change-Id: I83713e45ae59fb28004bb3cfa008f2d69edebbfa
BUG: 1359599
Signed-off-by: Raghavendra Bhat 
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/14442
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Fix snapshot failure in non-root geo-rep setup

2017-04-18T09:39:46+00:00

Geo-replication session directory name has the form
'__'. But in non-root
geo-replication setup, while preparing geo-replication
session directory name, glusterd is including 'user@'
resulting in "__".
Hence snapshot is failing to copy geo-rep specific
session files. Fixing the same.

Change-Id: Id214d3186e40997d2827a0bb60d3676ca2552df7
BUG: 1442760
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17067
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Amar Tumballi 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

dht: Add readdir-ahead in rebalance graph if parallel-readdir is on

2017-04-18T06:16:11+00:00

Issue:
The value of linkto xattr is generally the name of the dht's
next subvol, this requires that the next subvol of dht is not
changed for the life time of the volume. But with parallel
readdir enabled, the readdir-ahead loaded below dht, is optional.
The linkto xattr for first subvol, when:
- parallel readdir is enabled : "-readdir-head-0"
- plain distribute volume : "-client-0"
- distribute replicate volume : "-afr-0"

The value of linkto xattr is "-readdir-head-0" when
parallel readdir is enabled, and is "-client-0" if
its disabled. But the dht_lookup takes care of healing if it
cannot identify which linkto subvol, the xattr points to.

In dht_lookup_cbk, if linkto xattr is found to be "-client-0"
and parallel readdir is enabled, then it cannot understand the
value "-client-0" as it expects "-readdir-head-0".
In that case, dht_lookup_everywhere is issued and then the linkto file
is unlinked and recreated with the right linkto xattr. The issue is
when parallel readdir is enabled, mount point accesses the file
that is currently being migrated. Since rebalance process doesn't
have parallel-readdir feature, it expects "-client-0"
where as mount expects "-readdir-head-0". Thus at some point
either the mount or rebalance will fail.

Solution:
Enable parallel-readdir for rebalance as well and then do not
allow enabling/disabling parallel-readdir if rebalance is in
progress.

Change-Id: I241ab966bdd850e667f7768840540546f5289483
BUG: 1436090
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17056
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra G