glusterfs.git/xlators/mgmt, branch v3.10.1

glusterd: support filesystems with dynamic inode sizes

2017-03-28T12:04:12+00:00

btrfs and zfs are two filesystems that do not have fixed sizes for
inodes. Instead of logging an error, skip checking and mark the size as
"N/A" like other properties that can not be reported.

The error message that was reported by users on the mailinglist shows up
like:

  [glusterd-utils.c:5458:glusterd_add_inode_size_to_dict] 0-management: could not find (null) to getinode size for /dev/vdb (btrfs): (null) package missing?

Cherry picked from commit 12921693b572f642156d3167d1c92d3449dfc8ec:
> Change-Id: Ib10b7a3669f2f4221075715d9fd44ce1ffc35324
> Reported-by: Arman Khalatyan 
> URL: http://lists.gluster.org/pipermail/gluster-users/2017-March/030189.html
> BUG: 1433425
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/16867
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Prashanth Pai 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

Change-Id: Ib10b7a3669f2f4221075715d9fd44ce1ffc35324
Reported-by: Arman Khalatyan 
BUG: 1436411
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16959
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-13T14:19:00+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

master change https://review.gluster.org/16880
master BZ: 1430841

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430844
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16881
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Shyamsundar Ranganathan

glusterd: don't queue attach reqs before connecting

2017-03-10T19:50:11+00:00

This was causing USS tests to fail.  The underlying problem here is
that if we try to queue the attach request too soon after starting a
brick process then the socket code will get an error trying to write
to the still-unconnected socket.  Its response is to shut down the
socket, which causes the queued attach requests to be force-unwound.
There's nothing to retry them, so they effectively never happen and
those bricks (second and succeeding for a snapshot) never become
available.

We *do* have a retry loop for attach requests, but currently break out
as soon as a request is queued - not actually sent.  The fix is to
modify that loop so it will wait some more if the rpc connection isn't
even complete yet.  Now we break out only when we have a completed
connection *and* a queued request.

Backport of:
> 53e2c875cf97df8337f7ddb5124df2fc6dd37bca
> BUG: 1430148
> Reviewed-on: https://review.gluster.org/16868

Signed-off-by: Jeff Darcy 
BUG: 1431176
Change-Id: Ib6be13646f1fa9072b4a944ab5f13e1b29084841
Reviewed-on: https://review.gluster.org/16887
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

glusterd: take conn->lock around operations on conn->reconnect

2017-03-10T19:48:34+00:00

Failure to do this could lead to a race in which a timer would be
removed twice concurrently, corrupting the timer list (because
gf_timer_call_cancel has no internal protection against this) and
possibly causing a crash.

Backport of:
> 4e0d4b15717da1f6466133158a26927fb91384b8
> BUG: 1421721
> Reviewed-on: https://review.gluster.org/16662

Change-Id: Ic1a8b612d436daec88fd6cee935db0ae81a47d5c
BUG: 1431175
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16885
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd: disallow increasing replica count for arbiter volumes

2017-03-07T13:54:18+00:00

Problem: add-brick command to increase replica count in an arbiter
volume succeeds, causing undesirable effects like the 4th brick being
loaded with the arbiter xlator, the 3rd one losing the arbiter xlator
(when the brick process is restarted), arbitration logic in afr going
for a toss etc.

Fix: Arbiter configuration should always be a replica 3 volume (of
which 3rd brick is arbiter). Hence disallow increasing replica count for
arbiter volume configurations.

> Reviewed-on: https://review.gluster.org/16845
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
(cherry picked from commit b7ba77ab3ffb641d06223f7af5145d3d670b032a)

Change-Id: I9fe4edac880d0f711e6d44324ad5562974e53e51
BUG: 1429773
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/16863
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd: add portmap details in glusterd statedump

2017-03-03T15:13:06+00:00

glusterd statedump file doesn't have information on the ports and its
associated brick details. This is quite problematic if any setup ends up
with stale ports and the only way to find the issue out is to gdb into
the process which is always not available. This patch attempts to fill
in this gap.

>Reviewed-on: https://review.gluster.org/16764
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Prashanth Pai 
>(cherry picked from commit dbdfe6c353060b7af7b4250c8d6c30ed0c35783c)

Change-Id: I26b4fe753d752366ddf865ca3eeae3b4d577d555
BUG: 1427399
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16779
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System

core: Clean up pmap registry up correctly on volume/brick stop

2017-02-28T15:29:54+00:00

This commit changes the following:
1. In glusterfs_handle_terminate, send out individual pmap signout
requests to glusterd for every brick.
2. Add another parameter to glusterfs_mgmt_pmap_signout function to
pass the brickname that needs to be removed from the pmap registry.
3. Make sure pmap_registry_search doesn't break out from the loop
iterating over the list of bricks per port if the first brick entry
corresponding to a port is whitespaced out.
4. Make sure the pmap registry entries are removed for other
daemons like snapd.

> Reviewed-on: https://review.gluster.org/16689
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Gaurav Yadav 
> Reviewed-by: Jeff Darcy 

(cherry picked from commit 1e3538baab7abc29ac329c78182b62558da56d98)

Change-Id: I69949874435b02699e5708dab811777ccb297174
BUG: 1427461
Signed-off-by: Samikshan Bairagya 
Reviewed-on: https://review.gluster.org/16786
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd: unref brickinfo object on volume stop

2017-02-21T21:39:53+00:00

If brick multiplexing is enabled, on a volume stop glusterd was not
unrefing the brickinfo rpc object which lead to a flood of stale rpc
logs.

>Reviewed-on: https://review.gluster.org/16699
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Jeff Darcy 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Samikshan Bairagya 
>(cherry picked from commit 9cdfbdced23cd43b8738636a3ed906c8d4267d67)

Change-Id: I18fedcd6921042ef2e945605466194b7b53fe2f7
BUG: 1425556
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16703
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

posix: Fix creation of files with S_ISVTX on FreeBSD

2017-02-20T15:42:51+00:00

On FreeBSD the S_ISVTX flag is completely ignored when creating a
regular file. Since gluster needs to create files with this flag set,
specialy for DHT link files, it's necessary to force the flag.

This fix does this by calling fchmod() after creating a file that
must have this flag set.

> Change-Id: I51eecfe4642974df6106b9084a0b144835a4997a
> BUG: 1411228
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: https://review.gluster.org/16417
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Bhat 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I2087516383bd132c59bbab98eda8f2243a2163fe
BUG: 1424973
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/16686
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Shyamsundar Ranganathan

glusterd, readdir-ahead: Fix backward incompatibility

2017-02-19T14:18:08+00:00

Backport of https://review.gluster.org/#/c/16657/

Issue:
Any opion is spcified in two places: In the options[] of xlator
itself and glusterd-volume-set.c. The default value of this option
can be specified in both the places. If its specified only in xlator
then the volfile generated will not have the option and default value,
it will be assigned during graph initialization.
With patch [1] the option rda-request-size was changed from INT to SIZET
type, and default was changed from 131072 to 128KB, but was specified
only in the readdir-ahead.c. Thus with this patch alone the volfile
entry for readdir-ahead looks like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    subvolumes patchy-read-ahead
end-volume

With patch [2], the default of option rda-request-size was specified
in glusterd-volume-set.c as well(as it was necessary fr parallel readdir).
With this patch the readdir entry in the volfile will look like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    option rda-cache-limit 10MB
    option rda-request-size 128KB
    option parallel-readdir off
    subvolumes patchy-read-ahead
end-volume


Now consider the server has both these patches and client doesn't.
Server will generate a volfile with entry:

The old clients which thought the option rda-request-size is of type
INT will now recieve the value 128KB which it willn't understand,
and hence fail the mount.

The issue is seen only with the combination of [1] and [2].

Solution:
Instead of specifying 128KB as default in glusterd we specify 131072
so that the old clients will interpret as INT and new ones as 128KB

Credits: Raghavendra G

> Reviewed-on: https://review.gluster.org/16657
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Atin Mukherjee 

Change-Id: I0c269a5890957fd8a38e9a05bdec088645a6688a
BUG: 1423412
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16658
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee