glusterfs.git/xlators/mgmt/glusterd/src, branch v3.10.0

glusterd: unref brickinfo object on volume stop

2017-02-21T21:39:53+00:00

If brick multiplexing is enabled, on a volume stop glusterd was not
unrefing the brickinfo rpc object which lead to a flood of stale rpc
logs.

>Reviewed-on: https://review.gluster.org/16699
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Jeff Darcy 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Samikshan Bairagya 
>(cherry picked from commit 9cdfbdced23cd43b8738636a3ed906c8d4267d67)

Change-Id: I18fedcd6921042ef2e945605466194b7b53fe2f7
BUG: 1425556
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16703
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

posix: Fix creation of files with S_ISVTX on FreeBSD

2017-02-20T15:42:51+00:00

On FreeBSD the S_ISVTX flag is completely ignored when creating a
regular file. Since gluster needs to create files with this flag set,
specialy for DHT link files, it's necessary to force the flag.

This fix does this by calling fchmod() after creating a file that
must have this flag set.

> Change-Id: I51eecfe4642974df6106b9084a0b144835a4997a
> BUG: 1411228
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: https://review.gluster.org/16417
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Bhat 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I2087516383bd132c59bbab98eda8f2243a2163fe
BUG: 1424973
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/16686
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Shyamsundar Ranganathan

glusterd, readdir-ahead: Fix backward incompatibility

2017-02-19T14:18:08+00:00

Backport of https://review.gluster.org/#/c/16657/

Issue:
Any opion is spcified in two places: In the options[] of xlator
itself and glusterd-volume-set.c. The default value of this option
can be specified in both the places. If its specified only in xlator
then the volfile generated will not have the option and default value,
it will be assigned during graph initialization.
With patch [1] the option rda-request-size was changed from INT to SIZET
type, and default was changed from 131072 to 128KB, but was specified
only in the readdir-ahead.c. Thus with this patch alone the volfile
entry for readdir-ahead looks like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    subvolumes patchy-read-ahead
end-volume

With patch [2], the default of option rda-request-size was specified
in glusterd-volume-set.c as well(as it was necessary fr parallel readdir).
With this patch the readdir entry in the volfile will look like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    option rda-cache-limit 10MB
    option rda-request-size 128KB
    option parallel-readdir off
    subvolumes patchy-read-ahead
end-volume


Now consider the server has both these patches and client doesn't.
Server will generate a volfile with entry:

The old clients which thought the option rda-request-size is of type
INT will now recieve the value 128KB which it willn't understand,
and hence fail the mount.

The issue is seen only with the combination of [1] and [2].

Solution:
Instead of specifying 128KB as default in glusterd we specify 131072
so that the old clients will interpret as INT and new ones as 128KB

Credits: Raghavendra G

> Reviewed-on: https://review.gluster.org/16657
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Atin Mukherjee 

Change-Id: I0c269a5890957fd8a38e9a05bdec088645a6688a
BUG: 1423412
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16658
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd : Fix for error mesage while detaching peers

2017-02-17T15:18:49+00:00

When peer is detached from a cluster, an error log is being
generated in glusterd.log -"Failed to reconfigure all daemon
services". This log is seen in the originator node where the
detach is issued.

This happens in two cases.
Case 1: Detach peer with no volume been created in cluster.
Case 2: Detach peer after deleting all the volumes which were
created but never started.
In any one of the above two cases, in glusterd_check_files_identical()
GlusterD fails to retrieve nfs-server.vol file from /var/lib/glusterd/nfs
which gets created only when a volume is in place and and is started.

With this fix both the above cases have been handled by added
validation to skip reconfigure if there is no volume in started
state.

> Reviewed-on: https://review.gluster.org/16607
> Smoke: Gluster Build System 
> Tested-by: Atin Mukherjee 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 

(cherry picked from commit be44a1bd519af69b21acf682b0908d4d695f868e)

Change-Id: I039c0840e3d61ab54575e1e00c6a6a00874d84c0
BUG: 1423406
Signed-off-by: Gaurav Yadav 
Reviewed-on: https://review.gluster.org/16645
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Samikshan Bairagya 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

rpcsvc: Add rpchdr and proghdr to iobref before submitting to transport

2017-02-16T14:46:53+00:00

Backport of https://review.gluster.org/16613

Issue:
When fio is run on multiple clients (each client writes to its own files),
and meanwhile the clients does a readdirp, thus the client which did
a readdirp will now recieve the upcalls. In this scenario the client
disconnects with rpc decode failed error.

RCA:
Upcall calls rpcsvc_request_submit to submit the request to socket:
rpcsvc_request_submit currently:
rpcsvc_request_submit () {
   iobuf = iobuf_new
   iov = iobuf->ptr
   fill iobuf to contain xdrised upcall content - proghdr
   rpcsvc_callback_submit (..iov..)
   ...
   if (iobuf)
       iobuf_unref (iobuf)
}

rpcsvc_callback_submit (... iov...) {
   ...
   iobuf = iobuf_new
   iov1 = iobuf->ptr
   fill iobuf to contain xdrised rpc header - rpchdr
   msg.rpchdr = iov1
   msg.proghdr = iov
   ...
   rpc_transport_submit_request (msg)
   ...
   if (iobuf)
       iobuf_unref (iobuf)
}

rpcsvc_callback_submit assumes that once rpc_transport_submit_request()
returns the msg is written on to socket and thus the buffers(rpchdr, proghdr)
can be freed, which is not the case. In especially high workload,
rpc_transport_submit_request() may not be able to write to socket immediately
and hence adds it to its own queue and returns as successful. Thus, we have
use after free, for rpchdr and proghdr. Hence the clients gets garbage rpchdr
and proghdr and thus fails to decode the rpc, resulting in disconnect.

To prevent this, we need to add the rpchdr and proghdr to a iobref and send
it in msg:
   iobref_add (iobref, iobufs)
   msg.iobref = iobref;
The socket layer takes a ref on msg.iobref, if it cannot write to socket and
is adding to the queue. Thus we do not have use after free.

Thank You for discussing, debugging and fixing along:
Prashanth Pai 
Raghavendra G 
Rajesh Joseph 
Kotresh HR 
Mohammed Rafi KC 
Soumya Koduri 

> Reviewed-on: https://review.gluster.org/16613
> Reviewed-by: Prashanth Pai 
> Smoke: Gluster Build System 
> Reviewed-by: soumya k 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Ifa6bf6f4879141f42b46830a37c1574b21b37275
BUG: 1422363
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16623
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

glusterd: add a cli command to trigger a statedump on a client

2017-02-14T01:53:10+00:00

With this, we will be able to trigger statedumps on remote Gluster
clients, mainly targetted for applications using libgfapi.

Design:
SIGUSR signal is the most comman way of taking a statedump in Gluster.
But it cannot be used for libgfapi based processes, as the process
loading the library might have already consumed SIGUSR signal. Hence
going by the command way.

One has to issue a Gluster command to initiate a statedump on the
libgfapi based client. The command takes hostname and PID as an
argument. All the glusterds in the cluster, check if they are connected
to the specified hostname, and send an RPC request to all the connected
clients from that hostname (via the mgmt connection).

> URL: http://review.gluster.org/16357
> BUG: 1169302
> Signed-off-by: Poornima G 
> [ndevos: minor fixes and split patch in smaller pieces]
> Reviewed-on-master: https://review.gluster.org/9228
> Reviewed-by: Niels de Vos 
> Tested-by: Niels de Vos 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Samikshan Bairagya 

BUG: 1418981
Change-Id: Icbe4d2f026b32a2c7d5535e1bfb2cdaaff042e91
Signed-off-by: Shyam 
Reviewed-on: https://review.gluster.org/16601
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd: keep snapshot bricks separate from regular ones

2017-02-10T19:23:38+00:00

The problem here is that a volume's transport options can change, but
any snapshots' bricks don't follow along even though they're now
incompatible (with respect to multiplexing).  This was causing the
USS+SSL test to fail.  By keeping the snapshot bricks separate
(though still potentially multiplexed with other snapshot bricks
including those for other volumes) we can ensure that they remain
unaffected by changes to their parent volumes.

Also fixed various issues with how the test waits (or more precisely
didn't) for various events to complete before it continues.

Backport of:
> Change-Id: Iab4a8a44fac5760373fac36956a3bcc27cf969da
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/16544
`
Change-Id: I91c73e3fdf20d23bff15fbfcc03a8a1922acec27
BUG: 1418091
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16597
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

glusterd: put null check for mod_dict in build_shd_graph

2017-02-10T14:05:28+00:00

>Reviewed-on: https://review.gluster.org/16573
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Ravishankar N 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Prashanth Pai 
>Smoke: Gluster Build System 
>(cherry picked from commit 824a48fb5ce20e26ed78b26afd81ec876bc826ec)

Change-Id: I849f5f7eb5932d2dc2a76f8abc61e41231f9883e
BUG: 1420987
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16590
Reviewed-by: Ravishankar N 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

trash: fix problem with trash feature under multiplexing

2017-02-10T11:44:01+00:00

With multiplexing, the trash translator gets a reconfigure call before
a notify(CHILD_UP).  In this case, priv->trash_itable was not yet
initialized, so the reconfigure would get a SEGV.  Moving the itable
allocation to init seems to fix it, so trash can be reenabled.

Backport of:
> Change-Id: I21ac2d7fc66bac1bc4ec70fbc8bae306d73ac565
> BUG: 1420434
> Reviewed-on: https://review.gluster.org/16567

Change-Id: I43a6de6ac5070848619c5f905f075e4a4099c1bd
BUG: 1420808
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16582
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Anoop C S 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

glusterd: set default GLUSTERD_QUORUM_RATIO_KEY value to 51

2017-02-10T11:43:13+00:00

The default value of GLUSTERD_QUORUM_RATIO_KEY is set to 0, while this
doesn't harm any functionality as the actual quorum calculation logic
always take the factor that if the key is not set in the dictionary the
value is considered to be 51 but from an user perspective it gives an
incorrect output through volume get where quorum-ratio is always shown as
0 until reconfigured.

>Reviewed-on: https://review.gluster.org/16572
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Jeff Darcy 

Change-Id: I087712fa4736055859f29cc3bdbb3b111cbe7224
BUG: 1420989
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16592
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan