glusterfs.git/xlators/mgmt/glusterd/src/glusterd-utils.h, branch v3.10.8

glusterd: Make reset-brick work correctly if brick-mux is on

2017-05-12T19:59:02+00:00

Reset brick currently kills of the corresponding brick process.
However, with brick multiplexing enabled, stopping the brick
process would render all bricks attached to it unavailable. To
handle this correctly, we need to make sure that the brick process
is terminated only if brick-multiplexing is disabled. Otherwise,
we should send the GLUSTERD_BRICK_TERMINATE rpc to the respective
brick process to detach the brick that is to be reset.

> Signed-off-by: Samikshan Bairagya 
> Reviewed-on: https://review.gluster.org/17128
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 

(cherry picked from commit 74383e3ec6f8244b3de9bf14016452498c1ddcf0)

Change-Id: I69002d66ffe6ec36ef48af09b66c522c6d35ac58
BUG: 1449934
Signed-off-by: Samikshan Bairagya 
Reviewed-on: https://review.gluster.org/17253
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd: cleanup pidfile on pmap signout

2017-05-10T10:42:07+00:00

This patch ensures
1. brick pidfile is cleaned up on pmap signout
2. pmap signout evemt is sent for all the bricks when a brick process
shuts down.

>Reviewed-on: https://review.gluster.org/17168
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit 3d35e21ffb15713237116d85711e9cd1dda1688a)

Change-Id: I7606a60775b484651d4b9743b6037b40323931a2
BUG: 1449002
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/17209
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

glusterd: add a cli command to trigger a statedump on a client

2017-02-14T01:53:10+00:00

With this, we will be able to trigger statedumps on remote Gluster
clients, mainly targetted for applications using libgfapi.

Design:
SIGUSR signal is the most comman way of taking a statedump in Gluster.
But it cannot be used for libgfapi based processes, as the process
loading the library might have already consumed SIGUSR signal. Hence
going by the command way.

One has to issue a Gluster command to initiate a statedump on the
libgfapi based client. The command takes hostname and PID as an
argument. All the glusterds in the cluster, check if they are connected
to the specified hostname, and send an RPC request to all the connected
clients from that hostname (via the mgmt connection).

> URL: http://review.gluster.org/16357
> BUG: 1169302
> Signed-off-by: Poornima G 
> [ndevos: minor fixes and split patch in smaller pieces]
> Reviewed-on-master: https://review.gluster.org/9228
> Reviewed-by: Niels de Vos 
> Tested-by: Niels de Vos 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Samikshan Bairagya 

BUG: 1418981
Change-Id: Icbe4d2f026b32a2c7d5535e1bfb2cdaaff042e91
Signed-off-by: Shyam 
Reviewed-on: https://review.gluster.org/16601
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

Tier: remove warning related to the enum

2017-02-08T14:16:04+00:00

        back-port of : https://review.gluster.org/#/c/16539/

PROBLEM: In the tier as a service patch the enums for tier (from
gf1_op_command and gf_defrag_command) are put into a single enum
gf_defrag_command which causes a warning that will make the build
fail.

FIX: send both the enum and eliminate the warning.

>Change-Id: I899ff622dfb07134e6459aa65f65ea7252765293
>BUG: 1418973
>Signed-off-by: hari gowtham 
>Reviewed-on: https://review.gluster.org/16539
>Smoke: Gluster Build System 
>Tested-by: hari gowtham 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Atin Mukherjee 

Change-Id: I8d2ec89b8689066091ab58406d225e1058f435cf
BUG: 1419846
Signed-off-by: hari gowtham 
Reviewed-on: https://review.gluster.org/16553
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System

glusterd : do not load io-threads in client graph for replicate volumes

2017-02-06T14:53:33+00:00

client.io-threads has been turned on by default from release-3.9 onwards,
however this has an adverse effects on replicate volumes due to the design
limitations on replications, till that gets addressed through server side
replication as a preventive measure it is wiser not to load io-threads in the
client graph for replicate volumes.

>Reviewed-on: https://review.gluster.org/16502
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Prashanth Pai 
>Reviewed-by: Ravishankar N 
>Reviewed-by: Jeff Darcy 

Change-Id: Ibc576d4517da23fcdf55c6f4d17b90152a8817d7
BUG: 1419305
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16545
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Ravishankar N

core: run many bricks within one glusterfsd process

2017-02-02T00:54:58+00:00

This patch adds support for multiple brick translator stacks running in
a single brick server process.  This reduces our per-brick memory usage
by approximately 3x, and our appetite for TCP ports even more.  It also
creates potential to avoid process/thread thrashing, and to improve QoS
by scheduling more carefully across the bricks, but realizing that
potential will require further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global
option.  By default it's off, and bricks are started in separate
processes as before.  If multiplexing is enabled, then *compatible*
bricks (mostly those with the same transport options) will be started in
the same process.

Backport of:
> Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/14763

Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8
BUG: 1418091
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16496
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

tier : Tier as a service

2017-01-17T04:49:47+00:00

tierd is implemented by separating from rebalance process.

The commands affected:

1) Attach tier will trigger this process instead of old one
2) tier start and tier start force will also trigger this process.
3) volume status [tier] will show tier daemon as a process instead
of task and normal tier status and tier detach status works.
4) tier stop implemented.
5) detach tier implemented separately along with new detach tier
status
6) volume tier volname status will work using the changes.
7) volume set works

This patch has separated the tier translator from the legacy
DHT rebalance code. It now sends the RPCs from the CLI
to glusterd separate to the DHT rebalance code.
The daemon is now a service, similar to the snapshot daemon,
and can be viewed using the volume status command.

The code for the validation and commit phase are the same
as the earlier tier validation code in DHT rebalance.

The “brickop” phase has been changed so that the status
command can use this framework.

The service management framework is now used.
DHT rebalance does not use this framework.

This service framework takes care of :

*) spawning the daemon, killing it and other such processes.
*) volume set options , which are written on the volfile.
*) restart and reconfigure functions. Restart is to restart
the daemon at two points
        1)after gluster goes down and comes up.
        2) to stop detach tier.
*) reconfigure is used to make immediate volfile changes.
By doing this, we don’t restart the daemon.
it has the code to rewrite the volfile for topological
changes too (which comes into place during add and remove brick).

With this patch the log, pid, and volfile are separated
and put into respective directories.

Change-Id: I3681d0d66894714b55aa02ca2a30ac000362a399
BUG: 1313838
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/13365
Smoke: Gluster Build System 
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Atin Mukherjee

glusterd: Get maximum supported op-version in a cluster

2017-01-09T05:16:12+00:00

gluster volume get  cluster.opversion gives us the current
op-version on which the cluster is operating. There is no command
that lets the user know the maximum supported op-version that the
cluster can run on.

This patch adds a new global option cluster.max-op-version, that
can be used to retrieve the maximum supported op-version in a
cluster.

Usage:
	# gluster volume get all cluster.max-op-version

Example output:

Option                                  Value
------                                  -----
cluster.max-op-version                  30900

NOTE: The only way to test this feature for now is to set the
GD_OP_VERSION_MAX macro to different values (30800 for 3.8,30900
for 3.9, and so on) and rebuild glusterd. Since the regression test
framework currently doesn't have support to simulate these tests,
there are no accompanying regression tests for this feature. It
should be possible to add tests once glusto comes in and makes it
easier to run a heterogeneous cluster.

Change-Id: I547480ee5e7912664784643e436feb198b6d16d0
BUG: 1365822
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/16283
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd, cli: Get global options through volume get functionality

2016-12-31T04:47:27+00:00

Currently it is not possible to retrieve values of global options
by using the 'gluster volume get' functionality if there are no
volumes present. In order to get the global options one has to use
'gluster volume get' with a specific volume name. This usage makes
the illusion as though the option is set only on one volume, which
is incorrect. When setting the global options, 'gluster volume set'
provides a way to set them using the volume name as 'all'.

Similarly, retrieving the global options should be made possible by
using the volume name 'all' with the 'gluster volume get'
functionality. This patch adds that functionality to 'volume get'

Usage:
	# gluster volume get all

glusterd : Introduce reset brick

2016-08-30T02:55:53+00:00

The command basically allows replace brick with src and
dst bricks as same.

Usage:
gluster v reset-brick   start
This command kills the brick to be reset. Once this command is run,
admin can do other manual operations that they need to do,
like configuring some options for the brick. Once this is done,
resetting the brick can be continued with the following options.

gluster v reset-brick    commit {force}

Does the job of resetting the brick. 'force' option should be used
when the brick already contains volinfo id.

Problem: On doing a disk-replacement of a brick in a replicate volume
the following 2 scenarios may occur :

a) there is a chance that reads are served from this replaced-disk brick,
which leads to empty reads. b) potential data loss if next writes succeed
only on replaced brick, and heal is done to other bricks from this one.

Solution: After disk-replacement, make sure that reset-brick command is
run for that brick so that pending markers are set for the brick and it
is not chosen as source for reads and heal. But, as of now replace-brick
for the same brick-path is not allowed. In order to fix the above
mentioned problem, same brick-path replace-brick is needed.
With this patch reset-brick commit {force} will be allowed even when
source and destination  are identical as long as
1) destination brick is not alive
2) source and destination brick have the same brick uuid and path.
Also, the destination brick after replace-brick will use the same port
as the source brick.

Change-Id: I440b9e892ffb781ea4b8563688c3f85c7a7c89de
BUG: 1266876
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/12250
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ashish Pandey 
Reviewed-by: Pranith Kumar Karampuri