glusterfs.git/xlators/mgmt/glusterd/src/glusterd-pmap.c, branch v3.13.0beta

glusterd: clean up portmap on brick disconnect

2017-10-31T04:36:44+00:00

GlusterD's portmap entry for a brick is cleaned up when a PMAP_SIGNOUT event is
initiated by the brick process at the shutdown. But if the brick process crashes
or gets killed through SIGKILL then this event is not initiated and glusterd
ends up with a stale port. Since GlusterD's portmap traversal happens both ways,
forward for allocation and backward for registry search, there is a possibility
that glusterd might end up running with a stale port for a brick which
eventually will end up with clients to fail to connect to the bricks.

Solution is to clean up the port entry in case the process is down as
part of the brick disconnect event. Although with this the handling
PMAP_SIGNOUT event becomes redundant in most of the cases, but this is
the safeguard method to avoid glusterd getting into the stale port
issues.

Change-Id: I04c5be6d11e772ee4de16caf56dbb37d5c944303
BUG: 1503246
Signed-off-by: Atin Mukherjee

glusterd: introduce max-port range

2017-08-17T04:02:20+00:00

glusterd.vol file always had an option (commented out) to indicate the
base-port to start the portmapper allocation. This patch brings in the
max-port configuration where one can limit the range of ports which
gluster can be allowed to bind.

Fixes: #305
Change-Id: Id7a864f818227b9530a07e13d605138edacd9aa9
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/18016
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
Reviewed-by: Gaurav Yadav

glusterd: Block brick attach request till the brick's ctx is set

2017-08-08T22:28:30+00:00

Problem: In multiplexing setup in a container environment we hit a race
where before the first brick finishes its handshake with glusterd, the
subsequent attach requests went through and they actually failed and
glusterd has no mechanism to realize it. This resulted into all the such
bricks not to be active resulting into clients not able to connect.

Solution: Introduce a new flag port_registered in glusterd_brickinfo
          to make sure about pmap_signin finish before the subsequent
          attach bricks can be processed.

Test:     To reproduce the issue followed below steps
          1) Create 100 volumes on 3 nodes(1x3) in CNS environment
          2) Enable brick multiplexing
          3) Reboot one container
          4) Run below command
             for v in ‛gluster v list‛
             do
               glfsheal $v | grep -i "transport"
             done
          After apply the patch command should not fail.

Note:   A big thanks to Atin for suggest the fix.

BUG: 1478710
Change-Id: I8e1bd6132122b3a5b0dd49606cea564122f2609b
Signed-off-by: Mohit Agrawal 
Reviewed-on: https://review.gluster.org/17984
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

glusterd: Introduce option to limit no. of muxed bricks per process

2017-07-10T04:33:19+00:00

This commit introduces a new global option that can be set to limit
the number of multiplexed bricks in one process.

Usage:
`# gluster volume set all cluster.max-bricks-per-process `

If this option is not set then multiplexing will happen for now
with no limitations set; i.e. a brick process will have as many
bricks multiplexed to it as possible. In other words the current
multiplexing behaviour won't change if this option isn't set to
any value.

This commit also introduces a brick process instance that contains
information about brick processes, like the number of bricks
handled by the process (which is 1 in non-multiplexing cases), list
of bricks, and port number which also serves as an unique identifier
for each brick process instance. The brick process list is
maintained in 'glusterd_conf_t'.

Updates: #151
Change-Id: Ib987d14ab0a4f6034dac01b73a4b2839f7b0b695
Signed-off-by: Samikshan Bairagya 
Reviewed-on: https://review.gluster.org/17469
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd: fix brick start race

2017-06-06T02:24:28+00:00

This commit tries to handle a race where we might end up trying to spawn
the brick process twice with two different set of ports resulting into
glusterd portmapper having the same brick entry in two different ports
which will result into clients to fail connect to bricks because of
incorrect ports been communicated back by glusterd.

In glusterd_brick_start () checking brickinfo->status flag to identify
whether a brick has been started by glusterd or not is not sufficient as
there might be cases where while glusterd restarts
glusterd_restart_bricks () will be called through glusterd_spawn_daemons
() in synctask and immediately glusterd_do_volume_quorum_action () with
server-side-quorum set to on will again try to start the brick and in
case if the RPC_CLNT_CONNECT event for the same brick  hasn't been processed by
glusterd by that time, brickinfo->status will still be marked as
GF_BRICK_STOPPED resulting into a reattempt to start the brick with a different
port and that would result portmap go for a toss and resulting clients to fetch
incorrect port.

Fix would be to introduce another enum value called GF_BRICK_STARTING in
brickinfo->status which will be set when a brick start is attempted by
glusterd and will be set to started through RPC_CLNT_CONNECT event. For
brick multiplexing, on attach brick request given the brickinfo->status
flag is marked to started directly this value will not have any effect.
Also this patch removes started_here flag as it looks to be redundant as
brickinfo->status.

Change-Id: I9dda1a9a531b67734a6e8c7619677867b520dcb2
BUG: 1457981
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/17447
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

glusterd: Fix regression wrt add-brick on replica count change

2017-06-01T10:46:38+00:00

tests/bugs/glusterd/bug-1406411-fail-add-brick-on-replica-count-change.t
was failing on centos machines with brick multiplexing enabled.
This is because detaching individual bricks manually from the
backend like it is done in the regression test framework by
'kill_brick', fails to send a RPC_CLNT_DISCONNECT to glusterd when
multiplexing is enabled. This causes the add-brick command to not
fail when one of the bricks are killed using kill_brick in the
regression test framework. To fix this, set the brick status to
GF_BRICK_STOPPED on the glusterd end during portmap signout. This
commit also sets the brick status in glusterd_brick_stop() function
so that the brick status is correctly set to 'stopped' even when the
function is called independently for individual bricks.

Change-Id: I4d6f7b579069d0cfa53cb2b0cff78876e1f31594
BUG: 1456898
Signed-off-by: Samikshan Bairagya 
Reviewed-on: https://review.gluster.org/17422
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System

glusterd: Don't spawn new glusterfsds on node reboot with brick-mux

2017-05-18T16:45:28+00:00

With brick multiplexing enabled, upon a node reboot new bricks were
not being attached to the first spawned brick process even though
there wasn't any compatibility issues.

The reason for this is that upon glusterd restart after a node
reboot, since brick services aren't running, glusterd starts the
bricks in a "no-wait" mode. So after a brick process is spawned for
the first brick, there isn't enough time for the corresponding pid
file to get populated with a value before the compatibilty check is
made for the next brick.

This commit solves this by iteratively waiting for the pidfile to be
populated in the brick compatibility comparison stage before checking
if the brick process is alive.

Change-Id: Ibd1f8e54c63e4bb04162143c9d70f09918a44aa4
BUG: 1451248
Signed-off-by: Samikshan Bairagya 
Reviewed-on: https://review.gluster.org/17307
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd: cleanup pidfile on pmap signout

2017-05-08T13:13:41+00:00

This patch ensures
1. brick pidfile is cleaned up on pmap signout
2. pmap signout evemt is sent for all the bricks when a brick process
shuts down.

Change-Id: I7606a60775b484651d4b9743b6037b40323931a2
BUG: 1444596
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/17168
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

glusterd: Fix removing pmap entry on rpc disconnect

2017-04-28T17:15:30+00:00

Problem:
The following line of code intended to remove pmap entry for the
connection during disconnects:

    pmap_registry_remove (this, 0, NULL, GF_PMAP_PORT_NONE, xprt);

However, no pmap entry will have it's type set to GF_PMAP_PORT_NONE
at any point in time. So a call to pmap_registry_search_by_xprt() in
pmap_registry_remove() will always fail to find a match.

Fix:
Optionally ignore pmap entry's type in pmap_registry_search_by_xprt().

BUG: 1193929
Change-Id: I705f101739ab1647ff52a92820d478354407264a
Signed-off-by: Prashanth Pai 
Reviewed-on: https://review.gluster.org/17129
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

core: Clean up pmap registry up correctly on volume/brick stop

2017-02-27T22:59:03+00:00

This commit changes the following:
1. In glusterfs_handle_terminate, send out individual pmap signout
requests to glusterd for every brick.
2. Add another parameter to glusterfs_mgmt_pmap_signout function to
pass the brickname that needs to be removed from the pmap registry.
3. Make sure pmap_registry_search doesn't break out from the loop
iterating over the list of bricks per port if the first brick entry
corresponding to a port is whitespaced out.
4. Make sure the pmap registry entries are removed for other
daemons like snapd.

Change-Id: I69949874435b02699e5708dab811777ccb297174
BUG: 1421590
Signed-off-by: Samikshan Bairagya 
Reviewed-on: https://review.gluster.org/16689
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Gaurav Yadav 
Reviewed-by: Jeff Darcy