glusterfs.git/xlators/mgmt/glusterd/src/glusterd-utils.c, branch v3.8.12

glusterd/geo-rep: Fix snapshot create in geo-rep setup

2017-04-28T09:39:20+00:00

glusterd persists geo-rep sessions in glusterd
info file which is represented by dictionary
'volinfo->gsync_slaves' in memory. Glusterd also
maintains in memory active geo-rep sessions in
dictionary 'volinfo->gsync_active_slaves' whose key
is "::".

When glusterd is restarted while the geo-rep sessions
are active, it builds the 'volinfo->gsync_active_slaves'
from persisted glusterd info file. Since slave volume
uuid is added to "voinfo->gsync_slaves" with the commit
"http://review.gluster.org/13111", it builds it with key
":::" which is
wrong. So during snapshot pre-validation which checks
whether geo-rep is active or not, it always says it is
ACTIVE, as geo-rep stop would not deleted this key.
Fixed the same in this patch.


> BUG: 1443977
> Signed-off-by: Kotresh HR 
> Reviewed-on: https://review.gluster.org/17093
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
(cherry picked from commit f071d2a285ea4802fe8f328f9f275180983fbbba)

Change-Id: I185178910b4b8a62e66aba406d88d12fabc5c122
BUG: 1445213
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17109
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

glusterd: support filesystems with dynamic inode sizes

2017-04-07T11:57:22+00:00

btrfs and zfs are two filesystems that do not have fixed sizes for
inodes. Instead of logging an error, skip checking and mark the size as
"N/A" like other properties that can not be reported.

The error message that was reported by users on the mailinglist shows up
like:

  [glusterd-utils.c:5458:glusterd_add_inode_size_to_dict] 0-management: could not find (null) to getinode size for /dev/vdb (btrfs): (null) package missing?

Cherry picked from commit 12921693b572f642156d3167d1c92d3449dfc8ec:
> Change-Id: Ib10b7a3669f2f4221075715d9fd44ce1ffc35324
> Reported-by: Arman Khalatyan 
> URL: http://lists.gluster.org/pipermail/gluster-users/2017-March/030189.html
> BUG: 1433425
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/16867
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Prashanth Pai 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

Change-Id: Ib10b7a3669f2f4221075715d9fd44ce1ffc35324
Reported-by: Arman Khalatyan 
BUG: 1436412
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16960
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Zhou Zhengping

glusterd: ignore return code of glusterd_restart_bricks

2017-02-21T03:33:53+00:00

When GlusterD is restarted on a multi node cluster, while syncing the
global options from other GlusterD, it checks for quorum and based on
which it decides whether to stop/start a brick. However we handle the
return code of this function in which case if we don't want to start any
bricks the ret will be non zero and we will end up failing the import
which is incorrect.

Fix is just to ignore the ret code of glusterd_restart_bricks ()

>Reviewed-on: https://review.gluster.org/16574
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Samikshan Bairagya 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit 55625293093d485623f3f3d98687cd1e2c594460)

Change-Id: I37766b0bba138d2e61d3c6034bd00e93ba43e553
BUG: 1420993
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16594
Smoke: Gluster Build System 
Reviewed-by: Samikshan Bairagya 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

glusterd: clean up old port and allocate new one on every restart

2016-11-23T10:37:16+00:00

    Backport of http://review.gluster.org/#/c/15005/9.

GlusterD as of now was blindly assuming that the brick port which was
already allocated would be available to be reused and that assumption
is absolutely wrong.

Solution : On first attempt, we thought GlusterD should check if the
already allocated brick ports are free, if not allocate new port and
pass it to the daemon. But with that approach there is a possibility
that if PMAP_SIGNOUT is missed out, the stale port will be given back
to the clients where connection will keep on failing. Now given the
port allocation always start from base_port, if everytime a new port
has to be allocated for the daemons, the port range will still be
under control. So this fix tries to clean up old port using
pmap_registry_remove () if any and then goes for pmap_registry_alloc ()

This patch is being ported to 3.8 branch because, the brick process
blindly re-using old port, without registering with the pmap server,
causes snapd daemon to not start properly, even though snapd registers
with the pmap server. With this patch, all the brick processes and
snapd will register with the pmap server to either get the same port,
or a new port, and avoid port collision.

> Reviewed-on: http://review.gluster.org/15005
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Avra Sengupta 

(cherry picked from commit c3dee6d35326c6495591eb5bbf7f52f64031e2c4)

Change-Id: If54a055d01ab0cbc06589dc1191d8fc52eb2c84f
BUG: 1369766
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15308
Tested-by: Avra Sengupta 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos

snapshot: Fix for memory leaks in snapshot code path

2016-11-04T06:05:16+00:00

> Reviewed-on: http://review.gluster.org/15668
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Rajesh Joseph 

(cherry picked from commit 48a4c4e525665115f7e8c478d3bf51764427378d)

Change-Id: Idc2cb16574d166e3c0ee1f7c3a485f1acb19fc8c
BUG: 1388354
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/15720
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Rajesh Joseph

glusterd/geo-rep: Handle empty monitor.status during upgrade

2016-08-19T10:46:52+00:00

Problem:
Consider geo-replication is in Stopped state.
Following which, glusterfs is upgraded (where monitor.status is the 
new status file).

Now, When geo-replication status command is run,
empty monitor status file gets created.

Now, if glusterd is restarted, it reads empty monitor status
and  starts geo-replication session. This is incorrect as session
was in Stopped state earlier.

Solution:
If monitor status is empty, error out and avoid
starting geo-replication session.

Note: if monitor status is empty, geo-rep session is displayed
as Stopped state.

Change-Id: Ifb3db896e5ed92b927764cf1163503765cb08bb4
BUG: 1368053
Signed-off-by: Saravanakumar Arumugam 
> Reviewed-on: http://review.gluster.org/14830
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 
(cherry picked from commit f938b3a26ffab9482d5f910ee76d2bb2b370517f)
Reviewed-on: http://review.gluster.org/15196
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

glusterd: Fix volume restart issue upon glusterd restart

2016-08-18T08:39:34+00:00

http://review.gluster.org/#/c/14758/ introduces a check in
glusterd_restart_bricks that makes sure that if server quorum is
enabled and if the glusterd instance has been restarted, the bricks
do not get started. This prevents bricks which have been brought
down purposely, say for maintainence, from getting started
upon a glusterd restart. However this change introduced regression
for a situation that involves multiple volumes. The bricks from
the first volume get started, but then for the subsequent volumes
the bricks do not get started. This patch fixes that by setting
the value of conf->restart_done to _gf_true only after bricks are
started correctly for all volumes.

> Reviewed-on: http://review.gluster.org/15183
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 

(cherry picked from commit dd8d93f24a320805f1f67760b2d3266555acf674)

Change-Id: I2c685b43207df2a583ca890ec54dcccf109d22c3
BUG: 1366813
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/15186
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

changelog/rpc: Fix rpc_clnt_t mem leaks

2016-07-25T06:40:30+00:00

Backport of: http://review.gluster.org/13658

PROBLEM:
   1. Freeing up rpc_clnt object might lead to crashes. Well,
      it was not a necessity to free rpc-clnt object till now
      because all the existing use cases needs to reconnect
      back on disconnects. Hence timer code was not taking
      ref on rpc-clnt object.

      Glusterd had some use-cases that led to crash due to
      ping-timer and they fixed only those code paths that
      involve ping-timer.

      Now, since changelog has an use-case where rpc-clnt
      need to be freed up, we need to fix timer code to take
      refs

   2. In changelog, because of issue 1, only mydata was being
      freed which is incorrect. And there are races where
      rpc-clnt object would access the freed mydata which
      would lead to crashes.

      Since changelog xlator resides on brick side and is long
      living process, if multiple libgfchangelog consumers
      register to changelog and disconnect/reconnect mulitple
      times, it would result in leak of 'rpc-clnt' object
      for every connect/disconnect.

SOLUTION:
   1. Handle ref/unref of 'rpc_clnt' structure in timer
      functions properly.
   2. In changelog, unref 'rpc_clnt' in RPC_CLNT_DISCONNECT
      after disabling timers and free mydata on RPC_CLNT_DESTROY.

RPC SETUP IN CHANGELOG:
   1. changelog xlator initiates rpc server say 'changelog_rpc_server'
   2. libgfchangelog initiates one rpc server say 'libgfchangelog_rpc_server'
   3. libgfchangelog initiates rpc client and connects to 'changelog_rpc_server'
   4. In return changelog_rpc_server initiates a rpc client and connects back
      to 'libgfchangelog_rpc_server'

REF/UNREF HANDLING IN TIMER FUNCTIONS:
Let's say rpc clnt refcount = 1
   1. Take the ref before reigstering callback to timer queue
           >>>>  rpc_clnt_ref (say ref count becomes = 2)
   2. Register a callback to timer say 'callback1'
   3. If register fails:
           >>>> rpc_clnt_unref (ref count = 1)
   4. On timer expiration, 'callback1' gets called. So unref rpc clnt at the end
      in 'callback1'. This is corresponding to ref taken in step 1
           >>>> rpc_clnt_unref (ref count = 1)
   5. The cycle from step-1 to step-4 continues....until timer cancel event happens
   6. timer cancel of say 'callback1'
           If timer cancel fails:
                 Do nothing, Step-4 would have unrefd
           If timer cancel succeeds:
                 >>>> rpc_clnt_unref (ref count = 1)

Change-Id: I91389bc511b8b1a17824941970ee8d2c29a74a09
BUG: 1359364
Signed-off-by: Kotresh HR 
(cherry picked from commit 637ce9e2e27e9f598a4a6c5a04cd339efaa62076)
Reviewed-on: http://review.gluster.org/14994
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

feature/bitrot: Show whether scrub is in progress/idle

2016-07-18T10:53:50+00:00

Backport of http://review.gluster.org/14864/

Bitrot scrub status shows whether the scrub is paused
or active. It doesn't show whether the scrubber is
actually scrubbing or waiting in the timer wheel
for the next schedule. This patch shows this status
with "In Progress" and "Idle" respectively.

Change-Id: I995d8553d1ff166503ae1e7b46282fc3ba961f0b
BUG: 1355639
Signed-off-by: Kotresh HR 
(cherry picked from commit f4757d256e3e00132ef204c01ed61f78f705ad6b)
Reviewed-on: http://review.gluster.org/14901
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

glusterd: Fix gsyncd upgrade issue

2016-07-15T16:47:44+00:00

Backport of http://review.gluster.org/14898/

Problem:
    gluster upgrade is not generating new volfiles

Cause:
During upgrade, "glusterd --xlator-option *.upgrade=on -N"
is run to generate new volfiles. It is run post 'glusterfs'
rpm installation. The above command fails during upgrade
if geo-replication is installed. This is because on
glusterd start 'gsyncd' binary is called to configure
geo-replication related stuff. Since 'glusterfs' rpm is
installed prior to 'geo-rep' rpm, the 'gsyncd' binary
used to glusterd upgrade command is of old version and
hence it fails before generating new volfiles.

Solution:
Don't call geo-replication configure during upgrade/downgrade.
Geo-replication configuration happens during start of glusterd
after upgrade.

Change-Id: Id58ea44ead9f69982f86fb68dc5b9ee3f6cd11a1
BUG: 1356439
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14898
(cherry picked from commit 1b998788ece8c8b52657e8b9aae65d3279690c5b)
Reviewed-on: http://review.gluster.org/14916
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee