glusterfs.git/xlators/mgmt/glusterd/src/glusterd-peer-utils.c, branch v4.0dev

cli, glusterd: Address issues in get-state cli output

2016-10-20T10:12:26+00:00

This fixes the following data points:
1. Volume type
2. Peer state
3. List of other hostnames for a peer
4. Data unit information for rebalance

The following data points are removed:
1. Mount options and filesystem types for bricks
2. global-option-version from list of global options

The following data points are added:
1. Replica Count
2. Tier type for bricks belonging to hot/cold tier

Change-Id: I5011250e863fdc4929b203cdb345d79b2f16c6a5
BUG: 1385839
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/15662
Reviewed-by: mohammed rafi  kc 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd (peer-utils): fix unused variable warnings/errors

2016-08-29T16:31:54+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: I99e818d71e72aaa2dfb629f00a5245365f582b99
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15270
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai

glusterd/cli: cli to get local state representation from glusterd

2016-08-26T15:23:37+00:00

Currently there is no existing CLI that can be used to get the
local state representation of the cluster as maintained in glusterd
in a readable as well as parseable format.

The CLI added has the following usage:

 # gluster get-state [daemon] [odir ] [file ]

This would dump data points that reflect the local state
representation of the cluster as maintained in glusterd (no other
daemons are supported as of now) to a file inside the specified
output directory. The default output directory and filename is
/var/run/gluster and glusterd_state_ respectively. The
option for specifying the daemon name leaves room to add support for
other daemons in the future. Following are the data points captured
as of now to represent the state from the local glusterd pov:

 * Peer:
    - Primary hostname
    - uuid
    - state
    - connection status
    - List of hostnames

 * Volumes:
    - name, id, transport type, status
    - counts: bricks, snap, subvol, stripe, arbiter, disperse,
 redundancy
    - snapd status
    - quorum status
    - tiering related information
    - rebalance status
    - replace bricks status
    - snapshots

 * Bricks:
    - Path, hostname (for all bricks these info will be shown)
    - port, rdma port, status, mount options, filesystem type and
signed in status for bricks running locally.

 * Services:
    - name, online status for initialised services

 * Others:
    - Base port, last allocated port
    - op-version
    - MYUUID

Change-Id: I4a45cc5407ab92d8afdbbd2098ece851f7e3d618
BUG: 1353156
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/14873
Reviewed-by: Avra Sengupta 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

glusterd: spawn daemons from init() on a single or two node setup

2016-07-05T11:51:28+00:00

Allow glusterd to spawn the daemons at the time of initialization when peer
count is less than 2. This is required if user wants to set up a two node
cluster with out server side quorum and want the bricks to come up on a node
where the other node is down, however the behaviour will be overriden when
server side quorum is enabled.

Change-Id: I21118e996655822467eaf329f638eb9a8bf8b7d5
BUG: 1352277
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/14848
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

glusterd: fail volume delete if one of the node is down

2016-06-10T07:30:57+00:00

Deleting a volume on a cluster where one of the node in the cluster is down is
buggy since once that node comes back the resync of the same volume will happen.
Till we bring in the soft delete feature tracked in
http://review.gluster.org/12963 this is a safe guard to block the volume
deletion.

Change-Id: I9c13869c4a7e7a947f88842c6dc6f231c0eeda6c
BUG: 1344407
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/14681
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaushal M 
NetBSD-regression: NetBSD Build System

glusterd: Porting left out log messages to new framework

2015-06-27T06:32:01+00:00

Change-Id: I70d40ae3b5f49a21e1b93f82885cd58fa2723647
BUG: 1235538
Signed-off-by: Nandaja Varma 
Reviewed-on: http://review.gluster.org/11388
Tested-by: NetBSD Build System 
Reviewed-by: Anand Nekkunti 
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M

utils/glusterd: Porting to new logging framwork

2015-06-11T14:20:33+00:00

Change-Id: Iacb30eb675693bcdb3ef9de8e7f43892c5d79f6d
BUG: 1194640
Signed-off-by: Nandaja Varma 
Reviewed-on: http://review.gluster.org/9967
Tested-by: NetBSD Build System 
Reviewed-by: Poornima G 
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Kaushal M

glusterd: Use generation number to find peerinfo in RPC notifications

2015-05-07T07:10:43+00:00

The generation number for each peerinfo object is unique. It can be used
to find the exact peerinfo object, which is required for peer RPC
notifications.

Using hostname and uuid matching to find peerinfos can return incorrect
peerinfos to be returned in certain cases like multi network peer probe.
This could cause updates to happen to incorrect peerinfos.

Change-Id: Ia0aada8214fd6d43381e5afd282e08d53a277251
BUG: 1215018
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10495
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Atin Mukherjee

glusterd: Replace transaction peers lists

2015-04-13T06:30:02+00:00

Transaction peer lists were used in GlusterD to peers belonging to a
transaction. This was needed to prevent newly added peers performing
partial transactions, which could be incorrect.

This was accomplished by creating a seperate transaction peers list at
the beginning of every transaction. A transaction peers list referenced
the peerinfo data structures of the peers which were present at the
beginning of the transaction. RCU protection of peerinfos referenced by
the transaction peers list is a hard problem and difficult to do
correctly.

To have proper RCU protection of peerinfos, the transaction peers lists
have been replaced by an alternative method to identify peers that
belong to a transaction. The alternative method is to the global peers
list along with generation numbers to identify peers that should belong
to a transaction.

This change introduces a global peer list generation number, and a
generation number for each peerinfo object. Whenever a peerinfo object
is created, the global generation number is bumped, and the peerinfos
generation number is set to the bumped global generation.

With the above changes, the algorithm to identify peers belonging to a
transaction with RCU protection is as follows,
- At the beginning of a transaction, the current global generation
  number is saved
- To identify if a peers belonging to the transaction,
  - Start a RCU read critical section
  - For each peer in the global peers list,
    - If the peers generation number is not greater than the saved
      generation number, continue with the action on the peer
  - End the RCU read critical section

The above algorithm guarantees that,
- The peer list is not modified when a transaction is iterating through
  it
- The transaction actions are only done on peers that were present when
  the transaction started

But, as a transaction could iterate over the peers list multiple times,
the algorithm cannot guarantee that same set of peers will be selected
every time. A peer could get deleted between two iterations of the list
within a transaction. This problem existed with transaction peers list
as well, but unlike before now it will not lead to invalid memory access
and potential crashes. This problem will be addressed seprately.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  52ded5b Add timespec_cmp
  44aedd8 Add create timestamp to peerinfo
  7bcbea5 Fix some silly mistakes
  13e3241 Add start time to opinfo
  17a6727 Use timestamp comparisions to identify xaction peers instead
          of a xaction peer list
  3be05b6 Correct check for peerinfo age
  70d5b58 Use read-critical sections for peer list iteration
  ba4dbca Use peerinfo timestamp checks in op-sm instead of xaction peer
          list
  d63f811 Add more peer status checks when iterating peers list in
          glusterd-syncop
  1998a2a Timestamp based peer list traversal of mgmtv3 xactions
  f3c1a42 Remove transaction peer lists
  b8b08ee Remove unused labels
  32e5f5b Remove 'npeers' usage
  a075fb7 Remove 'npeers' from mgmt-v3 framework
  12c9df2 Use generation number instead of timestamps.
  9723021 Remove timespec_cmp
  80ae2c6 Remove timespec.h include
  a9479b0 Address review comments on 10147/4

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: I9be1033525c0a89276f5b5d83dc2eb061918b97f
BUG: 1205186
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10147
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Anand Nekkunti 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

Avoid conflict between contrib/uuid and system uuid

2015-04-04T17:48:35+00:00

glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.

Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.

A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.

BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos