glusterfs.git/xlators/mgmt/glusterd/src/glusterd-utils.h, branch v3.7.0beta1

glusterd: Replace transaction peers lists

2015-04-13T06:30:02+00:00

Transaction peer lists were used in GlusterD to peers belonging to a
transaction. This was needed to prevent newly added peers performing
partial transactions, which could be incorrect.

This was accomplished by creating a seperate transaction peers list at
the beginning of every transaction. A transaction peers list referenced
the peerinfo data structures of the peers which were present at the
beginning of the transaction. RCU protection of peerinfos referenced by
the transaction peers list is a hard problem and difficult to do
correctly.

To have proper RCU protection of peerinfos, the transaction peers lists
have been replaced by an alternative method to identify peers that
belong to a transaction. The alternative method is to the global peers
list along with generation numbers to identify peers that should belong
to a transaction.

This change introduces a global peer list generation number, and a
generation number for each peerinfo object. Whenever a peerinfo object
is created, the global generation number is bumped, and the peerinfos
generation number is set to the bumped global generation.

With the above changes, the algorithm to identify peers belonging to a
transaction with RCU protection is as follows,
- At the beginning of a transaction, the current global generation
  number is saved
- To identify if a peers belonging to the transaction,
  - Start a RCU read critical section
  - For each peer in the global peers list,
    - If the peers generation number is not greater than the saved
      generation number, continue with the action on the peer
  - End the RCU read critical section

The above algorithm guarantees that,
- The peer list is not modified when a transaction is iterating through
  it
- The transaction actions are only done on peers that were present when
  the transaction started

But, as a transaction could iterate over the peers list multiple times,
the algorithm cannot guarantee that same set of peers will be selected
every time. A peer could get deleted between two iterations of the list
within a transaction. This problem existed with transaction peers list
as well, but unlike before now it will not lead to invalid memory access
and potential crashes. This problem will be addressed seprately.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  52ded5b Add timespec_cmp
  44aedd8 Add create timestamp to peerinfo
  7bcbea5 Fix some silly mistakes
  13e3241 Add start time to opinfo
  17a6727 Use timestamp comparisions to identify xaction peers instead
          of a xaction peer list
  3be05b6 Correct check for peerinfo age
  70d5b58 Use read-critical sections for peer list iteration
  ba4dbca Use peerinfo timestamp checks in op-sm instead of xaction peer
          list
  d63f811 Add more peer status checks when iterating peers list in
          glusterd-syncop
  1998a2a Timestamp based peer list traversal of mgmtv3 xactions
  f3c1a42 Remove transaction peer lists
  b8b08ee Remove unused labels
  32e5f5b Remove 'npeers' usage
  a075fb7 Remove 'npeers' from mgmt-v3 framework
  12c9df2 Use generation number instead of timestamps.
  9723021 Remove timespec_cmp
  80ae2c6 Remove timespec.h include
  a9479b0 Address review comments on 10147/4

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: I9be1033525c0a89276f5b5d83dc2eb061918b97f
BUG: 1205186
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10147
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Anand Nekkunti 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

build: make contrib/uuid dependency optional

2015-04-10T11:39:52+00:00

On Linux systems we should use the libuuid from the distribution and not
bundle and statically link the contrib/uuid/ bits.

libglusterfs/src/compat-uuid.h has been introduced and should become an
abstraction layer for different UUID APIs. Non-Linux operating systems
should implement their compatibility layer there.

Once all operating systems have an implementation in compat-uuid.h, we
can remove contrib/uuid/ from the repository completely.

Change-Id: I345e5357644be2521685e00358bb8c83c4ea0577
BUG: 1206587
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10129
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: group server-quorum related code together

2015-04-01T13:07:14+00:00

Server-quorum implementation was spread in many files.  This patch
brings them all together into a single file, namely
glusterd-server-quorum.c. All exported functions are available via
glusterd-server-quorum.h

Change-Id: I8fd77114b5bc6b05127cb8a6a641e0295f0be7bb
BUG: 1205592
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/9492
Reviewed-by: Atin Mukherjee 
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M

glusterd: clean up global xaction_peer occurances

2015-03-26T07:12:32+00:00

With http://review.gluster.org/#/c/9972/ there is no more need to maintain
xaction_peers in glusterd_conf_t. This patch cleans up code for all the
occurances of xaction_peers.

Change-Id: I4fbf2df0fa9b8a8751029be36be7f76f6464cc76
BUG: 1204727
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/9980
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

glusterd: Maintain local xaction_peer list for op-sm

2015-03-26T07:10:45+00:00

http://review.gluster.org/9269 addresses maintaining local xaction_peers in
syncop and mgmt_v3 framework. This patch is to maintain local xaction_peers list
for op-sm framework as well.

Change-Id: Idd8484463fed196b3b18c2df7f550a3302c6e138
BUG: 1204727
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/9972
Reviewed-by: Anand Nekkunti 
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

glusterd: Prevent possible deadlock due to glusterd_quorum_count

2015-03-25T11:36:06+00:00

Also rename the macro glusterd_quorum_count to GLUSTERD_QUORUM_COUNT so
that it stands out as a macro.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  0fbd7ba Prevent possible deadlock due to glusterd_quorum_count
  5da3062 Rename glusterd_quorum_count to GLUSTERD_QUORUM_COUNT
  b3aa3c4 Enclose GLUSTERD_QUORUM_COUNT definition in a do-while block

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: Ic4b3949f303d72ce53e0139f62b83b8d13fb4e47
BUG: 1205186
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/9978
Reviewed-by: Atin Mukherjee 
Reviewed-by: Gaurav Kumar Garg 
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

cli/glusterd: cli command implementation for bitrot features

2015-03-19T01:34:07+00:00

CLI command for bitrot features.

volume bitrot  enable|disable

Above command will enable/disable bitrot feature for particular volume.

BUG: 1170075
Change-Id: Ie84002ef7f479a285688fdae99c7afa3e91b8b99
Signed-off-by: Gaurav Kumar Garg     
Signed-off-by: Anand nekkunti        
Signed-off-by: Dominic P Geevarghese 
Reviewed-on: http://review.gluster.org/9866
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Replace libglusterfs lists with liburcu lists

2015-03-04T07:50:22+00:00

This patch replaces usage of the libglusterfs lists data structures and
API in glusterd with the lists data structures and API from liburcu. The
liburcu data structes and APIs are a drop-in replacement for
libglusterfs lists.

All usages have been changed to keep the code consistent, and free from
confusion.

NOTE: glusterd_conf_t->xprt_list still uses the libglusterfs data
structures and API, as it holds rpc_transport_t objects, which is not a
part of glusterd and is not being changed in this patch.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  6dac576 Replace libglusterfs lists with liburcu lists
  a51b5ab Fix compilation issues
  d98a06f Fix merge issues
  a5d918e Remove merge remnant
  1cca113 More style cleanup
  1917be3 Address review comments on 9624/1
  8d10f13 Use cds_lists for glusterd_svc_t
  524ad5d Add rculist header in glusterd-conn-helper.c
  646f294 glusterd: add list_add_order API honouring rcu

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: Ic613c5b6e496a677b9d3de15fc042a0492109fb0
BUG: 1191030
Signed-off-by: Kaushal M 
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/9624
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Gaurav Kumar Garg 
Reviewed-by: Anand Nekkunti

glusterd: nfs,shd,quotad,snapd daemons refactoring

2015-02-20T12:04:08+00:00

This patch ports nfs, shd, quotad & snapd with the approach suggested in
http://www.gluster.org/pipermail/gluster-devel/2014-December/043180.html

Change-Id: I4ea5b38793f87fc85cc9d2cf873727351dedffd2
BUG: 1191486
Signed-off-by: Atin Mukherjee 
Signed-off-by:  Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/9428
Tested-by: Gluster Build System 
Reviewed-by: Anand Nekkunti

glusterd: quorum validatation in glusterd syncop framework

2015-01-21T06:38:36+00:00

Previously glusterd was not checking quorum validation in syncop framework.
So when there is loss in quorum then few operation (for eg. add-brick,
remove-brick, volume set) which is based on syncop framework passed
successfully with out doing quorum validation check.

With this change it will do quorum validation in syncop framework and it will
block all operation (except volume set  and "volume reset all"
commands) when there is loss in quorum.

Change-Id: I4c2ef16728d55c98a228bb86795023d9c1f4e9fb
BUG: 1177132
Signed-off-by: Gaurav Kumar Garg 
Reviewed-on: http://review.gluster.org/9349
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi