glusterfs.git/xlators/mgmt/glusterd/src/glusterd-syncop.c, branch v3.7.2

glusterd/snapshot: Return correct errno in events of failure - PATCH 2

2015-06-09T14:18:38+00:00

       Backport of http://review.gluster.org/#/c/10588/

ENUM           RETCODE        ERROR
-------------------------------------------------------------
EG_INTRNL      30800          Internal Error
EG_OPNOTSUP    30801          Gluster Op Not Supported
EG_ANOTRANS    30802          Another Transaction in Progress
EG_BRCKDWN     30803          One or more brick is down
EG_NODEDWN     30804          One or more node is down
EG_HRDLMT      30805          Hard Limit is reached
EG_NOVOL       30806          Volume does not exist
EG_NOSNAP      30807          Snap does not exist
EG_RBALRUN     30808          Rebalance is running
EG_VOLRUN      30809          Volume is running
EG_VOLSTP      30810          Volume is not running
EG_VOLEXST     30811          Volume exists
EG_SNAPEXST    30812          Snapshot exists
EG_ISSNAP      30813          Volume is a snap volume
EG_GEOREPRUN   30814          Geo-Replication is running
EG_NOTTHINP    30815          Bricks are not thinly provisioned

Change-Id: I49a170cdfd77df11fe677e09f4e063d99b159275
BUG: 1226117
Signed-off-by: Avra Sengupta 
(cherry picked from commit 2df57ab7dc7b9d7deb0eebad96036149760d607b)
Reviewed-on: http://review.gluster.org/11042
Tested-by: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Tested-by: NetBSD Build System 
Reviewed-by: Krishnan Parthasarathi

glusterd : allocate peerid to store in frame->cookie

2015-06-01T09:23:26+00:00

Backport of http://review.gluster.org/10842

commit a1de3b05 was using peerid from the stack and storing it in the
frame->cookie and in the subsequent callback it was referred. The existance of
this variable is not guranteed in the cbk since its not dynamically allocated.
Fix is to dynmacially manage peerid in the frame cookie.

This patch also fixes one problem in gd_sync_task_begin () where unlock is not
triggered if the cluster is running with lesser than 3.6 op-version resulting into
commands failing with another transaction is in progress.

Change-Id: I0d22cf663df53ef3769585703944577461061312
BUG: 1223215
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/10842
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M 
(cherry picked from commit 37f365843bed87728048da1f56de22290f5cb70f)
Reviewed-on: http://review.gluster.org/10959
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

glusterd/snapshot: Return correct errno in events of failure - PATCH 1

2015-06-01T07:18:15+00:00

       Backport of patch http://review.gluster.org/#/c/10313/

RETCODE              ERROR
-------------------------------------------
30800                Internal Error
30801                Another Transaction In Progress

>Change-Id: Ica7fd2e513b2c28717b6df73cfb2667725dbf057
>BUG: 1226117
>Signed-off-by: Avra Sengupta 
>Reviewed-on: http://review.gluster.org/10313
>Reviewed-by: Rajesh Joseph 
>Tested-by: Gluster Build System 
>Tested-by: NetBSD Build System
>Reviewed-by: Krishnan Parthasarathi 
>(cherry picked from commit 4397d7e72fdff6f01c59b72eebea421f23c1a392)

Change-Id: Iace6ac0f150919cead94e6c5d99a23d28d45046e
BUG: 1226117
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/11011
Tested-by: NetBSD Build System 
Reviewed-by: Kaushal M

glusterd: fix double-free of rebalance process' rpc object

2015-05-30T14:05:06+00:00

Change-Id: I0c79c4de47a160b1ecf3a8994eedc02e3f5002a9
BUG: 1225318
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/10872
Reviewed-by: Atin Mukherjee 
Reviewed-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10932
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: support for tier volumes 'detach start' and 'detach commit'

2015-05-09T15:35:14+00:00

        Back port of http://review.gluster.org/10108

These commands work in a manner analagous to rebalancing when removing a
brick. The existing migration daemon detects "detach start" and switches
to moving data off the hot tier. While in this state all lookups are
directed to the cold tier.

gluster v detach-tier  start
gluster v detach-tier  commit

The status and stop cli commands shall be submitted separately.

>Change-Id: I24fda5cc3ba74f5fb8aa9a3234ad51f18b80a8a0
>BUG: 1205540
>Signed-off-by: Dan Lambright 
>Signed-off-by: root 
>Signed-off-by: Dan Lambright 
>Reviewed-on: http://review.gluster.org/10108
>Reviewed-by: Kaleb KEITHLEY 

Change-Id: I212d748d077fb5870ee84b316c653acbafbea3f7
BUG: 1220047
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/10708
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

NFS-Ganesha : Locking global options file

2015-05-07T09:08:43+00:00

Global option gluster features.ganesha enable
writes into the global 'option' file. The snapshot
feature also writes into the same file.

To handle concurrent multiple transactions correctly,
a new lock has to be introduced on this file.
Every operation using this file needs
to contest for the new lock type.

This is a back-port of the patch,
http://review.gluster.org/#/c/10130/

Change-Id: I1fdd285814e615a13dbf8c88ad2b7ee311247f90
BUG: 1218963
Signed-off-by: Meghana Madhusudhan 
Reviewed-on: http://review.gluster.org/10606
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos 
Tested-by: NetBSD Build System
Reviewed-by: Avra Sengupta

libglusterfs: Implementation of sync lock as recursive lock to avoid crash.

2015-04-30T11:47:46+00:00

Problem : In glusterd,we are using big lock which is implemented based on sync
task frame work for thread synchronization and rcu lock for data consistency.
sync task frame work swap the threads  if there is no worker poll threads
available,due to this rcu lock and rcu unlock was happening in different threads
(urcu-bp will not allow this),resulting into glusterd crash.

fix : To avoid releasing the sync lock(big lock) in between rcu critical
section,implemented sync lock as recursive lock.

More details:
link : http://www.spinics.net/lists/gluster-devel/msg14632.html

Change-Id: I2b56c1caf3f0470f219b1adcaf62cce29cdc6b88
BUG: 1216942
Signed-off-by: anand 
Reviewed-on: http://review.gluster.org/10285
Reviewed-by: Atin Mukherjee 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur 
(cherry picked from commit ada6b3a8800867934af57a57d5312f5a5d8374f0)
Reviewed-on: http://review.gluster.org/10432
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Vijay Bellur

glusterd: Remove direct references to peerinfo in frame cookies

2015-04-28T12:49:05+00:00

RCU protection requires that we don't have  direct references to
protected data structures outside read-critical sections

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  82ebfdd Remove direct references to peerinfo in frame cookies
  dec4bec Remove incorrect and unneeded code from
          gd_syncop_mgmt_v3_unlock_cbk_fn
  7aced7b Use stack allocated uuid for frame cookie.
  38e4124 Address comments from 10192/2

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: Ic50e5fca0be72af5090f4cf318efa55d29075de9
BUG: 1205186
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10399
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

glusterd: Replace transaction peers lists

2015-04-13T06:30:02+00:00

Transaction peer lists were used in GlusterD to peers belonging to a
transaction. This was needed to prevent newly added peers performing
partial transactions, which could be incorrect.

This was accomplished by creating a seperate transaction peers list at
the beginning of every transaction. A transaction peers list referenced
the peerinfo data structures of the peers which were present at the
beginning of the transaction. RCU protection of peerinfos referenced by
the transaction peers list is a hard problem and difficult to do
correctly.

To have proper RCU protection of peerinfos, the transaction peers lists
have been replaced by an alternative method to identify peers that
belong to a transaction. The alternative method is to the global peers
list along with generation numbers to identify peers that should belong
to a transaction.

This change introduces a global peer list generation number, and a
generation number for each peerinfo object. Whenever a peerinfo object
is created, the global generation number is bumped, and the peerinfos
generation number is set to the bumped global generation.

With the above changes, the algorithm to identify peers belonging to a
transaction with RCU protection is as follows,
- At the beginning of a transaction, the current global generation
  number is saved
- To identify if a peers belonging to the transaction,
  - Start a RCU read critical section
  - For each peer in the global peers list,
    - If the peers generation number is not greater than the saved
      generation number, continue with the action on the peer
  - End the RCU read critical section

The above algorithm guarantees that,
- The peer list is not modified when a transaction is iterating through
  it
- The transaction actions are only done on peers that were present when
  the transaction started

But, as a transaction could iterate over the peers list multiple times,
the algorithm cannot guarantee that same set of peers will be selected
every time. A peer could get deleted between two iterations of the list
within a transaction. This problem existed with transaction peers list
as well, but unlike before now it will not lead to invalid memory access
and potential crashes. This problem will be addressed seprately.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  52ded5b Add timespec_cmp
  44aedd8 Add create timestamp to peerinfo
  7bcbea5 Fix some silly mistakes
  13e3241 Add start time to opinfo
  17a6727 Use timestamp comparisions to identify xaction peers instead
          of a xaction peer list
  3be05b6 Correct check for peerinfo age
  70d5b58 Use read-critical sections for peer list iteration
  ba4dbca Use peerinfo timestamp checks in op-sm instead of xaction peer
          list
  d63f811 Add more peer status checks when iterating peers list in
          glusterd-syncop
  1998a2a Timestamp based peer list traversal of mgmtv3 xactions
  f3c1a42 Remove transaction peer lists
  b8b08ee Remove unused labels
  32e5f5b Remove 'npeers' usage
  a075fb7 Remove 'npeers' from mgmt-v3 framework
  12c9df2 Use generation number instead of timestamps.
  9723021 Remove timespec_cmp
  80ae2c6 Remove timespec.h include
  a9479b0 Address review comments on 10147/4

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: I9be1033525c0a89276f5b5d83dc2eb061918b97f
BUG: 1205186
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10147
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Anand Nekkunti 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

Avoid conflict between contrib/uuid and system uuid

2015-04-04T17:48:35+00:00

glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.

Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.

A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.

BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos