glusterfs-nsr.git, branch master

Merge branch 'upstream'

2014-04-28T14:18:50+00:00

Conflicts:
	rpc/xdr/src/glusterfs3-xdr.c
	rpc/xdr/src/glusterfs3-xdr.h
	xlators/features/changelog/src/Makefile.am
	xlators/features/changelog/src/changelog-helpers.h
	xlators/features/changelog/src/changelog.c
	xlators/mgmt/glusterd/src/glusterd-sm.c

Change-Id: I9972a5e6184503477eb77a8b56c50a4db4eec3e2

features/libgfchangelog: APIs to process history changelogs.

2014-04-28T11:53:36+00:00

1. Create directories in following fashion for history API's
   usage when consumer is registered with libgfchangelog
   shared library through gf_changelog_register.
       scratch_dir/.history
       scratch_dir/.history/.current
       scratch_dir/.history/.processed
       scratch_dir/.history/.processing

2. Added new file 'gf-history-changelog.c' and following APIs
   are provided for consumers to process history changelogs.

    1. gf_history_changelog_scan:
            Move processed history changelog file from
            .processing to .processed
    2. gf_history_changelog_next_change:
            Return the next history changelog file entry.
            Zero means all history chanelogs are consumed.
    3. gf_history_changelog_done:
            Scan .processing directory and generate a list of
            change entries.
    4. gf_history_changelog_start_fresh:
            For a set of changelogs, start from the begining.

NOTE: Though this patch provides above funcationalities.
      It is considered functionally full with the
      patch (http://review.gluster.org/#/c/6930/).

Change-Id: I200780c7278e0a6c008910d93faad5858a4b3e76
Original-author: Kotresh H R 
Signed-off-by: Kotresh H R 
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/6998
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

glusterd/snapshot: Compare and update snapshots during peer handshake

2014-04-28T11:02:22+00:00

During a peer-handshake, after the volumes have synced, and the list of
missed snapshots have synced, the node will perform the pending deletes
and restores on this list. At this point, the current snapshot list in
the node will be updated, and hence in case of conflicts arising during
snapshot handshake, the peer hosting the bricks will be given precedence
Likewise, if there will be a conflict, and both peers will be in the same
state, i.e either both would be hosting bricks or both would not be hosting
bricks, then a decision can't be taken and a peer-reject will happen.

glusterd_compare_and_update_snap() implements the following algorithm to
perform the above task:
Step  1: Start.
Step  2: Check if the peer is missing a delete on the said snap.
         If yes, goto step 6.
Step  3: Check if there is a conflict between the peer's data and the
         local snap. If no, goto step 5.
Step  4: As there is a conflict, check if both the peer and the local nodes
         are hosting bricks. Based on the results perform the following:
         Peer Hosts Bricks    Local Node Hosts Bricks       Action
               Yes                     Yes                Goto Step 7
               No                      No                 Goto Step 7
               Yes                     No                 Goto Step 8
               No                      Yes                Goto Step 6
Step  5: Check if the local node is missing the peer's data.
         If yes, goto step 9.
Step  6: It's a no-op. Goto step 10
Step  7: Peer Reject. Goto step 10
Step  8: Delete local node's data.
Step  9: Accept Peer Data.
Step 10: Stop

Change-Id: I79be0f0f5f2a4f5c72277a4e77c2be732af432e1
BUG: 1061685
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/7525
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Vijay Bellur

glusterd: Rename the export dictionary as peer_data

2014-04-28T11:02:03+00:00

During a glusterd handshake, a dictionary is passed among
the peers which contains, info of volumes, global opts,
and now also info of snaps and list of missed snaps

As it now contains more than just volume specific data,
renaming the dict in the code-base from "vols" to "peer_data"

Change-Id: Ib457172789ddd0d8978b08bceab0988c48e9eea7
BUG: 1061685
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/7524
Tested-by: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Vijay Bellur

glusterd/snapshot: Recreate the mount dirs and mount the lvm snapshots on node reboot.

2014-04-28T10:51:29+00:00

The lvm snapshots of the bricks are mounted at /var/run/gluster/snaps/ or
/run/gluster/snaps. These paths being on a tempfs, on reboot are removed.
So when glusterd starts, we need to recreate these paths, activate the
respective logical volumes (lvm snapshots of the bricks), and mount
these logical volumes at their respective paths.

Change-Id: Ic5ef61e79a25d9830df717c592391965fe09db62
BUG: 1061685
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/7452
Tested-by: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Vijay Bellur

glusterd/snapshot: Perform missed snap deletes and restores.

2014-04-28T10:50:18+00:00

Replacing is_volume_restored(gf_boolean_t) with
restored_from_snap(uuid_t) in glusterd_volinfo_

Also removed gd_restore_snap_volume from glusterd-volgen.c
to glusterd-snapshot.c

Change-Id: Ic615a1658cfaffa98d4590506ac82f20bf709ad6
BUG: 1089906
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/7455
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Vijay Bellur

gNFS: gNFS drc cache failed to detecte duplicates.

2014-04-28T07:50:24+00:00

After the drc cache get full, message "DRC failed to detect duplicates" keep
printed in the log.
The root cause is drc_compare_reqs use the wrong compare type. This function
should use type drc_cache_op_t as its input type. Since all rbtree related code
(except function rpcsvc_drc_lookup) in drc cache pass drc_cache_op_t as compare
type. Only rpcsvc_drc_lookup use type rpcsvc_request_t. It has been modified
too.

Change-Id: I925c097debe6b82f267986961fd4e7755f3de9af
BUG: 1089676
Signed-off-by: Yuan Ding 
Reviewed-on: http://review.gluster.org/7519
Reviewed-by: Niels de Vos 
Reviewed-by: Santosh Pradhan 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

Barrier: Barrier translator options configuration

2014-04-28T05:25:32+00:00

barrier enable/disable, barrier-timeout configuration in barrier translator.

Change-Id: I7cbf9cd4f5e55d42dcc6b7cd6827234566c7b6f3
BUG: 1060002
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/7177
Reviewed-by: Kaushal M 
Tested-by: Gluster Build System

glusterd/snapshot: Adding snap_vol_id and snap_uuid to missed_snap_list

2014-04-28T05:00:20+00:00

Persisting missing snapshot info on disk as well as in memory in
the following format:
-------------NODE-UUID--------------:--------------SNAP-UUID-------------=---------SNAP-VOL-ID------------:BRICKNUM:-------BRICKPATH--------:OPERATION:STATUS
927cb5fe-63da-48f5-82f6-e6a09ddc81c4:8258b18f-d408-483d-8239-204039dc6397=a17b4fe42c5a45f7a916438643edaa13:   3    :/brick/brick-dirs/brick3:    1    :   1
927cb5fe-63da-48f5-82f6-e6a09ddc81c4:8258b18f-d408-483d-8239-204039dc6397=a17b4fe42c5a45f7a916438643edaa13:   3    :/brick/brick-dirs/brick3:    3    :   1
927cb5fe-63da-48f5-82f6-e6a09ddc81c4:8258b18f-d408-483d-8239-204039dc6397=83a3cc05453b46b2a7eda4c9a9208638:   3    :/brick/brick-dirs/brick3:    1    :   1

This data will be stored on disk at /var/lib/glusterd/snaps/missed_snaps_list

In memory we maintain the data as a list of glusterd_missed_snap_info
in conf, the key for this list are the first two fields,
i.e NODE-UUID:SNAP-UUID.

For every NODE-UUID:SNAP-UUID, there can be multiple operations missed
on multiple bricks. So we maintain a list of glusterd_snap_op_t
for every node of glusterd_missed_snap_info

This list is maintained or updated during snapshot create, delete, and restore
operations which are the only operations that if missed, are recorded in this
list.

During snapshot create, if a node is down, or a brick is down, we don't
receive their mount point infos. snap_status of such bricks is marked as
-1, and their brick details are added to this list.

During snapshot delete, we check from originator node, if any other
nodes, holding bricks of the said snap are down. Those are also added to the list.
Also if the node is up, but the snapshot was pending for a snap
brick, and its snap_status is -1, we add that to the list too.
When a subsequent delete entry is processed for an already existing
create entry, we just mark the create entries status as done (2), and don't
add the delete entry to the list.

During snapshot restore, we check from originator node, if any other
nodes, holding bricks of the said snap are down. Those are also added to the list.
Also if the node is up, but the snapshot was pending for a snap
brick, and its snap_status is -1, we add that to the list too.
Like delete when a subsequent restore entry is processed for an already existing
create entry, we just mark the create entries status as done (2), and don't
add the restore entry to the list.

Change-Id: I54f63e28d3c40555d0f84528f38227103171f594
BUG: 1061685
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/7454
Tested-by: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Vijay Bellur

Update references to the maillinglist to gluster-devel@gluster.org

2014-04-28T04:29:36+00:00

gluster-devel@nongnu.org has moved to gluster-devel@gluster.org. All
occurrences in the current (non legacy) documentation and code have been
adjusted.

Change-Id: I053162e633f7ea14fd3eed239ded017df165147c
BUG: 1091705
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/7573
Reviewed-by: Justin Clift 
Reviewed-by: Anand Avati 
Tested-by: Anand Avati