glusterfs.git/geo-replication/syncdaemon, branch v3.9rc0

geo-rep: add geo-rep events for server side changes

2016-08-31T14:14:08+00:00

Event Type defined in #15351 to avoid merge conflicts

Add geo-rep events applicable to changes in
geo-rep session in the server side.

Change-Id: Ia66574d2abccad7fce6a96667efbc7c6c8903fc6
BUG: 1370445
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/15328
Tested-by: Aravinda VK 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Post process Data and Meta Changelogs

2016-08-26T17:45:58+00:00

With this patch, Data and Meta GFIDs are post processed. If Changelog has
UNLINK entry then remove from Data and Meta GFIDs list(If stat on GFID is
ENOENT in Master).

While processing Changelogs,

- Collect all the data and meta operations in a temporary database
- Delete all Data and Meta GFIDs which are already unlinked as per Changelogs
  (unlink only if stat on GFID is ENOENT)
- Process all Entry operations as usual
- Process data and meta operations in batch(Fetch from Db in batch)
- Data sync is again batched based on number of changelogs(Default 1day
  changelogs). Once the sync is complete, Update last Changelog's time as last_synced
  time as usual.

Additionally maintain entry_stime on Brick root, ignore Entry ops if changelog
suffix time is less than entry_stime. If data stime is more than entry_stime,
this can happen only when passive worker updates stime by itself by getting
mount point stime. Use entry_stime = data_stime in this case.

New configurations:

max-rsync-retries - Default Value is 10
max-data-changelogs-in-batch - Max number of changelogs to be considered in a
batch for syncing. Default value is 5760(4 changelogs per min * 60 min *
24 hours)
max-history-changelogs-in-batch - Max number of history changelogs to be
processed at once. Default value 86400(4 changelogs per min * 60 min * 24
hours * 15 days)

BUG: 1364420
Change-Id: I7b665895bf4806035c2a8573d361257cbadbea17
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15110
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
CentOS-regression: Gluster Build System

features/libgfchangelog: Log failure in gf_histroy_changelog

2016-08-09T07:27:12+00:00

Add error logs if gf_history_changelog fails. If requested
changelog range is not available, log the error and exit
instead of continuing the loop and exiting in readdir
without logging. Also fixed the duplicate MSGID number in
'changelog-lib-messages.h'

Change-Id: Icd71b89ae23b48a71380657ba5649029c32fabfd
BUG: 1362151
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/15064
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Aravinda VK

geo-rep: Handle Config parser errors

2016-07-12T12:09:40+00:00

Python ConfigParser lib has two methods, readfp and read, it should
be used as follows.

    readfp(open("defaults.conf"))
    read("custom.conf")

ConfigParser.read(path) ignores any file errors, which is intentional
since errors are handled while loading default config.

Geo-rep uses only one config file(Session config in Master side and
Template config on Slave side) so we should use readfp to avoid
skipping OS errors.

config.read is retained in case of `--config-set-rx` where glusterd
creates new template config file.

BUG: 1349273
Change-Id: I15a14d3743facd7b8c7af0edc70fdefaa43efd04
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14777
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

geo-rep: fix wrong argument to find_library

2016-07-07T10:49:56+00:00

Argument passed to find_library function is wrong.
Pass proper argument to find_library.

Note: It still works as default handle is libc.

Change-Id: I4657a3ef9f5249e3eec4a20582856b58168126ce
BUG: 1352423
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/14852
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

georep: add reset-sync-time option for session delete

2016-06-29T06:41:54+00:00

Set the stime xattr at all the brick roots to (0,0) if the argument
reset-sync-time has been provided on the command-line.
To avoid testing against directory specific stime, the remote
stime is assumed to be minus_infinity, if the root directory
stime is set to (0,0), before the directory scan begins.
This triggers a full volume resync to slave in the case of a
geo-rep session recreation with the same master-slave volume
pair.

Command synopsis:
gluster volume geo-replication  :: delete \
    [reset-sync-time]

Update gluster cli man page to include new sub-command reset-sync-time.

Change-Id: Ie4ce03b9425ed9bb81eda8681058c0fc6f990948
BUG: 1311926
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/14051
Reviewed-by: Kotresh HR 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Safely handle if unliked GFID not present in data list

2016-06-20T06:37:00+00:00

If unlinked GFID is not present in data list to be synced then
Geo-rep worker was crashing with KeyError. Handled KeyError with
this patch.

BUG: 1345744
Change-Id: I5a1c9ca4473e32606df2e5c7e26c95faf55d44c0
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14706
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Error message cleanup

2016-06-15T17:28:54+00:00

If ssh returns 127 that means the remote gsyncd path is wrong
or push-pem failed during create. Existing error message was
pointing old documentation.

Change-Id: Ifbbb4a604fc0ae0fd5cb2746df6363bf28cde1e9
BUG: 1343943
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14673
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Kotresh HR

geo-rep: Handle stime/xtime set failures

2016-06-13T11:27:11+00:00

While setting stime/xtime, if the file or directory is already
deleted then Geo-rep will crash with ENOENT.

With this patch, Geo-rep will ignores ENOENT since stime/xtime can't
be applied on a deleted file/directory.

Change-Id: I2d90569e51565f81ae53fcb23323e4f47c9e9672
Signed-off-by: Aravinda VK 
BUG: 1339471
Reviewed-on: http://review.gluster.org/14529
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam 
Reviewed-by: Kotresh HR

glusterd/geo-rep: upgrade path when slave vol uuid involved

2016-06-04T02:41:15+00:00

slave volume uuid is involved in identifying a geo-replication
session.

This patch addresses upgrade path, where existing geo-rep session
is gracefully upgraded to involve slave volume uuid.

Change-Id: Ib7ff5109b161592f24fc86fc7e93a407655fab86
BUG: 1337473
Signed-off-by: Saravanakumar Arumugam 
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14425
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Aravinda VK