glusterfs.git/geo-replication, branch v3.9.0rc2

geo-rep: Upgrade conf file only if it is session config

2016-10-25T07:22:18+00:00

Ignore config upgrade if it is template config file present in
/var/lib/glusterd/geo-replication/gsyncd_template.conf

> Reviewed-on: http://review.gluster.org/15669
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 
> NetBSD-regression: NetBSD Build System 

BUG: 1388150
Change-Id: I2cbba3103b6801c16ff57f778a90b9a0bb2467cf
Signed-off-by: Aravinda VK 
(cherry picked from commit 1506c7a98d8d3b31e68d0f214ab331f28ffa9fb5)
Reviewed-on: http://review.gluster.org/15715
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Logging improvements

2016-10-25T07:22:01+00:00

- Redundant log messages removed.
- Worker and connected slave node details added in "starting worker" log
- Added log for Monitor state change
- Added log for Worker status change(Initializing/Active/Passive/Faulty)
- Added log for Crawl status Change
- Added log for config set and reset
- Added log for checkpoint set, reset and completion

> Reviewed-on: http://review.gluster.org/15684
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 

BUG: 1387990
Change-Id: Icc7173ff3c93de4b862bdb1a61760db7eaf14271
Signed-off-by: Aravinda VK 
(cherry picked from commit cdc30ed8eacb6772e0dabb863ef51cef794d60dd)
Reviewed-on: http://review.gluster.org/15710
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep/eventsapi: Additional Events

2016-10-20T06:50:24+00:00

Added following events
EVENT_GEOREP_ACTIVE
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_ACTIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH
        }
    }

EVENT_GEOREP_PASSIVE
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_PASSIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH
        }
    }
EVENT_GEOREP_CHECKPOINT_COMPLETED
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_ACTIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH,
            "checkpoint_time": CHECKPOINT_TIME,
            "checkpoint_completion_time": CHECKPOINT_COMPLETION_TIME
        }
    }

> Reviewed-on: http://review.gluster.org/15630
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 
> Tested-by: Kotresh HR 

BUG: 1386178
Change-Id: I90716175868c59dd65c8d202e73e0ede90347b6a
Signed-off-by: Aravinda VK 
(cherry picked from commit 3830b48b6a46854d6597a36b6f2089ac1e486eb5)
Reviewed-on: http://review.gluster.org/15672
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam

eventsapi/geo-rep: Geo-rep will not work without eventsapi rpms

2016-09-29T11:05:04+00:00

If glusterfs-events rpm is not installed, Geo-replication will
fail since it imports eventtypes.

Any call to gsyncd will fail with Import error. Glusterd start
fails since it runs `gsyncd.py --version`

Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py",
    line 29, in 
      from syncdutils import FreeObject, norm, grabpidfile, finalize
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py",
    line 28, in 
      from events import eventtypes
ImportError: No module named events

> Reviewed-on: http://review.gluster.org/15539
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Saravanakumar Arumugam 
> Reviewed-by: Kotresh HR 

BUG: 1380252
Change-Id: I1a9bc086c3d52449ec7296cb2f9ceb16cd41a8a4
Signed-off-by: Aravinda VK 
(cherry picked from commit 567dee257e092401cdf0a62d4b89e13c39a0a5aa)
Reviewed-on: http://review.gluster.org/15594
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Saravanakumar Arumugam

geo-rep: Fix Geo-rep status if monitor.pid file not exists

2016-09-15T06:48:01+00:00

If monitor.pid file not exists, gsyncd fails with following traceback

Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py",
  line 201, in main
    main_i()
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py",
  line 681, in main_i
    brick_status.print_status(checkpoint_time=checkpoint_time)
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py",
  line 343, in print_status
    for key, value in self.get_status(checkpoint_time).items():
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py",
  line 262, in get_status
    with open(self.monitor_pid_file, "r+") as f:
IOError: [Errno 2] No such file or directory: '/var/lib/glusterd/
  geo-replication/master_node_slave/monitor.pid'

If Georep status command this worker's status will not be displayed
since not returning expected status output.

> Reviewed-on: http://review.gluster.org/15416
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 

BUG: 1374630
Change-Id: I600a2f5d9617f993d635b9bc6e393108500db5f9
Signed-off-by: Aravinda VK 
(cherry picked from commit c7118a92f52a2fa33ab69f3e3ef1bdabfee847cf)
Reviewed-on: http://review.gluster.org/15446
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam 
Smoke: Gluster Build System

geo-rep: Defunct tar process after sync

2016-09-15T06:30:42+00:00

After every sync iteration with tarssh mode leaves defunct tar
process.

Added wait for tar process to prevent this issue.

> Reviewed-on: http://review.gluster.org/15426
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Saravanakumar Arumugam 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 

BUG: 1375543
Change-Id: I9953239ef601cc1970c814b00074b45eb00f481e
Signed-off-by: Aravinda VK 
(cherry picked from commit 6b30e9bf5a612e105eb7ded0a89ef25fd8530ba5)
Reviewed-on: http://review.gluster.org/15488
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam 
CentOS-regression: Gluster Build System

geo-rep: Fix ESTALE/EINVAL issue during set_{xtime,stime}

2016-09-15T06:30:02+00:00

Setfattr may get ESTALE/EINVAL if a file is being unlinked.
To prevent worker crashing, added retry for these error messages.

On second retry it will get ENOENT and that error is handled by
ignoring.

> Reviewed-on: http://review.gluster.org/15404
> Reviewed-by: Saravanakumar Arumugam 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 
> Smoke: Gluster Build System 

BUG: 1374626
Change-Id: Ic660fa13208366d57c8d3d492bbef611475e45b7
Signed-off-by: Aravinda VK 
(cherry picked from commit 9db2826cd8dbcc0fb58f5434c21d00ca5c15c491)
Reviewed-on: http://review.gluster.org/15445
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam

geo-rep: Use configured log_level for libgfchangelog logs

2016-09-15T06:29:47+00:00

libgfchangelog was not respecting the log_level configured
in Geo-replication. With this patch Libgfchangelog log level
can be configured using `config changelog_log_level TRACE`.
Default Changelog log level is INFO

> Reviewed-on: http://review.gluster.org/15078
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 

BUG: 1374608
Change-Id: Ida714931129f6a1331b9d0815da77efcb2b898e3
Signed-off-by: Aravinda VK 
(cherry picked from commit f41ec4fcfaa9ca976fddbe9e91aabf840c20111f)
Reviewed-on: http://review.gluster.org/15444
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam

geo-rep: Fix logging sync failures

2016-09-15T06:28:37+00:00

If Rsync/Tar subprocess dies, while logging error Geo-rep fails
with EBADF while accessing error file. Also worker dies while
accessing elines before it is set.

> Reviewed-on: http://review.gluster.org/15379
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 

BUG: 1374597
Change-Id: I9cfce116e8aafa4a98654f5190d40a455af8ec95
Signed-off-by: Aravinda VK 
(cherry picked from commit c0f877c0374d97e0bee17aac4850d7655a35e61b)
Reviewed-on: http://review.gluster.org/15441
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam

geo-rep: Handle EISDIR error during Unlink

2016-09-15T06:28:05+00:00

During Rename, If Source and Target has same inode then
Geo-rep unlinks source. But if source is a directory then
this will fail with below traceback

Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 113, in worker
    res = getattr(self.obj, rmeth)(*in_data[2:])
  File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 772,
    in entry_ops
    os.unlink(entry)
OSError: [Errno 21] Is a directory: '.gfid/12711ebf-7fdc-4f4b-9850-2d75581eb
   452/New folder'

With this patch, if EISDIR, rmdir is tried. Logs error in Slave log in case
of ENOTEMPTY.

> Reviewed-on: http://review.gluster.org/15132
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Kotresh HR 

BUG: 1374581
Change-Id: I099af4192adac5125c0a23988ceb6506f91e987f
Signed-off-by: Aravinda VK 
 (cherry picked from commit 8a6236e4cfc7bf86e881e5c770a19040a6060ad0)
Change-Id: Ic3fcf3fa3c049c0d820db86b93e04f6e239deb45
Reviewed-on: http://review.gluster.org/15435
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam