glusterfs.git/geo-replication/syncdaemon/monitor.py, branch v3.9.0

geo-rep: Logging improvements

2016-10-25T07:22:01+00:00

- Redundant log messages removed.
- Worker and connected slave node details added in "starting worker" log
- Added log for Monitor state change
- Added log for Worker status change(Initializing/Active/Passive/Faulty)
- Added log for Crawl status Change
- Added log for config set and reset
- Added log for checkpoint set, reset and completion

> Reviewed-on: http://review.gluster.org/15684
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 

BUG: 1387990
Change-Id: Icc7173ff3c93de4b862bdb1a61760db7eaf14271
Signed-off-by: Aravinda VK 
(cherry picked from commit cdc30ed8eacb6772e0dabb863ef51cef794d60dd)
Reviewed-on: http://review.gluster.org/15710
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep/eventsapi: Additional Events

2016-10-20T06:50:24+00:00

Added following events
EVENT_GEOREP_ACTIVE
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_ACTIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH
        }
    }

EVENT_GEOREP_PASSIVE
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_PASSIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH
        }
    }
EVENT_GEOREP_CHECKPOINT_COMPLETED
    {
        "nodeid": NODEID,
        "ts": TIMESTAMP,
        "event": "GEOREP_ACTIVE",
        "message": {
            "master_volume": MASTER_VOLUME_NAME,
            "slave_host": SLAVE_HOST,
            "slave_volume": SLAVE_VOLUME,
            "brick_path": BRICK_PATH,
            "checkpoint_time": CHECKPOINT_TIME,
            "checkpoint_completion_time": CHECKPOINT_COMPLETION_TIME
        }
    }

> Reviewed-on: http://review.gluster.org/15630
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 
> Tested-by: Kotresh HR 

BUG: 1386178
Change-Id: I90716175868c59dd65c8d202e73e0ede90347b6a
Signed-off-by: Aravinda VK 
(cherry picked from commit 3830b48b6a46854d6597a36b6f2089ac1e486eb5)
Reviewed-on: http://review.gluster.org/15672
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam

eventsapi/geo-rep: Geo-rep will not work without eventsapi rpms

2016-09-29T11:05:04+00:00

If glusterfs-events rpm is not installed, Geo-replication will
fail since it imports eventtypes.

Any call to gsyncd will fail with Import error. Glusterd start
fails since it runs `gsyncd.py --version`

Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py",
    line 29, in 
      from syncdutils import FreeObject, norm, grabpidfile, finalize
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py",
    line 28, in 
      from events import eventtypes
ImportError: No module named events

> Reviewed-on: http://review.gluster.org/15539
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Saravanakumar Arumugam 
> Reviewed-by: Kotresh HR 

BUG: 1380252
Change-Id: I1a9bc086c3d52449ec7296cb2f9ceb16cd41a8a4
Signed-off-by: Aravinda VK 
(cherry picked from commit 567dee257e092401cdf0a62d4b89e13c39a0a5aa)
Reviewed-on: http://review.gluster.org/15594
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Saravanakumar Arumugam

geo-rep: add geo-rep events for server side changes

2016-08-31T14:14:08+00:00

Event Type defined in #15351 to avoid merge conflicts

Add geo-rep events applicable to changes in
geo-rep session in the server side.

Change-Id: Ia66574d2abccad7fce6a96667efbc7c6c8903fc6
BUG: 1370445
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/15328
Tested-by: Aravinda VK 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

georep: add reset-sync-time option for session delete

2016-06-29T06:41:54+00:00

Set the stime xattr at all the brick roots to (0,0) if the argument
reset-sync-time has been provided on the command-line.
To avoid testing against directory specific stime, the remote
stime is assumed to be minus_infinity, if the root directory
stime is set to (0,0), before the directory scan begins.
This triggers a full volume resync to slave in the case of a
geo-rep session recreation with the same master-slave volume
pair.

Command synopsis:
gluster volume geo-replication  :: delete \
    [reset-sync-time]

Update gluster cli man page to include new sub-command reset-sync-time.

Change-Id: Ie4ce03b9425ed9bb81eda8681058c0fc6f990948
BUG: 1311926
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/14051
Reviewed-by: Kotresh HR 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Handle Worker kill gracefully if worker already died

2016-05-30T15:12:05+00:00

If Agent dies for any reason, monitor tries to kill Worker also. But
if worker is also died then kill command raises error ESRCH: No such
process.

[2016-05-23 16:49:33.903965] I [monitor(monitor):326:monitor] Monitor:
    Changelog Agent died, Aborting Worker(/bricks/brick0/master_brick0)
[2016-05-23 16:49:33.904535] E [syncdutils(monitor):276:log_raise_exception]
    : FAIL:
Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 306 in
  twrap
    tf(*aa)
  File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 393, in
  wmon
     slave_host, master)
  File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 327, in
  monitor
     os.kill(cpid, signal.SIGKILL)
     OSError: [Errno 3] No such process

With this patch, monitor will gracefully handle if worker is already died.

Change-Id: I3ae5f816a3a197343b64540cf46f5453167fb660
Signed-off-by: Aravinda VK 
BUG: 1339472
Reviewed-on: http://review.gluster.org/14512
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
CentOS-regression: Gluster Build System

geo-rep: Fix getting subvol count

2015-12-22T15:06:46+00:00

Tiering doesn't support disperse volume as hot tier,
hence xml output doesn't give 'hotdisperseCount'.
Remove the usage of 'hotdisperseCount' in geo-rep
and return 0 instead.

Change-Id: I736e29257de085a25e38eb02959caad3465ebcda
BUG: 1292084
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13062
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Vivek
Reviewed-by: Aravinda VK

geo-rep: Fix getting subvol number

2015-12-21T08:28:23+00:00

Fix getting subvol number if the volume
type is tier. If the volume type was tier,
the subvol number was calculated incorrectly
and hence few of workers didn't become ACTIVE
resulting in files not being replicated from
corresponding brick. This patch addresses
the same.

Change-Id: Ic10ad7f09a0fa91b4bf2aa361dea3bd48be74853
BUG: 1292084
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/12994
Tested-by: NetBSD Build System 
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System

geo-rep: use cold tier bricks for namespace operations

2015-12-03T10:34:53+00:00

Problem:
symlinks are not getting synced to slave in a Tiering based volume.

Solution:
Now, symlinks are created directly in cold tier bricks( in the backend).

Earlier, cold tier was avoided for namespace operations and only
hot tier was used while processing changelogs.

Now, cold tier is HASH subvolume in a Tiering volume.
So, carry out namespace operation only in cold tier subvolume and
avoid hot tier subvolume to avoid any races.

Earlier, XSYNC was used(and changeloghistory avoided) during initial sync
in order to avoid race while processing historychangelog in Hot tier.
This is no longer required as there is no race from Hot tier.

Also, avoid both live and history changelog ENTRY operations from Hot tier to avoid any race with cold tier.

Change-Id: Ia8fbb7ae037f5b6cb683f36c0df5c3fc2894636e
BUG: 1287519
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/12844
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar

geo-rep: Fix syntax errors in GsyncdError

2015-11-23T19:23:45+00:00

%s was not replaced by actual values in GsyncdError

BUG: 1279921
Change-Id: I3c0a10f07383ca72844a46f930b4aa3d3c29f568
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/12566
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur