glusterfs-afrv1.git/geo-replication/syncdaemon, branch master

geo-rep: Fix ValueError - signal only works in main thread

2014-03-21T06:25:43+00:00

When a worker process not confirmed within 60 seconds of start
then monitor thread was terminated instead of stopping and restarting
the worker thread.
Before terminate monitor thread tries to add a signal handler for
SIGTERM to cleanup the stuff before terminate. Signal handling will
not work inside thread, so ValueError was raised.

This patch will not terminate monitor thread, instead only kills and
restarts the worker.

Change-Id: I14df26c0cc3097af29293c81536c13b86075e28f
BUG: 1078068
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7294
Reviewed-by: Venky Shankar 
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

geo-rep: quick-fix for remote xtime set failed

2014-03-08T04:14:04+00:00

Remote xtime is required for failover/failback,
this patch is quick fix to avoid the OSError.

Code is masked out, this need to be resolved when
failover/failback is worked on.

Change-Id: If339d88a2ccd8ef18a3b3c015df765c93dcb020c
BUG: 1073844
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/7206
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: Config file upgrade

2014-02-11T01:05:44+00:00

When old config file is used with new geo-rep, config item like
'georep_session_working_dir' was missing in old config file.

With this patch geo-rep sets the default value for new items.

Following config options supported:
  - georep_session_working_dir
  - gluster_params
  - ssh_command_tar

BUG: 1036539
Change-Id: I389c62e749f3b567f9ecf96d4b41367ef962c025
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/6934
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System

gsyncd / geo-rep: invoke changelog process() on non-empty change list

2014-02-06T09:55:58+00:00

Change-Id: Ida4890abdc90d683a4a83127a1573bbb3829ea23
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6793
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gsyncd / geo-rep: ignore DHTs sticky bit file during crawl

2014-02-06T09:54:52+00:00

Change-Id: Ide927759c6a3d5301475eac9f6e785aa901d426e
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6792
Reviewed-by: Kotresh HR 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gsyncd / geo-rep: "patch" up missing stime

2014-02-06T09:53:08+00:00

In cases (mostly upgrade) of unavailability of "stime" key and
availability of "xtime" (slave's xtime), introduce "stime" key
on the fly by setting it to the value to "xtime".

Change-Id: Iaa424662d838154c8abc2cf00830c7f9d6be45ac
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6791
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gsyncd / geo-rep: cleanup the "tar" process

2014-02-06T09:50:46+00:00

A missing cleanup for the "tar" process (when tar+ssh
is used as the sync engine).

Change-Id: Ib9599b43e7ec606c70b7c5598793417142be3c0b
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6794
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: get new config value after config value reset

2014-01-23T17:45:30+00:00

When config.read is called it preserves the previous values
from the the previously opened config file.

For example
import ConfigParser
config = ConfigParser.RawConfigParser()
config.read("defaults.conf")
config.read("preferences.conf")

When change in config file is identified it will open new
instance of config to avoid getting old config values.

Change-Id: Iec677e61ebd2c59c95aea94481f569d78bd913e4
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/6747
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: optimizing update stime after directory synchronization

2014-01-23T05:02:38+00:00

Since xsync crawl generates new changelog when number of entries
reaches 8K or when directory is reached. If a directory has number
of files less than 8K then respective changelog file will have less
entries. Since xsync generated changelog files processed one after
the other, so syncjobs are underutilized. hence low bandwidth
utilization.

With this patch, changelog will be generated for 8K entries only,
but stime will be accumulated. Multiple dirs stime will be
updated together since the generated changelog will have entries
accross the dirs.

Change-Id: Ib0b40962a070f855f47f887d0840e412fb7928e1
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/6744
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

gsyncd / geo-rep: geo-replication fixes

2013-12-12T08:16:03+00:00

-> "threaded" hybrid crawl.
-> Enabling metatadata synchronization.
-> Handling EINVAL/ESTALE gracefully while syncing metadata.
-> Improvments to changelog crawl code.
-> Initial crawl changelog generation format.
-> No gsyncd restart when checkpoint updated.
-> Fix symlink handling in hybrid crawl.
-> Slave's xtime key is 'stime'.
-> tar+ssh as data synchronization.
-> Instead of 'raise', just log in warning level for xtime missing cases.
-> Fix for JSON object load failure
-> Get new config value after config value reset.
-> Skip already processed changelogs.
-> Saving status of each individual worker thread.
-> GFID fetch on slave for purges.
-> Add tar ssh keys and config options.
-> Fix nlink count when using backend.
-> Include "data" operation for hardlink.
-> Use changelog time prefix as slave's time.
-> Process changelogs in parallel.

Change-Id: I09fcbb2e2e418149a6d8435abd2ac6b2f015bb06
BUG: 1036539
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/6404
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati