glusterfs.git/geo-replication/syncdaemon, branch v3.5.4

geo-rep: Fix ValueError - signal only works in main thread

2014-03-24T17:30:32+00:00

When a worker process not confirmed within 60 seconds of start
then monitor thread was terminated instead of stopping and restarting
the worker thread.
Before terminate monitor thread tries to add a signal handler for
SIGTERM to cleanup the stuff before terminate. Signal handling will
not work inside thread, so ValueError was raised.

This patch will not terminate monitor thread, instead only kills and
restarts the worker.

BUG: 1078068
Change-Id: Icf0df7ef492da636d0d20e42750747e404d897df
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7294
Reviewed-on: http://review.gluster.org/7313
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: quick-fix for remote xtime set failed

2014-03-08T04:14:42+00:00

Remote xtime is required for failover/failback,
this patch is quick fix to avoid the OSError.

Code is masked out, this need to be resolved when
failover/failback is worked on.

Change-Id: If339d88a2ccd8ef18a3b3c015df765c93dcb020c
BUG: 1073844
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/7207
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: Config file upgrade

2014-02-11T01:07:42+00:00

When old config file is used with new geo-rep, config item like
'georep_session_working_dir' was missing in old config file.

With this patch geo-rep sets the default value for new items.

Following config options supported:
  - georep_session_working_dir
  - gluster_params
  - ssh_command_tar

BUG: 1036539
Change-Id: If277a14d21f4caee568908b73a2988352c08dcc8
Reviewed-on: http://review.gluster.org/6934
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/6935
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gsyncd / geo-rep: cleanup the "tar" process

2014-02-06T09:58:28+00:00

A missing cleanup for the "tar" process (when tar+ssh
is used as the sync engine).

Change-Id: Ib9599b43e7ec606c70b7c5598793417142be3c0b
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6794
Reviewed-on: http://review.gluster.org/6861
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gsyncd / geo-rep: invoke changelog process() on non-empty change list

2014-02-06T09:58:10+00:00

Change-Id: Ida4890abdc90d683a4a83127a1573bbb3829ea23
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6793
Reviewed-on: http://review.gluster.org/6860
Tested-by: Gluster Build System 
Reviewed-by: Aravinda VK

gsyncd / geo-rep: ignore DHTs sticky bit file during crawl

2014-02-06T09:56:40+00:00

Change-Id: I0a6eeee8ae192d086b34d316ee1b82c7d76634da
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6792
Reviewed-on: http://review.gluster.org/6859
Reviewed-by: Kotresh HR 
Tested-by: Gluster Build System 
Reviewed-by: Aravinda VK

gsyncd / geo-rep: "patch" up missing stime

2014-02-06T09:56:25+00:00

In cases (mostly upgrade) of unavailability of "stime" key and
availability of "xtime" (slave's xtime), introduce "stime" key
on the fly by setting it to the value to "xtime".

Change-Id: Iaa424662d838154c8abc2cf00830c7f9d6be45ac
BUG: 1036539
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/6791
Reviewed-on: http://review.gluster.org/6858
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: get new config value after config value reset

2014-02-03T04:26:40+00:00

When config.read is called it preserves the previous values
from the the previously opened config file.

For example
import ConfigParser
config = ConfigParser.RawConfigParser()
config.read("defaults.conf")
config.read("preferences.conf")

When change in config file is identified it will open new
instance of config to avoid getting old config values.

BUG: 1036539
Change-Id: Ie3226e4a75cbff9e69ad2d7cb5f4daf45c6ae0c8
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/6747
Reviewed-on: http://review.gluster.org/6845
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar

geo-rep: optimizing update stime after directory synchronization

2014-02-03T04:25:27+00:00

Since xsync crawl generates new changelog when number of entries
reaches 8K or when directory is reached. If a directory has number
of files less than 8K then respective changelog file will have less
entries. Since xsync generated changelog files processed one after
the other, so syncjobs are underutilized. hence low bandwidth
utilization.

With this patch, changelog will be generated for 8K entries only,
but stime will be accumulated. Multiple dirs stime will be
updated together since the generated changelog will have entries
accross the dirs.

BUG: 1036539
Change-Id: Ic4b9da6d8c1f11dd9ef7c13bcc7df80efe2c5caf
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/6744
Reviewed-on: http://review.gluster.org/6847
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-by: Vijay Bellur

gsyncd / geo-rep: geo-replication fixes

2014-01-27T17:43:19+00:00

-> "threaded" hybrid crawl.
-> Enabling metatadata synchronization.
-> Handling EINVAL/ESTALE gracefully while syncing metadata.
-> Improvments to changelog crawl code.
-> Initial crawl changelog generation format.
-> No gsyncd restart when checkpoint updated.
-> Fix symlink handling in hybrid crawl.
-> Slave's xtime key is 'stime'.
-> tar+ssh as data synchronization.
-> Instead of 'raise', just log in warning level for xtime missing cases.
-> Fix for JSON object load failure
-> Get new config value after config value reset.
-> Skip already processed changelogs.
-> Saving status of each individual worker thread.
-> GFID fetch on slave for purges.
-> Add tar ssh keys and config options.
-> Fix nlink count when using backend.
-> Include "data" operation for hardlink.
-> Use changelog time prefix as slave's time.
-> Process changelogs in parallel.

Change-Id: I09fcbb2e2e418149a6d8435abd2ac6b2f015bb06
BUG: 1036539
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/6404
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6809
Reviewed-by: Vijay Bellur