glusterfs.git/geo-replication, branch dht-stale-layout-fixes

build: make GLUSTERD_WORKDIR rely on localstatedir

2014-08-07T08:17:29+00:00

- Break-way from '/var/lib/glusterd' hard-coded previously,
  instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
  management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
  platforms.
- Define proper environment for running tests, define correct PATH
  and LD_LIBRARY_PATH when running tests, so that the desired version
  of glusterfs is used, regardless where it is installed.
  (Thanks to manu@netbsd.org for this additional work)

Change-Id: I2339a0d9275de5939ccad3e52b535598064a35e7
BUG: 1111774
Signed-off-by: Harshavardhana 
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8246
Tested-by: Gluster Build System

geo-rep: minimize xsync crawl usage and set upper limit to xsync crawl

2014-07-24T03:36:02+00:00

For effective handling of deletes and renames use history crawl
as much as possible. History crawl will run in loop till it syncs
all data before live changelog time.

When it uses xsync crawl(fallback when changelog not available, or
very first crawl) it sets upper limit to crawl.

After completing History crawl, it checks actual end time returned
by history api to compare with register time, if actual end is less
than register time then run history crawl one more time.

If first turn history processing time is less than the CHANGELOG
ROLLOVER TIME then sleep for the difference, After sleep if it is
guaranteed that rollover will happen and switches to live changelog
consumption without switching to xsync. This sleep is only when
history processing completed < CHANGELOG_ROLLOVER_TIME and sleep
only after the first turn, So will not affect the performance.

BUG: 1112238
Change-Id: I644ef24b07e42e81cec96a025ebd21244a555ec0
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8151
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: Avoid duplicate stat in xsync changelog processing

2014-07-09T08:52:24+00:00

When A file/dir is identified for metadata sync, it was doing
duplicate stat to get the metadata to sync.

With this patch it avoids doing one additional stat call. Xsync
performance will improve.

rsync will copy files metadata, so no need to include for
processing.

BUG: 1111490

Change-Id: I79dad6375fa4742d9aaca7d9856993c184a744dc
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8124
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: History Change detector method select issue

2014-07-01T10:59:01+00:00

Geo-rep does history crawl even if change-detector is set
to xsync.

This patch fixes by taking priority to user configured
change detector.

BUG: 1113525
Change-Id: Ic6c34e187c9cb6608c9ef8a010ea07015ba60a80
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8183
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: Fix the fd leak in worker/agent spawn

2014-06-30T10:40:45+00:00

worker and agent uses pipe to communicate, if worker dies
for some reason agent should get EOF and terminate.

Each worker-agent spawning is done in thread, Due to race
if multiple workers in same node retain the pipe refs of
other workers. Hence agent will not get EOF even if
worker dies.

BUG: 1114003
Change-Id: I36b9709b9392299483606bd3ef1db764fa3f2bff
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8194
Tested-by: Justin Clift 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

feature/geo-rep: Fix to retain pause state of gsyncd on restart

2014-06-20T08:17:22+00:00

On soft reboot, geo-rep monitor is writing 'faulty' into status
file. It should not do it if previous state is paused as
glusterd depend on the state file on node restart.

Change-Id: Idd45abf13350b087371935f1b4f6e1a346433d27
BUG: 1101410
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8097
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

feature/geo-rep: Fix for changelog agent becoming zombie.

2014-06-18T04:47:07+00:00

Monitor process spawns changelog agent and is not
wait on it, hence becoming zombie. When worker is
dies/killed, it respawns both worker and corresponding
agent leaving the earlier changelog agent in zombie
state. This patch addresses this issue by waiting
on agent process in montor process.

Change-Id: I571b7d6487133848edca67e7446f1caa70ae01c9
BUG: 1103643
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/7956
Tested-by: Gluster Build System 
Reviewed-by: Aravinda VK 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep/gverify: Never use ping to check for host reachability

2014-06-12T12:48:23+00:00

On many linux distributions with iptables enabled, ICMP traffic
is usually dropped even when port 22 is open for SSH service

So practically `ping` is an unreliable command

~~~
root@rhs1:/var/log/glusterfs # gluster volume geo-replication geo-test \
                          17.16.10.1::geo-test-slave create push-pem force
172.16.10.1 not reachable.
geo-replication command failed
~~~

~~~
root@rhs1:/var/log/glusterfs # ping 172.16.10.1
PING rhs2.sjc.redhat.com (172.16.10.1) 56(84) bytes of data.
From rhs2.sjc.redhat.com (172.16.10.1) icmp_seq=1 Destination Host Prohibited
...
...
~~~

~~~
root@rhs2:/var/log/glusterfs # service iptables status | grep 22
4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:22
root@rhs2:/var/log/glusterfs # service iptables status | grep icmp-host-prohibited
25   REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited
~~~

Change-Id: I33206ca071aa5d755c0762f7c486da222ec3c7db
BUG: 1105337
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/7997
Tested-by: Gluster Build System 
Reviewed-by: Aravinda VK 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: Making replica failover check interval configurable

2014-06-12T06:55:31+00:00

Replica failover check interval is hardcoded to 60 sec
by default. Now this option is made configurable and
defaulted to 1 sec.

To change the default value
gluster volume geo-replication  \
	:: config replica_failover_interval 15

Change-Id: Iada1b80d510452dcfedebd8a21bebd62394b0597
BUG: 1066410
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8003
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System 
Tested-by: Venky Shankar

geo-rep: entry_ops errors handling

2014-06-12T06:52:52+00:00

Xattr.lsetxattr_l call will not raise OSError which
errno_wrap can handle, so we need to use Xattr.lsetxattr
to get proper OSError and errno_wrap ignores or retries
accordingly.

Change-Id: Ie0a777152ddbaf9ed80c977e4704974fec997bea
BUG: 1105083
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7972
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar