glusterfs.git/xlators/features/marker/utils/syncdaemon, branch v3.3.2qa1

geo-rep / gsyncd: fixes to communication with child processes

2012-07-16T09:50:28+00:00

due to not using the proper Python keyword, errhandler thread
was possible to run into empty select

Signed-off-by: Csaba Henk 
BUG: 764678
Change-Id: I3c39e718e72545c27d50fd73aa6daf54062331b0
Reviewed-on: http://review.gluster.com/3562
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep / gsyncd: sanitize error log of external commands

2012-07-09T04:14:31+00:00

If a command invoked by gsyncd fails, gsyncd makes a log
of what comes out on its stderr. So far the log indeterministically
broke lines at random places. Now put some effort into reconstructing
original lines and having a faithful log.

BUG: 764678
Change-Id: I16fcc75d3e0f624c10c71d9b37c937ca677087cc
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3563
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep / gsyncd: fix cleanup of temporary mounts

2012-06-10T14:47:50+00:00

[This is a "forward port" of fafd5c17, http://review.gluster.com/2908]

The "finally" clause that was meant to cleanup after the
temp mount has not covered the case of getting signalled
(eg. by monitor, upon worker timing out).

So here we "outsource" the cleanup to an ephemeral child process.
Child calls setsid(2) so it won't be bothered by internal process
management. We use a pipe in between worker and the cleanup child;
when child sees the worker end getting closed, it performs the cleanup.
Worker end can get closed either because worker closes it (normal case),
or because worker has terminated (faulty case) -- thus as bonus, we get
a nice uniform handling with no need to differentiate between normal and
faulty cases.

The faulty case that was seen IRL -- ie., users of maintainance mounts
hang in chdir(2) to mount point -- can be simulated for testing purposes
by applying the following patch:

  diff --git a/xlators/mount/fuse/src/fuse-bridge.c b/xlators/mount/fuse/src/fuse-bridge.c
  index acd3c68..1ce5dc1 100644
  --- a/xlators/mount/fuse/src/fuse-bridge.c
  +++ b/xlators/mount/fuse/src/fuse-bridge.c
  @@ -2918,7 +2918,7 @@ fuse_init (xlator_t *this, fuse_in_header_t *finh, void *msg)
           if (fini->minor < 9)
                   *priv->msg0_len_p = sizeof(*finh) + FUSE_COMPAT_WRITE_IN_SIZE;
   #endif
  -        ret = send_fuse_obj (this, finh, &fino);
  +        ret = priv->client_pid_set ? 0 : send_fuse_obj (this, finh, &fino);
           if (ret == 0)
                   gf_log ("glusterfs-fuse", GF_LOG_INFO,
                           "FUSE inited with protocol versions:"

Change-Id: I14bad56a60a7fa82d0104fa4b9a20f4e42a7186f
BUG: 786291
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3542
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep / gsyncd: add "--super" to rsync invocation

2012-05-27T16:57:46+00:00

This forces rsync to perform supposedly privileged operations on
unprivileged slaves (like chown(2)).

For consistent behavior (with gsyncd's "chown" RPC call that's
being used for symlinks and directories), we also pass
"--numeric-ids" to rsync.

Also took the chance to retire gsyncd's "--rsync-extra" option
which was there for debugging purposes (related to a resolved
issue).

Change-Id: I4ee4d0d3a8c4e0f6746d34d7722c8a567a67491c
BUG: 822121
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3426
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati 
Reviewed-on: http://review.gluster.com/3453
Reviewed-by: Vijay Bellur

geo-rep / gsyncd: typo fix

2012-05-23T09:00:08+00:00

fix topy.

Change-Id: I84df3e850dd24d9e86713dfa401c603a84a81ca6
BUG: 763302
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3375
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.com/3407

geo-rep / gsyncd: recognize ECONNABORTED as termination of aux glusterfs

2012-05-23T08:59:47+00:00

Don't dump stack, rather log the "glusterfs session went down" message.

If the aux glusterfs is already dead when we try to do some file
operation, we get a failure with ENOTCONN, which is already handled
as above. However, it's also possible that glusterfs dies while we
are in a syscall into it -- in that case we get ECONNABORTED, and
so far then we end up with an ugly stack strace. From now on we
take ECONNABORTAD as well into consideration.

Nb. wrt. testing: it's not easy to synthetically force the aux glusterfs
to end this way; for that we have to provoke gsyncd into intensive
synchronization. I succeeded in that with the following ruby oneliner:

ruby -rcgi -e '
  Dir.chdir($*[0])
  a=[]
  Thread.new { loop { while a.size >= 100; File.delete a.shift; end; sleep 1 }}
  loop { a<
Reviewed-on: http://review.gluster.com/3078
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.com/3408

geo-rep / gsyncd: fixes regarding the command invocation framework

2012-05-22T18:25:23+00:00

Some of the bugs to fix were found by the following stress-test:
make "glusterfs --client-pid=-1" exit immediately on slave
side.

Also fix eintr_wrap which should not "adopt" exceptions generated
by the wrapped call, by re-raising them as GsyncdError.

Change-Id: Ia0d39e0635975ebbbf98d86e1e26f3122e1ed6ff
BUG: 764678
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3258
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-on: http://review.gluster.com/3409
Reviewed-by: Vijay Bellur

geo-rep / gsyncd: log sync failures as warnings

2012-04-13T07:25:47+00:00

Syncing of certain files can fail naturally if changes happen on
master (eg. file gets deleted). Therefore logging an error is
misleading.

Change-Id: I7b54904e5ec7c85e4e0fa1e330123d2c44c78ac5
BUG: 764510
Reported-by: Vijaykumar Koppad 
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3113
Tested-by: Gluster Build System 
Reviewed-by: Vijaykumar 
Reviewed-by: Venky Shankar

Fix compiler warnings and typos from Debian build.

2012-04-10T21:37:29+00:00

Mostly to do with "-Werror=format-security" being buggy, but while we're
here we might as well fix some typos and such.  Credit goes to Patrick
Matthäi  for pointing these out.

Change-Id: Ia32d1111d7c10b1f213df85d86b17a1326248ffd
BUG: 811387
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.com/3117
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

geo-rep / gsyncd: shuffle directory entries in crawl

2012-04-05T12:12:59+00:00

In order to randomize the walk of the file tree.

Change-Id: I9fc3b83d5804914a50faae8df7dbcfed2ba6f4b4
BUG: 809675
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3079
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar