glusterfs.git/xlators/features/marker/utils, branch v3.3.2qa1

localtime and ctime are not MT-SAFE

2013-02-08T20:04:42+00:00

There are a number of nit-level issues throughout the source with
the use of localtime and ctime. While they apparently aren't causing
too many problems, apart from the one in bz 828058, they ought to be
fixed. Among the "real" problems that are fixed in this patch:
1) general localtime and ctime not MT-SAFE. There's a non-zero chance
   that another thread calling localtime (or ctime) will over-write
   the static data about to be used in another thread
2) localtime(& <64-bit-type>) or ctime(& <64-bit-type>) generally
   not a problem on 64-bit or little-endian 32-bit. But even though
   we probably have zero users on big-ending 32-bit platforms, it's
   still incorrect.
3) multiple nested calls passed as params. Last one wins, i.e. over-
   writes result of prior calls.
4) Inconsistent error handling. Most of these calls are for logging,
   tracing, or dumping. I submit that if an error somehow occurs in
   the call to localtime or ctime, the log/trace/dump still should
   still occur.
5) Appliances should all have their clocks set to UTC, and all log
   entries, traces, and dumps should use GMT.
6) fix strtok(), change to strtok_r()

Other things this patch fixes/changes (that aren't bugs per se):
1) Change "%Y-%m-%d %H:%M:%S" and similar to their equivalent shorthand,
   e.g. "%F %T"
2) change sizeof(timestr) to sizeof timestr. sizeof is an operator,
   not a function. You don't use i +(32), why use sizeof().
   (And yes, you do use parens with sizeof().)
3) change 'char timestr[256]' to 'char timestr[32]' where appropriate.
   Per-thread stack is limited. Time strings are never longer than ~20
   characters, so why waste 220+ bytes on the stack?

Things this patch doesn't fix:
1) hodgepodge of %Y-%m-%d %H:%M:%S versus %Y/%m/%d-%H%M%S and other
   variations. It's not clear to me whether this ever matters, not to
   mention 3rd party log filtering tools may already rely on a
   particular format. Still it would be nice to have a single manifest
   constant and have every call to localtime/strftime consistently use
   the same format.

BUG: 832173

Change-Id: Iee9719db4576eacc6c75694d9107954d0912cba8
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/3613
Reviewed-by: Anand Avati 
Tested-by: Anand Avati

geo-rep / gsyncd: fixes to communication with child processes

2012-07-16T09:50:28+00:00

due to not using the proper Python keyword, errhandler thread
was possible to run into empty select

Signed-off-by: Csaba Henk 
BUG: 764678
Change-Id: I3c39e718e72545c27d50fd73aa6daf54062331b0
Reviewed-on: http://review.gluster.com/3562
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

NetBSD build fixes

2012-07-13T21:08:22+00:00

This is a backport of Change-Id: Icd7290f1e340675d763665a0d0c5f95bc14e0c55

BUG: 764655
Change-Id: Iaca3dc30e61c0864af673b90d2a7fdea6a3143cc
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.com/3577
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

geo-rep / gsyncd: sanitize error log of external commands

2012-07-09T04:14:31+00:00

If a command invoked by gsyncd fails, gsyncd makes a log
of what comes out on its stderr. So far the log indeterministically
broke lines at random places. Now put some effort into reconstructing
original lines and having a faithful log.

BUG: 764678
Change-Id: I16fcc75d3e0f624c10c71d9b37c937ca677087cc
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3563
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

NetBSD build fixes.

2012-07-05T18:28:31+00:00

This is a backport of Change-Id: Ib8183d4b585465d05a7adf3a4ceae93ae1bded15

BUG: 764655
Change-Id: I552b87b72c234b3a11af6ffd4a03975879602363
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.com/3574
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

geo-rep / gsyncd: fix cleanup of temporary mounts

2012-06-10T14:47:50+00:00

[This is a "forward port" of fafd5c17, http://review.gluster.com/2908]

The "finally" clause that was meant to cleanup after the
temp mount has not covered the case of getting signalled
(eg. by monitor, upon worker timing out).

So here we "outsource" the cleanup to an ephemeral child process.
Child calls setsid(2) so it won't be bothered by internal process
management. We use a pipe in between worker and the cleanup child;
when child sees the worker end getting closed, it performs the cleanup.
Worker end can get closed either because worker closes it (normal case),
or because worker has terminated (faulty case) -- thus as bonus, we get
a nice uniform handling with no need to differentiate between normal and
faulty cases.

The faulty case that was seen IRL -- ie., users of maintainance mounts
hang in chdir(2) to mount point -- can be simulated for testing purposes
by applying the following patch:

  diff --git a/xlators/mount/fuse/src/fuse-bridge.c b/xlators/mount/fuse/src/fuse-bridge.c
  index acd3c68..1ce5dc1 100644
  --- a/xlators/mount/fuse/src/fuse-bridge.c
  +++ b/xlators/mount/fuse/src/fuse-bridge.c
  @@ -2918,7 +2918,7 @@ fuse_init (xlator_t *this, fuse_in_header_t *finh, void *msg)
           if (fini->minor < 9)
                   *priv->msg0_len_p = sizeof(*finh) + FUSE_COMPAT_WRITE_IN_SIZE;
   #endif
  -        ret = send_fuse_obj (this, finh, &fino);
  +        ret = priv->client_pid_set ? 0 : send_fuse_obj (this, finh, &fino);
           if (ret == 0)
                   gf_log ("glusterfs-fuse", GF_LOG_INFO,
                           "FUSE inited with protocol versions:"

Change-Id: I14bad56a60a7fa82d0104fa4b9a20f4e42a7186f
BUG: 786291
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3542
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep / gsyncd: add "--super" to rsync invocation

2012-05-27T16:57:46+00:00

This forces rsync to perform supposedly privileged operations on
unprivileged slaves (like chown(2)).

For consistent behavior (with gsyncd's "chown" RPC call that's
being used for symlinks and directories), we also pass
"--numeric-ids" to rsync.

Also took the chance to retire gsyncd's "--rsync-extra" option
which was there for debugging purposes (related to a resolved
issue).

Change-Id: I4ee4d0d3a8c4e0f6746d34d7722c8a567a67491c
BUG: 822121
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3426
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati 
Reviewed-on: http://review.gluster.com/3453
Reviewed-by: Vijay Bellur

geo-rep / gsyncd: typo fix

2012-05-23T09:00:08+00:00

fix topy.

Change-Id: I84df3e850dd24d9e86713dfa401c603a84a81ca6
BUG: 763302
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3375
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.com/3407

geo-rep / gsyncd: recognize ECONNABORTED as termination of aux glusterfs

2012-05-23T08:59:47+00:00

Don't dump stack, rather log the "glusterfs session went down" message.

If the aux glusterfs is already dead when we try to do some file
operation, we get a failure with ENOTCONN, which is already handled
as above. However, it's also possible that glusterfs dies while we
are in a syscall into it -- in that case we get ECONNABORTED, and
so far then we end up with an ugly stack strace. From now on we
take ECONNABORTAD as well into consideration.

Nb. wrt. testing: it's not easy to synthetically force the aux glusterfs
to end this way; for that we have to provoke gsyncd into intensive
synchronization. I succeeded in that with the following ruby oneliner:

ruby -rcgi -e '
  Dir.chdir($*[0])
  a=[]
  Thread.new { loop { while a.size >= 100; File.delete a.shift; end; sleep 1 }}
  loop { a<
Reviewed-on: http://review.gluster.com/3078
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.com/3408

geo-rep / gsyncd: fixes regarding the command invocation framework

2012-05-22T18:25:23+00:00

Some of the bugs to fix were found by the following stress-test:
make "glusterfs --client-pid=-1" exit immediately on slave
side.

Also fix eintr_wrap which should not "adopt" exceptions generated
by the wrapped call, by re-raising them as GsyncdError.

Change-Id: Ia0d39e0635975ebbbf98d86e1e26f3122e1ed6ff
BUG: 764678
Signed-off-by: Csaba Henk 
Reviewed-on: http://review.gluster.com/3258
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-on: http://review.gluster.com/3409
Reviewed-by: Vijay Bellur