glusterfs.git/xlators/features/changelog, branch v3.10dev

core: fix unused variable warnings/errors

2016-08-29T16:21:56+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: I3d8a7a3de35058aa97eab59d3f59208396298b03
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15246
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

features/libgfchangelog: Log failure in gf_histroy_changelog

2016-08-09T07:27:12+00:00

Add error logs if gf_history_changelog fails. If requested
changelog range is not available, log the error and exit
instead of continuing the loop and exiting in readdir
without logging. Also fixed the duplicate MSGID number in
'changelog-lib-messages.h'

Change-Id: Icd71b89ae23b48a71380657ba5649029c32fabfd
BUG: 1362151
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/15064
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Aravinda VK

changelog/rpc: Fix rpc_clnt_t mem leaks

2016-07-22T15:12:52+00:00

PROBLEM:
   1. Freeing up rpc_clnt object might lead to crashes. Well,
      it was not a necessity to free rpc-clnt object till now
      because all the existing use cases needs to reconnect
      back on disconnects. Hence timer code was not taking
      ref on rpc-clnt object.

      Glusterd had some use-cases that led to crash due to
      ping-timer and they fixed only those code paths that
      involve ping-timer.

      Now, since changelog has an use-case where rpc-clnt
      need to be freed up, we need to fix timer code to take
      refs

   2. In changelog, because of issue 1, only mydata was being
      freed which is incorrect. And there are races where
      rpc-clnt object would access the freed mydata which
      would lead to crashes.

      Since changelog xlator resides on brick side and is long
      living process, if multiple libgfchangelog consumers
      register to changelog and disconnect/reconnect mulitple
      times, it would result in leak of 'rpc-clnt' object
      for every connect/disconnect.

SOLUTION:
   1. Handle ref/unref of 'rpc_clnt' structure in timer
      functions properly.
   2. In changelog, unref 'rpc_clnt' in RPC_CLNT_DISCONNECT
      after disabling timers and free mydata on RPC_CLNT_DESTROY.

RPC SETUP IN CHANGELOG:
   1. changelog xlator initiates rpc server say 'changelog_rpc_server'
   2. libgfchangelog initiates one rpc server say 'libgfchangelog_rpc_server'
   3. libgfchangelog initiates rpc client and connects to 'changelog_rpc_server'
   4. In return changelog_rpc_server initiates a rpc client and connects back
      to 'libgfchangelog_rpc_server'

REF/UNREF HANDLING IN TIMER FUNCTIONS:
Let's say rpc clnt refcount = 1
   1. Take the ref before reigstering callback to timer queue
           >>>>  rpc_clnt_ref (say ref count becomes = 2)
   2. Register a callback to timer say 'callback1'
   3. If register fails:
           >>>> rpc_clnt_unref (ref count = 1)
   4. On timer expiration, 'callback1' gets called. So unref rpc clnt at the end
      in 'callback1'. This is corresponding to ref taken in step 1
           >>>> rpc_clnt_unref (ref count = 1)
   5. The cycle from step-1 to step-4 continues....until timer cancel event happens
   6. timer cancel of say 'callback1'
           If timer cancel fails:
                 Do nothing, Step-4 would have unrefd
           If timer cancel succeeds:
                 >>>> rpc_clnt_unref (ref count = 1)

Change-Id: I91389bc511b8b1a17824941970ee8d2c29a74a09
BUG: 1316178
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13658
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

core: use readdir(3) with glibc, and associated cleanup

2016-07-18T11:59:42+00:00

Starting with glibc-2.23 (i.e. what's in Fedora 25), readdir_r(3)
is marked as deprecated. Specifically the function decl in 
has the deprecated attribute, and now warnings are thrown during the
compile on Fedora 25 builds.

The readdir(_r)(3) man page (on Fedora 25 at least) and World+Dog say
that glibc's readdir(3) is, and always has been, MT-SAFE as long as
only one thread is accessing the directory object returned by opendir().
World+Dog also says there is a potential buffer overflow in readdir_r().
World+Dog suggests that it is preferable to simply use readdir(). There's
an implication that eventually readdir_r(3) will be removed from glibc.
POSIX has, apparently deprecated it in the standard, or even removed it
entirely.

Over and above that, our source near the various uses of readdir(_r)(3)
has a few unsafe uses of strcpy()+strcat().

(AFAIK nobody has looked at the readdir(3) implemenation in *BSD to see
if the same is true on those platforms, and we can't be sure of MacOS
even though we know it's based on *BSD.)

Change-Id: I5481f18ba1eebe7ee177895eecc9a80a71b60568
BUG: 1356998
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/14838
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Jeff Darcy

build: correctly format some (s)size_t messages

2016-07-17T13:36:58+00:00

On 32-bit builds the are are warnings like these:

    posix.c:6438: warning: format '%ld' expects type 'long int', but argument 11 has type 'ssize_t'

Instead of using "%l" for (signed) size_t variables, "%z" should be
used.

BUG: 1198849
Change-Id: I6f57b5e8ea174dd9e3056aff5da685e497894ccf
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14933
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Smoke: Gluster Build System

libglusterfs (timer): race conditions, illegal mem access, mem leak

2016-06-01T13:50:13+00:00

While investigating gfapi memory consumption with valgrind, valgrind
reported several memory access issues.

Also see the timer 'registry' being recreated (shortly) after being
freed during teardown due to the way it's currently written.

Passing ctx as data to gf_timer_proc() is prone to memory access
issues if ctx is freed before gf_timer_proc() terminates. (And in
fact this does happen, at least in valgrind.) gf_timer_proc() doesn't
need ctx for anything, it only needs ctx->timer, so just pass that.

Nothing ever calls gf_timer_registry_init(). Nothing outside of
timer.c that is. Making it and gf_timer_proc() static.

Change-Id: Ia28454dda0cf0de2fec94d76441d98c3927a906a
BUG: 1333925
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14247
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Poornima G 
Reviewed-by: Niels de Vos 
Reviewed-by: Jeff Darcy

features/changelog: Change barrier notification mechanism

2016-05-31T18:59:58+00:00

The barrier notification mechanism was fd based and 'select'
was being used. 'select' breaks when number of fds opened
by brick process exceeds 1024. To avoid this and also
the maintainance of pipe between notify and 'changelog_rollover',
the pipe has been replaced with pthread condition signal
and timed wait mechanism.

Change-Id: I530ea90d9a06953f8b23b4e12d122872ee1925de
BUG: 1334314
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14272
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Aravinda VK 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Jeff Darcy

remove unused variables

2016-05-06T09:31:31+00:00

Cleaning dead initializations.

Change-Id: I53ae506593cd10441d61df3b2c9249544a7871f7
BUG: 1254067
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/11932
Tested-by: Prasanna Kumar Kalever 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur

features/changelog: Don't modify 'pargfid' in 'resolve_pargfid_to_path'

2016-03-31T04:23:19+00:00

If 'changelog' is enabled and 'changelog.capture-del-path' option is on
it calls 'resolve_pargfid_to_path' which modifies 'pargfid' sent by
caller.  'changelog_unlink' calls this routine directly with
'loc->pargfid' resulting it being modified and point to root instead of
actual pargfid. This is a nasty bug and could cause the deletion of
entry on root directory instead on actual parent when 'loc->path' is
not present.  Hence this fix to make 'pargfid' a const pointer and
'resolve_pargfid' to work on copy of pargfid.

Glusterfind session creation enables these options by default to
capture deleted entry path in changelog.

Thanks Pranith for root causing this.

Change-Id: I1d275a86f91c981b6889bedef93401c039d01d71
BUG: 1321955
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13845
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Venky Shankar 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System

changelog: fix typecasting of function

2016-03-17T15:22:13+00:00

fix compiler error:
"error: void value not ignored as it ought to be"

while the code tries to typecast return type of
'LOCK_DESTROY(lock)' to (void)

Change-Id: I38ed47f3a12719c7a4a59388c0d021858d4dfe6d
BUG: 1312354
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/13533
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy