glusterfs.git/xlators/features/changelog/lib/src/gf-changelog.c, branch v3.10.3

libglusterfs (timer): race conditions, illegal mem access, mem leak

2016-06-01T13:50:13+00:00

While investigating gfapi memory consumption with valgrind, valgrind
reported several memory access issues.

Also see the timer 'registry' being recreated (shortly) after being
freed during teardown due to the way it's currently written.

Passing ctx as data to gf_timer_proc() is prone to memory access
issues if ctx is freed before gf_timer_proc() terminates. (And in
fact this does happen, at least in valgrind.) gf_timer_proc() doesn't
need ctx for anything, it only needs ctx->timer, so just pass that.

Nothing ever calls gf_timer_registry_init(). Nothing outside of
timer.c that is. Making it and gf_timer_proc() static.

Change-Id: Ia28454dda0cf0de2fec94d76441d98c3927a906a
BUG: 1333925
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14247
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Poornima G 
Reviewed-by: Niels de Vos 
Reviewed-by: Jeff Darcy

libgfchangelog: Allocate logbuf_pool in master xlator

2015-12-22T09:33:14+00:00

The master xlator needs to allocate 'logbuf_pool'
else 'gf_msg' fails with EINVAL.

Change-Id: I6b2d3450250de7e77126d12b75b0dbc4db414bfb
BUG: 1292463
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/12997
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar

features/changelog: Porting log messages to new logging framework

2015-08-12T05:36:44+00:00

Change-Id: Ic7f842acca52908fd88e0796dc90b82650405b25
BUG: 1194640
Signed-off-by: Anusha Rao 
Reviewed-on: http://review.gluster.org/10532
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-by: Kotresh HR

libgfchangelog: Fix crash in gf_changelog_process

2015-06-18T06:34:13+00:00

Problem:
    Crash observed in gf_changelog_process and
    gf_changelog_callback_invoker.

Cause:
    Assignments to arguments passed to thread is done
    post thread creation. If the thread created gets
    scheduled before the assignment and access these
    variables, it would crash with segmentation fault.

Solution:
    Assignments to arguments are done prior to the thread
    creation.

Change-Id: I6afc8ccedd050cf4b50b967fef8287a0c834177b
BUG: 1232666
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/11273
Tested-by: NetBSD Build System 
Reviewed-by: Venky Shankar

features/changelog: fix possible illegal memory access (CID 1288821)

2015-04-06T06:33:19+00:00

Coverity CID 1288821

strncpy executed with a limit equal to the target array
size potentially leaves the target string not null terminated.
Make sure the copied string is a valid 0 terminated string.

Change-Id: I5ecc60549864cf50e3facbb8a2d9228cd67b57b5
BUG: 789278
Signed-off-by: Michael Adam 
Reviewed-on: http://review.gluster.org/10061
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

libgfchangelog: Use correct 'this' pointer on new thread creation

2015-04-03T10:30:23+00:00

When libgfchangelog is linked with non xlator application,
it should point to 'master' xlator which is initiated separately.
When ever a new thread is created, 'THIS' points to the global
xlator. 'THIS' should point to corresponding xlator even then.
This patch adjusts the pointer accordingly.

Change-Id: I2a199bb3c73146a0329540aedcbae697a00f6f0a
BUG: 1207643
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/10074
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

libgfchangelog: cleanups on disconnection

2015-03-24T13:32:39+00:00

[WIP patch as of now, just needs a little tweak]

A pending TODO in the code caused regressions to fail as
bitrot daemons are spawned during volume start (equivalent
to enabling bitrot by default). The problematic part that
casued such failures is during brick disconnections with
unsafe handling of event data structured in the code.

With this patch, data structures are properly cleaned up
with care taken to cleanup all accessors first. This also
fixes potential memory leaks which was bluntly ignored
before.

Change-Id: I70ed82cb1a0fb56c85ef390007e321a97a35c5ce
BUG: 1170075
Signed-off-by: Venky Shankar 
original-author: Venky Shankar 
Reviewed-on: http://review.gluster.org/9959
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

libgfchangelog: Fix faulty reference to xlator context

2015-03-24T13:25:15+00:00

Problem:
libgfchangelog initializes global xlator on library load (via
constructor: _ctor) and mangles it's xlator context thereby
messing with certain important members of the command structure.
On receiving an RPC disconnection event, if the point-of-execution
was in libgfchangelogs context, accessing ->cmd_args during RPC
notify resulted in a segfault.

Fix:
Since the libarary needs to be able to work with processes that
have a notion of an xlator (THIS in particular) and without it,
care needs to be taken to allocate the global xlator when needed.
Moreover, the actual fix is to use the correct xlator context
in both cases. A new API is introduces when needs to be invoked
by the conusmer (although this could have been done during
register() call, keeping it a separate API makes thing flexible
and easy).

Test:
The issue is observed when a brick process goes offline. This is
triggered when test cases (.t's) are run in bulk, since each
test essestially spawns bricks processes (on volume start) and
terminates them (volume stop). Since bitrot daemon, as of now,
spawns upon volume start, the issue is much observed when the
volume is taken offline at the end of each test case. With this
fix, running the basic and core test cases along with building
the linux kernel has passed without daemon segfaults.

Thanks to Johnny (rabhat@) for helping in debugging the issue
(and with the fix :)).

Change-Id: I8d3022bf749590b2ee816504ed9b1dfccc65559a
BUG: 1170075
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9953
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

features/changelog: RPC'fy {libgf}changelog

2015-03-19T01:22:36+00:00

This patch introduces RPC based communication between the changelog
translator and libgfchangelog. It replaces the old pathetic stream
based interaction that existed earlier (due to time constraints :-/).

Changelog, upon initialization starts a RPC server (rpcsvc) allowing
clients to invoke a probe API as a bootup mechanism to request for
event notifications. During probe, clients can choose an event
filter specifying the type(s) of events they are interested in. As
of now there is no way to change the event notification set once
the probe RPC call is made, but that is easier to implement.

The actual event notifications is done on a separate RPC session.
The client (libgfchangelog) itself starts and RPC server which the
changelog translator "connects back" during probe. Notifications
are dispatched by a bunch of threads from the server (translator)
and the client optionally orders them if ordered notifications
are requried. FOPs fill in their respective event details in a
buffer (rot-buffs to be particular) and a bunch of threads
(consumers) swap the buffers out of roatation and dispatch them
via RPC. To avoid writer starvation, then number of dispatcher
threads is one less than the number of buffer list in rot-buffs.x

libgfchangelog becomes purely callback based -- upon event
notification from the server (and re-ordering them if required)
invoke a callback routine specified by consumer(s).

A major part of the patch is also aimed at providing backward
compatibility for geo-replication, which was one of the main
consumer of the stream based API. Also, this patch does not\
"turn on" event notifications for all fops, just a bunch which
is currently in requirement. Another pain point is that the
server does not filter events before dispatching it to the
clients. That load is taken up by the client itself (although
it's done at the library layer rather than making it hard on
the callback implementor). This needs improvement and care
needs to be taken to not load the server up with expensive
filtering mechanisms.

Change-Id: Ibf60a432b68f2dfa60c6f9add2bcfd37a9c41395
BUG: 1170075
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9708
Reviewed-by: Jeff Darcy 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

changelog: Unchecked buffer fill in gf_history_changelog_next_change

2015-03-04T12:43:44+00:00

A gf_history_changelog_next_change() calls gf_readline() to fill a
buffer without checking buffer size. The size of maxlen is not verified
to be less than the lenght of buffer. This could result in the over
filling of buffer of maxlen is greater than PATH_MAX.

Check the size of maxlen to be less than PATH_MAX and return a fail code
as needed.

BUG: 1174017
Change-Id: Ic53b1a6e25af69a339bc15fb2d233dc1e457910f
Reported-by: Keith Schincke 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/9275
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar