glusterfs.git/xlators/protocol, branch v7.1

event: rename event_XXX with gf_ prefixed

2019-08-21T06:13:38+00:00

I hit one crash issue when using the libgfapi.

In the libgfapi it will call glfs_poller() --> event_dispatch()
in file api/src/glfs.c:721, and the event_dispatch() is defined
by libgluster locally, the problem is the name of event_dispatch()
is the extremly the same with the one from libevent package form
the OS.

For example, if a executable program Foo, which will also use and
link the libevent and the libgfapi at the same time, I can hit the
crash, like:

kernel: glfs_glfspoll[68486]: segfault at 1c0 ip 00007fef006fd2b8 sp
00007feeeaffce30 error 4 in libevent-2.0.so.5.1.9[7fef006ed000+46000]

The link for Foo is:
lib_foo_LADD = -levent $(GFAPI_LIBS)
It will crash.

This is because the glfs_poller() is calling the event_dispatch() from
the libevent, not the libglsuter.

The gfapi link info :
GFAPI_LIBS = -lacl -lgfapi -lglusterfs -lgfrpc -lgfxdr -luuid

If I link Foo like:
lib_foo_LADD = $(GFAPI_LIBS) -levent
It will works well without any problem.

And if Foo call one private lib, such as handler_glfs.so, and the
handler_glfs.so will link the GFAPI_LIBS directly, while the Foo won't
and it will dlopen(handler_glfs.so), then the crash will be hit everytime.

The link info will be:
foo_LADD = -levent
libhandler_glfs_LIBADD = $(GFAPI_LIBS)

I can avoid the crash temporarily by linking the GFAPI_LIBS in Foo too like:
foo_LADD = $(GFAPI_LIBS) -levent
libhandler_glfs_LIBADD = $(GFAPI_LIBS)

But this is ugly since the Foo won't use any APIs from the GFAPI_LIBS.

And in some cases when the --as-needed link option is added(on many dists
it is added as default), then the crash is back again, the above workaround
won't work.

Backport of:
> https://review.gluster.org/#/c/glusterfs/+/23110/
> Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa
> Fixes: #699
> Signed-off-by: Xiubo Li 

Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa
updates: bz#1740519
Signed-off-by: Xiubo Li 
(cherry picked from commit 799edc73c3d4f694c365c6a7c27c9ab8eed5f260)

protocol/client: propagte GF_EVENT_CHILD_PING only for connections to brick

2019-08-09T04:41:41+00:00

Two reasons:
* ping responses from glusterd may not be relevant for Halo
  replication. Instead, it might be interested in only knowing whether
  the brick itself is responsive.
* When a brick is killed, propagating GF_EVENT_CHILD_PING of ping
  response from glusterd results in GF_EVENT_DISCONNECT spuriously
  propagated to parent xlators. These DISCONNECT events are from the
  connections client establishes with glusterd as part of its
  reconnect logic. Without GF_EVENT_CHILD_PING, the last event
  propagated to parent xlators would be the first DISCONNECT event
  from brick and hence subsequent DISCONNECTS to glusterd are not
  propagated as protocol/client prevents same event being propagated
  to parent xlators consecutively. propagating GF_EVENT_CHILD_PING for
  ping responses from glusterd would change the last_sent_event to
  GF_EVENT_CHILD_PING and hence protocol/client cannot prevent
  subsequent DISCONNECT events

Signed-off-by: Raghavendra G 
Fixes: bz#1739334
Change-Id: I50276680c52f05ca9e12149a3094923622d6eaef
(cherry picked from commit 5d66eafec581fb3209af74595784be8854ca40a4)

across: coverity fixes

2019-06-03T04:00:39+00:00

* locks/posix.c: key was not freed in one of the cases.
* locks/common.c: lock was being free'd out of context.
* nfs/exports: handle case of missing free.
* protocol/client: handle case of entry not freed.
* storage/posix: handle possible case of double free

CID: 1398628, 1400731, 1400732, 1400756, 1124796, 1325526

updates: bz#789278
Change-Id: Ieeaca890288bc4686355f6565f853dc8911344e8
Signed-off-by: Amar Tumballi 
Signed-off-by: Sheetal Pamecha

stack: Make sure to have unique call-stacks in all cases

2019-05-30T15:54:33+00:00

At the moment new stack doesn't populate frame->root->unique in all cases. This
makes it difficult to debug hung frames by examining successive state dumps.
Fuse and server xlators populate it whenever they can, but other xlators won't
be able to assign 'unique' when they need to create a new frame/stack because
they don't know what 'unique' fuse/server xlators already used. What we need is
for unique to be correct. If a stack with same unique is present in successive
statedumps, that means the same operation is still in progress. This makes
'finding hung frames' part of debugging hung frames easier.

fixes bz#1714098
Change-Id: I3e9a8f6b4111e260106c48a2ac3a41ef29361b9e
Signed-off-by: Pranith Kumar K

Fix some "Null pointer dereference" coverity issues

2019-05-26T13:59:13+00:00

This patch fixes the following CID's:

  * 1124829
  * 1274075
  * 1274083
  * 1274128
  * 1274135
  * 1274141
  * 1274143
  * 1274197
  * 1274205
  * 1274210
  * 1274211
  * 1288801
  * 1398629

Change-Id: Ia7c86cfab3245b20777ffa296e1a59748040f558
Updates: bz#789278
Signed-off-by: Xavi Hernandez

protocol: remove compound fop

2019-04-29T05:29:44+00:00

Compound fops are kept on wire as a backward compatibility with
older AFR modules. The AFR module used beyond 4.x releases are
not using compound fops. Hence removing the compound fop in the
protocol code.

Note that, compound-fops was already an 'option' in AFR, and
completely removed since 4.1.x releases.

So, point to note is, with this change, we have 2 ways to upgrade
when clients of 3.x series are present.

i) set 'use-compound-fops' option to 'false' on any volume which
   is of replica type. And then upgrade the servers.

ii) Do a two step upgrade. First from current version (which will
    already be EOL if it's using compound) to a 4.1..6.x version,
    and then an upgrade to 7.x.

Consider the overall code which we are removing for the option
seems quite high, I believe it is worth it.

updates: bz#1693692
Signed-off-by: Amar Tumballi 
Change-Id: I0a8876d0367a15e1410ec845f251d5d3097ee593

Replace memdup() with gf_memdup()

2019-04-12T13:08:28+00:00

memdup() and gf_memdup() have the same implementation. Removed one API
as the presence of both can be confusing.

Change-Id: I562130c668457e13e4288e592792872d2e49887e
updates: bz#1193929
Signed-off-by: Vijay Bellur

client/fini: return fini after rpc cleanup

2019-04-11T03:47:41+00:00

There is a race condition in rpc_transport later
and client fini.

Sequence of events to happen the race condition
1) When we want to destroy a graph, we send a parent down
   event first
2) Once parent down received on a client xlator, we will
   initiates a rpc disconnect
3) This will in turn generates a child down event.
4) When we process child down, we first do fini for
   Every xlator
5) On successful return of fini, we delete the graph

Here after the step 5, there is a chance that the fini
on client might not be finished. Because an rpc_tranpsort
ref can race with the above sequence.

So we have to wait till all rpc's are successfully freed
before returning the fini from client

Change-Id: I20145662d71fb837e448a4d3210d1fcb2855f2d4
fixes: bz#1659708
Signed-off-by: Mohammed Rafi KC

protocol: add an option to force using old-protocol

2019-04-10T04:42:00+00:00

As protocol implements every fop, and in general a large part of
the codebase. Considering our regression is run mostly in 1 machine,
there was no way of forcing the client to use old protocol (while new
one is available). With this patch, a new 'testing' option is provided
which forces client to use old protocol if found.

This should help increase the code coverage by at least 10k lines overall.

updates: bz#1693692
Change-Id: Ie45256f7dea250671b689c72b4b6f25037cef948
Signed-off-by: Amar Tumballi

mgmt/shd: Implement multiplexing in self heal daemon

2019-04-01T03:44:23+00:00

Problem:

Shd daemon is per node, which means they create a graph
with all volumes on it. While this is a great for utilizing
resources, it is so good in terms of performance and managebility.

Because self-heal daemons doesn't have capability to automatically
reconfigure their graphs. So each time when any configurations
changes happens to the volumes(replicate/disperse), we need to restart
shd to bring the changes into the graph.

Because of this all on going heal for all other volumes has to be
stopped in the middle, and need to restart all over again.

Solution:

This changes makes shd as a per volume daemon, so that the graph
will be generated for each volumes.

When we want to start/reconfigure shd for a volume, we first search
for an existing shd running on the node, if there is none, we will
start a new process. If already a daemon is running for shd, then
we will simply detach a graph for a volume and reatach the updated
graph for the volume. This won't touch any of the on going operations
for any other volumes on the shd daemon.

Example of an shd graph when it is per volume

                           graph
                     -----------------------
                     |     debug-iostat    |
                     -----------------------
                    /         |             \
                   /          |              \
              ---------    ---------      ----------
              | AFR-1 |    | AFR-2 |      |  AFR-3 |
              --------     ---------      ----------

A running shd daemon with 3 volumes will be like-->

                           graph
                     -----------------------
                     |     debug-iostat    |
                     -----------------------
                    /           |           \
                   /            |            \
              ------------   ------------  ------------
              | volume-1 |   | volume-2 |  | volume-3 |
              ------------   ------------  ------------

Change-Id: Idcb2698be3eeb95beaac47125565c93370afbd99
fixes: bz#1659708
Signed-off-by: Mohammed Rafi KC