glusterfs.git/libglusterfs/src/graph.c, branch v7.1

glusterd/svc: update pid of mux volumes from the shd process

2019-07-24T10:29:17+00:00

For a normal volume, we are updating the pid from a the
process while we do a daemonization or at the end of the
init if it is no-daemon mode. Along with updating the pid
we also lock the file, to make sure that the process is
running fine.

With brick mux, we were updating the pidfile from gluterd
after an attach/detach request.

There are two problems with this approach.
1) We are not holding a pidlock for any file other than parent
   process.
2) There is a chance for possible race conditions with attach/detach.
   For example, shd start and a volume stop could race. Let's say
   we are starting an shd and it is attached to a volume.
   While we trying to link the pid file to the running process,
   this would have deleted by the thread that doing a volume stop.

Backport of : https://review.gluster.org/#/c/glusterfs/+/22935/
>Change-Id: I29a00352102877ce09ea3f376ca52affceb5cf1a
>Updates: bz#1722541
>Signed-off-by: Mohammed Rafi KC 

Change-Id: I29a00352102877ce09ea3f376ca52affceb5cf1a
Updates: bz#1732668
Signed-off-by: Mohammed Rafi KC

graph/shd: Use top down approach while cleaning xlator

2019-07-24T10:26:34+00:00

We were cleaning xlator from botton to top, which might
lead to problems when upper xlators trying to access
the xlator object loaded below.

One such scenario is when fd_unref happens as part of the
fini call which might lead to calling the releasedir to
lower xlator. This will lead to invalid mem access

Backport of:https://review.gluster.org/#/c/glusterfs/+/22968/
>Change-Id: I8a6cb619256fab0b0c01a2d564fc88287c4415a0
>Updates: bz#1716695
>Signed-off-by: Mohammed Rafi KC 

Change-Id: I8a6cb619256fab0b0c01a2d564fc88287c4415a0
Updates: bz#1730229
Signed-off-by: Mohammed Rafi KC

graph/shd: Use glusterfs_graph_deactivate to free the xl rec

2019-07-24T10:23:18+00:00

We were using glusterfs_graph_fini to free the xl rec from
glusterfs_process_volfp as well as glusterfs_graph_cleanup.

Instead we can use glusterfs_graph_deactivate, which is does
fini as well as other common rec free.

Backportof : https://review.gluster.org/22904
>Change-Id: Ie4a5f2771e5254aa5ed9f00c3672a6d2cc8e4bc1
>Updates: bz#1716695
>Signed-off-by: Mohammed Rafi KC 

Change-Id: Ie4a5f2771e5254aa5ed9f00c3672a6d2cc8e4bc1
Updates: bz#1730229
Signed-off-by: Mohammed Rafi KC

multiple files: another attempt to remove includes

2019-06-14T16:50:32+00:00

There are many include statements that are not needed.
A previous more ambitious attempt failed because of *BSD plafrom
(see https://review.gluster.org/#/c/glusterfs/+/21929/ )

Now trying a more conservative reduction.
It does not solve all circular deps that we have, but it
does reduce some of them. There is just too much to handle
reasonably (dht-common.h includes dht-lock.h which includes
dht-common.h ...), but it does reduce the overall number of lines
of include we need to look at in the future to understand and fix
the mess later one.

Change-Id: I550cd001bdefb8be0fe67632f783c0ef6bee3f9f
updates: bz#1193929
Signed-off-by: Yaniv Kaul

glusterd/shd: Optimize the glustershd manager to send reconfigure

2019-05-31T12:17:48+00:00

Traditionally all svc manager will execute process stop and then
followed by start each time when they called. But that is not
required by shd, because the attach request implemented in the shd
multiplex has the intelligence to check whether a detach is required
prior to attaching the graph. So there is no need to send an explicit
detach request if we are sure that the next call is an attach request

Change-Id: I9157c8dcaffdac038f73286bcf5646a3f1d3d8ec
fixes: bz#1710054
Signed-off-by: Mohammed Rafi KC

glusterfsd/cleanup: Protect graph object under a lock

2019-05-31T11:27:37+00:00

While processing a cleanup_and_exit function, we are
accessing a graph object. But this has not been protected
under a lock. Because a parallel cleanup of a graph is quite
possible which might lead to an invalid memory access

Change-Id: Id05ca70d5b57e172b0401d07b6a1f5386c044e79
fixes: bz#1708926
Signed-off-by: Mohammed Rafi KC

shd/mux: Fix coverity issues introduced by shd mux patch

2019-04-15T06:06:02+00:00

CID 1400475:  Null pointer dereferences  (FORWARD_NULL)
CID 1400474:  Null pointer dereferences  (FORWARD_NULL)
CID 1400471:  Code maintainability issues  (UNUSED_VALUE)
CID 1400470:  Null pointer dereferences  (FORWARD_NULL)
CID 1400469:  Memory - illegal accesses  (USE_AFTER_FREE)
CID 1400467:  Code maintainability issues  (UNUSED_VALUE)

Change-Id: I0ca1c733be335c6e5844f44850f8066626ac40d4
updates: bz#789278
Signed-off-by: Mohammed Rafi KC

graph.c: remove extra gettimeofday() - reuse the graph dob.

2019-04-15T02:25:26+00:00

It was written just before fill_void() call.

Note that there was a possible overflow if the hostname was too long
(unrelated to this patch), but it is now also fixed, as we use a smaller buffer
for the hostname. This, in turn, forces us to check if gethostname() failed
and add explicitly the terminating null to it.

Change-Id: I45fbc0a8e105f1247f3cbf61befac06fabbaea06
updates: bz#1193929
Signed-off-by: Yaniv Kaul

mgmt/shd: Implement multiplexing in self heal daemon

2019-04-01T03:44:23+00:00

Problem:

Shd daemon is per node, which means they create a graph
with all volumes on it. While this is a great for utilizing
resources, it is so good in terms of performance and managebility.

Because self-heal daemons doesn't have capability to automatically
reconfigure their graphs. So each time when any configurations
changes happens to the volumes(replicate/disperse), we need to restart
shd to bring the changes into the graph.

Because of this all on going heal for all other volumes has to be
stopped in the middle, and need to restart all over again.

Solution:

This changes makes shd as a per volume daemon, so that the graph
will be generated for each volumes.

When we want to start/reconfigure shd for a volume, we first search
for an existing shd running on the node, if there is none, we will
start a new process. If already a daemon is running for shd, then
we will simply detach a graph for a volume and reatach the updated
graph for the volume. This won't touch any of the on going operations
for any other volumes on the shd daemon.

Example of an shd graph when it is per volume

                           graph
                     -----------------------
                     |     debug-iostat    |
                     -----------------------
                    /         |             \
                   /          |              \
              ---------    ---------      ----------
              | AFR-1 |    | AFR-2 |      |  AFR-3 |
              --------     ---------      ----------

A running shd daemon with 3 volumes will be like-->

                           graph
                     -----------------------
                     |     debug-iostat    |
                     -----------------------
                    /           |           \
                   /            |            \
              ------------   ------------  ------------
              | volume-1 |   | volume-2 |  | volume-3 |
              ------------   ------------  ------------

Change-Id: Idcb2698be3eeb95beaac47125565c93370afbd99
fixes: bz#1659708
Signed-off-by: Mohammed Rafi KC

core: move logs which are only developer relevant to DEBUG level

2019-01-23T16:05:52+00:00

We had only changed the log level to DEBUG in release branch earlier.
But considering 90%+ of our deployments happen in same env, we can look
at these specific logs on need basis. With this change, the master
branch will be easier to debug with lesser logs.

Change-Id: I4157a7ec7d5ec9c2948b2bbc1e4cb8317f28d6b8
Updates: bz#1666833
Signed-off-by: Amar Tumballi