while creating rpc_tranpsort object, we store a dictionary without
taking a ref on dict but it does an unref during the cleaning of the
transport object.

So the rpc layer expect the caller to take a ref on the dictionary
before passing dict to rpc layer. This leads to a lot of confusion
across the code base and leads to ref leaks.

Semantically, this is not correct. It is the rpc layer responsibility
to take a ref when storing it, and free during the cleanup.

I'm listing down the total issues or leaks across the code base because
of this confusion. These issues are currently present in the upstream
master.

1) changelog_rpc_client_init

2) quota_enforcer_init

3) rpcsvc_create_listeners : when there are two transport, like tcp,rdma.

4) quotad_aggregator_init

5) glusterd: init

6) nfs3_init_state

7) server: init

8) client:init

This patch does the cleanup according to the semantics.

Change-Id: I46373af9630373eb375ee6de0e6f2bbe2a677425
updates: bz#1659708
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>

Problem: Multiple shd processes are spawned while starting volumes
         in the loop on brick_mux environment.glusterd spawn a process
         based on a pidfile and shd daemon is taking some time to
         update pid in pidfile due to that glusterd is not able to
         get shd pid

Solution: Commit cd249f4cb783f8d79e79468c455732669e835a4f changed
          the code to update pidfile in parent for any gluster daemon
          after getting the status of forking child in parent.To resolve
          the same correct the condition update pidfile in parent only
          for glusterd and for rest of the daemon pidfile is updated in
          child

Change-Id: Ifd14797fa949562594a285ec82d58384ad717e81
fixes: bz#1684404
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>

Otherwise, gnfs will crash in following situation.
Also see commit 2f9e555f.

Reproducible Steps:
1. kill gnfs process
2. service glusterd restart;gluster volume profile [vol] info nfs

dump trace info:
/lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xc2)[0x7fcf5cb6a872]
/lib64/libglusterfs.so.0(gf_print_trace+0x324)[0x7fcf5cb743a4]
/lib64/libc.so.6(+0x35670)[0x7fcf5b1d5670]
/usr/sbin/glusterfs(glusterfs_handle_nfs_profile+0x114)[0x7fcf5d066474]
/lib64/libglusterfs.so.0(synctask_wrap+0x12)[0x7fcf5cba1502]
/lib64/libc.so.6(+0x47110)[0x7fcf5b1e7110]

Fixes: bz#1677559

Change-Id: Id68edb3e4646c39544e0b4c90b5e0a9083b37b0d
Signed-off-by: hujianfei <hujianfei@cmss.chinamobile.com>

Adding a comment in the source code, so that anyone reading
the code will understand the changes done by d4fa29 better.

fixes: bz#1654270

Change-Id: I75aff4243420c434c47d69a4b310f77bf161bb29
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>

This patch implements a thread pool that is wait-free for adding jobs to
the queue and uses a very small locked region to get jobs. This makes it
possible to decrease contention drastically. It's based on wfcqueue
structure provided by urcu library.

It automatically enables more threads when load demands it, and stops
them when not needed. There's a maximum number of threads that can be
used. This value can be configured.

Depending on the workload, the maximum number of threads plays an
important role. So it needs to be configured for optimal performance.
Currently the thread pool doesn't self adjust the maximum for the
workload, so this configuration needs to be changed manually.

For this reason, the global thread pool has been made optional, so that
volumes can still use the thread pool provided by io-threads.

To enable it for bricks, the following option needs to be set:

   config.global-threading = on

This option has no effect if bricks are already running. A restart is
required to activate it. It's recommended to also enable the following
option when running bricks with the global thread pool:

   performance.iot-pass-through = on

To enable it for a FUSE mount point, the option '--global-threading'
must be added to the mount command. To change it, an umount and remount
is needed. It's recommended to disable the following option when using
global threading on a mount point:

   performance.client-io-threads = off

To enable it for services managed by glusterd, glusterd needs to be
started with option '--global-threading'. In this case all daemons, like
self-heal, will be using the global thread pool.

Currently it can only be enabled for bricks, FUSE mounts and glusterd
services.

The maximum number of threads for clients and bricks can be configured
using the following options:

   config.client-threads
   config.brick-threads

These options can be applied online and its effect is immediate most of
the times. If one of them is set to 0, the maximum number of threads
will be calcutated as #cores * 2.

Some distributions use a very old userspace-rcu library (version 0.7)
for this reason, some header files from version 0.10 have been copied
into contrib/userspace-rcu and are used if the detected version is 0.7
or older.

An additional change has been made to io-threads to prevent that threads
are started when iot-pass-through is set.

Change-Id: I09d19e246b9e6d53c6247b29dfca6af6ee00a24b
updates: #532
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>

in both `--help` text and man page

updates: bz#1193929
Change-Id: I9aa9367c6863ac8e2403255280697c9e6be26cf0
Signed-off-by: Amar Tumballi <amarts@redhat.com>

Auto invalidation is necessary when same (meta)data is shared/access
across multiple mounts. However, if (meta)data is not shared, all
relevant I/O goes through the cache of single mount and hence is
coherent with (meta)data on bricks always. So, fuse-auto-invalidation
can be disabled for this case which gives a huge performance boost for
workloads that write data and then immediately read the data they just
wrote.

From glusterfs --help,

<snip>
      --auto-invalidation[=BOOL]   controls whether fuse-kernel can
                             auto-invalidate attribute, dentry and page-cache.
                             Disable this only if same files/directories are
                             not accessed across two different mounts
                             concurrently [default: "on"]
</snip>

Details on how disabling auto-invalidation helped to reduce pgbench
init times can be found at [1]. Time taken for pgbench init of scale
8000 was 8340s. That will be an improvement of 86% (59280s vs 8340s)
with auto-invalidations turned off along with other
optimizations. Just disabling auto-invalidation contributed 56%
improvement by reducing the total time taken by 33260s.

[1] https://www.spinics.net/lists/gluster-devel/msg25907.html

Change-Id: I0ed730dba9064bd9c576ad1800170a21e100e1ce
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
updates: bz#1664934

Mostly, unlock before logging.
In some cases, moved different code that was not needed
to be under lock (for example, taking time, or malloc'ing)
to be executed before taking the lock.

Note: logging might be slightly less accurate in order, since it may
not be done now under the lock, so order of logs is racy. I think
it's a reasonable compromise.

Compile-tested only!
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

Change-Id: I2438710016afc9f4f62a176ef1a0d3ed793b4f89

The value rsp.xdata.xdata_val was being freed twice. It was assigned
to dict->extra_stdfree, dict_destroy would free it and also there was
an explicit free. Getting rid of explicit free in this patch.

Change-Id: Ia9c73454bec3970b33f154fa754398bf3b045645
fixes: bz#1668268
Signed-off-by: Poornima G <pgurusid@redhat.com>

This patch helps enable IPv6 connections in the cluster.
The default address-family is IPv4 without using this option explicitly.

When address-family is set to "inet6" in the /etc/glusterfs/glusterd.vol
file, the mount command-line also needs to have
-o xlator-option="transport.address-family=inet6" added to it.

This option also gets added to the brick command-line.
Snapshot and gfapi use-cases should also use this option to pass in the
inet6 address-family.

Change-Id: I97db91021af27bacb6d7578e33ea4817f66d7270
fixes: bz#1635863
Signed-off-by: Milind Changire <mchangir@redhat.com>