glusterfs.git/libglusterfs/src/glusterfs, branch v6.2

core: handle memory accounting correctly

2019-04-23T11:53:02+00:00

When a translator stops, memory accounting for that translator is not
destroyed (because there could remain memory allocated that references
it), but mutexes that coordinate updates of memory accounting were
destroyed. This caused incorrect memory accounting and even crashes in
debug mode.

This patch also fixes some other things:

* Reduce the number of atomic operations needed to manage memory
  accounting.
* Correctly account memory when realloc() is used.
* Merge two critical sections into one.
* Cleaned the code a bit.

Backport of:
> Change-Id: Id5eaee7338729b9bc52c931815ca3ff1e5a7dcc8
> BUG: bz#1659334
> Signed-off-by: Xavi Hernandez 

Change-Id: Id5eaee7338729b9bc52c931815ca3ff1e5a7dcc8
Fixes: bz#1702271
Signed-off-by: Xavi Hernandez

core: Log level changes do not effect on running client process

2019-04-16T10:59:36+00:00

Problem: commit c34e4161f3cb6539ec83a9020f3d27eb4759a975 set log-level
         per xlator during reconfigure only for a brick process not for
         the client process.

Solution: 1) Change per xlator log-level only if brick_mux is enabled.To make sure
             about brick multiplex introudce a flag brick_mux at ctx->cmd_args.

Note: There are two other changes done with this patch
      1) Ignore client-log-level option to attach a brick with
         already running brick if brick_mux is enabled
      2) Add a log to print pid of the running process to make easier
         debugging

> Change-Id: I39e85de778e150d0685cd9a79425ce8b4783f9c9
> Signed-off-by: Mohit Agrawal 
> Fixes: bz#1696046
> (Cherry picked from commit 798aadbe51a9a02dd98a0f861cc239ecf7c8ed57)
> (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/22495/)

Change-Id: If91682830f894ab8f6857f19dcb1797fc15ca64c
Fixes: bz#1699715
Signed-off-by: Mohit Agrawal

logging: Fix GF_LOG_OCCASSIONALLY API

2019-04-16T10:49:22+00:00

GF_LOG_OCCASSIONALLY doesn't log on the first instance rather at every
42nd iterations which isn't effective as in some cases we might not have
the code flow hitting the same log for as many as 42 times and we'd end
up suppressing the log.

Updates: bz#1679904
Change-Id: Iee293281d25a652b64df111d59b13de4efce06fa
Signed-off-by: Atin Mukherjee 
(cherry picked from commit d0d3e10d44366c68fc153e48b229e72a4aa26e61)

core: make compute_cksum function op_version compatible

2019-03-08T14:45:41+00:00

Problem: commit 5a152a changed the mechansim of computing the
checksum. In heterogeneous cluster, peers are running into
rejected state because we have different cksum computation
mechansims in upgraded and non-upgraded nodes.

Solution: add a check for op-version so that all the nodes
in the cluster follow the same mechanism for computing the
cksum.

fixes: bz#1684029

> Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193
> BUG: bz#1685120
> Signed-off-by: Sanju Rakonde 
> (cherry picked from commit 073444b693b7a91c42963512e0fdafb57ad46670)

Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193

socket: socket event handlers now return void

2019-03-02T11:54:24+00:00

Problem:
Returning any value from socket event handlers to the event sub-system
doesn't make sense since event sub-system cannot handle socket
sub-system errors.

Solution:
Change return type of all socket event handlers to 'void'

mainline:
> Change-Id: I70dc2c57f12b7ea2fae41120f71aa0d7fe0b2b6f
> Fixes: bz#1651246
> Signed-off-by: Milind Changire 
> Reviewed-on: https://review.gluster.org/c/glusterfs/+/22221

Change-Id: I70dc2c57f12b7ea2fae41120f71aa0d7fe0b2b6f
Fixes: bz#1683900
Signed-off-by: Milind Changire 
(cherry picked from commit 776ba851c6ee6c265253d44cf1d6e4e3d4a21772)

mount/fuse: expose auto-invalidation as a mount option

2019-02-02T03:07:35+00:00

Auto invalidation is necessary when same (meta)data is shared/access
across multiple mounts. However, if (meta)data is not shared, all
relevant I/O goes through the cache of single mount and hence is
coherent with (meta)data on bricks always. So, fuse-auto-invalidation
can be disabled for this case which gives a huge performance boost for
workloads that write data and then immediately read the data they just
wrote.

From glusterfs --help,


      --auto-invalidation[=BOOL]   controls whether fuse-kernel can
                             auto-invalidate attribute, dentry and page-cache.
                             Disable this only if same files/directories are
                             not accessed across two different mounts
                             concurrently [default: "on"]


Details on how disabling auto-invalidation helped to reduce pgbench
init times can be found at [1]. Time taken for pgbench init of scale
8000 was 8340s. That will be an improvement of 86% (59280s vs 8340s)
with auto-invalidations turned off along with other
optimizations. Just disabling auto-invalidation contributed 56%
improvement by reducing the total time taken by 33260s.

[1] https://www.spinics.net/lists/gluster-devel/msg25907.html

Change-Id: I0ed730dba9064bd9c576ad1800170a21e100e1ce
Signed-off-by: Raghavendra Gowdappa 
updates: bz#1664934

core: make gf_thread_create() easier to use

2019-02-01T09:37:20+00:00

This patch creates a specific function to set the thread name using a
string format and a variable argument list, like printf().

This function is used to set the thread name from gf_thread_create(),
which now accepts a variable argument list to create the full name. It's
not necessary anymore to use a local array to build the name of the
thread. This is done automatically.

Change-Id: Idd8d01fd462c227359b96e98699f8c6d962dc17c
Updates: bz#1193929
Signed-off-by: Xavi Hernandez

syncop: remove unnecessary call to gf_backtrace_save()

2019-01-31T15:41:30+00:00

A call to gf_backtrace_save() was done on each context switch of a
synctask. The backtrace is generated writing to the filesystem, so it
can have an important impact on latency.

The generated backtrace was not used anywhere, so it's been removed.

Change-Id: I399a93b932c5b6e981c696c72c3e1ef44710ba52
Updates: bz#1193929
Signed-off-by: Xavi Hernandez

core: heketi-cli is throwing error "target is busy"

2019-01-31T06:23:39+00:00

Problem: When rpc-transport-disconnect happens, server_connection_cleanup_flush_cbk() 
         is supposed to call rpc_transport_unref() after open-files on 
         that transport are flushed per transport.But open-fd-count is 
         maintained in bound_xl->fd_count, which can be incremented/decremented 
         cumulatively in server_connection_cleanup() by all transport 
         disconnect paths. So instead of rpc_transport_unref() happening 
         per transport, it ends up doing it only once after all the files 
         on all the transports for the brick are flushed leading to 
         rpc-leaks.

Solution: To avoid races maintain fd_cnt at client instead of maintaining
          on brick

Credits: Pranith Kumar Karampuri
Change-Id: I6e8ea37a61f82d9aefb227c5b3ab57a7a36850e6
fixes: bz#1668190
Signed-off-by: Mohit Agrawal

afr/self-heal:Fix wrong type checking

2019-01-24T12:16:50+00:00

gf_dirent struct has d_type variable which should check
with DT_DIR istead of IA_IFDIR or IA_IFDIR has to compare
with entry->d_stat.ia_type

Change-Id: Idf1059ce2a590734bc5b6adaad73604d9a708804
updates: bz#1653359
Signed-off-by: Mohammed Rafi KC