glusterfs.git/xlators/performance, branch v6.2

performance/readdir-ahead: fix deadlock

2019-03-08T14:08:29+00:00

This deadlock happens while processing dentry corresponding to current
directory (.) in rda_fill_readdirp. In this case following order is
followed:

LOCK(directory_fd_ctx->lock);
  rda_inode_ctx_get_iatt -> LOCK(directory_inode->lock);

However, in rda_mark_inode_dirty following lock order is followed:
LOCK(directory_inode->lock);
  LOCK(directory_fd_ctx->lock);

these two codepaths when executed concurrently resulted in a deadlock.

Current patch fixes this by removing locking directory inode and
fd-ctx in rda_fill_readdirp. This is fine as directory inode's stat
won't change due to writes to files within directory.

Change-Id: Ic93a67a0dac8229bb0d79582e526a512e6f2569c
Signed-off-by: Raghavendra Gowdappa 
Fixes: bz#1686399

io-threads: Prioritize fops with NO_ROOT_SQUASH pid

2019-03-06T03:16:53+00:00

There was 30% regression observed in mkdir path with commit
b139bc58eb504adf5ef81658896c9283ae21f390. On analysis it is found
that io-threads xlator deprioritzes fops with all -ve pid.

Some context in to the no-root-squash pid requirement:
DHT xlator does some of the internal fops with root privileges. This is
needed so that operations like layout healing should not be abandoned
because a non root user is operating.  If root-squash option is enabled
the layout set operation looses its root privilege as server xlator
converts the uid and pid to random numbers. Hence, the above mentioned
commit converted pid to GF_CLIENT_PID_NO_ROOT_SQUASH to continue fops
as root.

Combining the above I am proposing not to deprioritize fops with
no-root-squash pid.

> Change-Id: I54d056c01b25729304a77f9242fbaff39c5672ba
> fixes: bz#1676430
> Signed-off-by: Susant Palai 
(cherry picked from commit f5c3b1727f55ffaa3dcdb3c3a09b968ebb45dbb2)

Change-Id: I54d056c01b25729304a77f9242fbaff39c5672ba
fixes: bz#1676429
Signed-off-by: Susant Palai

md-cache: Adapt integer data types to avoid integer overflow

2019-02-25T15:25:57+00:00

The "struct iatt" in iatt.h is using int64_t types for storing
the atime, mtime and ctime. Therefore the struct 'struct md_cache' in
md-cache.c should also use this types to avoid an integer overflow.

This can happen e.g. if someone uses a very high default-retention-period
in the WORM-Xlator.

Change-Id: I605268d300ab622b9c8ab30e459dc00d9340aad1
fixes: bz#1680020
Signed-off-by: David Spisla 
(cherry picked from commit 15423e14f16dd1a15ee5e5cbbdbdd370e57ed59f)

performance/write-behind: fix use-after-free in readdirp

2019-02-22T03:33:07+00:00

Two issues were found:
1. in wb_readdirp_cbk, inode should unrefed after wb_inode is
unlocked. Otherwise, inode and hence the context wb_inode can be freed
by the type we try to unlock wb_inode
2. wb_readdirp_mark_end iterates over a list of wb_inodes of children
of a directory. But inodes could've been freed and hence the list
might be corrupted. To fix take a reference on inode before adding it
to invalidate_list of parent.

Change-Id: I911b0e0b2060f7f41ded0b05db11af6f9b7c09c5
Signed-off-by: Raghavendra Gowdappa 
Updates: bz#1678570
(cherry picked from commit 64cc4458918e8c8bfdeb114da0a6501b2b98491a)

performance/write-behind: handle call-stub leaks

2019-02-19T06:23:34+00:00

Change-Id: I7be9a5f48dcad1b136c479c58b1dca1e0488166d
Signed-off-by: Raghavendra Gowdappa 
Fixes: bz#1678570
(cherry picked from commit 6175cb10cd5f59f3c7ae4100bc78f359b68ca3e9)

performance/md-cache: introduce an option to control invalidation of inodes

2019-02-18T14:44:27+00:00

Explicit invalidation by calling inode_invalidate is necessary when
same (meta)data is shared/access across multiple mounts. Without an
explicit inode_invalidate call, caches in the mount which didn't
witness writes wouldn't be aware of changes as writes wouldn't have
passed through them. However, if (meta)data is not shared, all
relevant I/O goes through the cache of single mount and hence is
coherent with (meta)data on bricks always. So, explicit inode
invalidation can be disabled for this case which gives a huge
performance boost for workloads that write data and then immediately
read the data they just wrote. Note that otherwise, local writes
(which pass through the cache) will change ctime and cause unnecessary
invalidations.

The name of the option that controls this behavior is
"performance.global-cache-invalidation". This option is global and it
purges caches both in glusterfs and kernel stack for native FUSE
mounts. For non-native FUSE mounts, it purges cache only from
glusterfs stack. This option is effective only when
performance.stat-prefetch is on.

Note that there is a similar option "performance.cache-invalidation",
but the scope of that option is limited to quick-read and md-cache.

Change-Id: I462bb4b65ff9aae1f6ba76f50b1f2f94fb10323b
Signed-off-by: Raghavendra Gowdappa 
updates: bz#1674364
(cherry picked from commit 2b5aa4489de2017a03bcb6ec8986286f0c76a670)

core: make gf_thread_create() easier to use

2019-02-01T09:37:20+00:00

This patch creates a specific function to set the thread name using a
string format and a variable argument list, like printf().

This function is used to set the thread name from gf_thread_create(),
which now accepts a variable argument list to create the full name. It's
not necessary anymore to use a local array to build the name of the
thread. This is done automatically.

Change-Id: Idd8d01fd462c227359b96e98699f8c6d962dc17c
Updates: bz#1193929
Signed-off-by: Xavi Hernandez

readdir-ahead: do not zero-out iatt in fop cbk

2019-01-31T06:18:14+00:00

...when ctime is zero. ia_type and ia_gfid always need to be non-zero
for things to work correctly.

Problem:
Commit c9bde3021202f1d5c5a2d19ac05a510fc1f788ac zeroed out the iatt
buffer in the cbks of modification fops before unwinding if the ctime in
the buffer was zero. This was causing the fops to fail: noticeable when
AFR's 'consistent-metadata' option was enabled. (AFR zeros out the ctime
when the option is set. See commit
4c4624c9bad2edf27128cb122c64f15d7d63bbc8).

Fixes:
-Do not zero out the ia_type and ia_gfid of the iatt buff under any
circumstance.
-Also, fixed _rda_inode_ctx_update_iatts() to always update these values from
the incoming buf when ctime is zero. Otherwise we end up with zero
ia_type and ia_gfid the first time the function is called *and* the
incoming buf has ctime set to zero.

fixes: bz#1670253
Reported-By:Michael Hanselmann 
Change-Id: Ib72228892d42c3513c19fc6dfb543f2aa3489eca
Signed-off-by: Ravishankar N

performance/readdir-ahead: Fix deadlock in readdir ahead.

2019-01-23T13:59:54+00:00

This patch fixes a lock contention in reaadir-ahead xlator.

There are two issues, one is the processing of "." ".."
entry while holding an fd_ctx lock. The other one is destroying
the stack inside a fd_ctx lock.

Change-Id: Id0bf83a3d9fea6b40015b8d167525c59c6cfa25e
updates: bz#1659708
Signed-off-by: Mohammed Rafi KC

fix 32-bit-build-smoke warnings

2019-01-11T18:25:18+00:00

fixes: bz#1622665

Change-Id: I777d67b1b62c284c62a02277238ad7538eef001e
Signed-off-by: Iraj Jamali