glusterfs.git/xlators/performance/readdir-ahead/src/readdir-ahead.c, branch v5.0

Land part 2 of clang-format changes

2018-09-12T12:22:45+00:00

Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu

performance/readdir-ahead: keep stats of cached dentries in sync with modifications

2018-08-18T07:28:53+00:00

PROBLEM:

Stats of dentries that are readdirp'd ahead can become stale due to
fops like writes, truncate etc that modify the file pointed by
dentries. When a readdir is finally wound at offset corresponding to
these entries, the iatts that are returned to the application come
from readdir-ahead's cache, which are stale by now. This problem gets
further aggravated when caching translators/modules cache and continue
to serve this stale information.

FIX:

* Store the iatt in context of the inode pointed by dentry.
* Whenever the inode pointed by dentry undergoes modification, in cbk
  of modification fop, update the iatt stored in inode-ctx to reflect
  the modification.
* When serving a readdirp response from application, update iatts of
  dentries with the iatts stored in the context of inodes pointed by
  these dentries.
* Some fops don't have valid iatts in their responses. For eg., write
  response whose data is still cached in write-behind will have zeroed
  out stat. In this case keep only ia_type and ia_gfid and reset rest
  of the iatt members to zero.
  - fuse-bridge in this case just sends "entry" information back to
    kernel and attr is not sent.
  - gfapi sets entry->inode to NULL and zeroes out the entire stat
* There is one tiny race between the entry creation and a readdirp on
  its parent dir, which could cause the inode-ctx setting and inode
  ctx reading to happen on two different inode objects. To prevent
  this, when entry->inode doesn't eqaul to linked_inode,
  - fuse-bridge is made to send only "entry" information without
    attributes
  - gfapi sets entry->inode to NULL and zeroes out the entire stat.

Change-Id: Ia27ff49a61922e88c73a1547ad8aacc9968a69df
BUG: 1390050
Updates: bz#1390050
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Raghavendra G

Revert "performance/readdir-ahead: Invalidate cached dentries if they're modified while in cache"

2018-08-03T22:18:11+00:00

This reverts commit 7131de81f72dda0ef685ed60d0887c6e14289b8c.

With the latest master, I created a single brick volume and some files
inside it.

[root@rhgs313-6 ~]# umount -f /mnt/fuse1; mount -t glusterfs -s
192.168.122.6:/thunder /mnt/fuse1; ls -l /mnt/fuse1/; echo "Trying
again"; ls -l /mnt/fuse1
umount: /mnt/fuse1: not mounted
total 0
----------. 0 root root 0 Jan  1  1970 file-1
----------. 0 root root 0 Jan  1  1970 file-2
----------. 0 root root 0 Jan  1  1970 file-3
----------. 0 root root 0 Jan  1  1970 file-4
----------. 0 root root 0 Jan  1  1970 file-5
d---------. 0 root root 0 Jan  1  1970 subdir
Trying again
total 3
-rw-r--r--. 1 root root 33 Aug  3 14:06 file-1
-rw-r--r--. 1 root root 33 Aug  3 14:06 file-2
-rw-r--r--. 1 root root 33 Aug  3 14:06 file-3
-rw-r--r--. 1 root root 33 Aug  3 14:06 file-4
-rw-r--r--. 1 root root 33 Aug  3 14:06 file-5
d---------. 0 root root  0 Jan  1  1970 subdir
[root@rhgs313-6 ~]#

Conversation can be followed on gluster-devel on thread with subj:
tests/bugs/distribute/bug-1122443.t - spurious failure. git-bisected
pointed this patch as culprit.

Change-Id: I1eb46f6c196f44fde8ce991840a0e724e6f50862
Signed-off-by: Raghavendra G 
Updates: bz#1390050

performance/readdir-ahead: Invalidate cached dentries if they're modified while in cache

2018-07-28T09:27:38+00:00

PROBLEM:

Entries that are readdirp'd ahead can undergo modification in terms
of writes, truncates which could modify their iatts. When a readdir
is finally wound at offset corresponding to these entries, the iatts
that are returned to the application come from readdir-ahead's cache,
which are stale by now. This problem gets further aggravated when caching
translators/modules cache and continue to serve this stale information.

FIX:

Whenever a dentry undergoes modification, in the cbk of the modification fop,
a "dirty" flag (default 0) is set in its inode ctx. When it's time for
readdir-ahead to serve these entries, it will read the inode ctx and check
if the entry is "dirty", and if it is, set the entry's attrs to all zeroes,
as an indicator to fuse, md-cache etc not to cache these attributes.

Also there is one tiny race between the entry creation and a readdirp on its
parent dir, which could cause the inode-ctx setting and inode ctx reading to
happen on two different inode objects. To prevent this, fuse-bridge is made to
drop entries for which dentry->inode is not the same as linked inode,
in readdirp cbk.

Change-Id: If7396507632b5268442ca580473d5155fee9cbef
BUG: 1390050
Updates: bz#1390050
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Raghavendra G

readdir-ahead: Fix an issue with parallel-readdir and readdir-optimize

2018-05-17T06:47:52+00:00

Issue: When parallel-readdir is enabled, readdir-optimize automatically
stops working because of a bug in rda_opendir.

RCA: In rda_opendir, the xattrs that indicate readdir-optimize or not
is sent in xdata. This xdata is sent to all the readdirp prefetch
calls. A dict_ref is taken on xdata and kept in rda_opendir to be
used by rda_fill_fd, but dht_opendir deletes some elements in xdata
after calling rda_opendir. Hence dict_ref is not a right choice here,
dict_copy needs to used.

Change-Id: Ie7cc7ceb03117dd4179ef7905647f2f123f94966
fixes: bz#1578650
Signed-off-by: Poornima G

xlators/performance: Add pass-through option

2018-04-11T10:53:48+00:00

Add pass-through option in performance traslators. Set the option in
GF_OPTION_INIT() and GF_OPTION_RECONF()

Updates: #304

Change-Id: If1537450147d154905831e36f7162a32866d7ad6
Signed-off-by: Varsha Rao

readdir-ahead: Cleanup the xattr request code

2018-01-19T04:02:26+00:00

Updates #297

Change-Id: Ia0c697583751290a455da3cd1894e0c5685d1bd8
Signed-off-by: Poornima G

readdir-ahead: Add parallel-readdir option in readdir-ahead

2018-01-18T03:52:56+00:00

parallel-readdir option is defined as belonging to readdir-ahead
in glusterd-volume-set.c, but was not defined in options of
readdir-ahead, fixing the same.

Change-Id: I97cc88b38ab99ade5f066519ca1cb9bfed03a7da
BUG: 1506197
Signed-off-by: Poornima G

performance/readdir-ahead: fix cache usage update issue

2018-01-15T05:08:28+00:00

Use atomic operation to modify cache-size to protect it from concurrent
modification.

Change-Id: Ie73cdd4abbaf0232b1db4ac856c01d24603890ad
BUG: 1533804
Signed-off-by: Zhang Huan

readdir-ahead: Fix duplicate listing and cache size calculation

2017-06-12T10:18:24+00:00

Issue:
If a opendir is followed by a closedir without readdir, though
the prefetched entries were freed, the freed size was not accounted
in priv->rda_cache_size. Thus the cache limit will exceed if there
are multiple opendir followed by closedir.

Fix:
Fix the pric->rda_cache_size calculation. Also have removed the
inode_ctx_size. Each perf xlator has its own cache limit that
it works with. Also the inode_ctx size can change, if a forget/
invalidate or any other factor triggers the inode_ctx size.

Change-Id: I9707ec558076ce046e58a55989ec9513c70ea029
BUG: 1431908
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17504
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G