glusterfs.git/xlators/performance/readdir-ahead, branch v4.1dev

readdir-ahead: Cleanup the xattr request code

2018-01-19T04:02:26+00:00

Updates #297

Change-Id: Ia0c697583751290a455da3cd1894e0c5685d1bd8
Signed-off-by: Poornima G

readdir-ahead: Add parallel-readdir option in readdir-ahead

2018-01-18T03:52:56+00:00

parallel-readdir option is defined as belonging to readdir-ahead
in glusterd-volume-set.c, but was not defined in options of
readdir-ahead, fixing the same.

Change-Id: I97cc88b38ab99ade5f066519ca1cb9bfed03a7da
BUG: 1506197
Signed-off-by: Poornima G

performance/readdir-ahead: fix cache usage update issue

2018-01-15T05:08:28+00:00

Use atomic operation to modify cache-size to protect it from concurrent
modification.

Change-Id: Ie73cdd4abbaf0232b1db4ac856c01d24603890ad
BUG: 1533804
Signed-off-by: Zhang Huan

all: Simplify component message id's definition

2017-12-14T02:33:51+00:00

This patch creates a new way of defining message id's that is easier
and less error prone because it doesn't require so many manual changes
each time a new component is defined or a new message created.

Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b
Signed-off-by: Xavier Hernandez

readdir-ahead: Fix duplicate listing and cache size calculation

2017-06-12T10:18:24+00:00

Issue:
If a opendir is followed by a closedir without readdir, though
the prefetched entries were freed, the freed size was not accounted
in priv->rda_cache_size. Thus the cache limit will exceed if there
are multiple opendir followed by closedir.

Fix:
Fix the pric->rda_cache_size calculation. Also have removed the
inode_ctx_size. Each perf xlator has its own cache limit that
it works with. Also the inode_ctx size can change, if a forget/
invalidate or any other factor triggers the inode_ctx size.

Change-Id: I9707ec558076ce046e58a55989ec9513c70ea029
BUG: 1431908
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17504
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

rda, glusterd: Change the max of rda-cache-limit to INFINITY

2017-05-21T05:19:15+00:00

Issue:
The max value of rda-cache-limit is 1GB before this patch.
When parallel-readdir is enabled, there will be many instances of
readdir-ahead, hence the rda-cache-limit depends on the number of
instances. Eg: On a volume with distribute count 4, rda-cache-limit
when parallel-readdir is enabled, will be 4GB instead of 1GB.
Consider a followinf sequence of operations:
- Enable parallel readdir
- Set rda-cache-limit to lets say 3GB
- Disable parallel-readdir, this results in one instance of readdir-ahead
  and the rda-cache-limit will be back to 1GB, but the current value is 3GB
  and hence the mount will stop working as 3GB > max 1GB.

Solution:
To fix this, we can limit the cache to 1GB even when parallel-readdir
is enabled. But there is no necessity to limit the cache to 1GB, it
can be increased if the system has enough resources. Hence getting rid
of the rda-cache-limit max value is more apt. If we just change the
rda-cache-limit max to INFINITY, we will render older(<3.11) clients
broken, when the rda-cache-limit is set to > 1GB (as the older clients
still expect a value < 1GB). To safely change the max value of
rda-cache-limit to INFINITY, add a check in glusted to verify all
the clients are > 3.11 if the value exceeds 1GB.

Change-Id: Id0cdda3b053287b659c7bf511b13db2e45b92032
BUG: 1446516
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17338
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

dht: The xattrs sent in readdirp should be sent in opendir aswell

2017-04-06T04:04:03+00:00

As readdir-ahead can be loaded as a child of dht, dht has to specify
the xattrs it is intrested in, as part of opendir call itself.

Change-Id: I012ef96cc143b0cef942df78aa7150d85ec38606
BUG: 1431908
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16902
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

reddir-ahead: Fix EOD propagation problem

2017-04-05T06:24:32+00:00

In readdirp fop, op_errno is overloaded to indicate the EOD detection.
If op_errno contains ENOENT, then it indicates that there are no
further entries pending read in the directory. Currently NFS uses the
ENOENT to identify the EOD.

Issue:
NFS clients issues a 4K buffer for readdirp, readdir-ahead converts it
to 128K buffer as its reading ahead. If there are 100 entries in the
bricks, 128K can get all 100 and store in readdir-ahead, but only 23
entries that can be fit in 4K will be sent to NFS. Since the whole
100 entries were read from brick, the op_errno is set to ENOENT, and
the op_errno is propagated as is when sent to NFS. Hence NFS client
in reading 23 entries thinks it reached EOD.

Solution:
Do not propogate ENOENT errno, unless all the entries are read
from the readdir ahead buffer.

Change-Id: I4f173a77b21ab9e98ae35e291a45b8fc0cde65bd
BUG: 1436086
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16953
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd, readdir-ahead: Fix backward incompatibility

2017-02-18T07:03:11+00:00

Issue:
Any opion is spcified in two places: In the options[] of xlator
itself and glusterd-volume-set.c. The default value of this option
can be specified in both the places. If its specified only in xlator
then the volfile generated will not have the option and default value,
it will be assigned during graph initialization.
With patch [1] the option rda-request-size was changed from INT to SIZET
type, and default was changed from 131072 to 128KB, but was specified
only in the readdir-ahead.c. Thus with this patch alone the volfile
entry for readdir-ahead looks like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    subvolumes patchy-read-ahead
end-volume

With patch [2], the default of option rda-request-size was specified
in glusterd-volume-set.c as well(as it was necessary fr parallel readdir).
With this patch the readdir entry in the volfile will look like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    option rda-cache-limit 10MB
    option rda-request-size 128KB
    option parallel-readdir off
    subvolumes patchy-read-ahead
end-volume


Now consider the server has both these patches and client doesn't.
Server will generate a volfile with entry:

The old clients which thought the option rda-request-size is of type
INT will now recieve the value 128KB which it willn't understand,
and hence fail the mount.

The issue is seen only with the combination of [1] and [2].

Solution:
Instead of specifying 128KB as default in glusterd we specify 131072
so that the old clients will interpret as INT and new ones as 128KB

Credits: Raghavendra G

Change-Id: I0c269a5890957fd8a38e9a05bdec088645a6688a
BUG: 1423410
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16657
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Raghavendra G 
Reviewed-by: Atin Mukherjee

readdir-ahead: Remove unnecessary logging

2017-02-17T12:27:38+00:00

dict_get_int can return < 0 when key is not found is a valid case.
Hence no need to log.

Change-Id: If0795b0f178adbb94b10efc563506993f7411962
BUG: 1423369
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16654
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Jeff Darcy