readdir-ahead: Prefetch xattrs needed by md-cache

2016-05-10T09:05:38+00:00

Problem:
Negative cache feature implementation in md-cache requires xattrs
returned by posix to be intercepted for every call that can possibly
return xattrs. This includes readdirp(). This is crucial to treat
missing keys in cache as a case of negative entry (returns ENODATA)

md-cache puts names of xattrs that it wants to cache in xdata and
passes it down to posix which returns the specified xattrs in the
callback. This is done in lookup() and readdirp(). Hence, a xattr
that is cached can be invalidated during readdirp_cbk too.

This is based on the assumption that readdirp() will always return
all xattrs that md-cache is interested in. However, this is not the
case when readdirp() call is served from readdir-ahead's cache.
readdir-ahead xlator will pre-fetch dentries during opendir_cbk
and readdirp. These internal readdirp() calls made by readdir-ahead
xlator does not set xdata in it's requests. Hence, no xattrs are
fetched and stored in it's internal cache.

This causes metadata loss in gluster-swift. md-cache returns ENODATA
during getxattr() call even though the xattr for that object exists on
the brick. On receiving ENODATA, gluster-swift will create new metadata
and do setxattr(). This results in loss of information stored in
existing xattr.

Fix:
During opendir, md-cache will communicate to readdir-ahead asking it
to store the names of xattrs it's interested in so that readdir-ahead
can fetch those in all subsequent internal readdirp() calls issued by
it. This stored names of xattrs is invalidated/updated on the next
real readdirp() call issued by application. This readdirp() call will
have xdata set correctly by md-cache xlator.

BUG: 1333023
Change-Id: I32d46f93a99d4ec34c741f3c52b0646d141614f9
Reviewed-on: http://review.gluster.org/14214
Tested-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

performance/readdir-ahead: introduce directory read-ahead translator

2013-09-04T16:04:15+00:00

This is a translator to improve the performance of typical,
sequential directory reads (i.e., ls). readdir-ahead begins
preloading the contents of a directory on open and serves readdir
requests from the preloaded content. readdir-ahead is currently
implemented to only handle the single threaded directory read
case.

readdir-ahead is currently disabled by default. It can be enabled
with the following command:

	gluster volume set  readdir-ahead on

The following are results of a getdents test on a single brick
volume.

Test info:

- Single VM, gluster client/server.
- Volume mounted with native client using --gid-timeout=2.
- getdents on single directory with 100k 0-byte files.

Test results:

- !readdir-ahead

read 3120080 bytes from offset 0
3 MiB, 4348 ops, 0:00:07.00 (416.590 KiB/sec and 594.4737 ops/sec)

- readdir-ahead

read 3120080 bytes from offset 0
3 MiB, 4348 ops, 0:00:03.00 (820.116 KiB/sec and 1170.3043 ops/sec)

BUG: 980517
Change-Id: Ieceb9e1eb47d1d5b5af8da2bf03839537364653f
Signed-off-by: Brian Foster 
Reviewed-on: http://review.gluster.org/4519
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterfs.git/xlators/performance/readdir-ahead/src/readdir-ahead.h, branch v3.10dev

readdir-ahead: Prefetch xattrs needed by md-cache

performance/readdir-ahead: introduce directory read-ahead translator