glusterfs.git/xlators/performance/md-cache, branch round-robin2

md-cache: fix indention to silence Coverity

2016-07-21T18:05:59+00:00

Coverity complains about the incorrect indention:

    *** CID 1357867:  Control flow issues  (NESTING_INDENT_MISMATCH)
    ...
    2566                     if (conf->mdc_invalidation)
    2567                             ret = mdc_invalidate (this, data);
    >>>     CID 1357867:  Control flow issues  (NESTING_INDENT_MISMATCH)
    >>>     This 'if'  statement is indented to column 25, as if it were nested within the preceding parent statement, but it is not.
    2568                             if (default_notify (this, event, data) != 0)
    2569      	                         ret = -1;
    2570                     break;
    ...

Even when md-cache does not have cache-invalidation on, we need to pass
the upcall to the next xlator.

Change-Id: I6d2a174eb54e3df270920aae9673b5010c235f25
CID: 1357867
BUG: 1211863
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14971
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Poornima G 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur

md-cache: Add cache invalidation support to invalidate the meta data cache

2016-07-20T12:12:05+00:00

Problem:
md-cache currently updates its stat in cbks of selected fops.
The default cache time is 1 second, if this is increasd to reap the
benefits of caching, we may end up with stale cache for long time,
as there is no logic yet to notify md-cache of backend changes by
another client.

Solution:
Use the existing upcall mechanism to invalidate the cache.
For this feature to work, "features.cache-invalidation" volume
option should be enabled.

This patch as is doesn't improve any performance, the benifit of the
patch is that it provides coherency for stat cache, hence the cache
timeout can be quite longer which in turn can improve the performance.

Change-Id: I2dbb0afa7b5e4a5a248f910188e0918e02f18692
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/12951
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

core: add a basis function to reduce verbose code

2016-07-18T11:59:05+00:00

Change-Id: Icebe1b865edb317685e93f3ef11d98fd9b2c2e9a
BUG: 1357226
Signed-off-by: Zhou Zhengping 
Reviewed-on: http://review.gluster.org/14936
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

"md-cache: Enable caching of stat fetched from readdirp

2016-07-11T13:48:34+00:00

Patch http://review.gluster.org/11894 removed readdirp fop for
md-cache, but there is no mention of exact xlator which was
failing because of this. As mentioned by Rafi(author of patch 11894)
tiering and svc doesn't really need this as the inode_ctx is populated
in readdirp_cbk. Hence reverting this commit.
This reverts commit c8c9308134ae4ce24c630a1b0ccfcf4e8f9b0fe7.

Change-Id: Ib8d00b3f129596f3a54984f839199175f5c9b55b
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14879
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Zhou Zhengping 
Reviewed-by: Jeff Darcy

md-cache: Cache gluster-samba metadata

2016-07-05T11:00:50+00:00

Change-Id: I0a95f4897440c5bf6f54612d9c232e015c8bf983
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14824
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

readdir-ahead: Prefetch xattrs needed by md-cache

2016-05-10T09:05:38+00:00

Problem:
Negative cache feature implementation in md-cache requires xattrs
returned by posix to be intercepted for every call that can possibly
return xattrs. This includes readdirp(). This is crucial to treat
missing keys in cache as a case of negative entry (returns ENODATA)

md-cache puts names of xattrs that it wants to cache in xdata and
passes it down to posix which returns the specified xattrs in the
callback. This is done in lookup() and readdirp(). Hence, a xattr
that is cached can be invalidated during readdirp_cbk too.

This is based on the assumption that readdirp() will always return
all xattrs that md-cache is interested in. However, this is not the
case when readdirp() call is served from readdir-ahead's cache.
readdir-ahead xlator will pre-fetch dentries during opendir_cbk
and readdirp. These internal readdirp() calls made by readdir-ahead
xlator does not set xdata in it's requests. Hence, no xattrs are
fetched and stored in it's internal cache.

This causes metadata loss in gluster-swift. md-cache returns ENODATA
during getxattr() call even though the xattr for that object exists on
the brick. On receiving ENODATA, gluster-swift will create new metadata
and do setxattr(). This results in loss of information stored in
existing xattr.

Fix:
During opendir, md-cache will communicate to readdir-ahead asking it
to store the names of xattrs it's interested in so that readdir-ahead
can fetch those in all subsequent internal readdirp() calls issued by
it. This stored names of xattrs is invalidated/updated on the next
real readdirp() call issued by application. This readdirp() call will
have xdata set correctly by md-cache xlator.

BUG: 1333023
Change-Id: I32d46f93a99d4ec34c741f3c52b0646d141614f9
Reviewed-on: http://review.gluster.org/14214
Tested-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

md-cache: Cache gluster-swift metadata

2016-03-16T11:23:48+00:00

BUG: 1317785
Change-Id: Ie02b8fc294802f8fdf49dee8bf97f1e6177d92bd
Signed-off-by: Prashanth Pai 
Reviewed-on: http://review.gluster.org/13735
Smoke: Gluster Build System 
Reviewed-by: Poornima G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Gaurav Kumar Garg

build: export minimum symbols from xlators for correct resolution

2015-12-22T17:15:01+00:00

Revisiting http://review.gluster.org/#/c/11814/, which unintentionally
introduced warnings from libtool about the xlator .so names.

According to [1], the -module option must appear in the Makefile.am
file(s); if -module is defined in a macro, e.g. in configure(.ac),
then libtool will not recognize that this is a module and will emit a
warning.

[1]
http://www.gnu.org/software/automake/manual/automake.html#Libtool-Modules

Change-Id: Ifa5f9327d18d139597791c305aa10cc4410fb078
BUG: 1248669
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13003
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Niels de Vos

md-cache: Remove readdirp fop for md-cache

2015-11-09T10:49:39+00:00

readdirp call will return inode for each entry and
will share this nodeid with kernal, also md-cache
will cache this gfid and base name. So when a lookup
operation is perfromed on such an inode, md-cache
will wind the call, that prevents populating inode
ctx for other lower layer xlators.

Change-Id: I43c768703a3cc66d05b1c32909d1a2781001cb49
BUG: 1236032
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/11894
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G

build: export minimum symbols from xlators for correct resolution

2015-09-24T14:37:42+00:00

We've been lucky that we haven't had any symbol collisions until now.
Now we have a collision between the snapview-client's svc_lookup() and
libntirpc's svc_lookup() with nfs-ganesha's FSAL_GLUSTER and libgfapi.

As a short term solution all the snapview-client's FOP methods were
changed to static scope. See http://review.gluster.org/11805. This
works in snapview-client because all the FOP methods are defined in
a single source file. This solution doesn't work for other xlators
with FOP methods defined in multiple source files.

To address this we link with libtool's '-export-symbols $symbol-file'
(a wrapper around `ld --version-script ...` --- on linux anyway) and
only export the minimum required symbols from the xlator sharedlib.

N.B. the libtool man page says that the symbol file should be named
foo.sym, thus the rename of *.exports to *.sym. While foo.exports
worked, we will follow the documentation.

Signed-off-by: Kaleb S. KEITHLEY 
BUG: 1248669
Change-Id: I1de68b3e3be58ae690d8bfb2168bfc019983627c
Reviewed-on: http://review.gluster.org/11814
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Niels de Vos