glusterfs.git/xlators/performance, branch v3.9rc0

compound fops: Some fixes to compound fops framework

2016-08-31T06:50:40+00:00

Change-Id: I808fd5f9f002a35bff94d310c5d61a781e49570b
BUG: 1360169
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/15010
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

md-cache: Process all the cache invalidation flags

2016-08-31T06:40:52+00:00

Currently, md-cache only processes IATT_UPDATE_FLAGS, UP_XATTR and
UP_XATTR_RM. We also need to process UP_RENAME_FLAGS, UP_FORGET,
UP_PARENT_DENTRY_FLAGS and UP_NLINK_FLAGS. Otherwise the files
unlinked or renamed will not be reflected on other mounts.

Change-Id: Icb8b03da51482c3fc2e2a7292d16d56e11a341d9
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15324
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

dht, md-cache, upcall: Add invalidation of IATT when the layout changes

2016-08-31T06:08:54+00:00

Issue:
dht_layout is built as a part of lookup only. The layout can be
modified by rebalance process. Since every IO fop is preceded
by a lookup, there are very less issues of stale layout. But
with enhancements of aggressive caching of stats in md-cache,
the lookup will reduce and expose the stale layout issue often.

Solution:
Since stale layout is already an issue on dht, there is already
a plan to fix this at the dht layer, but this fix is not currently
planned for any release. Until this fix comes out, we can have
a workaround where, the upcall will send a notification to md-cache
when a layout xattr is changed. As a part of layout change notification
the existing cache is invalidated and the next lookup will fetch the
latest layout.

This is not a foolproof solution as the window between the layout change
and the next lookup(after invalidation of stat), where there will be stale
layout. But until the final fix comes in, this reduces the stale layout
window.

Change-Id: Iacf871a38b35880c1fc0bc68fe7ce291265e71d4
BUG: 1369638
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15300
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

md-cache: Register the list of xattrs with cache-invalidation

2016-08-31T06:07:01+00:00

Issue:
md-cache caches a specified list of xattrs, and when cache invalidation
is enabled, it makes sense to recieve invalidation only when those xattrs
are modified by other clients. But the current implementation of upcall
is that, it will send invalidation when any of the on-disk xattrs is modified.

Solution:
md-cache sends a list of xattrs that it is interested in, to upcall by
issuing an ipc(). The challenge here is to make sure everytime a brick
goes offline and comes back up, the ipc() needs to be issued to the
bricks. Hence ipc() is sent from md-cache every time there is a
CHILD_UP/CHILD_MODIFIED event.

TODO:
There will be patches following, in cluster xlators, to implement ipc fop.

Change-Id: I6efcf3df474f5ce6eabd3d6694c00c7bd89bc25d
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15002
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Raghavendra G

md-cache: Fix wrong cache time update for xattrs

2016-08-27T12:56:35+00:00

In md-cache, the cache has two times:
1. Time when the stat was last fetched for that inode
2. Time when the xattrs were last fetched for that inode. This
time should not be updated when only one of the xattrs is updated.
If, its updated when one of the cached xattr is changed, it can so
happen that the other xattrs have past their cache timeout, but are
still served from cache.

Solution:
Do not update the xattr cache time, when one of the xattrs being cached
is changed. With this, we may end up in cache timeout though it was
updated recently, but it is not a harm. The other way is to have
timeout for every xattr that is being cached. Its more complicated, and
may be not worth it, as we have lot of lookup fops, that are overloaded to
get all the xattrs.

Change-Id: Id77e547f403fc792348f1ea56b468b9260a5a34f
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15045
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

md-cache: Do not use features.cache-invalidation for both md-cache and upcall

2016-08-27T12:18:49+00:00

Currently, the volume set option features.cache-invalidation enables upcall
feature on server side and md-cache cache-invalidation on client side.
There are multiple problems that can arise from this:
1. The scenario when user wants to, enable upcall for nfs-ganesha setup,
   but do not want to enable md-cache cache-invalidation, as the
   nfs-clients have already cached the metadata and upcall is used to
   to invalidate the nfs-client cache. In this case, users should have
   a way of disabling md-cache invalidation without disabling upcall.

2. Upcall requires a op-version of GD_OP_VERSION_3_7_0, where as
   md-cache invalidation requires an op version of GD_OP_VERSION_3_9_0.
   Consider a setup where the servers are in op-version GD_OP_VERSION_3_7_0,
   and th clients are in op-version GD_OP_VERSION_3_9_0. if there is one
   single volume set option, user can enable this feature in this setup.
   But it can lead to stale xattr cache as the xattr invalidation was
   introduced in upcall only in release 3.8. Hence, we should not be
   able to enable md-cache invalidation, if all the servers and clients
   are not on opversion >= GD_OP_VERSION_3_9_0.

To solve the above mentioned issues, we have seperate volume options
for enabling md-cache invalidation and upcall. But this can lead to
issues when user enable md-cache invalidation and forgets to enable
upcall. Probably in the next release, these can be enables by default.

Change-Id: Ie70eff97fe12fcb623eec8f4f5861ac065bf483e
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15314
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Raghavendra G

md-cache: Add cache hit and miss counters

2016-08-27T11:14:36+00:00

These counters can be accessed either by .meta interface
or statedump.

From meta: cat on the private file in md-cache directory.
Eg: cat /mnt/glusterfs/0/.meta/graphs/active/patchy-md-cache/private
[performance/md-cache.patchy-md-cache]
stat_hit_count = 2
stat_miss_count = 8
xattr_hit_count = 4
xattr_miss_count = 3
nameless_lookup_count = 1
negative_lookup_count = 0
stat_invalidations_recieved = 1
xattr_invalidations_recieved = 1

Change-Id: Ib62a8822f263a9f75858b15832d0119fbe382629
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15185
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

md-cache: Add logging to increase debuggability

2016-08-16T04:20:44+00:00

Change-Id: I147d16ec3c20d3372892fdd5f62010e52f82f8bd
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15069
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-by: Raghavendra G

md-cache/upcall: In case of mode bit change invalidate xattr

2016-08-04T15:46:44+00:00

When the mode bits are changed, the ACL entries also do get affected.
Currently in upcall, setattr invalidates only the stat info.

With this patch, if mode bits are changed, the upcall will invalidate
all the xattrs.

Change-Id: Iccda2e1a7440ee845aa5442bf51970f74d9b0862
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15043
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Jeff Darcy

io-threads: distribute work fairly among clients

2016-07-28T18:52:06+00:00

This is the full "queue of queues" approach where each client gets its
own queue (per priority) and we round-robin among them.

Change-Id: I73955d1b9bb93f2ff781b48dfe2509009c519ec6
BUG: 1360402
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/14904
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System