glusterfs.git/libglusterfs/src/xlator.h, branch v3.10.7

afr: heal metadata in discover code path

2017-09-17T13:37:32+00:00

******************************************************
Backport of: https://review.gluster.org/18202
Also added loc_is_nameless() to libglusterfs since the patch that
introduced it in master was not backported to release-3.10.

Note: 18202 is a squash of  17850 and 18187 in master.
******************************************************

During graph switch, if fuse sends nameless (gfid) lookups, afr takes
the discover code path to serve it. If there are pending metadata heals,
they do not happen unless an inode refresh happens as a part of
discover (which is not guaranteed to happen always).

This patch fixes it by attempting metadata heal as a part of discover,
just like how it is done in lookup code path.

Change-Id: I87c493045b9225741cad173bf3f645848697032e
BUG: 1492010
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18304
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

libglusterfs: Fix a crash due to race between inode_ctx_set and inode_ref

2017-02-20T12:25:29+00:00

Issue:
Currently inode ref count is guarded by inode_table->lock, and
inode_ctx is guarded by inode->lock. With the new patch [1]
inode_ref was modified to change the inode_ctx to track the ref
count per xlator. Thus inode_ref performed under inode_table->lock
is modifying inode_ctx which has to be modified only under inode->lock

Solution:
When a inode is created, inode_ctx holder is allocated for all the xlators.
Hence in case of inode_ctx_set instead of using the first free index in
inode ctx holder, we can have predecided index for every xlator in the graph.

Credits Pranith K 

[1] http://review.gluster.org/13736

> Reviewed-on: https://review.gluster.org/16622
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Niels de Vos 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 

Change-Id: I1bfe111c211fcc4fcd761bba01dc87c4c69b5170
BUG: 1423385
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16655
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

core: run many bricks within one glusterfsd process

2017-02-02T00:54:58+00:00

This patch adds support for multiple brick translator stacks running in
a single brick server process.  This reduces our per-brick memory usage
by approximately 3x, and our appetite for TCP ports even more.  It also
creates potential to avoid process/thread thrashing, and to improve QoS
by scheduling more carefully across the bricks, but realizing that
potential will require further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global
option.  By default it's off, and bricks are started in separate
processes as before.  If multiplexing is enabled, then *compatible*
bricks (mostly those with the same transport options) will be started in
the same process.

Backport of:
> Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/14763

Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8
BUG: 1418091
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16496
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

libglusterfs: serialize init/reconfigure calls

2017-01-06T04:38:49+00:00

These functions do not generally "expect" to be called more than once
in parallel, and many are likely to misbehave in that case (one case
in DHT already).  Such parallel calls have not generally happened
because there are only a few places where we call these functions, and
those have been implicitly serialized until recently.  However, recent
changes in the epoll layer change that, as does brick multiplexing.
Therefore, the serialization is now explicit at the init/reconfigure
level.

It would be sufficient to serialize calls to a particular translator's
init and reconfigure functions, but that would require per-translator
locks and a bit more complexity in maintaining/using them.  Since
there's no clear reason why we would need or want to support a higher
level of parallelism, the simpler approach of a global lock should
suffice.

Change-Id: I26296c2826e91dc00b7f0c2061bcc2964ef90c4c
BUG: 1399134
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/16030
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

performance/readdir-ahead: limit cache size

2016-12-22T11:43:14+00:00

This patch introduces a new option called "rda-cache-limit", which is
the maximum value the entire readdir-ahead cache can grow into. Since,
readdir-ahead holds a reference to inode through dentries, this patch
also accounts memory stored by various xlators in inode contexts.

Change-Id: I84cc0ca812f35e0f9041f8cc71effae53a9e7f99
BUG: 1356960
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/16137
NetBSD-regression: NetBSD Build System 
Reviewed-by: Poornima G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

features/shard: Fill loc.pargfid too for named lookups on individual shards

2016-11-08T11:05:27+00:00

On a sharded volume when a brick is replaced while IO is going on, named
lookup on individual shards as part of read/write was failing with
ENOENT on the replaced brick, and as a result AFR initiated name heal in
lookup callback. But since pargfid was empty (which is what this patch
attempts to fix), the resolution of the shards by protocol/server used
to fail and the following pattern of logs was seen:

Brick-logs:

[2016-11-08 07:41:49.387127] W [MSGID: 115009]
[server-resolve.c:566:server_resolve] 0-rep-server: no resolution type
for (null) (LOOKUP)
[2016-11-08 07:41:49.387157] E [MSGID: 115050]
[server-rpc-fops.c:156:server_lookup_cbk] 0-rep-server: 91833: LOOKUP(null)
(00000000-0000-0000-0000-000000000000/16d47463-ece5-4b33-9c93-470be918c0f6.82)
==> (Invalid argument) [Invalid argument]

Client-logs:
[2016-11-08 07:41:27.497687] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-0: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.497755] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-1: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.498500] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-2: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.499680] E [MSGID: 133010]

Also, this patch makes AFR by itself choose a non-NULL pargfid even if
its ancestors fail to initialize all pargfid placeholders.

Change-Id: I5f85b303ede135baaf92e87ec8e09941f5ded6c1
BUG: 1392445
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15788
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System

core: add setactivelk () fop

2016-05-02T01:04:42+00:00

Change-Id: Ic2ba77a1fdd27801a6e579e04e6c0dd93cd7127b
BUG: 1326085
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14011
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: add getactivelk () fop

2016-05-02T01:04:21+00:00

Change-Id: Ifd0ff278dcf43da064021f5c25e5dcd34347fcde
BUG: 1326085
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/13970
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

performance/decompounder: Introducing decompounder xlator

2016-04-26T06:47:28+00:00

This xlator decompounds the compound fops received,
and executes them serially.

Change-Id: Ieddcec3c2983dd9ca7919ba9d7ecaa5192a5f489
BUG: 1303829
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/13577
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

core: add lease fop

2016-04-21T10:01:38+00:00

Change-Id: Ia27d66b1061b0377857827515590eb89b18515c9
BUG: 1319992
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/11596
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Pranith Kumar Karampuri