glusterfs.git/xlators/performance, branch v3.10.0

glusterd, readdir-ahead: Fix backward incompatibility

2017-02-19T14:18:08+00:00

Backport of https://review.gluster.org/#/c/16657/

Issue:
Any opion is spcified in two places: In the options[] of xlator
itself and glusterd-volume-set.c. The default value of this option
can be specified in both the places. If its specified only in xlator
then the volfile generated will not have the option and default value,
it will be assigned during graph initialization.
With patch [1] the option rda-request-size was changed from INT to SIZET
type, and default was changed from 131072 to 128KB, but was specified
only in the readdir-ahead.c. Thus with this patch alone the volfile
entry for readdir-ahead looks like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    subvolumes patchy-read-ahead
end-volume

With patch [2], the default of option rda-request-size was specified
in glusterd-volume-set.c as well(as it was necessary fr parallel readdir).
With this patch the readdir entry in the volfile will look like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    option rda-cache-limit 10MB
    option rda-request-size 128KB
    option parallel-readdir off
    subvolumes patchy-read-ahead
end-volume


Now consider the server has both these patches and client doesn't.
Server will generate a volfile with entry:

The old clients which thought the option rda-request-size is of type
INT will now recieve the value 128KB which it willn't understand,
and hence fail the mount.

The issue is seen only with the combination of [1] and [2].

Solution:
Instead of specifying 128KB as default in glusterd we specify 131072
so that the old clients will interpret as INT and new ones as 128KB

Credits: Raghavendra G

> Reviewed-on: https://review.gluster.org/16657
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Atin Mukherjee 

Change-Id: I0c269a5890957fd8a38e9a05bdec088645a6688a
BUG: 1423412
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16658
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee

readdir-ahead: Remove unnecessary logging

2017-02-17T16:56:00+00:00

dict_get_int can return < 0 when key is not found is a valid case.
Hence no need to log.

> Reviewed-on: https://review.gluster.org/16654
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Jeff Darcy  

Change-Id: If0795b0f178adbb94b10efc563506993f7411962
BUG: 1423429
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16659
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

performance/decompounder: Have terminal value in options[]

2017-02-17T11:46:15+00:00

Absence of terminal values is leading to buffer-over-flow errors in
address sanitizer.

 >BUG: 1422152
 >Change-Id: I769c0e4b5bbb3ef2849b8d1097b9def522ae08d9
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: https://review.gluster.org/16615
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Jeff Darcy 
 >(cherry picked from commit fca31ae5faa15a02e64143b9efb7e19c3b907c2d)

Change-Id: Ifec6f5b96cfe4c26a039f88649284bcdad5321f6
BUG: 1423070
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/16652
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

md-cache: initialize mdc_counter.lock

2017-02-16T17:53:25+00:00

add missed LOCK_INIT to fix INCREMENT_ATOMIC on
conf->mdc_counter.lock when pthread_spin_* using

Cherry picked from commit 22f02d8f1dcdf176744ab1536cb23a5fcd291243:
> Change-Id: I680bd6f41e3b8a1852ed969bf6794cbf4c1ccdd4
> BUG: 1417913
> Signed-off-by: Vitaly Lipatov 
> Reviewed-on: https://review.gluster.org/16515
> Reviewed-by: Niels de Vos 
> Tested-by: Niels de Vos 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Poornima G 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Vijay Bellur 

Change-Id: I680bd6f41e3b8a1852ed969bf6794cbf4c1ccdd4
BUG: 1417915
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16640
Smoke: Gluster Build System 
Reviewed-by: Poornima G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

performance/write-behind: access stub only if available during

2017-02-02T17:36:43+00:00

statedump

>Change-Id: Ia5dd718458a5e32138012f81f014d13fc6b28be2
>BUG: 1415115
>Signed-off-by: Raghavendra G 
>Reviewed-on: https://review.gluster.org/16440
>Reviewed-by: N Balachandran 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 

Change-Id: Ia5dd718458a5e32138012f81f014d13fc6b28be2
BUG: 1418623
Signed-off-by: Raghavendra G 
(cherry picked from commit 85d7f1d1ee24ac400d4aa223478727643532693a)
Reviewed-on: https://review.gluster.org/16519
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

libglusterfs+transport+io-threads: fix 256KB stack abuse

2017-02-02T17:32:49+00:00

Some functions were allocating 64K booleans, which are (crazily)
mapped to 4-byte ints, for a total of 256KB per call.  Changed to use
bitfields instead, so usage is now only 8KB per call.  This was the
impediment to changing the io-threads stack size, so that has been
adjusted too.

Backport of:
> Change-Id: I8781c4f2c8f2b830f4535e366995fac8dd0a8653
> BUG: 1418095
> Reviewed-on: https://review.gluster.org/15745

Change-Id: Ia5dada61703e6bea95f2511da71feb573fc9a429
BUG: 1418536
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16511
Reviewed-by: N Balachandran 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur 
CentOS-regression: Gluster Build System

md-cache: Cache security.ima xattrs

2017-01-30T14:25:03+00:00

Backport of http://review.gluster.org/16296

From kernel version 3.X or greater, creating of a file
results in removexattr call on security.ima xattr. But
this xattr is not set on the file unless IMA feature
is active. With this patch, removxattr call returns
ENODATA if it is not found in the cache.

> Change-Id: I8136096598a983aebc09901945eba1db1b2f93c9
> Signed-off-by: Poornima G 
> Reviewed-on: http://review.gluster.org/16296
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> (cherry picked from commit ac629e574935a8aed6526936bc83b1c6d295ae67)

Change-Id: I27abc23024c8fcf07389608df61ef6e64736d414
BUG: 1415918
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16460
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

Readdir-ahead : Honour readdir-optimise option of dht

2017-01-27T14:01:15+00:00

>Change-Id: I9c5e65b32e316e6a2fc7e1f5c79fce79386b78e2
>BUG: 1401812
>Signed-off-by: Poornima G 
>Reviewed-on: https://review.gluster.org/16071
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Raghavendra G 

Change-Id: I9c5e65b32e316e6a2fc7e1f5c79fce79386b78e2
BUG: 1417027
Signed-off-by: Poornima G 
(cherry picked from commit 7c6538f6c8f9a015663b4fc57c640a7c451c87f7)
Reviewed-on: https://review.gluster.org/16461
Tested-by: Raghavendra G 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Shyamsundar Ranganathan

readdir-ahead : Perform STACK_UNWIND outside of mutex locks

2017-01-10T04:52:28+00:00

Currently STACK_UNWIND is performnd within ctx->lock.
If readdir-ahead is loaded as a child of dht, then there
can be scenarios where the function calling STACK_UNWIND
becomes re-entrant. Its a good practice to not call
STACK_WIND/UNWIND within local mutex's

Change-Id: If4e869849d99ce233014a8aad7c4d5eef8dc2e98
BUG: 1401812
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/16068
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Raghavendra G

md-cache: Cache updated as a part of invalidate should not update time

2017-01-09T05:12:48+00:00

Currently when a invalidate happens we update the cache along with
the cache time. The problem with this is, upcall doesn't update the
last access time of a client when an invalidation is sent, thus resulting
in a timewindow where the md-cache has cached, but the upcall is unaware
and hence upcall will not further invalidate the cache(unless a fop is sent
from the same client, and upcall updates its database to reflect the same)

Change-Id: Ibceb8d2fc360582752846bbf7fd59697d5424754
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/16295
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G