glusterfs.git/tests, branch v3.10.7

glusterd: Gluster should keep PID file in correct location

2017-10-25T14:03:54+00:00

Currently Gluster keeps process pid information of all the daemons
and brick processes in Gluster configuration file directory
(ie., /var/lib/glusterd/*).

These pid files should be seperate from configuration files.
Deletion of the configuration file directory might result into serious problems.
Also, /var/run/gluster is the default placeholder directory for pid files.

So, with this fix Gluster will keep all process pid information of all
processes in /var/run/gluster/* directory.

> Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
> BUG: 1258561
> Signed-off-by: Gaurav Kumar Garg 
> Signed-off-by: Saravanakumar Arumugam 
> Reviewed-on: https://review.gluster.org/13580
> Tested-by: MOHIT AGRAWAL 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> (Cherry pick from commit 220d406ad13d840e950eef001a2b36f87570058d)

BUG: 1491059
Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
Signed-off-by: Mohit Agrawal

features/worm: Adding check to newloc when doing rename

2017-10-25T14:03:28+00:00

Problem: Since rename didn't check if newloc exists and it's
retention state it was possible to rename a new file that wasn't
in retention over a existing file that was in read-only state.

Cherry picked from commit 00a4dc0:
> Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229
> BUG: 1484490
> Signed-off-by: luneo7 
> Reviewed-on: https://review.gluster.org/18104
> Tested-by: jiffin tony Thottan 
> Tested-by: Prashanth Pai 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Prashanth Pai 
> Reviewed-by: Karthik U S 
> Reviewed-by: Amar Tumballi 

Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229
BUG: 1480788
Signed-off-by: luneo7

md-cache: avoid checking the xattr value buffer with string functions.

2017-10-25T14:00:03+00:00

xattrs may very well contain binary, non-text data with leading 0
values. Using strcmp for checking empty values is not the appropriate
thing to do: In the best case, it might treat a binary xattr value
starting with 0 from being cached (and hence also from being reported
back with xattr). In the worst case, we might read beyond the end
of a data blob that does contain any zero byte.

We fix this by checking the length of the data blob and checking
the first byte against 0 if the length is one.

> Signed-off-by: Guenther Deschner 
> Pair-Programmed-With: Michael Adam 
> Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11
> BUG: 1476324
> Reviewed-on: https://review.gluster.org/17910
> Reviewed-by: Michael Adam 
> Smoke: Gluster Build System 
> Reviewed-by: Poornima G 
> Tested-by: Poornima G 
> CentOS-regression: Gluster Build System 
> (cherry picked from commit ab4ffdac9dec1867f2d9b33242179cf2b347319d)

Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11
BUG: 1499893
Signed-off-by: Günther Deschner

gfapi: set lkowner in glfd

2017-10-17T10:50:19+00:00

We need a provision to be able to set lkowner (which is
used to distinguish locks maintained by server) in gfapi.
Since the same lk_owner need to be used to be able to
flush the lock while closing the fd, store the lkowner
in the glfd structure itself.

A new API has been added to be able to set lkowner in glfd.

This is backport of below mainline fix -
 https://review.gluster.org/#/c/18429
 https://review.gluster.org/#/c/18522/

Change-Id: I67591d6b9a89c20b9617d52616513ff9e6c06b47
BUG: 1501955
Signed-off-by: Soumya Koduri

mount/fuse: Make event-history feature configurable

2017-10-02T12:41:57+00:00

... and disable it by default.

        Backport of:
        > Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
        > Reviewed-on: https://review.gluster.org/18242
        > BUG: 1467614
        > cherry-picked from commit 956d43d6e89d40ee683547003b876f1f456f03b6

This is because having it disabled seems to improve performance.
This could be due to the lock contention by the different epoll threads
on the circular buff lock in the fop cbks just before writing their response
to /dev/fuse.

Just to provide some data - wrt ovirt-gluster hyperconverged
environment, I saw an increase in IOPs by 12K with event-history
disabled for randrom read workload.

Usage:
mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT
OR
glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT

Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
BUG: 1495430
Signed-off-by: Krutika Dhananjay

afr: auto-resolve split-brains for zero-byte files

2017-10-02T12:34:23+00:00

Backport of https://review.gluster.org/#/c/18283/

Problems:
As described in BZ 1491670, renaming hardlinks can result in data/mdata
split-brain of the DHT link-to files (T files) without any mismatch of
data and metadata.

As described in BZ 1486063, for a zero-byte file with only dirty bits
set, arbiter brick will likely be chosen as the source brick.

Fix:
For zero byte files in split-brain, pick first brick as
a) data source if file size is zero on all bricks.
b) metadata source if metadata is the same on all bricks

In arbiter case, if file size is zero on all bricks and there are no
pending afr xattrs, pick 1st brick as data source.

Change-Id: I0270a9a2f97c3b21087e280bb890159b43975e04
BUG: 1496321
Signed-off-by: Ravishankar N 
Reported-by: Rahul Hinduja 
Reported-by: Mabi

posix: add sanity checks for removing the gfid symlink for directories

2017-09-23T12:19:56+00:00

Backport of https://review.gluster.org/17945

...during mkdir and rmdir. Otherwise, during entry self-heal, the
directory could be left out without a .glusterfs symlink causing fops like
opendir, readdir to fail.  The only chance the missing symlink will be
created is when a fresh lookup comes on it.

Change-Id: I2e1cf1bce8962ea80187edd8f6d73e0a09cf9f8e
BUG: 1491966
Signed-off-by: Ravishankar N

features/shard: Return aggregated size in stbuf of LINK fop

2017-09-17T12:48:22+00:00

        Backport of:
        > Change-Id: I42df7679d63fec9b4c03b8dbc66c5625f097fac0
        > Reviewed-on: https://review.gluster.org/18209
        > BUG: 1488546
        > cherry-picked from 91430817ce5bcbeabf057e9c978485728a85fb2b

Change-Id: I42df7679d63fec9b4c03b8dbc66c5625f097fac0
BUG: 1488719
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/18212
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System

cluster/dht: Make rebalance throttle option tuned by number

2017-08-11T18:06:40+00:00

Current rebalance throttle options: lazy/normal/aggressive may not always be
sufficient for the purpose of throttling.  In our recent test, we observed for
certain setups, normal and aggressive modes behaved similarly consuming full
disk bandwidth. So in cases like this admin should be able to  tune it
down(or vice versa) depending on the need.

Along with old throttle configurations, thread counts are tuned based on number.
e.g. gluster v set vol-name cluster-rebal.throttle  5.

Admin can tune up/down between 0 and the number of cores available.

Note: For heterogenous servers, validation will fail on the old server if "number"
is given for throttle configuration.
The message looks something like this:
"volume set: failed: Staging failed on vm2. Error: cluster.rebal-throttle should be {lazy|normal|aggressive}"

Test: Manual test by logging active thread number after reconfiguring throttle option.
testcase: tests/basic/distribute/throttle-rebal.t

> Change-Id: I46e3cde546900307831028b344ecf601fd9b02c3
> BUG: 1438370
> Signed-off-by: Susant Palai 
> Reviewed-on: https://review.gluster.org/16980
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Raghavendra G 
> Signed-off-by: Susant Palai 

Change-Id: I46e3cde546900307831028b344ecf601fd9b02c3
BUG: 1473136
Signed-off-by: Susant Palai 
Reviewed-on: https://review.gluster.org/17835
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: Make rebalance honor min-free-disk

2017-08-11T12:18:11+00:00

test:  Manual

created files of size 1K on 2 brick(of size 1GB) setup .
added a brick of size 16GB.
set min-free-disk to 12GB(so that first two bricks won't receive any files).
removed one of the 1st brick of size 1GB.

Logs from test:
[2017-04-12 08:52:08.196484] W [MSGID: 0] [dht-rebalance.c:895:__dht_check_free_space]
 0-test1-dht: Write will cross min-free-disk for file - /tile32 on subvol - test1-client-1.
Looking for new subvol.

[2017-04-12 08:52:08.196904] I [MSGID: 0] [dht-rebalance.c:925:__dht_check_free_space]
0-test1-dht: new target found - test1-client-2 for file - /tile32

 - Post migration we have two files. The new destination (/brick/1) has the data file
[root@vm1 ~]# ll /brick/1/tile32
-rw-r--r--. 2 root root 0 Apr 12 14:22 /brick/1/tile32

 - On the old target the linkto file is there with linkto xattr pointing to /brick/1
[root@vm1 ~]# ll /tmp/2/tile32
---------T. 2 root root 1000 Apr 12 14:22 /tmp/2/tile32
[root@vm1 ~]# getfattr -m . -de text /tmp/2/tile32
getfattr: Removing leading '/' from absolute path names
security.selinux="unconfined_u:object_r:user_tmp_t:s0"
trusted.gfid="����:Aс�#�/'b2"
trusted.glusterfs.dht.linkto="test1-client-2"

Marking ./tests/features/worm_sh.t as bad test.
Reason being, this patch failed on master branch as well and it has nothing
to do with rebalance/remove-brick.

> BUG: 1441508
> Change-Id: I90bae251cda3d957a49cdceda90cd08311a392fb
> Signed-off-by: Susant Palai 
> Reviewed-on: https://review.gluster.org/17034
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
> Signed-off-by: Susant Palai 

Change-Id: I90bae251cda3d957a49cdceda90cd08311a392fb
BUG: 1473132
Signed-off-by: Susant Palai 
Reviewed-on: https://review.gluster.org/17831
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan