glusterfs.git/tests, branch release-3.10

features/index: Choose different base file on EMLINK error

2018-04-10T13:39:23+00:00

Change-Id: I4648816af908539efdc2528608aa2ebf7f0d0e2f
fixes: bz#1553777
BUG: 1553777
Signed-off-by: Pranith Kumar K 
(cherry picked from commit bb12f2109a01856e8184e13cf984210d20155b13)

cluster/ec: avoid delays in self-heal

2018-03-15T07:37:02+00:00

Self-heal creates a thread per brick to sweep the index looking for
files that need to be healed. These threads are started before the
volume comes online, so nothing is done but waiting for the next
sweep. This happens once per minute.

When a replace brick command is executed, the new graph is loaded and
all index sweeper threads started. When all bricks have reported, a
getxattr request is sent to the root directory of the volume. This
causes a heal on it (because the new brick doesn't have good data),
and marks its contents as pending to be healed. This is done by the
index sweeper thread on the next round, one minute later.

This patch solves this problem by waking all index sweeper threads
after a successful check on the root directory.

Additionally, the index sweep thread scans the index directory
sequentially, but it might happen that after healing a directory entry
more index entries are created but skipped by the current directory
scan. This causes the remaining entries to be processed on the next
round, one minute later. The same can happen in the next round, so
the heal is running in bursts and taking a lot to finish, specially
on volumes with many directory levels.

This patch solves this problem by immediately restarting the index
sweep if a directory has been healed.

Backport of:
> BUG: 1547662

Change-Id: I58d9ab6ef17b30f704dc322e1d3d53b904e5f30e
BUG: 1555203
Signed-off-by: Xavi Hernandez

tests: Mark tests/bugs/posix/bug-990028.t bad

2018-02-20T02:13:48+00:00

Change-Id: I92ecd4dd292118900b80833f116c518b7493da17
BUG: 1546912
Signed-off-by: Atin Mukherjee

tests: Fix a broken test case

2018-02-13T14:04:40+00:00

Issue: When using statedump command to take statedump of the
gfapi process, we specify the following things:

$gluster volume statedump  client :
   pid: Pid of the gfapi application
   host: This should be the IP/hostname as seen by the glusterd,
         the gfapi application is connected to.

In this test case, if gfapi application is running locally,
and is connected to $H1 glusterd, the  need not be $H1.
 could be localhost, 127.0.0.1, 127.1.1.1 etc. based on
the configuration of the system. Hence use netstat to find the
right  value.

>mainline patch : https://review.gluster.org/#/c/18893/

Change-Id: I6efb9d1ccaf9c6841a9ab7c9ebfecafc03c0bc5e
BUG: 1544787
Signed-off-by: Poornima G 
(cherry picked from commit 5529659dec7607bf9b94ea2195672ae553458785)

posix: delete stale gfid handles in nameless lookup

2018-01-18T18:21:37+00:00

..in order for self-heal of symlinks to work properly (see BZ for
details).

Backport of https://review.gluster.org/#/c/19070/
Signed-off-by: Ravishankar N 

Change-Id: I9a011d00b07a690446f7fd3589e96f840e8b7501
BUG: 1534848

mount/fuse: use fstat in getattr implementation if any opened fd is available

2018-01-03T14:54:19+00:00

The restriction of using fds opened by the same Pid means fds cannot
be shared across threads of multithreaded application. Note that fops
from kernel have different Pid for different threads. Imagine
following sequence of operations:

* Turn off performance.open-behind
* Thread t1 opens an fd - fd1 - on file "file". Let's assume nodeid of
  "file" is "nodeid-file".
* Thread t2 does RENAME ("newfile", "file"). Let's assume nodeid of
  "newfile" as "nodeid-newfile".
* t2 proceeds to do fstat (fd1)

The above set of operations can sometimes result in ESTALE/ENOENT
errors. RENAME overwrites "file" with "newfile" changing its nodeid
from "nodeid-file" to "nodeid-newfile" and post RENAME, "nodeid-file" is
removed from the backend. If fstat carries nodeid-file as argument,
which can happen if lookup has not refreshed the nodeid of "file" and
since t2 doesn't have an fd opened, fuse_getattr_resume uses STAT
which will fail as "nodeid-file" no longer exists.

Since the above set of operations and sharing of fds across
multiple threads are valid, this is a bug.

The fix is to use any fd opened on the inode. In this specific example
fuse_getattr_resume will find fd1 and winds down the call as fstat
(fd1) which won't fail.

Cross-checked with "Miklos Szeredi"  for
any security issues with this solution and he approves the solution.

Thanks to "Miklos Szeredi"  for all the
pointers and discussions.

>Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c
>BUG: 1510401
>Signed-off-by: Raghavendra G 

(cherry picked from commit 8b57378e5596f287a7b9d106dd6fb56a624b42ee)
Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c
BUG: 1529086
Signed-off-by: Raghavendra G

glusterd: delete source brick only once in reset-brick commit force

2017-10-31T18:08:14+00:00

While stopping the brick which is to be reset and replaced delete_brick
flag was passed as true which resulted glusterd to free up to source
brick before the actual operation. This results commit force to fail
failing to find the source brickinfo.

> mainline patch : https://review.gluster.org/#/c/18581/

Change-Id: I1aa7508eff7cc9c9b5d6f5163f3bb92736d6df44
BUG: 1507880
Signed-off-by: Atin Mukherjee 
(cherry picked from commit 0fb8acaa6ff80c43e46deac0ce66b29ae0df0ca4)

glusterd: Gluster should keep PID file in correct location

2017-10-25T14:03:54+00:00

Currently Gluster keeps process pid information of all the daemons
and brick processes in Gluster configuration file directory
(ie., /var/lib/glusterd/*).

These pid files should be seperate from configuration files.
Deletion of the configuration file directory might result into serious problems.
Also, /var/run/gluster is the default placeholder directory for pid files.

So, with this fix Gluster will keep all process pid information of all
processes in /var/run/gluster/* directory.

> Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
> BUG: 1258561
> Signed-off-by: Gaurav Kumar Garg 
> Signed-off-by: Saravanakumar Arumugam 
> Reviewed-on: https://review.gluster.org/13580
> Tested-by: MOHIT AGRAWAL 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> (Cherry pick from commit 220d406ad13d840e950eef001a2b36f87570058d)

BUG: 1491059
Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
Signed-off-by: Mohit Agrawal

features/worm: Adding check to newloc when doing rename

2017-10-25T14:03:28+00:00

Problem: Since rename didn't check if newloc exists and it's
retention state it was possible to rename a new file that wasn't
in retention over a existing file that was in read-only state.

Cherry picked from commit 00a4dc0:
> Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229
> BUG: 1484490
> Signed-off-by: luneo7 
> Reviewed-on: https://review.gluster.org/18104
> Tested-by: jiffin tony Thottan 
> Tested-by: Prashanth Pai 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Prashanth Pai 
> Reviewed-by: Karthik U S 
> Reviewed-by: Amar Tumballi 

Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229
BUG: 1480788
Signed-off-by: luneo7

md-cache: avoid checking the xattr value buffer with string functions.

2017-10-25T14:00:03+00:00

xattrs may very well contain binary, non-text data with leading 0
values. Using strcmp for checking empty values is not the appropriate
thing to do: In the best case, it might treat a binary xattr value
starting with 0 from being cached (and hence also from being reported
back with xattr). In the worst case, we might read beyond the end
of a data blob that does contain any zero byte.

We fix this by checking the length of the data blob and checking
the first byte against 0 if the length is one.

> Signed-off-by: Guenther Deschner 
> Pair-Programmed-With: Michael Adam 
> Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11
> BUG: 1476324
> Reviewed-on: https://review.gluster.org/17910
> Reviewed-by: Michael Adam 
> Smoke: Gluster Build System 
> Reviewed-by: Poornima G 
> Tested-by: Poornima G 
> CentOS-regression: Gluster Build System 
> (cherry picked from commit ab4ffdac9dec1867f2d9b33242179cf2b347319d)

Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11
BUG: 1499893
Signed-off-by: Günther Deschner