glusterfs.git/xlators, branch v3.12.5

glusterd: connect to an existing brick process when qourum status is NOT_APPLICABLE_QUORUM

2018-01-12T05:43:49+00:00

First of all, this patch reverts commit 635c1c3 as the same is causing a
regression with bricks not coming up on time when a node is rebooted.
This patch tries to fix the problem in a different way by just trying to
connect to an existing running brick when quorum status is not
applicable.

>mainline patch : https://review.gluster.org/#/c/19134/

Change-Id: I0efb5901832824b1c15dcac529bffac85173e097
BUG: 1511301
Signed-off-by: Atin Mukherjee

cluster/dht: Use percentages for space check

2018-01-12T05:37:34+00:00

With heterogenous bricks now being supported in DHT
we could run into issues where files are not migrated
even though there is sufficient space in newly added bricks
which just happen to be considerably smaller than older
bricks. Using percentages instead of absolute available
space for space checks can mitigate that to some extent.

Marking bug-1247563.t as that used to depend on the easier
code to prevent a file from migrating. This will be removed
once we find a way to force a file migration failure.

> Change-Id: I3452520511f304dbf5af86f0632f654a92fcb647
> BUG: 1529440
> Signed-off-by: N Balachandran 

Change-Id: I3452520511f304dbf5af86f0632f654a92fcb647
BUG: 1530455
Signed-off-by: N Balachandran

quota: fixes issue in quota.conf when setting large number of limits

2018-01-10T06:59:18+00:00

Problem: It was not possible to configure more than 7712 quota limits.
This was because a stack buffer of size 131072 was used to read from
quota.conf file. In the new format of quota.conf file each gfid entry
takes 17bytes (16byte gfid + 1 byte type). So, the buf_size was not a
multiple of gfid entry size and as per code this was considered as
corruption.

Solution: make buf size multiple of gfid entry size

Change-Id: Id036225505a47a4f6fa515a572ee7b0c958f30ed
BUG: 1489043
Signed-off-by: Sanoj Unnikrishnan 
(cherry picked from commit 2899a4f125735636fe7cd8db73c0b8a13289df9b)

Revert "mount/fuse: report ESTALE as ENOENT"

2018-01-10T06:56:57+00:00

This reverts commit 26d16b90ec7f8acbe07e56e8fe1baf9c9fa1519e.

Consider rename (index.new, store.idx) and open (store.idx) being
executed in parallel. When we break down operations following sequence
is possible.

* lookup (store.idx) - as part of open(store.idx) returns gfid1 as the
  result.
* rename (index.new, store.idx) changes gfid of store.idx to
  gfid2. Note that gfid2 was the nodeid of index.new. Since rename is
  successful, gfid2 is associated with store.idx.
* open (store.idx) resumes and issues open fop to glusterfs with
  gfid1. open in glusterfs fails as gfid1 doesn't exist and the error
  returned by glusterfs to kernel-fuse is ENOENT.
* kernel passes back the same error to application as a result to
  open.

This error could've been prevented if kernel retries open with
gfid2. Interestingly kernel do retry open when it receives ESTALE
error. Even though failure to find gfid resulted in ESTALE error,
commit 26d16b90ec7f8acb converted that error to ENOENT while sending
an error reply to kernel. This prevented kernel from retrying open
resulting in error.

>Change-Id: I2e752ca60dd8af1b989dd1d29c7b002ee58440b4
>BUG: 1500269
>Signed-off-by: Raghavendra G 

(cherry picked from commit 019a55e708375d2b1e576fcc948a691bcdc5c749)
Change-Id: I2e752ca60dd8af1b989dd1d29c7b002ee58440b4
BUG: 1529088
Signed-off-by: Raghavendra G

glusterd: Nullify pmap entry for bricks belonging to same port

2018-01-10T06:51:34+00:00

Commit 30e0b86 tried to address all the stale port issues glusterd had
in case of a brick is abruptly killed. For brick multiplexing case
because of a bug the portmap entry was not getting removed. This patch
addresses the same.

>mainline patch : https://review.gluster.org/#/c/19119/

Change-Id: Ib020b967a9b92f1abae9cab9492f0cacec59aaa1
BUG: 1530448
Signed-off-by: Atin Mukherjee

mount/fuse: never fail open(dir) with ENOENT

2018-01-03T10:03:02+00:00

open(dir) being an operation on inode should never fail with
ENOENT. If gfid is not present, the appropriate error is ESTALE. This
will enable kernel to retry open after a revalidate lookup.

>Change-Id: I8d07d2ebb5a0da6c3ea478317442cb42f1797a4b
>BUG: 1500269
>Signed-off-by: Raghavendra G 

(cherry picked from commit fb4b914ce84bc83a5f418719c5ba7c25689a9251)
Change-Id: I8d07d2ebb5a0da6c3ea478317442cb42f1797a4b
BUG: 1529088
Signed-off-by: Raghavendra G

performance/write-behind: fix bug while handling short writes

2018-01-02T09:51:52+00:00

The variabled "fulfilled" in wb_fulfill_short_write is not reset to 0
while handling every member of the list.

This has some interesting consequences:

* If we break from the loop while processing last member of the list
  head->winds, req is reset to head as the list is a circular
  one. However, head is already fulfilled and can potentially be
  freed. So, we end up adding a freed request to wb_inode->todo
  list. This is the RCA for the crash tracked by the bug associated
  with this patch (Note that we saw "holder" which is freed in todo
  list).

* If we break from the loop while processing any of the last but one
  member of the list head->winds, req is set to next member in the
  list, skipping the current request, even though it is not entirely
  synced. This can lead to data corruption.

The fix is very simple and we've to change the code to make sure
"fulfilled" reflects whether the current request is fulfilled or not
and it doesn't carry history of previous requests in the list.

>Change-Id: Ia3d6988175a51c9e08efdb521a7b7938b01f93c8
>BUG: 1528558
>Signed-off-by: Raghavendra G 

(cherry picked from commit 0bc22bef7f3c24663aadfb3548b348aa121e3047)
Change-Id: Ia3d6988175a51c9e08efdb521a7b7938b01f93c8
BUG: 1529095
Signed-off-by: Raghavendra G

mount/fuse: use fstat in getattr implementation if any opened fd is available

2018-01-02T08:26:42+00:00

The restriction of using fds opened by the same Pid means fds cannot
be shared across threads of multithreaded application. Note that fops
from kernel have different Pid for different threads. Imagine
following sequence of operations:

* Turn off performance.open-behind
* Thread t1 opens an fd - fd1 - on file "file". Let's assume nodeid of
  "file" is "nodeid-file".
* Thread t2 does RENAME ("newfile", "file"). Let's assume nodeid of
  "newfile" as "nodeid-newfile".
* t2 proceeds to do fstat (fd1)

The above set of operations can sometimes result in ESTALE/ENOENT
errors. RENAME overwrites "file" with "newfile" changing its nodeid
from "nodeid-file" to "nodeid-newfile" and post RENAME, "nodeid-file" is
removed from the backend. If fstat carries nodeid-file as argument,
which can happen if lookup has not refreshed the nodeid of "file" and
since t2 doesn't have an fd opened, fuse_getattr_resume uses STAT
which will fail as "nodeid-file" no longer exists.

Since the above set of operations and sharing of fds across
multiple threads are valid, this is a bug.

The fix is to use any fd opened on the inode. In this specific example
fuse_getattr_resume will find fd1 and winds down the call as fstat
(fd1) which won't fail.

Cross-checked with "Miklos Szeredi"  for
any security issues with this solution and he approves the solution.

Thanks to "Miklos Szeredi"  for all the
pointers and discussions.

>Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c
>BUG: 1510401
>Signed-off-by: Raghavendra G 
(cherry picked from commit 8b57378e5596f287a7b9d106dd6fb56a624b42ee)
Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c
BUG: 1529085
Signed-off-by: Raghavendra G

feature/bitrot: remove internal xattrs from lookup cbk

2017-12-19T04:39:05+00:00

Problem:
afr requests all xattrs in lookup via the list-xattr key. If bitrot is
enabled and later disabled, or if the bitrot xattrs were present due to
an older version of bitrot which used to create the xattrs without
enabling the feature, the xattrs (trusted.bit-rot.version in particular)
was not getting filtered and ended up reaching the client stack. AFR, on
noticing different values of the xattr across bricks of the replica,
started triggering spurious metadata heals.

Fix:
Filter all internal xattrs in bitrot xlator before unwinding lookup,
(f)getxattr.

Thanks to Kotresh for the help in RCA'ing.

Change-Id: I5bc70e4b901359c3daefc67b8e4fa6ddb47f046c
BUG: 1527276
Signed-off-by: Ravishankar N 
(cherry picked from commit d341f20230b9921391aff22337eaf9be82f44d88)

glusterd: Free up svc->conn on volume delete

2017-12-12T10:00:52+00:00

Daemons like snapd, tierd and gfproxyd are maintained on per volume
basis and on a volume delete we should destroy the rpc connection
established for them.

>mainline patch : https://review.gluster.org/#/c/18957/

Change-Id: Id1440e39da07b990fdb9b207df18da04b1ca8014
BUG: 1523048
Signed-off-by: Atin Mukherjee 
(cherry picked from commit 36ce4c614a3391043a3417aa061d0aa16e60b2d3)