glusterfs.git/tests/basic, branch v3.9.1

tests: Fix split-brain-favorite-child-policy.t failures

2017-01-05T11:42:04+00:00

Problem:
In CentOS-7, the file was receving an extra removexattr(security.ima)
FOP which changed its ctime, breaking the assumption that a particular brick
had the latest ctime based on the writevs done in the .t

Fix:
1. Compare the ctime of both files in the backend and pick the one with
the latest ctime for the fav-child policy. Also unmount the volume
before comparing, to avoid any further FOPS on the file that
can possibly modify the timestamps.

2. Added floating point handling in stat function. Thanks to Pranith for
the helping debugging the regex.

> Reviewed-on: http://review.gluster.org/16288
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit 76fff8cb2a164b596ca67e65c99623f5b68361fd)


Change-Id: I06041a0f39a29d2593b867af8685d65c7cd99150
BUG: 1410072
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16323
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

tests: Move tests/basic/gfapi/bug1291259.t to bad tests list

2016-12-19T09:33:10+00:00

See thread
http://www.gluster.org/pipermail/gluster-devel/2016-December/051714.html
for more information.

Change-Id: I9abe4b0e40499e53c1276a10a6bc192fd0f2cef7
BUG: 1405305
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/16160
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glfsheal: Explicitly enable self-heal xlator options

2016-12-16T01:34:04+00:00

Enable data, metadata and entry self-heal as xlator-options so that glfs-heal.c
can heal split-brain files even if they are disabled on the volume via volume
set commands.

> Reviewed-on: http://review.gluster.org/11333
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> Tested-by: Pranith Kumar Karampuri 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 209c2d447be874047cb98d86492b03fa807d1832)

Change-Id: Ic191a1017131db1ded94d97c932079d7bfd79457
BUG: 1405126
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16143
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/afr: Fix bugs in [f]inodelk/[f]entrylk

2016-12-08T06:56:55+00:00

Problems:
1) Inodelk is not taking quorum into account
2) finodelk, [f]entrylk are not implemented correctly
3) By default afr doesn't go for non-blocking parallel locks.

Fix:
Implemented a common framework which can be used by
[f]inodelk/[f]entrylk.  Used quorum for the same.

 >Change-Id: I239f13875a065298630d266941df10cfa3addc85
 >BUG: 1369077
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/15802
 >Tested-by: Krutika Dhananjay 
 >Reviewed-by: Krutika Dhananjay 
 >Smoke: Gluster Build System 
 >Reviewed-by: Ravishankar N 
 >CentOS-regression: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 

BUG: 1402482
Change-Id: I0c5fed6ca87c6432bb20d00f76cdf5c328a52a85
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/16056
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N

cluster/afr: CLI for granular entry heal enablement/disablement

2016-11-29T10:52:15+00:00

        Backport of: http://review.gluster.org/15747

When there are already existing non-granular indices created that are
yet to be healed, if granular-entry-heal option is toggled from 'off' to
'on', AFR self-heal whenever it kicks in, will try to look for granular
indices in 'entry-changes'. Because of the absence of name indices,
granular entry healing logic will fail to heal these directories, and
worse yet unset pending extended attributes with the assumption that
are no entries that need heal.

To get around this, a new CLI is introduced which will invoke glfsheal
program to figure whether at the time an attempt is made to enable
granular entry heal, there are pending heals on the volume OR there
are one or more bricks that are down. If either of them is true, the
command will be failed with the appropriate error.

New CLI: gluster volume heal  granular-entry-heal {enable,disable}

Change-Id: Ic79519468a087cd337df664b968188c4adcba43a
BUG: 1398500
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15941
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

features/index: Delete granular entry indices of already healed directories during crawl

2016-11-26T11:10:52+00:00

        Backport of: http://review.gluster.org/15880

If granular name indices are already in existence for a volume, and
before they are healed, granular entry heal be disabled, a crawl on
indices/xattrop will clear the changelogs on these directories. When
their corresponding entry-changes indices are crawled subsequently,
if it is found that the directories don't need heal anymore, the
granular indices are not cleaned up.
This patch fixes that problem by ensuring that the zero-xattrop
also deletes the stale indices at the level of index translator.

Change-Id: If4a2f14e33a78f2217e9fea8733ebb552af56059
BUG: 1398500
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15926
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

Remove experimental translators

2016-11-14T05:27:04+00:00

Change-Id: Id645918fa236f1fc00ab5fa427f394e853c44bf8
BUG: 1389675
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/15750
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

gfapi: async fops should unref in callbacks

2016-11-10T08:25:11+00:00

If fd is unref'd at the end of async call then the unref in cbks would
lead to double unref and possible crash. Removing duplicate unrefs.

Added unref only in failure cases.

A simple test case has been added to test async write case. Need to
extend the same for other async APIs too.

Details:
All glfd based calls in libgfapi, except for glfs_open and glfs_close,
behave in the same way. At the start of the operation, they take a ref
on glfd and fd. At the end of the operation, they unref it. Async calls
are a little different as they unref in the cbk function. A successfull
open call does not unref either the glfd or fd, thereby functioning as a
reference for a OPEN file object. glfs_close makes a syncop_flush call
sandwiched between a fd ref and unref(this can be removed, more on this
below), followed by a call to glfs_mark_glfd_for_deletion which unrefs
glfd and also calls glfs_fd_destroy as a release function thereby doing
a unref on fd too.

Functionally, there is no problem with how everything works when as
described above. However, it is a little non-intuitive that we need to
perform a fd_unref as a consequence of a implicit fd_ref that happens
within glfs_resolve_fd. As we perform a GF_REF_GET(glfd) at the start of
every operation, it would be worthwhile to remove the fd_ref that
glfs_resovle_fd takes and do away with explicit fd_unref()s at the end
of every operation. This is the same reason why we don't need the fd_ref
in glfs_close. This is however not in the scope of this patch.

Change-Id: I86b1d3b2ad846b16ea527d541dc82b5e90b0ba85
BUG: 1392286
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/15768
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Xavier Hernandez 
Reviewed-by: soumya k 
Reviewed-by: Prasanna Kumar Kalever 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit e65738818dd22462ec00dda021566654d1c702b1)
Reviewed-on: http://review.gluster.org/15778

afr,ec: Heal device files with correct major, minor numbers

2016-10-27T06:23:26+00:00

Thanks a lot to xiaoping.wu@nokia.com from Nokia for the bug and the
fix.

 >BUG: 1384297
 >Change-Id: Ie443237e85d34633b5dd30f85eaa2ac34e45754c
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/15728
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Xavier Hernandez 
 >CentOS-regression: Gluster Build System 

Change-Id: Ie7977caf7c98c91fca64752c56731c37ad27df4d
BUG: 1388912
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15734
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez

compound fops: Fix file corruption issue

2016-10-25T04:14:54+00:00

        Backport of: http://review.gluster.org/#/c/15654/

1. Address of a local variable @args is copied into state->req
in server3_3_compound (). But even after the function has gone out of
scope, in server_compound_resume () this pointer is accessed and
dereferenced. This patch fixes that.

2. Compound fops, by virtue of NOT having a vector sizer (like the one
writev has), ends up having both the header and the data (in case one of
its member fops is WRITEV) in the same hdr_iobuf. This buffer was not
being preserved through the lifetime of the compound fop, causing it to
be overwritten by a parallel write fop, even when the writev associated
with the currently executing compound fop is yet to hit the desk, thereby
corrupting the file's data. This is fixed by associating the hdr_iobuf with
the iobref so its memory remains valid through the lifetime of the fop.

3. Also fixed a use-after-free bug in protocol/client in compound fops cbk,
missed by Linux but caught by NetBSD.

Finally, big thanks to Pranith Kumar K and Raghavendra Gowdappa for their
help in debugging this file corruption issue.

Change-Id: I58da39ae544ad81192849926399a971c4c01c986
BUG: 1387984
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15709
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System