glusterfs.git/tests/basic/afr, branch v3.9.0rc2

compound fops: Fix file corruption issue

2016-10-25T04:14:54+00:00

        Backport of: http://review.gluster.org/#/c/15654/

1. Address of a local variable @args is copied into state->req
in server3_3_compound (). But even after the function has gone out of
scope, in server_compound_resume () this pointer is accessed and
dereferenced. This patch fixes that.

2. Compound fops, by virtue of NOT having a vector sizer (like the one
writev has), ends up having both the header and the data (in case one of
its member fops is WRITEV) in the same hdr_iobuf. This buffer was not
being preserved through the lifetime of the compound fop, causing it to
be overwritten by a parallel write fop, even when the writev associated
with the currently executing compound fop is yet to hit the desk, thereby
corrupting the file's data. This is fixed by associating the hdr_iobuf with
the iobref so its memory remains valid through the lifetime of the fop.

3. Also fixed a use-after-free bug in protocol/client in compound fops cbk,
missed by Linux but caught by NetBSD.

Finally, big thanks to Pranith Kumar K and Raghavendra Gowdappa for their
help in debugging this file corruption issue.

Change-Id: I58da39ae544ad81192849926399a971c4c01c986
BUG: 1387984
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15709
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

tests: change EXPECT_WITHIN timeouts

2016-08-29T06:09:42+00:00

Use defined HEAL and PROCESS_UP timeouts rather than
hard code them in self-heald.t.

Change-Id: I21586811904c8417b7208bb643f14dff20dc4832
BUG: 1370074
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/15316
Reviewed-by: Ravishankar N 
Tested-by: Ravishankar N 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Krutika Dhananjay 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

afr: some coverity fixes

2016-07-26T17:06:52+00:00

Thanks to Krutika for a cleaner way to track inode refs in
afr_set_split_brain_choice().

Change-Id: I2d968d05b815ad764b7e3f8aa9ad95a792b3c1df
BUG: 1355604
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/14895
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

tests: Fix spurious failures with split-brain-favorite-child-policy.t

2016-07-22T21:32:41+00:00

Problem:
It is not guranteed that the self-heal daemon would apply the new option
as soon as volume set is executed because all the command gurantees is that
the process is notified of the change in volfile. Shd still needs to fetch
volfile and reconfigure. If the next volume heal command comes even before
the reconfigure happens, then the heal won't happen.

Fix:
Restart shd to make sure it has the option loaded with new value.

BUG: 1358976
Change-Id: I3ed30ebbec17bd06caa632e79e9412564f431b19
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14978
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
Tested-by: Jeff Darcy 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

afr, index: Clean up stale directory and file indices in granular entry sh

2016-07-12T06:13:53+00:00

Specifically when a directory tree is removed (rm -rf)
while a brick is down, both the directory index and the
name indices of the files and subdirs under it will remain.
Self-heal will need to pick up these and remove them.

Towards this, afr sh will now also crawl indices/entry-changes
and call an rmdir on the dir if the directory index is stale.

On the brick side, rmdir fop has been implemented for index xl,
which would delete the directory index and its contents if present
in a synctask.

Change-Id: I8b527331c2547e6c141db6c57c14055ad1198a7e
BUG: 1331323
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14832
Reviewed-by: Ravishankar N 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System

features/index: Delete parent dir indices when heal on it is complete

2016-06-28T08:13:57+00:00

In this patch, the state information about whether a directory
gfid index is present or not is stored in the inode ctx with
values IN and NOTIN. This saves index xl the need to perform
stat() everytime an index_entry_create() is called.

When a brick is restarted these in-memory inode ctx records will
be gone. So when granular entry heal happens after a brick is restarted,
and a post-op is done on the parent, if the state gotten from inode ctx
is UNKNOWN, then index xl does a stat to initialize the state as IN or
NOTIN. Note that this is a one-time operation for the lifetime of the
brick. Such a change also helps avoid calling index_del() in
xattrop_index_action() periodically even when granular self-heal is
disabled or when the volume type is disperse.

Change-Id: Ib92c17350e6531b91ab81477410fe1e7a5856207
BUG: 1331323
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14781
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

cli: fix crash in arbiter keyword parsing

2016-06-17T10:56:37+00:00

A negative case like `gluster volume create volname arbiter 3 /bricks{1..3}`
must not crash. 'arbiter' keyword is valid only for (3 way) replica volumes.

The .t that is added will crash and create a core *without* the fix when
run but will still pass all TESTs. Since the regression framework fails
the .t if it creates a core,  we can consider it a valid test 'that
fails without the fix'.

Change-Id: Ie2d7ced66025ea3617d30f6f823b22401e6d2fde
BUG: 1346821
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/14738
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

tests: Add a test for conservative merge with granular esh

2016-06-09T08:28:49+00:00

Change-Id: Ib8ada912451bf80b640d25e6e3a824216d055b5b
BUG: 1332566
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14601
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

tests: Add more tests for granular entry self-heal feature

2016-05-31T04:23:55+00:00

Change-Id: I6f14e413c538e392c8ee5bf4bf9f283e8ac792b7
BUG: 1332566
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14542
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

afr: Automagic unsplit-brain by [ctime|mtime|size|majority]

2016-05-25T18:55:08+00:00

Introduce cluster.favorite-child-policy which when enabled with
[ctime|mtime|size|majority], automatically heals files that are in
split-brian.

The majority policy will not pick a source if there is no majority.
The other three policies pick the first brick with a valid reply and
non-zero ctime/mtime/size as source.

Change-Id: I3c099a0404082213860f74f2c9b4d207cfaedb76
BUG: 1328224
Original-author: Richard Wareing 
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/14026
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Anuradha Talur 
Reviewed-by: Jeff Darcy