glusterfs.git/tests/basic, branch exp

tests: Move tests/basic/gfapi/bug1291259.t to bad tests list

2016-12-19T09:30:50+00:00

See thread
http://www.gluster.org/pipermail/gluster-devel/2016-December/051714.html
for more information.

Change-Id: I9abe4b0e40499e53c1276a10a6bc192fd0f2cef7
BUG: 1405301
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/16159
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra Talur 
CentOS-regression: Gluster Build System

glfsheal: Explicitly enable self-heal xlator options

2016-12-15T15:46:26+00:00

Enable data, metadata and entry self-heal as xlator-options so that glfs-heal.c
can heal split-brain files even if they are disabled on the volume via volume
set commands.

Change-Id: Ic191a1017131db1ded94d97c932079d7bfd79457
BUG: 1234054
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/11333
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/afr: CLI for granular entry heal enablement/disablement

2016-11-28T11:56:33+00:00

When there are already existing non-granular indices created that are
yet to be healed, if granular-entry-heal option is toggled from 'off' to
'on', AFR self-heal whenever it kicks in, will try to look for granular
indices in 'entry-changes'. Because of the absence of name indices,
granular entry healing logic will fail to heal these directories, and
worse yet unset pending extended attributes with the assumption that
are no entries that need heal.

To get around this, a new CLI is introduced which will invoke glfsheal
program to figure whether at the time an attempt is made to enable
granular entry heal, there are pending heals on the volume OR there
are one or more bricks that are down. If either of them is true, the
command will be failed with the appropriate error.

New CLI: gluster volume heal  granular-entry-heal {enable,disable}

Change-Id: I1f4fe8162813b9068e198965d94169fee4adc099
BUG: 1370410
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15747
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

cluster/afr: Fix bugs in [f]inodelk/[f]entrylk

2016-11-26T15:34:59+00:00

Problems:
1) Inodelk is not taking quorum into account
2) finodelk, [f]entrylk are not implemented correctly
3) By default afr doesn't go for non-blocking parallel locks.

Fix:
Implemented a common framework which can be used by
[f]inodelk/[f]entrylk.  Used quorum for the same.

Change-Id: I239f13875a065298630d266941df10cfa3addc85
BUG: 1369077
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15802
Tested-by: Krutika Dhananjay 
Reviewed-by: Krutika Dhananjay 
Smoke: Gluster Build System 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

features/index: Delete granular entry indices of already healed directories during crawl

2016-11-25T03:03:14+00:00

If granular name indices are already in existence for a volume, and
before they are healed, granular entry heal be disabled, a crawl on
indices/xattrop will clear the changelogs on these directories. When
their corresponding entry-changes indices are crawled subsequently,
if it is found that the directories don't need heal anymore, the
granular indices are not cleaned up.
This patch fixes that problem by ensuring that the zero-xattrop
also deletes the stale indices at the level of index translator.

Change-Id: Ifbaa6bec2a14e3041addfee4054131babbf4d35e
BUG: 1370410
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15880
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System

jbr: Sending rollback from failed fop to fdl

2016-11-08T19:25:25+00:00

In case of a failed fop, the failure is detected
by the leader in the jbr-server in two places. First
during a quorum check of +ve responses when it
receives responses from all the followers. At this
point if the fop hasn't been successfully journaled
at a quorum of followers (as in there is no merit in
trying the fop in the leader as the quorum will never
be met), then we fail the fop.

Also if this quorum is met, then the fop is tried on
the leader, and after the leader completes the fop
a quorum check similar to the previous one is done
again, this time including the leaders outcome. If
quorum is not met, then we fail the fop.

In both these cases, when the fop fails we send a -ve
ack to the client. With this patch, now we will also
send a rollback through a GF_FOP_IPC to all the followers(and
also to the leader in the second case of failure). This
rollback will contain the index and term number of the
fop which failed. This will be recorded in the respective
journals of the bricks and will be used to rollback the
fop on that brick later.

A subsequent write, and it's respective rollback would
look something like the following in the journal.

The trusted.jbr.term and trusted.jbr.index present in the
dict of both the logs, relate them, and the presence of
"rollback-fop" in the dict of IPC indicates that it is a
rollback fop, and the value 13(stands for GF_FOP_WRITE)
indicates what kind of rollback operation it is.

=== GF_FOP_WRITE
fd = 
vector = <158 bytes>
offset = 0 (0x0)
flags = 32769 (0x8001)
xdata = dict {
 trusted.jbr.term = 0 <2 bytes>
 trusted.jbr.index = 4 <2 bytes>
}
=== GF_FOP_IPC
xdata = dict {
 trusted.jbr.term = 0 <2 bytes>
 trusted.jbr.index = 4 <2 bytes>
 rollback-fop = 13 <3 bytes>
}

Change-Id: I70b6a143d20697153d58e2f719e34ecd1ed160a5
BUG: 1349385
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/14783
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy 
Smoke: Gluster Build System

gfapi: async fops should unref in callbacks

2016-11-05T20:32:58+00:00

If fd is unref'd at the end of async call then the unref in cbks would
lead to double unref and possible crash. Removing duplicate unrefs.

Added unref only in failure cases.

A simple test case has been added to test async write case. Need to
extend the same for other async APIs too.

Details:
All glfd based calls in libgfapi, except for glfs_open and glfs_close,
behave in the same way. At the start of the operation, they take a ref
on glfd and fd. At the end of the operation, they unref it. Async calls
are a little different as they unref in the cbk function. A successfull
open call does not unref either the glfd or fd, thereby functioning as a
reference for a OPEN file object. glfs_close makes a syncop_flush call
sandwiched between a fd ref and unref(this can be removed, more on this
below), followed by a call to glfs_mark_glfd_for_deletion which unrefs
glfd and also calls glfs_fd_destroy as a release function thereby doing
a unref on fd too.

Functionally, there is no problem with how everything works when as
described above. However, it is a little non-intuitive that we need to
perform a fd_unref as a consequence of a implicit fd_ref that happens
within glfs_resolve_fd. As we perform a GF_REF_GET(glfd) at the start of
every operation, it would be worthwhile to remove the fd_ref that
glfs_resovle_fd takes and do away with explicit fd_unref()s at the end
of every operation. This is the same reason why we don't need the fd_ref
in glfs_close. This is however not in the scope of this patch.

Change-Id: I86b1d3b2ad846b16ea527d541dc82b5e90b0ba85
BUG: 1391086
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/15768
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Xavier Hernandez 
Reviewed-by: soumya k 
Reviewed-by: Prasanna Kumar Kalever 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

snapshot: Fix the failure to recreate clones with same name

2016-11-02T06:28:26+00:00

The brick path of snapshot clones contained the clonename,
thereby failing to create newer clones with the same name
after the original clone had been deleted.

This fix creates the brick path with the clone's vol id
instead of the clones name. Hence future clones with the
same name will not have the namespace clash.

Change-Id: I262712adc576122f051b5d1ce171d020efaefd1a
BUG: 1387160
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/15683
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Rajesh Joseph

afr,ec: Heal device files with correct major, minor numbers

2016-10-26T12:22:50+00:00

Thanks a lot to xiaoping.wu@nokia.com from Nokia for the bug and the
fix.

BUG: 1384297
Change-Id: Ie443237e85d34633b5dd30f85eaa2ac34e45754c
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15728
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Xavier Hernandez 
CentOS-regression: Gluster Build System

compound fops: Fix file corruption issue

2016-10-24T14:11:08+00:00

1. Address of a local variable @args is copied into state->req
in server3_3_compound (). But even after the function has gone out of
scope, in server_compound_resume () this pointer is accessed and
dereferenced. This patch fixes that.

2. Compound fops, by virtue of NOT having a vector sizer (like the one
writev has), ends up having both the header and the data (in case one of
its member fops is WRITEV) in the same hdr_iobuf. This buffer was not
being preserved through the lifetime of the compound fop, causing it to
be overwritten by a parallel write fop, even when the writev associated
with the currently executing compound fop is yet to hit the desk, thereby
corrupting the file's data. This is fixed by associating the hdr_iobuf with
the iobref so its memory remains valid through the lifetime of the fop.

3. Also fixed a use-after-free bug in protocol/client in compound fops cbk,
missed by Linux but caught by NetBSD.

Finally, big thanks to Pranith Kumar K and Raghavendra Gowdappa for their
help in debugging this file corruption issue.

Change-Id: I6d5c04f400ecb687c9403a17a12683a96c2bf122
BUG: 1378778
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15654
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System