glusterfs.git/xlators/protocol, branch v3.9.0rc2

compound fops: Fix file corruption issue

2016-10-25T04:14:54+00:00

        Backport of: http://review.gluster.org/#/c/15654/

1. Address of a local variable @args is copied into state->req
in server3_3_compound (). But even after the function has gone out of
scope, in server_compound_resume () this pointer is accessed and
dereferenced. This patch fixes that.

2. Compound fops, by virtue of NOT having a vector sizer (like the one
writev has), ends up having both the header and the data (in case one of
its member fops is WRITEV) in the same hdr_iobuf. This buffer was not
being preserved through the lifetime of the compound fop, causing it to
be overwritten by a parallel write fop, even when the writev associated
with the currently executing compound fop is yet to hit the desk, thereby
corrupting the file's data. This is fixed by associating the hdr_iobuf with
the iobref so its memory remains valid through the lifetime of the fop.

3. Also fixed a use-after-free bug in protocol/client in compound fops cbk,
missed by Linux but caught by NetBSD.

Finally, big thanks to Pranith Kumar K and Raghavendra Gowdappa for their
help in debugging this file corruption issue.

Change-Id: I58da39ae544ad81192849926399a971c4c01c986
BUG: 1387984
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15709
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

protocol-server: generate events

2016-09-08T07:59:22+00:00

This patch add/generate the events seen from/in the protocol server
side, which will be consumed by the new eventing framework.

-------------------------------------------------------------------------
|          event                |            description                |
|-------------------------------|---------------------------------------|
| EVENT_CLIENT_CONNECT          |          new client connected         |
|-------------------------------|---------------------------------------|
| EVENT_CLIENT_AUTH_REJECT      |     client cannot be authenticated    |
|-------------------------------|---------------------------------------|
| EVENT_CLIENT_DISCONNECT       |        client had disconnected        |
-------------------------------------------------------------------------

Thanks to "Raghavendra Talur"  for all the help

Backport of:
> Change-Id: I4fda83fae7747507f64d81a7645cc3a8a2fbaeae
> BUG: 1369403
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/15294
> Tested-by: Prasanna Kumar Kalever 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: I4fda83fae7747507f64d81a7645cc3a8a2fbaeae
BUG: 1373740
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/15415
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

compound fops: Some fixes to compound fops framework

2016-08-31T06:50:40+00:00

Change-Id: I808fd5f9f002a35bff94d310c5d61a781e49570b
BUG: 1360169
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/15010
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

protocol/server: fix unused variable warnings/errors

2016-08-27T11:17:49+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: I5a48055c4b0a4a07dd511ad4f2e9ef823b9c60b2
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15255
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

protocol/server: readlink rsp xdr will fail while readlink got an error

2016-08-25T02:06:25+00:00

set gfs3_readlink_rsp.path with an empty string while error happen, to
make xdr_gfs3_readlink_rsp happy. otherwise the original errno will be
lost, and return an rpc internal errno instead.

Change-Id: I36655b66df8b9f164e5bd21eb17244722c2f5a52
BUG: 1369530
Signed-off-by: Ryan Ding 
Reviewed-on: http://review.gluster.org/15299
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

protocol/client: Unserialize xdata even if lookup fails

2016-08-17T10:26:05+00:00

Problem:
AFR relies on xdata returned by lookup to determine if
there are any files that need healing. This info is further
used to optimize readdirp.

In case of lookups with negative return value, client
xlator was sending NULL xdata. Due to absence of xdata,
AFR conservatively assumes that there are files that
need healing, which is incorrect.

Solution:
Even in case of unsuccessful lookups, send the xdata received
by protocol client so that higher xlators can get the info
that they rely on.

Change-Id: Id3a1023eb536180888eb2c0b39050000b76f7226
BUG: 1366284
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/15120
Smoke: Gluster Build System 
Reviewed-by: Poornima G 
Tested-by: Poornima G 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ashish Pandey 
Reviewed-by: Pranith Kumar Karampuri

protocol/server: Fix client/server compatibility

2016-06-28T20:27:23+00:00

The 3.8 client expects a child_up key from the server
indicating the status of the server translators. This
key is not being sent by the servers running older
versions, thereby breaking compatibility.

With this patch we are treating the absence of the said
key as an indication that the server trying to connect
to this client is running an older version and hence
in such a case we are setting conf->child_up as
_gf_true explicitly. This should suffice in emulating
the older behavior.

Due to the nature of this bug, requiring two version to
be reproducible, there are no testcases added for the same.

Change-Id: I29e0a5c63b55380dc9db8e42852d7e95b64a2b2e
BUG: 1350327
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/14811
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Tested-by: Gluster Build System

protocol: Add framework to send transaction id with recall

2016-06-10T11:03:40+00:00

Issue:
The upcall(cache invalidation/recall) event is sent from the bricks
to clients. In AFR/EC setup, it can so happen that all the bricks
will send the upcall for the same event, and if AFR/EC doesn't filter
out these duplicate notifications, the logic above cluster xlators
can fail.

Solution:
Use transaction id to filter out duplicate notifications.

This patch adds framework for duplicate notifications.
AFR/EC can build up on this patch for deduping the notifications

Change-Id: I66b08e63b8799bc5932f2b2545376138a5701168
BUG: 1319992
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14647
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Raghavendra G

libglusterfs: Negate all but O_DIRECT flag if present on anon fds

2016-06-10T05:05:46+00:00

This is to prevent any unforeseen problems that might arise due to
writevs and readvs being wound with @flag parameter containing
O_TRUNC or O_APPEND especially wrt translators like sharding and ec
where O_TRUNC write or O_APPEND write on individual shards/fragments
is not the same as O_TRUNC write or O_APPEND write as expected by the
application.

Change-Id: I9e5206a6ce2b1b70df61ff23b1c961cf25bf7ff9
BUG: 1342171
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14665
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Tested-by: Gluster Build System 
NetBSD-regression: NetBSD Build System

core, shard: Make shards inherit main file's O_DIRECT flag if present

2016-06-07T07:19:26+00:00

If the application opens a file with O_DIRECT, the shards'
anon fds would also need to inherit the flag. Towards this,
shard xl would be passing the odirect flag in the @flags parameter
to the WRITEV fop. This will be used in anon fd resolution
and subsequent opening by posix xl.

Change-Id: Iddb75c9ed14ce5a8c5d2128ad09b749f46e3b0c2
BUG: 1342171
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14191
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri