glusterfs.git/xlators/protocol/client/src/client.c, branch v3.7.16

protocol/client: Filter o-direct in readv/writev

2016-07-30T01:37:53+00:00

 >Change-Id: I519c666b3a7c0db46d47e08a6a7e2dbecc05edf2
 >BUG: 1322214
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/14215
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Krutika Dhananjay 
 >(cherry picked from commit 74837896c38bafdd862f164d147b75fcbb619e8f)

BUG: 1360785
Pranith Kumar K 

Change-Id: Ib4013b10598b0b988b9f9f163296b6afa425f8fd
Reviewed-on: http://review.gluster.org/15050
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

protocol/client: prevent use-after-free of frame->root

2015-12-03T00:03:02+00:00

A regression failure generated a coredump on the glusterfs-client side:

  (gdb) f 0
  #0  0x00007fba6cd76432 in client_submit_request (this=0x7fba68006fc0,
                  req=0x7fba6579aa70, frame=0x7fba5c0058cc,
                  prog=0x7fba6cfb53c0 , procnum=41,
                  cbkfn=0x7fba6cd9206d ,
                  iobref=0x0, rsphdr=0x0, rsphdr_count=0,
                  rsp_payload=0x0, rsp_payload_count=0, rsp_iobref=0x0,
                  xdrproc=0x7fba79801075 ) at
  /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/protocol/client/src/client.c:324
  324                   frame->root->ngrps = ngroups;
  (gdb) l
  319                   gf_msg_debug (this->name, 0, "rpc_clnt_submit failed");
  320           }
  321
  322           if (!conf->send_gids) {
  323                   /* restore previous values */
  324                   frame->root->ngrps = ngroups;
  325                   if (ngroups <= SMALL_GROUP_COUNT)
  326                           frame->root->groups_small[0] = gid;
  327           }
  328
  (gdb) p *frame->root
  Cannot access memory at address 0x64185df000000000

After looking at this in more detail, the flow is like this:

  client_submit_request()
    |
    '- rpc_clnt_submit() // on line 314
         |
         '- cbkfn() // = client3_3_release_cbk
              |
              :- STACK_DESTROY (frame->root);
         .----'
    .----'
    |
    :- frame->root->ngrps = ngroups; // on line 324
    '

So, there is a use-after-free, and it is not needed to restore the
previous groups in frame->root.

Cherry picked from commit dc3aa7524e4974f9d02465e2e5dd6ed9b6d319e1:
> Change-Id: I9e7d712183692ed92cfc2f75cd3c2781a9db20e2
> BUG: 1281285 (was incorrect in original patch)
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/12575
> Reviewed-by: Dan Lambright 
> Tested-by: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

Change-Id: I9e7d712183692ed92cfc2f75cd3c2781a9db20e2
BUG: 1283138
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/12639
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Vijay Bellur

rpc: add owner xlator argument to rpc_clnt_new

2015-08-14T10:10:47+00:00

The @owner argument tells RPC layer the xlator that owns
the connection and to which xlator THIS needs be set during
network notifications like CONNECT and DISCONNECT.

Code paths that originate from the head of a (volume) graph and use
STACK_WIND ensure that the RPC local endpoint has the right xlator saved
in the frame of the call (callback pair). This guarantees that the
callback is executed in the right xlator context.

The client handshake process which includes fetching of brick ports from
glusterd, setting lk-version on the brick for the session, don't have
the correct xlator set in their frames. The problem lies with RPC
notifications. It doesn't have the provision to set THIS with the xlator
that is registered with the corresponding RPC programs. e.g,
RPC_CLNT_CONNECT event received by protocol/client doesn't have THIS set
to its xlator. This implies, call(-callbacks) originating from this
thread don't have the right xlator set too.

The fix would be to save the xlator registered with the RPC connection
during rpc_clnt_new. e.g, protocol/client's xlator would be saved with
the RPC connection that it 'owns'. RPC notifications such as CONNECT,
DISCONNECT, etc inherit THIS from the RPC connection's xlator.

Change-Id: I9dea2c35378c511d800ef58f7fa2ea5552f2c409
BUG: 1253212
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/11436
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G 
(cherry picked from commit f7668938cd7745d024f3d2884e04cd744d0a69ab)
Reviewed-on: http://review.gluster.org/11908
Reviewed-by: Pranith Kumar Karampuri

client, rpc: make ping-timeout configurable for glusterfs clients

2015-08-13T05:05:57+00:00

Change-Id: Idd94adb0457aaffce7330f56f98cebafa2c4dae8
BUG: 1250810
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/11818
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: NetBSD Build System 
(cherry picked from commit 3403370ebeaf16567b79022c6ac48b2e0cd50db5)
Reviewed-on: http://review.gluster.org/11848

protocol/client : porting log messages to new framework

2015-06-17T12:11:43+00:00

        Backport of http://review.gluster.org/#/c/10042/

Cherry picked from 379dbbfd683d2b0e1704c098b1f020567328122c
> Change-Id: I9bf2ca08fef969e566a64475d0f7a16d37e66eeb
> BUG: 1194640
> Signed-off-by: Manikandan Selvaganesh 
> Reviewed-on: http://review.gluster.org/10042
> Tested-by: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Tested-by: Raghavendra G 

Change-Id: I9bf2ca08fef969e566a64475d0f7a16d37e66eeb
BUG: 1217722
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/11240
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

protocol/client : logical dead code removed

2015-03-30T06:55:48+00:00

    CID :112438{2,3,4,5}
         11244{03,04,05,06,07,08,09,10,11,12,13,14,30,31,32,33}
         11244{34,35,36,37,38,39,40,41,42,43,44,56,57,58,59,60}
         11244{61,62,63,64,65,66,67,68,69,70}
         1128902

Change-Id: I1b53aeee83209728d5711540210d19314f4165bc
BUG: 789278
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/9601
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/dht: Change the subvolume encoding in d_off to be a "global"

2015-03-18T11:47:41+00:00

position in the graph rather than relative (local) to a particular
translator.

Encoding the volume in this way allows a single translator to manage
which brick is currently being scanned for directory entries. Using a
single translator minimizes allocated bits in the d_off. It also allows
multiple DHT translators in the same graph to have a common frame of
reference (the graph position) for which brick is being read. Multiple
DHT translators are needed for the Tiering feature.

The fix builds off a previous change (9332) which removed subvolume
encoding from AFR. The fix makes an equivalent change to the EC
translator.

More background can be found in fix 9332 and gluster-dev discussions [1].

DHT and AFR/EC are responsibile (as before) for choosing which brick to
enumerate directory entries in over the readdir lifecycle.

The client translator receiving the readdir fop encodes the dht_t. It
is referred to as the "leaf node" in the graph and corresponds to the
brick being scanned.

When DHT decodes the d_off, it translates the leaf node to a local
subvolume, which represents the next node in the graph leading to
the brick.

Tracking of leaf nodes is done in common utility functions. Leaf nodes
counts and positional information are updated on a graph switch.

[1] www.gluster.org/pipermail/gluster-devel/2015-January/043592.html

Change-Id: Iaf0ea86d7046b1ceadbad69d88707b243077ebc8
BUG: 1190734
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/9688
Reviewed-by: Xavier Hernandez 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

every/where: add GF_FOP_IPC for inter-translator communication

2015-03-17T14:02:15+00:00

Several features - e.g. encryption, erasure codes, or NSR - involve
multiple cooperating translators which sometimes need a "private" means
of communication amongst themselves.  Historically we've used virtual or
synthetic xattrs, but that's not very elegant and clutters up the
getxattr/setxattr path which must also handle real xattr requests.  This
new fop should address that.

The only argument is an int32_t "op" which should be recognized by the
target translator.  It is recommended that translators using these
feature follow some convention regarding the ops that they define, to
avoid conflicts.  Using a hash of the target translator's type string as
a base for a series of ops would probably be a good start.  Any other
information can be passed in both directions using xdata.

The default behavior for this fop, as with any other, is to pass through
to FIRST_CHILD.  That makes use of this fop "transparent" to other
translators that were written before it existed, but it also means that
it only really works with pass-through translators.  If a routing
translator (such as DHT) or a fan-out translator (such as AFR) is
involved, the IPC might not reach its intended destination unless those
translators are modified to forward IPC fops along all paths.

If an IPC gets all the way to storage/posix it is considered an error,
much like an uncaught exception.  We don't actually *do* anything in
that case, but we do log it send back an EOPNOTSUPP error.  This makes
the "unrecognized opcode" condition distinguishable from the "no IPC
support" condition (which would yield an RPC error instead) so clients
can probe for the presence of a handler for their own favorite opcode
and either use that or use old-school xattrs depending on the result.

BUG: 1158628
Signed-off-by: Venky Shankar 
Signed-off-by: Jeff Darcy 
Change-Id: I84af1b17babe5b30ec03ecf027ae37d09b873968
Reviewed-on: http://review.gluster.org/8812
Reviewed-by: Vijay Bellur

protocol: Fix typos in option description

2015-03-06T15:08:16+00:00

Change-Id: I7cf559fa5ffe3f6c437169820a86a7ee2f58b478
BUG: 1199382
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9816
Tested-by: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur

protocol/client: defer cleanup of private until RPC notifications are handled.

2015-03-02T09:16:51+00:00

This fix is required for glfs_fini to be able to perform fini on client
xlators in a graph. We are deferring freeing of client xlator's private
until all RPC related resources are destroyed. This guarantees that
client xlator would free RPC related resources provided its private
structures are still accessible via its this pointer.

'Weak' property: If there are no epoll threads executing after calling
fini() on a client xlator, then all its RPC related resources are
guaranteed to be freed. We can now free the corresponding 'this'
pointer.

Change-Id: Ie00b14dda096ac128e1c37e0032f07d17fd701ce
BUG: 1093594
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/9680
Reviewed-by: Rajesh Joseph 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur