glusterfs.git/rpc/rpc-lib/src/rpcsvc.h, branch v3.10.5

rpcsvc: Add rpchdr and proghdr to iobref before submitting to transport

2017-02-16T14:46:53+00:00

Backport of https://review.gluster.org/16613

Issue:
When fio is run on multiple clients (each client writes to its own files),
and meanwhile the clients does a readdirp, thus the client which did
a readdirp will now recieve the upcalls. In this scenario the client
disconnects with rpc decode failed error.

RCA:
Upcall calls rpcsvc_request_submit to submit the request to socket:
rpcsvc_request_submit currently:
rpcsvc_request_submit () {
   iobuf = iobuf_new
   iov = iobuf->ptr
   fill iobuf to contain xdrised upcall content - proghdr
   rpcsvc_callback_submit (..iov..)
   ...
   if (iobuf)
       iobuf_unref (iobuf)
}

rpcsvc_callback_submit (... iov...) {
   ...
   iobuf = iobuf_new
   iov1 = iobuf->ptr
   fill iobuf to contain xdrised rpc header - rpchdr
   msg.rpchdr = iov1
   msg.proghdr = iov
   ...
   rpc_transport_submit_request (msg)
   ...
   if (iobuf)
       iobuf_unref (iobuf)
}

rpcsvc_callback_submit assumes that once rpc_transport_submit_request()
returns the msg is written on to socket and thus the buffers(rpchdr, proghdr)
can be freed, which is not the case. In especially high workload,
rpc_transport_submit_request() may not be able to write to socket immediately
and hence adds it to its own queue and returns as successful. Thus, we have
use after free, for rpchdr and proghdr. Hence the clients gets garbage rpchdr
and proghdr and thus fails to decode the rpc, resulting in disconnect.

To prevent this, we need to add the rpchdr and proghdr to a iobref and send
it in msg:
   iobref_add (iobref, iobufs)
   msg.iobref = iobref;
The socket layer takes a ref on msg.iobref, if it cannot write to socket and
is adding to the queue. Thus we do not have use after free.

Thank You for discussing, debugging and fixing along:
Prashanth Pai 
Raghavendra G 
Rajesh Joseph 
Kotresh HR 
Mohammed Rafi KC 
Soumya Koduri 

> Reviewed-on: https://review.gluster.org/16613
> Reviewed-by: Prashanth Pai 
> Smoke: Gluster Build System 
> Reviewed-by: soumya k 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Ifa6bf6f4879141f42b46830a37c1574b21b37275
BUG: 1422363
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16623
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

compound fops: Fix file corruption issue

2016-10-24T14:11:08+00:00

1. Address of a local variable @args is copied into state->req
in server3_3_compound (). But even after the function has gone out of
scope, in server_compound_resume () this pointer is accessed and
dereferenced. This patch fixes that.

2. Compound fops, by virtue of NOT having a vector sizer (like the one
writev has), ends up having both the header and the data (in case one of
its member fops is WRITEV) in the same hdr_iobuf. This buffer was not
being preserved through the lifetime of the compound fop, causing it to
be overwritten by a parallel write fop, even when the writev associated
with the currently executing compound fop is yet to hit the desk, thereby
corrupting the file's data. This is fixed by associating the hdr_iobuf with
the iobref so its memory remains valid through the lifetime of the fop.

3. Also fixed a use-after-free bug in protocol/client in compound fops cbk,
missed by Linux but caught by NetBSD.

Finally, big thanks to Pranith Kumar K and Raghavendra Gowdappa for their
help in debugging this file corruption issue.

Change-Id: I6d5c04f400ecb687c9403a17a12683a96c2bf122
BUG: 1378778
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15654
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

build: do not #include "config.h" in each file

2015-05-29T12:43:17+00:00

Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).

When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.

BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Pranith Kumar Karampuri

rpc: Introduce attribute throttle for rpcsvc_t

2015-04-29T04:30:35+00:00

This attribute will be used to set/unset throttling for
a rpcsvc_t program subsequently.

Following APIs have been added to get/set throttle.

int
rpcsvc_set_throttle (rpcsvc_t svc, gf_boolean_t value);

gf_boolean_t
rpcsvc_get_throttle (rpcsvc_t svc);

Change-Id: Ica8a9166cef22eb92d81fe68e48d0a5e24a1ef95
BUG: 1212385
Signed-off-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/10267
Tested-by: NetBSD Build System
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

rpcsvc: New rpc routines defined to send callback requests

2015-03-02T05:30:43+00:00

Change-Id: I7f95682faada16308314bfbf84298b02d1198efa
BUG: 1188184
Signed-off-by: Poornima G 
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/9534
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

gNFS: Subdir mount does not work on UDP proto

2014-10-07T07:51:29+00:00

After enabling nfs.mount-udp, mounting a subdir on a volume over
NFS fails. Because mountudpproc3_mnt_3_svc() invokes nfs3_rootfh()
which internally calls mnt3_mntpath_to_export() to resolve the
mount path. mnt3_mntpath_to_export() just works if the mount path
requested is volume itself. It is not able to resolve, if the path
is a subdir inside the volume.

MOUNT over TCP uses mnt3_find_export() to resolve subdir path but
UDP can't use this routine because mnt3_find_export() needs the
req data (of type rpcsvc_request_t) and it's available only for
TCP version of RPC.

FIX:
(1) Use syncop_lookup() framework to resolve the MOUNT PATH by
    breaking it into components and resolve component-by-component.
    i.e. glfs_resolve_at () API from libgfapi shared object.
(2) If MOUNT PATH is subdir, then make sure subdir export is not
    disabled.
(3) Add auth mechanism to respect nfs.rpc-auth-allow/reject and
    subdir auth i.e. nfs.export-dir
(4) Enhanced error handling for MOUNT over UDP

Change-Id: I42ee69415d064b98af4f49773026562824f684d1
BUG: 1118311
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/8346
Reviewed-by: soumya k 
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

build: make GLUSTERD_WORKDIR rely on localstatedir

2014-08-07T08:17:29+00:00

- Break-way from '/var/lib/glusterd' hard-coded previously,
  instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
  management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
  platforms.
- Define proper environment for running tests, define correct PATH
  and LD_LIBRARY_PATH when running tests, so that the desired version
  of glusterfs is used, regardless where it is installed.
  (Thanks to manu@netbsd.org for this additional work)

Change-Id: I2339a0d9275de5939ccad3e52b535598064a35e7
BUG: 1111774
Signed-off-by: Harshavardhana 
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8246
Tested-by: Gluster Build System

protocol/server: do not do root-squashing for trusted clients

2014-02-11T07:32:05+00:00

* As of now clients mounting within the storage pool using that machine's
  ip/hostname are trusted clients (i.e clients local to the glusterd).

* Be careful when the request itself comes in as nfsnobody (ex: posix tests).
  So move the squashing part to protocol/server when it creates a new frame
  for the request, instead of auth part of rpc layer.

* For nfs servers do root-squashing without checking if it is trusted client,
  as all the nfs servers would be running within the storage pool, hence will
  be trusted clients for the bricks.

* Provide one more option for mounting which actually says root-squash
  should/should not happen. This value is given priority only for the trusted
  clients. For non trusted clients, the volume option takes the priority. But
  for trusted clients if root-squash should not happen, then they have to be
  mounted with root-squash=no option. (This is done because by default
  blocking root-squashing for the trusted clients will cause problems for smb
  and UFO clients for which the requests have to be squashed if the option is
  enabled).

* For geo-replication and defrag clients do not do root-squashing.

* Introduce a new option in open-behind for doing read after successful open.

Change-Id: I8a8359840313dffc34824f3ea80a9c48375067f0
BUG: 954057
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/4863
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gNFS: Set default outstanding RPC limit to 16

2014-01-15T21:47:32+00:00

With 64, NFS server hangs with large I/O load (~ 64 threads writing
to NFS server). The test results from Ben England (Performance expert)
suggest to set it as 16 instead of 64.

Change-Id: I418ff5ba0a3e9fdb14f395b8736438ee1bbd95f4
BUG: 1008301
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6696
Tested-by: Gluster Build System 
Reviewed-by: ben england

rpc/auth: Avoid NULL dereference in rpcsvc_auth_request_init()

2014-01-09T04:42:15+00:00

Code section is bogus!
------------------------------------------
370:       if (!auth->authops->request_init)
371:              ret = auth->authops->request_init (req, auth->authprivate);
------------------------------------------

Seems to have been never been used historically since
logically above code has never been true to actually execute
"authops->request_init() --> auth_glusterfs_{v2,}_request_init()"

On top of that under "rpcsvc_request_init()"
verf.flavour and verf.datalen are initialized from what is
provided through 'callmsg'.
------------------------------------------
        req->verf.flavour = rpc_call_verf_flavour (callmsg);
        req->verf.datalen = rpc_call_verf_len (callmsg);

        /* AUTH */
        rpcsvc_auth_request_init (req);
        return req;
------------------------------------------

So the code in 'auth_glusterfs_{v2,}_request_init()'
performing this operation will over-write the original
flavour and datalen.

------------------------------------------
      if (!req)
                return -1;
        memset (req->verf.authdata, 0, GF_MAX_AUTH_BYTES);
        req->verf.datalen = 0;
        req->verf.flavour = AUTH_NULL;
------------------------------------------

Refactoring the whole code into a more understandable version
and also avoiding a potential NULL dereference

Change-Id: I1a430fcb4d26de8de219bd0cb3c46c141649d47d
BUG: 1049735
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/6591
Reviewed-by: Santosh Pradhan 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur