glusterfs.git/rpc, branch v3.5.1

gNFS: Make NFS DRC off by default

2014-06-10T09:37:13+00:00

DRC in NFS causes memory bloat and there are known memory corruptions.
It would be good to disable drc by default till the feature is stable.

Cherry picked from 4215d071cec4fc8a62ca4fd6212d83f931838829:
> Change-Id: I93db6ef5298672c56fb117370bb582a5e5550b17
> BUG: 1105524
> Original-patch-by: Santosh Kumar Pradhan 
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/8004
> Tested-by: Gluster Build System 
> Reviewed-by: Krishnan Parthasarathi 
> Reviewed-by: Santosh Pradhan 
> Reviewed-by: Vijay Bellur 

Change-Id: I93db6ef5298672c56fb117370bb582a5e5550b17
BUG: 1105524
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/8013
Reviewed-by: Santosh Pradhan 
Tested-by: Gluster Build System

NetBSD build fixes for release-3.5

2014-06-08T09:18:13+00:00

1) Make sure __THROW is definedThis is a backport of I6e7cb1eb59b84988e155e9a8b696e842b7ff8f7f

2) include  before  so that XDR is defined
This was fixed in master within I20193d3f8904388e47344e523b3787dbeab044acbut weonly pull up

3) NetBSD's gettext is in libintl, hence search it at configure time
This is a backport of I651a74fe49c3f087fe135dab3453fd5b18b4268a

4) include  to have WEXITSTATUS defined
This problem does not exist in master as WEXITSTATUS is not used

5) Do not define popcountl() on NetBSD as it is in 
This is a backport of I4428a88b1e0d7c5f6740022861ffe230dbbd84bd

BUG: 764655
Change-Id: Ieea5a2a627e2b7930525d6c525f1602073574a97
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/7925
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System

rpc: warn and truncate grouplist if RPC/AUTH can not hold everything

2014-05-22T13:02:21+00:00

The GlusterFS protocol currently uses AUTH_GLUSTERFS_V2 in the RPC/AUTH
header. This header contains the uid, gid and auxiliary groups of the
user/process that accesses the Gluster Volume.

The AUTH_GLUSTERFS_V2 structure allows up to 65535 auxiliary groups to
be passed on. Unfortunately, the RPC/AUTH header is limited to 400 bytes
by the RPC specification: http://tools.ietf.org/html/rfc5531#section-8.2

In order to not cause complete failures on the client-side when trying
to encode a AUTH_GLUSTERFS_V2 that would result in more than 400 bytes,
we can calculate the expected size of the other elements:

    1 | pid
    1 | uid
    1 | gid
    1 | groups_len
   XX | groups_val (GF_MAX_AUX_GROUPS=65535)
    1 | lk_owner_len
   YY | lk_owner_val (GF_MAX_LOCK_OWNER_LEN=1024)
  ----+-------------------------------------------
    5 | total xdr-units

  one XDR-unit is defined as BYTES_PER_XDR_UNIT = 4 bytes
  MAX_AUTH_BYTES = 400 is the maximum, this is 100 xdr-units.
  XX + YY can be 95 to fill the 100 xdr-units.

  Note that the on-wire protocol has tighter requirements than the
  internal structures. It is possible for xlators to use more groups and
  a bigger lk_owner than that can be sent by a GlusterFS-client.

This change prevents overflows when allocating the RPC/AUTH header. Two
new macros are introduced to calculate the number of groups that fit in
the RPC/AUTH header, when taking the size of the lk_owner in account. In
case the list of groups exceeds the maximum possible, only the first
groups are passed over the RPC/GlusterFS protocol to the bricks.
A warning is added to the logs, so that most system administrators will
get informed.

The reducing of the number of groups is not a new inventions. The
RPC/AUTH header (AUTH_SYS or AUTH_UNIX) that NFS uses has a limit of 16
groups. Most, if not all, NFS-clients will reduce any bigger number of
groups to 16. (nfs.server-aux-gids can be used to workaround the limit
of 16 groups, but the Gluster NFS-server will be limited to a maximum of
93 groups, or fewer in case the lk_owner structure contains more items.)

Cherry picked from commit 8235de189845986a535d676b1fd2c894b9c02e52:
> BUG: 1053579
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/7202
> Tested-by: Gluster Build System 
> Reviewed-by: Harshavardhana 
> Reviewed-by: Santosh Pradhan 
> Reviewed-by: Vijay Bellur 

Change-Id: I8410e59d0fd246d601b54b961d3ae9cb5a858c10
BUG: 1096425
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/7829
Reviewed-by: Lalatendu Mohanty 
Tested-by: Gluster Build System

libgfapi: Added support to fetch volume info from glusterd and store in glfs object.

2014-05-12T15:14:52+00:00

Defined new APIs in the libgfapi module, given a glfs object,
 * to send handshake RPC call to glusterd process to fetch UUID of the volume
 * store it in the glusterfs_context linked to the glfs object.
 * to parse UUID from its cannonical string format into 16-byte array
   before sending it to the libgfapi users.

Defined a RPC call in glusterd which can be used to query volume related
info by other processes using 'clnt_handshake_procs'.

Note - Currently this RPC call to glusterd process is used only to fetch UUID.
But it can be extended to get other volume related structures as well.

In addition to the above, defined a new variable to keep track of such handshake
RPCs still in progress to make sure all the corresponding RPC callbacks have been
processed before libgfapi returns the glfs object initialized.

Also bumping up the GFAPI current version number since there is a new API
"glfs_get_volume_id" defined and exposed by libgfapi as part of these changes.

Cherry picked from commit 5adb10b9ac1c634334f29732e062b12d747ae8c5:
> Change-Id: I303f76d7177d32d25bdb301b1dbcf5cd73f42807
> BUG: 1095775
> Signed-off-by: Soumya Koduri 
> Reviewed-on: http://review.gluster.org/7218
> Reviewed-by: Anand Avati 
> Reviewed-by: Harshavardhana 
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

This change differs a little from the patch in the master branch:
- libgfapi in 3.5 does not have glfs_get_volfile(), so there were some
  merge conflicts resolved,
- libgfapi in 3.5 is not versioned, the configure.ac changes related to
  the versioning have been skipped,
- in the master branch only the XDR .x files are available, release-3.5
  requires a manual re-generation of the relates .h and .c files.

Change-Id: I52c32d0e69a52a7f4285f74164bca6fd83c4f3b3
BUG: 1095775
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/7741
Reviewed-by: Niels de Vos 
Tested-by: Niels de Vos

rpcsvc: Ignore INODELK/ENTRYLK/LK for throttling

2014-05-08T12:58:16+00:00

Problem:
When iozone is in progress, number of blocking inodelks sometimes becomes greater
than the threshold number of rpc requests allowed for that client
(RPCSVC_DEFAULT_OUTSTANDING_RPC_LIMIT). Subsequent requests from that client
will not be read until all the outstanding requests are processed and replied
to. But because no more requests are read from that client, unlocks on the
already granted locks will never come thus the number of outstanding requests
would never come down. This leads to a ping-timeout on the client.

Fix:
Do not account INODELK/ENTRYLK/LK for throttling

BUG: 1089470
Change-Id: I9cc2c259d2462159cea913d95f98a565acb8e0c0
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/7570
Tested-by: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
Reviewed-by: Niels de Vos

rdma: correct some spelling mistakes

2014-04-17T18:19:35+00:00

Change-Id: Ie99ad6db42d3a52bcc7748448c6d5e97e5bad69e
BUG: 1088848
Reported-by: Patrick Matthäi 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/7499
Reviewed-by: Vikhyat Umrao 
Reviewed-by: Humble Devassy Chirammal 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gNFS: Set default outstanding RPC limit to 16

2014-03-07T20:11:28+00:00

Backport of http://review.gluster.org/#/c/6696/

With 64, NFS server hangs with large I/O load (~ 64 threads writing
to NFS server). The test results from Ben England (Performance expert)
suggest to set it as 16 instead of 64.

Change-Id: Iaa9dda512904d2e359d8122a05e5bf65f99a7e78
BUG: 1073441
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/7200
Tested-by: Gluster Build System 
Reviewed-by: Santosh Pradhan

mgmt/glusterd: make sure quota enforcer has established connection with quotad before marking quota as enabled.

2014-01-29T06:46:35+00:00

without this patch there is a window of time when quota is marked as
enabled in quota-enforcer, but connection to quotad wouldn't have been
established. Any checklimit done during this period can result in a
failed fop because of unavailability of quotad.

Change-Id: I0d509fabc434dd55ce9ec59157123524197fcc80
Signed-off-by: Raghavendra G 
BUG: 969461
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/6572
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/6820

glusterd/geo-rep: more glusterd and cli fixes for geo-rep.

2014-01-27T17:44:33+00:00

    -> handle option validation cases in reset case.
    -> Creating valid conf path when glusterd restarts.
    -> Reading the gsyncd worker thread status and displaying it.
    -> Displaying status-detail per worker.
    -> Fetch checkpoint info in geo-rep status.
    -> use-tarssh value validation added.

misc: misc geo-rep fixes based on cluster, logrotate etc..
    -> cluster/dht: fix 'stime' getxattr getting overwritten.
    -> cluster/afr: return max of 'stime' values in subvol.
    -> geo-rep-logrotate: Sending SIGHUP to geo-rep auxiliary.
    -> cluster/dht: fix convoluted logic while aggregating.
    -> cluster/*: fix 'stime' min/max fetch logic.

Change-Id: I811acea0bbd6194797a3e55d89295d1ea021ac85
BUG: 1036552
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/6405
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6810
Reviewed-by: Vijay Bellur

rpc,glusterd: Use rpc_clnt notifyfn to cleanup mydata

2013-12-23T14:58:18+00:00

        Backport of http://review.gluster.org/5512

rpc:
- On a RPC_TRANSPORT_CLEANUP event, rpc_clnt_notify calls the registered
  notifyfn with a RPC_CLNT_DESTROY event. The notifyfn should properly
  cleanup the saved mydata on this event.
- Break the reconnect chain when an rpc client is disabled. This will
  prevent new disconnect events which can lead to crashes.

glusterd:
- Added support for RPC_CLNT_DESTROY in glusterd_brick_rpc_notify
- Use a common glusterd_rpc_clnt_unref() function throught glusterd in
  place of rpc_clnt_unref(). This function correctly gives up the
  big-lock before performing the unref.

Change-Id: I93230441c5089039643fc9f5632477ef1b695348
BUG: 962619
Signed-off-by: Kaushal M 
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6566
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur