glusterfs.git/xlators/nfs/server/src/nfs3.h, branch v3.5.0

gNFS: Client cache invalidation with bad fsid

2013-12-17T18:26:10+00:00

1. Problem:
Couple of issues are seen when NFS-ACL is turned ON. i.e.
i) NFS directory access is too slow, impacting customer workflows
   with ACL
ii)dbench fails with 100 directories.

2. Root cause: Frequent cache invalidation in the client side when ACL
is turned ON with NFS because NFS server getacl() code returns the
wrong fsid to the client.

3. This attr-cache invlaidation triggers the frequent LOOKUP ops for
each file instead of relying on the readdir or readdirp data. As
a result performance gets impacted.

4. In case of dbench workload, the problem is more severe. e.g.

Client side rpcdebug output:
===========================

Dec 16 10:16:53 santosh-3 kernel: NFS:
         nfs_update_inode(0:1b/12061953567282551806 ct=2 info=0x7e7f)
Dec 16 10:16:53 santosh-3 kernel: NFS:
         nfs_fhget(0:1b/12061953567282551806 ct=2)
Dec 16 10:16:53 santosh-3 kernel: <-- nfs_xdev_get_sb() = -116 [splat]
Dec 16 10:16:53 santosh-3 kernel: nfs_do_submount: done
Dec 16 10:16:53 santosh-3 kernel: <-- nfs_do_submount() =
ffffffffffffff8c
Dec 16 10:16:53 santosh-3 kernel: <-- nfs_follow_mountpoint() =
ffffffffffffff8c
Dec 16 10:16:53 santosh-3 kernel: NFS: dentry_delete(clients/client77,
20008)

As per Jeff Layton, This occurs when the client detects that the fsid on
a filehandle is different from its parent. At that point, it tries to
do a new submount of the new filesystem onto the correct point. It means
client got a superblock reference for the new fs and is now looking to
set up the root of the mount. It calls nfs_get_root to do that, which
basically takes the superblock and a filehandle and returns a dentry.
The problem here is that the dentry->d_inode you're getting back looks
wrong. It's not a directory as expected -- it's something else. So the
client gives up and tosses back an ESTALE.

Which clearly says that, In getacl() code while it does the stat() call
to get the attrs, it forgets to populate the deviceid or fsid before
going ahead and does getxattr().

FIX:
1. Fill the deviceid in iatt.
2. Do bit more clean up for the confusing part of the code.

NB: Many many thanks to Niels de Vos and Jeff Layton for their
help to debug the issue.

Change-Id: I44d8d2fa3ec7fb33a67dfdd4bbe2c45cdf67db8c
BUG: 1043737
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6526
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gNFS: More clean up for Gluster NFS

2013-11-25T13:40:57+00:00

1) Fix the typo in NFS default ACL
   The typo was introduced as part of the Fix to BZ 1009210 i.e.
   http://review.gluster.org/5980. The user ACL xattr structure
   was passed to default ACL xattr.

2) Clean up NFS code to avoid unnecessary SEGV in
   rpcsvc_drc_reconfigure() which was not validating the
   svc->drc. Add a routine rpcsvc_drc_deinit() to handle
   the clean up of DRC specific data structures. For init(),
   use rpcsvc_drc_init().

3) nfs_init_state() was returning wrong value even if the
   registration with portmapper failed, causing the NFS
   server process to hang around. As a result it used to
   get SEGV during rpcsvc_drc_reconfigure().

4) Clean up memfactor usage across nfs.c nfs3.c.

Change-Id: I5cea26cb68dd8a822ec0ae104952f67fe63fa703
BUG: 1009210
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6329
Reviewed-by: Rajesh Joseph 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gNFS: RFE for NFS connection behavior

2013-11-15T00:07:02+00:00

Implement reconfigure() for NFS xlator so that volume set/reset wont
restart the NFS server process. But few options can not be reconfigured
dynamically e.g. nfs.mem-factor, nfs.port etc which needs NFS to be
restarted.

Change-Id: Ic586fd55b7933c0a3175708d8c41ed0475d74a1c
BUG: 1027409
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6236
Tested-by: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Anand Avati

gNFS: Make NFS I/O size to 1MB by default

2013-10-28T06:15:12+00:00

For better NFS performance, make the default I/O size to 1MB, same as
kernel NFS. Also refactor the description for read-size, write-size
and readdir-size (i.e. it must be a multiple of 1KB but min value
is 4KB and max supported value is 1MB). On slower network, rsize/wsize
can be adjusted to 16/32/64-KB through nfs.read-size or nfs.write-size
respectively.

Change-Id: I142cff1c3644bb9f93188e4e890478177c9465e3
BUG: 1009223
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6103
Tested-by: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Anand Avati

dual license: update remaining files with correct license text

2013-10-24T14:58:19+00:00

Sayan Saha has previously approved changing everthing to dual license
but somehow we have missed changing these files.

I am explicitly not updating the copyright dates as nothing else that's
copyrightable has changed in these files with the license change

Change-Id: Ia965eeb7168447d69e28e939ad95ee388873b6e4
BUG: 951549
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/6128
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gNFS: NFS daemon is limiting IOs to 64KB

2013-09-20T18:29:34+00:00

Problem:
Gluster NFS server is hard-coding the max rsize/wsize to 64KB
which is very less for NFS running over 10GE NIC. The existing
options nfs.read-size, nfs.write-size are not working as
expected.

FIX:
Make the options nfs.read-size (for rsize) and nfs.write-size
(for wsize) work to tune the NFS I/O size. Value range would
be 4KB(Min)-64KB(Default)-1MB(max).

NB: Credit to "Richard Wareing" for catching it.

Change-Id: I2754ecb0975692304308be8bcf496c713355f1c8
BUG: 1009223
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/5964
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Anand Avati

nfs: log stale mount errors occasionally

2013-06-19T03:28:04+00:00

log the state mount errors only occasionally so
as not to fill log file with too many of them.

Change-Id: Ib5a2485dc2ce3a181cff34bbb6d7aba17a2e4d4d
BUG: 804301
Signed-off-by: Rajesh Amaravathi 
Reviewed-on: http://review.gluster.org/5229
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Vijay Bellur

ACLv3 - Access Control Lists V3

2012-11-14T07:24:46+00:00

Change-Id: I43e544d6cdeac5e3880141477461e7c22cbf6e91
BUG: 847622
Signed-off-by: Krishna Srinivas 
Reviewed-on: http://review.gluster.org/4045
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

nfs/nlm: procedures for PC clients

2012-05-17T09:12:49+00:00

*  This change introduces four NLMv4 procedures:

   NM_LOCK, SHARE, UNSHARE and FREE_ALL.

   These are used by PC clients (windows/dos) to control access
   to files.

   1. NM_LOCK: this lock is not monitored by statd.

   2. SHARE: A share reservation is a lock on the whole file
      that is taken whenever a file is opened on windows clients.
      This has ACCESS (N, R, W, RW) and DENY MODE (N, R, W, RW).
        ACCESS:    mode of access requested by the client;
        DENY MODE: what the requesting client wants to
                   deny other clients.

   3. UNSHARE: remove a share reservation obtained by SHARE.
      Called while closing a file.

   4. FREE_ALL: remove all share reservations and locks,
      both monitored and unmonitored, of the calling client.

*  lock and nm_lock use a common function with only
   a flag conveying whether or not to monitor a lock.

*  NOTES:
   1. SHARE reservations are not STACK_WIND'd to subsequent xlators.
      These are maintained in-memory in the nfs xlator.

   2. Consequently, for SHARE reservations to work effectively,
      all PC clients  *must* mount from the same gNfs server.
      Not doing so will result in different servers maintaining
      separate SHARE reservations which will not be enforced
      for obvious reasons.

Change-Id: Id4f22670a94ed58691a6a7f4c80aa8c11421a277
BUG: 800287
Signed-off-by: Rajesh Amaravathi 
Reviewed-on: http://review.gluster.com/3212
Tested-by: Gluster Build System 
Reviewed-by: Krishna Srinivas

glusterd: Include port in nfs status

2012-04-02T09:41:18+00:00

Change-Id: I679e491e801b694e8a0f93dd87cf540441dae927
BUG: 806877
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.com/3060
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur