glusterfs.git/xlators/features/locks, branch v3.11dev

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

feature/locks: Implement fops in locks Translator

2016-09-01T14:07:23+00:00

Implement those fop which were not present in
locks translator.
Also make sure to return xdata with the
entries which were requested for these fops.

Change-Id: I6e7f80e9eb5fba9e34c7034f6bc30557cde20927
BUG: 1231224
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/11204
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

core: fix unused variable warnings/errors

2016-08-29T16:24:04+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: Id9bf5732f8c94824769409f7895be9dbe6fbd5fa
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15249
Smoke: Gluster Build System 
Reviewed-by: Susant Palai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Jeff Darcy

features/locks: fix fdctx leak in locks xlators

2016-08-25T02:07:03+00:00

Problem:
Locks xlators is leaking fdctx in pl_release when
inode_ctx_get return non-zero

Fix:
This patch fixes fdctx leak in pl_release path

Change-Id: Icd5c5c681b7d890e7971b3b06d4258a51d45097d
Signed-off-by: Yanping.gao 
Reviewed-on: http://review.gluster.org/15302
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

features/locks: Add lock revocation functionality to posix locks translator

2016-07-18T09:30:38+00:00

Summary:
- Motivation: Prevents cluster instability by mis-behaving clients
causing bricks to OOM due to inode/entry lock pile-ups.
- Adds option to strip clients of entry/inode locks after N seconds
- Adds option to clear ALL locks should the revocation threshold get hit
- Adds option to clear all or granted locks should the max-blocked
threshold get hit (can be used in combination w/ revocation-clear-all).
- Options are:
features.locks-revocation-secs
features.locks-revocation-clear-all [on/off]
features.locks-revocation-max-blocked
- Adds monkey-locking option to ignore 1% of unlock requests (dev only)
features.locks-monkey-unlocking [on/off]
- Adds logging to indicate revocation event & reason

Test Plan:
First you will need TWO fuse mounts for this repro. Call them /mnt/patchy1 & /mnt/patchy2.

1. Enable monkey unlocking on the volume:
gluster vol set patchy features.locks-monkey-unlocking on

2. From the "patchy1", use DD or some other utility to begin writing to a file,
eventually the dd will hang due to the dropped unlocked requests. This now
simulates the broken client. Run:

for i in {1..1000};do dd if=/dev/zero of=/mnt/patchy1/testfile bs=1k count=10;done'

...this will eventually hang as the unlock request has been lost.

3. Goto another window and setup the mount "patchy2" @ /mnt/patchy2, and
observe that 'echo "hello" >> /mnt/patchy2/testfile" will hang due to the
inability of the client to take out the required lock.

4. Next, re-start the test this time enabling lock revocation; use a timeout of
2-5 seconds for testing:
'gluster vol set patchy features.locks-revocation-secs <2-5>'

5. Wait 2-5 seconds before executing step 3 above this time. Observe that this
time the access to the file will succeed, and the writes on patchy1 will
unblock until they hit another failed unlock request due to
"monkey-unlocking".

BUG: 1350867
Change-Id: I814b9f635fec53834a26db634d1300d9a61057d8
Signed-off-by: Pranith Kumar K
Reviewed-on: http://review.gluster.org/14816
NetBSD-regression: NetBSD Build System
Reviewed-by: Krutika Dhananjay
CentOS-regression: Gluster Build System
Smoke: Gluster Build System

posix/lock: implement meta-lock/unlock functionality

2016-06-03T10:06:31+00:00

problem : The lock state needs to be protected when rebalance is reading the
lock state on the source. Otherwise there will be locks left unmigrated.
Hence, to synchronize incoming lock requests with lock-migration, meta lock
is needed. Any new lock request will be queued if there is an active meta-lock
and with sucessful lock migration, will be unwound with EREMOTE, so that
dht module can wind the request to the correct destination.

On a successful lock migration, "pl_inode->migrated" flag is enabled. Hence,
any further request would be unwound with EREMOTE and will be redirected to
new destination.

More details can be found here:
https://github.com/gluster/glusterfs-specs/blob/master/accepted/Lock-Migration.md

design discussion:
https://www.gluster.org/pipermail/gluster-devel/2016-January/048088.html

Change-Id: Ief033d5652b5ca4ba6f499110a521cae283d6aba
BUG: 1331720
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14251
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

__inode_ctx_put: fix mem leak on failure

2016-06-01T13:28:48+00:00

up on failure case __inode_ctx_put need to free the allocated memory

Indirect leak of 104 byte(s) in 1 object(s) allocated from:
    #0 0x496669 in __interceptor_calloc (/usr/local/sbin/glusterfsd+0x496669)
    #1 0x7f8a288522f9 in __gf_calloc libglusterfs/src/mem-pool.c:117
    #2 0x7f8a17235962 in __posix_acl_ctx_get xlators/system/posix-acl/src/posix-acl.c:308

Change-Id: I0ce6da3967c55931a70f77d8551ccf52e4cdfda3
BUG: 1338733
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14505
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

core: Honour mandatory lock flags during lock migration

2016-05-20T11:23:19+00:00

lk_flags from posix_lock_t structure is the primary key used to
differentiate locks as either advisory and mandatory type. During
lock migration this field is not read in getactivelk() call path.
So in order to copy the exact lock state from source to destination
it is necessary to include lk_flags within lock_migration_info_t
structure to maintain accurate state. This change also includes
minor modifications to setactivelk() call to consider lk_flags
during lock migration.

Change-Id: I20a7b6b6a0f3bdac5734cce8a2cd2349eceff195
BUG: 1332501
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/14189
Smoke: Gluster Build System 
Reviewed-by: Susant Palai 
Reviewed-by: Poornima G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

locks: Fix a compile warning to use labs instead of abs

2016-05-12T02:54:43+00:00

This addresses below compile warning generated -

posix.c:2260:32: warning: absolute value function 'abs' given an argument of type 'off_t' (aka 'long') but has parameter of type 'int' which may cause truncation of value [-Wabsolute-value]
                flock->l_len = abs (flock->l_len);
                               ^
posix.c:2260:32: note: use function 'labs' instead
                flock->l_len = abs (flock->l_len);
                               ^~~
                               labs

Change-Id: Ifd8c5442de4076a2d4425487eb119ce89097779d
BUG: 1335231
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/14296
Smoke: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Kaleb KEITHLEY 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

features/locks: Implement mandatory locks

2016-05-02T11:18:44+00:00

Initial change to fix/enable the mandatory locking support in GlusterFS
as per the following design:

https://review.gluster.org/#/c/12014/

Accordingly 'locks.mandatory-locking' option is available as part of this
change which will accept one among the following values:

* off
* file
* forced
* optimal

See design doc for more details

Change-Id: I14c489b3f8af5ebcbfa155a03f0c175e9558ac46
BUG: 762184
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/9768
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Poornima G 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Pranith Kumar Karampuri