glusterfs.git/xlators/storage/posix/src, branch exp

events: Add FMT_WARN for gf_event

2016-11-18T09:56:39+00:00

Raghavendra G found that posix is trying to print %s
but passing an int when HEALTH_CHECK fails in posix.
These are the kind of bugs that should be caught
at compilation itself.
Also fixed the problematic gf_event() callers.

BUG: 1386097
Change-Id: Id7bd6d9a9690237cec3ca1aefa2aac085e8a1270
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15671
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System

storage/posix: Fix race in posix_pstat

2016-10-17T19:00:28+00:00

Problem:
When one thread is in the process of creating a file/directory
and the other thread is doing readdirp, there is a chance that
posix_pstat, creation fops race in the following manner which
will lead to wrong stat values to be read by parent xlators
like posix-acl.

Creation fops                           posix_pstat() as part of readdirp
1) file is created with uid/gid 0/0     1) does stat of the path that
                                           is created just now.
2) Does chown to set the correct
   uid/gid
3) Sets the acl/user/internal xattrs
4) Sets the gfid on the entry and
   completes the creation of the file/dir
                                        2) fills the gfid in the iatt

If unwind of readdirp hits server xlator before creation fop, then
posix-acl remembers uid/gid of the file to be root/root and fails
fops like open etc on it.

Fix:
Reverse the order of filling gfid and filling lstat() values in
posix_pstat() so that if there is gfid in iatt buffer uid/gid
are valid.

Change-Id: I46caa7f6da7abfa40a0b1d70e35b88de9c64959c
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15564
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

afr: Ignore gluster internal (virtual) xattrs in metadata heal check

2016-09-26T13:25:40+00:00

Problem:
In arbiter configuration, posix-xlator in the arbiter brick always sets
the GF_CONTENT_KEY in the response dict with a value 0. If the file size on
the data bricks is more than quick-read's max-file-size (64kb default),
those bricks don't set the key. Because of this difference in the no. of dict
elements, afr triggers metadata heal in lookup code path, in turn
leading to extra lookups+inodelks.

Fix:
Changed afr dict comparison logic to ignore all virtual xattrs and the
on-disk ones that we should not be healing.

Also removed is_virtual_xattr() function. The original callers to this
function (upcall) don't seem to need it anymore.

Change-Id: I05730bdd39d8fb0b9a49a5fc9c0bb01f0d3bb308
BUG: 1378684
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15548
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

storage/posix: Integrate important events with gf_event

2016-09-15T03:58:48+00:00

BUG: 1371353
Change-Id: I9346ca16222caf9d97d999751577399ae6aacf72
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15342
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

posix: fix unused variable warnings/errors

2016-08-29T17:30:51+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: Ibb3057cc072097434350c1dd2139b18610295825
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15257
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Prashanth Pai

storage/posix: Log EEXIST failures at DEBUG log-level

2016-08-16T09:24:48+00:00

Change-Id: I56f160058d2e2751ca75f1f75ea8254a1b8a6bf3
BUG: 1367258
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15161
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

posix: Do not move and recreate .glusterfs/unlink directory

2016-08-08T09:05:21+00:00

Problem:
At the time of start of a volume, it is checked if
.glusterfs/unlink exist or not. If it does, move it
to landfill and recreate unlink directory. If a volume
is mounted and we write data on it till we face ENOSPC,
restart of that volume fails as it will not be able to
create unlink dir. mkdir will fail with ENOSPC.
This will not allow volume to restart.

Solution:
If .glusterfs/unlink directory exist, don't move it to
landfill. Delete all the entries inside it.

Change-Id: Icde3fb36012f2f01aeb119a2da042f761203c11f
BUG: 1360679
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/15030
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

posix: honour fsync flags in posix_do_zerofill

2016-08-05T16:06:51+00:00

commit 6d994661db96cad43f26d0d1e2fba31dce9a76f5 introduced fallocate +
FALLOC_FL_ZERO_RANGE to do posix_do_zerofill but did not do an fsync if
O_SYNC or O_DSYNC fd flags were set. Prashant Pai pointed out performing
fallocate(2) does not remove the necessity to do fsyncs if durability is
desired: http://linux-fsdevel.vger.kernel.narkive.com/bDmrAUlh/fallocate-falloc-fl-punch-hole

So fixed it to honour the fd flags.

Change-Id: I27ae90b429185d0af29a5e632c4a8b242075a899
BUG: 1364449
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15100
Reviewed-by: Prashanth Pai 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

posix: leverage FALLOC_FL_ZERO_RANGE in zerofill fop

2016-07-29T07:09:54+00:00

posix_zerofill() implements zerofilling of a given (offset,length) by
doing a writev in a loop followed by an optional fsync on the file.
fallocate(2) has a FALLOC_FL_ZERO_RANGE flag which does away with all
this and provides the same result (from a userspace application point of
view) with a single syscall. This patch attempts the zerofill with the
latter and falls back to the former if it fails.

Tested using a libgfapi based C program on XFS and observed using gdb that
posix_zerofill()'s call to fallocate with FALLOC_FL_ZERO_RANGE was a
success.

Change-Id: I77e9b7de0d59c255f06b0c39c43a276990081727
BUG: 1361249
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15037
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System