glusterfs.git/xlators/storage, branch v3.10.0rc0

dht : file rename operation is successful but log has error File exists

2017-01-10T06:27:50+00:00

Problem: file rename operation is successful but log has error
         'key:trusted.glusterfs.dht.linkto error:File exists'

Solution: The error comes in log only when file already has set this xattr
          (trusted.glusterfs.dht.linkto) before run rename operation and
          in that case sys_lsetxattr throws this kind of message.
          To avoid the error message in logs update the condition in
          posix_handle_pair.

BUG: 1404905
Change-Id: Iafd8cb45f9d7f4fe247e297a6ef0af978a8d0b30
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/16185
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

posix: make sure atime and mtime are set when calling lutimes()

2017-01-06T17:33:10+00:00

When overwriting an existing file with O_TRUNC, the 'atime' was set to
0, meaning the Epoch (01-Jan-1970 UTC). However, the 'mtime' gets
updated correcty.

In case 'atime' or 'mtime' is not passed in the 'struct iatt', the time
values passed to the systemcall are taken from the current values are
returned by lstat().

Change-Id: I7021b7161dcd6c9a3e515d98f6d4847533c434b3
BUG: 1401777
Reported-by: Eivind Sarto 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/16034
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra Bhat 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

storage/posix: log the error when locking the export directory fails

2017-01-03T13:42:40+00:00

posix xlator tries to prevent external unmount of the brick export directory
by locking it. It locks the export directory by opening it (i.e. doing a opendir).

But if opendir fails it errors out resulting in the brick process going down. The
error due to which opendir failed is not logged.

Change-Id: I28648382b7f0b88fb1aeb36152e5ab5d812374c9
BUG: 1401095
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/16018
Smoke: Gluster Build System 
Tested-by: Vijay Bellur 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur

Fixes GlusterFS process crashes on deep directory hierarchy

2016-12-22T06:42:30+00:00

alloca() calls to recusive function posix_make_ancestryfromgfid
consumes the entire stack leading to segfault.
Changed the function to  iterative and reused the same linkname
buffer across iterations.

Change-Id: If065c8b9f7f85219bdd2d23259a115fe66f5f60d
BUG: 1405775
Signed-off-by: Sanoj Unnikrishnan 
Reviewed-on: http://review.gluster.org/16192
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

storage/posix: Do not create a directory which already exist

2016-12-22T05:41:03+00:00

Problem: While brick process starts, it tries to create trashcan
and internal_op directory with fixed gfids. For the first time
these directories will not be present. However, stop and start
of volume again tries to create these directories, which are
already present.
This throws a gfid exist warning and EEXIST error in logs.

Solution: Check if the directory exist on brick and gfid of
this dir and the requested gfid matches or not.

Change-Id: Ie4966a550ed44046e0f2d10e9dbd540182ea97d2
BUG: 1406348
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/16212
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

events: Add FMT_WARN for gf_event

2016-11-18T09:56:39+00:00

Raghavendra G found that posix is trying to print %s
but passing an int when HEALTH_CHECK fails in posix.
These are the kind of bugs that should be caught
at compilation itself.
Also fixed the problematic gf_event() callers.

BUG: 1386097
Change-Id: Id7bd6d9a9690237cec3ca1aefa2aac085e8a1270
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15671
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System

storage/posix: Fix race in posix_pstat

2016-10-17T19:00:28+00:00

Problem:
When one thread is in the process of creating a file/directory
and the other thread is doing readdirp, there is a chance that
posix_pstat, creation fops race in the following manner which
will lead to wrong stat values to be read by parent xlators
like posix-acl.

Creation fops                           posix_pstat() as part of readdirp
1) file is created with uid/gid 0/0     1) does stat of the path that
                                           is created just now.
2) Does chown to set the correct
   uid/gid
3) Sets the acl/user/internal xattrs
4) Sets the gfid on the entry and
   completes the creation of the file/dir
                                        2) fills the gfid in the iatt

If unwind of readdirp hits server xlator before creation fop, then
posix-acl remembers uid/gid of the file to be root/root and fails
fops like open etc on it.

Fix:
Reverse the order of filling gfid and filling lstat() values in
posix_pstat() so that if there is gfid in iatt buffer uid/gid
are valid.

Change-Id: I46caa7f6da7abfa40a0b1d70e35b88de9c64959c
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15564
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

afr: Ignore gluster internal (virtual) xattrs in metadata heal check

2016-09-26T13:25:40+00:00

Problem:
In arbiter configuration, posix-xlator in the arbiter brick always sets
the GF_CONTENT_KEY in the response dict with a value 0. If the file size on
the data bricks is more than quick-read's max-file-size (64kb default),
those bricks don't set the key. Because of this difference in the no. of dict
elements, afr triggers metadata heal in lookup code path, in turn
leading to extra lookups+inodelks.

Fix:
Changed afr dict comparison logic to ignore all virtual xattrs and the
on-disk ones that we should not be healing.

Also removed is_virtual_xattr() function. The original callers to this
function (upcall) don't seem to need it anymore.

Change-Id: I05730bdd39d8fb0b9a49a5fc9c0bb01f0d3bb308
BUG: 1378684
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15548
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

storage/posix: Integrate important events with gf_event

2016-09-15T03:58:48+00:00

BUG: 1371353
Change-Id: I9346ca16222caf9d97d999751577399ae6aacf72
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15342
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System