glusterfs.git/xlators, branch v3.6.1

cluster/afr: Perform post-op in entry selfheal inside locks

2014-10-31T13:09:28+00:00

        Backport of: http://review.gluster.org/#/c/9020

Take entrylks in xlator domain before doing post-op (undo-pending) in
entry self-heal. This is to prevent a parallel name self-heal on
an entry under @fd->inode from reading pending xattrs while it is
being modified by SHD after entry sh below, given that
name self-heal takes locks ONLY in xlator domain and is free to read
pending changelog in the absence of the following locking.

Change-Id: I0bc92978efc0741d6e3f2439540d008e31472313
BUG: 1136821
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/9030
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

features/snapview-server: verify the fs instance in revalidated lookups as well

2014-10-30T18:39:59+00:00

Change-Id: I691635e60aba72642c3c79d7da472884f1228301
BUG: 1158791
Reviewed-on: http://review.gluster.org/9008
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/snapview-server: check if the reference to the snapshot world is correct before doing any fop

2014-10-30T18:39:15+00:00

The following operations might lead to problems:
* Create a file on the glusterfs mount point
* Create a snapshot (say "snap1")
* Access the contents of the snapshot
* Delete the file from the mount point
* Delete the snapshot "snap1"
* Create a new snapshot "snap1"

Now accessing the new snapshot "snap1" gives problems. Because the inode and
dentry created for snap1 would not be deleted upon the deletion of the snapshot
(as deletion of snapshot is a gluster cli operation, not a fop). So next time
upon creation of a new snap with same name, the previous inode and dentry itself
will be used. But the inode context contains old information about the glfs_t
instance and the handle in the gfapi world. Directly accessing them without
proper check leads to ENOTCONN errors. Thus the glfs_t instance should be
checked before accessing. If its wrong, then right instance should be obtained
by doing the lookup.

Change-Id: I975245b8f6b7fea0a90eb5e36e8149d12457ac10
BUG: 1158791
Reviewed-on: http://review.gluster.org/9007
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

Avoid spurious EINVAL in posix_readdir()

2014-10-29T09:26:56+00:00

On non Linux systems, we check that seekdir() succeeds and we return
EINVAL if it does not. We need this to avoid infinite loops if some
other component in GlusterFS makes an invalid seekdir() usage. This
was introduced in this change: http://review.gluster.org/#/c/8760/

But seekdir() also fails when using the offset returned for the
last entry, and this is expected behavior. As a result, the seekdir()
test produces a spurious EINVAL when reaching end of directory. That
error is not propagated to calling process, but it may harm internal
GlusterFS processing. At least it produce a spurious error message
in brick's log.

We fix the problem by remembering the last entry offset in fd private
data. When a new posix_readdir() invocation requests that offset,
we avoid returning EINVAL.

Backport of I4e67a2ea46538aae63eea663dd4aa33b16ad24c7

BUG: 1138897
Change-Id: I4e98294d157f67ae1a1f0ece1562c77d1219da40
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8933
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

ec: Correctly handle xtime extended attribute

2014-10-28T08:40:02+00:00

Change-Id: I2bd34f063d6bf1835d5ae57a8e9aa03f3ec3deb3
BUG: 1156405
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/8976
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright

ec: Fix rebalance issues

2014-10-27T14:18:51+00:00

Some issues in ec xlator made that rebalance didn't complete
successfully and generated some warnings and errors in the
log. The most critical error was a race condition that caused
false corruption detection when two specific operations were
executed sequentially and they shared the same lock.

This explains the problem:

1. A setxattr is issued.
2. setxattr: ec locks the inode before updating the xattr.
3. setxattr: The xattr is updated.
4. setxattr: Upper xlator is notified that the operation completed.
5. setxattr: A background task is initiated to update the version
             of the file.
6. A stat is issued on the same file.
7. stat: Since the lock is already acquired, it's reused.
8. stat: A lookup is issued to determine version and size
         information of the file.

At this point, operations 5 and 8 can interfere. This can make that
lookup sees different information on each brick, determining that
some bricks are corrupted and incorrectly excluding them from the
operation and initiating a self-heal. In some cases this false
detection combined with self-heal could lead to invalid updates of
the trusted.ec.size xattr, leaving the file smaller than it should
be.

This only happens if the first operation does not perform a lookup,
because chained operations reuse the information returned by the
previous one, avoiding this kind of problems.

To solve this, now the background update is executed atomically with
the posterior unlock. This avoids some reuses of the lock while
updating. However this reduces performance because the window in
which new requests can reuse the lock is much smaller now. This has
been alleviated by using the same technique implemented in AFR (i.e.
waiting some time before releasing the lock).

Some minor changes also introduced in this patch:

* Bug in management of 'trusted.glusterfs.pathinfo' that was writing
  beyond the allocated space.
* Uninitialized variable.
* trusted.ec.config was not created for regular files created with
  mknod.
* An invalid state was used in access fop.

This is a backport of http://review.gluster.org/8947/

Change-Id: Idfaf69578ed04dbac97a62710326729715b9b395
BUG: 1152903
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/8948
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright

logs: Do selective logging for errnos

2014-10-22T10:04:26+00:00

        Backport of http://review.gluster.org/8918
                    http://review.gluster.org/8955

Problem:
Just after replace-brick the mount logs are filled with ENOENT/ESTALE
warning logs because the file is yet to be self-healed now that the
brick is new.

Fix:
Do conditional logging for the logs. ENOENT/ESTALE will be logged at
lower log level. Only when debug logs are enabled, these logs will
be written to the logfile.

Change-Id: If203d09e2479e8c2415ebc14fb79d4fbb81dfc95
BUG: 1155027
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8957
Tested-by: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
Reviewed-by: Vijay Bellur

cluster/afr: Add afr-v1 xattr compatibility

2014-10-22T10:03:51+00:00

        Backport of http://review.gluster.org/8536

All the special cases v1 handles and also
self-accusing pending changelog from v1 pre-op also is handled
in this patch.

BUG: 1155017
Change-Id: I86cf6b80492be5c1f240c74f91a0e1b0dd9b58b2
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8956
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

ec: Fix self-heal issues

2014-10-22T08:05:19+00:00

Problem: Doing an 'ls' of a directory that has been modified while one
         of the bricks was down, sometimes returns the old directory
         contents.

Cause: Directories are not marked when they are modified as files are.
       The ec xlator balances requests amongst available and healthy
       bricks. Since there is no way to detect that a directory is
       out of date in one of the bricks, it is used from time to time
       to return the directory contents.

Solution: Basically the solution consists in use versioning information
          also for directories, however some additional changes have
          been necessary.

Changes:

 * Use directory versioning:

     This required to lock full directory instead of a single entry for
     all requests that add or remove entries from it. This is needed to
     allow atomic version update. This affects the following fops:

         create, mkdir, mknod, link, symlink, rename, unlink, rmdir

     Another side effect is that opendir requires to do a previous
     lookup to get versioning information and discard out of date
     bricks for subsequent readdir(p) calls.

 * Restrict directory self-heal:

     Till now, when one discrepancy was found in lookup, a self-heal
     was automatically started. This caused the versioning information
     of a bad directory to be healed instantly, making the original
     problem to reapear again.

     To solve this, when a missing directory is detected in one or more
     bricks on lookup or opendir fops, only a partial self-heal is
     performed on it. A partial self-heal basically creates the
     directory but does not restore any additional information.

     This avoids that an 'ls' could repair the directory and cause the
     problem to happen again. With this change, output of 'ls' is
     always consistent. However, since the directory has been created
     in the brick, this allows any other operation on it (create new
     files, for example) to succeed on all bricks and not add additional
     work to the self-heal process.

     To force a self-heal of a directory, any other operation must be
     done on it. For example a getxattr.

     With these changes, the correct healing procedure that would avoid
     inconsistent directory browsing consists on a post-order traversal
     of directoriesi being healed. This way, the directory contents will
     be healed before healing the directory itslef.

 * Additional changes to fix self-heal errors

     - Don't use fop->fd to decide between fd/loc.

         open, opendir and create have an fd, but the correct data is in
         loc.

     - Fix incorrect management of bad bricks per inode/fd.

     - Fix incorrect selection of fop's target bricks when there are bad
       bricks involved.

     - Improved ec_loc_parent() to always return a parent loc as
       complete as possible.

This is a backport of http://review.gluster.org/8916/

Change-Id: Iaf3df174d7857da57d4a87b4a8740a7048b366ad
BUG: 1149727
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/8946
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System

glusterd: make bricks respect 'transport.socket.bind-address'

2014-10-21T17:15:58+00:00

When GlusterD starts the brick processes, these will listen on all
interfaces. When the 'transport.socket.bind-address' option is set in
glusterd.vol, the brick processes should only listen on the specified
hostname or IP-address.

Cherry picked from commit 430b874c4f1a171c106a9e1e6507e14e79805a1d:
> Change-Id: I8e7d1f294904081137c23f3446261329d0d13bba
> BUG: 1149863
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/8910
> Tested-by: Gluster Build System 
> Reviewed-by: Krishnan Parthasarathi 
> Tested-by: Krishnan Parthasarathi 

Change-Id: I8e7d1f294904081137c23f3446261329d0d13bba
BUG: 1151745
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/8951
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur