glusterfs.git/tests, branch v3.7.7

cluster/afr: Fix data loss due to race between sh and ongoing write.

2016-01-29T01:43:33+00:00

        Backport of: http://review.gluster.org/#/c/13001/

Problem:

When IO is happening on a file and a brick goes down comes back up
during this time, protocol/client translator attempts reopening of the
fd on the gfid handle of the file. But if another client renames this
file while a brick was down && writes were in progress on it, once this
brick is back up, there can be a race between reopening of the fd and
entry self-heal replaying the effect of the rename() on the sink brick.
If the reopening of the fd happens first, the application's writes
continue to go into the data blocks associated with the gfid.
Now entry-self-heal deletes 'src' and creates 'dst' file on the sink,
marking dst as a 'newentry'.  Data self-heal is also completed on 'dst'
as a result and self-heal terminates. If at this point the application
is still writing to this fd, all writes on the file after self-heal
would go into the data blocks associated with this fd, which would be
lost once the fd is closed. The result - the 'dst' file on the source
and sink are not the same and there is no pending heal on the file,
leading to silent corruption on the sink.

Fix:

Leverage http://review.gluster.org/#/c/12816/ to ensure the gfid handle
path gets saved in .glusterfs/unlink until the fd is closed on the file.
During this time, when self-heal sends mknod() with gfid of the file,
do the following:
link() the gfid handle under .glusterfs/unlink to the new path to be
created in mknod() and
rename() the gfid handle to go back under .glusterfs/ab/cd/.


Change-Id: I5dc49c127ef0a1bf3cf4ce1b24610b1527f84d6f
BUG: 1293265
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/13036
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

tests/quota : fix failing test on auxiliary mount point

2016-01-28T05:16:06+00:00

In test file tests/bugs/quota/bug-1049323.t, test "EXPECT "0" get_aux"
fails in Fedora.

In get_aux function we grep for "/var/run/gluster/" to check
if auxiliary mount point is created and we return 0 on success else we
return 1. In fedora, auxiliary mount point is created on
"/run/gluster/". So it fails on Fedora.

The patch fixes it by just grepping for "/run/gluster/".

        Backport of http://review.gluster.org/#/c/13228/

> Change-Id: Icb59395df4a98109eaa8199cbdbdedcd1cbef27a
> BUG: 1297740
> Signed-off-by: Manikandan Selvaganesh 
> Reviewed-on: http://review.gluster.org/13228
> Reviewed-by: Niels de Vos 
> Reviewed-by: Raghavendra Talur 
> Tested-by: NetBSD Build System 

Change-Id: Icb59395df4a98109eaa8199cbdbdedcd1cbef27a
BUG: 1300600
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/13273
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur

features/bitrot: Fail node-uuid getxattr if file is marked bad

2016-01-27T11:46:01+00:00

If xattr is node-uuid and the inode is marked bad, fail getxattr
and fgetxattr with EIO. Returning EIO would result in AFR to
choose correct node-uuid coresponding to the subvolume where
the good copy of the file resides.

BUG: 1296795
Change-Id: I3f8dc807794f9a82867807e7c4c73ded6c64fd8a
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13116
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/13194
Tested-by: Venky Shankar 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

features/bitrot: add check for corrupted object in f{stat}

2016-01-27T02:44:58+00:00

         Backport of http://review.gluster.org/13120

Check for corrupted objects is done bt bitrot stub component
for data operations and such fops are denied processing by
returning EIO. These checks were not done for operations such
as get/set extended attribute, stat and the likes - IOW, stub
only blocked pure data operations.

However, its necessary to have these checks for certain other
fops, most importantly stat (and fstat). This is due to the
fact that clients could possibly get stale stat information
(such as size, {a,c,m}time) resulting in incorrect operation
of the application that rely on these fields. Note that, the
data that replication would take care of fetching good (and
correct) data, but the staleness of stat information could
lead to data inconsistencies (e.g., rebalance, tier).

Change-Id: I5a22780373b182a13f8d2c4ca6b7d9aa0ffbfca3
BUG: 1297213
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/13276
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

storage/posix: Implement .unlink directory

2016-01-25T11:37:10+00:00

Problem: For EC volume, If a file descriptor is open and
file has been unlinked, any further write on that fd will
fail. When a write request comes, EC internally reads some
blocks using anonymous fd. This read will fail as the file
has already been unlinked.

Solution: To solve this issue, we are using .unlink directory
to keep track of unlinked file. If a file is to be unlinked
while its fd is open, move this to .unlink directory and unlink
it from .glusterfs and real path. Once all the fd will be closed,
remove this entry form .unlink directory.

master -
http://review.gluster.org/#/c/12816/

Change-Id: I8344edb0d340bdb883dc46458c16edbc336916b9
BUG: 1291557
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/12968
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

tests: Fix sparse-file-self-heal.t

2016-01-25T11:36:32+00:00

Backport of http://review.gluster.org/13233
Psuedo Problem:
https://build.gluster.org/job/rackspace-regression-2GB-triggered/16682/consoleFull

The 'zeroedfile' disk usage comparision which is failing in this .t file
fails so only on XFS. The test passes when the backend is on a s̶a̶n̶e̶r̶
different filesystem like EXT4 or BTRFS. This is due to the speculative
preallocation in XFS which can reserve different disk space on different
XFS mounts for the same version and same file operation. See BZ 1277992
for an example of XFS behaviour.

Fix:
Don't compare the disk usage of the file on the bricks of the replica:
instead, check that the disk space consumed is atleast equal to the size
of the file.

Also remove sparse-file-self-heal.t from is_bad_test()

Change-Id: If43f59549136ebf91f17ff9d958954b3587afe56
BUG: 1300210
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13265
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

features/shard: Implement fallocate FOP

2016-01-22T05:31:59+00:00

        Backport of: http://review.gluster.org/13196


Change-Id: Iab0c41319af42210c871a3ed6cf52a987c5d88d7
BUG: 1299712
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/13259
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

NetBSD regression reliability: properly cleanup loopback devices

2016-01-20T10:08:24+00:00

When a loopback device is configured and we forcibly unmount the
filesystem containing the backing store, further vnconfig -l will
complain "vnconfig: VNDIOCGET: Bad file descriptor" causing
failures.

We fix this by iterating on all loopback devices available in /dev,
testing for this condition and manually unconfiguring when it
happens.

Backport of: I17b956a8ed28a7767f2d0dda83b93c523d3238c2

BUG: 1212676
Change-Id: I9d6fdd859572b653e1144c7d9d98a977cc3e255a
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/13205
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System

all: reduce "inline" usage

2016-01-18T09:02:34+00:00

There are three kinds of inline functions: plain inline, extern inline,
and static inline.  All three have been removed from .c files, except
those in "contrib" which aren't our problem.  Inlines in .h files, which
are overwhelmingly "static inline" already, have generally been left
alone.  Over time we should be able to "lower" these into .c files, but
that has to be done in a case-by-case fashion requiring more manual
effort.  This part was easy to do automatically without (as far as I can
tell) any ill effect.

In the process, several pieces of dead code were flagged by the
compiler, and were removed.

backport of Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155,
http://review.gluster.org/11769, BUG: 1245331

Change-Id: Iba1efb0bc578ea4a5e9bf76b7bd93dc1be9eba44
BUG: 1283302
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/12646
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

glusterd: import/export brickinfo->uuid

2016-01-14T09:35:05+00:00

Backport of http://review.gluster.org/#/c/13047/ 

    Given a two node cluster with node N1 & N2, if a dummy node N3 is peer probed, the
    probed node N3  goes for importing volumes from the probing node (N1), but
    it still doesn't have information about the other node (N2) about its membership
    (since peer update happens post volume updates) and hence fail to update its
    brick's uuid. Post that even though N2 updates N3 about its membership the
    brick's uuid was never generated. Now as a consequence when N3 initiates a
    detach of N2, it checks whether the node to be detached has any bricks
    configured by its respective uuid which is NULL in this case and hence it goes
    ahead and removes the peer which ideally it shouldn't have (refer to
    glusterd_friend_contains_vol_bricks () for the logic)
    
    Fix is to export brick's uuid and import it at the probed node instead of
    resolving it.

Change-Id: I2d88c72175347550a45ab12aff0ae248e56baa87
BUG: 1297305
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/13047
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Gaurav Kumar Garg 
Reviewed-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/13210