glusterfs.git/tests/bugs, branch v3.10.0rc0

core: run many bricks within one glusterfsd process

2017-02-02T00:54:58+00:00

This patch adds support for multiple brick translator stacks running in
a single brick server process.  This reduces our per-brick memory usage
by approximately 3x, and our appetite for TCP ports even more.  It also
creates potential to avoid process/thread thrashing, and to improve QoS
by scheduling more carefully across the bricks, but realizing that
potential will require further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global
option.  By default it's off, and bricks are started in separate
processes as before.  If multiplexing is enabled, then *compatible*
bricks (mostly those with the same transport options) will be started in
the same process.

Backport of:
> Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/14763

Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8
BUG: 1418091
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16496
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

md-cache: Cache security.ima xattrs

2017-01-30T14:25:03+00:00

Backport of http://review.gluster.org/16296

From kernel version 3.X or greater, creating of a file
results in removexattr call on security.ima xattr. But
this xattr is not set on the file unless IMA feature
is active. With this patch, removxattr call returns
ENODATA if it is not found in the cache.

> Change-Id: I8136096598a983aebc09901945eba1db1b2f93c9
> Signed-off-by: Poornima G 
> Reviewed-on: http://review.gluster.org/16296
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> (cherry picked from commit ac629e574935a8aed6526936bc83b1c6d295ae67)

Change-Id: I27abc23024c8fcf07389608df61ef6e64736d414
BUG: 1415918
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16460
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd: daemon restart logic should adhere server side quorum

2017-01-30T14:13:52+00:00

Just like brick processes, other daemon services should also follow the same
logic of quorum checks to see if a particular service needs to come up if
glusterd is restarted or the incoming friend add/update request is received
(in glusterd_restart_bricks () function)

>Reviewed-on: https://review.gluster.org/15626
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Prashanth Pai 

Change-Id: I54a1fbdaa1571cc45eed627181b81463fead47a3
BUG: 1417042
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16472
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Samikshan Bairagya 
Reviewed-by: Prashanth Pai

features/trash: Create trash directory only when it is enabled

2017-01-23T16:07:29+00:00

Previously trash directory was being created as part of volume
start operation. And also the user/admin could not delete this
directory from volume even if it is not needed. This patch will
fix the same. From now onwards creation and enforcement on trash
directory will come into pictrure only when trash translator is
enabled. Similarly exact same behaviour is reflected on internal-op
directory inside trash directory.

Upstream reference :
>Change-Id: I3e58316a7b299a691885e458c960438bec2220fb
>BUG: 1264849
>Signed-off-by: Jiffin Tony Thottan 
>Reviewed-on: http://review.gluster.org/12256
>Smoke: Gluster Build System 
>Tested-by: Anoop C S 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Anoop C S 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit 07b9853ad0c92b341be33a6cd632013c416221c8)

Change-Id: I3e58316a7b299a691885e458c960438bec2220fb
BUG: 1415581
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: https://review.gluster.org/16454
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Shyamsundar Ranganathan

glusterd: bypass add-brick validation with force

2017-01-19T03:50:33+00:00

Commit c916a2f added a validation to restrict add-brick operation if a
replica configuration is changed and any of the bricks belonging to the
volume is down. However we should bypass this validation with a force
option if users really want to have add-brick to go through at the sake
of the corner cases of data loss issue.

The original problem of add-brick getting failed when layout is not set
will still be a problem with a force option as the issue has to be taken
care in the DHT layer.

Change-Id: I0ed3df91ea712f77674eb8afc6fdfa577f25a7bb
BUG: 1406411
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/16358
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System

tier : Tier as a service

2017-01-17T04:49:47+00:00

tierd is implemented by separating from rebalance process.

The commands affected:

1) Attach tier will trigger this process instead of old one
2) tier start and tier start force will also trigger this process.
3) volume status [tier] will show tier daemon as a process instead
of task and normal tier status and tier detach status works.
4) tier stop implemented.
5) detach tier implemented separately along with new detach tier
status
6) volume tier volname status will work using the changes.
7) volume set works

This patch has separated the tier translator from the legacy
DHT rebalance code. It now sends the RPCs from the CLI
to glusterd separate to the DHT rebalance code.
The daemon is now a service, similar to the snapshot daemon,
and can be viewed using the volume status command.

The code for the validation and commit phase are the same
as the earlier tier validation code in DHT rebalance.

The “brickop” phase has been changed so that the status
command can use this framework.

The service management framework is now used.
DHT rebalance does not use this framework.

This service framework takes care of :

*) spawning the daemon, killing it and other such processes.
*) volume set options , which are written on the volfile.
*) restart and reconfigure functions. Restart is to restart
the daemon at two points
        1)after gluster goes down and comes up.
        2) to stop detach tier.
*) reconfigure is used to make immediate volfile changes.
By doing this, we don’t restart the daemon.
it has the code to rewrite the volfile for topological
changes too (which comes into place during add and remove brick).

With this patch the log, pid, and volfile are separated
and put into respective directories.

Change-Id: I3681d0d66894714b55aa02ca2a30ac000362a399
BUG: 1313838
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/13365
Smoke: Gluster Build System 
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Atin Mukherjee

tests/distribute: remove bug-1063230.t

2017-01-13T18:57:37+00:00

bug-1063230.t does not add value and has diverged
from the original BZ it was supposed to test.

Change-Id: I13ae1c68c276233dd53548d1333e3eb4b936785d
BUG: 1412467
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/16379
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur

posix: make sure atime and mtime are set when calling lutimes()

2017-01-06T17:33:10+00:00

When overwriting an existing file with O_TRUNC, the 'atime' was set to
0, meaning the Epoch (01-Jan-1970 UTC). However, the 'mtime' gets
updated correcty.

In case 'atime' or 'mtime' is not passed in the 'struct iatt', the time
values passed to the systemcall are taken from the current values are
returned by lstat().

Change-Id: I7021b7161dcd6c9a3e515d98f6d4847533c434b3
BUG: 1401777
Reported-by: Eivind Sarto 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/16034
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra Bhat 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

glusterd: Fail add-brick on replica count change, if brick is down

2017-01-06T09:19:38+00:00

Problem:
1. Have a replica 2 volume with bricks b1 and b2
2. Before setting the layout, b1 goes down
3. Set the layout write some data, which gets populated on b2
4. b2 goes down then b1 comes up
5. Add another brick b3, and heal will take place from b1 to b3, which
   basically have no data
6. Write some data. Both b1 and b3 will mark b2 for pending writes
7. b1 goes down, and b2 comes up
8. b2 gets heald from b1. During heal it removes the data which is already
   in b2, considering that as stale data. This leads to data loss.

Solution:
1. In glusterd stage-op, while adding bricks, check whether the replica
   count is being increased
2. If yes, then check whether any of the bricks are down at that time
3. If yes, then fail the add-brick to avoid such data loss
4. Else continue the normal operation.

This check will work enen when we convert plain distribute volume to replicate

Test:
1. Create a replica 2 volume
2. Kill one brick from the volume
3. Try adding a brick to the volume
4. It should fail with all bricks are not up error
5. Cretae a distribute volume and kill one of the brick
6. Try to convert it to replicate volume, by adding bricks.
7. This should also fail.

Change-Id: I9c8d2ab104263e4206814c94c19212ab914ed07c
BUG: 1406411
Signed-off-by: karthik-us 
Reviewed-on: http://review.gluster.org/16330
Tested-by: Ravishankar N 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Atin Mukherjee

tests: fix tests/bugs/glusterd/bug-913555.t spurious failures

2017-01-02T05:37:35+00:00

Mainly replaced EXPECT instances with EXPECT_WITHIN

Change-Id: If48f444f6b2ba6713fdc5e31ff3a642092e62ada
BUG: 1408758
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/16289
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Raghavendra G