glusterfs.git/tests/bugs/glusterd, branch v3.11dev

glusterd: bypass add-brick validation with force

2017-01-19T03:50:33+00:00

Commit c916a2f added a validation to restrict add-brick operation if a
replica configuration is changed and any of the bricks belonging to the
volume is down. However we should bypass this validation with a force
option if users really want to have add-brick to go through at the sake
of the corner cases of data loss issue.

The original problem of add-brick getting failed when layout is not set
will still be a problem with a force option as the issue has to be taken
care in the DHT layer.

Change-Id: I0ed3df91ea712f77674eb8afc6fdfa577f25a7bb
BUG: 1406411
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/16358
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System

tier : Tier as a service

2017-01-17T04:49:47+00:00

tierd is implemented by separating from rebalance process.

The commands affected:

1) Attach tier will trigger this process instead of old one
2) tier start and tier start force will also trigger this process.
3) volume status [tier] will show tier daemon as a process instead
of task and normal tier status and tier detach status works.
4) tier stop implemented.
5) detach tier implemented separately along with new detach tier
status
6) volume tier volname status will work using the changes.
7) volume set works

This patch has separated the tier translator from the legacy
DHT rebalance code. It now sends the RPCs from the CLI
to glusterd separate to the DHT rebalance code.
The daemon is now a service, similar to the snapshot daemon,
and can be viewed using the volume status command.

The code for the validation and commit phase are the same
as the earlier tier validation code in DHT rebalance.

The “brickop” phase has been changed so that the status
command can use this framework.

The service management framework is now used.
DHT rebalance does not use this framework.

This service framework takes care of :

*) spawning the daemon, killing it and other such processes.
*) volume set options , which are written on the volfile.
*) restart and reconfigure functions. Restart is to restart
the daemon at two points
        1)after gluster goes down and comes up.
        2) to stop detach tier.
*) reconfigure is used to make immediate volfile changes.
By doing this, we don’t restart the daemon.
it has the code to rewrite the volfile for topological
changes too (which comes into place during add and remove brick).

With this patch the log, pid, and volfile are separated
and put into respective directories.

Change-Id: I3681d0d66894714b55aa02ca2a30ac000362a399
BUG: 1313838
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/13365
Smoke: Gluster Build System 
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Atin Mukherjee

glusterd: Fail add-brick on replica count change, if brick is down

2017-01-06T09:19:38+00:00

Problem:
1. Have a replica 2 volume with bricks b1 and b2
2. Before setting the layout, b1 goes down
3. Set the layout write some data, which gets populated on b2
4. b2 goes down then b1 comes up
5. Add another brick b3, and heal will take place from b1 to b3, which
   basically have no data
6. Write some data. Both b1 and b3 will mark b2 for pending writes
7. b1 goes down, and b2 comes up
8. b2 gets heald from b1. During heal it removes the data which is already
   in b2, considering that as stale data. This leads to data loss.

Solution:
1. In glusterd stage-op, while adding bricks, check whether the replica
   count is being increased
2. If yes, then check whether any of the bricks are down at that time
3. If yes, then fail the add-brick to avoid such data loss
4. Else continue the normal operation.

This check will work enen when we convert plain distribute volume to replicate

Test:
1. Create a replica 2 volume
2. Kill one brick from the volume
3. Try adding a brick to the volume
4. It should fail with all bricks are not up error
5. Cretae a distribute volume and kill one of the brick
6. Try to convert it to replicate volume, by adding bricks.
7. This should also fail.

Change-Id: I9c8d2ab104263e4206814c94c19212ab914ed07c
BUG: 1406411
Signed-off-by: karthik-us 
Reviewed-on: http://review.gluster.org/16330
Tested-by: Ravishankar N 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Atin Mukherjee

tests: fix tests/bugs/glusterd/bug-913555.t spurious failures

2017-01-02T05:37:35+00:00

Mainly replaced EXPECT instances with EXPECT_WITHIN

Change-Id: If48f444f6b2ba6713fdc5e31ff3a642092e62ada
BUG: 1408758
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/16289
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Raghavendra G

tests: fix bug-963541.t spurious failure

2016-09-12T02:32:00+00:00

wait for remove brick to complete before attempt for a commit.

Change-Id: I66ea6c48b6a69fe33d79f9d9080b6f2c1462578e
BUG: 1374993
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15457
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Fix volume restart issue upon glusterd restart

2016-08-17T09:53:20+00:00

http://review.gluster.org/#/c/14758/ introduces a check in
glusterd_restart_bricks that makes sure that if server quorum is
enabled and if the glusterd instance has been restarted, the bricks
do not get started. This prevents bricks which have been brought
down purposely, say for maintainence, from getting started
upon a glusterd restart. However this change introduced regression
for a situation that involves multiple volumes. The bricks from
the first volume get started, but then for the subsequent volumes
the bricks do not get started. This patch fixes that by setting
the value of conf->restart_done to _gf_true only after bricks are
started correctly for all volumes.

Change-Id: I2c685b43207df2a583ca890ec54dcccf109d22c3
BUG: 1367478
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/15183
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd : skip non directories inside /var/lib/glusterd/vols

2016-08-08T14:23:50+00:00

Right now glusterd won't come up if vols directory contains an invalid entry.
Instead of doing that with this change a message will be logged and then skip
that entry

Change-Id: I665b5c35291b059cf054622da0eec4db44ec5f68
BUG: 1318591
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/13764
Reviewed-by: Prashanth Pai 
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

tests: fix spurious failure in tests/bugs/glusterd/bug-1089668.t

2016-08-04T15:44:34+00:00

Instead of rebalance stop, its always better to wait for rebalance to complete
as the former doesn't have any purpose.

Change-Id: Ia1bc2a34d937a0a96543bebd257dcda619f12474
BUG: 1363948
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15085
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

glusterd: clean up old port and allocate new one on every restart

2016-08-04T04:43:34+00:00

GlusterD as of now was blindly assuming that the brick port which was already
allocated would be available to be reused and that assumption is absolutely
wrong.

Solution : On first attempt, we thought GlusterD should check if the already
allocated brick ports are free, if not allocate new port and pass it to the
daemon. But with that approach there is a possibility that if PMAP_SIGNOUT is
missed out, the stale port will be given back to the clients where connection
will keep on failing. Now given the port allocation always start from base_port,
if everytime a new port has to be allocated for the daemons, the port range will
still be under control. So this fix tries to clean up old port using
pmap_registry_remove () if any and then goes for pmap_registry_alloc ()

Change-Id: If54a055d01ab0cbc06589dc1191d8fc52eb2c84f
BUG: 1221623
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15005
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Avra Sengupta

afr: some coverity fixes

2016-07-26T17:06:52+00:00

Thanks to Krutika for a cleaner way to track inode refs in
afr_set_split_brain_choice().

Change-Id: I2d968d05b815ad764b7e3f8aa9ad95a792b3c1df
BUG: 1355604
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/14895
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System