glusterfs.git/tests/tier.rc, branch v4.0dev

tier : Tier as a service

2017-01-17T04:49:47+00:00

tierd is implemented by separating from rebalance process.

The commands affected:

1) Attach tier will trigger this process instead of old one
2) tier start and tier start force will also trigger this process.
3) volume status [tier] will show tier daemon as a process instead
of task and normal tier status and tier detach status works.
4) tier stop implemented.
5) detach tier implemented separately along with new detach tier
status
6) volume tier volname status will work using the changes.
7) volume set works

This patch has separated the tier translator from the legacy
DHT rebalance code. It now sends the RPCs from the CLI
to glusterd separate to the DHT rebalance code.
The daemon is now a service, similar to the snapshot daemon,
and can be viewed using the volume status command.

The code for the validation and commit phase are the same
as the earlier tier validation code in DHT rebalance.

The “brickop” phase has been changed so that the status
command can use this framework.

The service management framework is now used.
DHT rebalance does not use this framework.

This service framework takes care of :

*) spawning the daemon, killing it and other such processes.
*) volume set options , which are written on the volfile.
*) restart and reconfigure functions. Restart is to restart
the daemon at two points
        1)after gluster goes down and comes up.
        2) to stop detach tier.
*) reconfigure is used to make immediate volfile changes.
By doing this, we don’t restart the daemon.
it has the code to rewrite the volfile for topological
changes too (which comes into place during add and remove brick).

With this patch the log, pid, and volfile are separated
and put into respective directories.

Change-Id: I3681d0d66894714b55aa02ca2a30ac000362a399
BUG: 1313838
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/13365
Smoke: Gluster Build System 
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Atin Mukherjee

tier/cli : printing a warning instead of skipping the node

2016-05-20T14:25:37+00:00

Problem: skipping the status of the nodes down creates confusion
to the user as one might see the status as completed for all nodes
and while performing detach commit, the operation will fail as the
node is down

Fix: Display a warning message

Note: When the last node is down (as per the peer list) then
warning message can't be displayed as the total number of peers
participating in the transaction is considered to be the total count.

Change-Id: Ib7afbd1b26df3378e4d537db06f41f5c105ad86e
BUG: 1324439
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/14347
Tested-by: hari gowtham 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
NetBSD-regression: NetBSD Build System

tier/dht : Attach tier fix layout to run in background

2016-03-28T16:17:59+00:00

1. Spawn a thread for background fix-layout for tier process.

2. Once the fix-layout is completed a marker xttr is set on the root of
volume to mark the completion of the background fixlayout, so that
even if the tier process is spawned again, fixlayout will not be
issued, if it was completed last time.

3. Please note that promotion of legacy files will happen eventually as
the ctr lookup heal in the fixlayout slowly heals the ctr db for legacy
files OR the ctr lookup heal happend due to a name lookup.

4. When a detach tier is successful in evacuation data from hot tier, we remove
the marker xattr is removed. So that next attach tier runs the background
tier fixlayout.

what is remaining ?
1. Instead of clearing the marker xattr of tiering fix layout at the end of detach start
clear it during detach commit. But the issue is detach commit is a glusterd operation
and the volume is not mounted in glusterd.
The reason we want to do it in detach commit is that if the admin wants to attach the
same tier again, then a background fixlayout will be triggered, which would not be needed.
2. Clearing the CTR DB of the cold bricks when there is a detach commit, as it will be having
entries which will be stale when the volume is used, with ctr off (ctr is switched off only when
we have detach commit.)

Change-Id: Ibe343572e95865325cd0eef4d0b976b626a3c0c5
BUG: 1313228
Signed-off-by: Joseph Fernandes
Reviewed-on: http://review.gluster.org/13491
Smoke: Gluster Build System
Tested-by: Joseph Fernandes
NetBSD-regression: NetBSD Build System
CentOS-regression: Gluster Build System
Reviewed-by: Dan Lambright

tests/glusterd: Parse the new time format

2016-02-26T01:43:52+00:00

A recent change in cli changed elapsed time format
that broke a test.

This patch will fix the issue with parsing.

Change-Id: I9a4a4b28f654cf2ac223e25abfc9df6570607d74
BUG: 1312036
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13524
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Raghavendra Talur

glusterd/rebalance: initialize defrag variable after glusterd restart

2016-02-23T05:42:03+00:00

During reblance restart after glusterd restarted, we are not
connecting to rebalance process from glusterd, because the
defrag variable in volinfo will be null.

Initializing the variable will connect the rpc

Change-Id: Id820cad6a3634a9fc976427fbe1c45844d3d4b9b
BUG: 1303028
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13319
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
CentOS-regression: Gluster Build System

tier/glusterd : Validation for frequency thresholds and record-counters

2015-12-01T23:39:34+00:00

1) if record-counters is set to off
   check if both the frequency thresholds are non-zero, then pop
   an error message, with volume set failed.
2) if record-counters is set to on
   check if both the frequency thresholds are zero, then pop
   an note, but volume set is not failed.
3) If any of the frequency thresholds are set to a non-zero value,
   switch record-counters on, if not already on
4) If both the frequency thresholds are set to zero,
   switch record-counters off, if not already off

NOTE: In this fix we have
1) removed unnecessary ctr vol set options.
2) changed  ctr_hardlink_heal_expire_period to ctr_lookupheal_link_timeout

Change-Id: Ie7ccfd3f6e021056905a79de5a3d8f199312f315
BUG: 1286346
Signed-off-by: Joseph Fernandes 
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/12780
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

cluster/tier correct promotion cycle calculation

2015-11-07T18:30:22+00:00

The tier translator should only choose candidate files for promotion
from the most recent cycle, not a multiple of the most recent cycles.
Otherwise user observed behavior can be inconsistent. Remove related
test in tier.t that is subject to race condition.

Change-Id: I9ad1523cac00f904097ce468efa6ddd515857024
BUG: 1275524
Signed-off-by: root 
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/12480
Reviewed-by: Joseph Fernandes
Tested-by: Gluster Build System

cluster/tier : Files skipped during tier query parsing

2015-11-03T14:23:16+00:00

The tier query parsing code was using fscanf to read each record.
As space is a delimiter for fscanf, filenames containing spaces
caused the parsing to return unexpected values causing various
issues in the tier process, including crashes due to buffer
 overflows.

Change-Id: Ife602cb7ecb158fccbc2c89e4d2959bd97098a87
BUG: 1276562
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/12469
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tests/tier: Move common functions to tier.rc

2015-10-22T02:47:39+00:00

Move common functions in tier .t files to tier.rc

Change-Id: Ibc312d987be9d93e7cc7fc47d0bf598bb1c944c2
BUG: 1272319
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/12404
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright