glusterfs.git/xlators/cluster/dht/src/dht-shared.c, branch v3.8rc0

glusterd: volume set changes for lock migration

2016-05-02T01:05:03+00:00

Change-Id: I48c6f9cdda47503615ba65882acd5eedf0a70c89
BUG: 1326085
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14024
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/tier: add tunable to migrate files based on size

2016-03-17T06:12:04+00:00

This fix adds a paramater "tier-max_promote_size" to control wether
a file is migrated or not based on its size. By default the value
is 0, meaning all files are migrated. If set to a non-zero
value, files larger than the parameter won't be moved
in tiered volumes.

Change-Id: Ia6b88e9b2508935bef500d956f9192e59670fe00
BUG: 1313495
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/13570
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Joseph Fernandes

tier/dht : Default value for demote-freq, max files and mb

2016-01-26T17:16:05+00:00

Default value for tier-demote-frequency is 3600 sec to avoid
frequent demotions.

Default value for tier-max-mb is 4000 mb

Default value for tier-max-files is 10000 files

Change-Id: Ie60951c478a7462c425059699ab82511aa13fa0a
BUG: 1300412
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/13270
Smoke: Gluster Build System 
Tested-by: Dan Lambright 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

cluster/tier: fix loading tier.so into glusterd

2015-12-03T18:01:07+00:00

glusterd occasionally loads shared libraries of translators. This
failed for tiering due to a reference to dht_methods which is defined
as a global variable which is not necessary.
The global variable has been removed and this is now a member of
dht_conf and is now initialised in the *_init calls.

Change-Id: Ifa0a21e3962b5cd8d9b927ef1d087d3b25312953
BUG: 1287842
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/12863
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

cluster/tier: add pause tier for snapshots

2015-10-21T22:44:35+00:00

Snaps of tiered volumes cannot handle files undergoing migration.
We implement a helper mechanism to "pause" migration. Any files
undergoing migration are aborted. Clean up is done to remove
sticky bits and data at the destination. Migration is restarted
after snap completes.

For testing an internal switch is added. It is not exposed externally.

gluster volume set vol1 tier-pause [true|false]

Change-Id: Ia85bbf89ac142e9b7e73fcbef98bb9da86097799
BUG: 1267950
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/12304
Reviewed-by: N Balachandran 
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System

cluster/tier: add watermarks and policy driver

2015-10-10T13:00:04+00:00

This fix introduces infrastructure to support different
policies for promotion and demotion.

Currently the tier feature automatically promotes and demotes
files periodically based on access. This is good for testing
but too stringent for most real workloads. It makes it
difficult to fully utilize a hot tier- data will be demoted
before it is touched- its unlikely a 100GB hot SSD will have
all its data touched in a window of time.

A new parameter "mode" allows the user to pick promotion/demotion
polcies.

The "test mode" will be used for *.t and other general testing.
This is the current mechanism.

The "cache mode" introduces watermarks. The watermarks
represent levels of data residing on the hot tier.

"cache mode" policy:

The % the hot tier is full is called P.

Do not promote or demote more than D MB or F files.

A random number [0-100] is called R.

Rules for migration:

if (P < watermark_low) don't demote, always promote.

if (P >= watermark_low) && (P < watermark_hi) demote if R < P; promote if R > P.

if (P > watermark_hi) always demote, don't promote.

gluster volume set {vol} cluster.watermark-hi %
gluster volume set {vol} cluster.watermark-low %
gluster volume set {vol} cluster.tier-max-mb {D}
gluster volume set {vol} cluster.tier-max-files {F}
gluster volume set {vol} cluster.tier-mode {test|cache}

Change-Id: I157f19667ec95aa1d53406041c1e3b073be127c2
BUG: 1257911
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/12039
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd, dht: volume set for use-readdirp in dht

2015-10-01T07:09:02+00:00

Change-Id: Icab246b1d02808864d878d949fa56f9f889b538a
BUG: 1265677
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/12221
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Kaushal M

dht: Adding log messages to the new logging framework

2015-06-24T05:15:10+00:00

Change-Id: Ib3bb61c5223f409c23c68100f3fe884918d2dc3f
BUG: 1194640
Signed-off-by: arao 
Reviewed-on: http://review.gluster.org/10021
Reviewed-by: N Balachandran 
Reviewed-by: Joseph Fernandes
Tested-by: Joseph Fernandes
Reviewed-by: Dan Lambright 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

dht: Add lookup-optimize configuration option for DHT

2015-06-02T10:03:07+00:00

Currently with commit 4eaaf5 a mixed version cluster would
have issues if lookup-uhashed is set to auto, as older clients
would fail to validate the layouts if newer clients (i.e 3.7 or
upwards) create directories. Also, in a mixed version cluster
rebalance daemon would set commit hash for some subvolumes and
not for the others.

This commit fixes this problem by moving the enabling of the
functionality introduced in the above mentioned commit to a
new dht option. This option also has a op_version of 3_7_1
thereby preventing it from being set in a mixed version
cluster. It brings in the following changes,
- Option can be set only if min version of the cluster is
3.7.1 or more
- Rebalance and mkdir update the layout with the commit hashes
only if this option is set, hence ensuring rebalance works in a
mixed version cluster, and also directories created by newer
clients do not cause layout errors when read by older clients
- This option also supersedes lookup-unhased, to enable the
optimization for lookups more deterministic and not conflict
with lookup-unhashed settings.

Option added is cluster.lookup-optimize, which is a boolean.

Usage: # gluster volume set VOLNAME cluster.lookup-optimize on

Change-Id: Ifd1d4ce3f6438fcbcd60ffbfdbfb647355ea1ae0
BUG:  1222126
Signed-off-by: Shyam 
Reviewed-on: http://review.gluster.org/10797
Tested-by: NetBSD Build System 
Reviewed-by: Kaushal M 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

build: do not #include "config.h" in each file

2015-05-29T12:43:17+00:00

Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).

When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.

BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Pranith Kumar Karampuri