glusterfs.git/xlators/cluster/dht/src, branch v3.9.0rc2

cluster/dht: Incorrect volname in rebalance events

2016-10-26T02:57:10+00:00

The rebalance event code was using strtok to parse the
volume name which is incorrect.
Reworked the code to get the correct volume name using
strstr.

> Change-Id: Ib5f3305a34e6bf1ecfef677d87c5aff96bdeb0e6
> BUG: 1388010
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/15712
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
(cherry picked from commit f4efbf0e3092bd2181f62be1e1f30f202678c866)

Change-Id: Ia22d6a8ce23e9dcab8281b4995277cbb3a196fda
BUG: 1388563
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15725
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

dht/events: Added rebalance events

2016-09-17T10:51:48+00:00

The rebalance process will now send an event when it is
complete.
Also fixed a problem where the run-time was not always
set causing spurious rebalance failure events to be sent.

> Change-Id: Ib445171c78c9560940022bca20c887d31a9bb1ca
> BUG: 1371874
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/15501
> Reviewed-by: Raghavendra G 
> Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit e872917dbf4563703b8f396d7d30f8168ed646cd)

Change-Id: Ia24038c04548a98aac899df10414585b988caa76
BUG: 1376477
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15520
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

cluster/tier: add tiering events

2016-09-09T05:48:51+00:00

Add events for:
* tier attach and detach
* tier pause and resume
* tier rising and dropping hi and lo watermarks

Update eventskeygen.py with tiering events.
Update cli help with:
* attach: add optional force argument
* detach: make force available as non-optional argument on its own

> Reviewed-on: http://review.gluster.org/15232
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Dan Lambright 
> Tested-by: Dan Lambright 

(cherry picked from commit cd23242b8953040b231f402c334f16520ac8029f)

Change-Id: I43990d3a8742151a4a7889bafa19cb572fe661bd
BUG: 1374324
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/15431
Tested-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

dht, md-cache, upcall: Add invalidation of IATT when the layout changes

2016-08-31T06:08:54+00:00

Issue:
dht_layout is built as a part of lookup only. The layout can be
modified by rebalance process. Since every IO fop is preceded
by a lookup, there are very less issues of stale layout. But
with enhancements of aggressive caching of stats in md-cache,
the lookup will reduce and expose the stale layout issue often.

Solution:
Since stale layout is already an issue on dht, there is already
a plan to fix this at the dht layer, but this fix is not currently
planned for any release. Until this fix comes out, we can have
a workaround where, the upcall will send a notification to md-cache
when a layout xattr is changed. As a part of layout change notification
the existing cache is invalidated and the next lookup will fetch the
latest layout.

This is not a foolproof solution as the window between the layout change
and the next lookup(after invalidation of stat), where there will be stale
layout. But until the final fix comes in, this reduces the stale layout
window.

Change-Id: Iacf871a38b35880c1fc0bc68fe7ce291265e71d4
BUG: 1369638
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15300
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

dht/tiering: fix unused variable warnings/errors

2016-08-30T18:45:03+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: I367a737570dd7d2f6cc25f4bf4299d31bb6826aa
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15242
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Dan Lambright

dht: Implement ipc fop

2016-08-27T12:48:36+00:00

ipc is used by md-cache to communicate the list of xattrs that
it is caching, to the upcall xlator. Hence implement this in
dht, such that it winds to all the bricks if the ipc op is
GF_IPC_MDC_TARGET_UPCALL. The ips should not fail if any of
the bricks is down, as md-cache will replay the ipc late when
the brick comes back up.

Change-Id: Ica551a550c04cbb1240c0d211fe831c2e5eb6017
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15225
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

quotad: fix potential buffer overflows

2016-08-25T12:18:09+00:00

This converts sprintf to gf_asprintf in following components:                                                                                                          * quotad.c
* dht
* afr
* protocol/client
* rpc/rpc-lib
* rpc/rpc-transport

Change-Id: If8a267bab3d91003bdef3a92664077a0136745ee
BUG: 1332073
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/14102
Tested-by: Manikandan Selvaganesh 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Manikandan Selvaganesh

cluster/tier: dont promote if estimated block consumption > hi watermark

2016-07-29T14:14:37+00:00

Add test to fail promotion if estimated block consumption grows
beyond hi watermark.

Skip file migrations until next cycle if tier_get_fs_stat() fails
in tier_migrate_using_query_file()

Change-Id: Ice04572fa739c09109c4433e65965197482a7beb
BUG: 1349284
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/14780
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: mohammed rafi  kc 
Reviewed-by: Dan Lambright

dht/rebalance: allocate migrator thread pool dynamically

2016-07-28T12:46:30+00:00

Problems: The maximum number of migratior threads created was static set
to "40". And the number of these threads get created in rebalance depends
on the number of cores user has. If the number of cores exceeds 40, a 
crash or memory corruption can be seen.

Fix: Make the migratior thread pool dynamic.

Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
BUG: 1359711
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15000
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

cluster/tier: fix statfs for dht/tiered volumes

2016-07-27T13:03:48+00:00

Return the correct size of the tiered volume in statfs. It should
be the size of the cold tier, not the sum of the hot and cold tier,
because the hot tier is a cache and not an extension of the volume's
capacity. The number of free blocks, etc is the cold tier's capacity
subtracted by the sum of utilization on the hot and cold tiers. Note
if both tiers are part of the same file system this must be accounted
for as well.

The patch also fixes a pre-existing bug in the DHT/tier
translators. If statfs was taken on a file, the code only calculated
free space on the cached subvolume, not all subvolumes in the replica
group. With the fix, this is corrected, except in the case
where quota is used with the deem-statfs option set to "on".

Change-Id: I2b8bcb4511edf83f12130960aad0a609fcf8f513
BUG: 1339689
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/14536
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System