glusterfs.git/xlators/cluster/dht/src/dht-diskusage.c, branch v3.8rc0

dht: missleading indendentation, gcc-6

2016-01-06T02:07:37+00:00

gcc-6 now has -Wmisleading-indentation as part of -Wall.

compiling with gcc-6 gives this warning.

...
dht-diskusage.c: In function ‘dht_subvol_has_err’:
dht-diskusage.c:361:33: warning: statement is indented as if it were
guarded by... [-Wmisleading-indentation]
                                 goto out;
                                 ^~~~

dht-diskusage.c:358:25: note: ...this ‘if’ clause, but it is not
                         if (conf->decommissioned_bricks[i] &&
                         ^~
...

Inspection of the source shows that without braces the loop is
terminated prematurely.

Change-Id: Ica48a8c59ee5d0a206797827d7920259d33b47ec
BUG: 1295784
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13176
Reviewed-by: N Balachandran 
Reviewed-by: Shyamsundar Ranganathan 
Tested-by: Gluster Build System

dht: block/handle create op falling to decommissioned brick

2015-08-24T06:06:31+00:00

Problem:
Post remove-brick start till commit phase, the client layout
may not be in sync with disk layout because of lack of lookup.
Hence,a create call may fall on the decommissioned brick.

Solution:
Will acquire a lock on hashed subvol. So that a fix-layout or
selfheal can not step on layout while reading the layout.

Even if we read a layout before remove-brick fix-layout and the
file falls on the decommissioned brick, the file should be
migrated to a new brick as per the fix-layout.

Change-Id: If84a12ec34f981adb2b9b224e80f535cfe5bf9f2
BUG: 1232378
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/11260
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G

dht: Adding log messages to the new logging framework

2015-06-24T05:15:10+00:00

Change-Id: Ib3bb61c5223f409c23c68100f3fe884918d2dc3f
BUG: 1194640
Signed-off-by: arao 
Reviewed-on: http://review.gluster.org/10021
Reviewed-by: N Balachandran 
Reviewed-by: Joseph Fernandes
Tested-by: Joseph Fernandes
Reviewed-by: Dan Lambright 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

build: do not #include "config.h" in each file

2015-05-29T12:43:17+00:00

Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).

When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.

BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Pranith Kumar Karampuri

dht: suggest to add more bricks when min-free-disk is exceeded.

2015-03-18T11:33:01+00:00

Change-Id: I628fbd99c2478fcb8bb6e5be55e43467f25227bf
BUG: 1165870
Signed-off-by: Humble Devassy Chirammal 
Reviewed-on: http://review.gluster.org/9879
Tested-by: Gluster Build System 
Reviewed-by: Lalatendu Mohanty 
Reviewed-by: Kaleb KEITHLEY

dht: support heterogeneous brick sizes

2014-07-12T16:20:52+00:00

Calculation of layouts now considers the size of each brick, so that
smaller bricks don't get an "unfair" share of allocations and start
returning ENOSPC while the larger bricks still have plenty of space.

The observation has been made that some clients might get ENOTCONN when
trying to fetch disk-size information, and end up calculating layouts
differently.  The following meta-observations can be made.

(1) This scenario is extremely unlikely in configurations with AFR.

(2) The most likely consequence of this scenario is that some files will
be placed sub-optimally by the client with the obsolete (non-weighted)
layout.  They'll still be found anyway, so this isn't a show stopper.

(3) Without this patch it's *guaranteed* that some files will be placed
sub-optimally, because any layout that fails to account for brick sizes
is sub-optimal.

(4) We shouldn't be doing fix-layout from two nodes simultaneously
anyway.  That's inefficient at best.  Any instances of such behavior are
separate bugs, which should be fixed separately.

(5) In the most extreme edge case, two nodes doing weighted and
non-weighted layout fixes could race and end up creating an internally
inconsistent layout.  This condition is still transient; it will be
detected and repaired automatically the next time anyone fetches the
layout.  (If it's not that's also a preexisting bug that can show up in
other contexts.)

In conclusion, it's not the purpose of this patch to fix bugs elsewhere
in DHT.  Its purpose is to make life incrementally better for users who
add new hardware with larger disks etc. than the older equipment.  It's
only one part of an ongoing process to improve layout management and
repair, all the way up to support for multiple hash rings or tiering.

Change-Id: I05eb6f9eface9cdaf8622e0260c8c7f29020447f
BUG: 1114680
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/8093
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur

Cluster/DHT: New logging framework

2014-06-16T13:25:51+00:00

Moved all relevant DHT gf_log calls to the new logging
framework.

Change-Id: I3af3cfe0416e332774a6c4ff6a091d006c400af2
BUG: 1075611
Signed-off-by: Nithya Balachandran 
Reviewed-on: http://review.gluster.org/7929
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/dht: Fix min-free-disk calculations when quota-deem-statfs is on

2014-06-02T10:56:46+00:00

PROBLEM:

As part of file creation, DHT sends a statfs call to all of its
sub-volumes and expects in return the local space consumption and
availability on each one of them. This information is used by DHT to
ensure that atleast min-free-disk amount of space is left on each
sub-volume in the event that there ARE other sub-volumes with more
space available.
But when quota-deem-statfs is enabled, quota xlator on every brick
unwinds the statfs call with volume-wide consumption of disk space.
This leads to miscalculation in min-free-disk algo, thereby misleading
DHT at some point, into thinking all sub-volumes have equal available
space, in which case DHT keeps sending new file creates to subvol-0,
causing it to become 100% full at some point although there ARE other
subvols with ample space available.

FIX:

The fix is to make quota_statfs() behave as if quota xlator weren't
enabled, thereby making every brick return only its local consumption
and disk space availability.

Change-Id: I211371a1eddb220037bd36a128973938ea8124c2
BUG: 1099890
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/7845
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Vijay Bellur

cluster/dht: Ignore subvols with error in min-free-disk/inodes

2013-07-10T08:56:38+00:00

Currently when selecting a alternative subvolume when hashed
subvol has exceeded min-free-disk/inodes, we do not check if
layouts have errors (including decommissioning). This leads
to data being written to those subvolumes, and in case of
decommissioning, will lead to data loss.

Change-Id: Ie0c6cf4a29d7c53d8a6d8a8c1bd595cf58a0012a
BUG: 982919
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/5299
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/dht: Correct min_free_disk behaviour

2013-02-04T16:43:50+00:00

Problem:
Files were being created in subvol which had less than
min_free_disk available even in the cases where other
subvols with more space were available.

Solution:
Changed the logic to look for subvol which has more
space available.
In cases where all the subvols have lesser than
Min_free_disk available , the one with max space and
atleast one inode is available.

Known Issue: Cannot ensure that first file that is
created right after min-free-value is crossed on a
brick will get created in other brick because disk
usage stat takes some time to update in glusterprocess.
Will fix that as part of another bug.

Change-Id: If3ae0bf5a44f8739ce35b3ee3f191009ddd44455
BUG: 858488
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/4420
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati