glusterfs.git/xlators/cluster/ec/src, branch v3.7.2

cluster/ec: Avoid parallel executions of the same state machine

2015-06-19T06:35:15+00:00

        Backport of http://review.gluster.org/11317

In very rare circumstances it was possible that a subfop started
by another fop could finish fast enough to cause that two or more
instances of the same state machine be executing at the same time.

Change-Id: I319924a18bd3f88115e751a66f8f4560435e0e0e
BUG: 1233484
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11319
Tested-by: Gluster Build System 
Tested-by: Pranith Kumar Karampuri 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Wind unlock fops at all cost

2015-06-19T02:49:35+00:00

        Backport of http://review.gluster.org/11152

Problem:
While files are being created if more than redundancy number of bricks
go down, then unlock for these fops do not go to the bricks. This will
lead to stale locks leading to hangs.

Fix:
Wind unlock fops at all costs.

BUG: 1230350
Change-Id: I3312b0fe1694ad02af5307bcbaf233ac63058846
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11166
Tested-by: Gluster Build System

cluster/ec: Prevent double unwind

2015-06-18T12:47:15+00:00

        Backport of http://review.gluster.com/11111

Problem:
1) ec_access/ec_readlink_/ec_readdir[p] _cbks are trying to recover only from
ENOTCONN.
2) When the fop succeeds it unwinds right away. But when its
ec_fop_manager resumes, if the number of bricks that are up is less than
ec->fragments, the the state machine will resume with -EC_STATE_REPORT which
unwinds again. This will lead to crashes.

Fix:
- If fop fails retry on other subvols, as ESTALE/ENOENT/EBADFD etc are also
recoverable.
- unwind success/failure in _cbks

BUG: 1229331
Change-Id: I7510984a237761efba65e872313a8ede8b7543e5
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11128
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Prevent Null dereference in dht-rename

2015-06-16T08:29:43+00:00

        Backport of http://review.gluster.com/11178

BUG: 1230653
Change-Id: I708d4d84b1341fb7cb515808836721735dc63f14
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11179
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: EC_XATTR_DIRTY doesn't come in response

2015-06-07T05:36:07+00:00

         Backport of http://review.gluster.com/11078

Problem:
ec_update_size_version expects all the keys it did xattrop with to come in
response so that it can set the values again in ec_update_size_version_done.
But EC_XATTR_DIRTY is not combined so the value won't be present in the
response. So ctx->post/pre_dirty are not updated in
ec_update_size_version_done. So these values are still non-zero. When
ec_unlock_now is called as part of flush's unlock phase it again tries to
perform same xattrop for EC_XATTR_DIRTY. But ec_update_size_version is not
expected to be called in unlock phase of flush because ec_flush_size_version
should have reset everything to zero and unlock is never invoked from
ec_update_size_version_done for flush/fsync/fsyncdir. This leads to stale lock
which leads to hang.

Fix:
EC_XATTR_DIRTY is removed in ex_xattrop_cbk and is never combined with other
answers. So remove handling of this in the response.

BUG: 1228160
Change-Id: I657efca6e706e7acb541f98f526943f67562da9f
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11084
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

cluster/dht: Fix dht_setxattr to follow files under migration

2015-06-04T12:03:59+00:00

If a file is under migration, then any xattrs created on it
are lost post migration of the file. This is because
the xattrs are set only on the cached subvol of the source
and as the source is under migration, it becomes a linkto file
post migration.

Change-Id: Ib8e233b519cf954e7723c6e26b38fa8f9b8c85c0
BUG: 1225839
Signed-off-by: Nithya Balachandran 
Reviewed-on: http://review.gluster.org/10968
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

cluster/ec: Fix incorrect check for iatt differences

2015-06-03T10:35:15+00:00

        Backport of http://review.gluster.org/11018

A previous patch (http://review.gluster.org/10974) introduced a
bug that caused that some metadata differences could not be
detected in some circumstances. This could cause that self-heal
is not triggered and the file not repaired.

We also need to consider all differences for lookup requests, even
if there isn't any lock. Special handling of differences in lookup
is already done in lookup specific code.

Change-Id: I88e3678634d848a89f428a37209c31464e42cf8b
BUG: 1225796
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11027
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Ignore differences in non locked inodes

2015-05-30T12:35:44+00:00

        Backport of http://review.gluster.org/10974

When ec combines iatt structures from multiple bricks, it checks
for equality in important fields. This is ok for iatt related to
inodes involved in the operation that have been locked before
starting execution. However some fops return iatt information
from other inodes. For example a rename locks source and destination
parent directories, but it also returns an iatt from the entry
itself.

In these cases we ignore differences in some fields to avoid false
detection of inconsistencies and trigger unnecessary self-heals.

Another issue is solved in this patch that caused that the real
size of the file stored into the inode context was lost during
self-heal.

BUG: 1225796
Change-Id: I29f328a7b4895368ded859f3bae0359436c3588f
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/10983
Tested-by: Gluster Build System

cluster/ec: Fix all EIO errors in EC

2015-05-28T11:12:06+00:00

        Backport of http://review.gluster.org/10770
        Backport of http://review.gluster.org/10806
        Backport of http://review.gluster.org/10787
        Backport of http://review.gluster.org/10868
        Backport of http://review.gluster.com/10852

 - When a blocking lock is requested, lock request is succeeded even when
ec->fragment number of locks are acquired successfully in non-blocking locking
phase. This will lead to fop succeeding only on the bricks where the locks are
acquired, leading to the necessity of self-heals. To prevent these un-necessary
self-heals, if the remaining locks fail with EAGAIN in non-blocking lock phase
try blocking locking phase instead.

 -  Handle lookup failures while op in progress

 - cluster/ec: Correctly cleanup delayed locks
When a delayed lock is pending, a graph switch doesn't correctly
terminate it. This means that the update of version and size xattrs
is lost, causing EIO errors. This patch handles GF_EVENT_PARENT_DOWN
event to correctly finish pending udpdates before completing the
graph switch.

 - Fix use after free crash
ec_heal creates ec_fop_data but doesn't run ec_manager. ec_fop_data_allocate
adds this fop to ec->pending_fops, because ec_manager is not run on this heal
fop it is never removed from ec->pending_fops. When it is accessed after free
it leads to crash. It is better to not to add HEAL fops to ec->pending_fops
because we don't want graph switch to hang the mount because of a BIG
file/directory heal.

- Forced unlock when lock contention is detected
EC uses an eager lock mechanism to optimize multiple read/write
requests on the same entry or inode. This increases performance
but can have adverse results when other clients try to access the
same entry/inode. To solve this, this patch adds a functionality
to detect when this happens and force an earlier release to not
block other clients.

The method consists on requesting GF_GLUSTERFS_INODELK_COUNT and
GF_GLUSTERFS_ENTRYLK_COUNT for all fops that take a lock. When this
count is greater than one, the lock is marked to be released. All
fops already waiting for this lock will be executed normally before
releasing the lock, but new requests that also require it will be
blocked and restarted after the lock has been released and reacquired
again.

Another problem was that some operations did correctly lock the
parent of an entry when needed, but got the size and version xattrs
from the entry instead of the parent.

This patch solves this problem by binding all queries of size and
version to each lock and replacing all entrylk calls by inodelk ones
to remove concurrent updates on directory metadata.  This also allows
rename to correctly update source and destination directories.

BUG: 1225279
Change-Id: I02a6084b138dd38e018a462347cd9ce38610c7ef
Reviewed-on: http://review.gluster.org/10926
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

ec: Fix failures with missing files

2015-05-10T00:30:19+00:00

      Backport of http://review.gluster.com/9407

When a file does not exist on a brick but it does on others, there
could be problems trying to access it because there was some loc_t
structures with null 'pargfid' but 'name' was set. This forced
inode resolution based on /name instead of  which
would be the correct one. To solve this problem, 'name' is always
set to NULL when 'pargfid' is not present.

Another problem was caused by an incorrect management of errors
while doing incremental locking. The only allowed error during an
incremental locking was ENOTCONN, but missing files on a brick can
be returned as ESTALE. This caused an EIO on the operation.

This patch doesn't care of errors during an incremental locking. At
the end of the operation it will check if there are enough successfully
locked bricks to continue or not.

BUG: 1220011
Change-Id: I4a1e6235d80e20ef7ef12daba0807b859ee5c435
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/10701
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System