glusterfs.git/xlators/cluster/ec/src, branch v3.9.0rc2

cluster/ec: set/unset dirty flag for data/metadata update

2016-10-13T20:04:11+00:00

Currently, for all the update operations, metadata or data,
we set the dirty flag at the end of the operation only if
a brick is down. This leads to delay in healing and in some
cases not at all.
In this patch we set (+1) the dirty flag
at the start of the metadata or data update operations and
after successfull completion of the fop, we unset (-1) it again.

>Change-Id: Ide5668bdec7b937a61c5c840cdc79a967598e1e9
>BUG: 1316873
>Signed-off-by: Ashish Pandey 
>Reviewed-on: http://review.gluster.org/13733
>Tested-by: Pranith Kumar Karampuri 
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Xavier Hernandez 

Change-Id: Ide5668bdec7b937a61c5c840cdc79a967598e1e9
BUG: 1377570
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/15534
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/ec: Implement heal info with lock

2016-10-13T14:35:46+00:00

Problem: Currently heal info command prints all
the files/directories if the index for the
file/directory is present in .glusterfs/indices folder.
After implementing patch http://review.gluster.org/#/c/13733/
indices of the file which is going through update fop
will also be present in .glusterfs/indices even
if the fop is successful on all the brick. At this time
if heal info command is being used, it will also display this
file which is actually healthy and does not require any heal.

Solution: Take lock on a file corresponding to the indices
and inspect xattrs to decide if the file needs heal or not.

>Reviewed-on: http://review.gluster.org/15543
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Pranith Kumar Karampuri 
>Reviewed-by: Xavier Hernandez 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Signed-off-by: Ashish Pandey 

Change-Id: I6361e2813ece369be12d02e74816df4eddb81cfa
BUG: 1383913
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/15627
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Add support for hardware acceleration

2016-09-10T02:19:13+00:00

This patch implements functionalities for fast encoding/decoding
using hardware support. Currently optimized x86_64, SSE and AVX is
added.

Additionally this patch implements a caching mecanism for inverse
matrices to reduce computation time, as well as a new method for
computing the inverse that takes quadratic time instead of cubic.

Finally some unnecessary memory copies have been eliminated to
further increase performance.

 >Change-Id: I26c75f26fb4201bd22b51335448ea4357235065a
 >BUG: 1289922
 >Signed-off-by: Xavier Hernandez 
 >Reviewed-on: http://review.gluster.org/12837
 >Tested-by: Pranith Kumar Karampuri 
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Pranith Kumar Karampuri 

BUG: 1374841
Change-Id: I83731663922ed11ca84536deab5737463416e1e0
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15455
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/ec: Add events for EC translator

2016-09-08T12:21:52+00:00

This patch will generates events in following
cases which will be consumed by new event
framework.

Consider an EC volume with (K+M) configuration
K = Data bricks
M = Redundancy bricks

1- EVENT_EC_MIN_BRICKS_NOT_UP -
   When minimum "K" number of bricks, required
   for any ec fop, are not up.
2- EVENT_EC_MIN_BRICKS_UP
   When minimum "K" number of bricks, required
   for any ec fop, are up.

>Change-Id: I0414b8968c39740a171e5aa14b087afd524d574f
>BUG: 1371470
>Signed-off-by: Ashish Pandey 
>Reviewed-on: http://review.gluster.org/15348
>Tested-by: Pranith Kumar Karampuri 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 

Change-Id: I0414b8968c39740a171e5aa14b087afd524d574f
BUG: 1374167
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/15425
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/ec: Use locks for opendir

2016-08-25T13:48:33+00:00

Problem:
In some cases we see that readdir keeps winding to the brick that doesn't have
any blocked locks i.e. first brick. This is leading to the client assuming that
there are no blocking locks on the inode so it won't give away the lock. Other
clients end up blocked on the lock as if the command hung.

Fix:
Proper way to fix this issue is to use infra present in
http://review.gluster.org/14736 This is a stop gap fix where we start taking
inodelks in opendir which goes to all the bricks, this will detect if there is
any contention.

BUG: 1346719
Change-Id: I91109107a26f6535b945ac476338e9f21dc31eb9
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15309
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ashish Pandey

cluster/ec: Do multi-threaded self-heal

2016-08-24T22:24:22+00:00

BUG: 1368451
Change-Id: I5d6b91d714ad6906dc478a401e614115c89a8fbb
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15083
Smoke: Gluster Build System 
Reviewed-by: Ashish Pandey 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/afr: Prevent split-brain when bricks are brought off and on in cyclic order

2016-08-22T09:38:36+00:00

When the bricks are brought offline and then online in cyclic
order while writes are in progress on a file, thanks to inode
refresh in write txns, AFR will mostly fail the write attempt
when the only good copy is offline. However, there is still a
remote possibility that the file will run into split-brain if
the brick that has the lone good copy goes offline *after* the
inode refresh but *before* the write txn completes (I call it
in-flight split-brain in the patch for ease of reference),
requiring intervention from admin to resolve the split-brain
before the IO can resume normally on the file. To get around this,
the patch does the following things:
i) retains the dirty xattrs on the file
ii) avoids marking the last of the good copies as bad (or accused)
    in case it is the one to go down during the course of a write.
iii) fails that particular write with the appropriate errno.

This way, we still have one good copy left despite the split-brain situation
which when it is back online, will be chosen as source to do the heal.

Change-Id: I9ca634b026ac830b172bac076437cc3bf1ae7d8a
BUG: 1363721
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15080
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Oleksandr Natalenko 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: move alloca0 definition to common-utils

2016-08-12T08:53:23+00:00

...and remove their definitons from EC and AFR.
Also `s/alloca+memset0/alloca0` wherever it is used.

Change-Id: I3b71e596d12a7d8900f5d761af6b98305c8874d5
BUG: 1366226
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15147
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System

cluster/ec: Handle absence of keys in some callback dict

2016-07-26T06:07:03+00:00

Problem: This issue arises when we do a rolling update
from 3.7.5 to 3.7.9.
For 4+2 volume running 3.7.5, if we update 2 nodes
and after heal completion  kill 2 older nodes, this
problem can be seen. After update and killing of
bricks, 2 nodes will return inodelk count key in dict
while other 2 nodes will not have inodelk count in dict.
This is also true for get-link-count.
During dictionary match , ec_dict_compare, this will
lead to mismatch of answers and the file operation
on mount point will fail with IO error.

Solution:
Don't match inode, entry and link count keys while
comparing two dictionaries. However, while combining the
data in ec_dict_combine, go through all the dictionaries
and select the maximum values received in different dicts
for these keys.

Change-Id: I33546e3619fe8f909286ee48fb0df2009cd3d22f
BUG: 1347686
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/14761
Reviewed-by: Xavier Hernandez 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

core: add a basis function to reduce verbose code

2016-07-18T11:59:05+00:00

Change-Id: Icebe1b865edb317685e93f3ef11d98fd9b2c2e9a
BUG: 1357226
Signed-off-by: Zhou Zhengping 
Reviewed-on: http://review.gluster.org/14936
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System