glusterfs.git/xlators/cluster/ec/src/ec-generic.c, branch v3.10.3

ec: Implement ipc fop

2016-09-25T19:30:18+00:00

The ipc will be wound to all the bricks, but for it to be
successfull, the fop should succeed on minimum number of bricks.

Change-Id: I3f8cb6a349e87bafd0773583def9d4e3765aa140
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15387
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Ashish Pandey 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Add support for hardware acceleration

2016-09-08T17:08:25+00:00

This patch implements functionalities for fast encoding/decoding
using hardware support. Currently optimized x86_64, SSE and AVX is
added.

Additionally this patch implements a caching mecanism for inverse
matrices to reduce computation time, as well as a new method for
computing the inverse that takes quadratic time instead of cubic.

Finally some unnecessary memory copies have been eliminated to
further increase performance.

Change-Id: I26c75f26fb4201bd22b51335448ea4357235065a
BUG: 1289922
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/12837
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Use correct log levels

2016-05-24T12:41:14+00:00

Problem :
Misleading messages are getting logged in mount logs
and bricks log.
"Mismatching xdata" and "Heal failed" are getting logged

Solution :
Reduce the level of logs from INFO, WARNING and NOTICE
to DEBUG level wherever applicable OR use fop_log_level
to get proper log level.

Change-Id: Ia824c71e75ab683d3cb8949e1966ea09c9ccce72
BUG: 1231224
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/13266
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Do not ref dictionary in lookup

2016-03-14T11:39:58+00:00

Problem:
1) dict_for_each loops over the elements without any locks, so the members of
   the dictionary can be ref/unrefed while dict_for_each is executed by another
   thread leading to crashes.

Basically with distributed ec + disctributed replicate as cold, hot tiers. tier
sends a lookup which fails on ec. (By this time dict already contains ec
xattrs) After this lookup_everywhere code path is hit in tier which triggers
lookup on each of distribute's hash lookup but fails which leads to the cold,
hot dht's lookup_everywhere in two parallel epoll threads where in ec when it
tries to set trusted.ec.version/dirty/size as keys in the dictionary, the older
values against the same key get erased. While this erasing is going on if the
thread that is doing lookup on afr's subvolume accesses these keys either in
dict_copy_with_ref or client xlator trying to serialize, that can either lead
to crash or hang based on if the spin/mutex lock is called on invalid memory.

2) EC deletes GF_CONTENT_KEY from the dictionary, this may lead to extra reads
   in case of lookup-everwhere for tiered volumes.

Fix:
Do dict_copy_with_ref() for the lookup-dictionary.
This is avoiding the problem and is not actually fixing the 1st problem.
2nd problem will be fixed.

Change-Id: I5427aa14c48cb7572977d4de9a28c5ffff2b4b95
BUG: 1315560
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13680
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Allow read fops to be processed in parallel

2015-08-29T10:55:28+00:00

Currently ec only sends a single read request at a time for a given
inode. Since reads do not interfere between them, this patch allows
multiple concurrent read requests to be sent in parallel.

Change-Id: If853430482a71767823f39ea70ff89797019d46b
BUG: 1245689
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11742
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System

cluster/ec: Fix tracking of good bricks

2015-08-06T17:12:22+00:00

The bitmask of good and bad bricks was kept in the context of the
corresponding inode or fd. This was problematic when an external
process (another client or the self-heal process) did heal the
bricks but no one changed the bitmaks of other clients.

This patch removes the bitmask stored in the context and calculates
which bricks are healthy after locking them and doing the initial
xattrop. After that, it's updated using the result of each fop.

Change-Id: I225e31cd219a12af4ca58871d8a4bb6f742b223c
BUG: 1236065
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11844
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Minimize usage of EIO error

2015-07-28T11:12:17+00:00

Change-Id: I82e245615419c2006a2d1b5e94ff0908d2f5e891
BUG: 1245276
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11741
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: NetBSD Build System

quota: Fix statfs values in EC when quota_deem_statfs is enabled

2015-06-27T09:42:35+00:00

When quota_deem_statfs is enabled, quota sends aggregated statfs values
In EC we should not multiply statfs values with fragment number

Change-Id: I7ef8ea1598d84b86ba5c5941a2bbe0a6ab43c101
BUG: 1233162
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/11315
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

ec: Porting messages to new logging framework

2015-06-26T15:51:59+00:00

Change-Id: Ia05ae750a245a37d48978e5f37b52f4fb0507a8c
BUG: 1194640
Signed-off-by: Nandaja Varma 
Reviewed-on: http://review.gluster.org/10465
Tested-by: NetBSD Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Do not handle GF_CONTENT_KEY

2015-06-23T09:13:40+00:00

GF_CONTENT_KEY aggregation requires that the fragments on the bricks belong to
same data i.e. no operations are modifying the content while lookup is
performed on it. The only way to know it is to get at least ec->fragments+1
number of responses and see that two different sets of ec->fragments number of
fragments give same data. But at the moment we feel that this slows down
ec-lookup. So removing handling of this for now.

Change-Id: I2da5087f1311d5cdde999062607b143b48c17713
BUG: 1226279
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11003
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System