glusterfs-snapshot.git/xlators/cluster/afr/src, branch gluster-snap

cluster/afr: Change default_value for option self-heal-daemon

2014-01-27T15:47:06+00:00

Change-Id: Ic3c8e179a63e82a4e416aea620796f8bb3236c7c
BUG: 1052759
Signed-off-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/6706
Reviewed-by: Kaushal M 
Tested-by: Gluster Build System

syncop: Change return value of syncop

2014-01-20T07:05:15+00:00

Problem:
We found a day-1 bug when syncop_xxx() infra is used inside a synctask with
compilation optimization (CFLAGS -O2).

Detailed explanation of the Root cause:
We found the bug in 'gf_defrag_migrate_data' in rebalance operation:

Lets look at interesting parts of the function:

int
gf_defrag_migrate_data (xlator_t *this, gf_defrag_info_t *defrag, loc_t *loc,
                        dict_t *migrate_data)
{
.....
code section - [ Loop ]
        while ((ret = syncop_readdirp (this, fd, 131072, offset, NULL,
                                       &entries)) != 0) {
.....
code section - [ ERRNO-1 ] (errno of readdirp is stored in readdir_operrno by a
thread)
                /* Need to keep track of ENOENT errno, that means, there is no
                   need to send more readdirp() */
                readdir_operrno = errno;
.....
code section - [ SYNCOP-1 ] (syncop_getxattr is called by a thread)
                        ret = syncop_getxattr (this, &entry_loc, &dict,
                                               GF_XATTR_LINKINFO_KEY);
code section - [ ERRNO-2]   (checking for failures of syncop_getxattr(). This
may not always be executed in same thread which executed [SYNCOP-1])
                        if (ret < 0) {
                                if (errno != ENODATA) {
                                        loglevel = GF_LOG_ERROR;
                                        defrag->total_failures += 1;
.....
}

the function above could be executed by thread(t1) till [SYNCOP-1] and code
from [ERRNO-2] can be executed by a different thread(t2) because of the way
syncop-infra schedules the tasks.

when the code is compiled with -O2 optimization this is the assembly code that
is generated:
 [ERRNO-1]
1165                        readdir_operrno = errno; <<---- errno gets expanded
as *(__errno_location())
   0x00007fd149d48b60 <+496>:        callq  0x7fd149d410c0 
   0x00007fd149d48b72 <+514>:        mov    %rax,0x50(%rsp) <<------ Address
returned by __errno_location() is stored in a special location in stack for
later use.
   0x00007fd149d48b77 <+519>:        mov    (%rax),%eax
   0x00007fd149d48b79 <+521>:        mov    %eax,0x78(%rsp)
....
 [ERRNO-2]
1281                                        if (errno != ENODATA) {
   0x00007fd149d492ae <+2366>:        mov    0x50(%rsp),%rax <<-----  Because
it already stored the address returned by __errno_location(), it just
dereferences the address to get the errno value. BUT THIS CODE NEED NOT BE
EXECUTED BY SAME THREAD!!!
   0x00007fd149d492b3 <+2371>:        mov    $0x9,%ebp
   0x00007fd149d492b8 <+2376>:        mov    (%rax),%edi
   0x00007fd149d492ba <+2378>:        cmp    $0x3d,%edi

The problem is that __errno_location() value of t1 and t2 are different. So
[ERRNO-2] ends up reading errno of t1 instead of errno of t2 even though t2 is
executing [ERRNO-2] code section.

When code is compiled without any optimization for [ERRNO-2]:
1281                                        if (errno != ENODATA) {
   0x00007fd58e7a326f <+2237>:        callq  0x7fd58e797300
<<--- As it is calling __errno_location() again it gets the
location from t2 so it works as intended.
   0x00007fd58e7a3274 <+2242>:        mov    (%rax),%eax
   0x00007fd58e7a3276 <+2244>:        cmp    $0x3d,%eax
   0x00007fd58e7a3279 <+2247>:        je     0x7fd58e7a32a1


Fix:
Make syncop_xxx() return (-errno) value as the return value in
case of errors and all the functions which make syncop_xxx() will need to use
(-ret) to figure out the reason for failure in case of syncop_xxx() failures.

Change-Id: I314d20dabe55d3e62ff66f3b4adb1cac2eaebb57
BUG: 1040356
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/6475
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/afr: Unable to self heal symbolic links

2014-01-14T04:42:05+00:00

Problem:
Under the entry self heal, readlink is done at the
source and sink. When readlink is done at the sink,
because link is not present at the sink, afr expects
ENOENT. AFR translator takes decisions for new link
creation based on ENOENT but server translator is modified
to return ESTALE because of which afr xlator is not able
to heal.

Fix:
The check for inode absence at server includes ESTALE as
well.

Change-Id: I319e4cb4156a243afee79365b7b7a5a7823e9a24
BUG: 1046624
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on: http://review.gluster.org/6599
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/afr: Stop index crawl on pending full crawl

2014-01-08T17:25:06+00:00

Full crawl is executed when index self-heal is useless, like
disk replacement. So if there are on-going index crawls, they
should be stopped inorder to start full self-heals.

Change-Id: I9a1545f1ec4ad9999dc08523ce859e4fa152e214
BUG: 1049355
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/6659
Tested-by: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Vijay Bellur

cluster/afr: Don't accept heal commands until graph is up

2014-01-08T12:20:02+00:00

Change-Id: Icca6c23b6a5965f462db8b65af3eb2e141c7cd39
BUG: 1049355
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/6658
Tested-by: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Vijay Bellur

cluster/afr: avoid race due to afr_is_transaction_running()

2013-12-24T10:24:20+00:00

Problem:
------------------------------------------
afr_lookup_perform_self_heal() {
        if(afr_is_transaction_running())
                goto out
        else
                afr_launch_self_heal();
}
------------------------------------------

When 2 clients simultaneously access a file in split-brain, one of them
acquires the inode lock and proceeds with afr_launch_self_heal (which
eventually fails and sets "sh-failed" in the callback.)

The second client meanwhile bails out of afr_lookup_perform_self_heal()
because afr_is_transaction_running() returns true due to the lock obtained by
client-1. Consequetly in client-2, "sh-failed" does not get set in the dict,
causing quick-read translator to *not* invalidate the inode, thereby serving
data randomly from one of the bricks.

Fix:
If a possible split-brain is detected on lookup, forcefully traverse the
afr_launch_self_heal() code path in afr_lookup_perform_self_heal().

Change-Id: I316f9f282543533fd3c958e4b63ecada42c2a14f
BUG: 870565
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/6578
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Varun Shastry

Fix typos, and spacing issues.

2013-12-23T18:12:41+00:00

Change-Id: I459ba4e87e9bc4f1c373f7abe8701bfa8450253c
BUG: 1045690
Signed-off-by: James Shubin 
Reviewed-on: http://review.gluster.org/6556
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: For entry self heal, mark all source bricks

2013-12-19T17:34:00+00:00

Problem:
Whenever a new brick is added into a replicate volume, all
source bricks are not marked as source. Only one of them is
marked as source. Here marked as source refers to adding
extended attribute at the backend of a file corresponding to
the newly added brick. As well as source bricks should point
to the newly added brick so that heal can be triggered.

Fix:
All source bricks will now point to newly added bricks and heal
can be triggered based on the extended attributes.

Change-Id: I318e1f779a380c16c448a2d05c0140d8e4647fd4
BUG: 1037501
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on: http://review.gluster.org/6540
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Add foreground self-heal launch capability through lookup

2013-12-17T05:37:30+00:00

Also renamed allow-sh-for-running-transaction -> attempt-self-heal

Change-Id: I134cc79e663b532e625ffc342c59e49e71644ab3
BUG: 1039544
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/6463
Tested-by: Gluster Build System 
Reviewed-by: venkatesh somyajulu 
Reviewed-by: Vijay Bellur

pathinfo: Provide user namespace access.

2013-12-16T14:54:26+00:00

Locality can be now queried by unprivileged users with
key "glusterfs.pathinfo".

Setting both "glusterfs.pathinfo" and "trusted.glusterfs.pathinfo"
on disk is prevented with this patch.

Original Author: Vijay Bellur 

Change-Id: I4f7a0db8ad59165c4aeda04b23173255157a8b79
Signed-off-by: Vijaykumar M 
Reviewed-on: http://review.gluster.org/5101
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur