glusterfs.git/xlators/features/qemu-block/src, branch v3.8dev

libglusterfs/syncop: Add xdata to all syncop calls

2015-04-08T15:14:59+00:00

This patch adds support for xdata in both the
request and response path of syncops.

Few calls like lookup already had the support;
have renamed variables in few places to maintain
uniformity.

xdata passed downwards is known as xdata_in
and xdata passed upwards is known as xdata_out.

There is an old patch by Jeff Darcy at
http://review.gluster.org/#/c/8769/3 which does the
same for some selected calls. It also brings in
xdata support at gfapi level.

xdata support at gfapi level would be introduced
in subsequent patches.

Change-Id: I340e94ebaf2a38e160e65bc30732e8fe1c532dcc
BUG: 1158621
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/9859
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Avoid conflict between contrib/uuid and system uuid

2015-04-04T17:48:35+00:00

glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.

Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.

A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.

BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

qemu-block: Fixing coverity issue(Unused value)

2015-03-30T12:42:38+00:00

Coverity ID: 1124889 1124889 1124889 1124889 1124889 1124889 1124889
1124889 1124889 1124889 1124889

Change-Id: I82d57022c4a06fb70573b4e9436f934185f8fb84
BUG: 789278
Signed-off-by: Nandaja Varma 
Reviewed-on: http://review.gluster.org/9642
Tested-by: Gluster Build System 
Reviewed-by: Humble Devassy Chirammal 
Reviewed-by: Kaleb KEITHLEY

Do not assume sizeof(size_t)

2014-09-16T11:47:15+00:00

This fixes an assumption that sizeof(size_t) == sizeof(uint64_t), which
is not guaranteed. At least on NetBSD/i386, size_t is 32 bit long.

Caught by tests/basics/file-snapshot.t

BUG: 764655
Change-Id: Ib7620a2ffe8758521886af37bc280101a040d860
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8441
Reviewed-by: Humble Devassy Chirammal 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright

Build fix: -lz for qemu-block xlator

2014-09-05T17:17:48+00:00

LIBZ_LIBS has been changed into ZLIB_LIBS everywhere in the
sources, except in qemu-block xlator. Fix it so that it can
find -lz and link correctly.

BUG: 1129939
Change-Id: I34ded175f056d1a0898804fe602e3d2d2cba27f5
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8623
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System 
Reviewed-by: Harshavardhana 
Tested-by: Harshavardhana

osx: Enable qemu feature build on Darwin

2014-08-11T16:02:51+00:00

Change-Id: Ie5d437aa6c52c180fd8d54680c5f882e75c0bf7e
BUG: 1089172
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8448
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

features/qemu-block: Remove unref of freed iobuf

2014-01-30T03:39:34+00:00

Change-Id: I507a6504b379eef54be77b54d6e2beee63975ebf
BUG: 789278
Signed-off-by: Poornima 
Reviewed-on: http://review.gluster.org/6824
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Anand Avati

syncop: Change return value of syncop

2014-01-20T07:05:15+00:00

Problem:
We found a day-1 bug when syncop_xxx() infra is used inside a synctask with
compilation optimization (CFLAGS -O2).

Detailed explanation of the Root cause:
We found the bug in 'gf_defrag_migrate_data' in rebalance operation:

Lets look at interesting parts of the function:

int
gf_defrag_migrate_data (xlator_t *this, gf_defrag_info_t *defrag, loc_t *loc,
                        dict_t *migrate_data)
{
.....
code section - [ Loop ]
        while ((ret = syncop_readdirp (this, fd, 131072, offset, NULL,
                                       &entries)) != 0) {
.....
code section - [ ERRNO-1 ] (errno of readdirp is stored in readdir_operrno by a
thread)
                /* Need to keep track of ENOENT errno, that means, there is no
                   need to send more readdirp() */
                readdir_operrno = errno;
.....
code section - [ SYNCOP-1 ] (syncop_getxattr is called by a thread)
                        ret = syncop_getxattr (this, &entry_loc, &dict,
                                               GF_XATTR_LINKINFO_KEY);
code section - [ ERRNO-2]   (checking for failures of syncop_getxattr(). This
may not always be executed in same thread which executed [SYNCOP-1])
                        if (ret < 0) {
                                if (errno != ENODATA) {
                                        loglevel = GF_LOG_ERROR;
                                        defrag->total_failures += 1;
.....
}

the function above could be executed by thread(t1) till [SYNCOP-1] and code
from [ERRNO-2] can be executed by a different thread(t2) because of the way
syncop-infra schedules the tasks.

when the code is compiled with -O2 optimization this is the assembly code that
is generated:
 [ERRNO-1]
1165                        readdir_operrno = errno; <<---- errno gets expanded
as *(__errno_location())
   0x00007fd149d48b60 <+496>:        callq  0x7fd149d410c0 
   0x00007fd149d48b72 <+514>:        mov    %rax,0x50(%rsp) <<------ Address
returned by __errno_location() is stored in a special location in stack for
later use.
   0x00007fd149d48b77 <+519>:        mov    (%rax),%eax
   0x00007fd149d48b79 <+521>:        mov    %eax,0x78(%rsp)
....
 [ERRNO-2]
1281                                        if (errno != ENODATA) {
   0x00007fd149d492ae <+2366>:        mov    0x50(%rsp),%rax <<-----  Because
it already stored the address returned by __errno_location(), it just
dereferences the address to get the errno value. BUT THIS CODE NEED NOT BE
EXECUTED BY SAME THREAD!!!
   0x00007fd149d492b3 <+2371>:        mov    $0x9,%ebp
   0x00007fd149d492b8 <+2376>:        mov    (%rax),%edi
   0x00007fd149d492ba <+2378>:        cmp    $0x3d,%edi

The problem is that __errno_location() value of t1 and t2 are different. So
[ERRNO-2] ends up reading errno of t1 instead of errno of t2 even though t2 is
executing [ERRNO-2] code section.

When code is compiled without any optimization for [ERRNO-2]:
1281                                        if (errno != ENODATA) {
   0x00007fd58e7a326f <+2237>:        callq  0x7fd58e797300
<<--- As it is calling __errno_location() again it gets the
location from t2 so it works as intended.
   0x00007fd58e7a3274 <+2242>:        mov    (%rax),%eax
   0x00007fd58e7a3276 <+2244>:        cmp    $0x3d,%eax
   0x00007fd58e7a3279 <+2247>:        je     0x7fd58e7a32a1


Fix:
Make syncop_xxx() return (-errno) value as the return value in
case of errors and all the functions which make syncop_xxx() will need to use
(-ret) to figure out the reason for failure in case of syncop_xxx() failures.

Change-Id: I314d20dabe55d3e62ff66f3b4adb1cac2eaebb57
BUG: 1040356
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/6475
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

features/qemu-block: simplify coroutine model to use single synctask, ucontext

2013-11-11T07:45:46+00:00

The current coroutine model, mapping synctasks 1-1 with qemu internal
Coroutines, has some unresolved raciness issues. This problem usually
manifests as lifecycle mismatches between top-level (gluster created)
synctasks and the subsequently created internal coroutines from that
context. Qemu's internal queueing (and locking) can cause situations
where the top-level synctask is destroyed before the internal scheduler
has released references to memory, leading to use after free crashes
and asserts.

Simplify the coroutine model to use a single synctask as a coroutine
processor and rely on the existing native ucontext coroutine
implementation. The syncenv thread is donated to qemu and ensures a
single top-level coroutine is processed at a time. Qemu now has
complete control over coroutine scheduling.

BUG: 986775
Change-Id: I38223479a608d80353128e390f243933fc946fd6
Signed-off-by: Brian Foster 
Reviewed-on: http://review.gluster.org/6110
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

features/qemu-block: add qemu backing image support (clone)

2013-11-11T07:02:37+00:00

Add basic backing image support to the block-format mechanism. This
is a functionality checkpoint that enables the raw mechanism
required to support client driven "snapshot" and "clone" requests.

This change enhances the block-format setxattr command to support
an additional and optional backing image reference. For example:

setxattr -n trusted.glusterfs.block-format -v "qcow2:10GB:" ./newimage

... where  refers to the backing image for unallocated blocks
in newimage.  can be provided in one of two formats:

- a gfid string in the following format (assuming a valid gfid):



- or a filename that must be resident in the same directory as the
new clone file being formatted. E.g.,

setxattr -n trusted.glusterfs.block-format -v "qcow2:10GB:baseimg" ./newimage

This latter format is more restrictive, simply provided for
convenience or until something more refined is available.

This change makes no assumptions about the backing image file and
affords no additional protection. It is up to the user/client to
recognize the relationship between the files and manage them
appropriately (i.e., no writes to the backing image, etc.).

BUG: 986775
Change-Id: I7aff7bdc59b85a6459001a6bfeae4db6bf74f703
Signed-off-by: Brian Foster 
Reviewed-on: http://review.gluster.org/5967
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati