glusterfs.git/libglusterfs/src/iobuf.c, branch v4.1.0alpha

glusterfs: Use gcc builtin ATOMIC operator to increase/decreate refcount.

2017-12-12T09:05:56+00:00

Problem: In glusterfs code base we call mutex_lock/unlock to take
         reference/dereference for a object.Sometime it could be
         reason for lock contention also.

Solution: There is no need to use mutex to increase/decrease ref
          counter, instead of using mutex use gcc builtin ATOMIC
          operation.

Test:   I have not observed yet how much performance gain after apply
        this patch specific to glusterfs but i have tested same
        with below small program(mutex and atomic both) and
        get good difference.

static int numOuterLoops;
static void *
threadFunc(void *arg)
{
    int j;

    for (j = 0; j < numOuterLoops; j++) {
            __atomic_add_fetch (&glob, 1,__ATOMIC_ACQ_REL);
    }
    return NULL;
}

int
main(int argc, char *argv[])
{
    int opt, s, j;
    int numThreads;
    pthread_t *thread;
    int verbose;
    int64_t n = 0;

    if (argc < 2 ) {
     printf(" Please provide 2 args Num of threads && Outer Loop\n");
     exit (-1);
    }
    numThreads = atoi(argv[1]);
    numOuterLoops = atoi (argv[2]);

    if (1) {
        printf("\tthreads: %d; outer loops: %d;\n",
                numThreads, numOuterLoops);
    }

    thread = calloc(numThreads, sizeof(pthread_t));
    if (thread == NULL) {
        printf ("calloc error so exit\n");
        exit (-1);
    }

    __atomic_store (&glob, &n, __ATOMIC_RELEASE);
    for (j = 0; j < numThreads; j++) {
        s = pthread_create(&thread[j], NULL, threadFunc, NULL);
        if (s != 0) {
            printf ("pthread_create failed so exit\n");
            exit (-1);
        }
    }

    for (j = 0; j < numThreads; j++) {
        s = pthread_join(thread[j], NULL);
        if (s != 0) {
            printf ("pthread_join failed so exit\n");
            exit (-1);
        }
    }
    printf("glob value is %ld\n",__atomic_load_n (&glob,__ATOMIC_RELAXED));

    exit(0);
}

   time ./thr_count 800 800000
   threads: 800; outer loops: 800000;
   glob value is 640000000

real	1m10.288s
user	0m57.269s
sys	3m31.565s

time ./thr_count_atomic 800 800000
     threads: 800; outer loops: 800000;
glob value is 640000000

real	0m20.313s
user	1m20.558s
sys	0m0.028

Change-Id: Ie5030a52ea264875e002e108dd4b207b15ab7cc7
Signed-off-by: Mohit Agrawal

core: coverity, NULL potinter check

2016-07-22T15:12:31+00:00

iobuf_get_page_aligned should check the return pointer
after calling iobuf_get2.

CID: 1357868

Change-Id: I3aa5b09bce2225aa70cf3c60a6b6f4e4eca1dee6
BUG: 1358936
Signed-off-by: Zhou Zhengping 
Reviewed-on: http://review.gluster.org/14976
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

libglusterfs: Implement API that provides page-aligned iobufs

2016-06-28T21:52:42+00:00

One of the consumers of a page aligned buffer would be posix's
readv fop on O_DIRECT fds. Today the way it works is by getting
a page-aligned buffer through calloc, pread()ing into this buffer
and then copying its contents into a newly created iobuf's ptr.
This results in an extra memcpy() which can be avoided if we could
implement an api that would return an iobuf whose ptr is
page-aligned. That way the iobuf->ptr can be directly passed to
sys_pread() as a parameter by posix translator.

Change-Id: I385139bc6ee309fc501034b3af8f7814fab8cd65
BUG: 1343838
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14672
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Zhou Zhengping 
Reviewed-by: Jeff Darcy

experimental: add fdl (Full Data Logging) translator

2016-02-13T13:13:07+00:00

NSR needs logging that is different than our existing changelog in
several ways:

 * Full data, not just metadata

 * Pre-op, not post-op

 * High performance

 * Supports the concept of time-bounded "terms"

Others (for example EC) might need the same thing.  This patch adds such
a translator.  It also adds code to dump the resulting journals, and to replay
them using syncops, plus (very rudimentary) tests for all of the above.

Change-Id: I29680a1b4e0a9e7d5a8497fef302c46434b86636
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/12450
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

defaults,globals,iobuf,latency,logging,options,xlator/libglusterfs : porting to a new logging framework

2015-06-25T06:31:52+00:00

Change-Id: If6a55186cddc3d1c4d22e3d56b45358b84feeb49
BUG: 1194640
Signed-off-by: Mohamed Ashiq 
Reviewed-on: http://review.gluster.org/10826
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

iobuf: Do not call __iobuf_ref directly

2015-04-02T10:08:36+00:00

iobuf_get will be creating the iobuf and
hence lock is not necessary to increment ref.

However, it is a good practice to call
iobuf_ref instead of __iobuf_ref so that
we have a single point to get refs and
this can be used later to do mem
accounting etc.

Change-Id: I1fd328c3c463c23fd5f6df505ccb5c86f6207f28
BUG: 1199075
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/9812
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System

libglusterfs: Change iobuf_pool_destroy to destroy all the arenas.

2015-03-04T17:15:11+00:00

Change-Id: If5e9d4ce98f845d3b52565ac62970959e663497f
BUG: 1093594
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/9699
Reviewed-by: Raghavendra Talur 
Tested-by: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Shyamsundar Ranganathan

rdma: pre-register iobuf_pool with rdma devices.

2015-02-18T04:09:54+00:00

registring buffers with rdma device is a time consuming
operation. So performing registration in code path will
decrease the performance.
Using a pre registered memory will give a bettor performance,
ie, register iobuf_pool during rdma initialization. For
dynamically created arena, we can register with all the
device.

Change-Id: Ic79183e2efd014c43faf5911fdb6d5cfbcee64ca
BUG: 1187456
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/9506
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

iobufs: make iobref container size dynamic

2013-11-26T18:31:17+00:00

With gfapi we can receive read/write size beyond the natural limits
of FUSE and NFS server. iobref was hardcoded to hold iobuf refs up
to 16 in count, which imposes a natural limit of 2MB with 128KB page
sizes of read-ahead and io-cache. Fix this by making iobref's iobuf
ref container size dynamic.

Change-Id: I93d88104d6c5e7af96cc9f1bfcc870d80fa81dad
BUG: 1034398
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6348
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

iobuf: Added a function iobref_clear

2013-03-21T02:51:50+00:00

Original-author: Venky Shankar 

Change-Id: Ibf861db6c1b084b798d210962344487a1919aad2
BUG: 921942
Signed-off-by: Prashanth Pai 
Reviewed-on: http://review.gluster.org/4595
Reviewed-by: Jeff Darcy 
Tested-by: Gluster Build System