glusterfs.git/libglusterfs/src/dict.h, branch v4.1.0alpha

Quota: heal directory on newly added bricks when quota limit is reached

2018-03-28T04:07:12+00:00

Problem: if a lookup is done on a newly added brick for a path on which limit
has been reached, the lookup fails to heal the directory tree due to quota.

Solution: Tag the lookup as an internal fop and ignore it in quota.
Since marking internal fop does not usually give enough contextual information.
Introducing new flags to pass the contextual info.

Adding dict_check_flag and dict_set_flag to aid flag operations.
A flag is a single bit in a bit array (currently limited to 256 bits).

Change-Id: Ifb6a68bcaffedd425dd0f01f7db24edd5394c095
fixes: bz#1505355
BUG: 1505355
Signed-off-by: Sanoj Unnikrishnan

xlators/features/namespace: Add namespace xlator and link into brick graph

2018-02-21T09:52:17+00:00

The following release-3.8-fb branch patch is upstreamed:
> features/namespace: Add namespace xlator and link into brick graph
> Commit ID: dbd30776f26e
> https://review.gluster.org/#/c/18041/
> By Michael Goulet 

Changes in this patch:
Removes extra config.h and namespace.h file in namespace.c
Adds default_getspec_cbk to libglusterfs.sym
Rename dict_for_each to dict_foreach_inline
Remove fd.h header file stack.h
Add test case for truncate, open and symlink

This patch is required to forward port io-threads namespace patch.
Updates: #401

Change-Id: Ib88c95b89eecee9b8957df8a4c8712c899c761d1
Signed-off-by: Varsha Rao

performance/io-threads: expose io-thread queue depths

2018-02-08T17:01:12+00:00

The following release-3.8-fb branch patch is upstreamed:

> io-stats: Expose io-thread queue depths
> Commit ID: 69509ee7d2
> https://review.gluster.org/#/c/18143/
> By Shreyas Siravara 

Changes in this patch:
- Replace iot_pri_t with gf_fop_pri_t
- Replace IOT_PRI_{HI, LO, NORMAL, MAX, LEAST} with
  GF_FOP_PRI_{HI, LO, NORMAL, MAX, LEAST}
- Use dict_unref() instead of dict_destroy()

This patch is required to forward port io-threads namespace patch.
Updates: #401
Change-Id: I1b47a63185a441a30fbc423ca1015df7b36c2518
Signed-off-by: Varsha Rao

dict: add another type to handle backward compatibility

2018-01-17T03:53:37+00:00

This new type helps to avoid excessive logs. It should be
set only in case of
 * volume graph building (graph.y)
 * dict unserialize
   (happens once a dictionary is received on wire in old protocol)

All other dict set and get should have proper check and warning
logs if there is a mismatch.

updates #220

Change-Id: I1cccb304a877aa80c07aaac95f10f5005e35b9c5
Signed-off-by: Amar Tumballi

dict: add more types for values

2018-01-05T09:35:07+00:00

Added 2 more types which are present in gluster codebase, mainly
IATT and UUID.

Updates #203

Change-Id: Ib6d6d6aefb88c3494fbf93dcbe08d9979484968f
Signed-off-by: Amar Tumballi

dict: support better on-wire transfer

2017-12-27T05:20:30+00:00

This patch brings data type awareness to dictionary,
and also makes sure valid data is properly sent to the
other side of the wire using XDR.

Next step is to allow people to add more data types
(for example, Bool, UUID, iatt etc), and then make
it part of every fop signature in wire.

Fixes #203

Change-Id: Ie0eee2db847bea2bf7dad80dec89ce3e7c5917c1
Signed-off-by: Amar Tumballi

glusterfs: Use gcc builtin ATOMIC operator to increase/decreate refcount.

2017-12-12T09:05:56+00:00

Problem: In glusterfs code base we call mutex_lock/unlock to take
         reference/dereference for a object.Sometime it could be
         reason for lock contention also.

Solution: There is no need to use mutex to increase/decrease ref
          counter, instead of using mutex use gcc builtin ATOMIC
          operation.

Test:   I have not observed yet how much performance gain after apply
        this patch specific to glusterfs but i have tested same
        with below small program(mutex and atomic both) and
        get good difference.

static int numOuterLoops;
static void *
threadFunc(void *arg)
{
    int j;

    for (j = 0; j < numOuterLoops; j++) {
            __atomic_add_fetch (&glob, 1,__ATOMIC_ACQ_REL);
    }
    return NULL;
}

int
main(int argc, char *argv[])
{
    int opt, s, j;
    int numThreads;
    pthread_t *thread;
    int verbose;
    int64_t n = 0;

    if (argc < 2 ) {
     printf(" Please provide 2 args Num of threads && Outer Loop\n");
     exit (-1);
    }
    numThreads = atoi(argv[1]);
    numOuterLoops = atoi (argv[2]);

    if (1) {
        printf("\tthreads: %d; outer loops: %d;\n",
                numThreads, numOuterLoops);
    }

    thread = calloc(numThreads, sizeof(pthread_t));
    if (thread == NULL) {
        printf ("calloc error so exit\n");
        exit (-1);
    }

    __atomic_store (&glob, &n, __ATOMIC_RELEASE);
    for (j = 0; j < numThreads; j++) {
        s = pthread_create(&thread[j], NULL, threadFunc, NULL);
        if (s != 0) {
            printf ("pthread_create failed so exit\n");
            exit (-1);
        }
    }

    for (j = 0; j < numThreads; j++) {
        s = pthread_join(thread[j], NULL);
        if (s != 0) {
            printf ("pthread_join failed so exit\n");
            exit (-1);
        }
    }
    printf("glob value is %ld\n",__atomic_load_n (&glob,__ATOMIC_RELAXED));

    exit(0);
}

   time ./thr_count 800 800000
   threads: 800; outer loops: 800000;
   glob value is 640000000

real	1m10.288s
user	0m57.269s
sys	3m31.565s

time ./thr_count_atomic 800 800000
     threads: 800; outer loops: 800000;
glob value is 640000000

real	0m20.313s
user	1m20.558s
sys	0m0.028

Change-Id: Ie5030a52ea264875e002e108dd4b207b15ab7cc7
Signed-off-by: Mohit Agrawal

dict: Fix several coverity issues in dict

2017-11-15T14:37:31+00:00

This patch fixes issues 230,592,593,110,63 from [1]

[1] https://download.gluster.org/pub/gluster/glusterfs/static-analysis/master/glusterfs-coverity/2017-10-30-9aa574a5/html/

Note: Resolve FORWARD_NULL coverity issue in glusterfs_ctx_new is
      also fixed with this patch.

BUG: 789278
Change-Id: Ic4199a144a14cc9ead7366fb1c9699197141bc86
Signed-off-by: Mohit Agrawal

perf/qr: Use a ref-ed data to extract content

2017-08-28T06:42:27+00:00

qr_content_extract used dict_get to get the value of
the GF_CONTENT_KEY key. dict_get does not ref the data
before returning it so QR could be acting on freed memory
if another thread deletes the key before then.
This patch also fixes a race in dict_get_with_ref.

Fix: Use dict_get_with_ref to retrieve the file contents.

Change-Id: Ib1a7a70bb92eed7e70747ec530e0b3edc53127ec
BUG: 1484709
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18115
Smoke: Gluster Build System 
Reviewed-by: Amar Tumballi 
Tested-by: Raghavendra G 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

gfapi: Duplicate the buffer sent in setxattr calls

2017-08-07T16:28:45+00:00

Issue:
The caller of glfs_setxattr sends a buffer to set as the value.
We create a dict in which the pointer to the value is set.
Underlying layers like md-cache take a ref on this dict to store
the value for a longer time. But the moment setxattr is complete,
the caller of glfs_setxattr can free the value memory.

Solution:
memcpy the setxattr value to the gluster buffer.

Change-Id: I58753fe702e8b7d0f6c4f058714c65d0ad5d7a0a
BUG: 1477488
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17967
Reviewed-by: soumya k 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy