glusterfs.git/xlators/cluster/dht, branch v4.1dev

core: fix some of the dict_{get,set} with proper APIs

2018-01-17T00:00:11+00:00

updates #220

Change-Id: I6e25dbb69b2c7021e00073e8f025d212db7de0be
Signed-off-by: Amar Tumballi

cluster/dht: Update options for gd2

2018-01-12T05:55:55+00:00

Update DHT options for GD2

Updates gluster/glusterfs#302

Change-Id: Ia597fe364e97edd7bcf72d89f4ccdd50713a8837
Signed-off-by: N Balachandran

cluster/dht: Change datatype of search_unhashed variable

2018-01-10T12:21:12+00:00

Variable search_unhashed is of type boolean, change it to integer type.
This fixes the warning increment of a boolean expression.

BUG: 1531987

Change-Id: Ibf153f6a9ad704da38bff346b6a21a71323ed9bb
Signed-off-by: Varsha Rao

cluster/dht: Use percentages for space check

2018-01-02T18:39:55+00:00

With heterogenous bricks now being supported in DHT
we could run into issues where files are not migrated
even though there is sufficient space in newly added bricks
which just happen to be considerably smaller than older
bricks. Using percentages instead of absolute available
space for space checks can mitigate that to some extent.

Marking bug-1247563.t as that used to depend on the easier
code to prevent a file from migrating. This will be removed
once we find a way to force a file migration failure.

Change-Id: I3452520511f304dbf5af86f0632f654a92fcb647
BUG: 1529440
Signed-off-by: N Balachandran

cluster/dht: Add migration checks to dht_(f)xattrop

2017-12-26T05:25:33+00:00

The dht_(f)xattrop implementation did not implement
migration phase1/phase2 checks which could cause issues
with rebalance on sharded volumes.
This does not solve the issue where fops may reach the target
out of order.

Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c
BUG: 1471031
Signed-off-by: N Balachandran

dht: Fill first_up_subvol before use in dht_opendir

2017-12-15T05:10:01+00:00

Reported by: Sam McLeod

Change-Id: Ic8f9b46b173796afd70aff1042834b03ac3e80b2
BUG: 1512437
Signed-off-by: Poornima G

all: Simplify component message id's definition

2017-12-14T02:33:51+00:00

This patch creates a new way of defining message id's that is easier
and less error prone because it doesn't require so many manual changes
each time a new component is defined or a new message created.

Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b
Signed-off-by: Xavier Hernandez

glusterfs: Use gcc builtin ATOMIC operator to increase/decreate refcount.

2017-12-12T09:05:56+00:00

Problem: In glusterfs code base we call mutex_lock/unlock to take
         reference/dereference for a object.Sometime it could be
         reason for lock contention also.

Solution: There is no need to use mutex to increase/decrease ref
          counter, instead of using mutex use gcc builtin ATOMIC
          operation.

Test:   I have not observed yet how much performance gain after apply
        this patch specific to glusterfs but i have tested same
        with below small program(mutex and atomic both) and
        get good difference.

static int numOuterLoops;
static void *
threadFunc(void *arg)
{
    int j;

    for (j = 0; j < numOuterLoops; j++) {
            __atomic_add_fetch (&glob, 1,__ATOMIC_ACQ_REL);
    }
    return NULL;
}

int
main(int argc, char *argv[])
{
    int opt, s, j;
    int numThreads;
    pthread_t *thread;
    int verbose;
    int64_t n = 0;

    if (argc < 2 ) {
     printf(" Please provide 2 args Num of threads && Outer Loop\n");
     exit (-1);
    }
    numThreads = atoi(argv[1]);
    numOuterLoops = atoi (argv[2]);

    if (1) {
        printf("\tthreads: %d; outer loops: %d;\n",
                numThreads, numOuterLoops);
    }

    thread = calloc(numThreads, sizeof(pthread_t));
    if (thread == NULL) {
        printf ("calloc error so exit\n");
        exit (-1);
    }

    __atomic_store (&glob, &n, __ATOMIC_RELEASE);
    for (j = 0; j < numThreads; j++) {
        s = pthread_create(&thread[j], NULL, threadFunc, NULL);
        if (s != 0) {
            printf ("pthread_create failed so exit\n");
            exit (-1);
        }
    }

    for (j = 0; j < numThreads; j++) {
        s = pthread_join(thread[j], NULL);
        if (s != 0) {
            printf ("pthread_join failed so exit\n");
            exit (-1);
        }
    }
    printf("glob value is %ld\n",__atomic_load_n (&glob,__ATOMIC_RELAXED));

    exit(0);
}

   time ./thr_count 800 800000
   threads: 800; outer loops: 800000;
   glob value is 640000000

real	1m10.288s
user	0m57.269s
sys	3m31.565s

time ./thr_count_atomic 800 800000
     threads: 800; outer loops: 800000;
glob value is 640000000

real	0m20.313s
user	1m20.558s
sys	0m0.028

Change-Id: Ie5030a52ea264875e002e108dd4b207b15ab7cc7
Signed-off-by: Mohit Agrawal

cluster/dht: fix memory leaks in rebalance

2017-12-11T04:54:28+00:00

From code reading it was found that in gf_defrag_process_dir, GF_FREE
was called directly on dir_dfmeta->equeue leading to leaks of memory for
list of entries read from all the local subvols in case of a failure.

This patch frees the entries read from all the local subvols.

Change-Id: If5e8f557372a8fc2af86628b401e8de1b54986a1
BUG: 1430305
Signed-off-by: Susant Palai

dht: Send an event when disks get full

2017-12-09T00:47:46+00:00

Send an event if DHT determines that a subvol is getting
full or running out of inodes.

Change-Id: Ie026f4ee1832b5df1e80b16cb949b2cc31a25d6f
Bug: 1440659
Signed-off-by: Ankit raj 
Signed-off-by: N Balachandran