glusterfs.git/libglusterfs, branch release-3.10

libglusterfs: fix comparison of a NULL dict with a non-NULL dict

2018-04-19T09:03:52+00:00

Function are_dicts_equal() had a bug when the first argument was NULL and
the second one wasn't NULL. In this case it incorrectly returned that the
dicts were different when they could be equal.

Backport of:
> BUG: 1566732

BUG: 1569409
Change-Id: I0fc245c2e7d1395865a76405dbd05e5d34db3273
Signed-off-by: Xavi Hernandez

gfapi : Resolve "." and ".." only for named lookups

2018-03-22T11:28:55+00:00

The patch https://review.gluster.org/#/c/17177 resolves "." and ".."
to corrosponding inodes and names before sending the request to the
backend server. But this will only work if inode and its parent is
linked properly. Incase of nameless lookup(applications like ganesha)
the inode of parent can be NULL(only gfid is send). So this patch will
resolve "." and ".." only if proper parent is available

Upstream reference
>Change-Id: I4c50258b0d896dabf000a547ab180b57df308a0b
>BUG: 1460514
>Signed-off-by: Jiffin Tony Thottan 
>Reviewed-on: https://review.gluster.org/17502
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Poornima G 
>CentOS-regression: Gluster Build System 
>Reviewed-by: soumya k 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit a052b413242783f39cb3312a6a02bdd025b10f0c)

Change-Id: I4c50258b0d896dabf000a547ab180b57df308a0b
BUG: 1559352
Signed-off-by: Jiffin Tony Thottan

glusterd : introduce timer in mgmt_v3_lock

2018-03-20T13:58:38+00:00

Problem:
In a multinode environment, if two of the op-sm transactions
are initiated on one of the receiver nodes at the same time,
there might be a possibility that glusterd  may end up in
stale lock.

Solution:
During mgmt_v3_lock a registration is made to  gf_timer_call_after
which release the lock after certain period of time

>Change-Id: I16cc2e5186a2e8a5e35eca2468b031811e093843
>BUG: 1499004
>Signed-off-by: Gaurav Yadav 

Change-Id: I16cc2e5186a2e8a5e35eca2468b031811e093843
BUG: 1557304
Signed-off-by: Gaurav Yadav

cluster/dht: Add migration checks to dht_(f)xattrop

2018-01-30T13:38:09+00:00

The dht_(f)xattrop implementation did not implement
migration phase1/phase2 checks which could cause issues
with rebalance on sharded volumes.
This does not solve the issue where fops may reach the target
out of order.

> Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c
> BUG: 1471031
> Signed-off-by: N Balachandran 

Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c
BUG: 1498081
Signed-off-by: N Balachandran

gfapi: fix handling of dot and double dot in path

2018-01-03T15:19:25+00:00

This patch is to handle "." and ".." in file path. Which means
this special dentry names will be resolved before sending fops
on the path.

Change-Id: I5e92f6d1ad1412bf432eb2488e53fb7731edb013
BUG: 1530341
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: https://review.gluster.org/17177
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy 
(cherry picked from commit 269e2ccf45ddc662d8373eb887ae6cef96e2ef37)

glusterd: Gluster should keep PID file in correct location

2017-10-25T14:03:54+00:00

Currently Gluster keeps process pid information of all the daemons
and brick processes in Gluster configuration file directory
(ie., /var/lib/glusterd/*).

These pid files should be seperate from configuration files.
Deletion of the configuration file directory might result into serious problems.
Also, /var/run/gluster is the default placeholder directory for pid files.

So, with this fix Gluster will keep all process pid information of all
processes in /var/run/gluster/* directory.

> Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
> BUG: 1258561
> Signed-off-by: Gaurav Kumar Garg 
> Signed-off-by: Saravanakumar Arumugam 
> Reviewed-on: https://review.gluster.org/13580
> Tested-by: MOHIT AGRAWAL 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> (Cherry pick from commit 220d406ad13d840e950eef001a2b36f87570058d)

BUG: 1491059
Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
Signed-off-by: Mohit Agrawal

mount/fuse: Make event-history feature configurable

2017-10-02T12:41:57+00:00

... and disable it by default.

        Backport of:
        > Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
        > Reviewed-on: https://review.gluster.org/18242
        > BUG: 1467614
        > cherry-picked from commit 956d43d6e89d40ee683547003b876f1f456f03b6

This is because having it disabled seems to improve performance.
This could be due to the lock contention by the different epoll threads
on the circular buff lock in the fop cbks just before writing their response
to /dev/fuse.

Just to provide some data - wrt ovirt-gluster hyperconverged
environment, I saw an increase in IOPs by 12K with event-history
disabled for randrom read workload.

Usage:
mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT
OR
glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT

Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
BUG: 1495430
Signed-off-by: Krutika Dhananjay

afr: heal metadata in discover code path

2017-09-17T13:37:32+00:00

******************************************************
Backport of: https://review.gluster.org/18202
Also added loc_is_nameless() to libglusterfs since the patch that
introduced it in master was not backported to release-3.10.

Note: 18202 is a squash of  17850 and 18187 in master.
******************************************************

During graph switch, if fuse sends nameless (gfid) lookups, afr takes
the discover code path to serve it. If there are pending metadata heals,
they do not happen unless an inode refresh happens as a part of
discover (which is not guaranteed to happen always).

This patch fixes it by attempting metadata heal as a part of discover,
just like how it is done in lookup code path.

Change-Id: I87c493045b9225741cad173bf3f645848697032e
BUG: 1492010
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18304
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/ec: Node uuid xattr support update for EC

2017-09-13T06:41:52+00:00

Problem:
The change in EC to return list of node uuids for
GF_XATTR_NODE_UUID_KEY was causing problems with
geo-rep.

Fix:
This patch will allow to get the single node uuid
as it was doing before with the key
"GF_XATTR_NODE_UUID_KEY", and will also allow to get
the list of node uuids by using a new key
"GF_XATTR_LIST_NODE_UUIDS_KEY". This will solve
the problem with geo-rep and any other features which
were depending on this.

>BUG: 1462790
>Change-Id: I2d9214a9658d4a41a3d6de08600884d2bda5f3eb
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/17594
>Smoke: Gluster Build System 
>Reviewed-by: Xavier Hernandez 
>Reviewed-by: Pranith Kumar Karampuri 
>CentOS-regression: Gluster Build System 
>Signed-off-by: Sunil Kumar Acharya 

BUG: 1487647
Change-Id: I2d9214a9658d4a41a3d6de08600884d2bda5f3eb
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/17667
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/dht: rebalance perf enhancement

2017-08-11T17:40:29+00:00

Problem: Throttle settings "normal" and "aggressive" for rebalance
did not have performance difference.

normal mode spawns $(no. of cores - 4)/2 threads and aggressive
spawns $(no. of cores - 4) threads. Though aggressive mode has twice
the number of threads compared to that of normal mode, there was no
performance gain when switched to aggressive mode from normal mode.

RCA:
During the course of debugging the above problem, we tried assigning
migration job to migration threads spawned by rebalance, rather than
synctasks(as there is more overhead associated to manage the task
queue and threads). This gave us a significant improvement over rebalance
under synctasks. This patch does not really gurantee that there will be a
clear performance difference between normal and aggressive mode, but this
patch certainly maximized the disk utilization for 1GBfiles run.

Results:

Test enviroment:
Gluster Config:
Number of Bricks: 2 (one brick per disk(RAID-6 12 disk))
Bricks:
Brick1: server1:/brick/test1/1
Brick2: server2:/brick/test1/1
Options Reconfigured:
performance.readdir-ahead: on
server.event-threads: 4
client.event-threads: 4

1000 files with 1GB each were created/renamed such that all files will have
server1 as cached and server2 as hashed, so that all files will be migrated.

Test machines had 24 cores each.

Results  with/without synctask based migration:
-----------------------------------------------

mode                    normal(10threads)          aggressive(20threads)

timetaken               0:55:30 (h:m:s)            0:56:3 (h:m:s)
withsynctask

timetaken
with migrator           0:38:3 (h:m:s)             0:23:41 (h:m:s)
threads

From above table it can be seen that, there is a clear 2x perf gain between
rebalance with synctask vs rebalance with migrator threads.

Additionally this patch modifies the code so that caller will have the exact error
number returned by dht_migrate_file(earlier the errno meaning was overloaded). This
will help avoiding scenarios where migration failure due to ENOENT, can result in
rebalance abort/failure.

> Change-Id: I8904e2fb147419d4a51c1267be11a08ffd52168e
> BUG: 1420166
> Signed-off-by: Susant Palai 
> Reviewed-on: https://review.gluster.org/16427
> Smoke: Gluster Build System 
> Reviewed-by: N Balachandran 
> Reviewed-by: Raghavendra G 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Signed-off-by: Susant Palai 

Change-Id: I8904e2fb147419d4a51c1267be11a08ffd52168e
BUG: 1473134
Signed-off-by: Susant Palai 
Reviewed-on: https://review.gluster.org/17834
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan