glusterfs.git/libglusterfs, branch v3.12.2

cluster/ec: Improve performance with xattrop update

2017-10-12T18:52:34+00:00

Existing EC code updates the xattr on the subvolume
in a sequential pattern resulting in very poor performance.

With this fix EC now updates the xattr on the subvolume
in parallel which improves the xattr update performance.

>BUG: 1445663
>Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486
>Signed-off-by: Sunil Kumar Acharya 

BUG: 1499150
Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486
Signed-off-by: Sunil Kumar Acharya

glusterd: fix client io-threads option for replicate volumes

2017-10-09T12:21:08+00:00

Backport of https://review.gluster.org/#/c/18430/

Problem:
Commit ff075a3d6f9b142911d25c27fd209838782bfff0 disabled loading
client-io-threads for replicate volumes (it was set to on by default in
commit e068c1997314046658dd502e9118dab32decf879) due to performance
issues but in doing so, inadvertently failed to load the xlator even if
the user explicitly enabled the option using the volume set command.
This was despite returning returning sucess for the volume set.

Fix:
Modify the check in perfxl_option_handler() and add checks in volume
create/add-brick/remove-brick code paths, tying it all to
GD_OP_VERSION_3_12_2.

Change-Id: Ib612973a999a7da818cc926f5c2601b1f0794fcf
BUG: 1499158
Signed-off-by: Ravishankar N

mount/fuse: Make event-history feature configurable

2017-10-05T12:02:42+00:00

... and disable it by default.

        Backport of:
        > Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
        > Reviewed-on: https://review.gluster.org/18242
        > BUG: 1467614
        > cherry-picked from commit 956d43d6e89d40ee683547003b876f1f456f03b6

This is because having it disabled seems to improve performance.
This could be due to the lock contention by the different epoll threads
on the circular buff lock in the fop cbks just before writing their response
to /dev/fuse.

Just to provide some data - wrt ovirt-gluster hyperconverged
environment, I saw an increase in IOPs by 12K with event-history
disabled for randrom read workload.

Usage:
mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT
OR
glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT

Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
BUG: 1495397
Signed-off-by: Krutika Dhananjay

event/epoll: don't call handler for events received after a pollerr

2017-09-11T04:59:34+00:00

we register socket with EPOLLONESHOT, which means it has to be
explicitly added back through epoll_ctl to receive more
events. Normally we do this once the handler completes processing of
current event. But event_select_on_epoll is one asynchronous codepath
where socket can be added back for polling while an event on the same
socket is being processed. event_select_on_epoll has a check whether
an event is being processed in the form of slot->in_handler. But this
check is not sufficient enough to prevent parallel events as
slot->in_handler is not atomically incremented with respect to
reception of the event. This means following imaginary sequence of
events can happen:

* epoll_wait returns with a POLLERR - say POLLERR1 - on a socket
  (sock1) associated with slot s1. socket_event_handle_pollerr is yet
  to be invoked.
* an event_select_on called from __socket_ioq_churn which was called
  in request/reply/msg submission codepath (as opposed to
  __socket_ioq_churn called as part of POLLOUT handling - we cannot
  receive a POLLOUT due to EPOLLONESHOT) adds back sock1 for polling.
* since sock1 was added back for polling in step 2 and our polling is
  level-triggered, another thread picks up another POLLERR event - say
  POLLERR2. socket_event_handler is invoked as part of processing
  POLLERR2 and it completes execution setting priv->sock to -1.
* event_unregister_epoll called as part of __socket_reset due to
  POLLERR1 would receive fd as -1 resulting in assert failure.

Also, since the first pollerr event has done rpc_transport_unref,
subsequent parallel events (not just pollerr, but other events too)
could be acting on a freed up transport too.

>Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb
>BUG: 1486134
>Signed-off-by: Raghavendra G 
>Reviewed-on: https://review.gluster.org/18129
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: mohammed rafi  kc 

(cherry picked from commit b1b49997574eeb7c6a42e6e8257c81ac8d2d7578)
Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb
BUG: 1489296
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/18223
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan

perf/qr: Use a ref-ed data to extract content

2017-09-07T06:57:43+00:00

qr_content_extract used dict_get to get the value of
the GF_CONTENT_KEY key. dict_get does not ref the data
before returning it so QR could be acting on freed memory
if another thread deletes the key before then.
This patch also fixes a race in dict_get_with_ref.

Fix: Use dict_get_with_ref to retrieve the file contents.

> BUG: 1484709
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18115
> Smoke: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Tested-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Ib1a7a70bb92eed7e70747ec530e0b3edc53127ec
BUG: 1486538
(cherry picked from commit 414d3e92fc56f08e320a3aa65b6b18e65b384551)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18145
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: MOHIT AGRAWAL

glusterd: glusterd fails to start if peers file has blank line

2017-08-29T13:52:37+00:00

Problem:
On start of glusterd service, glusterd fetch data from store, while
parsing data from store if peers file consists of blank line glusterd
fails to start.

Fix:
With this fix while parsing peers file glusterd will skip blank lines
if it contains any.

>Reviewed-on: https://review.gluster.org/18066
>Tested-by: Gaurav Yadav 
>Smoke: Gluster Build System 
>Reviewed-by: Prashanth Pai 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Atin Mukherjee 
>Reviewed-by: Niels de Vos 

Change-Id: I53cd65a54de5f57baef292b2118b70ffb7f99388
BUG: 1486107
Signed-off-by: Gaurav Yadav 
Reviewed-on: https://review.gluster.org/18124
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

glusterd: Gluster should keep PID file in correct location

2017-08-12T13:38:30+00:00

Currently Gluster keeps process pid information of all the daemons
and brick processes in Gluster configuration file directory
(ie., /var/lib/glusterd/*).

These pid files should be seperate from configuration files.
Deletion of the configuration file directory might result into serious problems.
Also, /var/run/gluster is the default placeholder directory for pid files.

So, with this fix Gluster will keep all process pid information of all
processes in /var/run/gluster/* directory.

> BUG: 1258561
> Signed-off-by: Gaurav Kumar Garg 
> Signed-off-by: Saravanakumar Arumugam 
> Reviewed-on: https://review.gluster.org/13580
> Tested-by: MOHIT AGRAWAL 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> cherry pick from commit 220d406ad13d840e950eef001a2b36f87570058d

BUG: 1480459
Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
Signed-off-by: Mohit Agrawal 
Reviewed-on: https://review.gluster.org/18023
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

gfapi: Duplicate the buffer sent in setxattr calls

2017-08-12T13:33:55+00:00

Issue:
The caller of glfs_setxattr sends a buffer to set as the value.
We create a dict in which the pointer to the value is set.
Underlying layers like md-cache take a ref on this dict to store
the value for a longer time. But the moment setxattr is complete,
the caller of glfs_setxattr can free the value memory.

Solution:
memcpy the setxattr value to the gluster buffer.

> Reviewed-on: https://review.gluster.org/17967
> Reviewed-by: soumya k 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
> (cherry picked from commit e11296f8e52b7e3b13d21b41d4fa34baea878edf)

Change-Id: I58753fe702e8b7d0f6c4f058714c65d0ad5d7a0a
BUG: 1479655
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/18001
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

glusterfsd: allow subdir mount

2017-08-04T13:34:21+00:00

Changes:

1. Take subdir mount option in client (mount.gluster / glusterfsd)
2. Pass the subdir mount to server-handshake (from client-handshake)
3. Handle subdir-mount dir's lookup in server-first-lookup and handle
   all fops resolution accordingly with proper gfid of subdir
4. Change the auth/addr module to handle the multiple subdir entries
   in option, and valid parsing.

How to use the feature:

`# mount -t glusterfs $hostname:/$volname/$subdir /$mount_point`
Or
`# mount -t glusterfs $hostname:/$volname -osubdir_mount=$subdir /$mount_point`

Option can be set like:

`# gluster volume set  auth.allow "/subdir1(192.168.1.*),/(192.168.10.*),/subdir2(192.168.8.*)"`

Updates #175

> Reviewed-At: https://review.gluster.org/17141/

Change-Id: I7ea57f76ddbe6c3862cfe02e13f89e8a39719e11
Signed-off-by: Amar Tumballi 
Reviewed-on: https://review.gluster.org/17968
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

glusterd: Add geo-replication session details to get-state output

2017-08-04T13:32:52+00:00

This commit adds support to the get-state CLI to capture details
on geo-replication session as obtained in
`gluster volume geo-replication status detail` in its output.

> Reviewed-on: https://review.gluster.org/17941
> Smoke: Gluster Build System 
> Reviewed-by: Shubhendu Tripathi 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 

(cherry picked from commit 2e7daeffef05c6100cbcc39f1be62935711db3eb)

Fixes: #291
Change-Id: I2fbcba70bfdaf439522637234805545194777ed4
Signed-off-by: Samikshan Bairagya 
Reviewed-on: https://review.gluster.org/17971
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan