glusterfs.git/xlators/protocol, branch v3.12.6

protocol/server: fix the comparision logic in case of subdir mount

2017-11-06T11:43:42+00:00

without the fix, the stat entry on a file would return inode==1 for
many files, in case of subdir mount

This happened with the confusion of return value of 'gf_uuid_compare()',
it is more like strcmp, instead of a gf_boolean return value, and hence 
resulted in the bug.

Change-Id: I31b8cbd95eaa3af5ff916a969458e8e4020c86bb
BUG: 1505527
Signed-off-by: Amar Tumballi 
(cherry picked from commit 2ade36cd98ea0f5bd2a8f619a19c20438318afaf)

protocol/client: handle the subdir handshake properly for add-brick

2017-11-06T11:43:04+00:00

There should be different way we handle handshake in case of subdir
mount for the first time, and in case of subsequent graph changes.

Change-Id: I2a7ba836433bb0a0f4a861809e2bb0d7fbc4da54
BUG: 1505323
Signed-off-by: Amar Tumballi 
(cherry picked from commit 9aa574a51b84717c1f3949ed2e28a49e49840a93)

protocol-auth: use the proper validation method

2017-10-25T11:35:25+00:00

Currently, server protocol's init and glusterd's option
validation methods are different, causing an issue. They
should be same for having consistent behavior

Change-Id: Ibbf9a18c7192b2d77f9b7675ae7da9b8d2fe5de4
BUG: 1501315
Signed-off-by: Amar Tumballi

eventsapi: Fix issue with CLIENT_CONNECT event

2017-10-05T12:00:22+00:00

A mismatch in event format causing below error in events.log when
it detects CLIENT_CONNECT event.

    [2017-09-19 09:35:06,785] WARNING [glustereventsd - 46:handle] -
    Unable to parse Event 1505793906 97
    client_uid=f241-16363-2017/09/19-04:05:06:747558-gv1-client-
    0-0-0;client_identifier=192.168.122.208:49150;server_identifier=
    192.168.122.208:49152;brick_path=/bricks/b1,subdir_mount=(null)

> Reviewed-on: https://review.gluster.org/18322
> Reviewed-by: Atin Mukherjee 
> Smoke: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Prashanth Pai 
> (cherry picked from commit 23649f9ab73448737ee5d9509502f96e4775dca3)

BUG: 1492061
Change-Id: Ie6d507725a7e6b54fca44651f9c5e66eca2be244
Signed-off-by: Aravinda VK

rpc: destroy transport after client_t

2017-09-07T06:49:51+00:00

Problem:
1. Ref counting increment on the client_t object is done in
   rpcsvc_request_init() which is incorrect.
2. Ref not taken when delegating to grace_time_handler()

Solution:
1. Only fop requests which require processing down the graph via
   stack 'frames' now ref count the request in get_frame_from_request()
2. Take ref on client_t object in server_rpc_notify() but avoid
   dropping in RPCSVC_EVENT_TRANSPORT_DESRTROY. Drop the ref
   unconditionally when exiting out of grace_time_handler().
   Also, avoid dropping ref on client_t in
   RPCSVC_EVENT_TRANSPORT_DESTROY when ref mangement as been
   delegated to grace_time_handler()

mainline:
> BUG: 1481600
> Reported-by: Raghavendra G 
> Signed-off-by: Milind Changire 
> Reviewed-on: https://review.gluster.org/17982
> Tested-by: Raghavendra G 
> Reviewed-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
(cherry picked from commit 24b95089a18a6a40e7703cb344e92025d67f3086)

Change-Id: Ic16246bebc7ea4490545b26564658f4b081675e4
BUG: 1487033
Reported-by: Raghavendra G 
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/18156
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G 
CentOS-regression: Gluster Build System

glusterfsd: allow subdir mount

2017-08-04T13:34:21+00:00

Changes:

1. Take subdir mount option in client (mount.gluster / glusterfsd)
2. Pass the subdir mount to server-handshake (from client-handshake)
3. Handle subdir-mount dir's lookup in server-first-lookup and handle
   all fops resolution accordingly with proper gfid of subdir
4. Change the auth/addr module to handle the multiple subdir entries
   in option, and valid parsing.

How to use the feature:

`# mount -t glusterfs $hostname:/$volname/$subdir /$mount_point`
Or
`# mount -t glusterfs $hostname:/$volname -osubdir_mount=$subdir /$mount_point`

Option can be set like:

`# gluster volume set  auth.allow "/subdir1(192.168.1.*),/(192.168.10.*),/subdir2(192.168.8.*)"`

Updates #175

> Reviewed-At: https://review.gluster.org/17141/

Change-Id: I7ea57f76ddbe6c3862cfe02e13f89e8a39719e11
Signed-off-by: Amar Tumballi 
Reviewed-on: https://review.gluster.org/17968
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

program/GF-DUMP: Shield ping processing from traffic to Glusterfs

2017-07-18T10:45:29+00:00

Program

Since poller thread bears the brunt of execution till the request is
handed over to io-threads, poller thread experiencies lock
contention(s) in the control flow till io-threads, which slows it
down. This delay invariably affects reading ping requests from network
and responding to them, resulting in increased ping latencies, which
sometimes results in a ping-timer-expiry on client leading to
disconnect of transport. So, this patch aims to free up poller thread
from executing code of Glusterfs Program. We do this by making

* Glusterfs Program registering itself asking rpcsvc to execute its
  actors in its own threads.
* GF-DUMP Program registering itself asking rpcsvc to _NOT_ execute
  its actors in its own threads. Otherwise program's ownthreads become
  bottleneck in processing ping traffic. This means that poller thread
  reads a ping packet, invokes its actor and hands the response msg to
  transport queue.

Change-Id: I526268c10bdd5ef93f322a4f95385137550a6a49
Signed-off-by: Raghavendra G 
BUG: 1421938
Reviewed-on: https://review.gluster.org/17105
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Jeff Darcy

mgtm/core : use sha hash function for volfile check

2017-07-10T05:07:11+00:00

We are storing the entire volfile and using this to check
volfile change. With brick multiplexing there will be lot
of graphs per process which will increase the memory foot
print of the process. So instead of storing the entire
graph we could use sha256 and we can compare the hash to
see whether volfile change happened or not.

Also with Brick multiplexing, the direct comparison of vol
file is not correct. There are two problems.

Problem 1:

We are currently storing one single graph (the last
updated volfile) whereas, what we need is the entire
graph with all atttached bricks.

If we fix this issue, we have second problem

Problem 2:
With multiplexing we have a graph that contains multiple
bricks. But what we are checking as part of the reconfigure
is, comparing the entire graph with one single graph,
which will always fail.

Solution:
We create list in glusterfs_ctx_t that stores sha256 hash
of individual brick graphs. When a graph changes happens
we compare the stored hash and the current hash. If the
hash matches, then no need for reconfigure. Otherwise we
first do the reconfigure and then update the hash.

For now, gfapi has not changed this way. Meaning when gfapi
volfile fetch or reconfigure happens, we still store the
entire graph and compare, each memory.

This is fine, because libgfapi will not load brick graphs.
But changing the libgfapi will make the code similar in
both glusterfsd-mgmt and api. Also it helps to reduce some
memory.

Change-Id: I9df917a771a52b95622ab8f63af34ec390163a77
BUG: 1467986
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: https://review.gluster.org/17709
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Amar Tumballi

groups: don't allocate auxiliary gid list on stack

2017-07-06T18:26:08+00:00

When glusterfs wants to retrieve the list of auxiliary gids
of a user, it typically allocates a sufficiently big gid_t
array on stack and calls getgrouplist(3) with it. However,
"sufficiently big" means to be of maximum supported gid list
size, which in GlusterFS is GF_MAX_AUX_GROUPS = 64k.
That means a 64k * sizeof(gid_t) = 256k allocation, which is
big enough to overflow the stack in certain cases.

A further observation is that stack allocation of the gid list
brings no gain, as in all cases the content of the gid list
eventually gets copied over to a heap allocated buffer.

So we add a convenience wrapper of getgrouplist to libglusterfs
called gf_getgrouplist which calls getgrouplist with a sufficiently
big heap allocated buffer (it takes care of the allocation too).
We are porting all the getgrouplist invocations to gf_getgrouplist
and thus eliminate the huge stack allocation.

BUG: 1464327
Change-Id: Icea76d0d74dcf2f87d26cb299acc771ca3b32d2b
Signed-off-by: Csaba Henk 
Reviewed-on: https://review.gluster.org/17706
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Amar Tumballi 
CentOS-regression: Gluster Build System

Link against missed libraries to resolve symbols

2017-07-03T10:58:14+00:00

When external programs perform a dlopen("..so", RTLD_LAZY|RTLD_LOCAL)
on some shared objects like xlators, it can fail with dlerror set to
error string "undefined symbol ".

This was observed for the following shared objects: fuse.so, quota.so,
quotad.so, server.so, libgfrpc.so and socket.so

P.S: This was found while running a go program which fetches the list
of xlator options (volume_option_t) from xlator's shared object.

BUG: 1193929
Change-Id: I7b958409cf11fb67c2be32a3f85a96fb1260236b
Signed-off-by: Prashanth Pai 
Reviewed-on: https://review.gluster.org/17659
Smoke: Gluster Build System 
Reviewed-by: Amar Tumballi 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy