glusterfs.git/xlators/protocol/client/src, branch v3.10.0

protocol/client: Fix double free of client fdctx destroy

2017-02-15T12:33:24+00:00

This patch fixes the race between fd re-open code and fd release code,
both of which free the fd context due to a race in certain variable
checks as explained below:

1. client process (shd in the case of this BZ) sends an opendir to its
children (client xlators) which send the fop to the bricks to get a valid fd.

2. Client xlator loses connection to the brick. fdctx->remotefd is -1

3. Client re-establishes connection. After handshake, it reopens the dir
and sets fdctx->remotefd to a valid fd in client3_3_reopendir_cbk().

4. Meanwhile, shd sends a fd unref after it is done with the opendir.
This triggers a releasedir (since fd->refcount becomes 0).

5. client3_3_releasedir() sees that fdctx-->remotefd is a valid number
(i.e not -1), sets fdctx->released=1 and calls  client_fdctx_destroy()

6. As a continuation of step3, client_reopen_done() is called by
client3_3_reopendir_cbk(), which sees that fdctx->released==1 and
again calls client_fdctx_destroy().

Depending on when step-5 does GF_FREE(fdctx), we may crash at any place in
step-6 in client3_3_reopendir_cbk() when it tries to access
fdctx->{whatever}.


> Reviewed-on: https://review.gluster.org/16521
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 

(cherry picked from commit 25fc74f9d1f2b1e7bab76485a99f27abadd10b7b)
Change-Id: Ia50873d11763e084e41d2a1f4d53715438e5e947
BUG: 1422350
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/16619
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

core: run many bricks within one glusterfsd process

2017-02-02T00:54:58+00:00

This patch adds support for multiple brick translator stacks running in
a single brick server process.  This reduces our per-brick memory usage
by approximately 3x, and our appetite for TCP ports even more.  It also
creates potential to avoid process/thread thrashing, and to improve QoS
by scheduling more carefully across the bricks, but realizing that
potential will require further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global
option.  By default it's off, and bricks are started in separate
processes as before.  If multiplexing is enabled, then *compatible*
bricks (mostly those with the same transport options) will be started in
the same process.

Backport of:
> Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/14763

Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8
BUG: 1418091
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16496
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

glusterd: Add info on op-version for clients in vol status output

2017-01-12T18:20:59+00:00

Currently the `gluster volume status  clients` command
gives us the following information on clients:
1. Brick name
2. Client count for each brick
3. hostname:port for each client
4. Bytes read and written for each client

There is no information regarding op-version for each client. This
patch adds that to the output.

Change-Id: Ib2ece93ab00c234162bb92b7c67a7d86f3350a8d
BUG: 1409078
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/16303
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

socket: socket disconnect should wait for poller thread exit

2016-12-22T04:49:19+00:00

When SSL is enabled or if "transport.socket.own-thread" option is set
then socket_poller is run as different thread. Currently during
disconnect or PARENT_DOWN scenario we don't wait for this thread
to terminate. PARENT_DOWN will disconnect the socket layer and
cleanup resources used by socket_poller.

Therefore before disconnect we should wait for poller thread to exit.

Change-Id: I71f984b47d260ffd979102f180a99a0bed29f0d6
BUG: 1404181
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/16141
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaushal M 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Raghavendra G

client,server: Free xdr-allocated compound req/rsp arrays

2016-12-21T06:58:33+00:00

Change-Id: I9e1829f4f73f001bba33005032e030f289e8d003
BUG: 1406252
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/16210
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

protocol/client: Fix potential mem-leaks

2016-12-16T09:07:27+00:00

Commit 93eaeb9c93be3232f24e840044d560f9f0e66f71 introduces
leaks in INODELK callback where a dict is unserialized twice,
leading to dict leaks.

Change-Id: I219ccb2279f237ebc2e4fc366af4775a461929b8
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/16156
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

afr, client: More mem-leak fixes in COMPOUND fop cbk

2016-12-05T01:28:40+00:00

Bugs found and fixed:
1. Use correct subvolume index in pre-op-writev compound cbk
2. Prevent use-after-free of local->compound_args members in
   compound fops cbk in protocol/client
3. Fix xdata and xattr leaks in client_process_response
4. Fix possible leak of xdata in client_pre_writev() in
   test mode.
5. Free req->compound_req_array.compound_req_array_val as well
   after freeing its members
6. Free tmp_rsp->flock.lk_owner.lk_owner_val in LK fop.

Change-Id: I15b646d7d4e0e5cd4ea3d2d6452c815cf2eaf68f
BUG: 1401218
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/16020
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System

protocol/client: Fix iobref and iobuf leaks in COMPOUND fop

2016-11-17T06:25:54+00:00

Change-Id: I408879aa2bbd8ea176fbc0d0eba5567e5df1b2b3
BUG: 1395687
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15860
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

protocol/client: reduce memory usage

2016-10-25T06:08:45+00:00

readdirp calls use a lot of memory in case
of a large number of files. The dict->extra_free
is not used here so free buf immediately.

Change-Id: I097f5dde2df471f5834264152711110a3bdb7e9a
BUG: 1380249
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15593
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

compound fops: Fix file corruption issue

2016-10-24T14:11:08+00:00

1. Address of a local variable @args is copied into state->req
in server3_3_compound (). But even after the function has gone out of
scope, in server_compound_resume () this pointer is accessed and
dereferenced. This patch fixes that.

2. Compound fops, by virtue of NOT having a vector sizer (like the one
writev has), ends up having both the header and the data (in case one of
its member fops is WRITEV) in the same hdr_iobuf. This buffer was not
being preserved through the lifetime of the compound fop, causing it to
be overwritten by a parallel write fop, even when the writev associated
with the currently executing compound fop is yet to hit the desk, thereby
corrupting the file's data. This is fixed by associating the hdr_iobuf with
the iobref so its memory remains valid through the lifetime of the fop.

3. Also fixed a use-after-free bug in protocol/client in compound fops cbk,
missed by Linux but caught by NetBSD.

Finally, big thanks to Pranith Kumar K and Raghavendra Gowdappa for their
help in debugging this file corruption issue.

Change-Id: I6d5c04f400ecb687c9403a17a12683a96c2bf122
BUG: 1378778
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15654
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System