glusterfs.git/rpc, branch v3.7.15

rpc : build_prog_details should iterate program list inside critical section

2016-08-16T06:27:23+00:00

Backport of http://review.gluster.org/13428

While I was analyzing a glusterd crash from free_prog_details, a code
walkthrough detects that we iterate over the rpc svc program list without been
inside the criticial section. This opens up a possibility of a crash when there
is a concurrent writer updating the same list. Solution is to read the list
inside lock.

> Reviewed-on: http://review.gluster.org/13428
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Niels de Vos 
> Reviewed-by: Raghavendra G 

BUG: 1310970
Change-Id: Ib4b4b0022a9535e139cd3c00574aab23f07aa9d2
Signed-off-by: Atin Mukherjee 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/13488
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaushal M

socket: log the client identifier in ssl connect

2016-08-04T05:10:26+00:00

Backport of commit d308fb5e152d8c908bf4f5da81f553fbe3d0400a

> Change-Id: I4b463ecafb66de16cbe7ed23fae800bb1204f829
> BUG: 1333912
> Signed-off-by: Raghavendra Bhat 
> Reviewed-on: http://review.gluster.org/14242
> Tested-by: Vijay Bellur 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
> Smoke: Gluster Build System 
> (cherry picked from commit d308fb5e152d8c908bf4f5da81f553fbe3d0400a)

Change-Id: I2a57a206edab3e0c05ce28c299d78264c9a33d8b
BUG: 1351933
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/14844
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Atin Mukherjee

changelog/rpc: Fix rpc_clnt_t mem leaks

2016-07-27T10:13:33+00:00

Backport of http://review.gluster.org/13658

PROBLEM:
   1. Freeing up rpc_clnt object might lead to crashes. Well,
      it was not a necessity to free rpc-clnt object till now
      because all the existing use cases needs to reconnect
      back on disconnects. Hence timer code was not taking
      ref on rpc-clnt object.

      Glusterd had some use-cases that led to crash due to
      ping-timer and they fixed only those code paths that
      involve ping-timer.

      Now, since changelog has an use-case where rpc-clnt
      need to be freed up, we need to fix timer code to take
      refs

   2. In changelog, because of issue 1, only mydata was being
      freed which is incorrect. And there are races where
      rpc-clnt object would access the freed mydata which
      would lead to crashes.

      Since changelog xlator resides on brick side and is long
      living process, if multiple libgfchangelog consumers
      register to changelog and disconnect/reconnect mulitple
      times, it would result in leak of 'rpc-clnt' object
      for every connect/disconnect.

SOLUTION:
   1. Handle ref/unref of 'rpc_clnt' structure in timer
      functions properly.
   2. In changelog, unref 'rpc_clnt' in RPC_CLNT_DISCONNECT
      after disabling timers and free mydata on RPC_CLNT_DESTROY.

RPC SETUP IN CHANGELOG:
   1. changelog xlator initiates rpc server say 'changelog_rpc_server'
   2. libgfchangelog initiates one rpc server say 'libgfchangelog_rpc_server'
   3. libgfchangelog initiates rpc client and connects to 'changelog_rpc_server'
   4. In return changelog_rpc_server initiates a rpc client and connects back
      to 'libgfchangelog_rpc_server'

REF/UNREF HANDLING IN TIMER FUNCTIONS:
Let's say rpc clnt refcount = 1
   1. Take the ref before reigstering callback to timer queue
           >>>>  rpc_clnt_ref (say ref count becomes = 2)
   2. Register a callback to timer say 'callback1'
   3. If register fails:
           >>>> rpc_clnt_unref (ref count = 1)
   4. On timer expiration, 'callback1' gets called. So unref rpc clnt at the end
      in 'callback1'. This is corresponding to ref taken in step 1
           >>>> rpc_clnt_unref (ref count = 1)
   5. The cycle from step-1 to step-4 continues....until timer cancel event happens
   6. timer cancel of say 'callback1'
           If timer cancel fails:
                 Do nothing, Step-4 would have unrefd
           If timer cancel succeeds:
                 >>>> rpc_clnt_unref (ref count = 1)

Change-Id: I91389bc511b8b1a17824941970ee8d2c29a74a09
BUG: 1359363
Signed-off-by: Kotresh HR 
(cherry picked from commit 637ce9e2e27e9f598a4a6c5a04cd339efaa62076)
Reviewed-on: http://review.gluster.org/14993
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

rpc/socket: pthread resources are not cleaned up

2016-07-27T10:12:58+00:00

A socket_connect failure creates a new pthread which
is not a detached thread. As no pthread_join is called,
the thread resources are not cleaned up causing a memory leak.

Now, socket_connect creates a detached thread to handle failure.

> Change-Id: Idbf25d312f91464ae20c97d501b628bfdec7cf0c
> BUG: 1343374
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/14875
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
(cherry picked from commit 9886d568a7a8839bf3acc81cb1111fa372ac5270)

Change-Id: If0a65c50fef2a32148cf3a1d7992e63f044bf0ad
BUG: 1360553
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15019
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: Oleksandr Natalenko 
Reviewed-by: Raghavendra G

rpc/socket.c : Modify socket_poller code in case of ENODATA error code.

2016-07-26T17:07:28+00:00

Problem:  Polling failure errors are coming till volume is not come while
          SSL is enabled.

Solution: To avoid the message update one condition in socket_poller code
          It will not exit from thread in case of received ENODATA from
          ssl_do function.

Backport of commit 84e9fc2fb5fabf9d1e553a420854a306cdb8a168

> Change-Id: Ia514e99b279b07b372ee950f4368ac0d9c702d82
> BUG: 1349709
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/14786
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
> (cherry picked from commit 84e9fc2fb5fabf9d1e553a420854a306cdb8a168)

BUG: 1359651
Signed-off-by: Mohit Agrawal 

Change-Id: I86aa9955eca13d23120ba17b787f619c7de6be0c
Reviewed-on: http://review.gluster.org/14998
Tested-by: MOHIT AGRAWAL 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Jeff Darcy

rpc/socket.c: Modify approach to cleanup threads of socket_poller in socket_spawn.

2016-07-26T11:42:18+00:00

Problem: Current approach to cleanup threads of socket_poller is not appropriate.

Solution: Enable detach flag at the time of thread creation in socket_spawn.

Fix: Write a new wrapper(gf_create_detach_thread) to create detachable thread
         instead of store thread ids in a queue.

Test: Fix is verfied on gluster process, To test the patch followed below
      procedure
      Enable the client.ssl and server.ssl option on the volume
      Start the volume and count anon segment in pmap output for glusterd process
      pmap -x  | grep "\[ anon \]" | wc -l
      Stop the volume and check again count of anon segment it should not increase.

Backport of commit 2ee48474be32f6ead2f3834677fee89d88348382

> Signed-off-by: Mohit Agrawal 
> Change-Id: Ib8f7ec7504ec8f6f74b45ce6719b6fb47f9fdc37
> BUG: 1336508
> Reviewed-on: http://review.gluster.org/14694
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

BUG: 1354394
Change-Id: I271e83e7a210ecd27a7471c53147ceb837a33cad
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/14886
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

rpc: invalid argument when function setsockopt sets option TCP_USER_TIMEOUT

2016-07-12T12:07:16+00:00

If option "transport.tcp-user-timeout" hasn't been setted, glusterd's
priv->timeout will be -1, which will cause invalid argument when
set TCP_USER_TIMEOUT.

Cherry picked from commit b2c73cbf423de6201f956f522b7429615c88869d:
> Change-Id: Ibc16264ceac0e69ab4a217ffa27c549b9fa21df9
> BUG: 1349657
> Signed-off-by: Zhou Zhengping 
> Reviewed-on: http://review.gluster.org/14785
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

Change-Id: Ibc16264ceac0e69ab4a217ffa27c549b9fa21df9
BUG: 1354404
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14889
Smoke: Gluster Build System 
Reviewed-by: Zhou Zhengping 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

xdr/nfs: free complete groupnode structure

2016-07-03T11:15:15+00:00

The groupnode->gr_next pointer is not traversed upon free. This is
currently not a problem, because the pointer is never used. However the
correct way to free a groupnode should check the ->gr_next pointer and
free any of the groups that it encounters.

This problem was identified while correcting a problem with the MOUNT
protocol. The change "nfs: build exportlist with multiple groups" starts
to use ->gr_next.

This is backport of below mainline fix -
	http://review.gluster.org/#/c/14666/

Change-Id: I9d04eaf4c65bdb8db136321d60e70789da1739d7
BUG: 1343290
Signed-off-by: Niels de Vos 
Signed-off-by: Bipin Kunal 
Reviewed-on: http://review.gluster.org/14697
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

rpc: check the right variable after gf_strdup

2016-05-27T09:07:12+00:00

> Reviewed-on: http://review.gluster.org/13934
> Smoke: Gluster Build System 
> Tested-by: Jeff Darcy 
> Reviewed-by: Anoop C S 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Vijay Bellur 
> CentOS-regression: Gluster Build System 

(cherry picked from commit 1c9c776352c60deeda51be66fda6d44bf06d3796)

Change-Id: If4628bd37f2c85a070f6c3b3e0583d939100d7ec
BUG: 1332404
Signed-off-by: Zhou Zhengping 
Reviewed-on: http://review.gluster.org/14494
Smoke: Gluster Build System 
Tested-by: Anoop C S 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

rpc: change client insecure port ceiling from 65535 to 49151

2016-05-20T10:55:35+00:00

current port allocation to various processes (clumsy):

 1023 - 1       -> client ports range if bind secure is turned on
49151 - 1024    -> fall back to this, if in above case ports exhaust
65535 - 1024    -> client port range if bind insecure is on
49152 - 65535   -> brick port range

now, we have segregated port ranges 0 - 65535 to below 3 ranges

 1023 - 1       -> client ports range if bind secure is turned on
49151 - 1024    -> client port range if bind insecure is on
                   (fall back to this, if in above case ports exhaust)
49152 - 65535   -> brick port range

so now we have a clean segregation of port mapping

Backport of:
> Change-Id: Ie3b4e7703e0bbeabbe0adbdd6c60d9ef78ef7c65
> BUG: 1335776
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14326
> Tested-by: Prasanna Kumar Kalever 
> Reviewed-by: Raghavendra Talur 
> Tested-by: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Raghavendra G 

Change-Id: Ie3b4e7703e0bbeabbe0adbdd6c60d9ef78ef7c65
BUG: 1335813
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14412
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G