glusterfs.git/rpc/rpc-lib, branch v3.7.15

rpc : build_prog_details should iterate program list inside critical section

2016-08-16T06:27:23+00:00

Backport of http://review.gluster.org/13428

While I was analyzing a glusterd crash from free_prog_details, a code
walkthrough detects that we iterate over the rpc svc program list without been
inside the criticial section. This opens up a possibility of a crash when there
is a concurrent writer updating the same list. Solution is to read the list
inside lock.

> Reviewed-on: http://review.gluster.org/13428
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Niels de Vos 
> Reviewed-by: Raghavendra G 

BUG: 1310970
Change-Id: Ib4b4b0022a9535e139cd3c00574aab23f07aa9d2
Signed-off-by: Atin Mukherjee 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/13488
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaushal M

changelog/rpc: Fix rpc_clnt_t mem leaks

2016-07-27T10:13:33+00:00

Backport of http://review.gluster.org/13658

PROBLEM:
   1. Freeing up rpc_clnt object might lead to crashes. Well,
      it was not a necessity to free rpc-clnt object till now
      because all the existing use cases needs to reconnect
      back on disconnects. Hence timer code was not taking
      ref on rpc-clnt object.

      Glusterd had some use-cases that led to crash due to
      ping-timer and they fixed only those code paths that
      involve ping-timer.

      Now, since changelog has an use-case where rpc-clnt
      need to be freed up, we need to fix timer code to take
      refs

   2. In changelog, because of issue 1, only mydata was being
      freed which is incorrect. And there are races where
      rpc-clnt object would access the freed mydata which
      would lead to crashes.

      Since changelog xlator resides on brick side and is long
      living process, if multiple libgfchangelog consumers
      register to changelog and disconnect/reconnect mulitple
      times, it would result in leak of 'rpc-clnt' object
      for every connect/disconnect.

SOLUTION:
   1. Handle ref/unref of 'rpc_clnt' structure in timer
      functions properly.
   2. In changelog, unref 'rpc_clnt' in RPC_CLNT_DISCONNECT
      after disabling timers and free mydata on RPC_CLNT_DESTROY.

RPC SETUP IN CHANGELOG:
   1. changelog xlator initiates rpc server say 'changelog_rpc_server'
   2. libgfchangelog initiates one rpc server say 'libgfchangelog_rpc_server'
   3. libgfchangelog initiates rpc client and connects to 'changelog_rpc_server'
   4. In return changelog_rpc_server initiates a rpc client and connects back
      to 'libgfchangelog_rpc_server'

REF/UNREF HANDLING IN TIMER FUNCTIONS:
Let's say rpc clnt refcount = 1
   1. Take the ref before reigstering callback to timer queue
           >>>>  rpc_clnt_ref (say ref count becomes = 2)
   2. Register a callback to timer say 'callback1'
   3. If register fails:
           >>>> rpc_clnt_unref (ref count = 1)
   4. On timer expiration, 'callback1' gets called. So unref rpc clnt at the end
      in 'callback1'. This is corresponding to ref taken in step 1
           >>>> rpc_clnt_unref (ref count = 1)
   5. The cycle from step-1 to step-4 continues....until timer cancel event happens
   6. timer cancel of say 'callback1'
           If timer cancel fails:
                 Do nothing, Step-4 would have unrefd
           If timer cancel succeeds:
                 >>>> rpc_clnt_unref (ref count = 1)

Change-Id: I91389bc511b8b1a17824941970ee8d2c29a74a09
BUG: 1359363
Signed-off-by: Kotresh HR 
(cherry picked from commit 637ce9e2e27e9f598a4a6c5a04cd339efaa62076)
Reviewed-on: http://review.gluster.org/14993
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

rpc: check the right variable after gf_strdup

2016-05-27T09:07:12+00:00

> Reviewed-on: http://review.gluster.org/13934
> Smoke: Gluster Build System 
> Tested-by: Jeff Darcy 
> Reviewed-by: Anoop C S 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Vijay Bellur 
> CentOS-regression: Gluster Build System 

(cherry picked from commit 1c9c776352c60deeda51be66fda6d44bf06d3796)

Change-Id: If4628bd37f2c85a070f6c3b3e0583d939100d7ec
BUG: 1332404
Signed-off-by: Zhou Zhengping 
Reviewed-on: http://review.gluster.org/14494
Smoke: Gluster Build System 
Tested-by: Anoop C S 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd: add defence mechanism to avoid brick port clashes

2016-05-05T03:23:19+00:00

Intro:
Currently glusterd maintain the portmap registry which contains ports that
are free to use between 49152 - 65535, this registry is initialized
once, and updated accordingly as an then when glusterd sees they are been
used.

Glusterd first checks for a port within the portmap registry and gets a FREE
port marked in it, then checks if that port is currently free using a connect()
function then passes it to brick process which have to bind on it.

Problem:
We see that there is a time gap between glusterd checking the port with
connect() and brick process actually binding on it. In this time gap it could
be so possible that any process would have occupied this port because of which
brick will fail to bind and exit.

Case 1:
To avoid the gluster client process occupying the port supplied by glusterd :

we have separated the client port map range with brick port map range more @
http://review.gluster.org/#/c/13998/

Case 2: (Handled by this patch)
To avoid the other foreign process occupying the port supplied by glusterd :

To handle above situation this patch implements a mechanism to return EADDRINUSE
error code to glusterd, upon which a new port is allocated and try to restart
the brick process with the newly allocated port.

Note: Incase of glusterd restarts i.e. runner_run_nowait() there is no way to
handle Case 2, becuase runner_run_nowait() will not wait to get the return/exit
code of the executed command (brick process). Hence as of now in such case,
we cannot know with what error the brick has failed to connect.

This patch also fix the runner_end() to perform some cleanup w.r.t
return values.

Backport of:
> Change-Id: Iec52e7f5d87ce938d173f8ef16aa77fd573f2c5e
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14043
> Tested-by: Prasanna Kumar Kalever 
> Reviewed-by: Atin Mukherjee 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Signed-off-by: Prasanna Kumar Kalever 

Change-Id: Ief247b4d4538c1ca03e73aa31beb5fa99853afd6
BUG: 1323564
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14208
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd: try to connect on GF_PMAP_PORT_FOREIGN aswell

2016-05-02T03:53:04+00:00

This patch fix couple of things mentioned below:

1. previously we use to try to connect on only GF_PMAP_PORT_FREE
in the pmap_registry_alloc(), it could happen that some foreign process
would have freed the port by this time ?, hence it is worth giving a try on
GF_PMAP_PORT_FOREIGN ports as well instead of wasting them all.

2. fix pmap_registry_remove() to mark the port asGF_PMAP_PORT_FREE

3. added useful comments on gf_pmap_port_type enum members

Backport of:
> Change-Id: Id2aa7ad55e76ae3fdece21bed15792525ae33fe1
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14080
> Tested-by: Prasanna Kumar Kalever 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 

Change-Id: Ib7852aa1c55611e81c78341aace4d374d516f439
BUG: 1323564
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14117
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

Revert "glusterd: Bug fixes for IPv6 support"

2016-04-16T02:18:23+00:00

This reverts commit b33f3c95ec9c8112e6677e09cea05c4c462040d0.

This commit exposes some issues with management encryption that prevents
GlusterFS from operating properly. This will be added again once
problems with management encryption are fixed.

afr: add mtime based split-brain resolution to CLI

2016-04-04T17:33:56+00:00

Backport of http://review.gluster.org/#/c/13828/

Extended the CLI to include support for split-brain resolution based on
mtime. The command syntax is:

$:gluster volume heal  split-brain latest-mtime 

where  can be either the full file name as seen from the root of the
volume (or) the gfid-string representation of the file.

Change-Id: I7a16f72ff1a4495aa69f43f22758a9404e958b4f
BUG: 1321748
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13838
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Anuradha Talur 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

rpc: Connect back only if rpc is not disabled

2016-03-31T04:36:06+00:00

This is a backport of the below fix -
 http://review.gluster.org/13592

As discussed over http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/14284,
patch #13456 caused a regression where in if there are any pending
rpc invocations, we end up accessing freed object. This patch
fixes it by allowing reconnect during rpc submit only if rpc
is not disabled.

This is to fix regression caused by below patch -
http://review.gluster.org/#/c/13507/

Change-Id: I4ef4dd52bd42368bb89129f98bc973e46c6a39f4
BUG: 1311441
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13592
Reviewed-on: http://review.gluster.org/13824
Tested-by: soumya k 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

rpc: Fix for rpc_transport_t leak

2016-03-24T15:02:18+00:00

The transport object needs to get unref'ed when the rpc clnt
object is getting destroyed. But currently in rpc_clnt_disable()
we set conn->trans to NULL before it gets unref'ed leading to
transport object leak.

This change is to fix it by setting conn-tran to NULL only when
it is being unref'ed.

This is backport of the below patch
	- http://review.gluster.org/13456

Change-Id: I79ba34e28ae19eb616035f36bbed1c2f47875b94
BUG: 1311441
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/13456
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/13507
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd/rpc : Discard duplicate Disconnect events

2016-03-23T14:50:23+00:00

Backport of http://review.gluster.org/#/c/13790/

If a peer rpc disconnect event has been already processed, skip the furthers as
processing them are overheads and sometimes may lead to a crash like due to a
double free

Change-Id: Iec589ce85daf28fd5b267cb6fc82a4238e0e8adc
BUG: 1320374
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/13790
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/13813