rpc-transport: socket_poller fixes for proper working of mgmt encryption

socket_poller, the polling function used by ssl own_thread, had two
issues which lead to GlusterD crashes when using management encryption

Issue 1
-------
socket_poller calls functions which require THIS to be set. But, THIS
was being set conditionally. Because of this, functions could sometimes
be called without THIS being set. For example, rpc_transport_notify
could be called for an accepted client socket without THIS being set, as
THIS was only set it the transport wasn't yet connected. This would
cause the process to crash when THIS was accessed by the called
functions.

To fix this, THIS is being set at the start of socket_poller
unconditionally.

Issue 2
-------
DISCONNECT notify was being sent on the listener transport instead of
the client transport. The DISCONNECT event was converted to a
LISTENER_DEAD event in rpcsvc_handle_disconnect, as it could not find
the listener socket of the listener socket. GlusterD was notified of a
LISTENER_DEAD event instead of a DISCONNECT and failed to remove the
client transport from its xprt_list. The transport would subsequently
be freed, leaving the xprt_list with a corrupted/invalid entry. Later,
when GlusterD would iterate over the xprt_list to send notifications, it
would crash when the invalid entry was accessed.

To fix this, DISCONNECT notification in socket_poller is sent on the
client socket, as it is done in the epoll handler.

Change-Id: I0370b7c6d7eb13de10ebf08d91a4a39dc7d64c7a
BUG: 1242570
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/11650
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

1 files changed, 8 insertions, 5 deletions

diff --git a/rpc/rpc-transport/socket/src/socket.c b/rpc/rpc-transport/socket/src/socket.c
index e26407f63aa..38b4d6949fc 100644
--- a/rpc/rpc-transport/socket/src/socket.c
+++ b/rpc/rpc-transport/socket/src/socket.c
@@ -2422,6 +2422,12 @@ socket_poller (void *ctx)
         uint32_t          gen = 0;
         char             *cname = NULL;
 
+        GF_ASSERT (this);
+        /* Set THIS early on in the life of this thread, instead of setting it
+         * conditionally
+         */
+        THIS = this->xl;
+
         priv->ot_state = OT_RUNNING;
 
         if (priv->use_ssl) {
@@ -2450,7 +2456,6 @@ socket_poller (void *ctx)
         }
 
         if (priv->connected == 0) {
-		THIS = this->xl;
                 ret = socket_connect_finish (this);
                 if (ret != 0) {
                         gf_log (this->name, GF_LOG_WARNING,
@@ -2491,8 +2496,7 @@ socket_poller (void *ctx)
 			       "poll error on pipe");
 			break;
 		}
-		/* Only glusterd actually seems to need this. */
-		THIS = this->xl;
+
 		if (pfd[1].revents & POLL_MASK_INPUT) {
 			ret = socket_event_poll_in(this);
 			if (ret >= 0) {
@@ -2570,8 +2574,7 @@ err:
         priv->sock = -1;
         priv->ot_state = OT_IDLE;
         pthread_mutex_unlock(&priv->lock);
-        rpc_transport_notify (this->listener, RPC_TRANSPORT_DISCONNECT,
-                              this);
+        rpc_transport_notify (this, RPC_TRANSPORT_DISCONNECT, this);
         rpc_transport_unref (this);
 	return NULL;
 }