glusterfs.git/rpc/rpc-transport/socket/src, branch v3.10.10

rpc: TLSv1_2_method() is deprecated in OpenSSL-1.1

2017-09-17T12:55:40+00:00

Fedora 26 has OpenSSL-1.1. Compile-time warnings indicate
that TLSv1_2_method() is now deprecated. As per the SSL man page:

  TLS_method(), TLS_server_method(), TLS_client_method()
    These are the general-purpose version-flexible SSL/TLS methods.
    The actual protocol version used will be negotiated to the highest
    version mutually supported by the client and the server. The
    supported protocols are SSLv3, TLSv1, TLSv1.1 and TLSv1.2.
    Applications should use these methods, and avoid the version-
    specific methods described below.
  ...
  TLSv1_2_method(), ...
  ...

Note that OpenSSL-1.1 is the version of OpenSSL; Fedora 25 and RHEL 7.3
and other distributions (still) have OpenSSL-1.0.

TLS versions are orthogonal to the OpenSSL version.  TLS_method() is the
new — in OpenSSL-1.1 — version flexible function intended to replace the
TLSv1_2_method() function in OpenSSL-1.0 and the older (?), insecure
TLSv23_method(). (OpenSSL-1.0 does not have TLS_method())

master: https://review.gluster.org/18268
master BZ: 1491025
release-3.12: https://review.gluster.org/18284
release-3.12 BZ: 1491690

Change-Id: I190363ccffe7c25606ea2cf30a6b9ff1ec186057
BUG: 1491691
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/18285
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

refcount: typecast function for calling on free

2017-08-11T11:22:43+00:00

All of the functions called to free the refcounted structure are doing a
typecast from (void*) to their own type taht is being free'd. This
really is not needed and the refcount interface is made a little simpler
without the requirement of typecasting.

With this small improvement in the API, all callers are updated too.

Cherry picked from commit f2ca301bd741e3e3f076cd3f72fcd377bcef2a1a:
> Change-Id: I32473b6d1799f62861d4b2d78ea30c09e6c80ab1
> BUG: 1416889
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/16471
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Xavier Hernandez 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 

Backport note: This patch makes it easier to backport changes that use
               gf_refcount_t. There is no functional change.

Change-Id: I32473b6d1799f62861d4b2d78ea30c09e6c80ab1
BUG: 1471870
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17913
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

rpc: add options to manage socket keepalive lifespan

2017-06-20T04:58:25+00:00

Problem:
Default values for handling socket timeouts for brick responses are
insufficient for aggressive applications such as databases.

Solution:
Add 1:1 gluster options for keepalive, keepalive-idle,
keepalive-interval and keepalive-timeout as per the socket level options
available as per tcp(7) man page.

Default values for options are NOT agressive and continue to be values
which result in default timeout when only the keep alive option is
turned on.

These options are Linux specific and will not be applicable to the
*BSDs.

mainline:
> BUG: 1426059
> Signed-off-by: Milind Changire 
> Reviewed-on: https://review.gluster.org/16731
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit 6b8df081b46ac4f485c86a5052fc30472e74bfbb)

Change-Id: I2a08ecd949ca8ceb3e090d336ad634341e2dbf14
BUG: 1452038
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/17330
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

rpc: log more about socket disconnects

2017-05-31T05:57:36+00:00

Log more about the different paths leading to socket disconnect for 
ease of debugging.

Log via gf_log_callingfn() in __socket_disconnect() at loglevel
TRACE if socket connection is being torn down.

mainline:
> BUG: 1426125
> Signed-off-by: Milind Changire 
> Reviewed-on: https://review.gluster.org/16732
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
(cherry picked from commit 67a35ac54bfd61a920c1919fbde588a04ac3358a)

Change-Id: I1e551c2d685784b5ec747f481179f64d524c0461
BUG: 1451977
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/17321
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

rpc: avoid logging success on failure

2017-05-31T05:57:16+00:00

Avoid logging Success in the event of failure especially when errno
has no meaningful value w.r.t. the failure. In this case the errno 
is set to zero when there's indeed a failure at the RPC level.

mainline:
> BUG: 1426032
> Signed-off-by: Milind Changire 
> Reviewed-on: https://review.gluster.org/16730
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: N Balachandran 
> Reviewed-by: Jeff Darcy 
(cherry picked from commit 89c6bedc1c2e978f67ca29f212a357984cd8a2dd)

Change-Id: If2cc81aa1e590023ed22892dacbef7cac213e591
BUG: 1451995
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/17326
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

rpc: fix transport add/remove race on port probing

2017-05-11T05:51:08+00:00

Problem:
Spurious __gf_free() assertion failures seen all over the place with
header->magic being overwritten when running port probing tests with
'nmap'

Solution:
Fix sequence of:
1. add accept()ed socket connection fd to epoll set
2. add newly created rpc_transport_t object in RPCSVC service list

Correct sequence is #2 followed by #1.

Reason:
Adding new fd returned by accept() to epoll set causes an epoll_wait()
to return immediately with a POLLIN event. This races ahead to a readv()
which returms with errno:104 (Connection reset by peer) during port
probing using 'nmap'. The error is then handled by POLLERR code to
remove the new transport object from RPCSVC service list and later
unref and destroy the rpc transport object.
socket_server_event_handler() then catches up with registering the
unref'd/destroyed rpc transport object. This is later manifest as
assertion failures in __gf_free() with the header->magic field botched
due to invalid address references.
All this does not result in a Segmentation Fault since the address
space continues to be mapped into the process and pages still being
referenced elsewhere.

As a further note:
This race happens only in accept() codepath. Only in this codepath,
the notify will be referring to two transports:
1, listener transport and
2. newly accepted transport
All other notify refer to only one transport i.e., the transport/socket
on which the event is received. Since epoll is ONE_SHOT another event
won't arrive on the same socket till the current event is processed.
However, in the accept() codepath, the current event - ACCEPT - and the
new event - POLLIN/POLLER - arrive on two different sockets:
1. ACCEPT on listener socket and
2. POLLIN/POLLERR on newly registered socket.
Also, note that these two events are handled different thread contexts.

Cleanup:
Critical section in socket_server_event_handler() has been removed.
Instead, an additional ref on new_trans has been used to avoid ref/unref
race when notifying RPCSVC.

mainline:
> BUG: 1438966
> Signed-off-by: Milind Changire 
> Reviewed-on: https://review.gluster.org/17139
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Oleksandr Natalenko 
> Reviewed-by: Jeff Darcy 
(cherry picked from commit 4f7ef3020edcc75cdeb22d8da8a1484f9db77ac9)

Change-Id: I4417924bc9e6277d24bd1a1c5bcb7445bcb226a3
BUG: 1449169
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/17217
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

socket: GF_REF_PUT should be called outside lock

2017-02-07T11:49:43+00:00

GF_REF_PUT was called inside lock which can call
socket_poller_mayday which inturn tries to take the
same lock. This can lead to deadlock scenario.

>Reviewed-on: https://review.gluster.org/16343
>Reviewed-by: Raghavendra G 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 

BUG: 1419503
Change-Id: Ib3b161bcfeac810bd3593dc04c10ef984f996b17
Signed-off-by: Rajesh Joseph 
Reviewed-on: https://review.gluster.org/16548
Tested-by: Atin Mukherjee 
Reviewed-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

socket: retry connect immediately if it fails

2017-02-03T00:43:24+00:00

Previously we relied on a complex dance of setting flags, shutting
down the socket, tearing stuff down, getting an event, tearing more
stuff down, and waiting for a higher-level retry.  What we really
need, in the case where we're just trying to connect prematurely e.g.
to a brick that hasn't fully come up yet, is a simple retry of the
connect(2) call.

This was discovered by observing failures in ec-new-entry.t with
multiplexing enabled, but probably fixes other random failures as
well.

Backport of:
> Change-Id: Ibedb8942060bccc96b02272a333c3002c9b77d4c
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/16510

BUG: 1418091
Change-Id: I4bac26929a12cabcee4f9e557c8b4d520948378b
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16533
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

rpc/socket.c : Bonnie++ hangs during rewrites in ganesha + SSL

2017-02-02T17:33:45+00:00

Problem: Bonnie++ rewrite operation hangs in ganesha + SSL environment

Solution: Bonnie++ hangs during execution of rewrite operation in
          ganesha + SSL environment.It was hanged due to blocking on poll
          call in ssl_do because no POLLOUT event was getting on socket.
          Socket is not getting POLLOUT event because all other threads
          are waiting to get lock and lock is not released ssl_do
          because it is not getting any event on poll.To correct it
          update the condition in ssl_do as same in getting error
          SSL_ERROR_WANT_READ.

Test:     To test the patch followed below procedure
          1) Setup 2X2 Ganesha + SSL environment.
          2) Run bonnie from 3 nfs client parallely
          3) After run "Rewwrite operation" by bonnie it is hanged.
          4) After apply the patch it is not hanged.

> BUG: 1418213
> Change-Id: I5985cbbc4cfdac5d287268d791e31c274abc3c8d
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: https://review.gluster.org/16501
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Vijay Bellur 
> (cherry picked from commit d7077bca4b372a056d23416294e729637e9af94e)

Change-Id: Id029c71382025477bb5ff31f28ec537e4fe58b03
BUG: 1418541
Reviewed-on: https://review.gluster.org/16513
Tested-by: MOHIT AGRAWAL 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

libglusterfs+transport+io-threads: fix 256KB stack abuse

2017-02-02T17:32:49+00:00

Some functions were allocating 64K booleans, which are (crazily)
mapped to 4-byte ints, for a total of 256KB per call.  Changed to use
bitfields instead, so usage is now only 8KB per call.  This was the
impediment to changing the io-threads stack size, so that has been
adjusted too.

Backport of:
> Change-Id: I8781c4f2c8f2b830f4535e366995fac8dd0a8653
> BUG: 1418095
> Reviewed-on: https://review.gluster.org/15745

Change-Id: Ia5dada61703e6bea95f2511da71feb573fc9a429
BUG: 1418536
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16511
Reviewed-by: N Balachandran 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur 
CentOS-regression: Gluster Build System