core: run many bricks within one glusterfsd process

This patch adds support for multiple brick translator stacks running in
a single brick server process.  This reduces our per-brick memory usage
by approximately 3x, and our appetite for TCP ports even more.  It also
creates potential to avoid process/thread thrashing, and to improve QoS
by scheduling more carefully across the bricks, but realizing that
potential will require further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global
option.  By default it's off, and bricks are started in separate
processes as before.  If multiplexing is enabled, then *compatible*
bricks (mostly those with the same transport options) will be started in
the same process.

Backport of:
> Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/14763

Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8
BUG: 1418091
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: https://review.gluster.org/16496
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>

1 files changed, 63 insertions, 0 deletions

diff --git a/tests/basic/multiplex.t b/tests/basic/multiplex.t
new file mode 100644
index 00000000000..bff3efb0a2c
--- /dev/null
+++ b/tests/basic/multiplex.t
@@ -0,0 +1,63 @@
+#!/bin/bash
+
+. $(dirname $0)/../include.rc
+. $(dirname $0)/../traps.rc
+. $(dirname $0)/../volume.rc
+
+function count_up_bricks {
+        $CLI --xml volume status $V0 | grep '<status>1' | wc -l
+}
+
+function count_brick_pids {
+        $CLI --xml volume status $V0 | sed -n '/.*<pid>\([^<]*\).*/s//\1/p' \
+                                     | grep -v "N/A" | sort | uniq | wc -l
+}
+
+TEST glusterd
+TEST $CLI volume set all cluster.brick-multiplex yes
+push_trapfunc "$CLI volume set all cluster.brick-multiplex off"
+push_trapfunc "cleanup"
+TEST $CLI volume create $V0 $H0:$B0/brick{0,1}
+
+TEST $CLI volume start $V0
+# Without multiplexing, there would be two.
+EXPECT_WITHIN $PROCESS_UP_TIMEOUT 2 count_up_bricks
+EXPECT 1 online_brick_count
+
+TEST $CLI volume stop $V0
+EXPECT_WITHIN $PROCESS_DOWN_TIMEOUT 0 online_brick_count
+TEST $CLI volume start $V0
+EXPECT_WITHIN $PROCESS_UP_TIMEOUT 2 count_up_bricks
+EXPECT 1 online_brick_count
+
+TEST kill_brick $V0 $H0 $B0/brick1
+EXPECT_WITHIN $PROCESS_DOWN_TIMEOUT 1 count_up_bricks
+# Make sure the whole process didn't go away.
+EXPECT 1 online_brick_count
+
+TEST $CLI volume start $V0 force
+EXPECT_WITHIN $PROCESS_UP_TIMEOUT 2 count_up_bricks
+EXPECT 1 online_brick_count
+
+# Killing the first brick is a bit more of a challenge due to socket-path
+# issues.
+TEST kill_brick $V0 $H0 $B0/brick0
+EXPECT_WITHIN $PROCESS_DOWN_TIMEOUT 1 count_up_bricks
+EXPECT 1 online_brick_count
+TEST $CLI volume start $V0 force
+EXPECT_WITHIN $PROCESS_UP_TIMEOUT 2 count_up_bricks
+EXPECT 1 online_brick_count
+
+# Make sure that the two bricks show the same PID.
+EXPECT 1 count_brick_pids
+
+# Do a quick test to make sure that the bricks are acting as separate bricks
+# even though they're in the same process.
+TEST $GFS --volfile-id=$V0 --volfile-server=$H0 $M0
+for i in $(seq 10 99); do
+        echo hello > $M0/file$i
+done
+nbrick0=$(ls $B0/brick0/file?? | wc -l)
+nbrick1=$(ls $B0/brick1/file?? | wc -l)
+TEST [ $((nbrick0 + nbrick1)) -eq 90 ]
+TEST [ $((nbrick0 * nbrick1)) -ne 0 ]