core: run many bricks within one glusterfsd process

This patch adds support for multiple brick translator stacks running in
a single brick server process.  This reduces our per-brick memory usage
by approximately 3x, and our appetite for TCP ports even more.  It also
creates potential to avoid process/thread thrashing, and to improve QoS
by scheduling more carefully across the bricks, but realizing that
potential will require further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global
option.  By default it's off, and bricks are started in separate
processes as before.  If multiplexing is enabled, then *compatible*
bricks (mostly those with the same transport options) will be started in
the same process.

Backport of:
> Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/14763

Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8
BUG: 1418091
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: https://review.gluster.org/16496
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>

1 files changed, 43 insertions, 0 deletions

diff --git a/tests/basic/mpx-compat.t b/tests/basic/mpx-compat.t
new file mode 100644
index 00000000000..3de0f6fe7cb
--- /dev/null
+++ b/tests/basic/mpx-compat.t
@@ -0,0 +1,43 @@
+#!/bin/bash
+#This test tests that self-heals don't perform fsync when durability is turned
+#off
+
+. $(dirname $0)/../include.rc
+. $(dirname $0)/../traps.rc
+. $(dirname $0)/../volume.rc
+
+function count_processes {
+	# It would generally be a good idea to use "pgrep -x" to ensure an
+	# exact match, but the version of pgrep we have on NetBSD (a.k.a.
+	# the worst operating system ever) doesn't support that option.
+	# Fortunately, "glusterfsd" isn't the prefix of any other name,
+	# so this works anyway.  For now.
+	pgrep glusterfsd | wc -w
+}
+
+TEST glusterd
+TEST $CLI volume set all cluster.brick-multiplex yes
+push_trapfunc "$CLI volume set all cluster.brick-multiplex off"
+push_trapfunc "cleanup"
+
+# Create two vanilla volumes.
+TEST $CLI volume create $V0 $H0:$B0/brick-${V0}-{0,1}
+TEST $CLI volume create $V1 $H0:$B0/brick-${V1}-{0,1}
+
+# Start both.
+TEST $CLI volume start $V0
+TEST $CLI volume start $V1
+
+# There should be only one process for compatible volumes.  We can't use
+# EXPECT_WITHIN here because it could transiently see one process as two are
+# coming up, and yield a false positive.
+sleep $PROCESS_UP_TIMEOUT
+EXPECT "1" count_processes
+
+# Make the second volume incompatible with the first.
+TEST $CLI volume stop $V1
+TEST $CLI volume set $V1 server.manage-gids no
+TEST $CLI volume start $V1
+
+# There should be two processes this time (can't share protocol/server).
+EXPECT_WITHIN $PROCESS_UP_TIMEOUT "2" count_processes