glusterfs.git/extras/Makefile.am, branch v4.1.2

build: add --without-server option

2018-02-19T17:38:55+00:00

With Gluster 4.0 we will not provide the server components for EL6 and
older. At one point Gluster 4.x will get GlusterD2, which requires
Golang tools in the distribution. EL6 does not contain these at the
moment.

With this change, it is possible to `./configure --without-server` which
prevents building glusterd and the xlators for the bricks. Building RPMs
can pass `--without server` and the glusterfs-server sub-package will
not be created.

Change-Id: I97f5ccf9f2c76e60d9af83915fc59fae57ad6d25
BUG: 1074947
Signed-off-by: Niels de Vos

extras: scripts to control CPU/MEMORY for any gluster daemon during runtime

2017-11-13T11:07:06+00:00

Problem: Sometime gluster daemons like glustershd can consume a lot of cpu and/
or memory if there is a large amount of data/ entries to be healed.

Solution: Until we have some form of throttling/ QoS mechanisms built into
gluster, we can use control groups for regulating cpu and memory of any gluster
daemon using control-cpu-load.sh and control-mem.sh scripts respectively.

Test:    To test the control-cpu-load.sh script follow below procedure:
         1) Setup distribute replica environment
         2) Selfheal daemon off
         3) Down one node from replica nodes
         4) Create millions of files from mount point
         5) Start down node
         6) Check cpu usage for shd process in top command
         7) Run script after provide shd pid with CPU quota value
         8) Check again cpu usage for shd process in top command

Note: control-mem.sh script can cap the memory usage of the process to the set
limit, beyond which the process gets blocked. It resumes either when the memory
usage comes down or if the limit is increased.

BUG: 1496335
Change-Id: Id73c36b73ca600fa9f7905d84053d1e8633c996f
Signed-off-by: Mohit Agrawal

scripts/shared_storage : systemd helper scripts to mount shared storage post reboot

2017-06-20T12:42:01+00:00

Reported-by: Hendrik Visage 
Change-Id: Ibcff56b00f45c8af54c1ae04974267c2180f5f63
BUG: 1452527
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: https://review.gluster.org/17339
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Kaleb KEITHLEY

nl-cache: add group volume set option for ease of use

2017-06-12T03:52:16+00:00

Change-Id: Id03643a9598da53051a01ca09e1d2a62bc195ab6
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17495
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

extras: Provide group set for gluster-block workloads

2017-05-12T13:21:28+00:00

For gluster-block workloads I/O is always with o-direct so it doesn't
benefit by any of the perf xlators so disabling all of them to save
on memory.
performance.quick-read=off
performance.read-ahead=off
performance.io-cache=off
performance.stat-prefetch=off
performance.write-behind=off
performance.open-behind=off
performance.readdir-ahead=off

We want the I/O on the file to be with o-direct
network.remote-dio=enable

Options that are proven to give good performance with
VM workloads which is very similar to gluster-block
cluster.eager-lock=enable
cluster.quorum-type=auto
cluster.data-self-heal-algorithm=full
cluster.locking-scheme=granular
cluster.shd-max-threads=8
cluster.shd-wait-qlength=10000
features.shard=on

It is better to turn off things we are not using
user.cifs=off

It is better to have allow-insecure to be on so that
ports that are > 1024 in tcmu-runner are allowed.
server.allow-insecure=on

Change-Id: I9a21c824fa42242f02b57569feedd03d9b6f9439
BUG: 1450010
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17254
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

packaging: (ganesha) remove glusterfs-ganesha subpackage and related files)

2017-03-21T17:14:30+00:00

Indiana Jones and the Temple of Ganesha HA, part two.

remove glsuterfs-ganesha subpackage, superceded by storhaug

Change-Id: I42a1fc59159add108d77080b9b130696216aa76d
BUG: 1418417
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16506
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-13T14:18:07+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430841
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16880
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Joe Julian 
Reviewed-by: Shyamsundar Ranganathan

extras: Provide group set for md-cache and invalidation options

2017-02-03T09:33:17+00:00

To enable the integration of md-cache and invalidation features
we need to perform 3 volume set options in a specific order.
In order to ease this for user provide a group volume set option.

Usage: gluster vol set  group metadata-cache

Change-Id: I9bf0fd4217aa2a1c7ffbdc93e879b10f87addeac
BUG: 1418249
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16503
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Atin Mukherjee

extras/cliutils: Utils for creating CLI tools for Gluster

2016-07-12T16:07:29+00:00

Refer README.md for documentation.

BUG: 1342356
Change-Id: Ic88504177137136bbb4b8b2c304ecc4af9bcfe30
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14627
Reviewed-by: Prashanth Pai 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

clang-analyzer: adding clang static analysis support

2016-06-07T13:56:09+00:00

clang static analyzer is a source code analysis tool that
finds bugs in C, C++.

This patch automates clang analyzes part by integrating
it with 'make clang-check'

clang-checker.sh: runs clang analyzer with and without the
HEAD commit, and shows the bugs introduced by HEAD commit (if any)

sample report:

$[ glusterfs ]: make clang-check

================ Clang analyzer in progress ================
...

BASELINE BUGS LIST (before applying patch):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Out of bound array access --> 3
Memory leak --> 1
Unix API --> 24
Dead increment --> 5
Dereference of null pointer --> 1995
Uninitialized argument value --> 2
All Bugs --> 2872
Called function pointer is null null dereference --> 4
Dead initialization --> 49
Dead assignment --> 691
Undefined allocation of 0 bytes CERT MEM0 C CWE --> 5
Argument with nonnull attribute passed null --> 84
Result of operation is garbage or undefined --> 9

TARGET BUGS LIST (after applying patch):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Out of bound array access --> 3
Memory leak --> 1
Unix API --> 24
Dead increment --> 5
Dereference of null pointer --> 1995
Uninitialized argument value --> 2
All Bugs --> 2875
Called function pointer is null null
dereference --> 4
Dead initialization --> 49
Dead assignment --> 694
Undefined allocation of 0 bytes CERT MEM0 C CWE --> 5
Argument with nonnull attribute passed null --> 84
Result of operation is garbage or undefined --> 9

SUMMARY OF CLANG-ANALYZER:
~~~~~~~~~~~~~~~~~~~~~~~~~~
Extra 3 Bug[s] Introduced in: Dead assignment

Patch Value given by Clang analyzer '-1'

Explore complete results at
/home/user/work/glusterfs/baseline/results/index.html
/home/user/work/glusterfs/target/results/index.html

================= Done with Clang Analysis =================

Change-Id: I512e33cfc02885a111152fbc1832144261dda1da
BUG: 1228142
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/11083
Tested-by: Prasanna Kumar Kalever 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Kaleb KEITHLEY