glusterfs.git/extras, branch round-robin2

extras: fix failed_tests.py to work

2016-07-21T12:02:11+00:00

The text in the regression console output, now has the following prepended:
"08:27:48 "
with every line. Hence changing all the instances where the entire line
was compared, to find the pattern in the line.

Also, another usability change to take centos/netbsd as an argument
instead of the regression link.

Change-Id: I1648ab90fa6d34bb8b82a0a33401c5a2ecb86426
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14914
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

eventsapi: Gluster Eventing Feature implementation

2016-07-18T20:52:20+00:00

[Depends on http://review.gluster.org/14627]

Design is available in `glusterfs-specs`, A change from the design
is support of webhook instead of Websockets as discussed in the design

http://review.gluster.org/13115

Since Websocket support depends on REST APIs, I will add Websocket support
once REST APIs patch gets merged

Usage:
Run following command to start/stop Eventsapi server in all Peers,
which will collect the notifications from any Gluster daemon and emits
to configured client.

    gluster-eventsapi start|stop|restart|reload

Status of running services can be checked using,

    gluster-eventsapi status

Events listener is a HTTP(S) server which listens to events emited by
the Gluster. Create a HTTP Server to listen on POST and register that
URL using,

    gluster-eventsapi webhook-add  [--bearer-token ]

For example, if HTTP Server running in `http://192.168.122.188:9000`
then add that URL using,

    gluster-eventsapi webhook-add http://192.168.122.188:9000

If it expects a Token then specify it using `--bearer-token` or `-t`

We can also test Webhook if all peer nodes can send message or not
using,

    gluster-eventsapi webhook-test  [--bearer-token ]

Configurations can be viewed/updated using,

    gluster-eventsapi config-get [--name]
    gluster-eventsapi config-set  
    gluster-eventsapi config-reset 

If any one peer node was down during config-set/reset or webhook
modifications, Run sync command from good node when a peer node comes
back. Automatic update is not yet implemented.

    gluster-eventsapi sync

Basic Events Client(HTTP Server) is included with the code, Start
running the client with required port and start listening to the
events.

    /usr/share/glusterfs/scripts/eventsdash.py --port 8080

Default port is 9000, if no port is specified, once it started running
then configure gluster-eventsapi to send events to that client.

Eventsapi Client can be outside of the Cluster, it can be run event on
Windows. But only requirement is the client URL should be accessible
by all peer nodes.(Or ngrok(https://ngrok.com) like tools can be used)

Events implemented with this patch,
- Volume Create
- Volume Start
- Volume Stop
- Volume Delete
- Peer Attach
- Peer Detach

It is easy to add/support more events, since it touches Gluster cmd
code and to avoid merge conflicts I will add support for more events
once this patch merges.

BUG: 1334044
Change-Id: I316827ac9dd1443454df7deffe4f54835f7f6a08
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14248
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

extras/cliutils: Utils for creating CLI tools for Gluster

2016-07-12T16:07:29+00:00

Refer README.md for documentation.

BUG: 1342356
Change-Id: Ic88504177137136bbb4b8b2c304ecc4af9bcfe30
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14627
Reviewed-by: Prashanth Pai 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

ganesha/scripts : delete nfs-ganesha folder from shared storage during clean up

2016-06-24T00:48:38+00:00

A directory named "nfs-ganesha" will be created inside shared storage when
'gluster nfs-ganesha enable' is executed. Similarly this directory should be
removed when 'gluster nfs-ganesha disable' is executed.

Change-Id: Icc09b32010de07c9809e22aafbb2fd08a5c8252f
BUG: 1349398
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/14782
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

clang-analyzer: adding clang static analysis support

2016-06-07T13:56:09+00:00

clang static analyzer is a source code analysis tool that
finds bugs in C, C++.

This patch automates clang analyzes part by integrating
it with 'make clang-check'

clang-checker.sh: runs clang analyzer with and without the
HEAD commit, and shows the bugs introduced by HEAD commit (if any)

sample report:

$[ glusterfs ]: make clang-check

================ Clang analyzer in progress ================
...

BASELINE BUGS LIST (before applying patch):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Out of bound array access --> 3
Memory leak --> 1
Unix API --> 24
Dead increment --> 5
Dereference of null pointer --> 1995
Uninitialized argument value --> 2
All Bugs --> 2872
Called function pointer is null null dereference --> 4
Dead initialization --> 49
Dead assignment --> 691
Undefined allocation of 0 bytes CERT MEM0 C CWE --> 5
Argument with nonnull attribute passed null --> 84
Result of operation is garbage or undefined --> 9

TARGET BUGS LIST (after applying patch):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Out of bound array access --> 3
Memory leak --> 1
Unix API --> 24
Dead increment --> 5
Dereference of null pointer --> 1995
Uninitialized argument value --> 2
All Bugs --> 2875
Called function pointer is null null
dereference --> 4
Dead initialization --> 49
Dead assignment --> 694
Undefined allocation of 0 bytes CERT MEM0 C CWE --> 5
Argument with nonnull attribute passed null --> 84
Result of operation is garbage or undefined --> 9

SUMMARY OF CLANG-ANALYZER:
~~~~~~~~~~~~~~~~~~~~~~~~~~
Extra 3 Bug[s] Introduced in: Dead assignment

Patch Value given by Clang analyzer '-1'

Explore complete results at
/home/user/work/glusterfs/baseline/results/index.html
/home/user/work/glusterfs/target/results/index.html

================= Done with Clang Analysis =================

Change-Id: I512e33cfc02885a111152fbc1832144261dda1da
BUG: 1228142
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/11083
Tested-by: Prasanna Kumar Kalever 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Kaleb KEITHLEY

common-ha: race/timing issue setting up cluster

2016-06-04T04:10:02+00:00

The ganesha_grace resource agent can start before the ganesha_mon
resource agent, with the result that the crm_attribute that
ganesha_grace expects to find has not been created yet.

This is never (never? Or just so rarely that it has never actually
been seen during development) seen with four nodes, but with just
two nodes it's very repeatable.

Note that when long (FQDN) names are used it is not unexpected to
see Failed Actions in the output of `pcs status`, e.g.:

* nfs-grace_monitor_5000 on node1.fully.qualified.domain.name.com
'unknown error' (1): call=20, status=complete, exitreason='none',
last-rc-change='Wed Jun  1 12:32:32 2016', queued=0ms, exec=0ms
* nfs-grace_monitor_5000 on node2.fully.qualified.domain.name.com
'unknown error' (1): call=18, status=complete, exitreason='none',
last-rc-change='Wed Jun  1 12:32:42 2016', queued=0ms, exec=0ms

and as long as all the ganesha_grace_clone and cluster_ip-1
resource agents are in Started state then this is okay.

Change-Id: I726c9946ceb1ca92872b321612eb0f4c3cc039d8
BUG: 1341768
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14607
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

ganesha: fix the shebang for the copy-export script

2016-05-29T10:55:13+00:00

BUG: 1340488
Change-Id: I22061a8b8bc0ea43da91e5b2904a27a674a004be
Reported-by: Patrick Matthäi 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14548
Smoke: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

common-ha: wait for cluster to elect DC before accessing CIB

2016-05-24T09:35:58+00:00

access attempts, e.g. `pcs property set stonith-enabled=false`
will fail (or time out) if attempted "too early", i.e. before
the cluster has elected its DC.

Change-Id: Ifc0aa7ce652c1da339b9eb8fe17e40e8a09b1096
BUG: 1336945
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14426
CentOS-regression: Gluster Build System 
NetBSD-regression: Gluster Build System 
Tested-by: Gluster Build System 
Reviewed-by: soumya k 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: jiffin tony Thottan

common-ha: post fail-back, ganesha.nfsds are not put into NFS-GRACE

2016-05-24T08:41:18+00:00

A little known, rarely used feature of pacemaker called
"notification" is used to follow the status of the ganesha.nfsds
in the cluster. This is done with location constraints and other
Black Magick.

When a nfsd dies, the ganesha-active attribute is cleared, the
associated floating IP (VIP) fails over to another node, and the
ganesha_grace notify method is invoked with post-stop on all the
nodes where the ganesha.nfsd is still running. The notify methods
send dbus msgs to put their nfsds into NFS-GRACE, and the nfsds
perform their grace processing, e.g. taking over locks from the
failed nfsd.

N.B. Fail-back was originally not planned to be a feature for
glusterfs-3.7, but we sorta got it for free.

For fail-back, the opposite occurs. The ganesha-active attribute
is recreated, the floating IP fails back, and the notify method is
invoked with pre-start on all the nodes where the surviving
ganesha.nfsds continue to run. The notify methods send dbus msgs
again to put their nsfds into NFS-GRACE again, and the nfsds clean
up their locks.

Change-Id: I3fc64afa20ae3a928143d69aa533a8df68dd680e
BUG: 1338967
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14506
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Niels de Vos

common-ha: log flooded with Could not map name=xxxx to a UUID

2016-05-23T08:55:52+00:00

When the cluster is configured with long (FQDN) cluster members
the log is flooded with "Could not map name=$shortname to a UUID"
notices, and setting/getting the attribute is failing

Change-Id: I954d8cef7115659cc9c8b23dae75a5a247dc5db7
BUG: 1337650
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14435
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: soumya k