glusterfs.git/geo-replication/src, branch v4.1dev

geo-rep: Improve geo-rep pre-validation logs

2018-01-22T06:33:48+00:00

Geo-rep runs gverify.sh which does pre-validation.
As part of it, master and slave volume is mounted
to verify the size. If for some reason, the mount
fails, the error message does not point out the
mount log file location. Also both master and
slave mount logs are same.

Patch does following improvements.

1. Master and slave mount logs are separated and
   error message points the log file to be looked for.
2. The log location is changed to /var/log/glusterfs/geo-replication
   instead of /var/log/glusterfs/geo-replication-slaves
3. The log file name is changed to "gverify-mastermnt.log" and
   "gverify-slavemnt.log" for master and slave mount respectively

Fixes: #395
Change-Id: Ia644ec0afebbdaae92e01adf03c635e5f8866a02
Signed-off-by: Kotresh HR

geo-rep: Validate availability of gluster binary on slave

2018-01-19T03:53:24+00:00

1. Adds validation to check if gluster binary is available on slave
2. Add a simple geo-rep setup test case to verify whether setup is fine.
   It's named in such a way that it runs first.

BUG: 1532591
Change-Id: Ie777e55ae13db8fa97d4e32464ad82269ee5fd07
Signed-off-by: Kotresh HR

tests: Enable geo-rep test cases

2018-01-05T07:08:10+00:00

This patch re-enables the geo-rep test cases.
Along with it does following optimizations.

1. Use EXPECT_WITHIN instead of sleep
2. Clean up geo-rep ssh key after test
3. Changes to gverify.sh and S56glusterd-geo-rep-create-post.sh
   to use the given ssh identity file for geo-rep create
4. Make gluster-command-dir configurable and introduce
   slave-gluster-command-dir which points the parent directory
   of gluster binaries in master and slave respectively.

Change-Id: Ia7696278d9dd3ba04224dcd7c3564088ca970b04
BUG: 1480491
Signed-off-by: Kotresh HR

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-13T14:18:07+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430841
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16880
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Joe Julian 
Reviewed-by: Shyamsundar Ranganathan

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

geo-rep: Alternate command to generate SSH Keys

2016-08-31T10:09:12+00:00

`gluster system:: execute gsec_create` is used to generate SSH
Keys in all Master nodes and collect public keys in command initiated
node. But this tool will not provide details if a peer node is down and
unable to generate keys.

New command will be introduced to create SSH Keys in all peer nodes.

Usage:

    gluster-georep-sshkey generate
    or
    gluster-georep-sshkey generate --no-prefix

Generates two SSH keys(one for gsyncd access and other for tar) in all
peer nodes and collects the public keys to the local node where it is
initiated. Adds `command=` prefix to common_secret.pem.pub if `--no-prefix`
argument is not set.

Shows status as below,

+-----------+-------------+---------------+
|    NODE   | NODE STATUS | KEYGEN STATUS |
+-----------+-------------+---------------+
|    fvm2   |          UP |            OK |
| localhost |          UP |            OK |
+-----------+-------------+---------------+

BUG: 1356508
Change-Id: Ib202811f41f9986694f07d9eedba31db6ed4d18f
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14732
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Simplify Non root user(mountbroker) Setup

2016-08-31T05:22:55+00:00

Setting up Non root Geo-replication is troublesome. Lot of steps
involved like user setup, directory creation, setting up proper
permissions, editing glusterd.vol file etc.

Introducing `gluster-mountbroker` command, with this tool non root
user setup steps are(Run the following commands in any one Slave
node),

    gluster-mountbroker setup  

For example,

    gluster-mountbroker setup /var/mountbroker-root geogroup

Add user using,

    gluster-mountbroker add  

For example,

    gluster-mountbroker add slavevol geoaccount

Remove user or Volume using,

    gluster-mountbroker remove [--volume ] [--user ]

Example,

    gluster-mountbroker remove --volume slavevol --user geoaccount
    gluster-mountbroker remove --user geoaccount
    gluster-mountbroker remove --volume slavevol

Check the status of setup using,

    gluster-mountbroker status

Once the changes are complete, restart glusterd in all Slave nodes.
and follow the Geo-rep instructions.

Change-Id: Ied3fa009df6a886b24ddd86d39283fcfcff68c82
BUG: 1343333
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14544
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: address potential leak of memory

2016-07-19T11:25:35+00:00

fix memory leak reported by clang analyzer:
gsyncd.c:411:17: warning: Potential leak of 
                 memory pointed to by 'argv'
                 in main()


Change-Id: I179fb1c095cf4d331b5910df6388dc724cb4ac96
BUG: 1332419
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14151
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR

core: use readdir(3) with glibc, and associated cleanup

2016-07-18T11:59:42+00:00

Starting with glibc-2.23 (i.e. what's in Fedora 25), readdir_r(3)
is marked as deprecated. Specifically the function decl in 
has the deprecated attribute, and now warnings are thrown during the
compile on Fedora 25 builds.

The readdir(_r)(3) man page (on Fedora 25 at least) and World+Dog say
that glibc's readdir(3) is, and always has been, MT-SAFE as long as
only one thread is accessing the directory object returned by opendir().
World+Dog also says there is a potential buffer overflow in readdir_r().
World+Dog suggests that it is preferable to simply use readdir(). There's
an implication that eventually readdir_r(3) will be removed from glibc.
POSIX has, apparently deprecated it in the standard, or even removed it
entirely.

Over and above that, our source near the various uses of readdir(_r)(3)
has a few unsafe uses of strcpy()+strcat().

(AFAIK nobody has looked at the readdir(3) implemenation in *BSD to see
if the same is true on those platforms, and we can't be sure of MacOS
even though we know it's based on *BSD.)

Change-Id: I5481f18ba1eebe7ee177895eecc9a80a71b60568
BUG: 1356998
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/14838
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Jeff Darcy

core: assorted typos and spelling mistakes reported by Debian lintian

2016-05-18T09:21:42+00:00

Also missing bang (!) in #!/bin/bash in shell scripts.

Change-Id: I567a4be8f0f31f6285550f243fe802895f6bc43b
BUG: 1336793
Reported-by: Patrick Matthäi 
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14398
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR 
CentOS-regression: Gluster Build System