glusterfs.git/geo-replication/src, branch v3.7.2

build: fix compiling on older distributions

2015-06-11T14:29:03+00:00

data-tiering is disabled on RHEL-5 because it depends on a too new
SQLite version.

This change also prevents installing some of files that are used by
geo-replication, which is also not available on RHEL-5. geo-replication
depends on a too recent version of Python.

Due to an older version of OpenSSL, some of the newer functions can not
be used. A fallback to previous functions is done. Unfortunately RHEL-5
does not seem to have TLSv1.2 support, so only older versions can be
used.

Cherry picked from commit 0209b18fd65f9df5ebd0a8764ebf864d0d392998:
> Change-Id: I672264a673f5432358d2e83b17e2a34efd9fd913
> BUG: 1222317
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/10803
> Tested-by: NetBSD Build System 
> Tested-by: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Vijay Bellur 

Also including the changes from http://review.gluster.org/11140:

> build: improve detection of new OpenSSL features
>
> Building on Mac OS X revealed that the current check for
> CRYPTO_THREADID_set_callback() availability in OpenSSL is not correct.
>
> There also does not seem to be a guarantee that TLSv1_2_method() is
> available when TLS1_2_VERSION is #define'd.
>
> Change-Id: I21508065fc181a1c74bee4fd6d23bb5bdf7cea7a
> BUG: 1222317
> Reviewed-on: http://review.gluster.org/11140
> Original-author: Kaleb KEITHLEY 
> Signed-off-by: Niels de Vos 

Change-Id: I672264a673f5432358d2e83b17e2a34efd9fd913
BUG: 1228510
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11096
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Kaleb KEITHLEY

geo-rep/gverify: Ignore .trashcan while checking slave is empty.

2015-03-20T17:27:05+00:00

With trash translator, '.trashcan' is created in gluster volume
during init. So geo-rep create used to fail, with slave not
being empty. The gverify script should ignore '.trashcan' while
checking whether slave is empty.

Change-Id: Ib7ee265c6aa99b63c7ccf103747d47a6f756ad35
BUG: 1203086
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/9921
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: Remove Generated file From git

2015-03-18T09:55:29+00:00

geo-replication/src/peer_mountbroker

BUG: 1136312
Change-Id: Ib9b287b4e1183cb44acbf01184a240be7f09be7c
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/9923
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

feature/glusterfind: A tool to find incremental changes

2015-03-18T09:54:25+00:00

Documentation is available in patch:
http://review.gluster.org/#/c/9800/

A tool which helps to get list of modified files or list of all files in
GlusterFS Volume using Changelog or find command.

Usage
=====
glusterfind --help

Create:
-------
glusterfind create --help

The tool creates status file $GLUSTERD_WORKDIR/SESSION/VOLUME/status
and records current timestamp to initiate the session. This timestamp
will be used as start time for next runs.

As part of create also generates ssh key and distributes to all peers.
and enables build.pgfid and changelog using volume set command.

Pre:
----
glusterfind pre --help

This command is used to generate the list of files modified after session
creation time or after last run. To get list of all files/dirs in Volume,
run pre command with `--full` argument.

The tool gets all nodes details using gluster volume info and runs node
agent for each brick in respective nodes via ssh command. Once these node
agents generate the output file, tool copies to local using scp. Merges all
the output files to generate the final output file.

Post:
-----
glusterfind post --help

After consuming the list, this sub command is called to update the session
time based on pre command status file.

List:
-----
glusterfind list --help

To view all the sessions

Delete:
-------
glusterfind delete --help

Delete session.

Known Issues
------------
1. Deleted files will not get listed, since we can't convert GFID to
   Path if file/dir is deleted.
2. Only new name will get listed if Renamed.
3. All hardlinks will get listed.

Change-Id: I82991feb0aea85cb6ec035fddbf80a2b276e86b0
BUG: 1193893
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/9682
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-by: Prashanth Pai 
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur

geo-rep: mountbroker user management

2015-03-17T11:56:24+00:00

Non root geo-replication setup is now simplified. This
patch provides cli for mountbroker user and options management

To set Options,
    gluster system:: execute mountbroker opt  
    # for example,
    gluster system:: execute mountbroker opt mountbroker-root /var/mountbroker-root
    gluster system:: execute mountbroker opt geo-replication-log-group geogroup
    gluster system:: execute mountbroker opt rpc-auth-allow-insecure on

To remove option,
    gluster system:: execute mountbroker optdel 
    # for example,
    gluster system:: execute mountbroker optdel geo-replication-log-group

To add/edit user,
    gluster system:: execute mountbroker user  
    # for example
    gluster system:: execute mountbroker user geoaccount slavevol1,slavevol2

To remove user,
    gluster system:: execute mountbroker userdel 
    # for example
    gluster system:: execute mountbroker userdel geoaccount

For info,
    gluster system:: execute mountbroker info
    gluster system:: execute mountbroker -j info

For JSON output add -j after mountbroker, for example,
    gluster system:: execute mountbroker -j user geoaccount slavevol1,slavevol2

PS: Each peer prints its own JSON output, aggregator required from consumer side

BUG: 1136312
Change-Id: Ie52210c0bcc91ac2ffd3ba58988222ffca62b47f
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/9398
Tested-by: Gluster Build System 
Reviewed-by: darshan n 
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur

geo-rep: Avoid duplicate entries in authorized_keys file

2015-03-06T02:27:40+00:00

When Georep create(force) command is run multiple times
it appends master nodes public keys to Slave nodes without
checking the existence of the key.

With this patch, create push-pem force adds pub key only if
not available in authorized_keys.

BUG: 1197433
Change-Id: Iad57f6c45698e258ad1a547fa7a2e376a315f0cd
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/9776
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Saravanakumar Arumugam 
Reviewed-by: Vijay Bellur

geo-rep: Add support for non standard AuthorizedKeysFile location

2015-02-20T02:56:28+00:00

In /etc/ssh/sshd_config, AuthorizedKeysFile can be customized
using %u and %h variables, %u will be replaced by user name
and %h will be replaced by home dir name. Default location is
.ssh/authorized_keys

For example,
AuthorizedKeysFile .ssh/authorized_keys
AuthorizedKeysFile %h/.my_secret_dir/authorized_keys
AuthorizedKeysFile /etc/ssh/keys/%u/authorized_keys

PS: Support only added for %h and %u in sshd_config

BUG: 1181117
Signed-off-by: Aravinda VK 
Change-Id: Ic6ba20f9d202762dfdb6d0c73ea42e7f7c64e177
Reviewed-on: http://review.gluster.org/9436
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: Fixing the typo errors

2015-02-03T18:03:41+00:00

Change-Id: Iacc67e4ba9ac45e0858f3befe84ffb8fccf7e1c3
BUG: 1075417
Signed-off-by: arao 
Reviewed-on: http://review.gluster.org/9502
Tested-by: Gluster Build System 
Reviewed-by: Humble Devassy Chirammal

geo-rep: Handle copying of common_secret.pem.pub to slave correctly.

2015-01-22T06:58:41+00:00

Current Behaviour:
    1. Geo-replication gsec_create creates common_secret.pem.pub file
       containing public keys of the all the nodes of master cluster
       in the location /var/lib/glusterd/
    2. Geo-replication create push-pem copies the common_secret.pem.pub
       to the same location on all the slave nodes with same name.

Problem:
    Wrong public keys might get copied on to slave nodes in multiple
    geo-replication sessions simultaneosly.

     E.g.
      A geo-rep session is established between Node1(vol1:Master) to
      Node2 (vol2:Slave). And one more geo-rep session where
      Node2 (vol3) becomes master to Node3 (vol4) as below.

      Session1: Node1 (vol1) ---> Node2 (vol2)
      Session2: Node2 (vol3) ---> Node3 (vol4)

      If steps followed to create both geo-replication session is as
      follows, wrong public keys are copied on to Node3 from Node2.

      1. gsec_create is done on Node1 (vol1) -Session1
      2. gsec_create is done on Node2 (vol3) -Session2
      3. create push-pem is done Node1 - Session1.
          -This overwrites common_secret.pem.pub in Node2
           created by gsec_create in second step.
      4. create push-pem on Node2 (vol3) copies overwrited
         common_secret.pem.pub keys to Node3. -Session2

Consequence:
      Session2 fails to start with Permission denied because of wrong
      public keys

Solution:
      On geo-rep create push-pem, don't copy common_secret.pem.pub
      file with same name on to all slave nodes. Prefix master and
      slave volume names to the filename.

NOTE: This brings change in manual steps to be followed to setup
      non-root geo-replication (mountbroker). To copy ssh public
      keys, extra two arguments needs to be followed.

      set_geo_rep_pem_keys.sh   \
      
      
      Path to set_geo_rep_pem_keys.sh:
      Source Installation: 
              /usr/local/libexec/glusterfs/set_geo_rep_pem_keys.sh
      Rpm Installatino: 
              /usr/libexec/glusterfs/set_geo_rep_pem_keys.sh

Change-Id: If38cd4e6f58d674d5fe2d93da15803c73b660c33
BUG: 1183229
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/9460
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

Regression test portability: mktemp

2014-08-20T20:46:44+00:00

Linux mktemp accepts to run without a template, NetBSD mandates it. Since
the template option has the same syntax, add it everywhere. While there,
also do this in scripts outside of regression testing.

BUG: 764655
Change-Id: I3ec140afbc9009257c81a56d77afcc21fef74cc4
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8432
Tested-by: Gluster Build System 
Reviewed-by: Harshavardhana 
Tested-by: Harshavardhana