summaryrefslogtreecommitdiffstats
path: root/geo-replication/src
Commit message (Collapse)AuthorAgeFilesLines
* geo-rep: Avoid duplicate entries in authorized_keys fileAravinda VK2015-03-051-2/+9
| | | | | | | | | | | | | | | | | | When Georep create(force) command is run multiple times it appends master nodes public keys to Slave nodes without checking the existence of the key. With this patch, create push-pem force adds pub key only if not available in authorized_keys. BUG: 1197433 Change-Id: Iad57f6c45698e258ad1a547fa7a2e376a315f0cd Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9776 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Add support for non standard AuthorizedKeysFile locationAravinda VK2015-02-191-9/+32
| | | | | | | | | | | | | | | | | | | | | | In /etc/ssh/sshd_config, AuthorizedKeysFile can be customized using %u and %h variables, %u will be replaced by user name and %h will be replaced by home dir name. Default location is .ssh/authorized_keys For example, AuthorizedKeysFile .ssh/authorized_keys AuthorizedKeysFile %h/.my_secret_dir/authorized_keys AuthorizedKeysFile /etc/ssh/keys/%u/authorized_keys PS: Support only added for %h and %u in sshd_config BUG: 1181117 Signed-off-by: Aravinda VK <avishwan@redhat.com> Change-Id: Ic6ba20f9d202762dfdb6d0c73ea42e7f7c64e177 Reviewed-on: http://review.gluster.org/9436 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Fixing the typo errorsarao2015-02-031-1/+1
| | | | | | | | | Change-Id: Iacc67e4ba9ac45e0858f3befe84ffb8fccf7e1c3 BUG: 1075417 Signed-off-by: arao <arao@redhat.com> Reviewed-on: http://review.gluster.org/9502 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
* geo-rep: Handle copying of common_secret.pem.pub to slave correctly.Kotresh HR2015-01-212-12/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current Behaviour: 1. Geo-replication gsec_create creates common_secret.pem.pub file containing public keys of the all the nodes of master cluster in the location /var/lib/glusterd/ 2. Geo-replication create push-pem copies the common_secret.pem.pub to the same location on all the slave nodes with same name. Problem: Wrong public keys might get copied on to slave nodes in multiple geo-replication sessions simultaneosly. E.g. A geo-rep session is established between Node1(vol1:Master) to Node2 (vol2:Slave). And one more geo-rep session where Node2 (vol3) becomes master to Node3 (vol4) as below. Session1: Node1 (vol1) ---> Node2 (vol2) Session2: Node2 (vol3) ---> Node3 (vol4) If steps followed to create both geo-replication session is as follows, wrong public keys are copied on to Node3 from Node2. 1. gsec_create is done on Node1 (vol1) -Session1 2. gsec_create is done on Node2 (vol3) -Session2 3. create push-pem is done Node1 - Session1. -This overwrites common_secret.pem.pub in Node2 created by gsec_create in second step. 4. create push-pem on Node2 (vol3) copies overwrited common_secret.pem.pub keys to Node3. -Session2 Consequence: Session2 fails to start with Permission denied because of wrong public keys Solution: On geo-rep create push-pem, don't copy common_secret.pem.pub file with same name on to all slave nodes. Prefix master and slave volume names to the filename. NOTE: This brings change in manual steps to be followed to setup non-root geo-replication (mountbroker). To copy ssh public keys, extra two arguments needs to be followed. set_geo_rep_pem_keys.sh <mountbroker_user> <master vol name> \ <slave vol name> Path to set_geo_rep_pem_keys.sh: Source Installation: /usr/local/libexec/glusterfs/set_geo_rep_pem_keys.sh Rpm Installatino: /usr/libexec/glusterfs/set_geo_rep_pem_keys.sh Change-Id: If38cd4e6f58d674d5fe2d93da15803c73b660c33 BUG: 1183229 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/9460 Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* Regression test portability: mktempEmmanuel Dreyfus2014-08-201-2/+2
| | | | | | | | | | | | | | Linux mktemp accepts to run without a template, NetBSD mandates it. Since the template option has the same syntax, add it everywhere. While there, also do this in scripts outside of regression testing. BUG: 764655 Change-Id: I3ec140afbc9009257c81a56d77afcc21fef74cc4 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8432 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Harshavardhana <harsha@harshavardhana.net> Tested-by: Harshavardhana <harsha@harshavardhana.net>
* build: make GLUSTERD_WORKDIR rely on localstatedirHarshavardhana2014-08-074-14/+12
| | | | | | | | | | | | | | | | | | | | | | - Break-way from '/var/lib/glusterd' hard-coded previously, instead rely on 'configure' value from 'localstatedir' - Provide 's/lib/db' as default working directory for gluster management daemon for BSD and Darwin based installations - loff_t is really off_t on Darwin - fix-off the warnings generated by clang on FreeBSD/Darwin - Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all platforms. - Define proper environment for running tests, define correct PATH and LD_LIBRARY_PATH when running tests, so that the desired version of glusterfs is used, regardless where it is installed. (Thanks to manu@netbsd.org for this additional work) Change-Id: I2339a0d9275de5939ccad3e52b535598064a35e7 BUG: 1111774 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8246 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* geo-rep/gverify: Never use ping to check for host reachabilityHarshavardhana2014-06-121-1/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On many linux distributions with iptables enabled, ICMP traffic is usually dropped even when port 22 is open for SSH service So practically `ping` is an unreliable command ~~~ root@rhs1:/var/log/glusterfs # gluster volume geo-replication geo-test \ 17.16.10.1::geo-test-slave create push-pem force 172.16.10.1 not reachable. geo-replication command failed ~~~ ~~~ root@rhs1:/var/log/glusterfs # ping 172.16.10.1 PING rhs2.sjc.redhat.com (172.16.10.1) 56(84) bytes of data. From rhs2.sjc.redhat.com (172.16.10.1) icmp_seq=1 Destination Host Prohibited ... ... ~~~ ~~~ root@rhs2:/var/log/glusterfs # service iptables status | grep 22 4 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 root@rhs2:/var/log/glusterfs # service iptables status | grep icmp-host-prohibited 25 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited ~~~ Change-Id: I33206ca071aa5d755c0762f7c486da222ec3c7db BUG: 1105337 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/7997 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* glusterd/geo-rep: Not setting the user name as the user group by defaultAvra Sengupta2014-06-052-4/+3
| | | | | | | | | | | | | Also removing multiple mounts in gverify.sh Change-Id: I1567a9f711222c5f571a12f7c4ab49cef32d60c9 BUG: 1101948 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/7911 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* gsyncd : Use --remote-host option during cli invocationVenky Shankar2014-05-251-0/+1
| | | | | | | | | | | | Required for unprivileged geo-replication sessions to execute glusterd commands (which are however restricted). Change-Id: Ib83b81defa061717f4465ffa665450d0f5d3d20d BUG: 1077452 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/7833 Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* glusterd/geo-rep: Creating .ssh dir with right ownershipAvra Sengupta2014-05-252-6/+19
| | | | | | | | | | | | | | Also adding -P option to the usage of df for portability in gverify.sh Change-Id: I0be19d26ea63769a934c6ccbfc04ef80768ebc9a BUG: 1099041 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/7812 Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* glusterd/geo-rep: Use getent passwd instead of $HOMEAvra Sengupta2014-05-202-3/+3
| | | | | | | | | | | | | | | | $HOME might not be set in the env variables, as is the case when these scripts are executed using the runner framework. Hence using getent passwd instead of $HOME Change-Id: I99f6bcd788d727be534b3040600d66c8dbb7ee92 BUG: 1099041 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/7803 Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* glusterd/geo-rep: Allow gverify.sh and S56glusterd-geo-rep-create-post.shAvra Sengupta2014-05-144-39/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | to operate for non-root privileged slave volume Mounting the slave-volume on local node, to perform disk checks in order to allow gverify.sh to operate for non-root privileged slave volume Allowing the hook script S56glusterd-geo-rep-create-post.sh to operate for non-root privileged slave volume Modified peer_add_secret_pub.in to accept username as argument and add the pem keys to the users's_home_dir/.ssh/authorized_keys Wrote set_geo_rep_pem_keys.sh which accepts username as argument and copies the pem keys from the user's home directory to $GLUSTERD_WORKING_DIR/geo-replication/ and then copies the keys to other nodes in the cluster and add them to the respective authorized keys. The script takes as argument the user name and assumes that the user will be present in all the nodes in the cluster. It is not needed for root. To summarize: For a privileged slave user, execute the following on master node as super user: gluster system:: execute gsec_create gluster volume geo-replication <master_vol> [root@]<slave_ip>::<slave_vol> create push_pem For a non-privileged slave user execute the following on master node as super user: gluster system:: execute gsec_create gluster volume geo-replication <master_vol> <slave_user>@<slave_ip>::<slave_vol> create push_pem then on the slave node execute the following as super user: /usr/local/libexec/glusterfs/set_geo_rep_pem_keys.sh <slave_user> BUG: 1077452 Change-Id: I88020968aa5b13a2c2ab86b1d6661b60071f6f5e Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/7744 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* gsyncd / geo-rep: Partial support for Non-root geo-replication.Venky Shankar2014-05-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | This patch enables geo-replication to be run as an unprivileged user. As of now, this is just the partial support, but is very close to achieve full functionality. Current limitation * Geo-replication executed Gluster CLI commands on the slave via SSH. On a non-root setup, Gluster CLI would run as an unprivileged user, failing to execute the command. As a workaround (for testing), setuid(2) Gluster CLI executable or use the glusterd option to accept commands by unprivileged CLI process. The nature of cli commands are "system::" commands (for key management) and remote volume info fetching. Remote volume info fetching has been modified to use --remote-host gluster cli option rather than ssh and remote cli execution. Change-Id: Ica89e2ba9b7f48fd6e1c876c477d7822dc693617 BUG: 1077452 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/7658 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* gsyncd / geo-rep: Initialize default memory accountingVenky Shankar2014-05-061-4/+11
| | | | | | | | | | | | | | | | commit 7fba3a8 enables memory accounting by default. Since geo-replication binary (gsyncd) does not declare a memory accounting function, call like GF_{CALLOC,MALLOC} result int asserts. This patch initializes default memory accounting. Change-Id: I9ad1de8bca6745a9899a006a863a7cfeef73d52f BUG: 1094708 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/7683 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* gverify: Correcting variable namesAvra Sengupta2014-02-071-2/+2
| | | | | | | | | | Change-Id: I851a50e0ebb7554b821fb591375bb6ebd6240ade BUG: 1036539 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/6939 Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-replication: gverify should use disk_size and available_sizeHarshavardhana2014-02-061-14/+27
| | | | | | | | | | | | | | | "commit ef4bb42382a5c9c3ca0907136669a659c2a20311" was incomplete since reduction in sizes doesn't really have any affect on the problem when disks sizes are similar. Actual fix is to verify disk_size and then verify available_size Change-Id: Idb57f606df229c0bdaeda8852c74e84d29109ffc BUG: 1020154 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/6844 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Fix no error when slave size less than master sizeAravinda VK2014-01-231-2/+2
| | | | | | | | | | | | | | | | in df command output, default block-size is 1K, but BUFFER_SIZE is in bytes, so verify will succeed even if slave size is less than master size. Added -B1 to df command(that is --block-size=1) to get all values in bytes. BUG: 1056518 Change-Id: I2384d2a6f2505d967e0538b95fbf111c15b26f30 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/6746 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* gsyncd / geo-rep: geo-replication fixesAjeet Jha2013-12-121-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -> "threaded" hybrid crawl. -> Enabling metatadata synchronization. -> Handling EINVAL/ESTALE gracefully while syncing metadata. -> Improvments to changelog crawl code. -> Initial crawl changelog generation format. -> No gsyncd restart when checkpoint updated. -> Fix symlink handling in hybrid crawl. -> Slave's xtime key is 'stime'. -> tar+ssh as data synchronization. -> Instead of 'raise', just log in warning level for xtime missing cases. -> Fix for JSON object load failure -> Get new config value after config value reset. -> Skip already processed changelogs. -> Saving status of each individual worker thread. -> GFID fetch on slave for purges. -> Add tar ssh keys and config options. -> Fix nlink count when using backend. -> Include "data" operation for hardlink. -> Use changelog time prefix as slave's time. -> Process changelogs in parallel. Change-Id: I09fcbb2e2e418149a6d8435abd2ac6b2f015bb06 BUG: 1036539 Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/6404 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* geo-rep: Increasing the buffer_size for master-salve size comparision.Ajeet Jha2013-10-241-1/+2
| | | | | | | | | | | | The buffer size is increased to 100MB, considering the space required by slave filesystem bookkeeping. Change-Id: Ib296f0e03b4d7a2de6d5ff0ae9ab09a5261e3e7c BUG: 1020154 Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/6102 Reviewed-by: Avra Sengupta <asengupt@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* glusterd/gverify: Check for passwordless ssh in gverify.Venky Shankar2013-09-041-7/+15
| | | | | | | | | | Change-Id: I8c2d398114ad4534bcc052f9a5be8bbb2e7e2582 BUG: 999531 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/5677 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* glusterd/gverify.sh: Stops session being created with invalid slave detailsVenky Shankar2013-09-041-9/+16
| | | | | | | | | | | | | create force will fail with proper message, if the ip is not reachable, or is unable to fetch slave details. Change-Id: I44a3ba777b37702ffd0e48e9cb46c51e293327d4 BUG: 988314 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/5516 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* gverify/glusterd : Added check to warn the user of existing files in slave ↵Venky Shankar2013-09-041-11/+13
| | | | | | | | | | | | | volume Change-Id: I41c9fb44613386189d9f3c090729f4ded8e4dea8 BUG: 990997 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/5453 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* glusterd/cli changes for distributed geo-repAvra Sengupta2013-07-264-9/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commands: gluster system:: execute gsec_create gluster volume geo-rep <master> <slave-url> create [push-pem] [force] gluster volume geo-rep <master> <slave-url> start [force] gluster volume geo-rep <master> <slave-url> stop [force] gluster volume geo-rep <master> <slave-url> delete gluster volume geo-rep <master> <slave-url> config gluster volume geo-rep <master> <slave-url> status The geo-replication is distributed. The session will be created, and gsyncd will be spawned on all relevant nodes, instead of only one node. geo-rep: Collecting status detail related data Added persistent store for saving information about TotalFilesSynced, TotalSyncTime, TotalBytesSynced Changes in the status information in socket: Existing(Ex): FilesSynced=2;BytesSynced=2507;Uptime=00:26:01; New(Ex): FilesSynced=2;BytesSynced=2507;Uptime=00:26:01;SyncTime=0.69978; TotalSyncTime=2.890044;TotalFilesSynced=6;TotalBytesSynced=143640; Persistent details stored in /var/lib/glusterd/geo-replication/${mastervol}/${eSlave}-detail.status Change-Id: I1db7fc13ffca2e415c05200b0109b1254067f111 BUG: 847839 Original Author: Avra Sengupta <asengupt@redhat.com> Original Author: Venky Shankar <vshankar@redhat.com> Original Author: Aravinda VK <avishwan@redhat.com> Original Author: Amar Tumballi <amarts@redhat.com> Original Author: Csaba Henk <csaba@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/5132 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* gsyncd: distribute the crawling loadAvra Sengupta2013-07-265-2/+207
| | | | | | | | | | | | | | | | | | | * also consume changelog for change detection. * Status fixes * Use new libgfchangelog done API * process (and sync) one changelog at a time Change-Id: I24891615bb762e0741b1819ddfdef8802326cb16 BUG: 847839 Original Author: Csaba Henk <csaba@redhat.com> Original Author: Aravinda VK <avishwan@redhat.com> Original Author: Venky Shankar <vshankar@redhat.com> Original Author: Amar Tumballi <amarts@redhat.com> Original Author: Avra Sengupta <asengupt@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/5131 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* move 'xlators/marker/utils/' to 'geo-replication/' directoryAvra Sengupta2013-07-224-0/+534
Change-Id: Ibd0faefecc15b6713eda28bc96794ae58aff45aa BUG: 847839 Original Author: Amar Tumballi <amarts@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/5133 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-13 09:24:49 +0000