summaryrefslogtreecommitdiffstats
path: root/geo-replication/syncdaemon
Commit message (Collapse)AuthorAgeFilesLines
* geo-rep: Fix Geo-rep status if monitor.pid file not existsAravinda VK2016-09-151-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If monitor.pid file not exists, gsyncd fails with following traceback Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 201, in main main_i() File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 681, in main_i brick_status.print_status(checkpoint_time=checkpoint_time) File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py", line 343, in print_status for key, value in self.get_status(checkpoint_time).items(): File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py", line 262, in get_status with open(self.monitor_pid_file, "r+") as f: IOError: [Errno 2] No such file or directory: '/var/lib/glusterd/ geo-replication/master_node_slave/monitor.pid' If Georep status command this worker's status will not be displayed since not returning expected status output. > Reviewed-on: http://review.gluster.org/15416 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1374631 Change-Id: I600a2f5d9617f993d635b9bc6e393108500db5f9 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15447 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* geo-rep: Defunct tar process after syncAravinda VK2016-09-151-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | After every sync iteration with tarssh mode leaves defunct tar process. Added wait for tar process to prevent this issue. > Reviewed-on: http://review.gluster.org/15426 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1375541 Change-Id: I9953239ef601cc1970c814b00074b45eb00f481e Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 6b30e9bf5a612e105eb7ded0a89ef25fd8530ba5) Reviewed-on: http://review.gluster.org/15490 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* geo-rep: Use configured log_level for libgfchangelog logsAravinda VK2016-09-134-2/+21
| | | | | | | | | | | | | | | | | | | | | | libgfchangelog was not respecting the log_level configured in Geo-replication. With this patch Libgfchangelog log level can be configured using `config changelog_log_level TRACE`. Default Changelog log level is INFO > Reviewed-on: http://review.gluster.org/15078 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1374606 Change-Id: Ida714931129f6a1331b9d0815da77efcb2b898e3 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15449 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* geo-rep: Fix ESTALE/EINVAL issue during set_{xtime,stime}Aravinda VK2016-09-131-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | Setfattr may get ESTALE/EINVAL if a file is being unlinked. To prevent worker crashing, added retry for these error messages. On second retry it will get ENOENT and that error is handled by ignoring. > Reviewed-on: http://review.gluster.org/15404 > Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> BUG: 1374628 Change-Id: Ic660fa13208366d57c8d3d492bbef611475e45b7 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15451 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* geo-rep: Fix logging sync failuresAravinda VK2016-09-131-1/+8
| | | | | | | | | | | | | | | | | | | | | | If Rsync/Tar subprocess dies, while logging error Geo-rep fails with EBADF while accessing error file. Also worker dies while accessing elines before it is set. > Reviewed-on: http://review.gluster.org/15379 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1374595 Change-Id: I9cfce116e8aafa4a98654f5190d40a455af8ec95 Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit c0f877c0374d97e0bee17aac4850d7655a35e61b) Reviewed-on: http://review.gluster.org/15443 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* geo-rep: Handle EISDIR error during UnlinkAravinda VK2016-09-131-1/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During Rename, If Source and Target has same inode then Geo-rep unlinks source. But if source is a directory then this will fail with below traceback Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 113, in worker res = getattr(self.obj, rmeth)(*in_data[2:]) File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 772, in entry_ops os.unlink(entry) OSError: [Errno 21] Is a directory: '.gfid/12711ebf-7fdc-4f4b-9850-2d75581eb 452/New folder' With this patch, if EISDIR, rmdir is tried. Logs error in Slave log in case of ENOTEMPTY. > Reviewed-on: http://review.gluster.org/15132 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1374579 Change-Id: I099af4192adac5125c0a23988ceb6506f91e987f Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 8a6236e4cfc7bf86e881e5c770a19040a6060ad0) Reviewed-on: http://review.gluster.org/15436 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* features/libgfchangelog: Log failure in gf_histroy_changelogKotresh HR2016-08-183-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | Add error logs if gf_history_changelog fails. If requested changelog range is not available, log the error and exit instead of continuing the loop and exiting in readdir without logging. Also fixed the duplicate MSGID number in 'changelog-lib-messages.h' > Change-Id: Icd71b89ae23b48a71380657ba5649029c32fabfd > BUG: 1362151 > Signed-off-by: Kotresh HR <khiremat@redhat.com> > Reviewed-on: http://review.gluster.org/15064 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Aravinda VK <avishwan@redhat.com> Change-Id: Icd71b89ae23b48a71380657ba5649029c32fabfd BUG: 1365877 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/15139 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Handle Config parser errorsAravinda VK2016-07-282-10/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Python ConfigParser lib has two methods, readfp and read, it should be used as follows. readfp(open("defaults.conf")) read("custom.conf") ConfigParser.read(path) ignores any file errors, which is intentional since errors are handled while loading default config. Geo-rep uses only one config file(Session config in Master side and Template config on Slave side) so we should use readfp to avoid skipping OS errors. config.read is retained in case of `--config-set-rx` where glusterd creates new template config file. BUG: 1357759 Change-Id: I15a14d3743facd7b8c7af0edc70fdefaa43efd04 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14777 (cherry picked from commit d94bf608b16b82f2c8f8588a96459cb746773b32) Reviewed-on: http://review.gluster.org/14946 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* georep: add reset-sync-time option for session deleteMilind Changire2016-07-213-3/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Set the stime xattr at all the brick roots to (0,0) if the argument reset-sync-time has been provided on the command-line. To avoid testing against directory specific stime, the remote stime is assumed to be minus_infinity, if the root directory stime is set to (0,0), before the directory scan begins. This triggers a full volume resync to slave in the case of a geo-rep session recreation with the same master-slave volume pair. Command synopsis: gluster volume geo-replication <MASTERVOL> <SLAVE>::<SLAVEVOL> delete \ [reset-sync-time] Update gluster cli man page to include new sub-command reset-sync-time. Change-Id: Ie4ce03b9425ed9bb81eda8681058c0fc6f990948 BUG: 1357772 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/14051 Reviewed-by: Kotresh HR <khiremat@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 70fd68d94f768c098b3178c151fa92c5079a8cfd) Reviewed-on: http://review.gluster.org/14952
* geo-rep: Safely handle if unliked GFID not present in data listAravinda VK2016-06-271-1/+2
| | | | | | | | | | | | | | | | | If unlinked GFID is not present in data list to be synced then Geo-rep worker was crashing with KeyError. Handled KeyError with this patch. BUG: 1348085 Change-Id: I5a1c9ca4473e32606df2e5c7e26c95faf55d44c0 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14706 (cherry picked from commit 4797ca3778d82a671716d4913c14f285591ae959) Reviewed-on: http://review.gluster.org/14766 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* geo-rep: Handle stime/xtime set failuresAravinda VK2016-06-271-6/+10
| | | | | | | | | | | | | | | | | | | While setting stime/xtime, if the file or directory is already deleted then Geo-rep will crash with ENOENT. With this patch, Geo-rep will ignores ENOENT since stime/xtime can't be applied on a deleted file/directory. Change-Id: I2d90569e51565f81ae53fcb23323e4f47c9e9672 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1345882 Reviewed-on: http://review.gluster.org/14529 (cherry picked from commit 1a348bfaeb9f2a50ec8ce27e5477e9b430c58b3c) Reviewed-on: http://review.gluster.org/14713 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Error message cleanupAravinda VK2016-06-271-10/+13
| | | | | | | | | | | | | | | | | If ssh returns 127 that means the remote gsyncd path is wrong or push-pem failed during create. Existing error message was pointing old documentation. Change-Id: Ifbbb4a604fc0ae0fd5cb2746df6363bf28cde1e9 BUG: 1347206 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14673 (cherry picked from commit 3b69e25c0f3cf0b958fbd2b8459712ad21239caa) Reviewed-on: http://review.gluster.org/14740 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* glusterd/geo-rep: upgrade path when slave vol uuid involvedSaravanakumar Arumugam2016-06-131-0/+3
| | | | | | | | | | | | | | | | | | slave volume uuid is involved in identifying a geo-replication session. This patch addresses upgrade path, where existing geo-rep session is gracefully upgraded to involve slave volume uuid. Change-Id: Ib7ff5109b161592f24fc86fc7e93a407655fab86 BUG: 1342453 Reviewed-on: http://review.gluster.org/#/c/14425 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/14641 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Fix volume stop with geo-rep sessionAravinda VK2016-06-051-0/+12
| | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/14636 The slave volume uuid is appended to slave_url which is saved in 'gsync_slaves' dictionary with the commit "http://review.gluster.org/13111". But the gsyncd expects slave_url with out slave volume uuid otherwise it fails which in turn fails volume stop with geo-rep. This patch fixes the same. Change-Id: I036f4fce43115ff848daebfe1b2c58765cffab5a BUG: 1342431 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/14637 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: update peers section in gsyncd confSaravanakumar Arumugam2016-06-011-3/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Once Slave volume uuid is involved as part of a geo-rep session, it is possible to create the same geo-rep session with different (slave)host. But, it reflects default values for geo-rep configuration values originally configured for old geo-rep session. Reason is, slave host is used while saving config options in gsyncd.conf. With new slave host, it is not possible to retrieve those config values. Solution: Remove slave host related information from gsyncd.conf and have only master volume and slave volume as part of peers section. Also, during upgrade from old geo-rep session, update peers section to reflect only master volume and slave volume. Change-Id: I7debf35a09a28d030b706b0c3e5d82c9b0467d0e BUG: 1341121 Reviewed-on: http://review.gluster.org/#/c/14558/ Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/14566 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Handle Worker kill gracefully if worker already diedAravinda VK2016-06-011-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If Agent dies for any reason, monitor tries to kill Worker also. But if worker is also died then kill command raises error ESRCH: No such process. [2016-05-23 16:49:33.903965] I [monitor(monitor):326:monitor] Monitor: Changelog Agent died, Aborting Worker(/bricks/brick0/master_brick0) [2016-05-23 16:49:33.904535] E [syncdutils(monitor):276:log_raise_exception] <top>: FAIL: Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 306 in twrap tf(*aa) File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 393, in wmon slave_host, master) File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 327, in monitor os.kill(cpid, signal.SIGKILL) OSError: [Errno 3] No such process With this patch, monitor will gracefully handle if worker is already died. Change-Id: I3ae5f816a3a197343b64540cf46f5453167fb660 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1341068 Reviewed-on: http://review.gluster.org/14512 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> (cherry picked from commit 4f4a94a35a24d781f3f0e584a8cb59c019e50d6f) Reviewed-on: http://review.gluster.org/14562 Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* glusterd/geo-rep: slave volume uuid to identify a geo-rep sessionSaravanakumar Arumugam2016-05-182-0/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Currently, it is possible to create multiple geo-rep session from the Master host to Slave host(s), where Slave host(s) belonging to the same volume. For example: Consider Master Host M1 having volume tv1 and Slave volume tv2, which spans across two Slave hosts S1 and S2. Currently, it is possible to create geo-rep session from M1(tv1) to S1(tv2) as well as from M1(tv1) to S2(tv2). When the Slave Host is alone modified, it is identified as a new geo-rep session (as slave host and slave volume together are identifying Slave side). Also, it is possible to create both root and non-root geo-rep session between same Master volume and Slave volume. This should also be avoided. Solution: This multiple geo-rep session creation must be avoided and in order to avoid, use Slave volume uuid to identify a Slave. This way, we can identify whether a session is already created for the same Slave volume and avoid creating again (using different host). When the session creation is forced in the above scenario, rename the existing geo-rep session directory with new Slave Host mentioned. Change-Id: I9239759cbc0d15dad63c48b8cf62950bb687c7c8 BUG: 1335728 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13111 Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit a9128cda34b1f696b717ba09fa0ac5a929be8969) Reviewed-on: http://review.gluster.org/14322 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Fix Entry Creation issue with non root UID/GIDAravinda VK2016-03-091-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | During entry_ops RENAME Geo-rep sends stat info along with the recorded info from Changelog. In Slave side if Source file exists Geo-rep renames to Target file by calling os.rename. If source file does not exists, it tries to create Target file directly using available stat info from Master. If UID and GID are different in Master for that file then stat info will have different UID/GID during Create. Geo-rep gets EACCES when it tries to create a new entry using gfid-access with different UID/GID. With this patch, Entry creation with different UID/GID is split into two operations. Create Entry with UID:0 and GID:0 and then set UID/GID. Change-Id: I4987e3a205d8513c06fa66198cde145a87003a01 BUG: 1312762 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on:http://review.gluster.org/13542 Reviewed-on: http://review.gluster.org/13643 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: hard-link rename issues on changelog replayMilind Changire2016-03-091-4/+24
| | | | | | | | | | | | | | | | | | | | | | | Problem: LINK + RENAME changelog when replayed after worker restart causes stale hard-links to persist since VFS returns success for RENAME if hard-links point to same inode. Solution: Worker detects RENAME being issued on hard-links to the same inode and unlinks the source file-name. Conditionally rename by verifying that the source gfid matches with the on-disk gfid on the slave. Change-Id: I3ff1c30ef79e77503c8b246d46dab8ac3059ccf2 BUG: 1296175 Reviewed-on: http://review.gluster.org/13189 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/13638 Smoke: Gluster Build System <jenkins@build.gluster.com> Tested-by: Aravinda VK <avishwan@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* georep: avoid creating multiple entries with same gfidMilind Changire2016-03-081-4/+12
| | | | | | | | | | | | | | | | | | | | | | | | Problem: CREATE + RENAME changelogs replayed by geo-replication cause stale old-name entries with same gfid on slave nodes. A gfid is a unique key in the file-system and should not be assigned to multiple entries. Solution: Create entry on slave only if lstat(gfid) at aux-mount fails. This applies to files as well as directories. Change-Id: Ice3340f4ae1251c2dcef024a2388c4d33b5d4919 BUG: 1296208 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/13316 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> (cherry picked from commit 87d93fac9fcc4b258b7eb432ac4151cdd043534f) Reviewed-on: http://review.gluster.org/13571
* geo-rep: Fix CPU utilization 100% with PopenAravinda VK2016-03-081-0/+4
| | | | | | | | | | | | | | | | | | | | Errorstore is maintained by Geo-rep to collect errors from the child processes opened using Popen. If Popen.communicate is used then it closes stderr. When stderr is not available errorstore.tailer() will enter into infinite loop without gap. With this patch, sleep time added when stderr of Child process is already closed. Change-Id: Ic36aabd6de35b259467d0bab7952468432867a94 BUG: 1315582 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13637 Reviewed-on: http://review.gluster.org/13645 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Handling Rsync/Tar errors efficientlyAravinda VK2016-03-082-87/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Geo-rep processes Changelogs in Batch, if one file in batch fails with rsync error that Changelog file is reprocessed multiple times. After MAX_RETRY, it logs all the GFIDs from that batch as Skipped. This patch addresses following issues, 1. When Rsync/Tar fails do not parse Changelog again for retry 2. When Rsync/Tar fails do not replay Entry operations, only retry rsync/tar for those GFIDs 3. Log Error in Rsync/Tar only in the last Retry 4. Do not log Skipped GFIDs since Rsync/Tar errors are logged for only failed files. 5. Changed Entry failures as Error instead of Warning BUG: 1313309 Change-Id: Ie134ce2572693056ab9b9008cd8aa5b5d87f7975 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12856 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit d136a789258e8f600e536717da156a242d8ed9a5) Reviewed-on: http://review.gluster.org/13558
* geo-rep: Handle ERANGE error during listxattrAravinda VK2016-03-081-7/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | llistxattr in Geo-rep is two syscall instead of one SIZE = llistxattr(PATH, &BUF, 0); BUF = create_buf(SIZE); _ = llistxattr(PATH, &BUF, SIZE); So if any new xattrs added just after first call by any other worker, second syscall will fail with ERANGE error. Now Geo-rep sends BUF with large size(256*100) and gets value with only one syscall. Raises OSError if fails with ERANGE error even after sending large BUF. Change-Id: I8ade4bbe9a0a8ea908ed9dedcd3f2ff4c6fe6f51 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1313311 Reviewed-on: http://review.gluster.org/13106 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 656a0e64e5f465561ed29297421ec150de32f2a1) Reviewed-on: http://review.gluster.org/13560
* geo-rep: Handle hardlink in Tiering based volumeSaravanakumar Arumugam2016-03-071-10/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Hardlinks are synced as Sticky bit files to Slave in a Tiering based volume. In a Tiering based volume, cold tier is hashed subvolume and geo-rep captures all namespace operations in cold tier. While syncing a file and its corresponding hardlink, it is recorded as MKNOD in cold tier(for both) and We end up creating two different files in Slave. Solution: If MKNOD with Sticky bit set is present, record it as LINK. This way it will create a HARDLINK if source file exists (on slave), else it will create a new file. This way, Slave can create Hardlink file itself (instead of creating a new file) in case of hardlink. Change-Id: Ic50dc6e64df9ed01799c30539a33daace0abe6d4 BUG: 1302979 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13281 Reviewed-on: http://review.gluster.org/13315 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Symlink Rename issueAravinda VK2016-01-032-3/+15
| | | | | | | | | | | | | | | | | | | | | | | | If ENTRY creation failed for symlink in Slave and symlink renamed in Master. If Source not exists to Rename in Slave Geo-rep interprets as Create of Target file. Geo-rep sends blob of regular file to create symlink instead of sending blob of symlink. With this patch, Geo-rep identifies symlink and sends respective blob. BUG: 1292697 Change-Id: If9351974d1945141a1d3abb838b7d0de7591e48e Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12917 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Tested-by: Milind Changire <mchangir@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> (cherry picked from commit 3151194fad1aed3ab340b53aeeb4e8d131538d12) Reviewed-on: http://review.gluster.org/12998 Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Fix syntax errors in GsyncdErrorAravinda VK2016-01-031-3/+2
| | | | | | | | | | | | | | | | %s was not replaced by actual values in GsyncdError BUG: 1279644 Change-Id: I3c0a10f07383ca72844a46f930b4aa3d3c29f568 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12566 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> (cherry picked from commit 74699ddd777f7e862991cf3afad91823d30e5b84) Reviewed-on: http://review.gluster.org/12724 Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Fix getting subvol countKotresh HR2016-01-031-1/+4
| | | | | | | | | | | | | | | | Tiering doesn't support disperse volume as hot tier, hence xml output doesn't give 'hotdisperseCount'. Remove the usage of 'hotdisperseCount' in geo-rep and return 0 instead. BUG: 1293309 Change-Id: I3f50d21cb51db91e31faebf69af4f72360420b73 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/13062 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13068
* geo-rep: Fix getting subvol numberKotresh HR2016-01-032-17/+47
| | | | | | | | | | | | | | | | | | | Fix getting subvol number if the volume type is tier. If the volume type was tier, the subvol number was calculated incorrectly and hence few of workers didn't become ACTIVE resulting in files not being replicated from corresponding brick. This patch addresses the same. BUG: 1293309 Change-Id: I318de346657d330a2394507514bdff61feb92d27 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12994 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-on: http://review.gluster.org/13059
* Add support for sparse files to tarssh methodAlex Markelov2015-12-301-1/+1
| | | | | | | | | | | | | | | | | | | Without '--sparse' option tar will not properly archive sparse file and geo-replication will result in non-sparse file on the remote end. Here is more on how I arrived at this http://markelov.org/wiki/index.php/GlusterFS_3.6.1_on_CentOS_6.5:_geo-replication_and_sparse_files_problem Change-Id: I8d671964a1b48bbb916e4a064571221bf3631494 BUG: 1290719 Signed-off-by: Alex Markelov <alex@markelov.org> Reviewed-on: http://review.gluster.org/12476 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit eff14863fb6a6da0ccf8a40343160b09f29ea493) Reviewed-on: http://review.gluster.org/12949 Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: use cold tier bricks for namespace operationsSaravanakumar Arumugam2015-12-074-17/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: symlinks are not getting synced to slave in a Tiering based volume. Solution: Now, symlinks are created directly in cold tier bricks( in the backend). Earlier, cold tier was avoided for namespace operations and only hot tier was used while processing changelogs. Now, cold tier is HASH subvolume in a Tiering volume. So, carry out namespace operation only in cold tier subvolume and avoid hot tier subvolume to avoid any races. Earlier, XSYNC was used(and changeloghistory avoided) during initial sync in order to avoid race while processing historychangelog in Hot tier. This is no longer required as there is no race from Hot tier. Also, avoid both live and history changelog ENTRY operations from Hot tier to avoid any race with cold tier. Change-Id: Ia8fbb7ae037f5b6cb683f36c0df5c3fc2894636e BUG: 1288027 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/12844 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 93f31189ce8f6e2980a39b02568ed17088e0a667) Reviewed-on: http://review.gluster.org/12891
* geo-rep: fd close and fcntl issueAravinda VK2015-12-071-15/+19
| | | | | | | | | | | | | | | | | | | | | When any of the open fd of a file is closed on which fcntl lock is taken even though another fd of the same file is open on which lock is taken, all fcntl locks will be released. This causes both replica workers to be ACTIVE sometimes. This patche fixes that issue. BUG: 1287456 Change-Id: I8f113230d16075f18020044898a7eda5e2a5a0ac Original-Author: Aravinda VK <avishwan@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12752 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/12842
* geo-rep: Avoid cold tier bricks during ENTRY operationSaravanakumar Arumugam2015-12-074-5/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a series of patch which aims to fix geo-replication in a Tiering Volume. Problem: Consider, a file is placed in volume initially and then hot tier is attached. During any operation on the file, due to lookup a linkto file is created in hot tier. Now, any namespace operation carried out on the file is recorded in both cold and hot tier. There is a room for races when both changelogs are replayed. Solution: So, We are going to replay (namespace related)operations only in the hot tier. Why? a. If the file is directly placed in Hot tier , all fops will be recorded in HOT tier. b. If the file is already present in Cold tier, and if any fop is carried out, it creates linkto file in Hot tier. Now, operations like UNLINK, RENAME are captured in Hot tier(by means of linkto file). This way, we can get both tier's operation in HOT tier itself. Now, once the file is demoted to COLD tier, any namespace operation carried out on the cold tier can be avoided as we directly RECORD the same in HOT tier. How? 1. Check whether the brick is cold tier and skip ENTRY operation. 2. Also, if it is cold tier brick, use Xsync(which is used during initial run). This will help in getting all cold tier bricks changes using File System crawl and helps in avoiding races with hot tier brick(which can happen if historychangelog used in cold tier brick). Dependent patches: 1. http://review.gluster.org/12239 2. http://review.gluster.org/12326 Change-Id: I7692b1dbb8813a7e253451bca02f8f09a5782dde BUG: 1275173 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/12355 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 6188b5fcebc56b3d8af1956beeec9988f3e8f268) Reviewed-on: http://review.gluster.org/12429 Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: geo-rep to handle CAPS based HostnameSaravanakumar Arumugam2015-12-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Problem: geo-replication session creation fails when Hostname is having CAPS in it. Issue is with the regex pattern which handles only small lettered Hostname. Fix: Fix the regex pattern to handle CAPS based hostname as well. Change-Id: I5c99c102e9706acc2b1fab1e6bf158e68beed373 BUG: 1287960 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/12216 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> (cherry picked from commit f6142e97c6219d2c1c482a414690020ed1275bdb) Reviewed-on: http://review.gluster.org/12862 Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Fix Geo-rep logging to log datetime in GMT/UTCAravinda VK2015-12-021-0/+1
| | | | | | | | | | | | | | | | Geo-rep is logging in Local time, all other Gluster logs are in GMT/UTC. It is very difficult to co-relate Geo-rep logs with other Gluster logs. BUG: 1284737 Change-Id: Ieae8bda7e4788e587cf4595e21e0e772c210cfbb Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12583 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> (cherry picked from commit 41e900199d7f369862b21b739dd11602d0d7c48d) Reviewed-on: http://review.gluster.org/12723 Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* geo-rep: Fix FD leak from Active Geo-rep workerAravinda VK2015-12-012-0/+16
| | | | | | | | | | | | | | | | | | | | | Active worker tries to acquire lock in each iteration. On every successfull lock acqusition it was not closing previously opened lock fd. To see the leak, get the PID of worker, ps -ax | grep feedback-fd watch ls /proc/$pid/fd BUG: 1225567 Change-Id: Ic476c24c306e7ab372c5560fbb80ef39f4fb31af Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12332 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 42def948ac8e5d24278cb000cc8f8906b83a8592) Reviewed-on: http://review.gluster.org/12650 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Handle FXATTROP and XATTROPKotresh HR2015-11-271-1/+2
| | | | | | | | | | | | | | | | | | | GEO-REP INTEROP WITH SHARD FEATURE If it is FXATTROP or XATTROP in changelog, add the gfid to rsync queue. BUG: 1284453 Change-Id: If598da1543bbff7fe9228ba57c7e813ed2a3a7f2 Reviewed-on: http://review.gluster.org/12226 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12721 Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Fix syncing chown in xsync crawlKotresh HR2015-11-271-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GEO-REP INTEROP WITH SHARD FEATURE Problem: The sequence of entry creation and chown in master is recorded as creation of entry with resulted user:group in xsync changelog. During sync, entry creation is always split into two ops, MKNOD and SETATTR. Hence the issue is not being hit otherwise it would have failed with EPERM if parent is owned by different user. But with shard translator being enabled on slave, doing entry creation with MKNOD and SETATTR is not allowed, SETATTR fails as it accesses inode structure which is not linked. Solution: The sequence of entry creation and chown in master should be recorded as MKNOD and SETATTR separately always and do entry creation with single op in gfid-access xlator. The gfid-access patch will be sent separately. BUG: 1284453 Change-Id: Ia577aa5270cb96b86830885d6c4c01fb0133eeed Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12205 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12729 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: New Config option for ssh_portAravinda VK2015-11-242-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | If different port used for SSH instead of 22, Geo-replication was failing to establish SSH connection. ssh_port option can be added using config:ssh_command and config:ssh_command_tar, but user has to remember complete ssh command used with parameter to add/modify ssh port. This patch adds new config option for ssh_port, gluster volume geo-replication <MASTERVOL> <SLAVEHOST::<SLAVEVOL> \ config ssh_port 52022 Change-Id: I7753a09485f0b1f49d2b2a80b962c720817c96f4 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1283060 Reviewed-on: http://review.gluster.org/12444 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 7d35eb5926869ed084295600502a85ce13be506f) Reviewed-on: http://review.gluster.org/12607 Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Kill Geo-rep Worker when Agent process diesAravinda VK2015-11-243-14/+44
| | | | | | | | | | | | | | | | | | | | When Changelog agent process dies, Geo-replication fails to detect and worker will run without respective Changelog agent. Status shows Active/Passive without any progress. With this patch, Worker process gets killed whenever Changelog agent dies. Change-Id: I30b4cc77f924f7e8174b8bfe415ac17f0b3851b4 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1279362 Reviewed-on: http://review.gluster.org/12485 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 5d1ff7efd6ab3bd29a29922a9ea1e1aaf02544ad) Reviewed-on: http://review.gluster.org/12550
* geo-rep: Update last_synced_time in XSyncAravinda VK2015-11-091-14/+9
| | | | | | | | | | | | | | | | | During XSync crawl, last_synced time in status file was not updated. This patch fixes the issue by updating status file when stime xattr is updated after Xsync or Changelog Crawl. Change-Id: I4dc3a2d4c3d8378a939da0868caf1aef4f789599 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1279306 Reviewed-on: http://review.gluster.org/11771 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> (cherry picked from commit ee0b1a3bf11a4d05696212d91c932ddb7c7091ee) Reviewed-on: http://review.gluster.org/12545 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Update geo-rep status, if monitor process is killedSaravanakumar Arumugam2015-11-012-2/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: When the monitor process itself is getting killed, geo-rep session still shows as active. Status command will just pick up the content from the status file to show the output. Monitor process is the one which updates the Status file. When the monitor process itself gets killed, there is no way to update the status file. So, geo-rep session status command ends up showing last updated Status present in the status file. Solution: While getting the status output, check whether monitor process is running. If it is NOT running, update the status as STOPPED. Change-Id: I86a7ac1746dd8f27eef93658e992ef16f6068d9d BUG: 1276060 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/11873 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> (cherry picked from commit 4d4c7d5dc54850dcf916083b2b1398d9bfe2bfe6) Reviewed-on: http://review.gluster.org/12448 Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Add data operation if mknod with tier attributeSaravanakumar Arumugam2015-10-291-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a series of patches which aims to fix geo-replication in a Tiering Volume. Problem: Consider, a file is placed in volume initially and then hot tier is attached. During any operation on the file, due to lookup a linkto file is created in hot tier. Now, any namespace operation carried out on the file is recorded in both cold and hot tier. There is a room for races when both changelogs are replayed. Solution: So, We are going to replay (namespace related)operations only in the hot tier. Why? a. If the file is directly placed in Hot tier, all fops will be recorded in HOT tier. b. If the file is already present in Cold tier, and if any fop is carried out, it creates linkto file in Hot tier. Now, operations like UNLINK, RENAME are captured in Hot tier(by means of linkto file). This way, we can get both tier's operation in HOT tier itself. But, We may miss initial Data sync immediately after creating the file as it is only recording MKNOD. So, if MKNOD encountered with sticky bit set, queue DATA operation for the corresponding gfid. (This is addressed here in this patch) So, If tier-gfid linkto is set, we need to record the corresponding MKNOD. Earlier this was avoided as it was set as INTERNAL fop. (This changelog related changes are addressed in the patch: - http://review.gluster.org/12417) Change-Id: I2fa84cfa2b0f86506c3d15d484138ab9651e4f83 BUG: 1275173 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/12326 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit ffc39c9d8807464b5c78959bc43dc12b22f5a37b) Reviewed-on: http://review.gluster.org/12427 Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Do not crash worker on ESTALEKotresh HR2015-08-051-9/+1
| | | | | | | | | | | | | | | | Handle ESTALE returned by lstat gracefully by retrying it. Do not crash the worker. BUG: 1249547 Change-Id: I57fb9933900153ab41c3d9b73748b1cdaa8d89ca Reviewed-on: http://review.gluster.org/11772 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/11820 Reviewed-by: Milind Changire <mchangir@redhat.com>
* geo-rep: Fix history failureKotresh HR2015-08-052-5/+19
| | | | | | | | | | | | | | | | | | | | | | | | | Both ACTIVE and PASSIVE workers register to changelog at almost same time. When PASSIVE worker becomes ACTIVE, the start and end time would be current stime and register_time repectively for history API. Hence register_time would be less then stime for which history obviously fails. But it will be successful for the next restart as new register_time > stime. Fix is to pass current time as the end time to history call instead of the register_time. Also improvised the logging for ACTIVE/PASSIVE switching. BUG: 1247882 Change-Id: I40c582cc32fe29a6c30340ec81a3b5d30e461e71 Reviewed-on: http://review.gluster.org/11524 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/11784 Reviewed-by: Milind Changire <mchangir@redhat.com>
* geo-rep: Fix fd referenced before assignmentKotresh HR2015-07-121-2/+6
| | | | | | | | | | | | | | | | Fix fd reference before assignment in mgmt_lock function. BUG: 1240607 Change-Id: Ib8a5207a516eecb55a5f906ff5d49abc98cab2a6 Reviewed-on: http://review.gluster.org/11318 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/11563 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: ignore ESTALE as ENOENTAravinda VK2015-06-282-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | When DHT can't resolve a File it raises ESTALE, ignore ESTALE errors same as ENOENT after retry. Affected places: Xattr.lgetxattr os.listdir os.link Xattr.lsetxattr os.chmod os.chown os.utime os.readlink BUG: 1236093 Change-Id: I53f8dfa47911da93e0dcc20213afcbb47a14ccd8 Reviewed-On: http://review.gluster.org/11296 Original-Author: Aravinda VK <avishwan@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/11430 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Fix geo-rep fanout setup with meta volumeKotresh HR2015-06-251-2/+2
| | | | | | | | | | | | | | | | | | | Lock filename was formed with 'master volume id' and 'subvol number'. Hence multiple slaves try acquiring lock on same file and become PASSIVE ending up not syncing data. Using 'slave volume id' in lock filename will fix the issue making lock file unique across different slaves. Change-Id: I64c84670a5d9e1b0dfbdeb4479ee6b8e0c6b829e BUG: 1234898 Reviewed-On: http://review.gluster.org/11367 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/11366 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Fix toggling of use_meta_volume configKotresh HR2015-06-251-1/+1
| | | | | | | | | | | | | | | | | If meta-volume is deleted and use_meta_volume is set to false, geo-rep still fails complaining meta volume is not mounted. The patch fixes that issue. BUG: 1234695 Change-Id: I5ec7ad85399b90e85bca1d36750dd3fb29222db9 Reviewe-On: http://review.gluster.org/11358 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/11359 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Ignore ESTALE during unlink/rmdirAravinda VK2015-05-311-4/+6
| | | | | | | | | | | | | | | | | | during unlink/rmdir of Parent_GFID/Basename, if parent directory does not exists. Parent GFID will not get resolved and DHT raises ESTALE instead of ENOENT. Now ESTALE errors ignored during unlink/rmdir BUG: 1223286 Change-Id: If275c89fb9fc7d16004550805a4cd65be818540d Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed On: http://review.gluster.org/#/c/10837/ Reviewed-on: http://review.gluster.org/10913 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Fix Data counter issue in statusAravinda VK2015-05-311-2/+12
| | | | | | | | | | | | | | | | | ENTRY and META operations executed sequentially, DATA operations are handled async, increment happens when a changelog parsed. Decrement happens after the sync of all files. files_in_batch was reset multiple times in batch instead of once. BUG: 1224100 Change-Id: I87617f2fd5f4d3221a1c9f9d5a8efb0686c42bbe Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed On: http://review.gluster.org/#/c/10911/ Reviewed-on: http://review.gluster.org/10912 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Venky Shankar <vshankar@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-15 10:59:13 +0000