| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1. '--ignore-mising-args' option for rsync is not
being used even though the rsync version is
greater than 3.1.0. Fixed the same.
2. '--existing' option for rsync is also not being
used. Fixed the same.
3. geo-rep config fails to set rsync-options as the
value contains '--'. Interestingly, python argsparse
treats the value with '--' (e.g., --ignore-missing-args)
as option. But when passed with something like
--value=--ignore-missing-args, it succeeds. Fixed the
same.
Change-Id: Iaeb838acaff1c2920fee9c7f920c99edce13a0a1
Signed-off-by: Kotresh HR <khiremat@redhat.com>
fixes: bz#1629561
|
|
|
|
|
| |
Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu <nigelb@redhat.com>
|
|
|
|
| |
Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
|
|
|
|
|
|
| |
Change-Id: Iac241166d7a35dc7cc6cf07850f9f1bce38fe207
Updates: #411
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Commit af0d5a9b5375a5cd87ac10b429e2b9934718ce5b changes
python2 -> python3. This patch fixes a small issue introduced
by it.
Change-Id: Ib23c73683f570e8891f41476b661f37c89635fb5
updates: #411
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
see https://review.gluster.org/#/c/19788/,
https://review.gluster.org/#/c/19871/,
https://review.gluster.org/#/c/19952/,
https://review.gluster.org/#/c/20104/,
https://review.gluster.org/#/c/20162/,
https://review.gluster.org/#/c/20185/,
https://review.gluster.org/#/c/20207/,
https://review.gluster.org/#/c/20227/,
https://review.gluster.org/#/c/20307/,
https://review.gluster.org/#/c/20320/,
https://review.gluster.org/#/c/20332/,
https://review.gluster.org/#/c/20364/,
https://review.gluster.org/#/c/20441/, and
https://review.gluster.org/#/c/20484
shebangs changed from /usr/bin/python2 to /usr/bin/python3.
(Reminder, various distribution packaging guidelines require use
of explicit python version and don't allow '#!/usr/bin/env python',
regardless of how handy that idiom may be.)
glusterfs.spec(.in) package python{2,3}-gluster and python2 or
python3 dependencies as appropriate.
configure(.ac):
+ test for and use python2 or python3 as appropriate. If build
machine has python2 and python3, use python3. Override by
setting PYTHON=/usr/bin/python2 when running configure.
+ PYTHONDEV_CPPFLAGS from python[23]-config --includes is a
better match to the original python sysconfig.get_python_inc().
All those other extraneous flags breaks the build.
+ Only change the shebangs once. Changing them over and over
again, e.g., during a `make glusterrpms` in extras/LinuxRPM
just sends make (is it really make that's looping?) into an
infinite loop. If you figure out why, let me know.
+ Oldest python2 is python2.6 on CentOS 6 and Debian 8 (Jessie).
Everything else has 2.7 or 3.x
+ logic from https://review.gluster.org/c/glusterfs/+/21050, which
needs to be removed/merged after that patch is merged.
Builds on CentOS 6, CentOS 7, Fedora 28, Fedora rawhide, and the
mysterious RHEL > 7.
Change-Id: Idae21d3b6f58b32372e1daa0d234e491e563198f
updates: #411
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Autmatic gfid conflict resolution needs to be disabled
during failover/failback as it might lead to data loss
in the following scenario.
1. Master went down without syncing directory "dir1" to slave.
2. When slave is failed over to master, if a new file
is written inside "dir1", creating dir1 again if not
present, "dir1" ends up with different gfid on original
slave.
3. When original master is up and failed back, due to
automatic gfid conflict resolution, "dir1" present in
original master is deleted losing all files and only
new file created on original slave is restored.
Hence during failover/failback, automatic gfid conflict
resolution should be disabled. So in these cases, appropriate
decision is taken.
fixes: bz#1622076
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Change-Id: I433616f5d3e13d4b6eb675475bd554ca34928573
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes BAD_FREE at line number 75,105 and
UNUSED_VALUE at line number 108.
CID : 1274068
scan detail at [1].
[1] https://scan6.coverity.com/reports.htm#v42401/p10714/fileInstanceId=84384726&defectInstanceId=25600457&mergedDefectId=727233
Change-Id: I8dff42546204bf78c178c9dcaba534888c5354e9
updates: bz#789278
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Cleanup the Active/Passive logging code
which is redundant. With new status infra
implmented, this is redundant as every
status switch is logged by status infra.
fixes: bz#1619027
Change-Id: I0a6644cb998f3520e62a5189f21e4d66acc0e7c5
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Analysis:
Monitor process spawns monitor threads (one per brick).
Each monitor thread, forks worker and agent processes.
Each monitor thread, while intializing, updates the
monitor status file. It is synchronized using flock.
The race is that, some thread can fork worker while
other thread opened the status file resulting in
holding the reference of fd in worker process.
Cause:
flock gets unlocked either by specifically unlocking it
or by closing all duplicate fds referring to the file.
The code was relying on fd close, hence a reference
in worker/agent process by fork could cause the deadlock.
Fix:
1. flock is unlocked specifically.
2. Also made sure to update status file in approriate places so that
the reference is not leaked to worker/agent process.
With this fix, both the deadlock and possible fd
leaks is solved.
fixes: bz#1614799
Change-Id: I0d1ce93072dab07d0dbcc7e779287368cd9f093d
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is part of the reason why we use autoconf (i.e. configure).
For an ordinary clone+autogen.sh+configure SBIN_DIR is
/usr/local/sbin; for an rpm or dpkg build it will be /usr/sbin.
I wonder how many more are lurking in our sources? /usr/libexec is
one that frequently bites us on Debian and Ubuntu, which don't have
/usr/libexec. (But it's all Linux, right?)
See https://bugzilla.redhat.com/show_bug.cgi?id=1601532
Reported-by: lohmaier+rhbz@gmail.com
Change-Id: I6523894416cc06236ea1f99529efd36e957bd98e
updates: bz#1193929
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...
Only compile-tested!
Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1. MKDIR/RMDIR is recorded on all bricks. So if
one brick succeeds creating it, other bricks
should ignore it. But this was not happening.
The fix rename of directories in hybrid crawl,
was trying to rename the directory to itself
and in the process crashing with ENOENT if the
directory is removed.
2. If file is created, deleted and a directory is
created with same name, it was failing to sync.
Again the issue is around the fix for rename
of directories in hybrid crawl. Fixed the same.
If the same case was done with hardlink present
for the file, it was failing. This patch fixes
that too.
fixes: bz#1598884
Change-Id: I6f3bca44e194e415a3d4de3b9d03cc8976439284
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Geo-rep sometimes fails to sync the rename of symlink
if the I/O is as follows
1. touch file1
2. ln -s "./file1" sym_400
3. mv sym_400 renamed_sym_400
4. mkdir sym_400
The file 'renamed_sym_400' failed to sync to slave
Cause:
Assume there are three distribute subvolume (brick1, brick2, brick3).
The changelogs are recorded as follows for above I/O pattern.
Note that the MKDIR is recorded on all bricks.
1. brick1:
-------
CREATE file1
SYMLINK sym_400
RENAME sym_400 renamed_sym_400
MKDIR sym_400
2. brick2:
-------
MKDIR sym_400
3. brick3:
-------
MKDIR sym_400
The operations on 'brick1' should be processed sequentially. But
since MKDIR is recorded on all the bricks, The brick 'brick2/brick3'
processed MKDIR first before 'brick1' causing out of order syncing
and created directory sym_400 first.
Now 'brick1' processed it's changelog.
CREATE file1 -> succeeds
SYMLINK sym_400 -> No longer present in master. Ignored
RENAME sym_400 renamed_sym_400
While processing RENAME, if source('sym_400') doesn't
present, destination('renamed_sym_400') is created. But
geo-rep stats the name 'sym_400' to confirm source file's
presence. In this race, since source name 'sym_400' is
present as directory, it doesn't create destination.
Hence RENAME is ignored.
Fix:
The fix is not rely only on stat of source name during RENAME.
It should stat the name and if the name is present, gfid should
be same. Only then it can conclude the presence of source.
fixes: bz#1600405
Change-Id: I9fbec4f13ca6a182798a7f81b356fe2003aff969
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
| |
This module is no longer being used.
Fixes: bz#1597512
Change-Id: Ie5faf55c5961d9d7b5082c9c257351af712c41d7
Signed-off-by: Nigel Babu <nigelb@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
see https://review.gluster.org/#/c/19788/,
https://review.gluster.org/#/c/19871/,
https://review.gluster.org/#/c/19952/,
https://review.gluster.org/#/c/20104/,
https://review.gluster.org/#/c/20162/,
https://review.gluster.org/#/c/20185/,
https://review.gluster.org/#/c/20207/,
https://review.gluster.org/#/c/20227/,
https://review.gluster.org/#/c/20307/,
https://review.gluster.org/#/c/20320/,
https://review.gluster.org/#/c/20332/, and
https://review.gluster.org/#/c/20364/
Fixes glupy.py python2isms, iteritems -> items, and some overlooked
print() in georep/peer_mountbroker.in
Note: Fedora packaging guidelines and SUSE rpmlint require explicit
shebangs; popular practices like #!/usr/bin/env python and #!/usr/bin/python
are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3
Note: Selected small fixes from 2to3 utility. Specifically apply,
basestring, funcattrs, has_key, idioms, map, numliterals, raise,
set_literal, types, urllib, and zip have already been applied. Also
version agnostic imports for urllib, cpickle, socketserver, _thread,
queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1
Note: these 2to3 fixes report no changes are necessary: asserts, buffer,
exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern,
itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren,
raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw,
tuple_params, xreadlines.
Change-Id: Idda031c1ec975417c79323aea33e7b694e752b2a
updates: #411
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Geo-rep mounts are private to worker. It uses
mount namespace using unshare command to achieve
the same. Well, the unshare command has to support
'--propagation' option. So geo-rep breaks on the
systems with older unshare version. The patch
makes it fall back to lazy umount behaviour if
the unshare does not support propagation option.
fixes: bz#1589782
Change-Id: Ia614f068aede288d63ac62fea4461b1865066054
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
| |
1. Import configparser module correctly
2. Import thread module correctly
Updates: #411
Change-Id: I522453d23c256b694fa58d285f413b8c4dd6595c
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
see https://review.gluster.org/#/c/19788/,
https://review.gluster.org/#/c/19871/,
https://review.gluster.org/#/c/19952/, and
https://review.gluster.org/#/c/20104/
https://review.gluster.org/#/c/20162/
This patch changes uses of map() and raise(), and a few cases of print()
that were overlooked in the prior patch that fixed print.
Note: Fedora packaging guidelines require explicit shebangs, so popular
practices like #!/usr/bin/env python and #!/usr/bin/python are not
allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3
Note: Selected small fixes from 2to3 utility. Specifically apply,
basestring, funcattrs, idioms, numliterals, set_literal, types, urllib,
zip, map, and raise have already been applied. Also version agnostic
imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested
by Aravinda in https://review.gluster.org/#/c/19767/1
Note: these 2to3 fixes report no changes are necessary: asserts, buffer,
exec, execfile, exitfunc, filter, getcwdu, intern, itertools, metaclass,
methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce,
reload, renames, repr, standarderror, sys_exc, throw, tuple_params,
xreadlines.
Change-Id: Id62ea491e4ab5dd390075c5c6d9d889cf6f9da27
updates: #411
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
see https://review.gluster.org/#/c/19788/,
https://review.gluster.org/#/c/19871/, and
https://review.gluster.org/#/c/19952/
This patch adds version agnostic imports for urllib, cpickle,
socketserver, _thread, queue, etc., suggested by Aravinda in
https://review.gluster.org/#/c/19767/1
Note: Fedora packaging guidelines require explicit shebangs, so
popular practices like #!/usr/bin/env python and #!/usr/bin/python
are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3
Note: Selected small fixes from 2to3 utility. Specifically apply,
basestring, funcattrs, idioms, numliterals, set_literal, types, urllib,
and zip have already been applied.
Note: these 2to3 fixes report no changes are necessary: exec, execfile,
exitfunc, filter, getcwdu, intern, itertools, metaclass, methodattrs, ne,
next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr,
standarderror, sys_exc, throw, tuple_params, xreadlines.
Change-Id: I8d393064a1837874d8b4bc87c8ce05c679664642
updates: #411
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Cause and Analysis:
The last synced changelog for entry operations is
marked in current version to avoid re-processing
of already processed entry operations in a batch
during crash/restart of geo-rep. This was not
present in previous versoins.
The marker is maintained in the dictionary with the
key 'last_synced_entry' and dictionary is persisted
into status file. So upgrading to current version in
which the marker is present was failing with KeyError.
Solution:
Load the dictionary with default keys first which
contains all the keys including latest ones and then
load the values from status file instead of doing
otherwise.
fixes: bz#1575490
Change-Id: Ic654e6f9a3c97f616761f1362f890352a2186fb4
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
see https://review.gluster.org/#/c/19788/
use print fn from __future__
Change-Id: If5075d8d9ca9641058fbc71df8a52aa35804cda4
updates: #411
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
If symlink is created on master pointing
to current directory (e.g symlink -> ".") with
non root uid or gid, geo-rep worker crashes
with ENOTSUP.
Cause:
Geo-rep creates the symlink on slave and
fixes the uid and gid using chown cmd.
os.chown dereferences the symlink which is
pointing to ".gfid" which is not supported.
Note that geo-rep operates on aux-gfid-mount
(e.g. "/mnt/.gfid/<gfid-of-symlink-file>").
Solution:
The uid or gid change is acutally on symlink
file. So use os.lchown, i.e, don't deference.
BUG: 1567209
Change-Id: I63575fc589d71f987bef1d350c030987738c78ad
updates: bz#1567209
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Note 1) we're not supposed to be using #!/usr/bin/env python, see
https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines
Note 2) we're also not supposed to be using "!/usr/bin/python,
see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
The previous patch (https://review.gluster.org/19767) tried to do too
much in one patch, so it was abandoned.
This patch does two things:
1) minor cleanup of configure(.ac) to explicitly use python2
2) change all the shebang lines to #!/usr/bin/python2 and add them
where they were missing based on warnings emitted during rpmbuild.
In a follow-up patch python2 will eventually be changed to python3.
Before that python2-isms (e.g. print, string.join(), etc.) need to be
converted to python3. Some of those can be rewritten in version agnostic
python. E.g. print statements become print() with "from __future_ import
print_function". The python 2to3 utility will be used for some of those.
Also Aravinda has given guidance in the comments to the first patch for
changes.
updates: #411
Change-Id: I471730962b2526022115a1fc33629fb078b74338
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
| |
`find_library()` doesn't consider LD_LIBRARY_PATH on Python < 3.6.
Change-Id: Iee26085cb5d14061001f19f032c2664d69a378a8
BUG: 1450593
Signed-off-by: Niklas Hambüchen <mail@nh2.me>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Lazy umounting the master volume by worker causes
issues with rsync's usage of getcwd. Henc removing
the lazy umount and using private mount namespace
for the same. On the slave, the lazy umount is
retained as we can't use private namespace in non
root geo-rep setup.
Change-Id: I403375c02cb3cc7d257a5f72bbdb5118b4c8779a
BUG: 1546129
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
To append the default config file path, gsyncd calls gluster
command to get the workdir path and constructs config file path.
This is not required now since the Config management in Geo-replication
is changed with patch 18257(Issue #73)
BUG: 1539545
Change-Id: Ia7eb39e36ed59ece4de65ea7ec71a0f615e338bb
Signed-off-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
JSON output of `config-get` command now returns in the following
format
{
"name": CONFIG_NAME,
"value": CONFIG_VALUE,
"default_value": DEFAULT_VALUE, # Only if modified == true
"configurable": true|false,
"modified": true|false
}
Change-Id: I6193de48cd33655df7ecef5a0d83d7cb147089cf
Fixes: #361
Signed-off-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Once Geo-replication is started, it runs Gluster commands to get Volume
info from Master and Slave. With this patch, Georep can get Volume info
from Conf file if `--use-gconf-volinfo` argument is specified to monitor
Create a config(Or add to the config if exists) with following fields
[vars]
master-bricks=NODEID:HOSTNAME:PATH,..
slave-bricks=NODEID:HOSTNAME,..
master-volume-id=
slave-volume-id=
master-replica-count=
master-disperse_count=
Note: Exising Geo-replication is not affected since this is activated
only when `--use-gconf-volinfo` is passed while spawning `gsyncd
monitor`
Tiering support is not yet added since Tiering + Glusterd2 is still
under discussion.
Fixes: #396
Change-Id: I281baccbad03686c00f6488a8511dd6db0edc57a
Signed-off-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Geo-rep runs gverify.sh which does pre-validation.
As part of it, master and slave volume is mounted
to verify the size. If for some reason, the mount
fails, the error message does not point out the
mount log file location. Also both master and
slave mount logs are same.
Patch does following improvements.
1. Master and slave mount logs are separated and
error message points the log file to be looked for.
2. The log location is changed to /var/log/glusterfs/geo-replication
instead of /var/log/glusterfs/geo-replication-slaves
3. The log file name is changed to "gverify-mastermnt.log" and
"gverify-slavemnt.log" for master and slave mount respectively
Fixes: #395
Change-Id: Ia644ec0afebbdaae92e01adf03c635e5f8866a02
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
1. Adds validation to check if gluster binary is available on slave
2. Add a simple geo-rep setup test case to verify whether setup is fine.
It's named in such a way that it runs first.
BUG: 1532591
Change-Id: Ie777e55ae13db8fa97d4e32464ad82269ee5fd07
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch re-enables the geo-rep test cases.
Along with it does following optimizations.
1. Use EXPECT_WITHIN instead of sleep
2. Clean up geo-rep ssh key after test
3. Changes to gverify.sh and S56glusterd-geo-rep-create-post.sh
to use the given ssh identity file for geo-rep create
4. Make gluster-command-dir configurable and introduce
slave-gluster-command-dir which points the parent directory
of gluster binaries in master and slave respectively.
Change-Id: Ia7696278d9dd3ba04224dcd7c3564088ca970b04
BUG: 1480491
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
| |
BUG: 1529480
Change-Id: If4775ed9886990c0e1bcf4e44c7dfef95cc4f0c3
Signed-off-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
MD5 is not fips compliant. Hence replacing
with SHA256.
NOTE:
The hash is used to form the ctl_path for the ssh connection.
The length of ctl_path for ssh connection should not be > 108.
ssh fails with ctl_path too long if it is so. But when rsync
is piped to ssh, it is not taking > 90. rsync is failing with
error number 12. Hence using first 32 bytes of hash. Hash
collision doesn't matter as only one sock file is created
per directory.
Change-Id: I58aeb32a80b5422f6ac0188cf33fbecccbf08ae7
Updates: #230
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
| |
Fixes: #376
Change-Id: Ib92920c716c7d27e1eeb4bc4ebaf3efb48e0694d
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Fix access-mount and slave-access-mount
configs.
Change-Id: Ib586677755e76a51b9f20093e441b72789b4fecc
Signed-off-by: Kotresh HR <khiremat@redhat.com>
BUG: 1517633
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Slave gsyncd will not use session config files, Slave configs are
stored in Master config file itself and sent as argument to slave
gsyncd.
With this patch, gconf default values are overwritten if argument
name starts with "slave-"
Change-Id: Iebc51f52232c0cd30b29199f03015f97b70ce537
Signed-off-by: Aravinda VK <avishwan@redhat.com>
BUG: 1517068
|
|
|
|
|
|
|
|
|
| |
For Glusterd2 integration, JSON output of status and config
is very useful from gsyncd
Fixes: #361
Change-Id: I53c61f19033ad4ac601ea49469e4e7c7c8e9af3d
Signed-off-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Fixed Python pep8 issues
- Removed dead code
- Rewritten configuration management
- Rewritten Arguments/subcommands handling
- Added Args upgrade to accommodate all these changes without changing
glusterd code
- use of md5 removed, which was used to hash the brick path for workdir
Both Master and Slave nodes will have subdir for session in the
format "<mastervol>_<primary_slave_host>_<slavevol>
$GLUSTER_LOGDIR/geo-replication/<mastervol>_<primary_slave_host>_<slavevol>
$GLUSTER_LOGDIR/geo-replication-slaves/<mastervol>_<primary_slave_host>_<slavevol>
Log file paths renamed since session info is available with directory
name itself.
$LOG_DIR_MASTER/
- gsyncd.log - Gsyncd, Worker monitor logs
- mnt-<brick-path>.log - Aux mount logs, mounted by each worker
- changes-<brick-path>.log - Changelog related logs(One per brick)
$LOG_DIR_SLAVE/
- gsyncd.log - Slave Gsyncd logs
- mnt-<master-node>-<master-brick-path>.log - Aux mount logs,
mounted for each connection from master-node:master-brick
- mnt-mbr-<master-node>-<master-brick-path>.log - Same as above,
but mountbroker setup
Fixes: #73
Change-Id: I2ec2a21e4e2a92fd92899d026e8543725276f021
Signed-off-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
The data is not getting synced if master witnessed
IO as below.
1. echo "test_data" > f1
2. ln f1 f2
3. mv f2 f3
4. unlink f1
On master, 'f3' exists with data "test_data" but on
slave, only f3 exists with zero byte file without
backend gfid link.
Cause:
On master, since 'f2' no longer exists, the hardlink
is skipped during processing. Later, on trying to sync
rename, since source ('f2') doesn't exist, dst ('f3')
is created with same gfid. But in this use case, it
succeeds but backend gfid would not have linked as 'f1'
exists with the same gfid. So, rsync would fail with
ENOENT as backend gfid is not linked with 'f3' and 'f1'
is unlinked.
Fix:
On processing rename, if src doesn't exist on slave,
don't blindly create dst with same gfid. The gfid
needs to be checked, if it exists, hardlink needs
to be created instead of mknod.
Thanks Aravinda for helping in RCA :)
Change-Id: I5af4f99798ed1bcb297598a4bc796b701d1e0130
Signed-off-by: Kotresh HR <khiremat@redhat.com>
BUG: 1512483
Reporter: dimitri.ars@gmail.com
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In hybrid crawl, renames and unlink can't be
synced but directory renames can be detected.
While syncing the directory on slave, if the
gfid already exists, it should be rename.
Hence if directory gfid already exists, rename
it.
Change-Id: Ibf9f99e76a3e02795a3c2befd8cac48a5c365bb6
BUG: 1499566
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Passive brick's stime was not updated to the
status file immediately after updating the brick
root. As a result the last sync time was showing
'0' until it finishes first crawl if passive
worker becomes active after restart. Fix is to
update the status file immediately after upgrading
the brick root.
Change-Id: I248339497303bad20b7f5a1d42ab44a1fe6bca99
BUG: 1500346
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Worker occasionally crashed with EINTR on readlink.
This is not persistent and is transient. Worker restart
invovles re-processing of few entries in changenlogs.
So adding EINTR to retry list to avoid worker restart.
Change-Id: Iefe641437b5d5be583f079fc2a7a8443bcd19f9d
BUG: 1499393
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During xsync crawl, worker occasionally crashed
with ENODATA on getting gfid from backend. This
is not persistent and is transient. Worker restart
invovles re-processing of few entries in changenlogs.
So adding ENODATA to retry list to avoid worker
restart.
Change-Id: Ib78d1e925c0a83c78746f28f7c79792a327dfd3e
BUG: 1499391
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The status transition is as below which is
wrong.
Created->Initializing->Active->Active/Passive->Stopped
As soon as the monitor spawns the worker, the state
is changed from 'Initializing' to 'Active' and then to
'Active/Passive' based on whether worker gets the lock
or not. This is wrong and it should directly tranistion
as below.
Created->Initializing->Active/Passive->Stopped
Change-Id: Ibf5ca5c4fdf168c403c6da01db60b93f0604aae7
BUG: 1500284
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
os.listdir gives ENOTSUP on gfid path occasionally
which is not persistant. Adding it to retry list
to avoid worker to crash if it's transient error.
Change-Id: Ic795dd1f02a27c9e5d901e20722ee32451838feb
BUG: 1499180
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
If there is a hardlink to a symlink on master
and if the symlink file is deleted on master,
geo-rep fails to sync the hardlink.
Typical Usecase:
It's easily hit with rsnapshot use case where
it uses hardlinks.
Example Reproducer:
Setup geo-replication between master and slave
volume and in master mount point, do the following.
1. mkdir /tmp/symlinkbug
2. ln -f -s /does/not/exist /tmp/symlinkbug/a_symlink
3. rsync -a /tmp/symlinkbug ./
4. cp -al symlinkbug symlinkbug.0
5. ln -f -s /does/not/exist2 /tmp/symlinkbug/a_symlink
6. rsync -a /tmp/symlinkbug ./
7. cp -al symlinkbug symlinkbug.1
Cause:
If the source was not present while syncing hardlink,
it was always packing the blob as regular file.
Fix:
If the source was not present while syncing hardlink,
pack the blob based on the mode.
Change-Id: Iaa12d6f99de47b18e0650e7c4eb455f23f8390f2
BUG: 1432046
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reported-by: Christian Lohmaier <lohmaier+rhbz@gmail.com>
Reviewed-on: https://review.gluster.org/18011
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Convert the logs related to entry failures fix
due to gfid mismatch logs into structured logging
format
Change-Id: I9bce950c5339b48d3ec8b84bddee38b0473b7634
Updates: #246
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://review.gluster.org/17896
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
In a distribute replicate volume, if the hardlinks
are created when a subvolume is down, it gets
healed from other subvolume when it comes up.
If this subvolume becomes ACTIVE in geo-rep
there are chances that those hardlinks won't
be synced to slave.
Cause:
AFR can't detect hardlinks during self heal.
It just create those files using mknod and
the same is recorded in changelog. Geo-rep
processes these mknod and ignores it as
it finds gfid already on slave.
Solution:
Geo-rep should process the mknod as link
if the gfid already exists on slave.
Change-Id: I2f721b462b38a74c60e1df261662db4b99b32057
BUG: 1475308
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://review.gluster.org/17880
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Updates: #246
Change-Id: If0ce83fe8dd3068bfb671f398b2e82ac831288d0
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://review.gluster.org/17577
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
|