glusterfs.git/geo-replication/syncdaemon/gsyncdconfig.py, branch v7.1

geo-rep: Fix the name of changelog archive file

2019-11-06T16:59:14+00:00

Background:
The processed changelogs are archived each month in a single tar file.
The default format is "archive_YYYYMM.tar" which is specified as "%%Y%%m"
in configuration file.

Problem:
The created changelog archive file didn't have corresponding year
and month. It created as "archive_%Y%m.tar" on python2 only systems.

Cause and Fix:
Geo-rep expects "%Y%m" after the ConfigParser reads it from config file.
Since it was "%%Y%%m" in config file, geo-rep used to get correct value
"%Y%m" in python3 and "%%Y%%m" in python2 which is incorrect.
The fix can be to use "%Y%m" in config file but that fails in python3.
So the fix is to use "RawConfigParser" in geo-rep and use "%Y%m". This
works both in python2 and python3.

Backport of:
 > Patch: https://review.gluster.org/23248
 > Change-Id: Ie5b7d2bc04d0d53cd1769e064c2d67aaf95d557c
 > BUG: 1741890
 > Signed-off-by: Kotresh HR 

Change-Id: Ie5b7d2bc04d0d53cd1769e064c2d67aaf95d557c
fixes: bz#1764023
Signed-off-by: Kotresh HR

geo-rep: Fix Config Get Race

2019-11-06T16:59:14+00:00

When two threads(sync jobs) in Geo-rep worker calls `gconf.get` and
`gconf.getr`(realtime) at the sametime, `getr` resets the conf object
and other one gets None. Thread Lock is introduced to fix the issue.

```
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py",
  line 368, in twrap
    tf(*aargs)
  File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1987,
  in syncjob
    po = self.sync_engine(pb, self.log_err)
  File "/usr/libexec/glusterfs/python/syncdaemon/resource.py",
  line 1444, in rsync
    rconf.ssh_ctl_args + \
AttributeError: 'NoneType' object has no attribute 'split'
```

Backport of:
 > Patch: https://review.gluster.org/23158
 > Change-Id: I9c245e5c36338265354e158f5baa32b119eb2da5
 > BUG: 1737484
 > Signed-off-by: Aravinda VK 

Change-Id: I9c245e5c36338265354e158f5baa32b119eb2da5
fixes: bz#1764015
Signed-off-by: Kotresh HR

geo-rep: Test case for upgrading config file

2019-11-06T16:59:14+00:00

Added test case for the patch
https://review.gluster.org/#/c/glusterfs/+/22894/4

Also updated if else structure in gsyncdconfig.py to avoid
repeated occurance of values in new configfile.

Backport of:
 > Patch: https://review.gluster.org/22982
 > BUG: 1707731
 > Change-Id: If97e1d37ac52dbd17d47be6cb659fc5a3ccab6d7
 > Signed-off-by: Shwetha K Acharya 

fixes: bz#1764003
Change-Id: If97e1d37ac52dbd17d47be6cb659fc5a3ccab6d7
Signed-off-by: Kotresh HR

geo-rep: Upgrading config file to new version

2019-11-06T16:59:14+00:00

- configuration handling is enhanced with patch
https://review.gluster.org/#/c/glusterfs/+/18257/
- hence, the old configurations are not applied when
Geo-rep session is created in the old version and upgraded.

This patch solves the issue. It,
- checks if the config file is old.
- parses required values from old config file and stores in new
  config file, which ensures that configerations are applied on
  upgrade.
- stores old config file as backup.
- handles changes in options introduced in
  https://review.gluster.org/#/c/glusterfs/+/18257/

Backport of:
 > Patch: https://review.gluster.org/22894
 > BUG: bz#1707731
 > Change-Id: Iad8da6c1e1ae8ecf7c84dfdf8ea3ac6966d8a2a0
 > Signed-off-by: Shwetha K Acharya 

updates: bz#1764003
Change-Id: Iad8da6c1e1ae8ecf7c84dfdf8ea3ac6966d8a2a0
Signed-off-by: Kotresh HR

geo-rep: fix integer config validation

2019-03-27T14:34:31+00:00

ssh-port validation is mentioned as `validation=int` in template
`gsyncd.conf`, but not handled this during geo-rep config set.

Fixes: bz#1692666
Change-Id: I3f19d9b471b0a3327e4d094dfbefcc58ed2c34f6
Signed-off-by: Aravinda VK

geo-rep: Fix configparser import issue

2019-02-05T04:19:59+00:00

'configparser' is backported to python2 and can
be installed using pip (pip install configparser).
So trying to import 'configparser' first and later
'ConfigParser' can cause issues w.r.t unicode strings.

Always try importing 'ConfigParser' first and then
'configparser'. This solves python2/python3 compat
issues.

Change-Id: I2a87c3fc46476296b8cb547338f35723518751cc
fixes: bz#1671637
Signed-off-by: Kotresh HR

multiple: Fix python2 to python3 compat

2018-09-07T03:37:02+00:00

Change-Id: Iac241166d7a35dc7cc6cf07850f9f1bce38fe207
Updates: #411
Signed-off-by: Kotresh HR

core: switch to python3

2018-09-05T02:30:10+00:00

Commit af0d5a9b5375a5cd87ac10b429e2b9934718ce5b changes
python2 -> python3. This patch fixes a small issue introduced
by it.

Change-Id: Ib23c73683f570e8891f41476b661f37c89635fb5
updates: #411
Signed-off-by: Sanju Rakonde

All: run codespell on the code and fix issues.

2018-07-22T14:40:16+00:00

Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...

Only compile-tested!

Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul

geo-rep: Fix problems in python2 -> python3 compat

2018-06-11T06:59:44+00:00

1. Import configparser module correctly
2. Import thread module correctly

Updates: #411
Change-Id: I522453d23c256b694fa58d285f413b8c4dd6595c
Signed-off-by: Kotresh HR