glusterfs.git/tests/geo-rep.rc, branch v5.11

geo-rep: Fix syncing of files with non-ascii filenames

2018-12-26T16:50:01+00:00

Problem:
  Creation of files/directories with non-ascii names fails
  to sync to the slave. It crashes with below traceback on
  slave.
  ...
  File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/repce.py", line 118, in worker
    res = getattr(self.obj, rmeth)(*in_data[2:])
  File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/resource.py", line 709, in entry_ops
    [ESTALE, EINVAL, EBUSY])
  File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/syncdutils.py", line 546, in errno_wrap
    return call(*arg)
  File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/libcxattr.py", line 83, in lsetxattr
    cls.raise_oserr()
  File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/libcxattr.py", line 38, in raise_oserr
    raise OSError(errn, os.strerror(errn))
  OSError: [Errno 12] Cannot allocate memory

Cause:
  The length calculation arguments passed to blob creation was done before encoding. Hence
  was failing in gfid-access layer.

Fix:
  It appears that the calculating lenght properly fixes this issue. But it will cause
  issues in other places in 'python2' and not in 'python3'. So encoding and decoding
  each required string to make geo-rep compatible with both 'python2' and 'python3'
  is a nightmare and is not fool proof. Hence kept 'python2' code as is with out
  encode/decode and applied encode/decode only to 'python3'

Added non-ascii filename tests to regression

Backport of:
 > Patch: https://review.gluster.org/21668
 > BUG: 1650893
 > Change-Id: I35cfaf848e07b1a0b5cb93c01b98b472f08271a6
 > Signed-off-by: Kotresh HR 

fixes: bz#1648642
Change-Id: I35cfaf848e07b1a0b5cb93c01b98b472f08271a6
Signed-off-by: Kotresh HR

geo-rep/hook-script: Fix ssh/scp options

2018-08-03T03:18:58+00:00

Always use ssh and scp with "-oPasswordAuthentication=no"
and "-oStrictHostKeyChecking=no" options. It might hang
the post script otherwise leading geo-rep setup failure

Also increased geo-rep timeout. Occasionally, it's taking
more time to reach Active/Passive status. Especially, the
first start after create.

fixes: bz#1610405
Change-Id: I9560d64dbe0edf5db73446a9fc97dda19b88d233
Signed-off-by: Kotresh HR

tests/geo-rep: Add rsnapshot and hardlink rename test case

2018-07-13T07:36:24+00:00

1. rsnapshot creates a I/O pattern which involves
   create, rename, symlink, hardlink in specific
   order.

2. Hardlink rename on master with source unlinked use case

fixes: bz#1597540
Change-Id: Iedade47e5bf9905424a974df6ec33bc6f6695082
Signed-off-by: Kotresh HR

tests/geo-rep: Add symlink rename test case

2018-07-02T06:09:10+00:00

Added a test case of symlink rename and directory
creation with the name same as original symlink
file. Also fixed few other issues in geo-rep.rc

fixes: bz#1595726
Change-Id: I8e6acd3e742f3a0104cd37b87d1c0e0c902679b5
Signed-off-by: Kotresh HR

tests: Increase timeout for geo-rep testcases

2018-06-18T03:56:13+00:00

Change-Id: I11715c23fddc1dd218a3ab88a55f1d6355dcfd43
Updates: bz#1537602
Signed-off-by: Kotresh HR

tests: Enable geo-rep test cases

2018-01-05T07:08:10+00:00

This patch re-enables the geo-rep test cases.
Along with it does following optimizations.

1. Use EXPECT_WITHIN instead of sleep
2. Clean up geo-rep ssh key after test
3. Changes to gverify.sh and S56glusterd-geo-rep-create-post.sh
   to use the given ssh identity file for geo-rep create
4. Make gluster-command-dir configurable and introduce
   slave-gluster-command-dir which points the parent directory
   of gluster binaries in master and slave respectively.

Change-Id: Ia7696278d9dd3ba04224dcd7c3564088ca970b04
BUG: 1480491
Signed-off-by: Kotresh HR

georep: tests for logrotate, create+rename, hard-link rename

2016-08-09T07:29:35+00:00

* log rotate
  eg. with rotate count as 2 and with the following files created
  x.log, x.log.1 and x.log.2, another iteration of logrotate results in
  the following operations to be performed

  x.log.2 is renamed to x.log.3
  x.log.1 is renamed to x.log.2
  x.log is renamed to x.log.1
  x.log.3 is deleted
  x.log is created   [possible gfid allocated that belonged to x.log.3]

  when a file is created, there's a big possibility that a gfid used
  earlier can be reassigned and reused. This causes RENAME operations
  to fail on slave nodes when logs are replayed, typically on a Georep
  restart.
  A function called logrotate_simulate has been added to tests/geo-rep.rc
  to help simulate this situation. Starting from a clean state,
  logrotate_simulate has to be called 4 times with a rotate count of 2
  to help the logs roll over and cause a delete at the end and a gfid
  reallocation.
  With the bug-fix in place, this test should not cause the Georep
  session to go into a Faulty state.

* create+rename
  On log replay after Georep restart, a create+rename causes an entry
  to be created for the original file, but it cannot be linked to the
  gfid back-end since it is associated with the renamed file before
  the Georep restart.
  A function create_rename simulates the create+rename scenario and
  the function create_rename_ok tests if the dangling entry is present
  at the slave mount.
  With the bug-fix in place, a dangling entry with the original name
  should not be found at the slave mount after logs are replayed after
  a Georep restart.

* hard-link rename
  This case is similar to the create+rename case except that this is
  a case of renaming hard-link to one of its other names.
  A function hardlink_rename has been added to tests/geo-rep.rc which
  simulates the creation and rename of hard-link. After a Georep session
  restart, the test function hardlink_rename_ok should not find the
  source link name on the slave.
  With the bug-fix in place, this test should not fail.
  If changelogs have been enabled on the slave as well, then the logs
  should show an UNLINK entry for the source link name, since a rename
  operation of a hard link to one of its names essentially just drops
  the source name as per the 'mv' command semantics.

Change-Id: I85b196c00cf79a11bada25ef2fe5f1dc5c0c858a
BUG: 1316389
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/13663
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Aravinda VK

geo-rep: Fix syncing chown in xsync crawl

2015-11-09T07:09:08+00:00

GEO-REP INTEROP WITH SHARD FEATURE

Problem:
    The sequence of entry creation and chown in master
 is recorded as creation of entry with resulted
 user:group in xsync changelog. During sync, entry
 creation is always split into two ops, MKNOD and
 SETATTR. Hence the issue is not being hit otherwise
 it would have failed with EPERM if parent is owned
 by different user. But with shard translator being
 enabled on slave, doing entry creation with MKNOD and
 SETATTR is not allowed, SETATTR fails as it accesses
 inode structure which is not linked.

Solution:
    The sequence of entry creation and chown in master
 should be recorded as MKNOD and SETATTR separately always
 and do entry creation with single op in gfid-access
 xlator. The gfid-access patch will be sent separately.

Change-Id: I93e554bf9342397a7660503f5128e9709f8a0cd8
BUG: 1265148
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/12205
Tested-by: NetBSD Build System 
Reviewed-by: Aravinda VK

tests: Fix return value in geo-rep tests

2015-08-31T17:15:34+00:00

Remove the function 'data_tests' and TEST
each fop in testcase itself to determine
the exact test that fails.

Change-Id: Iffc3e2ac3b0fc0c7c64259fcdff8f27b146dcb9b
BUG: 1227624
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/11907
Tested-by: NetBSD Build System 
Reviewed-by: Jeff Darcy

tests: New simple geo-rep regression test suite

2015-08-12T06:38:06+00:00

This is a new simple regression test suite for
geo-replication. This is written keeping in mind
the run time for regression test. The existing
regression test suite is rigorous one and could
be run nightly. Hence the existing geo-rep tests
are being removed as part of this.

Also re-enable geo-rep regression with this patch.

Thanks Aravinda for initial template and plan.

Change-Id: If544ac295eaf67ac66e0b071903cc1096e71d437
BUG: 1227624
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/11058
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Aravinda VK