glusterfs.git/geo-replication/syncdaemon/libcxattr.py, branch v7.1

geo-rep: Fix syncing of files with non-ascii filenames

2018-12-04T09:15:44+00:00

Problem:
Creation of files/directories with non-ascii names fails
to sync to the slave. It crashes with below traceback on
slave.
...
File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/repce.py", line 118, in worker
res = getattr(self.obj, rmeth)(*in_data[2:])
File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/resource.py", line 709, in entry_ops
[ESTALE, EINVAL, EBUSY])
File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/syncdutils.py", line 546, in errno_wrap
return call(*arg)
File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/libcxattr.py", line 83, in lsetxattr
cls.raise_oserr()
File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/libcxattr.py", line 38, in raise_oserr
raise OSError(errn, os.strerror(errn))
OSError: [Errno 12] Cannot allocate memory

Cause:
The length calculation arguments passed to blob creation was done before encoding. Hence
was failing in gfid-access layer.

Fix:
It appears that the calculating lenght properly fixes this issue. But it will cause
issues in other places in 'python2' and not in 'python3'. So encoding and decoding
each required string to make geo-rep compatible with both 'python2' and 'python3'
is a nightmare and is not fool proof. Hence kept 'python2' code as is with out
encode/decode and applied encode/decode only to 'python3'

Added non-ascii filename tests to regression

fixes: bz#1650893
Change-Id: I35cfaf848e07b1a0b5cb93c01b98b472f08271a6
Signed-off-by: Kotresh HR

georep: python2 to python3 compat - syscalls

2018-10-08T16:36:04+00:00

1. ctypes/syscalls
    A) arguments is expected to be encoded
    B) Raw conversion of return value from bytearray into string
2. struct pack/unpack - Raw converstion of string to bytearray
3. basestring -> str

Updates: #411
Change-Id: I80f939adcdec0ed0022c87c0b76d057ad5559e5a
Signed-off-by: Kotresh HR

All: run codespell on the code and fix issues.

2018-07-22T14:40:16+00:00

Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...

Only compile-tested!

Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul

python: Remove all uses of find_library. Fixes #1450593

2018-03-24T05:10:31+00:00

`find_library()` doesn't consider LD_LIBRARY_PATH on Python < 3.6.

Change-Id: Iee26085cb5d14061001f19f032c2664d69a378a8
BUG: 1450593
Signed-off-by: Niklas Hambüchen

geo-rep: fix wrong argument to find_library

2016-07-07T10:49:56+00:00

Argument passed to find_library function is wrong.
Pass proper argument to find_library.

Note: It still works as default handle is libc.

Change-Id: I4657a3ef9f5249e3eec4a20582856b58168126ce
BUG: 1352423
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/14852
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Handle ERANGE error during listxattr

2016-02-26T10:12:16+00:00

llistxattr in Geo-rep is two syscall instead of one

SIZE = llistxattr(PATH, &BUF, 0);
BUF = create_buf(SIZE);
_ = llistxattr(PATH, &BUF, SIZE);

So if any new xattrs added just after first call by any other worker,
second syscall will fail with ERANGE error.

Now Geo-rep sends BUF with large size(256*100) and gets value
with only one syscall. Raises OSError if fails with ERANGE error
even after sending large BUF.

Change-Id: I8ade4bbe9a0a8ea908ed9dedcd3f2ff4c6fe6f51
Signed-off-by: Aravinda VK 
BUG: 1294588
Reviewed-on: http://review.gluster.org/13106
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Fix portability issues with NetBSD

2015-09-07T09:52:46+00:00

Fixes portability issues in gverify.sh and
libcxattr.py with NetBSD.

Change-Id: Idfaa6cf3815136e6a2343aab98d979b6ab451bbd
BUG: 1257847
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/12088
Reviewed-by: Emmanuel Dreyfus 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Venky Shankar

geo-rep: entry_ops errors handling

2014-06-12T06:52:52+00:00

Xattr.lsetxattr_l call will not raise OSError which
errno_wrap can handle, so we need to use Xattr.lsetxattr
to get proper OSError and errno_wrap ignores or retries
accordingly.

Change-Id: Ie0a777152ddbaf9ed80c977e4704974fec997bea
BUG: 1105083
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7972
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: code pep8/flake8 fixes

2014-04-08T04:56:55+00:00

pep8 is a style guide for python.
http://legacy.python.org/dev/peps/pep-0008/

pep8 can be installed using, `pip install pep8`

Usage: `pep8 `, For example, `pep8 master.py`
will display all the coding standard errors.

flake8 is used to identify unused imports and other issues
in code.

pip install flake8

cd $GLUSTER_REPO/geo-replication/
flake8 syncdaemon

Updated license headers to each source file.

Change-Id: I01c7d0a6091d21bfa48720e9fb5624b77fa3db4a
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7311
Reviewed-by: Kotresh HR 
Reviewed-by: Prashanth Pai 
Tested-by: Gluster Build System

gsyncd: distribute the crawling load

2013-07-26T20:18:57+00:00

* also consume changelog for change detection.
* Status fixes
* Use new libgfchangelog done API
* process (and sync) one changelog at a time

Change-Id: I24891615bb762e0741b1819ddfdef8802326cb16
BUG: 847839
Original Author: Csaba Henk 
Original Author: Aravinda VK 
Original Author: Venky Shankar 
Original Author: Amar Tumballi 
Original Author: Avra Sengupta 
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/5131
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur