| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
| |
Change-Id: I526bf9bfd889dd7aea19f71059042cd9a993e1d0
Signed-off-by: ShyamsundarR <srangana@redhat.com>
Fixes: bz#1640685
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As lookup is not a locked fop, we can not trust the
data received in this to be same.
Changing the log level to DEBUG in case lookup finds any
difference.
(cherry picked from commit 9be6bf3d90e3783b3ba559c93d41b933f8d53f03)
Change-Id: I39499c44688a2455c7c6c69a798762d045d21b39
updates: bz#1644622
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
lease_id is a 16 bits opaque data, copying it by gf_strdup is wrong.
Invalid read of size 2
at 0x483FA2F: memmove (vg_replace_strmem.c:1270)
by 0xE2EF6FB: ??? (in /usr/lib64/libtirpc.so.3.0.0)
by 0xE2EE047: xdr_opaque (in /usr/lib64/libtirpc.so.3.0.0)
by 0x107A97DC: xdr_gfx_value (glusterfs4-xdr.c:207)
by 0x107A98C0: xdr_gfx_dict_pair (glusterfs4-xdr.c:321)
by 0xE2EF35E: xdr_array (in /usr/lib64/libtirpc.so.3.0.0)
by 0x107A9A89: xdr_gfx_dict (glusterfs4-xdr.c:335)
by 0x107AA97B: xdr_gfx_write_req (glusterfs4-xdr.c:897)
by 0x107A181E: xdr_serialize_generic (xdr-generic.c:25)
by 0x231044A2: client_submit_request (client.c:205)
by 0x2314D3C1: client4_0_writev (client-rpc-fops_v2.c:3863)
by 0x230FD5FA: client_writev (client.c:956)
Address 0xad659e18 is 72 bytes inside a block of size 73 alloc'd
at 0x483880B: malloc (vg_replace_malloc.c:299)
by 0x106BA7EC: __gf_malloc (mem-pool.c:136)
by 0x1064521E: gf_strndup (mem-pool.h:166)
by 0x1064521E: gf_strdup (mem-pool.h:183)
by 0x1064521E: get_fop_attr_thrd_key (glfs.c:627)
by 0x1064D8E9: glfs_pwritev@@GFAPI_3.4.0 (glfs-fops.c:1154)
by 0x10610C0C: glusterfs_write2 (handle.c:2092)
by 0x54D30C: mdcache_write2 (mdcache_file.c:647)
by 0x48A3FC: nfs4_write (nfs4_op_write.c:459)
by 0x48A44D: nfs4_op_write (nfs4_op_write.c:487)
by 0x4634F5: nfs4_Compound (nfs4_Compound.c:947)
by 0x460155: nfs_rpc_process_request (nfs_worker_thread.c:1329)
by 0x4608A3: nfs_rpc_valid_NFS (nfs_worker_thread.c:1539)
by 0x488F12F: svc_vc_decode (svc_vc.c:825)
Backport of:
> Patch: https://review.gluster.org/21586/
> BUG: bz#1647651
> Change-Id: Ib9fff55c897bc43c15036a869888e763df133757
> Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
(cherry picked from commit 6d4cd8ce6c0d88d331ffed97c51d3061a3900561)
Updates bz#1648923
Change-Id: Ib9fff55c897bc43c15036a869888e763df133757
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Geo-rep's automatic error handling does gfid conflict
resolution. But if there are ENOENT errors because the
parent is not synced to slave, it doesn' handle them.
This patch adds the intelligence to create missing
parent directories on slave. It can create the missing
directories upto the depth of 10.
Backport of:
> Patch: https://review.gluster.org/21498/
> fixes: bz#1643402
> Change-Id: Ic97ed1fa5899c087e404d559e04f7963ed7bb54c
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 19775e0445411cca9ddd9d294fd54d0b6fbe6a03)
fixes: bz#1646896
Change-Id: Ic97ed1fa5899c087e404d559e04f7963ed7bb54c
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the commit febf5ed4848, during the volume create op,
we are setting volinfo->caps to 0, only if any of the bricks
belong to the same node and brickinfo->vg[0] is null.
Previously, we used to set volinfo->caps to 0, when
either brick doesn't belong to the same node or brickinfo->vg[0]
is null.
With this patch, we set volinfo->caps to 0, when either brick
doesn't belong to the same node or brickinfo->vg[0] is null.
(as we do earlier without commit febf5ed4848).
> BUG: bz#1635820
> Change-Id: I00a97415786b775fb088ac45566ad52b402f1a49
> Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
(cherry picked from commit aae1c402b74fd02ed2f6473b896f108d82aef8e3)
fixes: bz#1647968
Change-Id: I00a97415786b775fb088ac45566ad52b402f1a49
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, there are possibilities in few places, where a user-controlled
(like filename, program parameter etc) string can be passed as 'fmt' for
printf(), which can lead to segfault, if the user's string contains '%s',
'%d' in it.
While fixing it, makes sense to make the explicit check for such issues
across the codebase, by making the format call properly.
Fixes: CVE-2018-14661
Fixes: bz#1647666
Change-Id: Ib547293f2d9eb618594cbff0df3b9c800e88bde4
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Fixes CID 1396581
Change-Id: Ic04091b5783a75d8e1e605a9c1c28b77fea048d3
updates: bz#1647962
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Signed-off-by: Susant Palai <spalai@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In the current scheme of glusterfs where lock migration is
experimental, (ideally) the rebalance process which is migrating
the file should request for a metalock. Hence, the metalock count
should not be more than one for an inode. In future, if there is a
need for meta-lock from other clients, this patch can be reverted.
Since pl_metalk is called as part of setxattr operation, any client
process(non-rebalance) residing outside trusted network can exhaust
memory of the server node by issuing setxattr repetitively on the
metalock key. The current patch makes sure that more than
one metalock cannot be granted on an inode.
Fixes CVE-2018-14660
updates: bz#1647962
Change-Id: Ie1e697766388718804a9551bc58351808fe71069
Signed-off-by: Susant Palai <spalai@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Server stack needs to have all the sort of validation, assuming
clients can be compromized. It is possible for a compromized
client to send basenames with paths with '/', and with that
create files without permission on server. By sanitizing the basename,
and not allowing anything other than actual directory as the parent
for any entry creation, we can mitigate the effects of clients
not able to exploit the server.
Fixes: CVE-2018-14651
Fixes: bz#1647663
Change-Id: I5dc0da0da2713452ff2b65ac2ddbccf1a267dc20
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Currently for replica volume, even if only one brick is UP
SHD will keep crawling index entries even if it can not
heal anything.
In thin-arbiter volume which is also a replica 2 volume,
this causes inode lock contention which in turn sends
upcall to all the clients to release notify locks, even
if it can not do anything for healing.
This will slow down the client performance and kills the
purpose of keeping in memory information about bad brick.
Solution: Before starting heal or even crawling, check if
sufficient number of children are UP and available to check
and heal entries.
(cherry picked from commit f73b4476b15f9d6d3dc3c8e20c9742aacd857f9f)
Change-Id: I011c9da3b37cae275f791affd56b8f1c1ac9255d
updates: bz#1644645
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With commit 44e4db, we are not allowing user to create a volume
using glusterd's working directory as a brick or any sub directory
under glusterd's working directory as a brick.This has broken
shared-storage since the volume "gluster-shared-storage" is
created using the bricks under glusterd's working directory.
With this patch, we let the "gluster-shared-storage" volume
to use bricks under glusterd's working directory.
> BUG: bz#1647029
> Change-Id: Ifcbcf4576eea12cf46f199dea287b29bd3ec3bfd
> Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
(cherry picked from commit bdb4ca184913c82ccf9552298f5d5b597794f2aa)
fixes: bz#1647801
Change-Id: Ifcbcf4576eea12cf46f199dea287b29bd3ec3bfd
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
By allowing clients taking dump in a file on brick process, we are
allowing compromised clients to create io-stats dumps on server,
which can exhaust all the available inodes.
Fixes: CVE-2018-14659
Fixes: bz#1647665
Change-Id: I32bfde9d4fe646d819a45e627805b928cae2e1ca
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
as key size in xdr can be anything, it can be bigger than the
'NAME_MAX' allowed in the structure, which can allow for service denial
attacks.
Fixes: CVE-2018-14653
Fixes: bz#1647664
Change-Id: I2dc5e99af27ddf44c12c94b07e51adb8674cce80
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
'getspec' operation is not used between 'client' and 'server' ever since
we have off-loaded volfile management to glusterd, ie, at least 7 years.
No reason to keep the dead code! The removed option had no meaning,
as glusterd didn't provide a way to set (or unset) this option. So,
no regression should be observed from any of the existing glusterfs
deployment, supported or unsupported.
Updates: CVE-2018-14653
Updates: bz#1647664
Change-Id: I4a2e0f673c5bcd4644976a61dbd2d37003a428eb
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Patch https://review.gluster.org/#/c/glusterfs/+/19135/ has
optimised glusterd test cases by clubbing the similar test
cases into a single test case.
https://review.gluster.org/#/c/glusterfs/+/19135/15/tests/bugs/glusterd/bug-1293414-import-brickinfo-uuid.t
test case has been deleted and added as a part of
tests/bugs/glusterd/optimized-basic-testcases-in-cluster.t
In the original test case, we create a volume with two bricks,
each on a separate node(N1 & N2). From another node in cluster(N3),
we try to detach a node which is hosting bricks. It fails.
In the new test, we created volume with single brick on N1.
and from another node in cluster, we tried to detach N1. we
expect peer detach to fail, but peer detach was success as
the node is hosting all the bricks of volume.
Now, changing the new test case to cover the original test case scenario.
Please refer https://bugzilla.redhat.com/show_bug.cgi?id=1642597#c1 to
understand why the new test case is not failing in centos-regression.
> BUG: bz#1642597
> Change-Id: Ifda12b5677143095f263fbb97a6808573f513234
> Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
(cherry picked from commit 0ca6773eaf5aeb507ebc72d2c2f61902eeff414c)
fixes: bz#1643078
Change-Id: Ifda12b5677143095f263fbb97a6808573f513234
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When 'gluster-mountbroker status' was issued, it
crashes in a corner case with 'str object has not
attribute get'. Fixed the same.
Backport of:
> BUG: 1643929
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
> Change-Id: Iaf1a937ed0136b3b2058230c75fa89a215d8a5eb
(cherry picked from commit 5987b3388126a3c5e77481913cbaa4142117d19a)
fixes: bz#1644515
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Change-Id: Iaf1a937ed0136b3b2058230c75fa89a215d8a5eb
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
posix_update_utime_in_mdata() unconditionally logs an error if
consistent time attributes features is not enabled. This log
does not add any value, prints an incorrect errno & floods
the log file. Hence nuking this log message in this patch.
Backport of:
> Patch: https://review.gluster.org/21520/
> BUG: 1644129
> Change-Id: I9a1f9e7ada3366d2830f18d81f16a1461040092e
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
fixes: bz#1644526
Change-Id: I9a1f9e7ada3366d2830f18d81f16a1461040092e
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1. scheduler - Popen
2. syncdutils - corner case on failure
Backport of:
> Patch: https://review.gluster.org/21505
> BUG: 1643932
> Change-Id: I65af97a244a8790e976acedc2728db6ebbf2ae10
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 33e96100e17e9a293db6d63d9d5449d6c2d69376)
fixes: bz#1644514
Change-Id: I65af97a244a8790e976acedc2728db6ebbf2ae10
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
lvm2(-devel) 2.03.00 no longer has liblvm2app.so. (I expect a
similar change in fedora-30 before too much longer, but for
now fedora-30 still has lvm2 and lvm2-devel 2.02.181
rpcgen has been removed from glibc-common and unbundled rpcgen
is now required.
And I guess nobody has ever built rpms with '--without bd' or we
would have discovered the attempted inclusion of .../storage/bd.so
in the rpm when it hadn't actually been built.
Change-Id: I71e26c3d06af5d329ae89cc249a4ad88664ddf53
updates: bz#1644314
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
During gfid-conflict-resolution, geo-rep crashes
with 'ValueError: list.remove(x): x not in list'
Cause and Analysis:
During gfid-conflict-resolution, the entry blob is
passed back to master along with additional
information to verify it's integrity. If everything
looks fine, the entry creation is ignored and is
deleted from the original list. But it is crashing
during removal of entry from the list saying entry
not in list. The reason is that the stat information
in the entry blob was modified and sent back to
master if present.
Fix:
Send back the correct stat information for
gfid-conflict-resolution.
Backport of:
> BUG: bz#1642865
> Change-Id: I47a6aa60b2a495465aa9314eebcb4085f0b1c4fd
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit ff18121945bff394f3234e9f1a9d61ac97d4d493)
fixes: bz#1644158
Change-Id: I47a6aa60b2a495465aa9314eebcb4085f0b1c4fd
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Make Popen py2 and py3 compatiable
Backport of:
> BUG: 1643935
> Change-Id: Ife34cb38024dcdc0420436e7d76fd208223f9d86
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit bae584148761ad98cd3d5c380f8cea1ff83aa8c3)
fixes: bz#1644161
Change-Id: Ife34cb38024dcdc0420436e7d76fd208223f9d86
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
For lease operation, we allocate and store child nodes
data in lease structure. Use the same in afr_lease_cbk()
while checking for the quorum.
Change-Id: If1fdd5a0798888afd39ad3df57d96487baf9d1e6
updates: #350
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Patch in master: https://review.gluster.org/#/c/glusterfs/+/21534/
Problem:
A compromised client can set arbitrary values for the GF_XATTROP_ENTRY_IN_KEY
and GF_XATTROP_ENTRY_OUT_KEY during xattrop fop. These values are
consumed by index as a filename to be created/deleted according to the key.
Thus it is possible to create/delete random files even outside the gluster
volume boundary.
Fix:
Index expects the filename to be a basename, i.e. it must not contain any
pathname components like "/" or "../". Enforce this.
Fixes: CVE-2018-14654
Fixes: bz#1646204
Change-Id: I35f2a39257b5917d17283d0a4f575b92f783f143
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Glusterfs leases expects lease_id to be set and sent
for each fop to determine conflict resolution with the
existing lease.
Incase if not set (most likely if there is an older
client in a mixed cluster), it makes sense to consider
it as conflicitng fop and recall the lease.
Also fixed the return status check for __remove_lease(),
wherein non-negative value is considered as success case.
Change-Id: I5bcfba4f7c71a5af7cdedeb03436d0b818e85783
updates: #350
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit cf5b13896d65b6916634976a3a5f61ddeefbc19c)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
https://review.gluster.org/#/c/glusterfs/+/21427/ seems to be failing
this .t spuriously. On checking one of the failure logs, I see:
22:05:44 Launching heal operation to perform index self heal on volume patchy has been unsuccessful:
22:05:44 Self-heal daemon is not running. Check self-heal daemon log file.
22:05:44 not ok 20 , LINENUM:38
In glusterd log:
[2018-10-18 22:05:44.298832] E [MSGID: 106301] [glusterd-syncop.c:1352:gd_stage_op_phase] 0-management: Staging of operation 'Volume Heal' failed on localhost : Self-heal daemon is not running. Check self-heal daemon log file
But the tests which preceed this check whether via a statedump if the shd is
conected to the bricks, and they have succeeded and even started
healing. From glustershd.log:
[2018-10-18 22:05:40.975268] I [MSGID: 108026] [afr-self-heal-common.c:1732:afr_log_selfheal] 0-patchy-replicate-0: Completed data selfheal on 3b83d2dd-4cf2-4ea3-a33e-4275be40f440. sources=[0] 1 sinks=2
So the only reason I can see launching heal via cli failing is a race where
shd has been spawned but glusterd has not yet updated in-memory that it is up,
and hence failing the CLI.
Fix:
Check for shd up status before launching heal via CLI
Change-Id: Ic88abf14ad3d51c89cb438db601fae4df179e8f4
fixes: bz#1641872
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
(cherry picked from commit 3dea105556130abd4da0fd3f8f2c523ac52398d1)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of:
> Change-Id: Ic15ca41444dd04684a9458bd4a526b1d3e160499
> Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
> (cherry picked from commit e627977)
> BUG: 1605056
In __shard_update_shards_inode_list(), previously shard translator
was not holding a ref on the base inode whenever a shard was added to
the lru list. But if the base shard is forgotten and destroyed either
by fuse due to memory pressure or due to the file being deleted at some
point by a different client with this client still containing stale
shards in its lru list, the client would crash at the time of locking
lru_base_inode->lock owing to illegal memory access.
So now the base shard is ref'd into the inode ctx of every shard that
is added to lru list until it gets lru'd out.
The patch also handles the case where none of the shards associated
with a file that is about to be deleted are part of the LRU list and
where an unlink at the beginning of the operation destroys the base
inode (because there are no refkeepers) and hence all of the shards
that are about to be deleted will be resolved without the existence
of a base shard in-memory. This, if not handled properly, could lead
to a crash.
Change-Id: Ic15ca41444dd04684a9458bd4a526b1d3e160499
updates: bz#1641440
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of:
> Change-Id: I84a5e54d214b6c47ed85671a880bb1c767a29f4d
> Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
> (cherry picked from commit 15c9976)
> BUG: 1638453
PROBLEM:
tests/bugs/shard/bug-1251824.t fails occasionally with EIO due to gfid
mismatch across replicas on the same shard when dd is executed.
CAUSE:
Turns out this is due to a race between posix_mknod() and posix_lookup().
posix mknod does 3 operations, among other things:
1. creation of the entry itself under its parent directory
2. setting the gfid xattr on the file, and
3. creating the gfid link under .glusterfs.
Consider a case where the thread doing posix_mknod() (initiated by shard)
has executed steps 1 and 2 and is on its way to executing 3. And a
parallel LOOKUP from another thread on noting that loc->inode->gfid is NULL,
tries to perform gfid_heal where it attempts to create the gfid link
under .glusterfs and succeeds. As a result, posix_gfid_set() through
MKNOD (step 3) fails with EEXIST.
In the older code, MKNOD under such conditions was NOT being treated
as a failure. But commit e37ee6d changes this behavior by failing MKNOD,
causing the entry creation to be undone in posix_mknod() (it's another
matter that the stale gfid handle gets left behind if lookup has gone
ahead and gfid-healed it).
All of this happens on only one replica while on the other MKNOD succeeds.
Now if a parallel write causes shard translator to send another MKNOD
of the same shard (shortly after AFR releases entrylk from the first
MKNOD), the file is created on the other replica too, although with a
new gfid (since "gfid-req" that is passed now is a new UUID. This leads
to a gfid-mismatch across the replicas.
FIX:
The solution is to not fail MKNOD (or any other entry fop for that matter
that does posix_gfid_set()) if the .glusterfs link creation fails with EEXIST.
Change-Id: I84a5e54d214b6c47ed85671a880bb1c767a29f4d
fixes: bz#1641429
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
|
|
|
|
|
|
|
|
|
| |
- Added missing options
- Added bugs fixed
Change-Id: I3b788a093bc00fb977f792ce535ed50cc0cd9c9e
Updates: bz#1628620
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes below issues in gfapi lease code-path
* 'glfs_setfsleasid' should allow NULL input to be
able to reset leaseid
* Applications should be allowed to (un)register for
upcall notifications of type GLFS_EVENT_LEASE_RECALL
* APIs added to read contents of GLFS_EVENT_LEASE_RECALL
argument which is of type "struct glfs_upcall_lease"
This is backport of the below mainline patch -
https://review.gluster.org/#/c/glusterfs/+/21391
Change-Id: I3320ddf235cc82fad561e13b9457ebd64db6c76b
updates: #350
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The #include "uuid.h" left over from using .../contrib/uuid is debatably
incorrect now that we use the "system header" file /usr/include/uuid/uuid.h
from libuuid-devel.
Unfortunately this is complicated by things like FreeBSD having its own
/usr/include/uuid.h, and the e2fsprogs-libuuid uuid.h in installed - as
most third-party packages in FreeBSD are - in /usr/local as
/usr/local/include/uuid/uuid.h
With a system header file it should at least be #include <uuid.h>, and
even better as #include <uuid/uuid.h>, much like the way <sys/types.h>
and <net/if.h> are included. Using #include <uuid/uuid.h> guarantees
not getting the /usr/include/uuid.h on FreeBSD, but clang/cc knows to
find "system" header files like this in /usr/local/include; with or
without the -I/... from uuid.pc. Also using #include "uuid.h" leaves
the compiler free to find a uuid.h from any -I option it might be passed.
(Fortunately we don't have any at this time.)
As we now require libuuid-devel or e2fsprogs-libuuid and configure will
exit with an error if the uuid.pc file doesn't exist, the HAVE_LIBUUID
(including the #elif FreeBSD) tests in compat-uuid.h are redundant. We
are guaranteed to have it, so testing for it is a bit silly IMO. It may
also break building third party configure scripts if they omit defining
it. (Just how hard do we want to make things for third party developers?)
Change-Id: I7317f63c806281a5d27de7d3b2208d86965545e1
updates: bz#1639688
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As dict_unserialize does not null terminate the value,
using snprintf adds garbage characters to the buffer
used to create the filename.
The code also used this->name in the filename which
will be the same for all bricks for a volume. The
files were thus overwritten if a node contained
multiple bricks for a volume. The code now uses
the conf->unique instead if available.
Change-Id: I2c72534b32634b87961d3b3f7d53c5f2ca2c068c
fixes: bz#1640392
Signed-off-by: N Balachandran <nbalacha@redhat.com>
(cherry picked from commit 219cd649fdbd7bfd6c2268a0a4f66bcc15918e31)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
translators like readdir-ahead selectively retain entry information of
iatt (gfid and type) when rest of the iatt is invalidated (for write
invalidating ia_size, (m)(c)times etc). Fuse-bridge uses this
information and sends only entry information in readdirplus
response. However such option doesn't exist in gfapi. This patch
modifies gfapi to populate the stat by forcing an extra lookup.
Thanks to Shyamsundar Ranganathan <srangana@redhat.com> and Prashanth
Pai <ppai@redhat.com> for tests.
Change-Id: Ieb5f8fc76359c327627b7d8420aaf20810e53000
Fixes: bz#1630804
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 6257276d9de3f15643f159b2ec627a67c84fc23d)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of https://review.gluster.org/#/c/glusterfs/+/21135/
Problem:
When a directory has dirty xattrs due to failed post-ops or when
replace/reset brick is performed, AFR does a conservative merge as
expected, but heal-info reports it as split-brain because there are no
clear sources.
Fix:
Modify pending flag to contain information about pending heals and
split-brains. For directories, if spit-brain flag is not set,just show
them as needing heal and not being in split-brain.
Change-Id: I09ef821f6887c87d315ae99e6b1de05103cd9383
fixes: bz#1638163
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of https://review.gluster.org/#/c/glusterfs/+/21380/
Problem:
In an arbiter volume, if there is a pending data heal of a file only on
arbiter brick, self-heal takes inodelks twice due to a code-bug but unlocks
it only once, leaving behind a stale lock on the brick. This causes
the next write to the file to hang.
Fix:
Fix the code-bug to take lock only once. This bug was introduced master
with commit eb472d82a083883335bc494b87ea175ac43471ff
Thanks to Pranith Kumar K <pkarampu@redhat.com> for finding the RCA.
fixes: bz#1638159
Change-Id: I15ad969e10a6a3c4bd255e2948b6be6dcddc61e1
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes import issue in cliutils.
Provided solution is to use relative import.
Change-Id: I14c9a0b528ef52e7c91f6b17b569c68c2ced8912
updates: #411
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
(cherry picked from commit 8d4c5e022bba1b99786ce13f407c27024beccc23)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1. ctypes/syscalls
A) arguments is expected to be encoded
B) Raw conversion of return value from bytearray into string
2. struct pack/unpack - Raw converstion of string to bytearray
3. basestring -> str
Updates: #411
Change-Id: I80f939adcdec0ed0022c87c0b76d057ad5559e5a
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit fb6e8d0d0ca21b16d331fa69da9b9dadf6c5c35d)
|
|
|
|
|
|
|
|
|
|
|
| |
1. Fix imports
2. Use universal_newlines flag in Popen
3. encode msg used in 'sendto'
Updates: #411
Change-Id: Ieacdc2ba86953ff3a2e9d9b9a97157e8e0e36836
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit b5f75e337c19d4fed85d4dcf15cf47f02fea5f6a)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
No default value was specified for `export-statfs-size` in posix
option table. Glusterd2 sets default value as `off` since the
option type is `bool`. Posix treats `export-statfs-size=on` if
not specified in volfile(That means default value is `on`)
This patch sets default value as `on`
> Change-Id: I5c6341183be9b62a78fdbc94621220f9284e1382
> updates: #302
> Signed-off-by: Aravinda VK <avishwan@redhat.com>
(cherry picked from commit 07088d95e450f847722e5decbfa5da18a0dbd9de)
Change-Id: Ib6b3accdb9921376c16040bd2312b99b0226a26f
Fixes: bz#1636842
Signed-off-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
| |
Change-Id: Id2ed7e04fc7fe708a144ac09c69b837956aedb94
Updates: bz#1628620
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For both Virt and block workloads the file is opened multiple times
leading to dynamically setting eager-lock to off for the workload.
Instead of depending on the number-of-open-fds, if we change the
logic to depend on number of inodelks, then it will give better
performance than the earlier logic. When there is an eager-lock
and number of inodelks is more than 1 we know that there is a
conflicting lock, so depend on that information to decide whether
to keep the current transaction go through delayed-post-op or not.
Locks xlator doesn't have implementation to query number of locks in
fxattrop in releases older than 3.10 so to keep things backward
compatible in 3.12, data transactions will use new logic where as
fxattrop transactions will use old logic. I am planning to send one
more patch which makes metadata domain locks also depend on
inodelk-count
Profile info for a dd of 500MB to a file with another fd opened
on the file using exec 250>filename
Without this patch:
0.14 67.41 us 16.72 us 3870.82 us 892 FINODELK
0.59 279.87 us 95.71 us 2085.89 us 898 FXATTROP
3.46 366.43 us 81.75 us 6952.79 us 4000 WRITE
95.79 148733.99 us 50568.12 us 919127.86 us 273 FSYNC
With this patch:
0.00 51.01 us 38.07 us 80.16 us 4 FINODELK
0.00 235.43 us 235.43 us 235.43 us 1 TRUNCATE
0.00 125.07 us 56.80 us 193.33 us 2 GETXATTR
0.00 135.86 us 62.13 us 209.59 us 2 INODELK
0.00 197.88 us 155.39 us 253.90 us 4 FXATTROP
0.00 450.59 us 394.28 us 506.89 us 2 XATTROP
0.00 56.96 us 19.06 us 406.59 us 23 FLUSH
37.81 273648.93 us 48.43 us 6017657.05 us 44 LOOKUP
62.18 4951.86 us 93.80 us 1143154.75 us 3999 WRITE
postgresql benchmark performance changed from ~1130 TPS to ~2300TPS
randio fio job inside Ovirt based VM went from ~600IOPs to ~2000IOPS
fixes bz#1635972
Change-Id: If7f7388d2f08cf7f17ca517a4ea222560661dc36
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
When eager-lock is disabled because of multiple-fds opened and app
writes come on conflicting regions, the number of locks grows very
fast leading to all the CPU being spent just in locking and unlocking
by traversing huge queues in locks xlator for granting locks.
Fix:
Reduce the number of locks in transit by bundling the writes in the
same lock and disable delayed piggy-pack when we learn that multiple
fds are open on the file. This will reduce the size of queues in the
locks xlator. This also reduces the number of network calls like
inodelk/fxattrop.
Please note that this problem can still happen if eager-lock is
disabled as the writes will not be bundled in the same lock.
fixes bz#1635975
Change-Id: I8fd1cf229aed54ce5abd4e6226351a039924dd91
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Till now, glusterd was generating the volfile path for the snapshot
volume's bricks like this.
/snaps/<snap name>/<brick volfile>
But in reality, the path to the brick volfile for a snapshot volume is
/snaps/<snap name>/<snap volume name>/<brick volfile>
The above workaround was used to distinguish between a mount command used
to mount the snapshot volume, and a brick of the snapshot volume, so that
based on what is actually happening, glusterd can return the proper volfile
(client volfile for the former and the brick volfile for the latter). But,
this was causing problems for snapshot restore when brick multiplexing is
enabled. Because, with brick multiplexing, it tries to find the volfile
and sends GETSPEC rpc call to glusterd using the 2nd style of path i.e.
/snaps/<snap name>/<snap volume name>/<brick volfile>
So, when the snapshot brick (which is multiplexed) sends a GETSPEC rpc
request to glusterd for obtaining the brick volume file, glusterd was
returning the client volume file of the snapshot volume instead of the
brick volume file.
Change-Id: I28b2dfa5d9b379fe943db92c2fdfea879a6a594e
fixes: bz#1636162
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
(cherry picked from commit 83a89296a3d12a3fc2a643c0630be5ce659204ea)
|
|
|
|
|
|
|
|
|
|
| |
'%' needs special handling in config and also removed
duplicate misspelled 'changelog-archive-format' config.
Updates: #411
Change-Id: I33621a62bdf5f781ee62e6cedec0c2df3f5d70cf
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 62164ac0d242a271f19b0e0bb352af5f7df082ce)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The file objects for python3 by default is opened
in binary mode where as in python2 it's opened
as text by default.
The geo-rep code parses the output of Popen assuming
it as text, hence used the 'universal_newlines' flag
which provides backward compatibility for the same.
Change-Id: I371a03b6348af9666164cb2e8b93d47475431ad9
Updates: #411
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 65aed1070cc2e44959cf3a0fbfde635de7e03103)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
'os.pipe' returns pair of file descriptors
which are non-inheritable by child processes.
But geo-rep uses te inheritable nature of
pipe fds to communicate between parent and
child processes. Hence wrote a compatiable
pipe routine which works well both with python2
and python3 with inheritable nature.
Updates: #411
Change-Id: I869d7a52eeecdecf3851d44ed400e69b32a612d9
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 173e89a6506bc8c727ce6d8e5ac84b59ad2e21de)
|
|
|
|
|
|
|
|
|
| |
1. Fix fdopen used for pid file
2. Fix sha256 checksum calculation
Updates: #411
Change-Id: Ic173d104a73822c29aca260ba6de872cd8d23f86
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
python3 expects byte string for os.write. This works
for both py2 and py3. Fixed the same for geo-rep
mount testing code path.
Updates: #411
Change-Id: I2dfedcb0869457707bcca4d2847ef0d52bff1987
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Handle py2 and py3 compatibility for pickling and unpickling.
Geo-rep pickles and unpickles reading sys.stdin and sys.stdout streams.
py2 and py3 compatibility expects the streams to be opened in binary
mode but the sys.stdout objects are different in python2
and python3
python2:
>>> type(sys.stdout)
<type 'file'>
python3:
>>> type(sys.stdout)
<class '_io.TextIOWrapper'>
So in order to access binary stream, using sys.stdin.buffer in python3
Updates: #411
Change-Id: I1a633ccdddff5baf0cf05a8b493add39ddf75bd7
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The char pointer mdc_xattr_str in function mdc_xattr_list_populate
is malloc'd and doing a strcat into a malloc'd region can
overflow content allocated based on prior contents of the
memory region.
Added a NULL terimation to the malloc'd region to prevent
the overflow, and treat it as an empty string.
Change-Id: If0decab669551581230a8ede4c44c319ff04bac9
Updates: bz#1635373
Signed-off-by: ShyamsundarR <srangana@redhat.com>
(cherry picked from commit d00a2a1b398346bbdc5ac9b3ba4b09fb1ce1e699)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Most of the applications are {c|m}time dependant
and very few are atime dependant. So provide noatime
option to not update atime when ctime feature is
enabled.
Also this option has to be enabled with ctime
feature to avoid unnecessary self heal. Since
AFR/EC reads data from single subvolume, atime
is only updated in one subvolume triggering self
heal.
Backport of:
> Patch: https://review.gluster.org/21073
> BUG: 1593538
> Change-Id: I085fb33c882296545345f5df194cde7b6cbc337e
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 89636be4c73b12de2e11c75d8e59527bb243f147)
updates: bz#1633015
Change-Id: I085fb33c882296545345f5df194cde7b6cbc337e
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|