glusterfs.git/xlators/cluster, branch v3.4.6beta1 cluster/afr: Fix memory leak of file-path in self-heal-daemon 2014-08-27T13:46:38+00:00 Pranith Kumar K pkarampu@redhat.com 2014-08-26T07:29:47+00:00 f0ddba7e0913db505f1295e9b3b7d35ead9c4407 Backport of http://review.gluster.org/4790 Note: Only the part which fixes the memory leak is backported shd event has path which needs to be freed as part of circular buffer cleanup. This patch introduces the functionality so that self-heal-daemon can use it. Change-Id: I3f3823d5587eda2fcb278f0fdb89123a31c9d786 BUG: 1119894 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/8541 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> 
        Backport of http://review.gluster.org/4790

Note: Only the part which fixes the memory leak is backported

shd event has path which needs to be freed as part of circular buffer cleanup.
This patch introduces the functionality so that self-heal-daemon can use it.

Change-Id: I3f3823d5587eda2fcb278f0fdb89123a31c9d786
BUG: 1119894
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/8541
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
NFS: stripe-xlator should pass EOF at end of READDIR 2014-08-27T13:43:33+00:00 Niels de Vos ndevos@redhat.com 2014-08-21T16:24:33+00:00 1679b72c5b023884fb4bc6a2a85b06016fb3cab8 NFS READDIR replies are made of a header, a sequence of entries, and a EOF flag. When GlusterFS's NFS server is used along with stripe xlator, it fails to set the EOF flag, which violates NFS RFC and confuses some clients. The bug is caused because nfs xlator sets EOF if it gets op_errno set to ENOENT. That value is produced in storage xlator and propagated through server, client, and other xlators until stripe xlator handles it. stripe only passed op_errno if op_ret < 0, which is not the case here. This change set adds a special case for that situation to fix the problem. Cherry picked from commit 9b5231e5c98b8cfa116838287c7a14042702795f: > Change-Id: Ie6db94b0515292387cfb04c1e4a9363f34fcd19a > BUG: 1130969 > Reported-by: Emmanuel Dreyfus <manu@netbsd.org> > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: http://review.gluster.org/8493 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Santosh Pradhan <spradhan@redhat.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> > Reviewed-by: Emmanuel Dreyfus <manu@netbsd.org> > Tested-by: Emmanuel Dreyfus <manu@netbsd.org> Change-Id: Ie6db94b0515292387cfb04c1e4a9363f34fcd19a BUG: 1132392 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/8510 Reviewed-by: Santosh Pradhan <spradhan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> 
NFS READDIR replies are made of a header, a sequence of
entries, and a EOF flag. When GlusterFS's NFS server is
used along with stripe xlator, it fails to set the EOF
flag, which violates NFS RFC and confuses some clients.

The bug is caused because nfs xlator sets EOF if it gets
op_errno set to ENOENT. That value is produced in storage
xlator and propagated through server, client, and other
xlators until stripe xlator handles it. stripe only passed
op_errno if op_ret < 0, which is not the case here. This
change set adds a special case for that situation to fix
the problem.

Cherry picked from commit 9b5231e5c98b8cfa116838287c7a14042702795f:
> Change-Id: Ie6db94b0515292387cfb04c1e4a9363f34fcd19a
> BUG: 1130969
> Reported-by: Emmanuel Dreyfus <manu@netbsd.org>
> Signed-off-by: Niels de Vos <ndevos@redhat.com>
> Reviewed-on: http://review.gluster.org/8493
> Tested-by: Gluster Build System <jenkins@build.gluster.com>
> Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
> Reviewed-by: Emmanuel Dreyfus <manu@netbsd.org>
> Tested-by: Emmanuel Dreyfus <manu@netbsd.org>

Change-Id: Ie6db94b0515292387cfb04c1e4a9363f34fcd19a
BUG: 1132392
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/8510
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cluster/dht: Don't do extra unref in dht-migration checks 2014-06-18T12:17:34+00:00 Vijay Bellur vbellur@redhat.com 2014-06-10T16:51:28+00:00 2b789331dc933b186360fc8cbffb06289ee60ee9 Problem: syncop_open used to perform a ref in syncop_open_cbk so the extra unref was needed but now syncop_open_cbk does not take a ref so no need to do extra unref. Fix: remove the extra fd_unref and let dht_local_wipe do the final unref. Change-Id: Ibe8f9a678d456a0c7bff175306068b5cd297ecc4 BUG: 961615 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8029 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: Joe Julian <joe@julianfamily.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> 
Problem:
syncop_open used to perform a ref in syncop_open_cbk so the extra
unref was needed but now syncop_open_cbk does not take a ref so no
need to do extra unref.

Fix:
remove the extra fd_unref and let dht_local_wipe do the final unref.

Change-Id: Ibe8f9a678d456a0c7bff175306068b5cd297ecc4
BUG: 961615
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/8029
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: Joe Julian <joe@julianfamily.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
core: fix Ubuntu code audit (cppcheck) results 2014-05-25T10:52:01+00:00 Kaleb S. KEITHLEY kkeithle@redhat.com 2014-04-28T18:25:09+00:00 d5e317718f4137431c3996ac5f38e9226620e760 These block inclusion in Ubuntu Main repo. AFAICT these are false positives: [rpc/rpc-transport/rdma/src/rdma.c:3074]: (error) Address of local auto-variable assigned to a function parameter. [xlators/features/marker/utils/src/gsyncd.c:99]: (error) Memory leak: str [xlators/features/marker/utils/src/gsyncd.c:354]: (error) Memory leak: argv [xlators/nfs/server/src/nlm4.c:1176]: (error) Possible null pointer dereference: fde The remainder are fixed with this change-set: [api/src/glfs-fops.c:700]: (error) Possible null pointer dereference: gio [api/src/glfs-fops.c:702]: (error) Possible null pointer dereference: frame [xlators/cluster/afr/src/afr-inode-write.c:375]: (error) Possible null pointer dereference: frame [xlators/cluster/afr/src/afr-self-heal-common.c:1522]: (error) Possible null pointer dereference: local [xlators/cluster/dht/src/dht-rebalance.c:1574]: (error) Possible null pointer dereference: ctx [xlators/cluster/stripe/src/stripe.c:4407]: (error) Possible null pointer dereference: local [xlators/mgmt/glusterd/src/glusterd-mountbroker.c:675]: (error) Possible null pointer dereference: cookieswitch [xlators/mgmt/glusterd/src/glusterd-mountbroker.c:677]: (error) Possible null pointer dereference: cookieswitch [xlators/mgmt/glusterd/src/glusterd-replace-brick.c:924]: (error) Resource leak: file [xlators/mgmt/glusterd/src/glusterd-replace-brick.c:1008]: (error) Resource leak: file [xlators/mgmt/glusterd/src/glusterd-sm.c:248]: (error) Possible null pointer dereference: new_ev_ctx [xlators/mgmt/glusterd/src/glusterd-store.c:1250]: (error) Possible null pointer dereference: handle [xlators/mgmt/glusterd/src/glusterd-utils.c:4272]: (error) Possible null pointer dereference: this [xlators/mgmt/glusterd/src/glusterd-utils.c:5113]: (error) Possible null pointer dereference: this [xlators/mount/fuse/src/fuse-bridge.c:4432]: (error) Uninitialized variable: finh [xlators/mount/fuse/src/fuse-bridge.c:2927]: (error) Possible null pointer dereference: state [xlators/mount/fuse/src/fuse-bridge.c:3226]: (error) Possible null pointer dereference: state [xlators/storage/bd_map/src/bd_map.c:1504]: (error) Possible null pointer dereference: bd_fd [xlators/storage/bd_map/src/bd_map.c:1728]: (error) Possible null pointer dereference: n_entry [xlators/storage/bd_map/src/bd_map.c:1741]: (error) Possible null pointer dereference: n_entry [xlators/performance/quick-read/src/quick-read.c:585]: (error) Possible null pointer dereference: iobuf rerunning cppcheck --force afterwards: Test code, don't care: [extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments. False positive after fix [xlators/cluster/stripe/src/stripe.c:4407]: (error) Possible null pointer dereference: local Still false positive: [xlators/features/marker/utils/src/gsyncd.c:354]: (error) Memory leak: argv [xlators/nfs/server/src/nlm4.c:1176]: (error) Possible null pointer dereference: fde Not built, don't care: [xlators/cluster/ha/src/ha.c:2699]: (error) Possible null pointer dereference: priv Change-Id: I1fb849e9c042d3a3701cb05121d413e58e73d505 BUG: 1086460 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/7583 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> 
These block inclusion in Ubuntu Main repo.

AFAICT these are false positives:

[rpc/rpc-transport/rdma/src/rdma.c:3074]: (error) Address of local auto-variable assigned to a function parameter.
[xlators/features/marker/utils/src/gsyncd.c:99]: (error) Memory leak: str
[xlators/features/marker/utils/src/gsyncd.c:354]: (error) Memory leak: argv
[xlators/nfs/server/src/nlm4.c:1176]: (error) Possible null pointer dereference: fde

The remainder are fixed with this change-set:

[api/src/glfs-fops.c:700]: (error) Possible null pointer dereference: gio
[api/src/glfs-fops.c:702]: (error) Possible null pointer dereference: frame
[xlators/cluster/afr/src/afr-inode-write.c:375]: (error) Possible null pointer dereference: frame
[xlators/cluster/afr/src/afr-self-heal-common.c:1522]: (error) Possible null pointer dereference: local
[xlators/cluster/dht/src/dht-rebalance.c:1574]: (error) Possible null pointer dereference: ctx
[xlators/cluster/stripe/src/stripe.c:4407]: (error) Possible null pointer dereference: local
[xlators/mgmt/glusterd/src/glusterd-mountbroker.c:675]: (error) Possible null pointer dereference: cookieswitch
[xlators/mgmt/glusterd/src/glusterd-mountbroker.c:677]: (error) Possible null pointer dereference: cookieswitch
[xlators/mgmt/glusterd/src/glusterd-replace-brick.c:924]: (error) Resource leak: file
[xlators/mgmt/glusterd/src/glusterd-replace-brick.c:1008]: (error) Resource leak: file
[xlators/mgmt/glusterd/src/glusterd-sm.c:248]: (error) Possible null pointer dereference: new_ev_ctx
[xlators/mgmt/glusterd/src/glusterd-store.c:1250]: (error) Possible null pointer dereference: handle
[xlators/mgmt/glusterd/src/glusterd-utils.c:4272]: (error) Possible null pointer dereference: this
[xlators/mgmt/glusterd/src/glusterd-utils.c:5113]: (error) Possible null pointer dereference: this
[xlators/mount/fuse/src/fuse-bridge.c:4432]: (error) Uninitialized variable: finh
[xlators/mount/fuse/src/fuse-bridge.c:2927]: (error) Possible null pointer dereference: state
[xlators/mount/fuse/src/fuse-bridge.c:3226]: (error) Possible null pointer dereference: state
[xlators/storage/bd_map/src/bd_map.c:1504]: (error) Possible null pointer dereference: bd_fd
[xlators/storage/bd_map/src/bd_map.c:1728]: (error) Possible null pointer dereference: n_entry
[xlators/storage/bd_map/src/bd_map.c:1741]: (error) Possible null pointer dereference: n_entry
[xlators/performance/quick-read/src/quick-read.c:585]: (error) Possible null pointer dereference: iobuf

rerunning cppcheck --force afterwards:

Test code, don't care:
[extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments.

False positive after fix
[xlators/cluster/stripe/src/stripe.c:4407]: (error) Possible null pointer dereference: local

Still false positive:
[xlators/features/marker/utils/src/gsyncd.c:354]: (error) Memory leak: argv
[xlators/nfs/server/src/nlm4.c:1176]: (error) Possible null pointer dereference: fde

Not built, don't care:
[xlators/cluster/ha/src/ha.c:2699]: (error) Possible null pointer dereference: priv

Change-Id: I1fb849e9c042d3a3701cb05121d413e58e73d505
BUG: 1086460
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/7583
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
dht: dht_lookup_dir_cbk should set op_errno as local->op_errno 2014-03-11T16:31:57+00:00 shishir gowda gowda.shishir@gmail.com 2013-12-10T09:32:49+00:00 010a9a7867c7135dfedf52e5d2b34122a9cb1984 Two glusterfs clients return inconsistent errnos when the bricks of the volume were down. Consider two gluster mounts. Mount 1 was done when the bricks were online. Mount 2 was done after the bricks were killed, (using the 'glusterfs' command instead of the mount script). For any request, mount 1 will return ENOTCONN, where as mount 2 will return ENOENT. This happens because for the 2nd mount, a fuse would send a lookup on '/' for any request, as it hadn't been done yet. The client xlator returns ENOTCONN, but the dht_lookup_dir_cbk changed this to ENOENT unconditionally when aggregating. So, fuse returned ENOENT, even though the errno should have been ENOTCONN. backporting http://review.gluster.org/6072 BUG: 1019095 Change-Id: Iaa40dffefddfcaf1ab7736f5423d7f9d2ece1363 Original-author: Kaushal M <kaushal@redhat.com> Signed-off-by: shishir gowda <gowda.shishir@gmail.com> Reviewed-on: http://review.gluster.org/6471 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-by: Anand Avati <avati@redhat.com> 
Two glusterfs clients return inconsistent errnos when the bricks of the volume
were down. Consider two gluster mounts. Mount 1 was done when the bricks were
online. Mount 2 was done after the bricks were killed, (using the 'glusterfs'
command instead of the mount script).

For any request, mount 1 will return ENOTCONN, where as mount 2 will return
ENOENT.

This happens because for the 2nd mount, a fuse would send a lookup on '/' for
any request, as it hadn't been done yet. The client xlator returns ENOTCONN,
but the dht_lookup_dir_cbk changed this to ENOENT unconditionally when
aggregating. So, fuse returned ENOENT, even though the errno should have been
ENOTCONN.

backporting  http://review.gluster.org/6072

BUG: 1019095
Change-Id: Iaa40dffefddfcaf1ab7736f5423d7f9d2ece1363
Original-author: Kaushal M <kaushal@redhat.com>
Signed-off-by: shishir gowda <gowda.shishir@gmail.com>
Reviewed-on: http://review.gluster.org/6471
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Treat FOOL condition as split-brain for entry self-heal 2014-02-12T17:34:00+00:00 Pranith Kumar K pkarampu@redhat.com 2014-01-25T05:13:52+00:00 945c6de4e6c3f55ab05bebf84868d6183b44260f BUG: 1057846 Change-Id: I19051c19a54c8aab37eb7cb32dde9f7e9e77c073 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/6854 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
BUG: 1057846
Change-Id: I19051c19a54c8aab37eb7cb32dde9f7e9e77c073
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/6854
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/dht: Make sure loc has gfid 2014-01-13T12:58:45+00:00 Pranith Kumar K pkarampu@redhat.com 2014-01-13T10:43:30+00:00 8eda793ed6762ca71521698eb80e2262b5b63092 Problem: In some code paths neither loc->gfid nor loc->inode->gfid is populated which leads to EINVAL for linkfile setattr in dht_linkfile_attr_heal. Fix: Populate loc->gfid before dht_linkfile_attr_heal. BUG: 971805 Change-Id: I8e4b7510ee5c38aa9ccf5283c7165c7df25ec62b Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/6691 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
Problem:
In some code paths neither loc->gfid nor loc->inode->gfid
is populated which leads to EINVAL for linkfile setattr
in dht_linkfile_attr_heal.

Fix:
Populate loc->gfid before dht_linkfile_attr_heal.

BUG: 971805
Change-Id: I8e4b7510ee5c38aa9ccf5283c7165c7df25ec62b
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/6691
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/dht: Ignore ENOENT errors for unlink of linkfiles 2013-12-24T11:32:23+00:00 Anand Avati avati@redhat.com 2013-12-12T10:04:15+00:00 828a6eea202e9195f323a7c4b4656c73ecb34256 Backport of http://review.gluster.org/4971 If unlink of linkfile returns ENOENT, do not fail unlink. Proceed with unlinking of cached file. Change-Id: If7cec92b40c39d68dd9c3606c6c2c3a6bd67d27b BUG: 966848 Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/6586 Reviewed-by: Harshavardhana <harsha@harshavardhana.net> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
Backport of http://review.gluster.org/4971

If unlink of linkfile returns ENOENT, do not fail unlink.
Proceed with unlinking of cached file.

Change-Id: If7cec92b40c39d68dd9c3606c6c2c3a6bd67d27b
BUG: 966848
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/6586
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Revert "core: fix errno for non-existent GFID" 2013-12-24T11:31:53+00:00 Vijay Bellur vbellur@redhat.com 2013-12-23T10:39:00+00:00 63a3a0dd297f0615517de399f54df99041577a35 This reverts commit 837422858c2e4ab447879a4141361fd382645406 Change-Id: I0909f26ce088454bb14b3694b489c672286a4ae6 Reviewed-on: http://review.gluster.org/6575 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
This reverts commit 837422858c2e4ab447879a4141361fd382645406

Change-Id: I0909f26ce088454bb14b3694b489c672286a4ae6
Reviewed-on: http://review.gluster.org/6575
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/dht: interim fix for reverting 837422858c 2013-12-24T09:53:48+00:00 Vijay Bellur vbellur@redhat.com 2013-12-23T17:25:15+00:00 92ad6c28936904ed2a43d254892a325bc2c695dc Change-Id: I74818a03f7c5d7891561515af2fa35ea3775255c BUG: 1032894 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/6582 Tested-by: Gluster Build System <jenkins@build.gluster.com> 
Change-Id: I74818a03f7c5d7891561515af2fa35ea3775255c
BUG: 1032894
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/6582
Tested-by: Gluster Build System <jenkins@build.gluster.com>