glusterfs.git/xlators, branch v8.0rc0 md-cache: fix several NULL dereferences 2020-05-31T05:56:08+00:00 Xavi Hernandez xhernandez@redhat.com 2020-04-17T10:10:05+00:00 18bd1bdaa6ea5d589b21865769d6183e4e201006 This patch includes the following CID from Coverity Scan: * 1425196 * 1425197 * 1425198 * 1425199 * 1525200 Change-Id: Iddcfea449d3dd56d4dfcc39f4c3c608518e611e4 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com> Updates: #1060 (cherry picked from commit b53ba17dbfd2d18c10e2c308b8899d36726ab440) 
This patch includes the following CID from Coverity Scan:

  * 1425196
  * 1425197
  * 1425198
  * 1425199
  * 1525200

Change-Id: Iddcfea449d3dd56d4dfcc39f4c3c608518e611e4
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Updates: #1060
(cherry picked from commit b53ba17dbfd2d18c10e2c308b8899d36726ab440)
syncop: improve scaling and implement more tools 2020-05-30T05:17:12+00:00 Xavi Hernandez xhernandez@redhat.com 2020-04-30T09:19:01+00:00 67cd2b1ab5c9a8db637506b9a6be57fc79672c99 The current scaling of the syncop thread pool is not working properly and can leave some tasks in the run queue more time than necessary when the maximum number of threads is not reached. This patch provides a better scaling condition to react faster to pending work. Condition variables and sleep in the context of a synctask have also been implemented. Their purpose is to replace regular condition variables and sleeps that block synctask threads and prevent other tasks to be executed. The new features have been applied to several places in glusterd. Change-Id: Ic50b7c73c104f9e41f08101a357d30b95efccfbf Fixes: #1116 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com> 
The current scaling of the syncop thread pool is not working properly
and can leave some tasks in the run queue more time than necessary
when the maximum number of threads is not reached.

This patch provides a better scaling condition to react faster to
pending work.

Condition variables and sleep in the context of a synctask have also
been implemented. Their purpose is to replace regular condition
variables and sleeps that block synctask threads and prevent other
tasks to be executed.

The new features have been applied to several places in glusterd.

Change-Id: Ic50b7c73c104f9e41f08101a357d30b95efccfbf
Fixes: #1116
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
tests: skip tests on absence of reflink in xfs 2020-05-29T07:02:11+00:00 Pranith Kumar K pkarampu@redhat.com 2020-05-06T12:06:20+00:00 1572f98486bf3204b5d2d89da880b92e26734c3c Fixes: #1223 Change-Id: I36cb72d920ffd77405051546615c5262c392daef Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> (cherry picked from commit b85f01abab658d1d704cd6caf84dd64eddafbff7) 
Fixes: #1223
Change-Id: I36cb72d920ffd77405051546615c5262c392daef
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
(cherry picked from commit b85f01abab658d1d704cd6caf84dd64eddafbff7)
fuse: occasional logging for fuse device 'weird' write errors 2020-05-28T07:29:02+00:00 Csaba Henk csaba@redhat.com 2020-05-08T21:01:04+00:00 2ae5b28030b08348ec5e4c08ada8600943db0366 This change is a followup to I510158843e4b1d482bdc496c2e97b1860dc1ba93. In referred change we pushed log messages about 'weird' write errors to fuse device out of sight, by reporting them at Debug loglevel instead of Error (where 'weird' means errno is not POSIX compliant but having meaningful semantics for FUSE protocol). This solved the issue of spurious error reporting. And so far so good: these messages don't indicate an error condition by themselves. However, when they come in high repetitions, that indicates a suboptimal condition which should be reported.[1] Therefore now we shall emit a Warning if a certain errno occurs a certain number of times[2] as the outcome of a write to the fuse device. ___ [1] typically ENOENTs and ENOTDIRs accumulate when glusterfs' inode invalidation lags behind the kernel's internal inode garbage collection (in this case above errnos mean that the inode which we requested to be invalidated is not found in kernel). This can be mitigated with the invalidate-limit command line / mount option, cf. bz#1732717. [2] 256, as of the current implementation. Change-Id: I8cc7fe104da43a88875f93b0db49d5677cc16045 Updates: #1000 Signed-off-by: Csaba Henk <csaba@redhat.com> (cherry picked from commit c1baf3c68b87584aea5389af958326f6ed01d7ec) 
This change is a followup to
I510158843e4b1d482bdc496c2e97b1860dc1ba93.

In referred change we pushed log messages about 'weird'
write errors to fuse device out of sight, by reporting
them at Debug loglevel instead of Error (where
'weird' means errno is not POSIX compliant but having
meaningful semantics for FUSE protocol).

This solved the issue of spurious error reporting.
And so far so good: these messages don't indicate
an error condition by themselves. However, when they
come in high repetitions, that indicates a suboptimal
condition which should be reported.[1]

Therefore now we shall emit a Warning if a certain
errno occurs a certain number of times[2] as the
outcome of a write to the fuse device.

___
[1] typically ENOENTs and ENOTDIRs accumulate
when glusterfs' inode invalidation lags behind
the kernel's internal inode garbage collection
(in this case above errnos mean that the inode
which we requested to be invalidated is not found
in kernel). This can be mitigated with the
invalidate-limit command line / mount option,
cf. bz#1732717.

[2] 256, as of the current implementation.

Change-Id: I8cc7fe104da43a88875f93b0db49d5677cc16045
Updates: #1000
Signed-off-by: Csaba Henk <csaba@redhat.com>
(cherry picked from commit c1baf3c68b87584aea5389af958326f6ed01d7ec)
cluster/ec: Return correct error code and log message 2020-05-28T07:16:40+00:00 Ashish Pandey aspandey@redhat.com 2020-05-05T12:47:49+00:00 984a8dd8fa3e371fa1336730d6dc0041ceef877c In case of readdir was send with an FD on which opendir was failed, this FD will be useless and we return it with error. For now, we are returning it with EINVAL without logging any message in log file. Return a correct error code and also log the message to improve thing to debug. fixes: #1220 Change-Id: Iaf035254b9c5aa52fa43ace72d328be622b06169 (cherry picked from commit af70cb5eedd80207cd184e69f2a4fb252b72d070) 
In case of readdir was send with an FD on which opendir
was failed, this FD will be useless and we return it with error.
For now, we are returning it with EINVAL without logging any
message in log file.

Return a correct error code and also log the message to improve thing to debug.

fixes: #1220
Change-Id: Iaf035254b9c5aa52fa43ace72d328be622b06169
(cherry picked from commit af70cb5eedd80207cd184e69f2a4fb252b72d070)
afr: event gen changes 2020-04-24T12:50:13+00:00 Ravishankar N ravishankar@redhat.com 2020-04-10T09:00:57+00:00 1c8af2522458e3763302ec408fe1832dff150efa The general idea of the changes is to prevent resetting event generation to zero in the inode ctx, since event gen is something that should follow 'causal order'. Change #1: For a read txn, in inode refresh cbk, if event_generation is found zero, we are failing the read fop. This is not needed because change in event gen is only a marker for the next inode refresh to happen and should not be taken into account by the current read txn. Change #2: The event gen being zero above can happen if there is a racing lookup, which resets even get (in afr_lookup_done) if there are non zero afr xattrs. The resetting is done only to trigger an inode refresh and a possible client side heal on the next lookup. That can be acheived by setting the need_refresh flag in the inode ctx. So replaced all occurences of resetting even gen to zero with a call to afr_inode_need_refresh_set(). Change #3: In both lookup and discover path, we are doing an inode refresh which is not required since all 3 essentially do the same thing- update the inode ctx with the good/bad copies from the brick replies. Inode refresh also triggers background heals, but I think it is okay to do it when we call refresh during the read and write txns and not in the lookup path. The .ts which relied on inode refresh in lookup path to trigger heals are now changed to do read txn so that inode refresh and the heal happens. Change-Id: Iebf39a9be6ffd7ffd6e4046c96b0fa78ade6c5ec Fixes: #1179 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reported-by: Erik Jacobson <erik.jacobson at hpe.com> (cherry picked from commit f0fcd909ad4535b60c9208d4804ebe6afe421a09) 
The general idea of the changes is to prevent resetting event generation
to zero in the inode ctx, since event gen is something that should
follow 'causal order'.

Change #1:
For a read txn, in inode refresh cbk, if event_generation is
found zero, we are failing the read fop. This is not needed
because change in event gen is only a marker for the next inode refresh to
happen and should not be taken into account by the current read txn.

Change #2:
The event gen being zero above can happen if there is a racing lookup,
which resets even get (in afr_lookup_done) if there are non zero afr
xattrs. The resetting is done only to trigger an inode refresh and a
possible client side heal on the next lookup. That can be acheived by
setting the need_refresh flag in the inode ctx. So replaced all
occurences of resetting even gen to zero with a call to
afr_inode_need_refresh_set().

Change #3:
In both lookup and discover path, we are doing an inode refresh which is
not required since all 3 essentially do the same thing- update the inode
ctx with the good/bad copies from the brick replies. Inode refresh also
triggers background heals, but I think it is okay to do it when we call
refresh during the read and write txns and not in the lookup path.

The .ts which relied on inode refresh in lookup path to trigger heals are
now changed to do read txn so that inode refresh and the heal happens.

Change-Id: Iebf39a9be6ffd7ffd6e4046c96b0fa78ade6c5ec
Fixes: #1179
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reported-by: Erik Jacobson <erik.jacobson at hpe.com>
(cherry picked from commit f0fcd909ad4535b60c9208d4804ebe6afe421a09)
dht - Remove "tier" code (part 1) 2020-04-17T04:59:18+00:00 Barak Sason Rofman sason922@gmail.com 2020-04-12T15:19:41+00:00 0e94dbb81154edcc7d1182527663a7340fbeab01 This patch is removing some of the "tier" code in dht xlator, as it is no longer being used. Not all of the not-needed code is removed at once, so reviewing is easier. Follow up patches removing additional unused code will follow. This is based in the work done in https://review.gluster.org/#/c/glusterfs/+/23935/ Change-Id: I3cb6a0c5d8f14afcd87cf021ef8f74b91c0f908a updates: #1097 Signed-off-by: Barak Sason Rofman <bsaonro@redhat.com> 
This patch is removing some of the "tier" code in dht xlator, as it is no longer
being used.
Not all of the not-needed code is removed at once, so reviewing is easier.
Follow up patches removing additional unused code will follow.

This is based in the work done in https://review.gluster.org/#/c/glusterfs/+/23935/

Change-Id: I3cb6a0c5d8f14afcd87cf021ef8f74b91c0f908a
updates: #1097
Signed-off-by: Barak Sason Rofman <bsaonro@redhat.com>
md-cache: avoid clearing cache when not necessary 2020-04-16T11:15:09+00:00 Xavi Hernandez xhernandez@redhat.com 2020-03-30T09:09:39+00:00 f507c20bd11565c329d103075a15da11dba461d5 mdc_inode_xatt_set() blindly cleared current cache when dict was not NULL, even if there was no xattr requested. This patch fixes this by only calling mdc_inode_xatt_set() when we have explicitly requested something to cache. Change-Id: Idc91a4693f1ff39f7059acde26682ccc361b947d Fixes: #1140 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com> 
mdc_inode_xatt_set() blindly cleared current cache when dict was not
NULL, even if there was no xattr requested.

This patch fixes this by only calling mdc_inode_xatt_set() when we have
explicitly requested something to cache.

Change-Id: Idc91a4693f1ff39f7059acde26682ccc361b947d
Fixes: #1140
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
posix: fix GF_VALIDATE_OR_GOTO(this->name, this, out) 2020-04-16T11:05:19+00:00 Sanju Rakonde srakonde@redhat.com 2019-12-25T17:26:12+00:00 453861088dfabde2c41bec674bed2a6886b4ed37 Remove GF_VALIDATE_OR_GOTO(this->name, this, out) when this is passed as an argument and is checked for NULL in the caller itself. GF_VALIDATE_OR_GOTO(this->name, this, out) is modified to use xlator name instead of this->name as we are still verifying whether this is NULL. updates: #1000 Change-Id: Ide3180da29d0d4a35b2c5b9a7604fdf2ff4a9ffb Signed-off-by: Sanju Rakonde <srakonde@redhat.com> 
Remove GF_VALIDATE_OR_GOTO(this->name, this, out) when this
is passed as an argument and is checked for NULL in the caller
itself.

GF_VALIDATE_OR_GOTO(this->name, this, out) is modified to use
xlator name instead of this->name as we are still verifying
whether this is NULL.

updates: #1000

Change-Id: Ide3180da29d0d4a35b2c5b9a7604fdf2ff4a9ffb
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
mgmt/glusterd: use stat() syscall wrapper 2020-04-13T14:50:29+00:00 Dmitry Antipov dmantipov@yandex.ru 2020-04-10T07:47:17+00:00 0644fce4e52cd738f7ebb5e0596481a256fed3da Found with 0-symbol-check.t: ./tests/basic/0symbol-check.t .. 1..2 ./xlators/mgmt/glusterd/src/.libs/glusterd_la-glusterd-volume-set.o should call sys_stat, not stat ok 1 [ 40/ 41011] < 40> 'find . -name *.o -exec ./tests/basic/symbol-check.sh {} \;' not ok 2 [ 11/ 1] < 42> '[ ! -e ./.symbol-check-errors ]' -> '' Failed 1/2 subtests Change-Id: I8962f487cd88738a1f7a962049d513712687088c Fixes: #1160 Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru> 
Found with 0-symbol-check.t:

./tests/basic/0symbol-check.t ..
1..2
./xlators/mgmt/glusterd/src/.libs/glusterd_la-glusterd-volume-set.o should call sys_stat, not stat
ok   1 [     40/  41011] <  40> 'find . -name *.o -exec ./tests/basic/symbol-check.sh {} \;'
not ok   2 [     11/      1] <  42> '[ ! -e ./.symbol-check-errors ]' -> ''
Failed 1/2 subtests

Change-Id: I8962f487cd88738a1f7a962049d513712687088c
Fixes: #1160
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>