glusterfs.git/xlators/performance/quick-read/src/quick-read.c, branch release-3.13

perf/qr: Use a ref-ed data to extract content

2017-08-28T06:42:27+00:00

qr_content_extract used dict_get to get the value of
the GF_CONTENT_KEY key. dict_get does not ref the data
before returning it so QR could be acting on freed memory
if another thread deletes the key before then.
This patch also fixes a race in dict_get_with_ref.

Fix: Use dict_get_with_ref to retrieve the file contents.

Change-Id: Ib1a7a70bb92eed7e70747ec530e0b3edc53127ec
BUG: 1484709
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18115
Smoke: Gluster Build System 
Reviewed-by: Amar Tumballi 
Tested-by: Raghavendra G 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

libglusterfs: stop special casing "cache-size" in size_t validation

2017-05-08T13:37:54+00:00

The original situation was as follows:

The function that validates xlator options indicating a size,
xlator_option_validate_sizet(), handles the case when the name
of the option is "cache-size" in a special way.

- Xlator options (things of type volume_option_t) has a
  min and max attribute of type double.
- An xlator option is endowed with a gluster specific type (not
  C type). An instance of an xlator option goes through a validation
  process by a type specific validator function (which are collected
  in option.c).
- Validators of numeric types - size being one of them - make use the
  min and max attributes to perform a range check, except in one case:
  if an option is defined with min = max = 0, then this option will be
  exempt of range checking. (Note: the volume_option_t definition
  features the following comments along the min, max fields:

   double                  min;  /* 0 means no range */
   double                  max;  /* 0 means no range */

  which is slightly misleading as it lets one to conclude that
  zeroing min or max buys exemption from low or high boundary check,
  which is not true -- only *both* being zero buys exemption.)
- Besides this, the validator for options of size type,
  xlator_option_validate_sizet() special cases options
  named "cache-size" so that only min is enforced. (The only consequence
  of a value exceeding max is that glusterd logs a warning about it, but
  the cli user who makes such a setting gets no feedback on it.)
- This was introduced because a hard coded limit is not useful for
  io-cache and quick-read. They rather use a runtime calculated
  upper limit. (See changes
  I7dd4d8c53051b89a293696abf1ee8dc237e39a20
  I9c744b5ace10604d5a814e6218ca0d83c796db80
  about the last two points.)
- As an unintended consequence, the upper limit check of
  cache-size of write-behind, for which a conventional hard coded limit
  is specified, is defeated.

What we do about it:

- Remove the special casing clause for cache-size in
  xlator_option_validate_sizet. Thus the general range
  check policy (as described above) will apply to
  cache-size too.
- To implement a lower bound only check by the validator
  for cache-size of io-cache and quick-read, change the
  max attribute of these options to INFINITY.

The only behavioral difference is the omission of the warnings
about cache-size of io-cache and quick-read exceeding the former max
values. (They were rather heuristic anyway.)

BUG: 1445609
Change-Id: I0bd8bd391fa7d926f76e214a2178833fe4673b4a
Signed-off-by: Csaba Henk 
Reviewed-on: https://review.gluster.org/17125
Smoke: Gluster Build System 
Reviewed-by: Amar Tumballi 
Tested-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: fix Ubuntu code audit (cppcheck) results

2015-11-01T14:43:04+00:00

This change includes an additional fix (forward port) of a fix
made on the release-3.x branches to address a comment made after
the original change was merged on the master branch.

* release-3.7
* Change-Id: Ie15c5919e5bf9b0a1c66e20dc42d80fdfa8bd7f4
* BZ: 1227808
*  http://review.gluster.org/11069

Change-Id: I4fc2672ab1a17998b2e40bc43eb6a3e15058a086
BUG: 1109180
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/11067
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

performance translators : porting the missing gf_log to gf_msg

2015-09-01T03:28:53+00:00

Change-Id: I5cc2b4669b164fe09637c86da05d2d94589dd7e4
BUG: 1253149
Signed-off-by: Hari Gowtham 
Reviewed-on: http://review.gluster.org/11906
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G

features/bit-rot-stub: deny access to bad objects

2015-06-27T11:17:33+00:00

* Access to bad objects (especially operations such as open, readv, writev)
  should be denied to prevent applications from getting wrong data.

* Do not allow anyone apart from scrubber to set bad object xattr.

* Do not allow bad object xattr to be removed.

Change-Id: Ia9185a067233a9f26e3d41d41d11d9a4eb0da827
BUG: 1210689
Signed-off-by: Raghavendra Bhat 
Signed-off-by: Venky Shankar 
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/11126
Tested-by: NetBSD Build System

Logging: Porting the performance translator

2015-06-27T09:29:27+00:00

         logs to new logging framework.

Change-Id: Ie6aaf8d30bd4457bb73c48e23e6b1dea27598644
BUG: 1194640
Signed-off-by: arao 
Reviewed-on: http://review.gluster.org/9822
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

quick-read: Do a null check before unref

2015-04-17T06:48:16+00:00

Commit 4ea5b8d2046b9e0bc7f24cdf1b2e72ab8b462c9e seems to have removed the check
as a part of static analyis fixes but I'm seeing errors in the client log.

--------------------
touch  /mnt/fuse_mnt/zero-byte-file
echo 3 > /proc/sys/vm/drop_caches
cat  /mnt/fuse_mnt/zero-byte-file

mount log:
[2015-04-13 05:52:21.683256] E [iobuf.c:790:iobuf_unref] (-->
/usr/local/lib/libglusterfs.so.0(_gf_log_callingfn+0x232)[0x7feda12c0e24] (-->
/usr/local/lib/libglusterfs.so.0(iobuf_unref+0x56)[0x7feda1304c8e] (-->
/usr/local/lib/glusterfs/3.7dev/xlator/performance/quick-read.so(qr_readv_cached+0x466)[0x7fed95b7e2fc]
(-->
/usr/local/lib/glusterfs/3.7dev/xlator/performance/quick-read.so(qr_readv+0x70)[0x7fed95b7e385]
(-->
/usr/local/lib/libglusterfs.so.0(default_readv_resume+0x270)[0x7feda12d4401]
))))) 0-iobuf: invalid argument: iobuf

--------------------

Hence re-adding the checks.
Note: I'm using the same BZ Id used for the original commit though it is in
MODIFIED state just for correlation.

Change-Id: I79749814a9d4082933e3b306ce449492ee5b43a5
BUG: 1109180
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/10206
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Vijay Bellur

Performance: Replace ASSERTS in xlator fini() with info messages, for the known leaks.

2015-03-10T11:42:22+00:00

There are few known leaks in read-ahead and quick-read xlator fini().
Until they are fixed replacing the ASSERTS with info, else any call
to glfs_fini() in debug mode will core dump.

Change-Id: Id60c6f952574863fc77c7d101cb5d5e9113090d8
BUG: 1199436
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/9819
Reviewed-by: Raghavendra Talur 
Reviewed-by: Ravishankar N 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

core: fix Ubuntu code audit (cppcheck) results

2014-11-25T17:27:36+00:00

See also http://review.gluster.org/#/c/7693/, BZ 1091677

AFAICT these are false positives:

[geo-replication/src/gsyncd.c:100]: (error) Memory leak: str
[geo-replication/src/gsyncd.c:403]: (error) Memory leak: argv
[xlators/nfs/server/src/nlm4.c:1201]: (error) Possible null pointer dereference: fde
[xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr

Test program:

[extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments.
[tests/basic/fops-sanity.c:55]: (error) Buffer overrun possible for long command line arguments.

the remainder are fixed with this change-set:

[cli/src/cli-rpc-ops.c:8883]: (error) Possible null pointer dereference: local
[cli/src/cli-rpc-ops.c:8886]: (error) Possible null pointer dereference: local
[contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 2) requires 'long *' but the argument type is 'unsigned long *'.
[contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 3) requires 'long *' but the argument type is 'unsigned long *'.
[xlators/cluster/dht/src/dht-rebalance.c:1734]: (error) Possible null pointer dereference: ctx
[xlators/cluster/stripe/src/stripe.c:4940]: (error) Possible null pointer dereference: local
[xlators/mgmt/glusterd/src/glusterd-geo-rep.c:1718]: (error) Possible null pointer dereference: command
[xlators/mgmt/glusterd/src/glusterd-replace-brick.c:942]: (error) Resource leak: file
[xlators/mgmt/glusterd/src/glusterd-replace-brick.c:1026]: (error) Resource leak: file
[xlators/mgmt/glusterd/src/glusterd-sm.c:249]: (error) Possible null pointer dereference: new_ev_ctx
[xlators/mgmt/glusterd/src/glusterd-snapshot.c:6917]: (error) Possible null pointer dereference: volinfo
[xlators/mgmt/glusterd/src/glusterd-utils.c:4517]: (error) Possible null pointer dereference: this
[xlators/mgmt/glusterd/src/glusterd-utils.c:6662]: (error) Possible null pointer dereference: this
[xlators/mgmt/glusterd/src/glusterd-utils.c:7708]: (error) Possible null pointer dereference: this
[xlators/mount/fuse/src/fuse-bridge.c:4687]: (error) Uninitialized variable: finh
[xlators/mount/fuse/src/fuse-bridge.c:3080]: (error) Possible null pointer dereference: state
[xlators/nfs/server/src/nfs-common.c:89]: (error) Dangerous usage of 'volname' (strncpy doesn't always null-terminate it).
[xlators/performance/quick-read/src/quick-read.c:586]: (error) Possible null pointer dereference: iobuf

Rerunning cppcheck after fixing the above:

As before, test program:

[extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments.
[tests/basic/fops-sanity.c:55]: (error) Buffer overrun possible for long command line arguments.

As before, false positive:

[geo-replication/src/gsyncd.c:100]: (error) Memory leak: str
[geo-replication/src/gsyncd.c:403]: (error) Memory leak: argv
[xlators/nfs/server/src/nlm4.c:1201]: (error) Possible null pointer dereference: fde
[xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr

False positive after fix:

[xlators/performance/quick-read/src/quick-read.c:584]: (error) Possible null pointer dereference: iobuf

Change-Id: I20e0e3ac1d600b2f2120b8d8536cd6d9e17023e8
BUG: 1109180
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/8064
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

build: MacOSX Porting fixes

2014-04-24T21:41:48+00:00

git@forge.gluster.org:~schafdog/glusterfs-core/osx-glusterfs

Working functionality on MacOSX

 - GlusterD (management daemon)
 - GlusterCLI (management cli)
 - GlusterFS FUSE (using OSXFUSE)
 - GlusterNFS (without NLM - issues with rpc.statd)

Change-Id: I20193d3f8904388e47344e523b3787dbeab044ac
BUG: 1089172
Signed-off-by: Harshavardhana 
Signed-off-by: Dennis Schafroth 
Tested-by: Harshavardhana 
Tested-by: Dennis Schafroth 
Reviewed-on: http://review.gluster.org/7503
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati