glusterfs.git/xlators/performance/io-cache/src/io-cache.c, branch release-4.0

perf/io-cache: Update options for gd2

2018-01-12T15:34:32+00:00

Update io-cache options for GD2

Updates gluster/glusterfs#302

Change-Id: I7fabf912d1e3cd024989404526db0a29644768c7
Signed-off-by: N Balachandran

xlators/performance/io-cache/: DEADCODE in ioc_open_cbk

2017-11-03T17:17:17+00:00

Coverity ID: 173

Issue: Event dead_error_line: Execution cannot reach this statement:
"fd_ctx_set(fd, this, 1UL);".

Removed if block as "weight == 0U" cannot be true, so if block will
never execute. Also removed weight variable because it is unused.

Change-Id: I4f028df29bfde91167fb15befa99b1fe1892adc6
BUG: 789278
Signed-off-by: Girjesh Rajoria

mem-pool: track glusterfs_ctx_t in struct mem_pool

2017-08-29T12:37:40+00:00

In order to generate statedumps per glusterfs_ctx_t, it is needed to
place all the memory pools in a structure that the context can reach.
The 'struct mem_pool' has been extended with a 'list_head owner' that is
linked with the glusterfs_ctx_t->mempool_list.

All callers of mem_pool_new() have been updated to pass the current
glusterfs_ctx_t along. This context is needed to add the new memory pool
to the list and for grabbing the ctx->lock while updating the
glusterfs_ctx_t->mempool_list.

Updates: #307
Change-Id: Ia9384424d8d1630ef3efc9d5d523bf739c356c6e
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/18075
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

performance/io-cache: update inode contexts of each entry in readdirplus

2017-07-27T03:58:10+00:00

io-cache stores read-cache in inode which is currently created only in
lookup. But, with readdirplus and md-cache absorbing lookups, io-cache
need not receive a lookup before a fop like readv.

Change-Id: I6eba995b0a90d4d5055a4aef0489707b852da1b8
BUG: 1474180
Signed-off-by: Raghavendra G 
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/5029
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

libglusterfs: stop special casing "cache-size" in size_t validation

2017-05-08T13:37:54+00:00

The original situation was as follows:

The function that validates xlator options indicating a size,
xlator_option_validate_sizet(), handles the case when the name
of the option is "cache-size" in a special way.

- Xlator options (things of type volume_option_t) has a
  min and max attribute of type double.
- An xlator option is endowed with a gluster specific type (not
  C type). An instance of an xlator option goes through a validation
  process by a type specific validator function (which are collected
  in option.c).
- Validators of numeric types - size being one of them - make use the
  min and max attributes to perform a range check, except in one case:
  if an option is defined with min = max = 0, then this option will be
  exempt of range checking. (Note: the volume_option_t definition
  features the following comments along the min, max fields:

   double                  min;  /* 0 means no range */
   double                  max;  /* 0 means no range */

  which is slightly misleading as it lets one to conclude that
  zeroing min or max buys exemption from low or high boundary check,
  which is not true -- only *both* being zero buys exemption.)
- Besides this, the validator for options of size type,
  xlator_option_validate_sizet() special cases options
  named "cache-size" so that only min is enforced. (The only consequence
  of a value exceeding max is that glusterd logs a warning about it, but
  the cli user who makes such a setting gets no feedback on it.)
- This was introduced because a hard coded limit is not useful for
  io-cache and quick-read. They rather use a runtime calculated
  upper limit. (See changes
  I7dd4d8c53051b89a293696abf1ee8dc237e39a20
  I9c744b5ace10604d5a814e6218ca0d83c796db80
  about the last two points.)
- As an unintended consequence, the upper limit check of
  cache-size of write-behind, for which a conventional hard coded limit
  is specified, is defeated.

What we do about it:

- Remove the special casing clause for cache-size in
  xlator_option_validate_sizet. Thus the general range
  check policy (as described above) will apply to
  cache-size too.
- To implement a lower bound only check by the validator
  for cache-size of io-cache and quick-read, change the
  max attribute of these options to INFINITY.

The only behavioral difference is the omission of the warnings
about cache-size of io-cache and quick-read exceeding the former max
values. (They were rather heuristic anyway.)

BUG: 1445609
Change-Id: I0bd8bd391fa7d926f76e214a2178833fe4673b4a
Signed-off-by: Csaba Henk 
Reviewed-on: https://review.gluster.org/17125
Smoke: Gluster Build System 
Reviewed-by: Amar Tumballi 
Tested-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

io-cache: Fix a read hang

2016-11-23T13:11:07+00:00

Issue:
=====
In certain cases, there was no unwind of read
from read-ahead xlator, thus resulting in hang.

RCA:
====
In certain cases, ioc_readv() issues STACK_WIND_TAIL() instead
of STACK_WIND(). One such case is when inode_ctx for that file
is not present (can happen if readdirp was called, and populates
md-cache and serves all the lookups from cache).

Consider the following graph:
...
io-cache (parent)
   |
readdir-ahead
   |
read-ahead
...

Below is the code snippet of ioc_readv calling STACK_WIND_TAIL:
ioc_readv()
{
...
 if (!inode_ctx)
   STACK_WIND_TAIL (frame, FIRST_CHILD (frame->this),
                    FIRST_CHILD (frame->this)->fops->readv, fd,
                    size, offset, flags, xdata);
   /* Ideally, this stack_wind should wind to readdir-ahead:readv()
      but it winds to read-ahead:readv(). See below for
      explaination.
    */
...
}

STACK_WIND_TAIL (frame, obj, fn, ...)
{
  frame->this = obj;
  /* for the above mentioned graph, frame->this will be readdir-ahead
   * frame->this = FIRST_CHILD (frame->this) i.e. readdir-ahead, which
   * is as expected
   */
  ...
  THIS = obj;
  /* THIS will be read-ahead instead of readdir-ahead!, as obj expands
   * to "FIRST_CHILD (frame->this)" and frame->this was pointing
   * to readdir-ahead in the previous statement.
   */
  ...
  fn (frame, obj, params);
  /* fn will call read-ahead:readv() instead of readdir-ahead:readv()!
   * as fn expands to "FIRST_CHILD (frame->this)->fops->readv" and
   * frame->this was pointing ro readdir-ahead in the first statement
   */
  ...
}

Thus, the readdir-ahead's readv() implementation will be skipped, and
ra_readv() will be called with frame->this = "readdir-ahead" and
this = "read-ahead". This can lead to corruption / hang / other problems.
But in this perticular case, when 'frame->this' and 'this' passed
to ra_readv() doesn't match, it causes ra_readv() to call ra_readv()
again!. Thus the logic of read-ahead readv() falls apart and leads to
hang.

Solution:
=========
Ideally, STACK_WIND_TAIL() should be modified as:
STACK_WIND_TAIL (frame, obj, fn, ...)
{
  next_xl = obj /* resolve obj as the variables passed in obj macro
                   can be overwritten in the further instrucions */
  next_xl_fn = fn /* resolve fn and store in a tmp variable, before
                     modifying any variables */
  frame->this = next_xl;
  ...
  THIS = next_xl;
  ...
  next_xl_fn (frame, next_xl, params);
  ...
}
But for this solution, knowing the type of variable 'next_xl_fn' is
a challenge and is not easy. Hence just modifying all the existing
callers to pass "FIRST_CHILD (this)" as obj, instead of
"FIRST_CHILD (frame->this)".

Change-Id: I179ffe3d1f154bc5a1935fd2ee44e912eb0fbb61
BUG: 1388292
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15901
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

performance/open-behind: Pass O_DIRECT flags for anon fd reads when required

2016-09-23T05:32:14+00:00

Writes are already passing the correct flags at the time of open().

Also, make io-cache honor direct-io for anon-fds with
O_DIRECT flag during reads.

Change-Id: I215cb09ef1b607b9f95cabf0ef3065c00edd9e78
BUG: 1377556
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15537
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

all: reduce "inline" usage

2015-09-01T11:55:15+00:00

There are three kinds of inline functions: plain inline, extern inline,
and static inline.  All three have been removed from .c files, except
those in "contrib" which aren't our problem.  Inlines in .h files, which
are overwhelmingly "static inline" already, have generally been left
alone.  Over time we should be able to "lower" these into .c files, but
that has to be done in a case-by-case fashion requiring more manual
effort.  This part was easy to do automatically without (as far as I can
tell) any ill effect.

In the process, several pieces of dead code were flagged by the
compiler, and were removed.

Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155
BUG: 1245331
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/11769
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Niels de Vos 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Venky Shankar

Logging: Porting the performance translator

2015-06-27T09:29:27+00:00

         logs to new logging framework.

Change-Id: Ie6aaf8d30bd4457bb73c48e23e6b1dea27598644
BUG: 1194640
Signed-off-by: arao 
Reviewed-on: http://review.gluster.org/9822
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

build: do not #include "config.h" in each file

2015-05-29T12:43:17+00:00

Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).

When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.

BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Pranith Kumar Karampuri