glusterfs.git/xlators/performance/io-threads/src/io-threads.h, branch v6.8

libglusterfs: Move devel headers under glusterfs directory

2018-12-05T21:47:04+00:00

libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.

Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs

This change although big, is just moving around the headers and
making it correct when including these headers from other sources.

This helps us correctly include libglusterfs includes without
namespace conflicts.

Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR

core: glusterfsd keeping fd open in index xlator

2018-10-12T15:22:36+00:00

Problem: At the time of processing GF_EVENT_PARENT_DOWN
         at brick xlator, it forwards the event to next xlator
         only while xlator ensures no stub is in progress.
         At io-thread xlator it decreases stub_cnt before the process
         a stub and notify EVENT to next xlator

Solution: Introduce a new counter to save stub_cnt and decrease
          the counter after process the stub completely at io-thread
          xlator.
          To avoid brick crash at the time of call xlator_mem_cleanup
          move only brick xlator if detach brick name has found in
          the graph

Note: Thanks to pranith for sharing a simple reproducer to
      reproduce the same

fixes bz#1637934
Change-Id: I1a694a001f7a5417e8771e3adf92c518969b6baa
Signed-off-by: Mohit Agrawal

core: glusterfsd keeping fd open in index xlator

2018-10-08T15:45:47+00:00

Problem: Current resource cleanup sequence is not
         perfect while brick mux is enabled

Solution: 1) Destroying xprt after cleanup all fd associated
             with a client
          2) Before call fini for brick xlators ensure no stub
             should be running on a brick

Change-Id: I86195785e428f57d3ef0da3e4061021fafacd435
fixes: bz#1631357
Signed-off-by: Mohit Agrawal

Land clang-format changes

2018-09-12T11:52:48+00:00

Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b

perfomance/io-threads: Add option to disable client disconnect feature

2018-02-28T01:45:51+00:00

> Add options to disable new features
> Commit ID: c071992e8d
> https://review.gluster.org/#/c/18291/
> By Michael Goulet 

This patch is required to forward port io-threads namespace patch.
Updates: #401

Change-Id: Ice477fdf4b8934f9fac0b4a2f6c93db97429a586
Signed-off-by: Varsha Rao

performance/io-threads: Add threads to priority based stagnant queues

2018-02-22T17:25:13+00:00

> performance/io-threads: Add watchdog to cover up a possible thread leak
> Commit ID: 8b6804f75c
> https://review.gluster.org/#/c/18239/
> By Shreyas Siravara 

This patch is required to forward port io-threads namespace patch.
Updates: #401

Change-Id: Id057c34a2abb9fc6dfb4afcd5c7bbbfe5693bbb8
Signed-off-by: Varsha Rao

performance/io-threads: expose io-thread queue depths

2018-02-08T17:01:12+00:00

The following release-3.8-fb branch patch is upstreamed:

> io-stats: Expose io-thread queue depths
> Commit ID: 69509ee7d2
> https://review.gluster.org/#/c/18143/
> By Shreyas Siravara 

Changes in this patch:
- Replace iot_pri_t with gf_fop_pri_t
- Replace IOT_PRI_{HI, LO, NORMAL, MAX, LEAST} with
  GF_FOP_PRI_{HI, LO, NORMAL, MAX, LEAST}
- Use dict_unref() instead of dict_destroy()

This patch is required to forward port io-threads namespace patch.
Updates: #401
Change-Id: I1b47a63185a441a30fbc423ca1015df7b36c2518
Signed-off-by: Varsha Rao

libglusterfs+transport+io-threads: fix 256KB stack abuse

2017-02-02T00:59:25+00:00

Some functions were allocating 64K booleans, which are (crazily) mapped to
4-byte ints, for a total of 256KB per call.  Changed to use bitfields instead,
so usage is now only 8KB per call.  This was the impediment to changing the
io-threads stack size, so that has been adjusted too.

Change-Id: I8781c4f2c8f2b830f4535e366995fac8dd0a8653
BUG: 1418095
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/15745
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Shyamsundar Ranganathan

performance/io-threads: Exit threads in fini() as well

2016-11-22T07:01:18+00:00

Problem:
io-threads starts the thread in 'init()' but doesn't clean them up
on 'fini()'. It relies on PARENT_DOWN to exit threads but there can
be cases where event before PARENT_UP the graph init code can think
of issuing fini(). This code path is hit when glfs_init() is called
on a volume that is in 'stopped' state. It leads to a crash in ganesha
process, because the io-thread tries to access freed memory.

Fix:
Ideal fix would be to wait for all fops in io-thread list to be completed on
PARENT_DOWN, and have fini() do cleanup of threads. Because there is no proper
documentation about how PARENT_DOWN/fini are supposed to be used,
we are getting different kinds of sequences in different higher level protocols.
So for now cleaning up in both PARENT_DOWN and fini(). Fuse doesn't call fini()
gfapi is not calling PARENT_DOWN in some cases, so for now I don't see
another way out.

BUG: 1396793
Change-Id: I9c9154e7d57198dbaff0f30d3ffc25f6d8088aec
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15888
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

performance/io-threads: Exit all threads on PARENT_DOWN

2016-10-23T09:32:05+00:00

Problem:
When glfs_fini() is called on a volume where client.io-threads is enabled,
fini() will free up iothread xl's private structure but there would be some
threads that are sleeping which would wakeup after the timedwait completes
leading to accessing already free'd memory.

Fix:
As part of parent-down, exit all sleeping threads.

BUG: 1381830
Change-Id: I0bb8d90241112c355fb22ee3fbfd7307f475b339
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15620
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G