glusterfs.git/libglusterfs/src/mem-pool.c, branch v3.10.7

libglusterfs: make memory pools more thread-friendly

2017-02-03T00:44:09+00:00

Early multiplexing tests revealed *massive* contention on certain
pools' global locks - especially for dictionaries and secondarily for
call stubs.  For the thread counts that multiplexing can create, a
more lock-free solution is clearly needed.  Also, the current mem-pool
implementation does a poor job releasing memory back to the system,
artificially inflating memory usage to match whatever the worst case
was since the process started.  This is bad in general, but especially
so for multiplexing where there are more pools and a major point of
the whole exercise is to reduce memory consumption.

The basic ideas for the new design are these

  There is one pool, globally, for each power-of-two size range.
  Every attempt to create a new pool within this range will instead
  add a reference to the existing pool.

  Instead of adding pools for each translator within each multiplexed
  brick (potentially infinite and quite possibly thousands), we
  allocate one set of size-based pools per *thread* (hundreds at
  worst).

  Each per-thread pool is divided into hot and cold lists.  Every
  allocation first attempts to use the hot list, then the cold list.
  When objects are freed, they always go on the hot list.

  There is one global "pool sweeper" thread, which periodically
  reclaims everything in each pool's cold list and then "demotes" the
  current hot list to be the new cold list.

  For normal allocation activity, only a per-thread lock need be
  taken, and even that only to guard against very rare contention from
  the pool sweeper.  When threads start and stop, a global lock must
  be taken to add them to the pool sweeper's list.  Lock contention is
  therefore extremely low, and the hot/cold lists also provide good
  locality.

A more complete explanation (of a similar earlier design) can be found
here:

 http://www.gluster.org/pipermail/gluster-devel/2016-October/051160.html

Backport of:
> Change-Id: I5bc8a1ba57cfb553998f979a498886e0d006e665
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/15645

BUG: 1418091
Change-Id: Id09bbea41f65fcd245822607bc204f3a34904dc2
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16531
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

core: Disable the memory pooler in Gluster via a build flag

2017-01-04T11:42:40+00:00

Summary:
Passing --disable-mempool to configure will disable the mempool.

Change-Id: I60d5f70d54de507fe9f4695d7589f7ae1ba4bb0f
BUG: 1405165
Signed-off-by: Shreyas Siravara 
Reviewed-on: http://review.gluster.org/16148
Smoke: Gluster Build System 
Tested-by: Jeff Darcy 
Reviewed-by: Niels de Vos 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System 
Reviewed-by: Anoop C S 
NetBSD-regression: NetBSD Build System

libglusterfs:Now mempool is added to ctx pool list under a lock

2016-11-22T12:45:34+00:00

mempool is added to ctx pool list without any lock. This can cause undefined
behaviour in case of multithreaded environment.

Fix: modify the list only under ctx->lock

Change-Id: I7bdbb3db48a899bb0e41427e149b13c0facaedba
Signed-off-by: Rajesh Joseph 
BUG: 1394719
Reviewed-on: http://review.gluster.org/15842
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Poornima G 
Reviewed-by: Niels de Vos

libglusterfs/mem-pool: fix unused variable warnings/errors

2016-09-16T07:07:09+00:00

http://review.gluster.org/14085 fixes a "pragma leak" where the
generated rpc/xdr headers have a pair of pragmas that disable these
warnings. With the warnings disabled, many unused variables have
crept into the code base.

And 14085 won't pass its own smoke test until all these warnings are
fixed.

BUG: 1369124
Change-Id: I8885e4d4aa44307c240413ba6f35ecd59ab45444
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15516
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

libglusterfs: add gf_get_mem_type()

2016-08-15T07:56:54+00:00

gfapi needs to provide a function towards applications to free memory
that it allocated. Depending on how the application is compiled/linked,
it could use a different memory allocator than Gluster itself. Therefore
it is not safe for gfapi to request applications to free memory with
'standard' free().

Examples for this are Gluster allocated structures with GF_CALLOC() when
memory accounting is enabled (the default). Some gfapi functions use
malloc() to allocate memory as a workaround, but the free() from the
jemalloc implementation should not be combined with the malloc() from
glibc.

Change-Id: I626cd1a60abf8965f9263290f4045d1f69fc2093
BUG: 1344714
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15108
Smoke: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: jiffin tony Thottan 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

libglusterfs: fix glusterd statedump crash

2016-08-04T15:44:09+00:00

commit 3c04a91 removed setting typeStr to NULL if num_allocs is set to 0, this
has caused this regression. Code has been put back like earlier and to avoid
statedump printing all the NULL values check is modified to see skip the records
if num_allocs is 0 instead of total_allocs

Change-Id: Ib8bcc2fba908e88cf52b641c3f6bcba74f5e667c
BUG: 1359190
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/14987
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Prashanth Pai 
Reviewed-by: Jeff Darcy

statedump: Prevent (null) typestr to be printed

2016-04-24T13:44:44+00:00

Problem:
After the commits: 7e44c783ad731856956929f6614bbe045c26ea3a - lock: use
spinlock only on multicore systems
a6aecae2cd8171b8538bfe5d2800bdd157380b85 - nfs: fix lock variable type

we see a lot of "[global.glusterfs - usage-type (null) memusage]" in statedump
because lock status is not all-zeros after init, and the memcmp to check that
a datatype is never allocated is invalid.

Fix:
Changed if a datatype is allocated or not check based on total_allocs. Also
removed setting typestr to NULL on gf_free even when num_allocs is 0. Because
even that is leading to 'null' memusage string to be printed in statedump.

BUG: 1329870
Change-Id: If2b01a557cbdc787625db32e276e06cee3ac46ee
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14054
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

libglusterfs: coverity fix

2016-04-14T12:55:50+00:00

fix missing varargs cleanup

CID 1124856: string overflow

CID 1124656: NULL return

CID 1124374: constant expression

Change-Id: Iead530c599bdfef05a40c68b892215f4e4f02247
BUG: 789278
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/9630
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

mem-pool,stack,store,syncop,timer/libglusterfs : Porting to a new logging framework

2015-06-27T05:52:01+00:00

Change-Id: Idd3dcaf7eeea5207b3a5210676ce3df64153197f
BUG: 1194640
Signed-off-by: Mohamed Ashiq 
Reviewed-on: http://review.gluster.org/10827
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: NetBSD Build System

core: use reference counting for mem_acct structures

2015-05-09T13:28:13+00:00

When freeing memory, our memory-accounting code expects to be able to
dereference from the (previously) allocated block to its owning
translator.  However, as we have already found once in option
validation and twice in logging, that translator might itself have
been freed and the dereference attempt causes on of our daemons to
crash with SIGSEGV.  This patch attempts to fix that as follows:

 * We no longer embed a struct mem_acct directly in a struct xlator,
   but instead allocate it separately.

 * Allocated memory blocks now contain a pointer to the mem_acct
   instead of the xlator.

 * The mem_acct structure contains a reference count, manipulated in
   both the normal and translator allocate/free code using atomic
   increments and decrements.

 * Because it's now a separate structure, we can defer freeing the
   mem_acct until its reference count reaches zero (either way).

 * Some unit tests were disabled, because they embedded their own
   copies of the implementation for what they were supposedly testing.
   Life's too short to spend time fixing tests that seem designed to
   impede progress by requiring a certain implementation as well as
   behavior.

Change-Id: Id929b11387927136f78626901729296b6c0d0fd7
BUG: 1211749
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/10417
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Niels de Vos 
Reviewed-by: Pranith Kumar Karampuri