glusterfs.git/xlators/performance, branch v3.11.0

nl-cache: Remove the max limit for nl-cache-limit and nl-cache-timeout

2017-05-26T16:34:17+00:00

The max limit is better unset when arbitrary. Otherwise in the future
if max has to be changed, it can break backward compatility.

>Reviewed-on: https://review.gluster.org/17261
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit 64f41b962b643b966e376a10a16671c569bf6299)

Change-Id: I4337a3789a2d0d5cc8e2bf687a22536c97608461
BUG: 1453152
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17400
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

nl-cache: In case of nameless operations do not cache

2017-05-25T15:30:34+00:00

Issue:
In nameless lookup/other fops, parent inode will be NULL, when we try
to add the cache to the NULL inode, it causes a crash.

Hence handle the scenario of nameless fops, and do not cache/serve
the nameless fops.

>Reviewed-on: https://review.gluster.org/17316
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Pranith Kumar Karampuri 
>CentOS-regression: Gluster Build System 
>(cherry picked from commit 284cd8851bfe60984d2f11b5c52fe3204ff43b06)

Change-Id: I3b90f882ac89e6aaf3419db89e6f890797f37700
BUG: 1454569
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17361
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

rda, glusterd: Change the max of rda-cache-limit to INFINITY

2017-05-22T15:05:43+00:00

Issue:
The max value of rda-cache-limit is 1GB before this patch.
When parallel-readdir is enabled, there will be many instances of
readdir-ahead, hence the rda-cache-limit depends on the number of
instances. Eg: On a volume with distribute count 4, rda-cache-limit
when parallel-readdir is enabled, will be 4GB instead of 1GB.
Consider a followinf sequence of operations:
- Enable parallel readdir
- Set rda-cache-limit to lets say 3GB
- Disable parallel-readdir, this results in one instance of readdir-ahead
  and the rda-cache-limit will be back to 1GB, but the current value is 3GB
  and hence the mount will stop working as 3GB > max 1GB.

Solution:
To fix this, we can limit the cache to 1GB even when parallel-readdir
is enabled. But there is no necessity to limit the cache to 1GB, it
can be increased if the system has enough resources. Hence getting rid
of the rda-cache-limit max value is more apt. If we just change the
rda-cache-limit max to INFINITY, we will render older(<3.11) clients
broken, when the rda-cache-limit is set to > 1GB (as the older clients
still expect a value < 1GB). To safely change the max value of
rda-cache-limit to INFINITY, add a check in glusted to verify all
the clients are > 3.11 if the value exceeds 1GB.

>Reviewed-on: https://review.gluster.org/17338
>Smoke: Gluster Build System 
>Reviewed-by: Atin Mukherjee 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>(cherry picked from commit e43b40296956d132c70ffa3aa07b0078733b39d4)

Change-Id: Id0cdda3b053287b659c7bf511b13db2e45b92032
BUG: 1453152
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17354
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

performance/read-ahead: prevent stale data being returned to application.

2017-05-18T14:53:06+00:00

Assume that fd is shared by two application threads/processes.

T0 read is triggered from app-thread t1 and read call passes through
   write-behind.
T1 app-thread t2 issues a write. The page on which read from t1 is
   waiting is marked stale
T2 write-behind caches write and indicates to application as write
   complete.
T3 app-thread t2 issues read to same region. Since, there is already a
   page for that region (created as part of read at T0), this read
   request waits on that page to be filled (though it is stale, which
   is a bug).
T4 read (triggered at T0) completes from brick (with write still
   pending). Now both read requests from t1 and t2 are served this data
   (though data is stale from app-thread t2's perspective - which is a
   bug)
T5 write is flushed to brick by write-behind.

Fix is to not to serve data from a stale page, but instead initiate a
fresh read to back-end.

>Change-Id: Id6af733464fa41bb4e81fd29c7451c73d06453fb
>BUG: 1414242
>Signed-off-by: Raghavendra G 
>Reviewed-on: https://review.gluster.org/7447
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Csaba Henk 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Zhou Zhengping 
>Reviewed-by: Amar Tumballi 

(cherry picked from commit 2ff39c5cbea6fbda0d7a442f55e6dc2a72efb171)
Change-Id: Id6af733464fa41bb4e81fd29c7451c73d06453fb
BUG: 1449311
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/17221
Reviewed-by: Zhou Zhengping 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

nl-cache: free nlc_conf_t in fini()

2017-05-16T00:29:12+00:00

The (xlator_t*)->private structure in negative-lookup-cache is allocated
in the init() function of the xlator, but never free'd. Valgrind
detected this as:

    656 bytes in 1 blocks are definitely lost in loss record X of Y
       at 0x..+ calloc (/builddir/build/BUILD/valgrind-3.11.0/coregrind/m_replacemalloc/vg_replace_malloc.c:711)
       by 0x.. __gf_calloc (/usr/src/debug/glusterfs-3.11dev/libglusterfs/src/mem-pool.c:117)
       by 0x.. init (/usr/src/debug/glusterfs-3.11dev/xlators/performance/nl-cache/src/nl-cache.c:669)
       by 0x.. __xlator_init (/usr/src/debug/glusterfs-3.11dev/libglusterfs/src/xlator.c:472)
       by 0x.. xlator_init (/usr/src/debug/glusterfs-3.11dev/libglusterfs/src/xlator.c:498)
       by 0x.. glusterfs_graph_init (/usr/src/debug/glusterfs-3.11dev/libglusterfs/src/graph.c:321)
       by 0x.. glusterfs_graph_activate (/usr/src/debug/glusterfs-3.11dev/libglusterfs/src/graph.c:693)
       by 0x.. glfs_process_volfp (/usr/src/debug/glusterfs-3.11dev/api/src/glfs-mgmt.c:79)
       by 0x.. glfs_volumes_init (/usr/src/debug/glusterfs-3.11dev/api/src/glfs.c:160)
       by 0x.. glfs_init_common (/usr/src/debug/glusterfs-3.11dev/api/src/glfs.c:868)
       by 0x.. glfs_init@@GFAPI_3.4.0 (/usr/src/debug/glusterfs-3.11dev/api/src/glfs.c:913)
       by 0x.. main (/root/gluster-debug/gfapi-load-volfile/gfapi-load-volfile.c:54)

When the xlators is unloaded, it should free the resources it allocated.
This can easily be done in the fini() function.

Cherry picked from commit d7e9dcfad228f385ad64526b1f06b55e98b06964:
> Change-Id: I079e78cc207145bc542e2282fc4cf2bb4dadc28a
> BUG: 1442569
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17143
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

Change-Id: I079e78cc207145bc542e2282fc4cf2bb4dadc28a
BUG: 1450267
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17263
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Zhou Zhengping 
Reviewed-by: Poornima G

core: make the per glusterfs_ctx_t timer-wheel refcounted

2017-05-12T13:32:32+00:00

xlators can use a 'global' timer-wheel for scheduling events. This
timer-wheel is managed per glusterfs_ctx_t, but does not need to be
allocated for every graph. When an xlator wants to use the timer-wheel,
it will be instanciated on demand, and provided to xlators that request
it later on.

By adding a reference counter to the glusterfs_ctx_t for the
timer-wheel, the threads and structures can be cleaned up when the last
xlator does not have a need for it anymore. In general, the xlators
request the timer-wheel in init(), and they should return it in fini().

Because the timer-wheel is managed per glusterfs_ctx_t, the functions
can be added to ctx.c and do not need to live in their very minimal
tw.[ch] files.


>Reported-by: Poornima G 
>Signed-off-by: Niels de Vos 
>Reviewed-on: https://review.gluster.org/17068
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Amar Tumballi 
>Reviewed-by: Zhou Zhengping 
>Reviewed-by: Kaleb KEITHLEY 
>(cherry picked from commit 73fcf3a874b2049da31d01b8363d1ac85c9488c2)

Change-Id: I19d225b39aaa272d9005ba7adc3104c3764f1572
BUG: 1450267
Reviewed-on: https://review.gluster.org/17262
Tested-by: Poornima G 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

nl-cache: Fix the issue in refd_inode counting and prune the cache

2017-04-26T09:02:06+00:00

Change-Id: I5b9beb8502667bc3876385900bc01b6491348716
BUG: 1442569
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17110
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

Implement negative lookup cache

2017-04-20T04:18:52+00:00

Before creating any file negative lookups(1 in Fuse, 4 in SMB etc.)
are sent to verify if the file already exists. By serving these
lookups from the cache when possible, increases the create
performance by multiple folds in SMB access and some percentage
in Fuse/NFS access.

Feature page: https://review.gluster.org/#/c/16436

Updates #82
Change-Id: Ib1c0e7ac7a386f943d84f6398c27f9a03665b2a4
BUG: 1442569
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16952
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

dht: The xattrs sent in readdirp should be sent in opendir aswell

2017-04-06T04:04:03+00:00

As readdir-ahead can be loaded as a child of dht, dht has to specify
the xattrs it is intrested in, as part of opendir call itself.

Change-Id: I012ef96cc143b0cef942df78aa7150d85ec38606
BUG: 1431908
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16902
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

libglusterfs: provide standardized atomic operations

2017-04-05T13:14:26+00:00

The current macros ATOMIC_INCREMENT() and ATOMIC_DECREMENT() expect a
lock as first argument. There are at least two issues with this
approach:

  1. this lock is unused on architectures that have atomic operations
  2. some structures use a single lock for multiple variables

By defining a gf_atomic_t type, the unused lock can be removed, saving a
few bytes on modern architectures.

Because the gf_atomic_t type locates the lock for the variable (in case
of older architectures), each variable is protected the same on all
architectures. This makes the behaviour across all architectures more
equal (per variable locking, by a gf_lock_t or compiler optimization).

BUG: 1437037
Change-Id: Ic164892b06ea676e6a9566f8a98b7faf0efe76d6
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16963
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Jeff Darcy