glusterfs.git/xlators/performance/md-cache, branch v3.3.2qa3

performance/md-cache: use mdc_fsetattr_cbk as the callback if setattr is on a fd

2012-05-23T09:04:23+00:00

Change-Id: I114a16055540e0cd3317b83b329600251ffe03c3
BUG: 823886
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.com/3404
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Vijay Bellur

license: dual license under GPLV2 and LGPLV3+

2012-05-11T05:06:16+00:00

Note that the license was not changed in any of the following:
	.../argp-standalone/...
        .../booster/...
        .../cli/...
        .../contrib/...
        .../extras/...
        .../glusterfsd/...
        .../glusterfs-hadoop/...
        .../mod_clusterfs/...
        .../scheduler/...
        .../swift/...

The license was not changed in any of the non-building xlators. The
license was not changed in any of the xlators that seemed — to me — to
be clearly server-side only, e.g. protocol/server

Note too that copyright was changed along with the license; I did
not change the copyright in files where the license did not change.

If you find any errors or ommissions please don't hesitate to let me know.

The complete list of files with the license change is:
	libglusterfs/src/byte-order.h
	libglusterfs/src/call-stub.c
	libglusterfs/src/call-stub.h
	libglusterfs/src/checksum.c
	libglusterfs/src/checksum.h
	libglusterfs/src/circ-buff.c
	libglusterfs/src/circ-buff.h
	libglusterfs/src/common-utils.c
	libglusterfs/src/common-utils.h
	libglusterfs/src/compat-errno.c
	libglusterfs/src/compat-errno.h
	libglusterfs/src/compat.c
	libglusterfs/src/compat.h
	libglusterfs/src/daemon.c
	libglusterfs/src/daemon.h
	libglusterfs/src/defaults.c
	libglusterfs/src/defaults.h
	libglusterfs/src/dict.c
	libglusterfs/src/dict.h
	libglusterfs/src/event-history.c
	libglusterfs/src/event-history.h
	libglusterfs/src/event.c
	libglusterfs/src/event.h
	libglusterfs/src/fd-lk.c
	libglusterfs/src/fd-lk.h
	libglusterfs/src/fd.c
	libglusterfs/src/fd.h
	libglusterfs/src/gf-dirent.c
	libglusterfs/src/gf-dirent.h
	libglusterfs/src/globals.c
	libglusterfs/src/globals.h
	libglusterfs/src/glusterfs.h
	libglusterfs/src/graph-print.c
	libglusterfs/src/graph-utils.h
	libglusterfs/src/graph.c
	libglusterfs/src/hashfn.c
	libglusterfs/src/hashfn.h
	libglusterfs/src/iatt.h
	libglusterfs/src/inode.c
	libglusterfs/src/inode.h
	libglusterfs/src/iobuf.c
	libglusterfs/src/iobuf.h
	libglusterfs/src/latency.c
	libglusterfs/src/latency.h
	libglusterfs/src/list.h
	libglusterfs/src/lkowner.h
	libglusterfs/src/locking.h
	libglusterfs/src/logging.c
	libglusterfs/src/logging.h
	libglusterfs/src/mem-pool.c
	libglusterfs/src/mem-pool.h
	libglusterfs/src/mem-types.h
	libglusterfs/src/options.c
	libglusterfs/src/options.h
	libglusterfs/src/rbthash.c
	libglusterfs/src/rbthash.h
	libglusterfs/src/run.c
	libglusterfs/src/run.h
	libglusterfs/src/scheduler.c
	libglusterfs/src/scheduler.h
	libglusterfs/src/stack.c
	libglusterfs/src/stack.h
	libglusterfs/src/statedump.c
	libglusterfs/src/statedump.h
	libglusterfs/src/syncop.c
	libglusterfs/src/syncop.h
	libglusterfs/src/syscall.c
	libglusterfs/src/syscall.h
	libglusterfs/src/timer.c
	libglusterfs/src/timer.h
	libglusterfs/src/trie.c
	libglusterfs/src/trie.h
	libglusterfs/src/xlator.c
	libglusterfs/src/xlator.h
	libglusterfsclient/src/libglusterfsclient-dentry.c
	libglusterfsclient/src/libglusterfsclient-internals.h
	libglusterfsclient/src/libglusterfsclient.c
	libglusterfsclient/src/libglusterfsclient.h
	rpc/rpc-lib/src/auth-glusterfs.c
	rpc/rpc-lib/src/auth-null.c
	rpc/rpc-lib/src/auth-unix.c
	rpc/rpc-lib/src/protocol-common.h
	rpc/rpc-lib/src/rpc-clnt.c
	rpc/rpc-lib/src/rpc-clnt.h
	rpc/rpc-lib/src/rpc-transport.c
	rpc/rpc-lib/src/rpc-transport.h
	rpc/rpc-lib/src/rpcsvc-auth.c
	rpc/rpc-lib/src/rpcsvc-common.h
	rpc/rpc-lib/src/rpcsvc.c
	rpc/rpc-lib/src/rpcsvc.h
	rpc/rpc-lib/src/xdr-common.h
	rpc/rpc-lib/src/xdr-rpc.c
	rpc/rpc-lib/src/xdr-rpc.h
	rpc/rpc-lib/src/xdr-rpcclnt.c
	rpc/rpc-lib/src/xdr-rpcclnt.h
	rpc/rpc-transport/rdma/src/name.c
	rpc/rpc-transport/rdma/src/name.h
	rpc/rpc-transport/rdma/src/rdma.c
	rpc/rpc-transport/rdma/src/rdma.h
	rpc/rpc-transport/socket/src/name.c
	rpc/rpc-transport/socket/src/name.h
	rpc/rpc-transport/socket/src/socket.c
	rpc/rpc-transport/socket/src/socket.h
	xlators/cluster/afr/src/afr-common.c
	xlators/cluster/afr/src/afr-dir-read.c
	xlators/cluster/afr/src/afr-dir-read.h
	xlators/cluster/afr/src/afr-dir-write.c
	xlators/cluster/afr/src/afr-dir-write.h
	xlators/cluster/afr/src/afr-inode-read.c
	xlators/cluster/afr/src/afr-inode-read.h
	xlators/cluster/afr/src/afr-inode-write.c
	xlators/cluster/afr/src/afr-inode-write.h
	xlators/cluster/afr/src/afr-lk-common.c
	xlators/cluster/afr/src/afr-mem-types.h
	xlators/cluster/afr/src/afr-open.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.h
	xlators/cluster/afr/src/afr-self-heal-common.c
	xlators/cluster/afr/src/afr-self-heal-common.h
	xlators/cluster/afr/src/afr-self-heal-data.c
	xlators/cluster/afr/src/afr-self-heal-entry.c
	xlators/cluster/afr/src/afr-self-heal-metadata.c
	xlators/cluster/afr/src/afr-self-heal.h
	xlators/cluster/afr/src/afr-self-heald.c
	xlators/cluster/afr/src/afr-self-heald.h
	xlators/cluster/afr/src/afr-transaction.c
	xlators/cluster/afr/src/afr-transaction.h
	xlators/cluster/afr/src/afr.c
	xlators/cluster/afr/src/afr.h
	xlators/cluster/afr/src/pump.c
	xlators/cluster/afr/src/pump.h
	xlators/cluster/dht/src/dht-common.c
	xlators/cluster/dht/src/dht-common.h
	xlators/cluster/dht/src/dht-diskusage.c
	xlators/cluster/dht/src/dht-hashfn.c
	xlators/cluster/dht/src/dht-helper.c
	xlators/cluster/dht/src/dht-inode-read.c
	xlators/cluster/dht/src/dht-inode-write.c
	xlators/cluster/dht/src/dht-layout.c
	xlators/cluster/dht/src/dht-linkfile.c
	xlators/cluster/dht/src/dht-mem-types.h
	xlators/cluster/dht/src/dht-rebalance.c
	xlators/cluster/dht/src/dht-rename.c
	xlators/cluster/dht/src/dht-selfheal.c
	xlators/cluster/dht/src/dht.c
	xlators/cluster/dht/src/nufa.c
	xlators/cluster/dht/src/switch.c
	xlators/cluster/stripe/src/stripe-helpers.c
	xlators/cluster/stripe/src/stripe-mem-types.h
	xlators/cluster/stripe/src/stripe.c
	xlators/cluster/stripe/src/stripe.h
	xlators/features/index/src/index-mem-types.h ¹
	xlators/features/index/src/index.c ¹
	xlators/features/index/src/index.h ¹
	xlators/performance/io-cache/src/io-cache.c
	xlators/performance/io-cache/src/io-cache.h
	xlators/performance/io-cache/src/ioc-inode.c
	xlators/performance/io-cache/src/ioc-mem-types.h
	xlators/performance/io-cache/src/page.c
	xlators/performance/io-threads/src/io-threads.c
	xlators/performance/io-threads/src/io-threads.h
	xlators/performance/io-threads/src/iot-mem-types.h
	xlators/performance/md-cache/src/md-cache-mem-types.h
	xlators/performance/md-cache/src/md-cache.c
	xlators/performance/quick-read/src/quick-read-mem-types.h
	xlators/performance/quick-read/src/quick-read.c
	xlators/performance/quick-read/src/quick-read.h
	xlators/performance/read-ahead/src/page.c
	xlators/performance/read-ahead/src/read-ahead-mem-types.h
	xlators/performance/read-ahead/src/read-ahead.c
	xlators/performance/read-ahead/src/read-ahead.h
	xlators/performance/symlink-cache/src/symlink-cache.c
	xlators/performance/write-behind/src/write-behind-mem-types.h
	xlators/performance/write-behind/src/write-behind.c
	xlators/protocol/auth/addr/src/addr.c ¹
	xlators/protocol/auth/login/src/login.c ¹
	xlators/protocol/client/src/client-callback.c
	xlators/protocol/client/src/client-handshake.c
	xlators/protocol/client/src/client-helpers.c
	xlators/protocol/client/src/client-lk.c
	xlators/protocol/client/src/client-mem-types.h
	xlators/protocol/client/src/client.c
	xlators/protocol/client/src/client.h
	xlators/protocol/client/src/client3_1-fops.c

¹ Copyright only, license reverted to original

Change-Id: If560e826c61b6b26f8b9af7bed6e4bcbaeba31a8
BUG: 820551
Signed-off-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.com/3304
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

quick-read, md-cache: selectively cache xattr data to conserve memory

2012-05-08T22:34:01+00:00

The md-cache translator can cache items for a long period of time and
thus is sensitive to memory usage associated with the items it
references. This implements two changes to help conserve memory:

- quick-read - Migrate the file content data to a local dict and
	delete from the xdata dict that passes through the stack.
- md-cache - Create a local dict to selectively store the xattr
	data md-cache is interested in. This includes a slight
	optimization to not allocate an empty dict in the case
	where we have not received any xattr's of interest.

I've tested both changes independently and together by running a
readdirp test against several compiled source trees (~340k files).
The base test results in a 7.7GB RSS on the client. The quick-read
modification cuts RSS down to 4.1GB, which is still large due
to md-cache unintelligently caching a large number of empty
dictionaries. The combined modification cuts RSS down to 462MB
on the same workload.

NOTE: Theoretically the md-cache change should supercede the
quick-read change, but practically I save an extra 150MB or so with
both. I already had the change and consider it an improvement that
quick-read clean up after itself.

BUG: 812876
Change-Id: Id59734d12dd6476b0e32480939e633448adb6884
Signed-off-by: Brian Foster 
Reviewed-on: http://review.gluster.com/3268
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Amar Tumballi 
Reviewed-by: Jeff Darcy

md-cache: fix dict memory leak in mdc_inode_xatt_update

2012-04-12T21:21:39+00:00

We currently either dict_ref() or dict_copy() into mdc->xattr.
The double dict_ref() call path is obvious. Alternatively,
dict_copy() does not ref the dict in the situation where it
creates a new dict, but we only call dict_copy() in this
context when we already have a dict, which should mean the
extra ref is unnecessary.

BUG: 801582
Change-Id: I72e3f1a16f942f04752c033812d13d50ee7c2c2c
Signed-off-by: Brian Foster 
Reviewed-on: http://review.gluster.com/3138
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

md-cache: fix time comparison to support 0 timeout

2012-04-02T18:25:12+00:00

The time comparison currently causes caching when the current
time matches the timeout. This causes caching to occur even
when the timeout parameter is set to 0. Fix the condition to
support the no caching scenario. A longer term approach is to
use gettimeofday() for more accurate granularity, but this
apparently fixes a known bug.

Change-Id: I6552c5961743edb22a1aec2e6a3a6e041d0647b2
BUG: 809193
Signed-off-by: Brian Foster 
Reviewed-on: http://review.gluster.com/3066
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

core: adding extra data for fops

2012-03-22T23:40:27+00:00

with this change, the xlator APIs will have a dictionary as extra
argument, which is passed between all the layers. This can be
utilized for overloading in some of the operations.

Change-Id: I58a8186b3ef647650280e63f3e5e9b9de7827b40
Signed-off-by: Amar Tumballi 
BUG: 782265
Reviewed-on: http://review.gluster.com/2960
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

Fix memory leaks found in readdir

2012-03-01T19:42:56+00:00

Change-Id: I0e221e4de9ee12586b09cd8bf7f394e9d4b88a11
BUG: 765785
Reviewed-on: http://review.gluster.com/2853
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Vijay Bellur

perf/md-cache: hold lock on modification of md_cache structure

2012-03-01T03:45:45+00:00

as it is possible that multiple fops can alter the content of
md-cache structure from inode-ctx, we need to safe-guard the
issue of corruption due to race conditions.

Change-Id: Iea051f8f6adff7690d6d60f3cf82eda75150b449
Signed-off-by: Amar Tumballi 
BUG: 798179
Reviewed-on: http://review.gluster.com/2834
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/md-cache: Change strdup to gf_strdup

2012-02-21T10:28:54+00:00

Change-Id: Ic96e7d38b988efab49f723c9d1d46f65037a84ed
BUG: 795682
Reviewed-on: http://review.gluster.com/2783
Reviewed-by: Anand Avati 
Tested-by: Gluster Build System

performance/md-cache: Add mem_acct_init

2012-02-21T07:22:12+00:00

Change-Id: I1b91ee0bde160762548c39646ceb96c3c778e06c
BUG: 795651
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.com/2778
Reviewed-by: Vijay Bellur 
Tested-by: Gluster Build System