| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The md-cache translator can cache items for a long period of time and
thus is sensitive to memory usage associated with the items it
references. This implements two changes to help conserve memory:
- quick-read - Migrate the file content data to a local dict and
delete from the xdata dict that passes through the stack.
- md-cache - Create a local dict to selectively store the xattr
data md-cache is interested in. This includes a slight
optimization to not allocate an empty dict in the case
where we have not received any xattr's of interest.
I've tested both changes independently and together by running a
readdirp test against several compiled source trees (~340k files).
The base test results in a 7.7GB RSS on the client. The quick-read
modification cuts RSS down to 4.1GB, which is still large due
to md-cache unintelligently caching a large number of empty
dictionaries. The combined modification cuts RSS down to 462MB
on the same workload.
NOTE: Theoretically the md-cache change should supercede the
quick-read change, but practically I save an extra 150MB or so with
both. I already had the change and consider it an improvement that
quick-read clean up after itself.
BUG: 812876
Change-Id: Id59734d12dd6476b0e32480939e633448adb6884
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.com/3268
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We currently either dict_ref() or dict_copy() into mdc->xattr.
The double dict_ref() call path is obvious. Alternatively,
dict_copy() does not ref the dict in the situation where it
creates a new dict, but we only call dict_copy() in this
context when we already have a dict, which should mean the
extra ref is unnecessary.
BUG: 801582
Change-Id: I72e3f1a16f942f04752c033812d13d50ee7c2c2c
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.com/3138
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The time comparison currently causes caching when the current
time matches the timeout. This causes caching to occur even
when the timeout parameter is set to 0. Fix the condition to
support the no caching scenario. A longer term approach is to
use gettimeofday() for more accurate granularity, but this
apparently fixes a known bug.
Change-Id: I6552c5961743edb22a1aec2e6a3a6e041d0647b2
BUG: 809193
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.com/3066
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
with this change, the xlator APIs will have a dictionary as extra
argument, which is passed between all the layers. This can be
utilized for overloading in some of the operations.
Change-Id: I58a8186b3ef647650280e63f3e5e9b9de7827b40
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 782265
Reviewed-on: http://review.gluster.com/2960
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I0e221e4de9ee12586b09cd8bf7f394e9d4b88a11
BUG: 765785
Reviewed-on: http://review.gluster.com/2853
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendrabhat@gluster.com>
Reviewed-by: Krishnan Parthasarathi <kp@gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pranithk@gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
as it is possible that multiple fops can alter the content of
md-cache structure from inode-ctx, we need to safe-guard the
issue of corruption due to race conditions.
Change-Id: Iea051f8f6adff7690d6d60f3cf82eda75150b449
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 798179
Reviewed-on: http://review.gluster.com/2834
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: Ic96e7d38b988efab49f723c9d1d46f65037a84ed
BUG: 795682
Reviewed-on: http://review.gluster.com/2783
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I1b91ee0bde160762548c39646ceb96c3c778e06c
BUG: 795651
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/2778
Reviewed-by: Vijay Bellur <vijay@gluster.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
- preserve CLI set option key as "performance.stat-prefetch"
- create a symlink stat-prefetch.so to point to md-cache.so
Change-Id: Ib95e7c30073f13ae04c39e9466967ba1db5a0614
BUG: 765785
Reviewed-on: http://review.gluster.com/2714
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
This is a metadata caching translator which is well integrated with
glusterfs core framework and leverages some of the recent protocol
changes to do a better job at caching.
- It uses the attributes returned along callbacks of all calls to
update its attribute cache as frequently as possible without
issuing calls on its own (i.e, very low overhead)
- It caches attributes returned via readdirp into the inode contexts
corresponding to those entries (i.e, well integrated)
- It caches and updates xattrs and not just inode attributes (i.e,
eliminates the need for a separate xattr-prefetch translator)
In its current form it has a timeout based consistency model
Change-Id: I891f6225c1a4c08bb111e287571d5f6d326dbe97
BUG: 765785
Reviewed-on: http://review.gluster.com/2713
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|