| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
| |
It was assumed that hard and soft limits are stored as two different
xattrs on disk. However they are stored as two members of a structure
which is stored as a value for a single key.
Change-Id: I947fa5c375209c31fe1511bda0d5cb0e249af9ba
BUG: 969461
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
| |
Change-Id: I38077c7adc497b314f4037cbbb116458a26ed589
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* hard and soft limits are persisted in xattrs of the
inode. Associating limits with inode instead of maintaining as a
global list helps us to scale better.
* quotad-aggregator acts as a special client to provide cluster view
through an rpc program. Quota enforcer loaded on brick uses this to get
aggregated directory sizes. Aggregated sizes are cached for a
timeout period in in-memory inode contexts.
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Change-Id: I2ab508d9d4fe224bc1d8cf01cf2b7969dd4200bb
BUG: 969461
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Old implementation
* Client side implementation of quota
- Not secure
- Increased traffic in updating the ctx
New Implementation
* 2 stages of quota implementation is done: Soft and hard quota
Upon reaching soft quota limit on the directory it logs/alerts in the quota
daemon log (ie DEFAULT_LOG_DIR/quotad.log) and no more writes allowed after
hard quota limit. After reaching the soft-limit the daemon alerts the
user/admin repeatively for every 'alert-time', which is configurable.
* Quota is moved to server-side.
There will be 2 quota xlators
i. Quota Server
It takes care of the enforcing the quota and maintains the context
specific to the brick. Since this doesn't have the complete picture of
the cluster, cluster wide usage is updated from the quota daemon. This
updated context is saved and used for the enforcement.
It updates its context by searching the QUOTA_UPDATE_KEY from the dict
in the setxattr call, and is updated from nowhere else.
The quota is always loaded in the server graph and is by passed if the
feature is not enabled.
Options specific to quota-server:
server-quota - Specifies whether the features is on/off. It is used
to by pass the quota if turned off.
deem-statfs - If set to on, it takes quota limits into
consideration while estimating fs size. (df command)
ii. Quota Daemon
This is the new xlator introduced with this patch. Its the
*gluster client* process with no mount point, started upon enabling
quota or restarting the volume. This is a single process for all the
volumes in the cluster. Its volfile stored in
GLUSTERD_DEFAULT_WORKI_DIR/quotad/quotad.vol.
It queries for the sizes on all the bricks, aggregates the size and
sends back the updated size, periodically. The timeout between
successive updation is configurable and typically/by default more for
below-soft-quota usage and less for above-soft-quota usage. It
maintains the timeout inside the limit structure based on the usage;
below soft limit and above soft limit.
There will be thread running per volume which iterates through the list
and decides whether the size to be queried in the current iteration
based on its timeout. It takes the next iteration time taking the least
of the timeouts in the list of entries.
Maintains a separate inode table for each volume in the quotad. In the
first iteration it builds the table for quota-dirs (dirs on which limit
is set) and its components.
Options specific to quotad:
hard-timeout - Timeout for updation of usage to the quota-server
when the usage is crosses the soft-limit.
soft-timeout - Timeout for the updation of usage to the
quota-server when the usage is below soft-limit.
alert-time - Frequency of logging after the usage reached
soft limit.
Options common to both:
default-soft-limit - This is used when individual paths are not
configured with soft-limit and default value of
this option is 90% of the hard-limit.
limit-set - String containing all the limits.
Thus in the current implementation we'll have 2 quota xlators: one in server
graph and one in trusted client (quota daemon) of which the sole
purpose will be to aggregate the quota size xattrs from all the bricks and
send the same to server quota xlator.
* Changes in glusterd and CLI
A single volfile is created for all the volumes, similar to nfs volfile.
All files related to quota client (volfile, pid etc) are stored in
GLUSTERD_DEFAULT_WORK_DIR/quotad/.
The new pattern of the quota limit stores in
limit-set = <single-dir-limit>[,<single-dir-limit>]
single-dir-limit = <abs-path>:<hard-limit>[:<soft-limit-in-percent>]
It also introduces new options:
volume quota <VOLNAME> {enable|disable|list [<path> ...]|remove <path>| default-soft-limit <percent>} |
volume quota <VOLNAME> {limit-usage <path> <size> |soft-limit <path> <percent>} |
volume quota <VOLNAME> {alert-time|soft-timeout|hard-timeout} {<time>}
Credit:
Raghavendra Bhat <rabhat@redhat.com>
Varun Shastry <vshastry@redhat.com>
Shishir Gowda <sgowda@redhat.com>
Kruthika Dhananjay <kdhananj@redhat.com>
Brian Foster <bfoster@redhat.com>
Krishnan Parthasarathi <kparthas@redhat.com>
Change-Id: I16ec5be0c2faaf42b14034b9ccaf17796adef082
BUG: 969461
Signed-off-by: Varun Shastry <vshastry@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds an option, features.quota-deem-statfs (default off) to consider the
quota limits while calculating the volume stats.
Eg: Backend is of size 10GB and limit set on / is 5GB. If the option is off
df show actual size to be 10GB and when it is on df shows 5GB.
Change-Id: Ib30733bb69afecce1dea9d0491af89d551d214cc
BUG: 905425
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/4511
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
QUOTA_STACK_UNWIND does not check if local is NULL before mem_put.
This leads to a callingfn log for every setxattr in quota.
Fix:
Made sure the mem_put happens only on non-null local in quota.
Additional changes:
Removed unused QUOTA_STACK_DESTROY macro.
Tests:
Peformed setfattr on the mount point and the Invalid argument
logs don't come anymore. Checked that the change does not introduce
any memory leak in quota.
Change-Id: Ie4534c7af0fff80e35d8510311055f930262594f
BUG: 856541
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.org/3936
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
License message changed for server-side, dual license GPLV2 and LGPLv3+.
Change-Id: Ia9e53061b9d2df3b3ef3bc9778dceff77db46a09
BUG: 852318
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/3940
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The license message is changed to
Copyright (c) 2008-2012 Red Hat, Inc. <http://www.redhat.com>
This file is part of GlusterFS.
This file is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3 or
later), or the GNU General Public License, version 2 (GPLv2), in all
cases as published by the Free Software Foundation.
Change-Id: I07d2b63ed5fbbbd1884f1e74f2dd56013d15b0f4
BUG: 852318
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/3858
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Till this patch, quota was relying on lookups after reconfigure, for
new limits to be effective. However, this might be restrictive for
users who don't want to change directories after resetting quota
limits.
Change-Id: I236aa31fdc7dcbc01d40d38c9b5b7b0a28a4d2ea
BUG: 802905
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.com/3022
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
in each translator, which uses 'frame->local', we are using
GF_CALLOC/GF_FREE, which would be costly considering the
number of allocation happening in a lifetime of 'fop'. It
would be good to utilize the mem pool framework for xlator's
local structures, so there is no allocation overhead.
Change-Id: Ida6e65039a24d9c219b380aa1c3559f36046dc94
Signed-off-by: Amar Tumballi <amar@gluster.com>
BUG: 765336
Reviewed-on: http://review.gluster.com/2772
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I0f078d1753db65d2f2e0380d1b0450c114cf40dd
BUG: 3518
Reviewed-on: http://review.gluster.com/522
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I2d10f2be44f518f496427f257988f1858e888084
BUG: 3348
Reviewed-on: http://review.gluster.com/200
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I3914467611e573cccee0d22df93920cf1b2eb79f
BUG: 3348
Reviewed-on: http://review.gluster.com/182
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
|
|
|
|
|
|
|
|
| |
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Signed-off-by: Anand Avati <avati@gluster.com>
BUG: 2604 (Quota: crossing the set limit)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2604
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- call-stub is resumed in fops calling quota_check_limit and quota_check_limit
when validate_count is zero, indicating no pending validates. During
validates, validate_count was decremented in quota_validate_cbk, but check
for validate_count being zero was done in quota_check_limit (which is
called by quota_validate_cbk). Hence there is a time window b/w decrementing
validate_count in quota_validate_cbk and checking whether
validate_count is zero in quota_check_limit, during which if the
control is passed to code in fop checking for validate_count being zero,
same stub will be resumed twice - once in fop and once in quota_check_limit.
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Signed-off-by: Anand Avati <avati@gluster.com>
BUG: 2686 ([glusterfs-3.2.0qa8]: nfs server crashed)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2686
|
|
|
|
|
|
|
|
|
|
|
|
| |
inode for files.
- It also fixes limit checks if limit set in the directory.
Signed-off-by: Junaid <junaid@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
BUG: 2604 (Quota: crossing the set limit)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2604
|
|
|
|
|
|
|
|
| |
Signed-off-by: Junaid <junaid@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
BUG: 2473 (Support for volume and directory level quota)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2473
|
|
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
BUG: 2473 (Support for volume and directory level quota)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2473
|