features/quota : Introducing inode quota

==========================================================================
                             Inode quota
==========================================================================
= Currently, the only way to retrieve the number of files/objects in a   =
= directory or volume is to do a crawl of the entire directory/volume.   =
= This is expensive and is not scalable.                                 =
=                                                                        =
= The proposed mechanism will provide an easier alternative to determine =
= the count of files/objects in a directory or volume.                   =
=                                                                        =
= The new mechanism proposes to store count of objects/files as part of  =
= an extended attribute of a directory. Each directory's extended        =
= attribute value will indicate the number of files/objects present      =
= in a tree with the directory being considered as the root of the tree. =
=                                                                        =
= The count value can be accessed by performing a getxattr().            =
= Cluster translators like afr, dht and stripe will perform aggregation  =
= of count values from various bricks when getxattr() happens on the key =
= associated with file/object count.                                     =

A new interface is introduced:
------------------------------
        limit-objects  : limit the number of inodes at directory level
        list-objects   : list the directories where the limit is set
        remove-objects : remove the limit from the directory

==========================================================================

CLI COMMAND:
gluster volume quota <volname> limit-objects <path> <number> [<percent>]

* <number> is a hard-limit for number of objects limitation for path "<path>"
  If hard-limit is exceeded, creation of file/directory is no longer
permitted.

* <percent> is a soft-limit for number of objects creation for path "<path>"
  If soft-limit is exceeded, a warning is issued for each creation.

CLI COMMAND:
gluster volume quota <volname> remove-objects [path]

==========================================================================

CLI COMMAND:
gluster volume quota <volname> list-objects [path] ...

Sample output:
------------------
  Path                   Hard-limit Soft-limit   Used  Available
Soft-limit exceeded?
Hard-limit exceeded?
  ------------------------------------------------------------------------
--------------------------------------
  /dir                      10       80%          10       0
Yes
        Yes

==========================================================================

[root@snapshot-28 dir]# ls
a  b  file11  file12  file13  file14  file15  file16  file17
[root@snapshot-28 dir]# touch a1
touch: cannot touch `a1': Disk quota exceeded
* Nine files are created in directory "dir" and directory is included in
* the
count too. Hence the limit "10" is reached and further file creation
fails

==========================================================================

Note: We have also done some re-factoring in cli for volume name
validation. New function cli_validate_volname is created

==========================================================================

Change-Id: I1823497de4f790a2a20ebb1770293472ea33ee2b
BUG: 1190108
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/9769
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>

1 files changed, 132 insertions, 0 deletions

diff --git a/tests/bugs/quota/inode-quota.t b/tests/bugs/quota/inode-quota.t
new file mode 100644
index 00000000000..12a18945d8b
--- /dev/null
+++ b/tests/bugs/quota/inode-quota.t
@@ -0,0 +1,132 @@
+#!/bin/bash
+
+. $(dirname $0)/../../include.rc
+. $(dirname $0)/../../volume.rc
+. $(dirname $0)/../../nfs.rc
+
+function quota_list_field () {
+        local QUOTA_PATH=$1
+        local FIELD=$2
+        $CLI volume quota $V0 list $QUOTA_PATH | grep $QUOTA_PATH\
+                                               | awk '{print $FIELD}'
+}
+
+function quota_object_list_field () {
+        local QUOTA_PATH=$1
+        local FIELD=$2
+        $CLI volume quota $V0 list-objects $QUOTA_PATH | grep $QUOTA_PATH\
+                                                       | awk '{print $FIELD}'
+}
+
+cleanup;
+
+TESTS_EXPECTED_IN_LOOP=9
+
+TEST glusterd
+TEST pidof glusterd
+
+# --------------------------------------------------
+# Create, start and mount a volume with single brick
+# --------------------------------------------------
+
+TEST $CLI volume create $V0 $H0:$B0/{V0}
+EXPECT "$V0" volinfo_field $V0 'Volume Name'
+EXPECT 'Created' volinfo_field $V0 'Status'
+
+TEST $CLI volume start $V0
+EXPECT 'Started' volinfo_field $V0 'Status'
+
+TEST $GFS -s $H0 --volfile-id $V0 $M0
+TEST mkdir -p $M0/test_dir
+
+#--------------------------------------------------------
+# Enable quota of the volume and set hard and soft timeout
+#------------------------------------------------------
+
+TEST $CLI volume quota $V0 enable
+EXPECT 'on' volinfo_field $V0 'features.quota'
+TEST $CLI volume quota $V0 soft-timeout 0
+EXPECT '0' volinfo_field $V0 'features.soft-timeout'
+TEST $CLI volume quota $V0 hard-timeout 0
+EXPECT '0' volinfo_field $V0 'features.hard-timeout'
+
+
+#-------------------------------------------------------
+# Set and remove quota limits on the directory and
+# verify if the limits are being reflected properly
+#------------------------------------------------------
+
+TEST $CLI volume quota $V0 limit-usage /test_dir 100MB
+EXPECT "100.0MB" quota_list_field "/test_dir" 2
+
+TEST $CLI volume quota $V0 limit-objects /test_dir 100
+EXPECT "100" quota_object_list_field "/test_dir" 2
+
+TEST $CLI volume quota $V0 remove /test_dir
+EXPECT "" quota_list_field "/test_dir" 2
+
+# Need to verify this once
+#TEST $CLI volume quota $V0 remove-objects /test_dir
+#EXPECT "" quota_object_list_field "/test_dir" 2
+
+# Set back the limits
+
+TEST $CLI volume quota $V0 limit-usage /test_dir 10MB
+EXPECT "10.0MB" quota_list_field "/test_dir" 2
+
+TEST $CLI volume quota $V0 limit-objects /test_dir 10
+EXPECT "10" quota_object_list_field "/test_dir" 2
+
+#-----------------------------------------------------
+# Check the quota enforcement mechanism for usage
+#-----------------------------------------------------
+
+# Compile the program which basically created a file
+# of required size
+TEST $CC $(dirname $0)/../../basic/quota.c -o $(dirname $0)/quota
+
+# try creating a 8MB file and it should fail
+TEST $(dirname $0)/quota $M0/test_dir/test1.txt '8388608'
+EXPECT_WITHIN $MARKER_UPDATE_TIMEOUT "8.0MB" quota_list_field "/test_dir" 2
+TEST rm -f $M0/test_dir/test1.txt
+EXPECT_WITHIN $MARKER_UPDATE_TIMEOUT "0Bytes" quota_list_field "/test_dir" 2
+
+# try creating a 15MB file and it should succeed
+TEST ! $(dirname $0)/quota $M0/test_dir/test2.txt '15728640'
+TEST rm -f $M0/test_dir/test2.txt
+
+
+#------------------------------------------------------
+# Check the quota enforcement mechanism for object count
+#-------------------------------------------------------
+
+# Try creating 9 files and it should succeed as object limit
+# is set to 10, since directory where limit is set is accounted
+# as well.
+
+for i in {1..9}; do
+        TEST_IN_LOOP touch $M0/test_dir/test$i.txt
+done
+EXPECT_WITHIN $MARKER_UPDATE_TIMEOUT "10" quota_object_list_field "/test_dir" 4
+
+# Check available limit
+EXPECT_WITHIN $MARKER_UPDATE_TIMEOUT "0" quota_object_list_field "/test_dir" 5
+
+# Check if hard-limit exceeded
+EXPECT "Yes" quota_object_list_field "/test_dir" 7
+
+# Check if soft-limit exceeded
+EXPECT "Yes" quota_object_list_field "/test_dir" 6
+
+# Creation of 11th file should throw out an error
+TEST ! touch $M0/test_dir/test11.txt
+
+TEST rm -rf $M0/test_dir/test*
+EXPECT_WITHIN $MARKER_UPDATE_TIMEOUT "0" quota_object_list_field "/test_dir" 4
+
+TEST $CLI volume quota $V0 remove-objects /test_dir
+
+TEST $CLI volume stop $V0
+TEST $CLI volume delete $V0
+
+cleanup;