nfs: Introduce tunable for memory consumption

NFS has used a common define called the mem-factor to determine
how much memory is allocated for the following items:

o inode table
o local structures used by nfs for storing fop state
o local structures used by nfs3 for storing nfs op state

These are all allocated through the mem-pool. The factor is multiplied by
a multiple that is specific to each data structure. For eg.

define GF_NFS_CONCURRENT_OPS_MULT     15
define GF_NFS_INODE_LRU_MULT          6000

The first value is used for allocating a mem-pool for storing
state for each fop or nfs op that is currently being handled. Knowing that linux
allows at most 128 in-flight requests, this multiple combined with the default
mem-factor of 15 gives us 225 slots in the mem-pool for the local structures.

Similarly, 6000*15 gives us a space of 90k inodes in the lru.

That means, increasing the common mem-factor will allow increasing the perf
under some conditions. This patch introduces the mem-factor as a configurable
option.

Signed-off-by: Shehjar Tikoo <shehjart@gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>

BUG: 2277 (Regression in Gluster NFS re-read performance)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2277

1 files changed, 27 insertions, 1 deletions

diff --git a/xlators/nfs/server/src/nfs.c b/xlators/nfs/server/src/nfs.c
index 979fd2fb04d..f7def106005 100644
--- a/xlators/nfs/server/src/nfs.c
+++ b/xlators/nfs/server/src/nfs.c
@@ -493,6 +493,22 @@ nfs_init_state (xlator_t *this)
         }
 
         nfs->memfactor = GF_NFS_DEFAULT_MEMFACTOR;
+        if (dict_get (this->options, "nfs.mem-factor")) {
+                ret = dict_get_str (this->options, "nfs.mem-factor",
+                                    &optstr);
+                if (ret < 0) {
+                        gf_log (GF_NFS, GF_LOG_ERROR, "Failed to parse dict");
+                        goto free_foppool;
+                }
+
+                ret = gf_string2uint (optstr, &nfs->memfactor);
+                if (ret < 0) {
+                        gf_log (GF_NFS, GF_LOG_ERROR, "Failed to parse uint "
+                                "string");
+                        goto free_foppool;
+                }
+        }
+
         fopspoolsize = nfs->memfactor * GF_NFS_CONCURRENT_OPS_MULT;
         /* FIXME: Really saddens me to see this as xlator wide. */
         nfs->foppool = mem_pool_new (struct nfs_fop_local, fopspoolsize);
@@ -897,6 +913,16 @@ struct volume_options options[] = {
           .description = "Use this option on systems that need Gluster NFS to "
                          "be associated with a non-default port number."
         },
-	{ .key  = {NULL} },
+        { .key  = {"nfs.mem-factor"},
+          .type = GF_OPTION_TYPE_INT,
+          .description = "Use this option to make NFS be faster on systems by "
+                         "using more memory. This option specifies a multiple "
+                         "that determines the total amount of memory used. "
+                         "Default value is 15. Increase to use more memory in "
+                         "order to improve performance for certain use cases."
+                         "Please consult gluster-users list before using this "
+                         "option."
+        },
+        { .key  = {NULL} },
 };