glusterfs.git/xlators/features/marker, branch v3.7.0beta2

quota/marker: turn off inode quotas by default

2015-05-08T12:42:20+00:00

inode quota is a new feature implemented in glusterfs-3.7
if quota is enabled in the older version and is upgraded
to a new version, we can hit setxattr spike during self-heal
of inode quotas. So, when a quota is enabled, turn off
inode-quotas with a xlator option.

With this patch, we still account for inode quotas but only
when a write operation is performed for a particular file.
User will be able to query inode quotas once the Inode-quota
xlator option is enabled.

Change-Id: I52fb28bf7024989ce7bb08ac63a303bf3ec1ec9a
BUG: 1218243
Signed-off-by: vmallika 
Signed-off-by: Sachin Pandit 
Reviewed-on: http://review.gluster.org/10152
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Signed-off-by: Sachin Pandit 
Reviewed-on: http://review.gluster.org/10621

quota: Validate NULL inode from the entries received in readdirp_cbk

2015-05-02T09:35:51+00:00

This is a backport of http://review.gluster.org/#/c/10416/

> In quota readdirp_cbk, inode ctx filled for the all entries
> received.
> In marker readdirp_cbk, files/directories are inspected for
> dirty
>
> There is no guarantee that entry->inode is populated.
> If entry->inode is NULL, this needs to be treated as readdir
>
> Change-Id: Id2d17bb89e4770845ce1f13d73abc2b3c5826c06
> BUG: 1215550
> Signed-off-by: vmallika 
> Reviewed-on: http://review.gluster.org/10416
> Tested-by: NetBSD Build System
> Tested-by: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Tested-by: Raghavendra G 

Change-Id: I6e01b16bae1555efb6c20e49c9c473d81c6d46a0
BUG: 1217406
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/10468
Reviewed-by: Raghavendra G 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

quota/marker: fix inode quota healing after glusterfs upgrade

2015-04-24T11:47:32+00:00

This is a backport of http://review.gluster.org/10239

> There is a problem during upgrade where, inode quotas are not healed in
> the contri xattrs.
> Healing happens if contri xattrs are missing.
> But healing doesn't happen if contri xattrs are present and inode quota
> values are missing in the contri xattrs.
>
> This patch fixes the problem
>
> Change-Id: I6c88b74b5bb333a97c5419e24cc4ada82839f474
> BUG: 1211808
> Signed-off-by: vmallika 
> Reviewed-on: http://review.gluster.org/10239
> Tested-by: NetBSD Build System
> Reviewed-by: Sachin Pandit 
> Tested-by: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Tested-by: Raghavendra G 

Change-Id: I3245ee62fd69b60ec56ebe28043c273dbb58dd58
BUG: 1212348
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/10265
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

build: make contrib/uuid dependency optional

2015-04-10T11:39:52+00:00

On Linux systems we should use the libuuid from the distribution and not
bundle and statically link the contrib/uuid/ bits.

libglusterfs/src/compat-uuid.h has been introduced and should become an
abstraction layer for different UUID APIs. Non-Linux operating systems
should implement their compatibility layer there.

Once all operating systems have an implementation in compat-uuid.h, we
can remove contrib/uuid/ from the repository completely.

Change-Id: I345e5357644be2521685e00358bb8c83c4ea0577
BUG: 1206587
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10129
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

libglusterfs/syncop: Add xdata to all syncop calls

2015-04-08T15:14:59+00:00

This patch adds support for xdata in both the
request and response path of syncops.

Few calls like lookup already had the support;
have renamed variables in few places to maintain
uniformity.

xdata passed downwards is known as xdata_in
and xdata passed upwards is known as xdata_out.

There is an old patch by Jeff Darcy at
http://review.gluster.org/#/c/8769/3 which does the
same for some selected calls. It also brings in
xdata support at gfapi level.

xdata support at gfapi level would be introduced
in subsequent patches.

Change-Id: I340e94ebaf2a38e160e65bc30732e8fe1c532dcc
BUG: 1158621
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/9859
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Avoid conflict between contrib/uuid and system uuid

2015-04-04T17:48:35+00:00

glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.

Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.

A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.

BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

marker, quota: Fixing the coverity issues

2015-03-31T08:09:53+00:00

Coverity CIDs:
1124601
1124690
1134008
1134009

Change-Id: I93992b11bb6d8f7edd065f602aec2cd7a8b433d0
BUG: 789278
Signed-off-by: Nandaja Varma 
Reviewed-on: http://review.gluster.org/9614
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/quota : Introducing inode quota

2015-03-19T01:24:12+00:00

==========================================================================
                             Inode quota
==========================================================================
= Currently, the only way to retrieve the number of files/objects in a   =
= directory or volume is to do a crawl of the entire directory/volume.   =
= This is expensive and is not scalable.                                 =
=                                                                        =
= The proposed mechanism will provide an easier alternative to determine =
= the count of files/objects in a directory or volume.                   =
=                                                                        =
= The new mechanism proposes to store count of objects/files as part of  =
= an extended attribute of a directory. Each directory's extended        =
= attribute value will indicate the number of files/objects present      =
= in a tree with the directory being considered as the root of the tree. =
=                                                                        =
= The count value can be accessed by performing a getxattr().            =
= Cluster translators like afr, dht and stripe will perform aggregation  =
= of count values from various bricks when getxattr() happens on the key =
= associated with file/object count.                                     =

A new interface is introduced:
------------------------------
        limit-objects  : limit the number of inodes at directory level
        list-objects   : list the directories where the limit is set
        remove-objects : remove the limit from the directory

==========================================================================

CLI COMMAND:
gluster volume quota  limit-objects   []

*  is a hard-limit for number of objects limitation for path ""
  If hard-limit is exceeded, creation of file/directory is no longer
permitted.

*  is a soft-limit for number of objects creation for path ""
  If soft-limit is exceeded, a warning is issued for each creation.

CLI COMMAND:
gluster volume quota  remove-objects [path]

==========================================================================

CLI COMMAND:
gluster volume quota  list-objects [path] ...

Sample output:
------------------
  Path                   Hard-limit Soft-limit   Used  Available
Soft-limit exceeded?
Hard-limit exceeded?
  ------------------------------------------------------------------------
--------------------------------------
  /dir                      10       80%          10       0
Yes
        Yes

==========================================================================

[root@snapshot-28 dir]# ls
a  b  file11  file12  file13  file14  file15  file16  file17
[root@snapshot-28 dir]# touch a1
touch: cannot touch `a1': Disk quota exceeded
* Nine files are created in directory "dir" and directory is included in
* the
count too. Hence the limit "10" is reached and further file creation
fails

==========================================================================

Note: We have also done some re-factoring in cli for volume name
validation. New function cli_validate_volname is created

==========================================================================

Change-Id: I1823497de4f790a2a20ebb1770293472ea33ee2b
BUG: 1190108
Signed-off-by: Sachin Pandit 
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/9769
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

marker: fix compile time warning on buf arg.

2015-03-18T14:43:56+00:00

Problem:
marker-quota.c: In function 'mq_inspect_directory_xattr_task':
marker-quota.c:3451:31: warning: variable 'buf' set but not
used [-Wunused-but-set-variable]
         struct iatt           buf                          = {0,};

Change-Id: I211378328bdb2509a5d2a186d173f7f30a670c8a
BUG: 1198849
Signed-off-by: Humble Devassy Chirammal 
Reviewed-on: http://review.gluster.org/9928
Reviewed-by: Vijaikumar Mallikarjuna 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System

Quota: Build ancestry in the lookup

2015-03-18T13:59:54+00:00

Marker can fail or can account incorrect numbers when it doesn't find a
ancestry for a inode.

Solution:
Current build_ancestry is done only on demand in the write/create FOPs
in quota enforcer.
It is good to do this in the quota_lookup as well.

Change-Id: I8aaf5b3e05a3ca51e7ab1eaa1b636a90f659a872
BUG: 1184885
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/9478
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Vijay Bellur