glusterfs.git/libglusterfs/src/glusterfs.h, branch v3.7.9

fuse: Add a new mount option capability

2016-03-10T03:09:43+00:00

Originally all security.* xattrs were forbidden if selinux is disabled,
which was causing Samba's acl_xattr module to not work, as it would
store the NTACL in security.NTACL. To fix this http://review.gluster.org/#/c/12826/
was sent, which forbid only security.selinux. This opened up a getxattr
call on security.capability before every write fop and others.

Capabilities can be used without selinux, hence if selinux is disabled,
security.capability cannot be forbidden. Hence adding a new mount
option called capability.

Only when "--capability" or "--selinux" mount option is used,
security.capability is sent to the brick, else it is forbidden.

Backport of : http://review.gluster.org/#/c/13540/ &
              http://review.gluster.org/#/c/13653/

BUG: 1309462
Change-Id: Ib8d4f32d9f1458f4d71a05785f92b526aa7033ff
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13626
Tested-by: Vijay Bellur 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur

tier:delete the linkfile if data file creation fails

2016-02-22T14:53:07+00:00

If we are creating data file in a hot subvolume
then we will create a linkfile in cold subvolume.
Linkfile creation happens first. If linkfile creation
was successful and data file creation failed, then
linkfile in cold subvolume will become stale

This patch will delete the linkfile as well, if data
file creation fails

Also this code duplicates dht_create to make tier_create

backport of>
>Change-Id: I377a90dad47f288e9576c7323b23cf694a91a7a3
>BUG: 1290677
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/12948
>Reviewed-by: N Balachandran 
>Tested-by: NetBSD Build System 
>Reviewed-by: Raghavendra G 
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 

Change-Id: I3689e8ee387fb6731b4b3a7fe8f8190143482eaa
BUG: 1295359
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13161
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

tier:unlink during migration

2016-02-22T14:50:54+00:00

files deleted during promotion were not deleting as the
files are moving from hashed to non-hashed

On deleting a file that is undergoing promotion,
the unlink call is not sent to the dst file as the
hashed subvol == cached subvol. This causes
the file to reappear once the migration is complete.

This patch also fixes a problem with stale linkfile
deleting.

Backport of>
>Change-Id: I4b02a498218c9d8eeaa4556fa4219e91e7fa71e5
>BUG: 1282390
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/12829
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 

(cherry picked from commit b5de382afa8c5777e455c7a376fc4f1f01d782d1)

Change-Id: I951adb4d929926bcd646dd7574f7a2d41d57479d
BUG: 1282388
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/12991
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

glusterd/afr : Readdirp performance improvement

2015-12-21T09:14:24+00:00

        Backport of http://review.gluster.org/#/c/12467/

Add xlator options to index xlator with xattrs that
it needs to keep track of.

Change-Id: If818673be5e626f77e65cc3a340f8cdd624179c2
BUG: 1287531
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/12467
Tested-by: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
Tested-by: NetBSD Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Pranith Kumar Karampuri 
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/12848

glusterd: cli command implementation for bitrot scrub status

2015-11-23T03:53:58+00:00

This patch is backport of: http://review.gluster.org/10231

CLI command  for bitrot scrub status will be :

gluster volume bitrot  scrub status

Above command will show the statistics of bitrot scrubber.

Upon execution of this command it will show some common
scrubber tunable value of volume  followed by
statistics of scrubber statistics of individual nodes.

sample ouput for single node:

Volume name : 

State of scrub: Active

Scrub frequency: biweekly

Bitrot error log location: /var/log/glusterfs/bitd.log

Scrubber error log location: /var/log/glusterfs/scrub.log

=========================================================

Node name:

Number of Scrubbed files:

Number of Unsigned files:

Last completed scrub time:

Duration of last scrub:

Error count:

=========================================================

This is just infrastructure. list of bad file, last scrub
time, error count value will be taken care by
http://review.gluster.org/#/c/12503/  and
http://review.gluster.org/#/c/12654/ patches.

    >> Change-Id: I3ed3c7057c9d0c894233f4079a7f185d90c202d1
    >> BUG: 1207627
    >> Signed-off-by: Gaurav Kumar Garg 
    >> Reviewed-on: http://review.gluster.org/10231
    >> Reviewed-by: Atin Mukherjee 
    >> Tested-by: NetBSD Build System 
    >> Tested-by: Gluster Build System 

Change-Id: I45ed94e5e0e78a1e007c30eb0b252f74cf3c9187
BUG: 1283881
Signed-off-by: Gaurav Kumar Garg 
Reviewed-on: http://review.gluster.org/12704
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee

marker: do remove xattr only for last link

2015-11-09T11:12:55+00:00

This is a backport of http://review.gluster.org/#/c/12033/

With unlink, rename, rmdir, contribution xattrs
are removed. If the file is a last link
then remove_xattr will fail with ENOENT.

So it better to perform remove_xattr
only if there are more links to the file

> Change-Id: Ifc1e7fda4d310fd87f6f28a635c9ea78b8f3929d
> BUG: 1257694
> Signed-off-by: vmallika 

Change-Id: Icf5fdd86bbb8eef0adeb9518e89e5b612e9e0705
BUG: 1279331
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/12549
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G

quota: add version to quota xattrs

2015-11-03T05:03:46+00:00

This is a backport of http://review.gluster.org/#/c/12386/

When a quota is disable and the clean-up process terminated
without completely cleaning-up the quota xattrs.
Now when quota is enabled again, this can mess-up the accounting

A version number is suffixed for all quota xattrs and this version
number is specific to marker xaltor, i.e when quota xattrs are
requested by quotad/client marker will remove the version suffix in the
key before sending the response

> Change-Id: I1ca2c11460645edba0f6b68db70d476d8d26e1eb
> BUG: 1272411
> Signed-off-by: vmallika 
> Reviewed-on: http://review.gluster.org/12386
> Tested-by: NetBSD Build System 
> Reviewed-by: Krishnan Parthasarathi 
> Tested-by: Gluster Build System 
> Reviewed-by: Manikandan Selvaganesh 
> Reviewed-by: Raghavendra G 

Change-Id: I67b1b930b28411d76b2d476a4e5250c52aa495a0
BUG: 1277080
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/12487
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

tier/ctr: CTR DB named lookup heal of cold tier during attach tier

2015-10-11T01:10:18+00:00

Heal hardlink in the db for already existing data in the cold
tier during attach tier. i.e during fix layout do lookup to files
in the cold tier.

CTR xlator on the  brick/server side does db update/insert of the hardlink on a namelookup.
Currently the namedlookup is done synchronous to the fixlayout that is
triggered by attach tier. This is not performant, adding more time to
fixlayout. The performant approach is record the hardlinks on a compressed
datastore and then do the namelookup asynchronously later, giving the ctr db
eventual consistency

master patch : http://review.gluster.org/#/c/11828/

>>Change-Id: I4ffc337fffe7d447804786851a9183a51b5044a9
>>BUG: 1252586
>>Signed-off-by: Joseph Fernandes 
>>Reviewed-on: http://review.gluster.org/11828
>>Tested-by: Gluster Build System 
>>Reviewed-by: Dan Lambright 
>>Tested-by: Dan Lambright 
Signed-off-by: Joseph Fernandes 

Change-Id: I61b185a54ae4e8c1d82804b95a278bfbea870987
BUG: 1261146
Reviewed-on: http://review.gluster.org/12331
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

posix: xattrop 'GF_XATTROP_ADD_ARRAY_WITH_DEFAULT' implementation

2015-09-29T05:35:19+00:00

This is a backport of http://review.gluster.org/#/c/11702

Implementation of xattrop type:
GF_XATTROP_ADD_ARRAY_WITH_DEFAULT
GF_XATTROP_ADD_ARRAY64_WITH_DEFAULT

These operations are similar to 'GF_XATTROP_ADD_ARRAY',
except that it adds a default value if xattr is missing
or its value is zero on disk.

One use-case of this operation is in inode-quota.
When a new directory is created, its default dir_count
should be set to 1. So when a xattrop performed setting
inode-xattrs, it should account initial dir_count
1 if the xattrs are not present

Here is the usage of this operation

value required in xdata for each key
struct array {
    int32_t   newvalue_1;
    int32_t   newvalue_2;
    ...
    int32_t   newvalue_n;
    int32_t   default_1;
    int32_t   default_2;
    ...
    int32_t   default_n;
};

or

struct array {
    int32_t   value_1;
    int32_t   value_2;
    ...
    int32_t   value_n;
} data[2];
fill data[0] with new value to add
fill data[1] with default value

xattrop GF_XATTROP_ADD_ARRAY_WITH_DEFAULT
for i from 1 to n
{
    if (xattr (dest_i) is zero or not set in the disk)
        dest_i = newvalue_i + default_i
    else
        dest_i = dest_i + newvalue_i
}

value in xdata after xattrop is successful
struct array {
    int32_t   dest_1;
    int32_t   dest_2;
    ...
    int32_t   dest_n;
};

> Change-Id: Ic6a08473e99fd98299a839d4d8416081a7534efd
> BUG: 1243946
> Signed-off-by: vmallika 
> Reviewed-on: http://review.gluster.org/11702
> Tested-by: NetBSD Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Ie0c8285d9d582afbc808b0fd878f6c02957ff928
BUG: 1266882
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/12241
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G

fuse: add "resolve-gids" mount option to overcome 32-groups limit

2015-09-28T09:51:08+00:00

Add a --resolve-gids commandline option to the glusterfs binary. This
option gets set when executing "mount -t glusterfs -o resolve-gids ...".

This option is most useful in combination with the "acl" mount option.
POSIX ACL permission checking is done on the FUSE-client side to improve
performance (in addition to the checking on the bricks).

The fuse-bridge reads /proc/$PID/status by default, and this file
contains maximum 32 groups. Any local (client-side) permission checking
that requires more than the first 32 groups will fail.

By enabling the "resolve-gids" option, the fuse-bridge will call
getgrouplist() to retrieve all the groups from the user accessing the
mountpoint. This is comparable to how "nfs.server-aux-gids" works.

Note that when a user belongs to more than ~93 groups, the volume option
server.manage-gids needs to be enabled too. Without this option, the
RPC-layer will need to reduce the number of groups to make them fit in
the RPC-header.

Cherry picked from commit 64a5bf3749c67fcc00773a2716d0c7b61b0b4417:
> Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
> BUG: 1246275
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/11732
> Tested-by: NetBSD Build System 
> Reviewed-by: Ravishankar N 
> Tested-by: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
BUG: 1246397
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11875
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri