glusterfs.git/xlators/features, branch v3.10.3

features/worm: Adding implementation for ftruncate

2017-04-13T15:37:23+00:00

Problem:
Since the ftruncate fop was not handled in the worm feature, when
truncate and write was happening on a worm-retained/worm file, it was
giving the EROFS error but truncating the file, which is not correct.

> Change-Id: I1a7e904655210d78bce9e01652ac56f3783b5aed
> BUG: 1438810
> Signed-off-by: karthik-us 
> Reviewed-on: https://review.gluster.org/16995
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Niels de Vos 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Ravishankar N 
> Smoke: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra Talur 
(cherry picked from commit c5a4a77848024d2adf8cd4f35d550ba90c174fc7)

Change-Id: I815049d37d95597021e11b1e3d25d56bb83623c4
BUG: 1437763
Signed-off-by: karthik-us 
Reviewed-on: https://review.gluster.org/16999
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Shyamsundar Ranganathan

features/shard: Fix vm corruption upon fix-layout

2017-04-13T15:35:57+00:00

Backport of:
> Change-Id: I8a2e97d91ba3275fbc7174a008c7234fa5295d36
> BUG: 1440051
> Reviewed on: https://review.gluster.org/17010
> (cherry-picked from commit 99c8c0b03a3368d81756440ab48091e1f2430a5f)

shard's writev implementation, as part of identifying
presence of participant shards that aren't in memory,
first sends an MKNOD on these shards, and upon EEXIST error,
looks up the shards before proceeding with the writes.

The VM corruption was caused when the following happened:
1. DHT had n subvolumes initially.
2. Upon add-brick + fix-layout, the layout of .shard changed
   although the existing shards under it were yet to be migrated
   to their new hashed subvolumes.
3. During this time, there were writes on the VM falling in regions
   of the file whose corresponding shards were already existing under
   .shard.
4. Sharding xl sent MKNOD on these shards, now creating them in their
   new hashed subvolumes although there already exist shard blocks for
   this region with valid data.
5. All subsequent writes were wound on these newly created copies.

The net outcome is that both copies of the shard didn't have the correct
data. This caused the affected VMs to be unbootable.

FIX:
For want of better alternatives in DHT, the fix changes shard fops to do
a LOOKUP before the MKNOD and upon EEXIST error, perform another lookup.

Change-Id: I8a2e97d91ba3275fbc7174a008c7234fa5295d36
BUG: 1426508
RCA'd-by: Raghavendra Gowdappa 
Reported-by: Mahdi Adnan 
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/17021
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

features/shard: Initialize local->fop in readv

2017-04-13T15:35:51+00:00

Backport of:
> Change-Id: I9008ca9960df4821636501ae84f93a68f370c67f
> BUG: 1440051
> Reviewed on: https://review.gluster.org/17014
> (cherry-picked from commit a4bb716be1f27be50e44d8167300e8b078a1f862)

Change-Id: I9008ca9960df4821636501ae84f93a68f370c67f
BUG: 1426508
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/17022
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

features/shard: Pass the correct iatt for cache invalidation

2017-03-30T13:56:05+00:00

Backport of:
> Change-Id: Ib0349291d2d01f3782d6d0bdd90c6db5e0609210
> BUG: 1436739
> Reviewed on: https://review.gluster.org/16961
> (cherry picked from commit 5369fe2d545e0a0d31ab9c9d2797744c130af259)
          
This fixes a performance issue with shard which was causing
the translator to trigger unusually high number of lookups
for cache invalidation even when there was no modification to
the file.

In shard_common_stat_cbk(), it is local->prebuf that contains the
aggregated size and block count as opposed to buf which only holds the
attributes for the physical copy of base shard. Passing buf for
inode_ctx invalidation would always set refresh to true since the file
size in inode ctx contains the aggregated size and would never be same
as @buf->ia_size. This was leading to every write/read being preceded
by a lookup on the base shard even when the file underwent no
modification.

Change-Id: Ib0349291d2d01f3782d6d0bdd90c6db5e0609210
BUG: 1437326
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16966
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

features/locks: Fix leak of posix_lock_t's client_uid

2017-03-16T01:38:59+00:00

> Change-Id: I3bc14998ed6a8841f77a004c24a456331048a521
> BUG: 1428510
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: https://review.gluster.org/16838
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Jeff Darcy 

Change-Id: I3bc14998ed6a8841f77a004c24a456331048a521
BUG: 1431591
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/16895
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-13T14:19:00+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

master change https://review.gluster.org/16880
master BZ: 1430841

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430844
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16881
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Shyamsundar Ranganathan

features/shard: Fix EIO error on add-brick

2017-02-27T16:10:47+00:00

        Backport of: https://review.gluster.org/14419

DHT seems to link inode during lookup even before initializing
inode ctx with layout information, which comes after
directory healing.

Consider two parallel writes. As part of the first write,
shard sends lookup on .shard which in its return path would
cause DHT to link .shard inode. Now at this point, when a
second write is wound, inode_find() of .shard succeeds and
as a result of this, shard goes to create the participant
shards by issuing MKNODs under .shard. Since the layout is
yet to be initialized, mknod fails in dht call path with EIO,
leading to VM pauses.

The fix involves shard maintaining a flag to denote whether
a fresh lookup on .shard completed one network trip. If it
didn't, all inode_find()s in fop path will be followed by a
lookup before proceeding with the next stage of the fop.

Big thanks to Raghavendra G and Pranith Kumar K for the RCA
and subsequent inputs and feedback on the patch.

Change-Id: Ibe59f6804a9c2ec95fbeaef1dc26858f16b8fcb5
BUG: 1426508
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16748
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Smoke: Gluster Build System

features/shard: Put onus of choosing the inode to resolve on individual fops

2017-02-27T15:34:44+00:00

        Backport of: https://review.gluster.org/16709

... as opposed to adding checks in "common" functions to choose the inode
to resolve based local->fop, which is rather ugly and prone to errors.

Change-Id: I55ede087b6ff8e9a76276c2636410c69f567bc0f
BUG: 1426508
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16747
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System

marker: Fix inode value in loc, in setxattr fop

2017-02-20T12:24:01+00:00

On recieving a rename fop, marker_rename() stores the,
oldloc and newloc in its 'local' struct, once the rename
is done, the xtime marker(last updated time) is set on
the file, but sending a setxattr fop. When upcall
receives the setxattr fop, the loc->inode is NULL and
it crashes. The loc->inode can be NULL only in one valid
case, i.e. in rename case where the inode of new loc
can be NULL. Hence, marker should have filled the inode
of the new_loc before issuing a setxattr.

marker_rename_cbk was already fixed in a previous commit.
Fixing marker_rename_done to send valid inode in this commit.

Also in upcall check for NULL inode so that there is no crash.

>Reviewed-on: https://review.gluster.org/16633
>Reviewed-by: Kotresh HR 
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: soumya k 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Raghavendra G 
>(cherry picked from commit 73defab8be16b73241225bb1c2588a61e3e425d5)

Change-Id: I3ed2a05118fed3367dfe3251ce4477310cb480d0
BUG: 1424937
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16684
Tested-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

libglusterfs+changetimerecorder: reduce log noise

2017-02-10T19:23:08+00:00

The logging about translator options is so verbose that it
significantly slows down scalability tests - sometimes even to the
point where it induces timing-related failures.  Quiet, please.

Backport of:
> Change-Id: If0766e2a80746bba586e67e6019ff7084d68b425
> Reviewed-on: https://review.gluster.org/16569

Change-Id: I65117e69427ce1d6a2490832c5c9ab57ee29004e
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16599
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan