glusterfs.git/xlators/features, branch v3.8.13

features/shard: Fix vm corruption upon fix-layout

2017-04-10T15:45:51+00:00

        Backport of: https://review.gluster.org/17010

shard's writev implementation, as part of identifying
presence of participant shards that aren't in memory,
first sends an MKNOD on these shards, and upon EEXIST error,
looks up the shards before proceeding with the writes.

The VM corruption was caused when the following happened:
1. DHT had n subvolumes initially.
2. Upon add-brick + fix-layout, the layout of .shard changed
   although the existing shards under it were yet to be migrated
   to their new hashed subvolumes.
3. During this time, there were writes on the VM falling in regions
   of the file whose corresponding shards were already existing under
   .shard.
4. Sharding xl sent MKNOD on these shards, now creating them in their
   new hashed subvolumes although there already exist shard blocks for
   this region with valid data.
5. All subsequent writes were wound on these newly created copies.

The net outcome is that both copies of the shard didn't have the correct
data. This caused the affected VMs to be unbootable.

FIX:
For want of better alternatives in DHT, the fix changes shard fops to do
a LOOKUP before the MKNOD and upon EEXIST error, perform another lookup.

Change-Id: I1a5d3515b42e2e5583c407d1b4aff44d7ce472eb
BUG: 1440635
RCA'd-by: Raghavendra Gowdappa 
Reported-by: Mahdi Adnan 
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/17019
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: jiffin tony Thottan

features/shard: Initialize local->fop in readv

2017-04-10T11:27:44+00:00

        Backport of: https://review.gluster.org/17014

Change-Id: I4d2f0a3f533009038d48579db5a8a2a048b77ca1
BUG: 1440635
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/17020
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System

features/worm: Adding implementation for ftruncate

2017-04-07T12:09:30+00:00

Problem:
Since the ftruncate fop was not handled in the worm feature, when
truncate and write was happening on a worm-retained/worm file, it was
giving the EROFS error but truncating the file, which is not correct.

> Change-Id: I1a7e904655210d78bce9e01652ac56f3783b5aed
> BUG: 1438810
> Signed-off-by: karthik-us 
> Reviewed-on: https://review.gluster.org/16995
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Niels de Vos 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Ravishankar N 
> Smoke: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra Talur 
(cherry picked from commit c5a4a77848024d2adf8cd4f35d550ba90c174fc7)

Change-Id: Ic5e904b5bb3d76954a143f92fbfd8959fec884b8
BUG: 1439112
Signed-off-by: karthik-us 
Reviewed-on: https://review.gluster.org/17000
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Niels de Vos

features/locks: Fix leak of posix_lock_t's client_uid

2017-04-07T11:54:42+00:00

> Change-Id: I3bc14998ed6a8841f77a004c24a456331048a521
> BUG: 1428510
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: https://review.gluster.org/16838
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Jeff Darcy 

Change-Id: I3bc14998ed6a8841f77a004c24a456331048a521
BUG: 1431592
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/16896
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

features/shard: Pass the correct iatt for cache invalidation

2017-04-04T08:54:24+00:00

        Backport of: https://review.gluster.org/16961

This fixes a performance issue with shard which was causing
the translator to trigger unusually high number of lookups
for cache invalidation even when there was no modification to
the file.

In shard_common_stat_cbk(), it is local->prebuf that contains the
aggregated size and block count as opposed to buf which only holds the
attributes for the physical copy of base shard. Passing buf for
inode_ctx invalidation would always set refresh to true since the file
size in inode ctx contains the aggregated size and would never be same
as @buf->ia_size. This was leading to every write/read being preceded
by a lookup on the base shard even when the file underwent no
modification.

Change-Id: I85940b4b33e77b98e97e277d880ab35b1496c89a
BUG: 1437330
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16968
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-21T19:04:27+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

master change https://review.gluster.org/16880
master BZ: 1430841
release-3.10 change https://review.gluster.org/16881
release-3.10 BZ: 1430845

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430845
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16882
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire

features/shard: Fix EIO error on add-brick

2017-03-10T22:47:05+00:00

        Backport of: https://review.gluster.org/14419

DHT seems to link inode during lookup even before initializing
inode ctx with layout information, which comes after
directory healing.

Consider two parallel writes. As part of the first write,
shard sends lookup on .shard which in its return path would
cause DHT to link .shard inode. Now at this point, when a
second write is wound, inode_find() of .shard succeeds and
as a result of this, shard goes to create the participant
shards by issuing MKNODs under .shard. Since the layout is
yet to be initialized, mknod fails in dht call path with EIO,
leading to VM pauses.

The fix involves shard maintaining a flag to denote whether
a fresh lookup on .shard completed one network trip. If it
didn't, all inode_find()s in fop path will be followed by a
lookup before proceeding with the next stage of the fop.

Big thanks to Raghavendra G and Pranith Kumar K for the RCA
and subsequent inputs and feedback on the patch.

Change-Id: I66a7adf177e338a7691f441f199dde7c2b90c292
BUG: 1387878
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16750
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

features/shard: Put onus of choosing the inode to resolve on individual fops

2017-03-10T22:44:47+00:00

        Backport of: https://review.gluster.org/16709

... as opposed to adding checks in "common" functions to choose the inode
to resolve based local->fop, which is rather ugly and prone to errors.

Change-Id: Ib26d3dd5a7ae43cd27839752bdae2cce56d73e8a
BUG: 1387878
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16749
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

features/changelog: Fix htime xattr during brick crash

2017-01-31T09:31:14+00:00

The htime file contains the path of all the changelogs
that is rolloved over till now. It also maintains xattr
which tracks the latest changelog file rolloved over
and the number of changelogs. The path and and xattr
update happens in two different system calls. If the
brick is crashed between them, the xattr value becomes
stale and can lead to the failure of gf_history_changelog.
To identify this, the total number of changelogs is being
calculated based on htime file size and the record
length. The above value is used in case of mismatch.

> Change-Id: Ia1c3efcfda7b74227805bb2eb933c9bd4305000b
> BUG: 1413967
> Signed-off-by: Kotresh HR 
> Reviewed-on: http://review.gluster.org/16420
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Aravinda VK 

Change-Id: Ia1c3efcfda7b74227805bb2eb933c9bd4305000b
BUG: 1415053
Signed-off-by: Kotresh HR 
(cherry picked from commit 6f4811ca9331eee8c00861446f74ebe23626bbf8)
Reviewed-on: https://review.gluster.org/16437
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Aravinda VK

Upcall: Fix possible memleak when inode_ctx_set fails

2017-01-24T10:15:59+00:00

In __upcall_inode_ctx_set(), if inode_ctx_set fails we should
free allocated memory for ctx. This patch takes care of the same.

This is backport of below mainline fix.
http://review.gluster.org/16381

>Change-Id: Iafb42787151a579caf6f396c9b414ea48d16e6b4
>BUG: 1412489
>Reported-by: Nithya Balachandran 
>Signed-off-by: Soumya Koduri 
>Reviewed-on: http://review.gluster.org/16381
>Reviewed-by: N Balachandran 
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit 84271e12efb783bfc83133329b0fd18aba729c84)

Change-Id: Ia258f3fb12b92795aa7546708c6da5c91f70a08a
BUG: 1414655
Reported-by: Nithya Balachandran 
Signed-off-by: Soumya Koduri 
Reviewed-on: https://review.gluster.org/16431
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos