glusterfs.git/xlators/cluster/dht/src/dht-rename.c, branch release-3.9

cluster/dht: Do rename cleanup as root

2017-01-20T07:18:58+00:00

Problem:
Rename linkfile cleanup is done as non-root which may not have priviliges to do
the rename so it fails with EACCESS. MKDIR on that name in future will start to
hole on this subvolume. It is not easy to hit on fuse mounts because vfs takes
care of the permission checks even before rename fop is wound. But with
nfs-ganesha mounts it happens.

Fix:
Do rename cleanup as root

 >BUG: 1409727
 >Change-Id: I414c1eb6dce76b4516a6c940557b249e6c3f22f4
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/16317
 >Smoke: Gluster Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Raghavendra G 
 >Reviewed-by: N Balachandran 
 >NetBSD-regression: NetBSD Build System 

BUG: 1413061
Change-Id: If94121275b141c5f52084b8aafac86451e667d3d
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/16412
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran

dht/rename : Incase of failure remove linkto file properly

2016-12-07T02:06:35+00:00

Generally linkto file is created using root user. Consider following
case, a user is trying to rename a file which he is not permitted.
So the rename fails with EACESS and when rename tries to cleanup the
linkto file, it fails.

The above issue happens when rename/00.t test executed on nfs-ganesha
clients :
Steps executed in script
* create a file "abc" using root
* rename the file "abc" to "xyz" using a non root user, it fails with EACESS
* delete "abc"
* create directory "abc" using root
* again try ot rename "abc" to "xyz" using non root user, test hungs here
which slowly leds to OOM kill of ganesha process

RCA put forwarded by Du for OOM kill of ganesha
Note that when we hit this bug, we've a scenario of a dentry being
present as:
    * a linkto file on one subvol
    * a directory on rest of subvols

When a lookup happens on the dentry in such a scenario, the control flow
goes into an infinite loop of:

    dht_lookup_everywhere
    dht_lookup_everywhere_cbk
    dht_lookup_unlink_cbk
    dht_lookup_everywhere_done
    dht_lookup_directory (as local->dir_count > 0)
    dht_lookup_dir_cbk (sets to local->need_selfheal = 1 as the entry is a linkto file on one of the subvol)
    dht_lookup_everywhere (as need_selfheal = 1).

This infinite loop can cause increased consumption of memory due to:
1) dht_lookup_directory assigns a new layout to local->layout unconditionally
2)  Most of the functions in this loop do a stack_wind of various fops.

This results in growing of call stack (note that call-stack is destroyed only after lookup response is
received by fuse - which never happens in this case)

Thanks Du for root causing the oom kill and Sushant for suggesting the fix

Upstream reference :
>Change-Id: I1e16bc14aa685542afbd21188426ecb61fd2689d
>BUG: 1397052
>Signed-off-by: Jiffin Tony Thottan 
>Reviewed-on: http://review.gluster.org/15894
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Raghavendra G 
>(cherry picked from commit 57d59f4be205ae0c7888758366dc0049bdcfe449)

Change-Id: I1e16bc14aa685542afbd21188426ecb61fd2689d
BUG: 1401023
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/16014
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

features/shard: Get hard-link-count in {unlink,rename}_cbk before deleting shards

2016-05-20T14:40:00+00:00

Change-Id: I0606b74f11f5412c4d9af44a6505635ed9022c15
BUG: 1335858
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14334
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

dht: rename takes lock on parent directory if destination exists

2016-05-18T09:59:25+00:00

For directory rename if destination exists the source directory
is created as a child of the given destination directory. Since
the new child directory does not exist take lock on parent of the
child directory.

Change-Id: I24a34605a2cd65984910643ff5462f35e8fc7e71
BUG: 1336698
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/14371
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

dht: lock on subvols to prevent rename and lookup selfheal race

2016-04-06T15:28:09+00:00

This patch addresses two races while renaming directories:
1) While renaming src to dst, if a lookup selfheal is triggered
it can recreate src on those subvols where rename was successful.
This leads to multiple directories (src and dst) having same gfid.
To avoid this we must take locks on all subvols with src.

2) While renaming if the dst exists and a lookup selfheal is
triggered it will find anomalies in the dst layout and try to
heal the stale layout. To avoid this we must take lock on any
one subvol with dst.

Change-Id: I637f637d3241d9065cd5be59a671c7e7ca3eed53
BUG: 1252244
Signed-off-by: Sakshi 
Reviewed-on: http://review.gluster.org/11880
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

dht: lock on subvols to prevent lookup vs rmdir race

2016-04-06T05:06:02+00:00

There is a possibility that while an rmdir is completed on
some non-hashed subvol and proceeding to others, a lookup
selfheal can recreate the same directory on those subvols
for which the rmdir had succeeded. Now the deletion of the
parent directory will fail with an ENOTEMPTY.

To fix this take blocking inodelk on the subvols before
starting rmdir. Selfheal must also take blocking inodelk
before creating the entry.

Change-Id: I168a195c35ac1230ba7124d3b0ca157755b3df96
BUG: 1245065
Signed-off-by: Sakshi 
Reviewed-on: http://review.gluster.org/13528
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

dht: report constant directory size

2016-03-20T21:37:43+00:00

Directory size is meaningless.  Every filesystem has its own
unpredictable way of increasing or decreasing it, based on internal data
structures and even transient conditions.  Some filesystems (e.g. ext4)
never decrease it at all.  Others (e.g. btrfs) don't even report it.
Very few programs look at it, and those that do are broken.
Unfortunately, one such program is GNU tar, which will complain when it
sees different values because at different times we got the value from
different DHT subvolumes.  To avoid such problems, just report a
constant value.

Change-Id: Id64ce917c75b5f7ff50cb55b6e997f3b3556e7e3
BUG: 1302948
Original-author: Shyam 
Signed-off-by: Jeff Darcy 
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/13770
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur

dht: cleanup dict and free memory in rename code path

2016-02-25T05:11:32+00:00

Change-Id: I2458e18197bdf7565563a85e9021b5b2850c1825
BUG: 1303945
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/13392
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

dht: file rename must take blocking inode locks

2016-02-22T06:39:31+00:00

Currently DHT takes non-blocking locks for file rename.
Due to this during parallel renames some clients fail
with EBUSY or ESTALE errors. Hence to avoid application
discontinuity file rename must take blocking inode locks.

Change-Id: I986e9d08b3be359f20b1a3e1564e049b0f3dffd3
BUG: 1304966
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/13366
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

all: fix various cppcheck warnings

2016-02-15T09:32:14+00:00

fixes for various warnings reported by cppcheck

N.B. cppcheck output is in the bugzilla

Change-Id: I33acec127bc4536935fdd8d52a0c490ec54d50b2
BUG: 1292954
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13006
Reviewed-by: Jeff Darcy 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System