glusterfs.git/xlators/cluster/dht/src, branch exp

cluster/dht: Fix memory corruption while accessing regex stored in

2016-12-08T17:56:41+00:00

private

If reconfigure is executed parallely (or concurrently with dht_init),
there are races that can corrupt memory. One such race is modification
of regexes stored in conf (conf->rsync_regex_valid and
conf->extra_regex_valid) through dht_init_regex. With change [1],
reconfigure codepath can get executed parallely (with itself or with
dht_init) and this fix is needed.

Also, a reconfigure can race with any thread doing dht_layout_search,
resulting in dht_layout_search accessing regex freed up by reconfigure
(like in bz 1399134).

[1] http://review.gluster.org/15046

Change-Id: I039422a65374cf0ccbe0073441f0e8c442ebf830
BUG: 1399134
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/15945
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

dht/md-cache: Filter invalidate if the file is made a linkto file

2016-12-02T10:46:58+00:00

Upcall as a part of setattr, sends an invalidation and the
invalidation carries the resulting stat value. When a file
is converted to linkto files, even then an invalidation
is set and as a result the mountpoint shows the sticky
bit in the stat of the file.
eg: ---------T. 945 root root 0 Nov  8 10:14 hardlink.999

Fix:
When dht recieves a notification of sticky bit change, it updates
the flag, to indicate md-cache to send the subsequent lookup.

Change-Id: Ic2fd7a5b196db0754f9b97072e644e6bf69da606
BUG: 1392713
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15789
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
Reviewed-by: Susant Palai 
Reviewed-by: Rajesh Joseph

dht/rename : Incase of failure remove linkto file properly

2016-12-01T15:47:34+00:00

Generally linkto file is created using root user. Consider following
case, a user is trying to rename a file which he is not permitted.
So the rename fails with EACESS and when rename tries to cleanup the
linkto file, it fails.

The above issue happens when rename/00.t test executed on nfs-ganesha
clients :
Steps executed in script
* create a file "abc" using root
* rename the file "abc" to "xyz" using a non root user, it fails with EACESS
* delete "abc"
* create directory "abc" using root
* again try ot rename "abc" to "xyz" using non root user, test hungs here
which slowly leds to OOM kill of ganesha process

RCA put forwarded by Du for OOM kill of ganesha
Note that when we hit this bug, we've a scenario of a dentry being
present as:
    * a linkto file on one subvol
    * a directory on rest of subvols

When a lookup happens on the dentry in such a scenario, the control flow
goes into an infinite loop of:

    dht_lookup_everywhere
    dht_lookup_everywhere_cbk
    dht_lookup_unlink_cbk
    dht_lookup_everywhere_done
    dht_lookup_directory (as local->dir_count > 0)
    dht_lookup_dir_cbk (sets to local->need_selfheal = 1 as the entry is a linkto file on one of the subvol)
    dht_lookup_everywhere (as need_selfheal = 1).

This infinite loop can cause increased consumption of memory due to:
1) dht_lookup_directory assigns a new layout to local->layout unconditionally
2)  Most of the functions in this loop do a stack_wind of various fops.

This results in growing of call stack (note that call-stack is destroyed only after lookup response is
received by fuse - which never happens in this case)

Thanks Du for root causing the oom kill and Sushant for suggesting the fix

Change-Id: I1e16bc14aa685542afbd21188426ecb61fd2689d
BUG: 1397052
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/15894
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

cluster/dht: A hard link is lost during rebalance + lookup

2016-11-29T08:34:23+00:00

Problem:  A hard link is lost during rebalance + lookup.Rebalance skip files
          if file has hardlink.In dht_migrate_file __is_file_migratable ()
          function checks if a file has hardlink, if yes file is not migrated
          but if link is created after call this function then link will lost.

Solution: Call __check_file_has_hardlink to check hardlink existence after (S+T) bits
          in migration process ,if file has hardlink then skip the file for
          migrate rebalance process.

BUG: 1396048
Change-Id: Ia53c07ef42f1128c2eedf959a757e8df517b9d12
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/15866
Reviewed-by: Susant Palai 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Raghavendra G

cluster/dht Set layout after mkdir as root

2016-11-21T14:40:15+00:00

DHT does not set the layout for newly created
directories as root. This causes EPERM failures
when a non-root user with insufficient permissions
creates directories.

credit: srangana@redhat.com for RCA

Change-Id: Ia646e41665ce172c43c5f01d2707455e8eb374ed
BUG: 1392772
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15794
Reviewed-by: Susant Palai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Jeff Darcy

afr,dht,ec: Replace GF_EVENT_CHILD_MODIFIED with event SOME_DESCENDENT_DOWN/UP

2016-11-21T09:32:05+00:00

Currently these are few events related to child_up/down:
GF_EVENT_CHILD_UP :  Issued when any of the protocol client
connects.
GF_EVENT_CHILD_MODIFIED : Issued by afr/dht/ec
GF_EVENT_CHILD_DOWN : Issued when any of the protocol client
disconnects.
These events get modified at the dht/afr/ec layers. Here is a
brief on the same.

DHT:
- All the subvolumes reported once, and atleast one child came
  up, then GF_EVENT_CHILD_UP is issued
- connect GF_EVENT_CHILD_UP is issued
- disconnect GF_EVENT_CHILD_MODIFIED is issued
- All the subvolumes disconnected, GF_EVENT_CHILD_DOWN is issued

AFR:
- First subvolume came up, then GF_EVENT_CHILD_UP is issued
- Subsequent subvolumes coming up, results in GF_EVENT_CHILD_MODIFIED
- Any of the subvolumes go down, then GF_EVENT_SOME_CHILD_DOWN is issued
- Last up subvolume goes down, then GF_EVENT_CHILD_DOWN is issued

Until the patch [1] introduced GF_EVENT_SOME_CHILD_UP,
GF_EVENT_CHILD_MODIFIED was issued by afr/dht when any of the subvolumes
go up or down.

Now with md-cache changes, there is a necessity to differentiate between
child up and down. Hence, introducing GF_EVENT_SOME_DESCENDENT_DOWN/UP and
getting rid of GF_EVENT_CHILD_MODIFIED.

[1] http://review.gluster.org/12573

Change-Id: I704140b6598f7ec705493251d2dbc4191c965a58
BUG: 1396038
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15764
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Rajesh Joseph

events: Add FMT_WARN for gf_event

2016-11-18T09:56:39+00:00

Raghavendra G found that posix is trying to print %s
but passing an int when HEALTH_CHECK fails in posix.
These are the kind of bugs that should be caught
at compilation itself.
Also fixed the problematic gf_event() callers.

BUG: 1386097
Change-Id: Id7bd6d9a9690237cec3ca1aefa2aac085e8a1270
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15671
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System

cluster/dht: Check for null inode

2016-11-16T03:42:33+00:00

Check for NULL inode before attempting to
set dht inode ctx.

Change-Id: I7693c18445f138221d8417df5e95b118cedb818a
BUG: 1395261
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15847
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

cluster/tier: break the monolith processing function

2016-10-26T20:53:05+00:00

Break tier_migrate_using_query_file() into a more manageable
tier_migrate_link() and helpers.

Change-Id: I5eb2d2cff9e7a2a2da567c3c4c2d53aab195f477
BUG: 1358296
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/14957
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

cluster/dht: Incorrect volname in rebalance events

2016-10-25T11:09:22+00:00

The rebalance event code was using strtok to parse the
volume name which is incorrect.
Reworked the code to get the correct volume name using
strstr.

Change-Id: Ib5f3305a34e6bf1ecfef677d87c5aff96bdeb0e6
BUG: 1388010
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15712
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System