glusterfs.git/rpc, branch release-3.9

socket: log the client identifier in ssl connect

2017-01-10T06:32:50+00:00

Problem: client identifier is not logged in message in ssl_setup_connection

Solutuion: In ssl_setup_connection xl_private is not available in rpc_transport
           so changed to this peerinfo.identifier.

Backport of commit 2e23c62cc50037c8e61bcd9c04348409e7627181
BUG: 1383879
Signed-off-by: Mohit Agrawal 

> BUG: 1380275
> Change-Id: I05006a3d63e46de8c388298c22faa9a3329eb6f3
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/15596
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Vijay Bellur 
> (cherry picked from commit 2e23c62cc50037c8e61bcd9c04348409e7627181)

Change-Id: I904f267937e0e12b0b1e19027c017e5d979b8197
Reviewed-on: http://review.gluster.org/15625
Tested-by: MOHIT AGRAWAL 
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

rpc/socket.c : Modify gf_log message in socket_poller code in case of error

2017-01-10T06:28:45+00:00

Problem:  In case of SSL after stopping the volume if client(mount point) is
          still trying to write the data on socket then it will throw an EIO
          error on that socket and given this log message is captured at every
          attempt this would flood the log file.

Solution: To reduce the frequency of stored log message use GF_LOG_OCCASIONALLY
          instead of gf_log.

> BUG: 1381115
> Change-Id: I66151d153c2cbfb017b3ebc4c52162278c0f537c
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/15605
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> (cherry picked from commit 070145750006c87099f945b4990a4460d814c21f)

Change-Id: I9536dcb2cbf3f8082dccf049cbfb21a034f5eac1
BUG: 1384363
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/15632
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra G

dht/md-cache: Filter invalidate if the file is made a linkto file

2017-01-05T06:10:12+00:00

Backport of http://review.gluster.org/15789

Upcall as a part of setattr, sends an invalidation and the
invalidation carries the resulting stat value. When a file
is converted to linkto files, even then an invalidation
is set and as a result the mountpoint shows the sticky
bit in the stat of the file.
eg: ---------T. 945 root root 0 Nov  8 10:14 hardlink.999

Fix:
When dht recieves a notification of sticky bit change, it updates
the flag, to indicate md-cache to send the subsequent lookup.

>Reviewed-on: http://review.gluster.org/15789
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Niels de Vos 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Susant Palai 
>Reviewed-by: Rajesh Joseph 
>(cherry picked from commit 4536f7bdf16f8286d67598eda9a46c029f0c0bf4)

Change-Id: Ic2fd7a5b196db0754f9b97072e644e6bf69da606
BUG: 1401376
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/16022
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

rpc: fix for race between rpc and protocol/client

2016-12-20T17:11:57+00:00

It is possible that the notification thread which notifies
protocol/client layer about the disconnection is put to sleep
and meanwhile, a fuse thread or a timer thread initiates and
completes reconnection to the brick. The notification thread
is then woken up and protocol/client layer updates its flags
to indicate that network is disconnected. No reconnection is
initiated because reconnection is rpc-lib layer's responsibility
and its flags indicate that connection is connected.

Fix: Serialize connect and disconnect notify

> Credit: Raghavendra Talur 
> Reviewed-on: http://review.gluster.org/15916
> Reviewed-by: Raghavendra G 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit aa22f24f5db7659387704998ae01520708869873)

Change-Id: I8ff5d1a3283b47f5c26848a42016a40bc34ffc1d
BUG: 1388323
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/16026
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra Talur

afr,dht,ec: Replace GF_EVENT_CHILD_MODIFIED with event SOME_DESCENDENT_DOWN/UP

2016-12-13T09:46:14+00:00

Backport of http://review.gluster.org/#/c/15764/

Currently these are few events related to child_up/down:
GF_EVENT_CHILD_UP :  Issued when any of the protocol client
connects.
GF_EVENT_CHILD_MODIFIED : Issued by afr/dht/ec
GF_EVENT_CHILD_DOWN : Issued when any of the protocol client
disconnects.
These events get modified at the dht/afr/ec layers. Here is a
brief on the same.

DHT:
- All the subvolumes reported once, and atleast one child came
  up, then GF_EVENT_CHILD_UP is issued
- connect GF_EVENT_CHILD_UP is issued
- disconnect GF_EVENT_CHILD_MODIFIED is issued
- All the subvolumes disconnected, GF_EVENT_CHILD_DOWN is issued

AFR:
- First subvolume came up, then GF_EVENT_CHILD_UP is issued
- Subsequent subvolumes coming up, results in GF_EVENT_CHILD_MODIFIED
- Any of the subvolumes go down, then GF_EVENT_SOME_CHILD_DOWN is issued
- Last up subvolume goes down, then GF_EVENT_CHILD_DOWN is issued

Until the patch [1] introduced GF_EVENT_SOME_CHILD_UP,
GF_EVENT_CHILD_MODIFIED was issued by afr/dht when any of the subvolumes
go up or down.

Now with md-cache changes, there is a necessity to differentiate between
child up and down. Hence, introducing GF_EVENT_SOME_DESCENDENT_DOWN/UP and
getting rid of GF_EVENT_CHILD_MODIFIED.

[1] http://review.gluster.org/12573

>Reviewed-on: http://review.gluster.org/15764
>CentOS-regression: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Smoke: Gluster Build System 
>Reviewed-by: N Balachandran 
>Reviewed-by: Pranith Kumar Karampuri  
>Reviewed-by: Rajesh Joseph 
(cherry picked from commit f7ab6c45963fa0da68acedfb14281cd2456abc68)

Change-Id: I704140b6598f7ec705493251d2dbc4191c965a58
BUG: 1396880
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15890
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/afr: CLI for granular entry heal enablement/disablement

2016-11-29T10:52:15+00:00

        Backport of: http://review.gluster.org/15747

When there are already existing non-granular indices created that are
yet to be healed, if granular-entry-heal option is toggled from 'off' to
'on', AFR self-heal whenever it kicks in, will try to look for granular
indices in 'entry-changes'. Because of the absence of name indices,
granular entry healing logic will fail to heal these directories, and
worse yet unset pending extended attributes with the assumption that
are no entries that need heal.

To get around this, a new CLI is introduced which will invoke glfsheal
program to figure whether at the time an attempt is made to enable
granular entry heal, there are pending heals on the volume OR there
are one or more bricks that are down. If either of them is true, the
command will be failed with the appropriate error.

New CLI: gluster volume heal  granular-entry-heal {enable,disable}

Change-Id: Ic79519468a087cd337df664b968188c4adcba43a
BUG: 1398500
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15941
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

Revert "rpc: Fix the race between notification and reconnection"

2016-11-12T05:06:49+00:00

This reverts commit e6c38ae1d3f3c53f8739ab2db7c4ecfdbc58fc44.

Mount is intermittently failing with this patch, which means
this patch introduced another race. So until we fix that
reverting this patch and making the release.

BUG: 1388323
Change-Id: I6e110caf38fcc6a309b2abdc864bc4fbdb3a7588
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15825
Reviewed-by: Raghavendra Talur 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd/quota: upgrade quota.conf file during an upgrade

2016-11-11T10:11:36+00:00

Problem
=======
When quota is enabled on 3.6, it will have quota conf version in quota.conf
as v1.1. This node gets upgraded to 3.7 but it will still have quota conf
version as v1.1 until a quota enable/disable/set limit is initiated. When
this is not initiated and when this node tries to peer probe a node which
is a fresh install of 3.7 (which will have quota conf version as v1.2), then this
will result in "Peer rejected" state. This patch fixes the issue.

Solution
========
When an upgrade happens from 3.6 to 3.7, quota.conf file needs
to be modified as well. With 3.6, in quota.conf the version will be
v1.1 and it needs to be changed to v1.2 from 3.7. This is because in
3.7, inode quota feature is introduced. So when an op-version bumpup
happens quota.conf needs to be upgraded with quota conf version v1.2
and all the 16 byte uuid needs to be changed to 17 bytes uuid as well.

Previously, when the cluster version is upgraded to 3.7, the quota.conf
got upgraded as well. But, the upgradation was done only when quota
enable/disable/set limit is done. With this patch, the upgradation is done
during a cluster op version bump up as well.


>Reviewed-on: http://review.gluster.org/15352
>Tested-by: Atin Mukherjee 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Atin Mukherjee 
>(cherry picked from commit 4b2cff614462508eef529c5d128e0974720e3f50)

Change-Id: Idb5ba29d3e1ea0e45c85d87c952c75da9e0f99f0
BUG: 1392718
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/15792
Tested-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Manikandan Selvaganesh 
CentOS-regression: Gluster Build System

rpc/socket: Close pipe on disconnection

2016-10-25T11:09:08+00:00

Encrypted connections create a pipe, which isn't closed when the
connection disconnects. This leaks fds, and gluster eventually ends up
in a situation with fd starvation which leads to operation failures.

> Change-Id: I144e1f767cec8c6fc1aa46b00cd234129d2a4adc
> BUG: 1336371
> Signed-off-by: Kaushal M 
> Reviewed-on: http://review.gluster.org/14356
> Tested-by: MOHIT AGRAWAL 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra G 
> Signed-off-by: Kaushal M 

Change-Id: I144e1f767cec8c6fc1aa46b00cd234129d2a4adc
BUG: 1387960
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/15702
Tested-by: Atin Mukherjee 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra G

rpc: Fix the race between notification and reconnection

2016-10-25T11:00:46+00:00

Problem:
There was a hang because unlock on an entry failed with
ENOTCONN.
Client thinks the connection is down where as server thinks
the connection is up.

This is the race we are seeing:
1) Connection from client to the brick disconnects.
2) Saved frames unwind is called which unwinds all
   frames that were wound before disconnect.
3) connection from client to the brick happens and
   setvolume.
4) Disconnect notification for the connection in 1)
   comes now and calls client_rpc_notify() which
   marks the connection to be offline even when the
   connection is up.

This is happening because I/O can retrigger connection
before disconnect notification is sent to the higher
layers in rpc.

Fix:
Notify the higher layers that a disconnect happened and then
go ahead with reconnect logic.

For the logs which point to the information above check:
https://bugzilla.redhat.com/show_bug.cgi?id=1386626#c1

Thanks to Raghavendra G for suggesting the correct fix.

 >BUG: 1386626
 >Change-Id: I3c84ba1f17010bd69049fa88ec5f0ae431f8cda9
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/15681
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Niels de Vos 
 >CentOS-regression: Gluster Build System 
 >Smoke: Gluster Build System 
 >Reviewed-by: Raghavendra G 
 >(cherry picked from commit a6b63e11b7758cf1bfcb67985e25ec02845f0995)

Change-Id: Ifa721193c26b70e26b47b7698c077da0ad5f2e1d
BUG: 1388323
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15717
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System