glusterfs.git, branch v3.11.2

doc: Added release notes for 3.11.2 release

2017-07-21T15:37:46+00:00

Change-Id: I03f20ace01a29cf8c22538adf4c11bddd343bc0e
BUG: 1466024
Signed-off-by: Shyam 
Reviewed-on: https://review.gluster.org/17852
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

nfs/nlm: keep track of the call-state and frame for notifications

2017-07-21T15:03:29+00:00

When blocking locks are used, a new frame is allocated that is used to
send the notification to the client once once the lock becomes
available. In all other cases, the frame that contains the request from
the client will be used for the reply.

Because there was no way to track the different clients with their
requests (captured in the call-state), the call-state could be free'd
before the notification was sent to the client. This caused a
use-after-free of the call-state and could trigger segfaults of the
Gluster/NFS server or incorrect replies on (un)lock requests.

By introducing a nlm4_notify_args structure, the call-state and frame
can be tracked better. This prevents the possibility of segfaulting when
the call-state is used after being free'd.

Cherry picked from commit b81997264f079983fa02bd5fa2b3715224942b00:
> BUG: 1467313
> Change-Id: I285d2bc552f509e5145653b7a50afcff827cd612
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17700
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: jiffin tony Thottan 

Change-Id: I285d2bc552f509e5145653b7a50afcff827cd612
BUG: 1471869
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17801
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

nfs/nlm: use refcounting for nfs3_call_state_t

2017-07-21T15:03:22+00:00

In order to track down a potential use-after-free of the
nfs3_call_state_t structure in the NLM component, add reference counting
where teh structure is used. This should prevent premature free'ing of
the structure.

Cherry picked from commit 01bfdd4d1759423681d311da33f4ac2346ace445:
> Change-Id: Ib1f13b0463ab1e012b7b49a623c91f0f3e73e1fb
> BUG: 1467313
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17699
> Reviewed-by: jiffin tony Thottan 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 

Change-Id: Ib1f13b0463ab1e012b7b49a623c91f0f3e73e1fb
BUG: 1471869
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17800
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

nfs/nlm: handle reconnect for non-NLM4_LOCK requests

2017-07-21T14:21:22+00:00

When a reply on an NLM-procedure gets stuck, the NFS-client will resend
the request. This can happen through a re-connect in case the connection
was terminated (long delay in the reply on the initial request). Once
that happens, not all NLM-procedures are handled correctly.

Testing this is difficult and time-consuming. There still may be
problems with certain operations, but this definitely makes it behave
much better than before.

The problem occured due to a problem in EC, change-id I18a782903ba
addressed the root cause.

Cherry picked from commit fafe1491ead527ba1024c521013aa90d2ee2b355:
> Change-Id: I23b385568e27232951fa3fbd7198a0e5d775a8c2
> BUG: 1467313
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17698
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 

Change-Id: I23b385568e27232951fa3fbd7198a0e5d775a8c2
BUG: 1471869
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17799
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

nfs/nlm: unref fds in nlm_client_free()

2017-07-21T14:21:17+00:00

When a nlm_clnt is getting free'd, the FDs associated with this client
should be unref'd as well.

Cherry picked from commit e9a482f94e748ea12e73ddd2e275bad9aa314b4c:
> Change-Id: Ifa4ea4b7ed45a454413cfc0c820f2516c534a9aa
> BUG: 1467313
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17697
> Smoke: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> CentOS-regression: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: Ifa4ea4b7ed45a454413cfc0c820f2516c534a9aa
BUG: 1471869
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17798
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

nfs: make nfs3_call_state_t refcounted

2017-07-21T14:21:11+00:00

There is no refcounting done of the nfs3_call_state_t structure, which
seems to result in use-after-free problems in the NLM part of
Gluster/NFS. The structure is initialized with two different functions,
it is easier to have a single place to do this.

The Gluster/NFS part will not use the refcounting, for now. This is
being added to make the NLM code more stable. nfs3_call_state_wipe()
will behave as before for Gluster/NFS, but cleanup is triggered through
the refcounting now. This prevents major changes to the stable part of
the NFS-server, and makes it possible to improve the NLM component
separately.

Cherry picked from commit daed52b8ebcac7ef36f11e944f83826f46593867:
> Change-Id: I2e15bcf12af74e8a46c2727e4a160e9444d29ece
> BUG: 1467313
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17696
> Smoke: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: jiffin tony Thottan 

Change-Id: I2e15bcf12af74e8a46c2727e4a160e9444d29ece
BUG: 1471869
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17797
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/ec: Test script failing with brick multiplexing enabled

2017-07-20T11:51:55+00:00

Problem:
Killing the bricks(using kill signal) in test scripts will
result in test failures with brick multiplexing enabled.

Solution:
Updated the script to use kill_brick function to bring down
the bricks.

>BUG: 1472094
>Change-Id: Ibbf1fdc1be660ad3cd93e95af2838c0aae0181af
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/17809
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 

BUG: 1472794
Change-Id: Ibbf1fdc1be660ad3cd93e95af2838c0aae0181af
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/17823
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/ec: Non-disruptive upgrade on EC volume fails

2017-07-19T11:26:21+00:00

Problem:
Enabling optimistic changelog on EC volume was not
handling node down scenarios appropriately resulting
in volume data inaccessibility.

Solution:
Update dirty xattr appropriately on good bricks whenever
nodes are down. This would fix the metadata information
as part of heal and thus ensures data accessibility.

>BUG: 1468261
>Change-Id: I08b0d28df386d9b2b49c3de84b4aac1c729ac057
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/17703
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 

BUG: 1470938
Change-Id: I08b0d28df386d9b2b49c3de84b4aac1c729ac057
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/17773
Smoke: Gluster Build System 
Reviewed-by: Ashish Pandey 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

cluster/dht: Clear clean_dst flag on target change

2017-07-19T11:25:43+00:00

If the target of a file migration was changed because
of min-free-disk limits, the dst_fd was closed but the
clean_dst flag was not set to false. If the file could
not be created on the new target for some reason, the
ftruncate call to clean up the dst was sent on the now
invalid fd causing the process to deadlock.

> BUG: 1469029
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17735
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra G 

(cherry picked from commit bd71ca4fdf2554dd22c0db70af132a11b966ef38)
Change-Id: I5bfa80f519b04567413d84229cf62d143c6e2f04
BUG: 1469459
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17745
Smoke: Gluster Build System 
Reviewed-by: Susant Palai 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: Fix fd check race

2017-07-19T11:25:10+00:00

There is a another race between the cached subvol
being updated in the inode_ctx and the fd being opened on
the target.

1. fop1 -> fd1 -> subvol0
2. file migrated from subvol0 to subvol1 and cached_subvol
   changed to subvol1 in inode_ctx
3. fop2 -> fd1 -> subvol1 [takes new cached subvol]
4. fop2 -> checks fd ctx (fd not open on subvol1) -> opens fd1 on subvol1
5. fop1 -> checks fd ctx (fd not open on subvol0)
   -> tries to open fd1 on subvol0 -> fails with "No such file on directory".

Fix:
If dht_fd_open_on_dst fails with ENOENT or ESTALE, wind to old subvol
and let the phase1/phase2 checks handle it.

> BUG: 1465075
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17731
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Amar Tumballi 
(cherry picked from commit f7a450c17fee7e43c544473366220887f0534ed7)
Change-Id: I34f8011574a8b72e3bcfe03b0cc4f024b352f225
BUG: 1465123
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17751
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System