glusterfs.git, branch release-3.9

features/shard: Fix EIO error on add-brick

2017-02-27T05:45:54+00:00

        Backport of: https://review.gluster.org/14419

DHT seems to link inode during lookup even before initializing
inode ctx with layout information, which comes after
directory healing.

Consider two parallel writes. As part of the first write,
shard sends lookup on .shard which in its return path would
cause DHT to link .shard inode. Now at this point, when a
second write is wound, inode_find() of .shard succeeds and
as a result of this, shard goes to create the participant
shards by issuing MKNODs under .shard. Since the layout is
yet to be initialized, mknod fails in dht call path with EIO,
leading to VM pauses.

The fix involves shard maintaining a flag to denote whether
a fresh lookup on .shard completed one network trip. If it
didn't, all inode_find()s in fop path will be followed by a
lookup before proceeding with the next stage of the fop.

Big thanks to Raghavendra G and Pranith Kumar K for the RCA
and subsequent inputs and feedback on the patch.

Change-Id: Id0d160157ad8f6bcd52801a2173c5869517d0a96
BUG: 1426512
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16752
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

features/shard: Put onus of choosing the inode to resolve on individual fops

2017-02-27T05:00:35+00:00

        Backport of: https://review.gluster.org/16709

... as opposed to adding checks in "common" functions to choose the inode
to resolve based local->fop, which is rather ugly and prone to errors.

Change-Id: I84c5b26160150f2fd87e7f245190c500a4b36bd8
BUG: 1426512
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16751
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

extras: Add more options to group virt

2017-02-27T05:00:28+00:00

        Backport of: https://review.gluster.org/16577

Apart from some of the option configurations already listed
in the group-virt.example file, we also recommend that the
users set certain other options added by this patch for
VM use-case. This also helps Gluster-oVirt users in configuring
virt options for new volumes at the click of a button as opposed
to setting them manually through volume-set command.

Change-Id: I3a0c72b514e717268ccc4d02f23660e1d19ce232
BUG: 1424932
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16682
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

common-ha: unable to start HA, Connection Error

2017-02-26T19:14:44+00:00

See BZ 1284404. pcsd behavior has changed and pcsd will not accept
connections until SSL certificates have fully propagated throughout
all the nodes

HA devels suggest a 12 second delay between the `pcs cluster setup ...`
and the `pcs cluster start --all`

Change-Id: If94b6991a62f346dbead023c7e7f8282a995728c
BUG: 1425110
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16690
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: soumya k

cluster/ec: Change log level of messages to DEBUG

2017-02-20T08:37:06+00:00

Heal failed or passed should not be logged as info. These can be
observed from heal info if the heal is happening or not. If we require
to debug a case where heal is not happening, we can set the level to
DEBUG.

>Change-Id: I062668eadd145ef809b25e818e6bca1094f54cd6
>BUG: 1420619
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/16580
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Ashish Pandey 

Change-Id: I91700a1960b5feb03ef186e4d0ddba1338152469
BUG: 1422783
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/16636
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/ec: mark ec-background-heal.t as bad

2017-02-20T08:23:51+00:00

> Change-Id: I0c54c62cdeb40b983da2392296762471a5474652
> BUG: 1416689
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: https://review.gluster.org/16470
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Atin Mukherjee 
> CentOS-regression: Gluster Build System 

Change-Id: Ieb99ae7f728d787de0d9dc47a08b81536638e51e
BUG: 1422838
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/16643
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ashish Pandey

protocol/client: Fix double free of client fdctx destroy

2017-02-20T05:13:52+00:00

This patch fixes the race between fd re-open code and fd release code,
both of which free the fd context due to a race in certain variable
checks as explained below:

1. client process (shd in the case of this BZ) sends an opendir to its
children (client xlators) which send the fop to the bricks to get a valid fd.

2. Client xlator loses connection to the brick. fdctx->remotefd is -1

3. Client re-establishes connection. After handshake, it reopens the dir
and sets fdctx->remotefd to a valid fd in client3_3_reopendir_cbk().

4. Meanwhile, shd sends a fd unref after it is done with the opendir.
This triggers a releasedir (since fd->refcount becomes 0).

5. client3_3_releasedir() sees that fdctx-->remotefd is a valid number
(i.e not -1), sets fdctx->released=1 and calls  client_fdctx_destroy()

6. As a continuation of step3, client_reopen_done() is called by
client3_3_reopendir_cbk(), which sees that fdctx->released==1 and
again calls client_fdctx_destroy().

Depending on when step-5 does GF_FREE(fdctx), we may crash at any place in
step-6 in client3_3_reopendir_cbk() when it tries to access
fdctx->{whatever}.

> Reviewed-on: https://review.gluster.org/16521
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit 25fc74f9d1f2b1e7bab76485a99f27abadd10b7b)

Change-Id: Ia50873d11763e084e41d2a1f4d53715438e5e947
BUG: 1422351
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/16620
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

nfs: revalidate lookup converted to fresh lookup

2017-02-16T18:53:43+00:00

Backport of http://review.gluster.org/15580

when an inode ctx is missing for a linked inode the revalidate
lookups are converted to fresh.

This could result in sending ESTALE when the gfid are recreated

We are not able to reproduce the issue with normal setup, most part of
RCA was done with code reading.

Possible scenario in which  this bug can reproduce,
Delete a file and recreate a new file with same name, at the same time
from another client process try to list/or access the file.

In this case the second client may throw an ESTALE error for such files

Thanks to Soumya and Pranith for doing the complete RCA

>Change-Id: I73992a65844b09a169cefaaedc0dcfb129d66ea1
>BUG: 1379720
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/15580
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: soumya k 
>Reviewed-by: Kaleb KEITHLEY 
Signed-off-by: Mohammed Rafi KC 

Change-Id: I44c3770fb07e84183f8bc6eceb533efbc67fb67f
BUG: 1394634
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: https://review.gluster.org/15839
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System

afr: all children of AFR must be up to resolve s-brain

2017-02-15T15:07:32+00:00

Problem:
The various split-brain resolution policies (favorite-child-policy based,
CLI based and mount (get/setfattr) based) attempt to resolve split-brain
even when not all bricks of replica are up. This can be a problem when
say in a replica 3, the only good copy is down and the other 2 bricks
are up and blame each other (i.e. split-brain). We end up healing the
file in such a  case and allow I/O on it.

Fix:
A decision on whether the file is in split-brain or not must be taken
only if we are able to examine the afr xattrs of *all* bricks of a given
replica.

> Reviewed-on: https://review.gluster.org/16476
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit 0e03336a9362e5717e561f76b0c543e5a197b31b)

Change-Id: Icddb1268b380005799990f5379ef957d84639ef9
BUG: 1420983
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/16588
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

snapshot: Fix restore rollback to reassign snap volume ids to bricks

2017-02-03T06:25:56+00:00

Added further checks to ensure we do not go beyond prevalidate
when trying to restore a snapshot which has a nfs-gansha conf
file, in a cluster when nfs-ganesha is not enabled

The error message for the particular scenario is:
"Snapshot() has a nfs-ganesha export conf
file. cluster.enable-shared-storage and nfs-ganesha
should be enabled before restoring this snapshot."

> Reviewed-on: http://review.gluster.org/16116
> Reviewed-by: Rajesh Joseph 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

Change-Id: I1b87e9907e0a5e162f26ef1ca89fe76e8da8610f
BUG: 1405909
Signed-off-by: Avra Sengupta 
Reviewed-on: https://review.gluster.org/16195
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: jiffin tony Thottan 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK