glusterfs.git/xlators, branch v3.8rc1

client: Fix the message ids

2016-05-15T11:04:11+00:00

The message id of PC_MSG_GFID_NULL was changed as a part of rebase
of http://review.gluster.org/#/c/11597. Fixing the same

Backport of http://review.gluster.org/#/c/14276/

Cherry picked from commit f3699f32fd8d468f757697fdacf4949b8d5312d5
> Change-Id: I773e02fb5695b6b55700046f4a4298ec475f8991
> Signed-off-by: Poornima G 
> Reviewed-on: http://review.gluster.org/14276
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> CentOS-regression: Gluster Build System 

Change-Id: I773e02fb5695b6b55700046f4a4298ec475f8991
BUG: 1334994
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14280
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System

cluster/afr: Don't let NFS cache stat after writes

2016-05-14T14:30:21+00:00

Problem:
Afr does post-ops after write but the stat buffer it unwinds is at the
time of write, so if nfs client caches this, it will see different
ctime when it does stat on it after post-op is done. From NFS client's
perspective it thinks the file is changed. Tar which depends on this
to be correct keeps giving 'file changed as we read it' warning.
If Afr instead has to choose to unwind after post-op, eager-lock,
delayed-post-op will have to be disabled which will lead to bad
performance for all write usecases.

Fix:
Don't let client cache stat after write.

 >Change-Id: Ic6062acc6e5cdd97a9c83c56bd529ec83cee8a23
 >BUG: 1302948
 >Signed-off-by: Pranith Kumar K 
 >Signed-off-by: Anuradha Talur 
 >Reviewed-on: http://review.gluster.org/13785
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Niels de Vos 

BUG: 1335285
Change-Id: Ibef4fc80496d12acd15db57713af2e3a1c9109a7
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14300
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Niels de Vos

mount/fuse: Log gfid and fd ptr as well when writev/readv fail

2016-05-14T14:17:12+00:00

        Backport of: http://review.gluster.org/14291

Change-Id: I9100c7d75417b0c9bc3df112912cf773994b3a97
BUG: 1335730
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14324
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

cluster/afr: Do heals with shd pid

2016-05-14T14:14:37+00:00

Multi-threaded healing doesn't create synctask with shd pid, this
leads to healing problems when quota exceeds.

 >BUG: 1332994
 >Change-Id: I80f57c1923756f3298730b8820498127024e1209
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/14211
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Ravishankar N 

BUG: 1335283
Change-Id: If59d8f88d8f4a3ca6a3b6e1c9dfd594dd93f542b
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14298
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

cluster/afr: Handle non-zero source in heal-info decision

2016-05-14T14:12:38+00:00

        Backport of http://review.gluster.org/14302

Problem:
Spurious entries are reported in heal info when the mount is on second/third
brick of the replica pair because local-child is given preference in selecting
source. The code is supposed to suggest the file needs heal if the (source < 0)
(failure code path), but instead it is written as if any non-zero value
is considered failure.

Fix:
Treat +ve source as success case

BUG: 1335433
Change-Id: Iede983b6560622964e91306405587da3f1de5748
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14303
Reviewed-by: Krutika Dhananjay 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Anuradha Talur 
Reviewed-by: Niels de Vos

Revert "features/shard: Make o-direct writes work with sharding"

2016-05-14T14:09:31+00:00

        Backport of: http://review.gluster.org/#/c/14328/

This reverts commit c272c71391cea9db817f4e7e38cfc25a7cff8bd5.

This is for two reasons:
1) It introduces high fop latencies
2) Even with the patch, there is no true odirect behavior since the
   workaround in the patch doesn't reduce the caching done in kernel's
   page cache as far as writes on anon fds associated with individual
   shards is concerned.

Change-Id: I4137816a8bff9f0f77d42041a2d17e63dff82b5d
BUG: 1335822
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14330
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

readdir-ahead: Prefetch xattrs needed by md-cache

2016-05-10T16:42:38+00:00

Problem:
Negative cache feature implementation in md-cache requires xattrs
returned by posix to be intercepted for every call that can possibly
return xattrs. This includes readdirp(). This is crucial to treat
missing keys in cache as a case of negative entry (returns ENODATA)

md-cache puts names of xattrs that it wants to cache in xdata and
passes it down to posix which returns the specified xattrs in the
callback. This is done in lookup() and readdirp(). Hence, a xattr
that is cached can be invalidated during readdirp_cbk too.

This is based on the assumption that readdirp() will always return
all xattrs that md-cache is interested in. However, this is not the
case when readdirp() call is served from readdir-ahead's cache.
readdir-ahead xlator will pre-fetch dentries during opendir_cbk
and readdirp. These internal readdirp() calls made by readdir-ahead
xlator does not set xdata in it's requests. Hence, no xattrs are
fetched and stored in it's internal cache.

This causes metadata loss in gluster-swift. md-cache returns ENODATA
during getxattr() call even though the xattr for that object exists on
the brick. On receiving ENODATA, gluster-swift will create new metadata
and do setxattr(). This results in loss of information stored in
existing xattr.

Fix:
During opendir, md-cache will communicate to readdir-ahead asking it
to store the names of xattrs it's interested in so that readdir-ahead
can fetch those in all subsequent internal readdirp() calls issued by
it. This stored names of xattrs is invalidated/updated on the next
real readdirp() call issued by application. This readdirp() call will
have xdata set correctly by md-cache xlator.

Cherry picked from commit 0c73e7050c4d30ace0c39cc9b9634e9c1b448cfb:
> BUG: 1333023
> Change-Id: I32d46f93a99d4ec34c741f3c52b0646d141614f9
> Reviewed-on: http://review.gluster.org/14214
> Tested-by: Prashanth Pai 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Tested-by: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Tested-by: Raghavendra G 

BUG: 1334699
Change-Id: I32d46f93a99d4ec34c741f3c52b0646d141614f9
Signed-off-by: Prashanth Pai 
Reviewed-on: http://review.gluster.org/14281
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System

glusterd: add defence mechanism to avoid brick port clashes

2016-05-10T07:11:33+00:00

Intro:
Currently glusterd maintain the portmap registry which contains ports that
are free to use between 49152 - 65535, this registry is initialized
once, and updated accordingly as an then when glusterd sees they are been
used.

Glusterd first checks for a port within the portmap registry and gets a FREE
port marked in it, then checks if that port is currently free using a connect()
function then passes it to brick process which have to bind on it.

Problem:
We see that there is a time gap between glusterd checking the port with
connect() and brick process actually binding on it. In this time gap it could
be so possible that any process would have occupied this port because of which
brick will fail to bind and exit.

Case 1:
To avoid the gluster client process occupying the port supplied by glusterd :

we have separated the client port map range with brick port map range more @
http://review.gluster.org/#/c/13998/

Case 2: (Handled by this patch)
To avoid the other foreign process occupying the port supplied by glusterd :

To handle above situation this patch implements a mechanism to return EADDRINUSE
error code to glusterd, upon which a new port is allocated and try to restart
the brick process with the newly allocated port.

Note: Incase of glusterd restarts i.e. runner_run_nowait() there is no way to
handle Case 2, becuase runner_run_nowait() will not wait to get the return/exit
code of the executed command (brick process). Hence as of now in such case,
we cannot know with what error the brick has failed to connect.

This patch also fix the runner_end() to perform some cleanup w.r.t
return values.

Backport of:
> Change-Id: Iec52e7f5d87ce938d173f8ef16aa77fd573f2c5e
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14043
> Tested-by: Prasanna Kumar Kalever 
> Reviewed-by: Atin Mukherjee 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Id7d8351a0082b44310177e714edc0571ad0f7195
BUG: 1333711
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14235
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

rpc: define client port range

2016-05-10T07:11:20+00:00

Problem:
when bind-insecure is 'off', all the clients bind to secure ports,
if incase all the secure ports exhaust the client will no more bind
to secure ports and tries gets a random port which is obviously insecure.

we have seen the client obtaining a port number in the range 49152-65535
which are actually reserved as part of glusterd's pmap_registry for bricks,
hence this will lead to port clashes between client and brick processes.

Solution:
If we can define different port ranges for clients incase where secure ports
exhaust, we can avoid the maximum port clashes with in gluster processes.

Still we are prone to have clashes with other non-gluster processes, but
the chances being very low, but that's a different story on its own,
which will be handled in upcoming patches.

Backportof:
> Change-Id: Ib5ce05991aa1290ccb17f6f04ffd65caf411feaf
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/13998
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Raghavendra G 

Change-Id: I2ab9608ddbefcdf5987d817c23dd066010148e19
BUG: 1333711
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14234
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd/bitrot: Fix bitrot xlator_type

2016-05-08T18:43:18+00:00

Fix "voltype" value for bitrot in 'glusterd_volopt_map'.
Routines such as 'gd_is_boolean_option' will use
the value of "voltype" to load the shared
library and figure out data type of specific options.
If the value of "voltype" does not match with
the actual shared library name, it will break such
routines.

>Change-Id: I36af4bb3e7043cc87dd19aea1985f3167ff82c18
>BUG: 1332465
>Signed-off-by: Kotresh HR 
>Reviewed-on: http://review.gluster.org/14183
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Atin Mukherjee 
>Reviewed-by: Kaushal M 
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
(cherry picked from commit 29254449b19697b93ebf376541ea5062b4a99fcb)

Change-Id: Iae7b00272d2d5231ce46944064badacb1f09bc3b
BUG: 1332864
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14207
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Niels de Vos