glusterfs.git/xlators, branch v3.9.0

Remove experimental translators

2016-11-14T05:27:04+00:00

Change-Id: Id645918fa236f1fc00ab5fa427f394e853c44bf8
BUG: 1389675
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/15750
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd/quota: upgrade quota.conf file during an upgrade

2016-11-11T10:11:36+00:00

Problem
=======
When quota is enabled on 3.6, it will have quota conf version in quota.conf
as v1.1. This node gets upgraded to 3.7 but it will still have quota conf
version as v1.1 until a quota enable/disable/set limit is initiated. When
this is not initiated and when this node tries to peer probe a node which
is a fresh install of 3.7 (which will have quota conf version as v1.2), then this
will result in "Peer rejected" state. This patch fixes the issue.

Solution
========
When an upgrade happens from 3.6 to 3.7, quota.conf file needs
to be modified as well. With 3.6, in quota.conf the version will be
v1.1 and it needs to be changed to v1.2 from 3.7. This is because in
3.7, inode quota feature is introduced. So when an op-version bumpup
happens quota.conf needs to be upgraded with quota conf version v1.2
and all the 16 byte uuid needs to be changed to 17 bytes uuid as well.

Previously, when the cluster version is upgraded to 3.7, the quota.conf
got upgraded as well. But, the upgradation was done only when quota
enable/disable/set limit is done. With this patch, the upgradation is done
during a cluster op version bump up as well.


>Reviewed-on: http://review.gluster.org/15352
>Tested-by: Atin Mukherjee 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Atin Mukherjee 
>(cherry picked from commit 4b2cff614462508eef529c5d128e0974720e3f50)

Change-Id: Idb5ba29d3e1ea0e45c85d87c952c75da9e0f99f0
BUG: 1392718
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/15792
Tested-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Manikandan Selvaganesh 
CentOS-regression: Gluster Build System

posix-acl: check dictionary before using it

2016-11-09T19:17:18+00:00

If extended attributes are not present in md-cache it returns NULL as xattr.
posix acl xlator should check for NULL before using xattr.

If normal and default ACLs are not set on file then md-cache will not contain
system.posix_acl_access and system.posix_acl_default extended attributes in
its cache.

Therefore posix_acl_lookup_cbk should check xattr before using it, otherwise
the logs will get filled with dictionary errors.

> Reviewed-on: http://review.gluster.org/15769
> Reviewed-by: Raghavendra Talur 
> Smoke: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Vijay Bellur 
> CentOS-regression: Gluster Build System 

(cherry picked from commit de7fe24663713fff364cfc2b52b675e3e979ee68)

Change-Id: Icebf73cf0b313bd3e82ca8cbda63786dd0fa47da
BUG: 1392869
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15801
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur

features/shard: Fill loc.pargfid too for named lookups on individual shards

2016-11-09T07:34:31+00:00

        Backport of: http://review.gluster.org/#/c/15788/

On a sharded volume when a brick is replaced while IO is going on, named
lookup on individual shards as part of read/write was failing with
ENOENT on the replaced brick, and as a result AFR initiated name heal in
lookup callback. But since pargfid was empty (which is what this patch
attempts to fix), the resolution of the shards by protocol/server used
to fail and the following pattern of logs was seen:

Brick-logs:

[2016-11-08 07:41:49.387127] W [MSGID: 115009]
[server-resolve.c:566:server_resolve] 0-rep-server: no resolution type
for (null) (LOOKUP)
[2016-11-08 07:41:49.387157] E [MSGID: 115050]
[server-rpc-fops.c:156:server_lookup_cbk] 0-rep-server: 91833: LOOKUP(null)
(00000000-0000-0000-0000-000000000000/16d47463-ece5-4b33-9c93-470be918c0f6.82)
==> (Invalid argument) [Invalid argument]

Client-logs:
[2016-11-08 07:41:27.497687] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-0: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.497755] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-1: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.498500] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-2: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.499680] E [MSGID: 133010]

Also, this patch makes AFR by itself choose a non-NULL pargfid even if
its ancestors fail to initialize all pargfid placeholders.

Change-Id: I1ad027b7b2ccd6b5c0162726b63b06d261b108e8
BUG: 1392844
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15795
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Ravishankar N 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

md-cache: Invalidate cache entry for open() with O_TRUNC

2016-11-04T03:19:55+00:00

When a file is opened with O_TRUNC flag set, its size gets
set to '0'. This case needs to be handled in md-cache to
avoid sending incorrect cached stat.

This is backport of below mainline patch -
http://review.gluster.org/#/c/15618/

> Change-Id: I95d1f8a6634734898883ede010c3e7b0b7eb97d9
> BUG: 1382266
> Signed-off-by: Soumya Koduri 
> Reviewed-on: http://review.gluster.org/15618
> Smoke: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Tested-by: jiffin tony Thottan 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> (cherry picked from commit 6ca5d6382f03685b31b12accb095093cf1486603)

Change-Id: I9780a4051adaa9030bd33da68dbc0a31212de054
BUG: 1391448
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/15770
Reviewed-by: jiffin tony Thottan 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

performance/write-behind: fix flush stuck by former failed writes

2016-11-03T09:28:20+00:00

the issue is happened in this case:
assume a file is opened with fd1 and fd2.
1. some WRITE opto fd1 got error, they were add back to 'todo' queue
   because of those error.
2. fd2 closed, a FLUSH op is send to write-behind.
3. FLUSH can not be unwind because it's not a legal waiter for those
   failed write(as func __wb_request_waiting_on() say). and those failed
   WRITE also can not be ended if fd1 is not closed. fd2 stuck in close
   syscall.

to resolve this issue, we can change the way we determine 2 requests is
'conflict': flush/fsync is not conflict with those write that is not
belonged to them. so __wb_pick_winds() can wind the FLUSH op.

below is some information when the stuck issue happen:
glusterdump logs:
[xlator.performance.write-behind.wb_inode]
path=/ltp-F9eG0ZSOME/rw-buffered-16436
inode=0x7fdbe8039b9c
window_conf=1048576
window_current=249856
transit-size=0
dontsync=0

[.WRITE]
request-ptr=0x7fdbe8020200
refcount=1
wound=no
generation-number=4
req->op_ret=-1
req->op_errno=116
sync-attempts=3
sync-in-progress=no
size=131072
offset=1220608
lied=-1
append=0
fulfilled=0
go=0

[.WRITE]
request-ptr=0x7fdbe8068c30
refcount=1
wound=no
generation-number=5
req->op_ret=-1
req->op_errno=116
sync-attempts=2
sync-in-progress=no
size=118784
offset=1351680
lied=-1
append=0
fulfilled=0
go=0

[.FLUSH]
request-ptr=0x7fdbe8021cd0
refcount=1
wound=no
generation-number=6
req->op_ret=0
req->op_errno=0
sync-attempts=0

gdb detail about above 3 requests:
(gdb) print *((wb_request_t *)0x7fdbe8021cd0)
$2 = {all = {next = 0x7fdbe803a608, prev = 0x7fdbe8068c30}, todo = {next
= 0x7fdbe803a618, prev = 0x7fdbe8068c40}, lie = {next = 0x7fdbe8021cf0,
    prev = 0x7fdbe8021cf0}, winds = {next = 0x7fdbe8021d00, prev =
0x7fdbe8021d00}, unwinds = {next = 0x7fdbe8021d10, prev =
0x7fdbe8021d10}, wip = {
    next = 0x7fdbe8021d20, prev = 0x7fdbe8021d20}, stub =
0x7fdbe80224dc, write_size = 0, orig_size = 0, total_size = 0, op_ret =
0, op_errno = 0,
  refcount = 1, wb_inode = 0x7fdbe803a5f0, fop = GF_FOP_FLUSH, lk_owner
= {len = 8, data = "W\322T\f\271\367y$", '\000' },
  iobref = 0x0, gen = 6, fd = 0x7fdbe800f0dc, wind_count = 0, ordering =
{size = 0, off = 0, append = 0, tempted = 0, lied = 0, fulfilled = 0,
    go = 0}}
(gdb) print *((wb_request_t *)0x7fdbe8020200)
$3 = {all = {next = 0x7fdbe8068c30, prev = 0x7fdbe803a608}, todo = {next
= 0x7fdbe8068c40, prev = 0x7fdbe803a618}, lie = {next = 0x7fdbe8068c50,
    prev = 0x7fdbe803a628}, winds = {next = 0x7fdbe8020230, prev =
0x7fdbe8020230}, unwinds = {next = 0x7fdbe8020240, prev =
0x7fdbe8020240}, wip = {
    next = 0x7fdbe8020250, prev = 0x7fdbe8020250}, stub =
0x7fdbe8062c3c, write_size = 131072, orig_size = 4096, total_size = 0,
op_ret = -1,
  op_errno = 116, refcount = 1, wb_inode = 0x7fdbe803a5f0, fop =
GF_FOP_WRITE, lk_owner = {len = 8, data = '\000' },
  iobref = 0x7fdbe80311a0, gen = 4, fd = 0x7fdbe805c89c, wind_count = 3,
ordering = {size = 131072, off = 1220608, append = 0, tempted = -1,
    lied = -1, fulfilled = 0, go = 0}}
(gdb) print *((wb_request_t *)0x7fdbe8068c30)
$4 = {all = {next = 0x7fdbe8021cd0, prev = 0x7fdbe8020200}, todo = {next
= 0x7fdbe8021ce0, prev = 0x7fdbe8020210}, lie = {next = 0x7fdbe803a628,
    prev = 0x7fdbe8020220}, winds = {next = 0x7fdbe8068c60, prev =
0x7fdbe8068c60}, unwinds = {next = 0x7fdbe8068c70, prev =
0x7fdbe8068c70}, wip = {
    next = 0x7fdbe8068c80, prev = 0x7fdbe8068c80}, stub =
0x7fdbe806746c, write_size = 118784, orig_size = 4096, total_size = 0,
op_ret = -1,
  op_errno = 116, refcount = 1, wb_inode = 0x7fdbe803a5f0, fop =
GF_FOP_WRITE, lk_owner = {len = 8, data = '\000' },
  iobref = 0x7fdbe8052b10, gen = 5, fd = 0x7fdbe805c89c, wind_count = 2,
ordering = {size = 118784, off = 1351680, append = 0, tempted = -1,
    lied = -1, fulfilled = 0, go = 0}}

you can see they are all on 'todo' queue, and FLUSH op fd is not the
same WRITE op fd.

> Change-Id: Id687f9cd3b9f281e1a97c83f1ce981ede272b8ab
> BUG: 1372211
> Signed-off-by: Ryan Ding 

Change-Id: Id687f9cd3b9f281e1a97c83f1ce981ede272b8ab
BUG: 1390837
Signed-off-by: Ryan Ding 
Reviewed-on: http://review.gluster.org/15761
Tested-by: Raghavendra G 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

snapshot: Fix for memory leaks in snapshot code path

2016-11-02T06:31:57+00:00

> Reviewed-on: http://review.gluster.org/15668
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Rajesh Joseph 

(cherry picked from commit 48a4c4e525665115f7e8c478d3bf51764427378d)

Change-Id: Idc2cb16574d166e3c0ee1f7c3a485f1acb19fc8c
BUG: 1388350
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/15719
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph

crypt: changes needed for openssl-1.1 (coming in Fedora 26)

2016-10-27T18:46:12+00:00

Fedora has updated openssl-1.1.0b in/for Fedora 26

HMAC_CTX is now an opaque type and instances of it must be
created and released by calling HMAC_CTX_new() and
HMAC_CTX_free().

Change-Id: I3a09751d7b0d9fc25fe18aac6527e5431e9ab19a
BUG: 1388579
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15726
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

afr,ec: Heal device files with correct major, minor numbers

2016-10-27T06:23:26+00:00

Thanks a lot to xiaoping.wu@nokia.com from Nokia for the bug and the
fix.

 >BUG: 1384297
 >Change-Id: Ie443237e85d34633b5dd30f85eaa2ac34e45754c
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/15728
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Xavier Hernandez 
 >CentOS-regression: Gluster Build System 

Change-Id: Ie7977caf7c98c91fca64752c56731c37ad27df4d
BUG: 1388912
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15734
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez

cluster/dht: Incorrect volname in rebalance events

2016-10-26T02:57:10+00:00

The rebalance event code was using strtok to parse the
volume name which is incorrect.
Reworked the code to get the correct volume name using
strstr.

> Change-Id: Ib5f3305a34e6bf1ecfef677d87c5aff96bdeb0e6
> BUG: 1388010
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/15712
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
(cherry picked from commit f4efbf0e3092bd2181f62be1e1f30f202678c866)

Change-Id: Ia22d6a8ce23e9dcab8281b4995277cbb3a196fda
BUG: 1388563
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15725
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri