glusterfs.git/xlators, branch v3.7.18

xlators/trash : Remove upper limit for trash max file size

2016-11-28T12:27:58+00:00

Currently file which size exceeds more than 1GB never moved to
trash directory. This is due to the hard coded check using
GF_ALLOWED_MAX_FILE_SIZE.

Upstream reference :
>Change-Id: I2ed707bfe1c3114818896bb27a9856b9a164be92
>BUG: 1386766
>Signed-off-by: Jiffin Tony Thottan 
>Reviewed-on: http://review.gluster.org/15689
>Smoke: Gluster Build System 
>Reviewed-by: Anoop C S 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Vijay Bellur 
>(cherry picked from commit cd9be49f6fe05d424989c0686a7e55a3f3ead27e)

Change-Id: I2ed707bfe1c3114818896bb27a9856b9a164be92
BUG: 1392366
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/15786
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Anoop C S 
Reviewed-by: Niels de Vos

glusterd/syncop: double free of frame stack

2016-11-28T10:17:24+00:00

Backport of http://review.gluster.org/13854

If rpc message from glusterd during brick op phase
fails without sending, then frame was freed from
the caller function and call back function.

>Change-Id: I63cb3be30074e9a074f6895faa25b3d091f5b6a5
>BUG: 1322262
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/13854
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Jeff Darcy 

Change-Id: I39b32f64fd66ee8a6d30c60bb0a42faa45e78814
BUG: 1395245
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/15917
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee

marker: Fix inode value in loc, in setxattr fop

2016-11-22T05:44:23+00:00

Backport of http://review.gluster.org/15826

On recieving a rename fop, marker_rename() stores the,
oldloc and newloc in its 'local' struct, once the rename
is done, the xtime marker(last updated time) is set on
the file, but sending a setxattr fop. When upcall
receives the setxattr fop, the loc->inode is NULL and
it crashes. The loc->inode can be NULL only in one valid
case, i.e. in rename case where the inode of new loc
can be NULL. Hence, marker should have filled the inode
of the new_loc before issuing a setxattr.

> Reviewed-on: http://review.gluster.org/15826
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kotresh HR 
> Smoke: Gluster Build System 
> Reviewed-by: Rajesh Joseph 
(cherry picked from commit 46e5466850311ee69e6ae9a11c2bba2aabadd5de)

Change-Id: Id638f678c3daaf4a5c29b970b58929d377ae8977
BUG: 1396419
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15879
Reviewed-by: Rajesh Joseph 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

upcall: Fix a log level

2016-11-18T11:20:03+00:00

In upcall_cache_invalidation(), the gfid can be NULL in certain
valid test cases(eg: entry for ".." in readdirp), hence change
the log level from WARNING to DEBUG.

Backport of http://review.gluster.org/15777

> Reviewed-on: http://review.gluster.org/15777
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Rajesh Joseph 
> Reviewed-by: Jeff Darcy 
(cherry picked from commit 68d1480e6056d1be91cde5129a6809642eeee857)

Change-Id: Ic90167a0e2076694e9131913114460df7b939b30
BUG: 1394188
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15830
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Smoke: Gluster Build System

cluster/afr: When failing fop due to lack of quorum, also log error string

2016-11-11T12:23:39+00:00

        Backport of: http://review.gluster.org/#/c/15800

Change-Id: I1e73b4518bcf26196d6326065ad404f878e70bd4
BUG: 1393631
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15814
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

posix-acl: check dictionary before using it

2016-11-09T19:17:36+00:00

If extended attributes are not present in md-cache it returns NULL as xattr.
posix acl xlator should check for NULL before using xattr.

If normal and default ACLs are not set on file then md-cache will not contain
system.posix_acl_access and system.posix_acl_default extended attributes in
its cache.

Therefore posix_acl_lookup_cbk should check xattr before using it, otherwise
the logs will get filled with dictionary errors.

> Reviewed-on: http://review.gluster.org/15769
> Reviewed-by: Raghavendra Talur 
> Smoke: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Vijay Bellur 
> CentOS-regression: Gluster Build System 

(cherry picked from commit de7fe24663713fff364cfc2b52b675e3e979ee68)

Change-Id: Icebf73cf0b313bd3e82ca8cbda63786dd0fa47da
BUG: 1392867
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15798
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur

features/shard: Fill loc.pargfid too for named lookups on individual shards

2016-11-09T06:39:14+00:00

        Backport of: http://review.gluster.org/#/c/15788/

On a sharded volume when a brick is replaced while IO is going on, named
lookup on individual shards as part of read/write was failing with
ENOENT on the replaced brick, and as a result AFR initiated name heal in
lookup callback. But since pargfid was empty (which is what this patch
attempts to fix), the resolution of the shards by protocol/server used
to fail and the following pattern of logs was seen:

Brick-logs:

[2016-11-08 07:41:49.387127] W [MSGID: 115009]
[server-resolve.c:566:server_resolve] 0-rep-server: no resolution type
for (null) (LOOKUP)
[2016-11-08 07:41:49.387157] E [MSGID: 115050]
[server-rpc-fops.c:156:server_lookup_cbk] 0-rep-server: 91833: LOOKUP(null)
(00000000-0000-0000-0000-000000000000/16d47463-ece5-4b33-9c93-470be918c0f6.82)
==> (Invalid argument) [Invalid argument]

Client-logs:
[2016-11-08 07:41:27.497687] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-0: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.497755] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-1: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.498500] W [MSGID: 114031]
[client-rpc-fops.c:2930:client3_3_lookup_cbk] 2-rep-client-2: remote
operation failed. Path: (null) (00000000-0000-0000-0000-000000000000)
[Invalid argument]
[2016-11-08 07:41:27.499680] E [MSGID: 133010]

Also, this patch makes AFR by itself choose a non-NULL pargfid even if
its ancestors fail to initialize all pargfid placeholders.

Change-Id: I34b9f90d0f09766b6d87b3994d5cd7a77b622dcb
BUG: 1392853
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15797
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Ravishankar N 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

glusterd/quota: upgrade quota.conf file during an upgrade

2016-11-08T13:32:13+00:00

Problem
=======
When quota is enabled on 3.6, it will have quota conf version in quota.conf
as v1.1. This node gets upgraded to 3.7 but it will still have quota conf
version as v1.1 until a quota enable/disable/set limit is initiated. When
this is not initiated and when this node tries to peer probe a node which
is a fresh install of 3.7 (which will have quota conf version as v1.2), then this
will result in "Peer rejected" state. This patch fixes the issue.

Solution
========
When an upgrade happens from 3.6 to 3.7, quota.conf file needs
to be modified as well. With 3.6, in quota.conf the version will be
v1.1 and it needs to be changed to v1.2 from 3.7. This is because in
3.7, inode quota feature is introduced. So when an op-version bumpup
happens quota.conf needs to be upgraded with quota conf version v1.2
and all the 16 byte uuid needs to be changed to 17 bytes uuid as well.

Previously, when the cluster version is upgraded to 3.7, the quota.conf
got upgraded as well. But, the upgradation was done only when quota
enable/disable/set limit is done. With this patch, the upgradation is done
during a cluster op version bump up as well.

>Reviewed-on: http://review.gluster.org/15352
>Tested-by: Atin Mukherjee 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Atin Mukherjee 

Change-Id: Idb5ba29d3e1ea0e45c85d87c952c75da9e0f99f0
BUG: 1392715
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/15790
Tested-by: sanoj-unnikrishnan 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Manikandan Selvaganesh

md-cache: Invalidate cache entry for open() with O_TRUNC

2016-11-04T10:10:36+00:00

When a file is opened with O_TRUNC flag set, its size gets
set to '0'. This case needs to be handled in md-cache to
avoid sending incorrect cached stat.

This is backport of below mainline patch -
http://review.gluster.org/#/c/15618/

> Change-Id: I95d1f8a6634734898883ede010c3e7b0b7eb97d9
> BUG: 1382266
> Signed-off-by: Soumya Koduri 
> Reviewed-on: http://review.gluster.org/15618
> Smoke: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Tested-by: jiffin tony Thottan 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> (cherry picked from commit 6ca5d6382f03685b31b12accb095093cf1486603)

Change-Id: I00d2b0a6dbafa690235bbc7cd714ec7cc8a13808
BUG: 1391451
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/15772
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

performance/write-behind: fix flush stuck by former failed writes

2016-11-03T09:28:17+00:00

the issue is happened in this case:
assume a file is opened with fd1 and fd2.
1. some WRITE opto fd1 got error, they were add back to 'todo' queue
   because of those error.
2. fd2 closed, a FLUSH op is send to write-behind.
3. FLUSH can not be unwind because it's not a legal waiter for those
   failed write(as func __wb_request_waiting_on() say). and those failed
   WRITE also can not be ended if fd1 is not closed. fd2 stuck in close
   syscall.

to resolve this issue, we can change the way we determine 2 requests is
'conflict': flush/fsync is not conflict with those write that is not
belonged to them. so __wb_pick_winds() can wind the FLUSH op.

below is some information when the stuck issue happen:
glusterdump logs:
[xlator.performance.write-behind.wb_inode]
path=/ltp-F9eG0ZSOME/rw-buffered-16436
inode=0x7fdbe8039b9c
window_conf=1048576
window_current=249856
transit-size=0
dontsync=0

[.WRITE]
request-ptr=0x7fdbe8020200
refcount=1
wound=no
generation-number=4
req->op_ret=-1
req->op_errno=116
sync-attempts=3
sync-in-progress=no
size=131072
offset=1220608
lied=-1
append=0
fulfilled=0
go=0

[.WRITE]
request-ptr=0x7fdbe8068c30
refcount=1
wound=no
generation-number=5
req->op_ret=-1
req->op_errno=116
sync-attempts=2
sync-in-progress=no
size=118784
offset=1351680
lied=-1
append=0
fulfilled=0
go=0

[.FLUSH]
request-ptr=0x7fdbe8021cd0
refcount=1
wound=no
generation-number=6
req->op_ret=0
req->op_errno=0
sync-attempts=0

gdb detail about above 3 requests:
(gdb) print *((wb_request_t *)0x7fdbe8021cd0)
$2 = {all = {next = 0x7fdbe803a608, prev = 0x7fdbe8068c30}, todo = {next
= 0x7fdbe803a618, prev = 0x7fdbe8068c40}, lie = {next = 0x7fdbe8021cf0,
    prev = 0x7fdbe8021cf0}, winds = {next = 0x7fdbe8021d00, prev =
0x7fdbe8021d00}, unwinds = {next = 0x7fdbe8021d10, prev =
0x7fdbe8021d10}, wip = {
    next = 0x7fdbe8021d20, prev = 0x7fdbe8021d20}, stub =
0x7fdbe80224dc, write_size = 0, orig_size = 0, total_size = 0, op_ret =
0, op_errno = 0,
  refcount = 1, wb_inode = 0x7fdbe803a5f0, fop = GF_FOP_FLUSH, lk_owner
= {len = 8, data = "W\322T\f\271\367y$", '\000' },
  iobref = 0x0, gen = 6, fd = 0x7fdbe800f0dc, wind_count = 0, ordering =
{size = 0, off = 0, append = 0, tempted = 0, lied = 0, fulfilled = 0,
    go = 0}}
(gdb) print *((wb_request_t *)0x7fdbe8020200)
$3 = {all = {next = 0x7fdbe8068c30, prev = 0x7fdbe803a608}, todo = {next
= 0x7fdbe8068c40, prev = 0x7fdbe803a618}, lie = {next = 0x7fdbe8068c50,
    prev = 0x7fdbe803a628}, winds = {next = 0x7fdbe8020230, prev =
0x7fdbe8020230}, unwinds = {next = 0x7fdbe8020240, prev =
0x7fdbe8020240}, wip = {
    next = 0x7fdbe8020250, prev = 0x7fdbe8020250}, stub =
0x7fdbe8062c3c, write_size = 131072, orig_size = 4096, total_size = 0,
op_ret = -1,
  op_errno = 116, refcount = 1, wb_inode = 0x7fdbe803a5f0, fop =
GF_FOP_WRITE, lk_owner = {len = 8, data = '\000' },
  iobref = 0x7fdbe80311a0, gen = 4, fd = 0x7fdbe805c89c, wind_count = 3,
ordering = {size = 131072, off = 1220608, append = 0, tempted = -1,
    lied = -1, fulfilled = 0, go = 0}}
(gdb) print *((wb_request_t *)0x7fdbe8068c30)
$4 = {all = {next = 0x7fdbe8021cd0, prev = 0x7fdbe8020200}, todo = {next
= 0x7fdbe8021ce0, prev = 0x7fdbe8020210}, lie = {next = 0x7fdbe803a628,
    prev = 0x7fdbe8020220}, winds = {next = 0x7fdbe8068c60, prev =
0x7fdbe8068c60}, unwinds = {next = 0x7fdbe8068c70, prev =
0x7fdbe8068c70}, wip = {
    next = 0x7fdbe8068c80, prev = 0x7fdbe8068c80}, stub =
0x7fdbe806746c, write_size = 118784, orig_size = 4096, total_size = 0,
op_ret = -1,
  op_errno = 116, refcount = 1, wb_inode = 0x7fdbe803a5f0, fop =
GF_FOP_WRITE, lk_owner = {len = 8, data = '\000' },
  iobref = 0x7fdbe8052b10, gen = 5, fd = 0x7fdbe805c89c, wind_count = 2,
ordering = {size = 118784, off = 1351680, append = 0, tempted = -1,
    lied = -1, fulfilled = 0, go = 0}}

you can see they are all on 'todo' queue, and FLUSH op fd is not the
same WRITE op fd.

> Change-Id: Id687f9cd3b9f281e1a97c83f1ce981ede272b8ab
> BUG: 1372211
> Signed-off-by: Ryan Ding 

Change-Id: Id687f9cd3b9f281e1a97c83f1ce981ede272b8ab
BUG: 1390840
Signed-off-by: Ryan Ding 
Reviewed-on: http://review.gluster.org/15763
Tested-by: Raghavendra G 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System