glusterfs.git/xlators/mount/fuse/src, branch v3.6.6

fuse: add "resolve-gids" mount option to overcome 32-groups limit

2015-09-21T06:45:07+00:00

Add a --resolve-gids commandline option to the glusterfs binary. This
option gets set when executing "mount -t glusterfs -o resolve-gids ...".

This option is most useful in combination with the "acl" mount option.
POSIX ACL permission checking is done on the FUSE-client side to improve
performance (in addition to the checking on the bricks).

The fuse-bridge reads /proc/$PID/status by default, and this file
contains maximum 32 groups. Any local (client-side) permission checking
that requires more than the first 32 groups will fail.

By enabling the "resolve-gids" option, the fuse-bridge will call
getgrouplist() to retrieve all the groups from the user accessing the
mountpoint. This is comparable to how "nfs.server-aux-gids" works.

Note that when a user belongs to more than ~93 groups, the volume option
server.manage-gids needs to be enabled too. Without this option, the
RPC-layer will need to reduce the number of groups to make them fit in
the RPC-header.

Cherry picked from commit 64a5bf3749c67fcc00773a2716d0c7b61b0b4417:
> Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
> BUG: 1246275
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/11732
> Tested-by: NetBSD Build System 
> Reviewed-by: Ravishankar N 
> Tested-by: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
BUG: 1252072
Tested-by: Patrick Glomski 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11876
Tested-by: Gluster Build System 
Reviewed-by: Anoop C S 
Reviewed-by: Raghavendra Bhat

core: fix Ubuntu code audit (cppcheck) results

2015-06-10T12:03:01+00:00

See also http://review.gluster.org/#/c/8064/, BZ 1109180, and
http://review.gluster.org/#/c/7693/, BZ 1091677

AFAICT these are false positives:

[geo-replication/src/gsyncd.c:100]: (error) Memory leak: str
[geo-replication/src/gsyncd.c:403]: (error) Memory leak: argv
[xlators/nfs/server/src/nlm4.c:1201]: (error) Possible null pointer dereference: fde
[xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr

Test program:

[extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments.
[tests/basic/fops-sanity.c:55]: (error) Buffer overrun possible for long command line arguments.

the remainder are fixed with this change-set:

[cli/src/cli-rpc-ops.c:8883]: (error) Possible null pointer dereference: local
[cli/src/cli-rpc-ops.c:8886]: (error) Possible null pointer dereference: local
[contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 2) requires 'long *' but the argument type is 'unsigned long *'.
[contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 3) requires 'long *' but the argument type is 'unsigned long *'.
[xlators/cluster/dht/src/dht-rebalance.c:1734]: (error) Possible null pointer dereference: ctx
[xlators/cluster/stripe/src/stripe.c:4940]: (error) Possible null pointer dereference: local
[xlators/mgmt/glusterd/src/glusterd-geo-rep.c:1718]: (error) Possible null pointer dereference: command
[xlators/mgmt/glusterd/src/glusterd-replace-brick.c:942]: (error) Resource leak: file
[xlators/mgmt/glusterd/src/glusterd-replace-brick.c:1026]: (error) Resource leak: file
[xlators/mgmt/glusterd/src/glusterd-sm.c:249]: (error) Possible null pointer dereference: new_ev_ctx
[xlators/mgmt/glusterd/src/glusterd-snapshot.c:6917]: (error) Possible null pointer dereference: volinfo
[xlators/mgmt/glusterd/src/glusterd-utils.c:4517]: (error) Possible null pointer dereference: this
[xlators/mgmt/glusterd/src/glusterd-utils.c:6662]: (error) Possible null pointer dereference: this
[xlators/mgmt/glusterd/src/glusterd-utils.c:7708]: (error) Possible null pointer dereference: this
[xlators/mount/fuse/src/fuse-bridge.c:4687]: (error) Uninitialized variable: finh
[xlators/mount/fuse/src/fuse-bridge.c:3080]: (error) Possible null pointer dereference: state
[xlators/nfs/server/src/nfs-common.c:89]: (error) Dangerous usage of 'volname' (strncpy doesn't always null-terminate it).
[xlators/performance/quick-read/src/quick-read.c:586]: (error) Possible null pointer dereference: iobuf

Rerunning cppcheck after fixing the above:

As before, test program:

[extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments.
[tests/basic/fops-sanity.c:55]: (error) Buffer overrun possible for long command line arguments.

As before, false positive:

[geo-replication/src/gsyncd.c:100]: (error) Memory leak: str
[geo-replication/src/gsyncd.c:403]: (error) Memory leak: argv
[xlators/nfs/server/src/nlm4.c:1201]: (error) Possible null pointer dereference: fde
[xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr

False positive after fix:

[xlators/performance/quick-read/src/quick-read.c:584]: (error) Possible null pointer dereference: iobuf

Change-Id: Ia5a256281156dd1df2fa900caea7694d9d0a7077
BUG: 1122290
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/8351
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-by: Raghavendra Bhat

fuse: squash 64-bit inodes in readdirp when enable-ino32 is set

2015-06-03T11:47:27+00:00

The structures returned by readdirp contain the inode 2x. Only one of
them was squashed into 32-bits when enable-ino32 is enabled.

Backport of:
> Change-Id: I33a6d28fb118bb23971f918ffeb983d7f033106e
> BUG: 1223889
> Reviewed-on: http://review.gluster.org/10881
> Signed-off-by: Niels de Vos 
> Tested-by: Cyril Peponnet  [on release-3.5]

Change-Id: I33a6d28fb118bb23971f918ffeb983d7f033106e
BUG: 1223891
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10883
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat

build: only include sys/sysctl.h when sysctl() is used

2015-05-19T13:35:08+00:00

Backport of BZ 1198849 http://review.gluster.org/9963

(fails to build on X32. X32 ABI for amd64/x86_64 using
32-bit pointers.)

Change-Id: I92496eeadec5a339abcfe9b7fab4bef896d80da4
BUG: 1215421
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/10424
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat

32 bit fix: use off_t and not size_t for truncate()

2014-09-25T18:05:52+00:00

Make sure off_t and not size_t is used when holding file offsets for
ftruncate()/truncate(). It works on 64 bit machines where
sizeof(size_t) == sizeof(off_t) == 8, but breaks for big offsets on 32 bit
machines because sizeof(size_t) == 4 and sizeof(off_t) == 8

This is backport of Ia2637be772ba9b11731d59fdbffbd269f0ff56c8

BUG: 1138897
Change-Id: I8fe77a86831f0db4eff5b5c89efe004b9a0b29e9
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8743
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

mount/fuse: Handle fd resolution failures

2014-09-12T09:17:42+00:00

        Backport of http://review.gluster.org/8402

Problem:
Even when the fd resolution failed, the fop is continuing on the
new graph which may not have valid inode. This lead to NULL layout
subvols in dht which lead to crash in fsync after graph migration.

Fix:
- Remove resolution error handling in FUSE_FOP as it was only added
  to handle fd migration failures.
- check in fuse_resolve_done for fd resolution failures and fail the
  fop right away.
- loc resolution failures are already handled in the corresponding
  fops.
- Return errno from state->resolve.op_errno in resume functions.
- Send error to fuse on frame allocation failures.
- Removed unused variable state->resolved
- Removed unused macro FUSE_FOP_COOKIE

BUG: 1136827
Change-Id: I4010b7fccd7d8caf0ce4e7629e81b605102d8fb4
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8592
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Vijay Bellur

Always check for ENODATA with ENOATTR

2014-09-08T06:23:42+00:00

Linux defines ENODATA and ENOATTR with the same value, which means that
code can miss on on the two without breaking.

FreeBSD does not have ENODATA and GlusterFS defines it as ENOATTR just
like Linux does.

On NetBSD, ENODATA != ENOATTR, hence we need to check for both values
to get portable behavior.

This is a backport of  I003a3af055fdad285d235f2a0c192c9cce56fab8

BUG: 1138897
Change-Id: I272cd53e637993c7fd2ac74bd607001d3581ced7
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8634
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

fuse: fuse_readlink_cbk() - linkname NULL termination unnecessary

2014-08-03T14:56:44+00:00

op_ret incremented to compensate for NULL terminating character
leads to self referential loop where OSXFUSE which would reply
on a same READLINK() over and over again

~~~
[2014-07-12 08:41:29.815473]
   T [fuse-bridge.c:1372:fuse_readlink_cbk] 0-glusterfs-fuse:
                                              1: /a/b/1 => ../../1
[2014-07-12 08:41:29.815820]
   T [fuse-bridge.c:1372:fuse_readlink_cbk] 0-glusterfs-fuse:
                                              0: /a/b/1 => ../../1
[2014-07-12 08:41:29.816165]
   T [fuse-bridge.c:1372:fuse_readlink_cbk] 0-glusterfs-fuse:
                                              1: /a/b/1 => ../../1
~~~

It happens due to the problem being (op_ret + 1) > strlen(linkname),
for some odd reason this isn't an issue on Linux where there are odd
safegaurds on these things - Example of following code

~~~
((char *)linkname)[op_ret] = '\0';
send_fuse_data (this, finh, (void *)linkname, op_ret + 2048); <---- Here!
~~~

This behaves normally with no issue, the reasoning i have is that
internally 'readlink()' is verified with strlen() again or perhaps the size
is re-adjusted to the strlen() of `linkname`

This isn't the case on OSX, one needs to make sure that
~~~
strlen(linkname) == op_ret
~~~
Otherwise you would get READLINK() loops as shown above.

This patch fixes the problem.

Many thanks to Anand Avati for helping me out on this.

Change-Id: Ia35818de78a5e4d89bad03ab06e2c5ed6e6753a4
BUG: 1095525
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8336
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

porting: Port for FreeBSD rebased from Mike Ma's efforts

2014-07-03T00:20:34+00:00

- Provides a working Gluster Management Daemon, CLI
- Provides a working GlusterFS server, GlusterNFS server
- Provides a working GlusterFS client
- execinfo port from FreeBSD is moved into ./contrib/libexecinfo
  for ease of portability on NetBSD. (FreeBSD 10 and OSX provide
  execinfo natively)
- More portability cleanups for Darwin, FreeBSD and NetBSD
- Provides a new rc script for FreeBSD

Change-Id: I8dff336f97479ca5a7f9b8c6b730051c0f8ac46f
BUG: 1111774
Original-Author: Mike Ma 
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8141
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

Fix resolution issues across fuse/server/afr

2014-06-14T11:10:06+00:00

Problems with fuse/server:
Fuse loc touch up sets loc->name even when pargfid
is not known. Server lookup does (pargfid, name) based
lookup when name is set ignoring the gfid. Because of this server
resolver finds that the lookup came on (null-pargfid, name) and
fails the lookup with EINVAL.

Fix:
Don't set loc->name in loc_touchup if the pargfid is not known.
Did the same even for server-resolver

Problem with afr:
Lets say there is a directory hierarchy a/b/c/d on the mount and the
user is cd'ed into the directory. Bring down one of the bricks of replica and
remove all directories/files to simulate disk replacement on that brick. Now
this brick is brought back up. Creates on the cd'ed directory fail with ESTALE.
Basically before sending a create of 'f' inside 'd', fuse sends a lookup to
make sure the file is not present.  On one of the bricks  'd' is present and
'f' is not so it sends ENOENT as response. On the new brick 'd' itself is not
present. So it sends ESTALE. In afr ESTALE is considered to be special errno on
witnessing which lookup has to fail. And ESTALE is given more priority than
ENOENT. Due to these reasons lookup fails with ESTALE rather than ENOENT. Since
lookup didn't fail with ENOENT, 'create' can't be issued so the command is
failed with ESTALE.

Solution:
Afr needs to consider ESTALE errno normally and ENOENT needs to
be given more priority so that operations like create can proceed even when
only one of the brick is up and running. Whenever client xlator identifies
that gfid-changed, it sets that information in lookup xdata. Afr uses this
information to fail the lookup with ESTALE so that top xlator can send
fresh lookup.

Change-Id: Ica6ce01baef08620154050a635e6f97d51029ef6
BUG: 1106408
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8015
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur