glusterfs.git/xlators/mount/fuse/utils, branch v3.12.14

mount/fuse : Fix parsing of vol_id for snapshot volume

2017-10-15T19:39:35+00:00

For supporting sub-dir mount, we changed the volid. Which means anything
after a '/' in volume_id will be considered as sub-dir path.

But snapshot volume has vol_id stracture of /snaps//
which has to be considered as during the parsing.

Note 1: sub-dir mount is not supported on snapshot volume
Note 2: With sub-dir mount changes brick based mount for quota cannot be
        executed via mount command. It has to be a direct call via glusterfs

Backport of>
>Change-Id: I0d824de0236b803db8a918f683dabb0cb523cb04
>BUG: 1501235
>Signed-off-by: Mohammed Rafi KC 
Change-Id: I0d824de0236b803db8a918f683dabb0cb523cb04
BUG: 1501238
Signed-off-by: Mohammed Rafi KC 
(cherry picked from commit 067f38063e13fc75d4e3f7adf93441d15099c557)

mount/fuse: Make event-history feature configurable

2017-10-05T12:02:42+00:00

... and disable it by default.

        Backport of:
        > Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
        > Reviewed-on: https://review.gluster.org/18242
        > BUG: 1467614
        > cherry-picked from commit 956d43d6e89d40ee683547003b876f1f456f03b6

This is because having it disabled seems to improve performance.
This could be due to the lock contention by the different epoll threads
on the circular buff lock in the fop cbks just before writing their response
to /dev/fuse.

Just to provide some data - wrt ovirt-gluster hyperconverged
environment, I saw an increase in IOPs by 12K with event-history
disabled for randrom read workload.

Usage:
mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT
OR
glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT

Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
BUG: 1495397
Signed-off-by: Krutika Dhananjay

scripts: mount.glusterfs contains non-portable bashisms

2017-09-06T07:45:01+00:00

Debian's default shell is dash, i.e. /bin/sh -> dash, which doesn't
support bash extensions

Cherry picked from commit c0406501f0b4295a9917d31574baa2e462a18799:
> Reported-by: "Michael Lundkvist" 
> Reported-by: pmatthaei@debian.org
> Debian BZ: 873878
> Change-Id: I33003183b9bc6459cae28c565125e6b2bd1eaa47
> BUG: 1487830
> Signed-off-by: Kaleb S. KEITHLEY 
> Reviewed-on: https://review.gluster.org/18184
> Reviewed-by: Niels de Vos 
> Tested-by: Niels de Vos 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 

Reported-by: Michael Lundkvist 
Reported-by: Patrick Matthäi 
Debian BZ: 873878
Change-Id: I33003183b9bc6459cae28c565125e6b2bd1eaa47
BUG: 1488119
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/18205
Smoke: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System

glusterfsd: allow subdir mount

2017-08-04T13:34:21+00:00

Changes:

1. Take subdir mount option in client (mount.gluster / glusterfsd)
2. Pass the subdir mount to server-handshake (from client-handshake)
3. Handle subdir-mount dir's lookup in server-first-lookup and handle
   all fops resolution accordingly with proper gfid of subdir
4. Change the auth/addr module to handle the multiple subdir entries
   in option, and valid parsing.

How to use the feature:

`# mount -t glusterfs $hostname:/$volname/$subdir /$mount_point`
Or
`# mount -t glusterfs $hostname:/$volname -osubdir_mount=$subdir /$mount_point`

Option can be set like:

`# gluster volume set  auth.allow "/subdir1(192.168.1.*),/(192.168.10.*),/subdir2(192.168.8.*)"`

Updates #175

> Reviewed-At: https://review.gluster.org/17141/

Change-Id: I7ea57f76ddbe6c3862cfe02e13f89e8a39719e11
Signed-off-by: Amar Tumballi 
Reviewed-on: https://review.gluster.org/17968
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

fuse: implement "-oauto_unmount"

2017-05-23T13:21:25+00:00

libfuse has an auto_unmount option which,
if enabled, ensures that the file system
is unmounted at FUSE server termination
by running a separate monitor process
that performs the unmount when that
occurs. (This feature would probably
better be called "robust auto-unmount",
as FUSE servers usually do try to unmount
their file systems upon termination,
it's just this mechanism is not crash
resilient.)

This change implements that option and
behavior for glusterfs.

Note that "auto unmount" (robust or not) is
a leaky abstraction, as the kernel cannot
guarantee that at the path where the FUSE
fs is mounted is actually the toplevel mount
at the time of the umount(2) call, for
multiple reasons, among others, see:

  fuse-devel: "fuse: feasible to distinguish between umount and abort?"
  http://fuse.996288.n3.nabble.com/fuse-feasible-to-distinguish-between-umount-and-abort-tt14358.html
  https://github.com/libfuse/libfuse/issues/122

Updates #153

Change-Id: Ia4432580c9fd2c156d9c73c3a44f4bfd42437599
Signed-off-by: Csaba Henk 
Reviewed-on: https://review.gluster.org/17230
Tested-by: Amar Tumballi 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Amar Tumballi 
NetBSD-regression: NetBSD Build System

Halo Replication feature for AFR translator

2017-05-02T10:23:53+00:00

Summary:
Halo Geo-replication is a feature which allows Gluster or NFS clients to write
locally to their region (as defined by a latency "halo" or threshold if you
like), and have their writes asynchronously propagate from their origin to the
rest of the cluster.  Clients can also write synchronously to the cluster
simply by specifying a halo-latency which is very large (e.g. 10seconds) which
will include all bricks.

In other words, it allows clients to decide at mount time if they desire
synchronous or asynchronous IO into a cluster and the cluster can support both
of these modes to any number of clients simultaneously.

There are a few new volume options due to this feature:
  halo-shd-latency:  The threshold below which self-heal daemons will
  consider children (bricks) connected.

  halo-nfsd-latency: The threshold below which NFS daemons will consider
  children (bricks) connected.

  halo-latency: The threshold below which all other clients will
  consider children (bricks) connected.

  halo-min-replicas: The minimum number of replicas which are to
  be enforced regardless of latency specified in the above 3 options.
  If the number of children falls below this threshold the next
  best (chosen by latency) shall be swapped in.

New FUSE mount options:
  halo-latency & halo-min-replicas: As descripted above.

This feature combined with multi-threaded SHD support (D1271745) results in
some pretty cool geo-replication possibilities.

Operational Notes:
- Global consistency is gaurenteed for synchronous clients, this is provided by
  the existing entry-locking mechanism.
- Asynchronous clients on the other hand and merely consistent to their region.
  Writes & deletes will be protected via entry-locks as usual preventing
  concurrent writes into files which are undergoing replication.  Read operations
  on the other hand should never block.
- Writes are allowed from _any_ region and propagated from the origin to all
  other regions.  The take away from this is care should be taken to ensure
  multiple writers do not write the same files resulting in a gfid split-brain
  which will require resolution via split-brain policies (majority, mtime &
  size).  Recommended method for preventing this is using the nfs-auth feature to
  define which region for each share has RW permissions, tiers not in the origin
  region should have RO perms.

TODO:
- Synchronous clients (including the SHD) should choose clients from their own
  region as preferred sources for reads.  Most of the plumbing is in place for
  this via the child_latency array.
- Better GFID split brain handling & better dent type split brain handling
  (i.e. create a trash can and move the offending files into it).
- Tagging in addition to latency as a means of defining which children you wish
  to synchronously write to

Test Plan:
- The usual suspects, clang, gcc w/ address sanitizer & valgrind
- Prove tests

Reviewers: jackl, dph, cjh, meyering

Reviewed By: meyering

Subscribers: ethanr

Differential Revision: https://phabricator.fb.com/D1272053

Tasks: 4117827

Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1
BUG: 1428061
Signed-off-by: Kevin Vigor 
Reviewed-on: http://review.gluster.org/16099
Reviewed-on: https://review.gluster.org/16177
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

glusterfsd/main: Add ability to set oom_score_adj

2016-06-01T13:27:10+00:00

Give the administrator a possibility to set oom_score_adj for glusterfs
process. Applies to Linux only.

Change-Id: Iff13c2f4cb28457871c6ebeff6130bce4a8bf543
BUG: 1336818
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/14399
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

fuse: accept the -s option to allow automounting

2016-05-31T09:01:41+00:00

autofs passes the -s option when mounting. All /sbin/mount. helpers
accept this, except mount.glusterfs. Because the helper fails when -s is
passed accessing the mountpoint through autofs gives the following
error:

  $ ls /lan/storage.lan.example.net/repos
  ls: cannot open directory /lan/storage.lan.example.net/repos: Too many levels of symbolic links

BUG: 1340936
Change-Id: I84755cdac59e630618cb745c0eb3228cc1e93a1a
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14559
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Oleksandr Natalenko

fuse: Add a new mount option capability

2016-03-08T05:57:28+00:00

Originally all security.* xattrs were forbidden if selinux is disabled,
which was causing Samba's acl_xattr module to not work, as it would
store the NTACL in security.NTACL. To fix this http://review.gluster.org/#/c/12826/
was sent, which forbid only security.selinux. This opened up a getxattr
call on security.capability before every write fop and others.

Capabilities can be used without selinux, hence if selinux is disabled,
security.capability cannot be forbidden. Hence adding a new mount
option called capability.

Only when "--capability" or "--selinux" mount option is used,
security.capability is sent to the brick, else it is forbidden.

Change-Id: I77f60e0fb541deaa416159e45c78dd2ae653105e
BUG: 1309462
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13540
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd: Bug fixes for IPv6 support

2016-02-20T17:16:42+00:00

Problem:
Glusterd not working using ipv6 transport. The idea is with proper glusterd.vol configuration,
1. glusterd needs to listen on default port (240007) as IPv6 TCP listner.
2. Volume creation/deletion/mounting/add-bricks/delete-bricks/peer-probe
   needs to work using ipv6 addresses.
3. Bricks needs to listen on ipv6 addresses.
All the above functionality is needed to say that glusterd supports ipv6 transport and this is broken.

Fix:
When "option transport.address-family inet6" option is present in glusterd.vol
file, it is made sure that glusterd creates listeners using ipv6 sockets only and also the same information is saved
inside brick volume files used by glusterfsd brick process when they are starting.

Tests Run:
Regression tests using ./run-tests.sh
    IPv4: Ran manually till tests/basic/rpm.t .
    IPv6: (Need to add the above mentioned config and also add an entry for "hostname ::1" in /etc/hosts)
        Started failing at ./tests/basic/glusterd/arbiter-volume-probe.t and ran successfully till here

Unit Tests using Ipv6
    peer probe
    add-bricks
    remove-bricks
    create volume
    replace-bricks
    start volume
    stop volume
    delete volume

Change-Id: Iebc96e6cce748b5924ce5da17b0114600ec70a6e
BUG: 1117886
Signed-off-by: Nithin D 
Reviewed-on: http://review.gluster.org/11988
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Jeff Darcy