glusterfs.git/xlators/protocol/server/src/server-resolve.c, branch v7.1

copy_file_range support in GlusterFS

2018-12-12T15:56:55+00:00

    * libglusterfs changes to add new fop

    * Fuse changes:
      - Changes in fuse bridge xlator to receive and send responses

    * posix changes to perform the op on the backend filesystem

    * protocol and rpc changes for sending and receiving the fop

    * gfapi changes for performing the fop

    * tools: glfs-copy-file-range tool for testing copy_file_range fop

      - Although, copy_file_range support has been added to the upstream
	    fuse kernel module, no release has been made yet of a kernel
        which contains the support. It is expected to come in the
        upcoming release of linux-4.20

        So, as of now, executing copy_file_range fop on a fused based
        filesystem results in fuse kernel module sending read on the
	    source fd and write on the destination fd.

	    Therefore a small gfapi based tool has been written to be able
        test the copy_file_range fop. This tool is similar (in functionality)
	    to the example program given in copy_file_range man page.

	    So, running regular copy_file_range on a fuse mount point and
	    running gfapi based glfs-copy-file-range tool gives some idea about
	    how fast, the copy_file_range (or reflink) can be.

	    On the local machine this was the result obtained.

	    mount -t glusterfs workstation:new /mnt/glusterfs
	    [root@workstation ~]# cd /mnt/glusterfs/
	    [root@workstation glusterfs]# ls
	    file
	    [root@workstation glusterfs]# cd
	    [root@workstation ~]# time /tmp/a.out /mnt/glusterfs/file /mnt/glusterfs/new
	    real  0m6.495s
	    user  0m0.000s
	    sys   0m1.439s
	    [root@workstation ~]# time glfs-copy-file-range $(hostname) new /tmp/glfs.log /file /rrr
	    OPEN_SRC: opening /file is success
	    OPEN_DST: opening /rrr is success
	    FSTAT_SRC: fstat on /rrr is success
	    copy_file_range successful

        real  0m0.309s
        user  0m0.039s
        sys   0m0.017s

        This tool needs following arguments
         1) hostname
         2) volume name
         3) log file path
         4) source file path (relative to the gluster volume root)
         5) destination file path (relative to the gluster volume root)

        "glfs-copy-file-range     "

      - Added a testcase as well to run glfs-copy-file-range tool

    * io-stats changes to capture the fop for profiling

    * NOTE:

      - Added conditional check to see whether the copy_file_range syscall
        is available or not. If not, then return ENOSYS.

      - Added conditional check for kernel minor version in fuse_kernel.h
        and fuse-bridge while referring to copy_file_range. And the kernel
        minor version is kept as it is. i.e. 24. Increment it in future
        when there is a kernel release which contains the support for
        copy_file_range fop in fuse kernel module.

    * The document which contains a writeup on this enhancement can be found at
      https://docs.google.com/document/d/1BSILbXr_knynNwxSyyu503JoTz5QFM_4suNIh2WwrSc/edit

Change-Id: I280069c814dd21ce6ec3be00a884fc24ab692367
updates: #536
Signed-off-by: Raghavendra Bhat

server: don't allow '/' in basename

2018-11-05T18:50:10+00:00

Server stack needs to have all the sort of validation, assuming
clients can be compromized. It is possible for a compromized
client to send basenames with paths with '/', and with that
create files without permission on server. By sanitizing the basename,
and not allowing anything other than actual directory as the parent
for any entry creation, we can mitigate the effects of clients
not able to exploit the server.

Fixes: CVE-2018-14651

Fixes: bz#1644755
Change-Id: I5dc0da0da2713452ff2b65ac2ddbccf1a267dc20
Signed-off-by: Amar Tumballi

Land part 2 of clang-format changes

2018-09-12T12:22:45+00:00

Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu

server-protocol: don't allow '../' path in 'name'

2018-09-05T09:23:26+00:00

This will prevent any arbitrary file creation through glusterfs
by modifying the client bits.

Also check for the similar flaw inside posix too, so we prevent any
changes in layers in-between.

Fixes: bz#1625095

Signed-off-by: Amar Tumballi 
Change-Id: Id9fe0ef6e86459e8ed85ab947d977f058c5ae06e

server/resolver: don't trust inode-table for RESOLVE_NOT

2018-04-30T04:58:08+00:00

There have been known races between fops which add a dentry (like
lookup, create, mknod etc) and fops that remove a dentry (like rename,
unlink, rmdir etc) due to which stale dentries are left out in inode
table even though the dentry doesn't exist on backend. For eg.,
consider a lookup (parent/bname) and unlink (parent/bname) racing in
the following order:

* lookup hits storage/posix and finds that dentry exists
* unlink removes the dentry on storage/posix
* unlink reaches protocol/server where the dentry (parent/bname) is
  unlinked from the inode
* lookup reaches protocol/server and creates a dentry (parent/bname)
  on the inode

Now we've a stale dentry (parent/bname) associated with the inode in
itable. This situation is bad for fops like link, create etc which
invoke resolver with type RESOLVE_NOT. These fops fail with EEXIST
even though there is no such dentry on backend fs. This issue can be
solved in two ways:

* Enable "dentry fop serializer" xlator [1].
  # gluster volume set features.sdfs on

* Make sure resolver does a lookup on backend when it finds a dentry
  in itable and validates the state of itable.
   - If a dentry is not found, unlink those stale dentries from itable
     and continue with fop
   - If dentry is found, fail the fop with EEXIST

This patch implements second solution as sdfs is not enabled by
default in brick xlator stack. Once sdfs is enabled by default, this
patch can be reverted.

[1] https://github.com/gluster/glusterfs/issues/397

Change-Id: Ia8bb0cf97f97cb0e72639bce8aadb0f6d3f4a34a
updates: bz#1543279
BUG: 1543279
Signed-off-by: Raghavendra G

protocol/server: fix unused variable warnings/errors

2016-08-27T11:17:49+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: I5a48055c4b0a4a07dd511ad4f2e9ef823b9c60b2
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15255
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

libglusterfs: Negate all but O_DIRECT flag if present on anon fds

2016-06-10T05:05:46+00:00

This is to prevent any unforeseen problems that might arise due to
writevs and readvs being wound with @flag parameter containing
O_TRUNC or O_APPEND especially wrt translators like sharding and ec
where O_TRUNC write or O_APPEND write on individual shards/fragments
is not the same as O_TRUNC write or O_APPEND write as expected by the
application.

Change-Id: I9e5206a6ce2b1b70df61ff23b1c961cf25bf7ff9
BUG: 1342171
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14665
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Tested-by: Gluster Build System 
NetBSD-regression: NetBSD Build System

core, shard: Make shards inherit main file's O_DIRECT flag if present

2016-06-07T07:19:26+00:00

If the application opens a file with O_DIRECT, the shards'
anon fds would also need to inherit the flag. Towards this,
shard xl would be passing the odirect flag in the @flags parameter
to the WRITEV fop. This will be used in anon fd resolution
and subsequent opening by posix xl.

Change-Id: Iddb75c9ed14ce5a8c5d2128ad09b749f46e3b0c2
BUG: 1342171
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14191
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

server: send lookup on root inode when itable is created

2016-03-30T19:39:42+00:00

 * xlators like quota, marker, posix_acl can cause problems
   if inode-ctx are not created.
   sometime these xlarors may not get lookup on root inode
   with below cases
   1) client may not send lookup on root inode (like NSR leader)
   2) if the xlators on one of the bricks are not up,
      and client sending lookup during this time: brick
      can miss the lookup
   It is always better to make sure that there is one lookup
   on root. So send a first lookup when the inode table is created

 * When sending lookup on root, new inode is created, we need to
   use itable->root instead

Change-Id: Iff2eeaa1a89795328833a7761789ef588f11218f
BUG: 1320818
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/13837
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

protocol/server : porting missing gf_log's to gf_msg

2015-08-26T12:08:24+00:00

Change-Id: I8818931fafea3c013551a5de23a9f77c81164841
BUG: 1252808
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/11895
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G