BUG: 1534602
Change-Id: Ide42cf9cffe462d0cc46272b327c2a05999f09ba
Signed-off-by: Csaba Henk <csaba@redhat.com>

Added 2 more types which are present in gluster codebase, mainly
IATT and UUID.

Updates #203

Change-Id: Ib6d6d6aefb88c3494fbf93dcbe08d9979484968f
Signed-off-by: Amar Tumballi <amarts@redhat.com>

- call_stack_set_group() will take the ownership of passed
  buffer from caller;
- to indicate the change, its signature is changed from
  including the buffer directly to take a pointer to it;
- either the content of the buffer is copied to the
  groups_small embedded buffer of the call stack, or
  the buffer is set as groups_large member of the call
  stack;
- the groups member of the call stack is set to,
  respectively, groups_small or groups_large, according
  to the memory management conventions of the call stack;
- the buffer address is overwritten with junk to effectively
  prevent the caller from using it further on.

Also move call_stack_set_group to stack.c from stack.h
to prevent "defined but not used [-Wunused-function]"
warnings (not using it anymore in call_stack_alloc_group()
implementation, which saved us from this so far).

protocol/server: refactor gid_resolve()

In gid_resolve there are two cases:
either the gid_cache_lookup() call returns
a value or not. The result is caputured in
the agl variable, and throughout the function,
each particular stage of the implementation
comes with an agl and a no-agl variant.

In most cases this is explicitly indicated
via an

   if (agl) {
      ...
   } else {
      ...
   }

but some of this branching are expressed via
goto constructs (obfuscating the fact we stated
above, that is, each particular stage having
an agl/no-agl variant).

In the current refactor, we bring the agl
conditional to the top, and present the
agl/non-agl implementations sequentially.

Also we take the opportunity to clean up and
fix the agl case:
- remove the spurious
    gl.gl_list = agl->gl_list;
  setting, as gl is not used in the agl caae
- populate the group list of call stack from
  agl, fixing thus referred BUG.

Also fixes BUG: 1513920

Change-Id: I61f4574ba21969f7661b9ff0c9dce202b874025d
BUG: 1513928
Signed-off-by: Csaba Henk <csaba@redhat.com>

The restriction of using fds opened by the same Pid means fds cannot
be shared across threads of multithreaded application. Note that fops
from kernel have different Pid for different threads. Imagine
following sequence of operations:

* Turn off performance.open-behind
* Thread t1 opens an fd - fd1 - on file "file". Let's assume nodeid of
  "file" is "nodeid-file".
* Thread t2 does RENAME ("newfile", "file"). Let's assume nodeid of
  "newfile" as "nodeid-newfile".
* t2 proceeds to do fstat (fd1)

The above set of operations can sometimes result in ESTALE/ENOENT
errors. RENAME overwrites "file" with "newfile" changing its nodeid
from "nodeid-file" to "nodeid-newfile" and post RENAME, "nodeid-file" is
removed from the backend. If fstat carries nodeid-file as argument,
which can happen if lookup has not refreshed the nodeid of "file" and
since t2 doesn't have an fd opened, fuse_getattr_resume uses STAT
which will fail as "nodeid-file" no longer exists.

Since the above set of operations and sharing of fds across
multiple threads are valid, this is a bug.

The fix is to use any fd opened on the inode. In this specific example
fuse_getattr_resume will find fd1 and winds down the call as fstat
(fd1) which won't fail.

Cross-checked with "Miklos Szeredi" <mszeredi.at.redhat.dot.com> for
any security issues with this solution and he approves the solution.

Thanks to "Miklos Szeredi" <mszeredi.at.redhat.dot.com> for all the
pointers and discussions.

Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c
BUG: 1510401
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>

This reduces the space used from four bytes to one, and allows
new code to use familiar C99 types/values interoperably with our
old cruft. It does *not* change current declarations or code;
that will be left for a separate - much larger - patch.

Updates: #80

Change-Id: I5baedd17d3fb05b38f0d8b8bb9dd62824475842e
Signed-off-by: Jeff Darcy <jdarcy@fb.com>

Updates: #242
BUG: 1428063
Change-Id: Iaaf2edf99b2ecc75f6d30762c752a6d445c1c826
Signed-off-by: Poornima G <pgurusid@redhat.com>

open(dir) being an operation on inode should never fail with
ENOENT. If gfid is not present, the appropriate error is ESTALE. This
will enable kernel to retry open after a revalidate lookup.

Change-Id: I8d07d2ebb5a0da6c3ea478317442cb42f1797a4b
BUG: 1500269
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>

This reverts commit 26d16b90ec7f8acbe07e56e8fe1baf9c9fa1519e.

Consider rename (index.new, store.idx) and open (store.idx) being
executed in parallel. When we break down operations following sequence
is possible.

* lookup (store.idx) - as part of open(store.idx) returns gfid1 as the
  result.
* rename (index.new, store.idx) changes gfid of store.idx to
  gfid2. Note that gfid2 was the nodeid of index.new. Since rename is
  successful, gfid2 is associated with store.idx.
* open (store.idx) resumes and issues open fop to glusterfs with
  gfid1. open in glusterfs fails as gfid1 doesn't exist and the error
  returned by glusterfs to kernel-fuse is ENOENT.
* kernel passes back the same error to application as a result to
  open.

This error could've been prevented if kernel retries open with
gfid2. Interestingly kernel do retry open when it receives ESTALE
error. Even though failure to find gfid resulted in ESTALE error,
commit 26d16b90ec7f8acb converted that error to ENOENT while sending
an error reply to kernel. This prevented kernel from retrying open
resulting in error.

Change-Id: I2e752ca60dd8af1b989dd1d29c7b002ee58440b4
BUG: 1500269
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>

... and disable it by default.

This is because having it disabled seems to improve performance.
This could be due to the lock contention by the different epoll threads
on the circular buff lock in the fop cbks just before writing their response
to /dev/fuse.

Just to provide some data - wrt ovirt-gluster hyperconverged
environment, I saw an increase in IOPs by 12K with event-history
disabled for randrom read workload.

Usage:
mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT
OR
glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT

Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
BUG: 1467614
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>

background: Various xlators used to populate their ctx, on
an explicit lookup. That means without a lookup, the translator will have
either null or stale data to function. E.g. dht would depend on lookup to
create linkto files on the correct node/hashed subvol, afr would rely on
this lookup to heal pending data/metadata etc.

So to complete above actions a lookup used to be issued on files,
even their inode was populated on a readdirp_cbk. This was done
by setting the need_lookup flag on all the files those were read
on readdirp fop.

We tried a small test on "ACL client". For listing 50k files on root
itself, it took around 50seconds with readdirp enabled while
the same operation took 5-6 seconds with readdirp disabled. Both the
times md-cache was enabled.

We observed that on the 1st test case (readdirp enabled), post readdirp
a getxattr is done. The number of getxattr depends on the number of acl
xattrs (I saw requests on these two: system.posix_acl_default,
system.posix_acl_access). Since need_lookup flag is set, during fuse_resolve
a nameless lookup is executed on the inode(getxattr being inode operation,
hence the nameless lookup). Since md-cache does not serve nameless lookup,
a network hop is needed for each file, costing the time.

With readdirp disabled, the getxattrs are served from md-cache itself(note:
we are discussing the 2nd attempt of ls -l use case).

_Current affairs around need of lookup for a file to populate it's ctx_:

For the xlators on client stack we discussed quite extensively about the need
for a lookup fop post readdirp in all three cluster translators - afr, EC and
dht. EC and dht don't really need a nameless lookup post readdirp. For afr too,
the need for lookup was negated with patch (http://review.gluster.org/6010 - AFRV2),
where  afr added a function called afr_inode_refresh() which does a lookup and
populates its inode context in case a FOP came to AFR without a lookup being issued
prior to it.

We ran a thread on gluster-devel asking for feedback on the need of explicit lookup
post readdirp. For responses refer [1]. Refer [2] for discussions happened on gerrit.
After gathering inputs from [1] and [2], it looks like there is no xlator in
current state that requires an explicit lookup post readdirp to function properly.

* A separate similar patch will be sent for gfapi/nfs/nfs-ganesha.

Note: Only file's inode is built with readdirp.

[1] http://lists.gluster.org/pipermail/gluster-devel/2017-August/053505.html
[2] https://review.gluster.org/#/c/17985/

Change-Id: Ie1d68ce7bea5e1f8a1fab9a62217f478322554f5
BUG: 1492996
Signed-off-by: Susant Palai <spalai@redhat.com>