glusterfs.git/api/src, branch v3.9.0rc2

api: revert addition of glfs_ipc_xd()

2016-10-14T20:29:15+00:00

It is intended for 4.0

Cherry picked from commit f4e5841eb6f132174e65ccad91fd2f6021771f29:
> Change-Id: Ieffe1d966234652091a4a9ae0b2c4b23f1297147
> BUG: 1345977
> Signed-off-by: Kaleb S KEITHLEY 
> Reviewed-on: http://review.gluster.org/14716
> Smoke: Gluster Build System 
> Reviewed-by: Jeff Darcy 
> Reviewed-by: Shyamsundar Ranganathan 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Niels de Vos 
> CentOS-regression: Gluster Build System 

Note that this 'backport' is done on a patch from the release-3.8
branch, and it not part of the master branch.

Change-Id: Ieffe1d966234652091a4a9ae0b2c4b23f1297147
BUG: 1364529
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15642
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

gfapi: warn when glfs_realpath() returned malloc'd memory

2016-10-11T18:54:21+00:00

glfs_realpath() may return memory allocated with malloc(). Depending on
the memory allocator that the application uses, calling free() on the
returned string can cause segmentation faults or other problems.

Functions that allocate memory, need to match the free'ing of the same
memory allocator and memory accounting. glibc/malloc and jemalloc/free
do not match together (other allocators could probably trigger these
problems as well).

Applications need to provide a pre-allocated buffer, or in case
glfs_realpath() allocates the memory, glfs_free() should be used to free
it.

Cherry picked from commit 85e959052148ec481823d55c8b91cdee36da2b43:
> Change-Id: I5d721a7425674aa700db8a7a436cbedb95a5927f
> BUG: 1370931
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/15332
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I5d721a7425674aa700db8a7a436cbedb95a5927f
BUG: 1383591
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15621
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

gfapi: redesign the public interface for upcall consumers

2016-10-11T12:18:56+00:00

The glfs_callback_arg and glfs_callback_inode_arg were allocated by
gfapi, and expected to be free()'d by the application. However it is not
reasonable to expect that applications use the same memory allocator to
as the compiled libgfapi.so. For instance, it is possible that gfapi
uses glibc malloc/free, and an application like NFS-Ganesha the versions
from jemalloc. Mismatching of the malloc() and free() functions causes
segmentation faults at best.

In order to prevent problems like this in the future, the API for
applications that consume upcalls has been remodeled. Any of the
structures that gfapi allocates, should be free'd with glfs_free(). The
members of the structures can not be accessed directly anymore, each
has its own function to access now.

Correcting the naming of the functions, structures and constants is a
continuation of commit 2775dc64101ed37c8d9809bf9852dbf0746ee2b6. These
new improvements not only have correct prefixes for the functions and
structures, the naming also reflects more to the upcall framework and
does not use "callback" anymore.

Cherry picked from commit 4721188a154acd9a0a4c096d8d73e97f3bf1b2a9:
> Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
> BUG: 1344714
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/14701
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: soumya k 
> Reviewed-by: jiffin tony Thottan 

Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93
BUG: 1378948
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15597
NetBSD-regression: NetBSD Build System 
Reviewed-by: jiffin tony Thottan 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

md-cache: Register the list of xattrs with cache-invalidation

2016-08-31T06:07:01+00:00

Issue:
md-cache caches a specified list of xattrs, and when cache invalidation
is enabled, it makes sense to recieve invalidation only when those xattrs
are modified by other clients. But the current implementation of upcall
is that, it will send invalidation when any of the on-disk xattrs is modified.

Solution:
md-cache sends a list of xattrs that it is interested in, to upcall by
issuing an ipc(). The challenge here is to make sure everytime a brick
goes offline and comes back up, the ipc() needs to be issued to the
bricks. Hence ipc() is sent from md-cache every time there is a
CHILD_UP/CHILD_MODIFIED event.

TODO:
There will be patches following, in cluster xlators, to implement ipc fop.

Change-Id: I6efcf3df474f5ce6eabd3d6694c00c7bd89bc25d
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15002
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Raghavendra G

gfapi: SSL connection for mgmt connection is not working

2016-08-29T06:19:33+00:00

Problem: libgfapi does not enable SSL on mgmt connection.

Fix: Enable SSL when it is enabled on mgmt connection is enabled,
i.e. presence of /var/lib/glusterd/secure-access file

Change-Id: I1ce4935b04e6140aeab819e42076defd580b0727
BUG: 1362602
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15073
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Kaushal M

gfapi: do not cache upcalls if the application is not interested

2016-08-25T20:35:43+00:00

When the volume option 'features.cache-invalidation' is enabled, upcall
events are sent from the brick process to the client. Even if the client
is not interested in upcall events itself, md-cache or other xlators may
benefit from them.

By adding a new 'cache_upcalls' boolean in the 'struct glfs', we can
enable the caching of upcalls when the application called
glfs_h_poll_upcall(). NFS-Ganesha sets up a thread for handling upcalls
in the initialization phase, and calls glfs_h_poll_upcall() before any
NFS-client accesses the NFS-export.

In the future there will be a more flexible registration API for
enabling certain kind of upcall events. Until that is available, this
should work just fine.

Verificatio of this change is not trivial within our current regression
test framework. The bug report contains a description on how to reliably
reproduce the problem with the glusterfs-coreutils.

Change-Id: I818595c92db50e6e48f7bfe287ee05103a4a30a2
BUG: 1368842
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15191
Smoke: Gluster Build System 
Reviewed-by: Poornima G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Kaleb KEITHLEY

gfapi: add missing glfs_truncate

2016-08-11T13:19:33+00:00

Change-Id: I80b016090a4d9d86278a0a5144dd58c0cbfe9bb2
BUG: 1365489
Signed-off-by: Jeff Darcy 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/13927
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY

gfapi: use const qualifier for glfs_*timens()

2016-08-10T11:26:47+00:00

glfs_*timens() functions have the last argument
of struct timespec type that is supposed to be
const. Now using glfs_*timens() in fops, implemented
on top of FUSE library, leads to compiler-time warning
about discarding const qualifier.

Introducing const qualifier does not break ABI,
so let's just fix it.

Change-Id: Iea2a1018de7dce67f67a8229671a5978246de800
BUG: 1365506
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15119
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

gfapi: Fix IO error caused when there is consecutive graph switches

2016-08-10T10:40:58+00:00

This is part 2 of the fix, the part 1 can be found at:
http://review.gluster.org/#/c/14656/

Problem:
=======
Consider a race between, __glfs_active_subvol() and graph_setup().
Lets say @TIME T1:
fs->active_subvol = A
fs->next_subvol = B
__glfs_active_subvol()                //under lock fs->mutex
{
  ....
  new_subvol = fs->next_subvol       //which is B
  ....                               //Start migration from A to B
  __glfs_first_lookup(){
     ....
     unlock fs->mutex                //@TIME T2
     network fop
     lock fs->mutex
     ....
  }
  ....                                //migration continue on B
  fs->active_subvol = fs->next_subvol //which is C (explained below)
  ....
}

@Time T2, lets say in another thread, graph_setup() is called with C,
note that at T2, fs->mutex is unlocked.

graph_stup(C...)
{
  lock fs->mutex
  ....
  if (fs->next_subvol)                // which is B
      destroy subvol (fs->next_subvol)
  ....
  fs->next_subvol = C
  ....
  unlock fs->mutex
}

Thus at the end of this,
fs->old_subvol = A;
fs->active_subvol = C;
fs->next_subvol = NULL;
which is wrong, as B completed migration, but was destroyed by
graph_setup, and C never was migrated.

Solution:
=========
Any new graph can be in one of the 2 states:
- Picked for migration, migration in progress (fs->mip_subvol)
- Not picked so far for migration (fs->next_subvol)
graph_setup() updates fs->next_subvol only, __glfs_active_subvol()
moves fs->next_subvol to fs->mip_subvol and fs->next_subvol = NULL
atomically, and then once the migration is complete, make that the
fs->active_subvol

Change-Id: Ib6ff0565105c5eedb912a43da4017cd413243612
BUG: 1343038
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14722
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Niels de Vos

gfapi: fix glfs_set_volfile_server doc

2016-08-02T17:06:22+00:00

Gluster supports volume file fetching over transports tcp and unix only.
The current glfs_set_volfile_server() doc misleads by mentioning rdma,
which is removed by this patch.

More about the discussion around @
http://www.gluster.org/pipermail/gluster-devel/2016-July/050114.html

Change-Id: I08cfe262fb9a82cb341f6adbe93594c8e628b7fe
BUG: 1359370
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14995
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur