glusterfs.git/api, branch v3.10.1

gfapi: OBS build fails in post build analysis

2017-02-23T19:15:22+00:00

Originally gfapi: create statedump when glusterd requests it

When GlusterD sends the STATEDUMP procedure to the libgfapi client, the
client checks if it matches the PID that should take the statedump. If
so, it will do a statedump for the glfs_t that is connected to this mgmt
connection.

See https://bugzilla.redhat.com/show_bug.cgi?id=1169302#c25 for the
OpenSuSE Build System post build analysis error.

original Change-Id: I70d6a1f4f19d525377aebc8fa57f51e513b92d84
master       https://review.gluster.org/#/c/16415/
release-3.10 https://review.gluster.org/#/c/16602/

BUG: 1418981
Change-Id: I1cff59ed34ac963334416d4271f7a187cd88bff6
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16723
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

gfapi: create statedump when glusterd requests it

2017-02-14T01:53:27+00:00

When GlusterD sends the STATEDUMP procedure to the libgfapi client, the
client checks if it matches the PID that should take the statedump. If
so, it will do a statedump for the glfs_t that is connected to this mgmt
connection.

> BUG: 1169302
> See-also: http://review.gluster.org/9228
> Signed-off-by: Poornima G 
> [ndevos: separated patch from 9228]
> Reviewed-on: https://review.gluster.org/16415
> Reviewed-by: Niels de Vos 
> Tested-by: Niels de Vos 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Prashanth Pai 

BUG: 1418981
Change-Id: I70d6a1f4f19d525377aebc8fa57f51e513b92d84
Signed-off-by: Shyam 
Reviewed-on: https://review.gluster.org/16602
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

gfapi: add API to trigger events for debugging and troubleshooting

2017-02-14T01:53:23+00:00

Introduce glfs_sysrq() as a generic API for triggering debug and
troubleshoot events. This interface will be used by the feature to get
statedumps for applications using libgfapi.

The current events that can be requested through this API are:
 - 'h'elp: log a mesage with all supported events
 - 's'tatedump: trigger a statedump for the passed glfs_t

In future, this API can be used by a CLI to trigger statedumps from
storage servers. At the moment it is limited to take statedumps, but it
is extensible to set the log-level, clear caches, force reconnects and
much more.

> BUG: 1169302
> Change-Id: I18858359a3957870cea5139c79efe1365a15a992
> Original-author: Poornima G 
> Signed-off-by: Niels de Vos 
> Reviewed-on-master: http://review.gluster.org/16414
> Reviewed-by: Prashanth Pai 
> Reviewed-by: Kaleb KEITHLEY 

BUG: 1418981
Change-Id: I18858359a3957870cea5139c79efe1365a15a992
Signed-off-by: Shyam 
Reviewed-on: https://review.gluster.org/16600
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: run many bricks within one glusterfsd process

2017-02-02T00:54:58+00:00

This patch adds support for multiple brick translator stacks running in
a single brick server process.  This reduces our per-brick memory usage
by approximately 3x, and our appetite for TCP ports even more.  It also
creates potential to avoid process/thread thrashing, and to improve QoS
by scheduling more carefully across the bricks, but realizing that
potential will require further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global
option.  By default it's off, and bricks are started in separate
processes as before.  If multiplexing is enabled, then *compatible*
bricks (mostly those with the same transport options) will be started in
the same process.

Backport of:
> Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/14763

Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8
BUG: 1418091
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16496
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

libgfapi: remove/revert glfs_ipc() changes targeted for 4.0

2017-01-24T10:42:07+00:00

revert glfs_ipc() to 3.7.0 signature and associated symbol versioning

Change-Id: Ieffe1d966234652091a4a9ae0b2c4b23f1297147
BUG: 1415279
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16450
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

Make getvolfile.py example pep8 compliant

2017-01-12T19:02:01+00:00

Change-Id: I7d3c1e871b9fa3859745f4d67acf05f16c7ccf5f
BUG: 1412002
Signed-off-by: Patrick Uiterwijk 
Reviewed-on: http://review.gluster.org/16372
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

gfapi: glfs_subvol_done should NOT wait for graph migration.

2016-11-30T07:52:42+00:00

In graph_setup function glfs_subvol_done is called which
is executed in an epoll thread. glfs_lock waits on other
thread to finish graph migration. This can lead to dead lock
if we consume all the epoll threads.

In general any call-back function executed in epoll thread
should not call any blocking call which waits on a network
reply either directly or indirectly, e.g. syncop functions
should not be called in these threads.

As a fix we should not wait for migration in the call-back path.

Change-Id: If96d0689fe1b4d74631e383048cdc30b01690dc2
BUG: 1397754
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15913
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System

gfapi: Fix memory leak in glfs-mgmt

2016-11-28T08:49:03+00:00

dictionary was not freed after serialization

Change-Id: I495f2f823b0d53a0d858876bde41fde5f0705113
BUG: 1397177
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/15895
Smoke: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-by: Poornima G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

libglusterfs:Now mempool is added to ctx pool list under a lock

2016-11-22T12:45:34+00:00

mempool is added to ctx pool list without any lock. This can cause undefined
behaviour in case of multithreaded environment.

Fix: modify the list only under ctx->lock

Change-Id: I7bdbb3db48a899bb0e41427e149b13c0facaedba
Signed-off-by: Rajesh Joseph 
BUG: 1394719
Reviewed-on: http://review.gluster.org/15842
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Poornima G 
Reviewed-by: Niels de Vos

gfapi: async fops should unref in callbacks

2016-11-05T20:32:58+00:00

If fd is unref'd at the end of async call then the unref in cbks would
lead to double unref and possible crash. Removing duplicate unrefs.

Added unref only in failure cases.

A simple test case has been added to test async write case. Need to
extend the same for other async APIs too.

Details:
All glfd based calls in libgfapi, except for glfs_open and glfs_close,
behave in the same way. At the start of the operation, they take a ref
on glfd and fd. At the end of the operation, they unref it. Async calls
are a little different as they unref in the cbk function. A successfull
open call does not unref either the glfd or fd, thereby functioning as a
reference for a OPEN file object. glfs_close makes a syncop_flush call
sandwiched between a fd ref and unref(this can be removed, more on this
below), followed by a call to glfs_mark_glfd_for_deletion which unrefs
glfd and also calls glfs_fd_destroy as a release function thereby doing
a unref on fd too.

Functionally, there is no problem with how everything works when as
described above. However, it is a little non-intuitive that we need to
perform a fd_unref as a consequence of a implicit fd_ref that happens
within glfs_resolve_fd. As we perform a GF_REF_GET(glfd) at the start of
every operation, it would be worthwhile to remove the fd_ref that
glfs_resovle_fd takes and do away with explicit fd_unref()s at the end
of every operation. This is the same reason why we don't need the fd_ref
in glfs_close. This is however not in the scope of this patch.

Change-Id: I86b1d3b2ad846b16ea527d541dc82b5e90b0ba85
BUG: 1391086
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/15768
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Xavier Hernandez 
Reviewed-by: soumya k 
Reviewed-by: Prasanna Kumar Kalever 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan