glusterfs.git/libglusterfs/src/common-utils.c, branch v3.10.11

libglusterfs : Fix crash in glusterd while peer probing

2017-06-20T04:56:08+00:00

glusterd crashes when port is being set explcitly to a
range which is outside greater than short data type range.
Eg. sysctl net.ipv4.ip_local_reserved_ports="49152-49156"
In above case glusterd crashes while parsing the port.

With this fix glusterd will be able to handle port range
between INT_MIN to INT_MAX

> Reviewed-on: https://review.gluster.org/17359
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
>  Reviewed-by: Samikshan Bairagya 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Niels de Vos 
> Reviewed-by: Jeff Darcy 

Change-Id: I7c75ee67937b0e3384502973d96b1c36c89e0fe1
BUG: 1459760
Signed-off-by: Gaurav Yadav 
Reviewed-on: https://review.gluster.org/17494
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Samikshan Bairagya 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

glusterd: socketfile & pidfile related fixes for brick multiplexing feature

2017-05-10T10:42:05+00:00

Problem: While brick-muliplexing is on after restarting glusterd, CLI is
         not showing pid of all brick processes in all volumes.

Solution: While brick-mux is on all local brick process communicated through one
          UNIX socket but as per current code (glusterd_brick_start) it is trying
          to communicate with separate UNIX socket for each volume which is populated
          based on brick-name and vol-name.Because of multiplexing design only one
          UNIX socket is opened so it is throwing poller error and not able to
          fetch correct status of brick process through cli process.
          To resolve the problem write a new function glusterd_set_socket_filepath_for_mux
          that will call by glusterd_brick_start to validate about the existence of socketpath.
          To avoid the continuous EPOLLERR erros in  logs update socket_connect code.

Test:     To reproduce the issue followed below steps
          1) Create two distributed volumes(dist1 and dist2)
          2) Set cluster.brick-multiplex is on
          3) kill glusterd
          4) run command gluster v status
          After apply the patch it shows correct pid for all volumes

> BUG: 1444596
> Change-Id: I5d10af69dea0d0ca19511f43870f34295a54a4d2
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: https://review.gluster.org/17101
> Smoke: Gluster Build System 
> Reviewed-by: Prashanth Pai 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> (cherry picked from commit 21c7f7baccfaf644805e63682e5a7d2a9864a1e6)

Change-Id: I1892c80b9ffa93974f20c92d421660bcf93c4cda
BUG: 1449002
Signed-off-by: Mohit Agrawal 
Reviewed-on: https://review.gluster.org/17210
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Prashanth Pai

glusterd: add a cli command to trigger a statedump on a client

2017-02-14T01:53:10+00:00

With this, we will be able to trigger statedumps on remote Gluster
clients, mainly targetted for applications using libgfapi.

Design:
SIGUSR signal is the most comman way of taking a statedump in Gluster.
But it cannot be used for libgfapi based processes, as the process
loading the library might have already consumed SIGUSR signal. Hence
going by the command way.

One has to issue a Gluster command to initiate a statedump on the
libgfapi based client. The command takes hostname and PID as an
argument. All the glusterds in the cluster, check if they are connected
to the specified hostname, and send an RPC request to all the connected
clients from that hostname (via the mgmt connection).

> URL: http://review.gluster.org/16357
> BUG: 1169302
> Signed-off-by: Poornima G 
> [ndevos: minor fixes and split patch in smaller pieces]
> Reviewed-on-master: https://review.gluster.org/9228
> Reviewed-by: Niels de Vos 
> Tested-by: Niels de Vos 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Samikshan Bairagya 

BUG: 1418981
Change-Id: Icbe4d2f026b32a2c7d5535e1bfb2cdaaff042e91
Signed-off-by: Shyam 
Reviewed-on: https://review.gluster.org/16601
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

libglusterfs+transport+io-threads: fix 256KB stack abuse

2017-02-02T17:32:49+00:00

Some functions were allocating 64K booleans, which are (crazily)
mapped to 4-byte ints, for a total of 256KB per call.  Changed to use
bitfields instead, so usage is now only 8KB per call.  This was the
impediment to changing the io-threads stack size, so that has been
adjusted too.

Backport of:
> Change-Id: I8781c4f2c8f2b830f4535e366995fac8dd0a8653
> BUG: 1418095
> Reviewed-on: https://review.gluster.org/15745

Change-Id: Ia5dada61703e6bea95f2511da71feb573fc9a429
BUG: 1418536
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16511
Reviewed-by: N Balachandran 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur 
CentOS-regression: Gluster Build System

core: run many bricks within one glusterfsd process

2017-02-02T00:54:58+00:00

This patch adds support for multiple brick translator stacks running in
a single brick server process.  This reduces our per-brick memory usage
by approximately 3x, and our appetite for TCP ports even more.  It also
creates potential to avoid process/thread thrashing, and to improve QoS
by scheduling more carefully across the bricks, but realizing that
potential will require further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global
option.  By default it's off, and bricks are started in separate
processes as before.  If multiplexing is enabled, then *compatible*
bricks (mostly those with the same transport options) will be started in
the same process.

Backport of:
> Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/14763

Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8
BUG: 1418091
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16496
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

system/posix-acl: Log reason for EACCES

2016-11-17T09:56:14+00:00

It is becoming increasingly difficult to debug the reason why posix-acl decides
to fail a fop with EACCES. This patch prints a big log everytime such
a condition occurs giving out the details that may help in finding why the fop
is errored out.

Change-Id: I2505baaafb5d77ef6c187554ff027df9b20468db
BUG: 1394548
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15837
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

afr: Ignore gluster internal (virtual) xattrs in metadata heal check

2016-09-26T13:25:40+00:00

Problem:
In arbiter configuration, posix-xlator in the arbiter brick always sets
the GF_CONTENT_KEY in the response dict with a value 0. If the file size on
the data bricks is more than quick-read's max-file-size (64kb default),
those bricks don't set the key. Because of this difference in the no. of dict
elements, afr triggers metadata heal in lookup code path, in turn
leading to extra lookups+inodelks.

Fix:
Changed afr dict comparison logic to ignore all virtual xattrs and the
on-disk ones that we should not be healing.

Also removed is_virtual_xattr() function. The original callers to this
function (upcall) don't seem to need it anymore.

Change-Id: I05730bdd39d8fb0b9a49a5fc9c0bb01f0d3bb308
BUG: 1378684
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15548
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

glusterd/cli: cli to get local state representation from glusterd

2016-08-26T15:23:37+00:00

Currently there is no existing CLI that can be used to get the
local state representation of the cluster as maintained in glusterd
in a readable as well as parseable format.

The CLI added has the following usage:

 # gluster get-state [daemon] [odir ] [file ]

This would dump data points that reflect the local state
representation of the cluster as maintained in glusterd (no other
daemons are supported as of now) to a file inside the specified
output directory. The default output directory and filename is
/var/run/gluster and glusterd_state_ respectively. The
option for specifying the daemon name leaves room to add support for
other daemons in the future. Following are the data points captured
as of now to represent the state from the local glusterd pov:

 * Peer:
    - Primary hostname
    - uuid
    - state
    - connection status
    - List of hostnames

 * Volumes:
    - name, id, transport type, status
    - counts: bricks, snap, subvol, stripe, arbiter, disperse,
 redundancy
    - snapd status
    - quorum status
    - tiering related information
    - rebalance status
    - replace bricks status
    - snapshots

 * Bricks:
    - Path, hostname (for all bricks these info will be shown)
    - port, rdma port, status, mount options, filesystem type and
signed in status for bricks running locally.

 * Services:
    - name, online status for initialised services

 * Others:
    - Base port, last allocated port
    - op-version
    - MYUUID

Change-Id: I4a45cc5407ab92d8afdbbd2098ece851f7e3d618
BUG: 1353156
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/14873
Reviewed-by: Avra Sengupta 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

quotad: fix potential buffer overflows

2016-08-25T12:18:09+00:00

This converts sprintf to gf_asprintf in following components:                                                                                                          * quotad.c
* dht
* afr
* protocol/client
* rpc/rpc-lib
* rpc/rpc-transport

Change-Id: If8a267bab3d91003bdef3a92664077a0136745ee
BUG: 1332073
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/14102
Tested-by: Manikandan Selvaganesh 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Manikandan Selvaganesh

cluster/afr: Prevent split-brain when bricks are brought off and on in cyclic order

2016-08-22T09:38:36+00:00

When the bricks are brought offline and then online in cyclic
order while writes are in progress on a file, thanks to inode
refresh in write txns, AFR will mostly fail the write attempt
when the only good copy is offline. However, there is still a
remote possibility that the file will run into split-brain if
the brick that has the lone good copy goes offline *after* the
inode refresh but *before* the write txn completes (I call it
in-flight split-brain in the patch for ease of reference),
requiring intervention from admin to resolve the split-brain
before the IO can resume normally on the file. To get around this,
the patch does the following things:
i) retains the dirty xattrs on the file
ii) avoids marking the last of the good copies as bad (or accused)
    in case it is the one to go down during the course of a write.
iii) fails that particular write with the appropriate errno.

This way, we still have one good copy left despite the split-brain situation
which when it is back online, will be chosen as source to do the heal.

Change-Id: I9ca634b026ac830b172bac076437cc3bf1ae7d8a
BUG: 1363721
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15080
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Oleksandr Natalenko 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri