glusterfs.git/glusterfsd, branch v3.7.15

glusterd: add defence mechanism to avoid brick port clashes

2016-05-05T03:23:19+00:00

Intro:
Currently glusterd maintain the portmap registry which contains ports that
are free to use between 49152 - 65535, this registry is initialized
once, and updated accordingly as an then when glusterd sees they are been
used.

Glusterd first checks for a port within the portmap registry and gets a FREE
port marked in it, then checks if that port is currently free using a connect()
function then passes it to brick process which have to bind on it.

Problem:
We see that there is a time gap between glusterd checking the port with
connect() and brick process actually binding on it. In this time gap it could
be so possible that any process would have occupied this port because of which
brick will fail to bind and exit.

Case 1:
To avoid the gluster client process occupying the port supplied by glusterd :

we have separated the client port map range with brick port map range more @
http://review.gluster.org/#/c/13998/

Case 2: (Handled by this patch)
To avoid the other foreign process occupying the port supplied by glusterd :

To handle above situation this patch implements a mechanism to return EADDRINUSE
error code to glusterd, upon which a new port is allocated and try to restart
the brick process with the newly allocated port.

Note: Incase of glusterd restarts i.e. runner_run_nowait() there is no way to
handle Case 2, becuase runner_run_nowait() will not wait to get the return/exit
code of the executed command (brick process). Hence as of now in such case,
we cannot know with what error the brick has failed to connect.

This patch also fix the runner_end() to perform some cleanup w.r.t
return values.

Backport of:
> Change-Id: Iec52e7f5d87ce938d173f8ef16aa77fd573f2c5e
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14043
> Tested-by: Prasanna Kumar Kalever 
> Reviewed-by: Atin Mukherjee 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Signed-off-by: Prasanna Kumar Kalever 

Change-Id: Ief247b4d4538c1ca03e73aa31beb5fa99853afd6
BUG: 1323564
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14208
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterfsd: fix to return actual exit status on mount process

2016-05-02T11:30:10+00:00

Problem:
Currently, we always exit mount process with the pid as the exit number
which is return value of the waitpid(), it is not the exit value of the
child process

Solution:
Extract the actual exit code/status in case if the child terminated normally,
that is, by calling exit(3) or _exit(2), or by returning from main()

Backport of:
> Change-Id: Iefec6e27b5a5a98a22f016e49967978853662e37
> BUG: 1331042
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14094
> Tested-by: Prasanna Kumar Kalever 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Jeff Darcy 

Change-Id: Ib9fd878461b0b409c9d55d526cf60d7276c31775
BUG: 1331938
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14129
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra G

Revert "glusterd: Bug fixes for IPv6 support"

2016-04-16T02:18:23+00:00

This reverts commit b33f3c95ec9c8112e6677e09cea05c4c462040d0.

This commit exposes some issues with management encryption that prevents
GlusterFS from operating properly. This will be added again once
problems with management encryption are fixed.

glusterd: Bug fixes for IPv6 support

2016-03-21T17:52:50+00:00

Backport of http://review.gluster.org/#/c/11988/

Problem:
Glusterd not working using ipv6 transport. The idea is with proper glusterd.vol configuration,
1. glusterd needs to listen on default port (240007) as IPv6 TCP listner.
2. Volume creation/deletion/mounting/add-bricks/delete-bricks/peer-probe
needs to work using ipv6 addresses.
3. Bricks needs to listen on ipv6 addresses.
All the above functionality is needed to say that glusterd supports ipv6 transport and this is broken.

Fix:
When "option transport.address-family inet6" option is present in glusterd.vol
file, it is made sure that glusterd creates listeners using ipv6 sockets only and also the same information is saved
inside brick volume files used by glusterfsd brick process when they are starting.

Tests Run:
Regression tests using ./run-tests.sh
IPv4: Regression tests using ./run-tests.sh for release-3.7 branch verified by comparing with clean repo.
IPv6: (Need to add the above mentioned config and also add an entry for "hostname ::1" in /etc/hosts)
    Started failing at ./tests/basic/glusterd/arbiter-volume-probe.t and ran successfully till here

Change-Id: Idd7513aa2347ce0de2b1f68daeecce1b7a39a7af
BUG: 1310445
Signed-off-by: Nithin D 
Reviewed-on: http://review.gluster.org/13787
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd: fixing few memory leak in glusterd

2016-03-10T15:20:04+00:00

This patch is backport of: http://review.gluster.org/#/c/12927/

Current glusterd code base having memory leak. This is because of
memory allocate by dict_allocate_and_serialize function in
"gd_syncop_mgmt_v3_lock" and "gd_syncop_mgmt_v3_unlock"
function is not freeing up memory upon exit.

Fix is to free the memory after exit of the above function.

Thanks Carlos and Roman for finding out the issue and fix.

 >> Change-Id: Id67aa794c84969830ca7ea8c2374f80c64d7a639
 >> BUG: 1287517
 >> Signed-off-by: Gaurav Kumar Garg 
 >> Signed-off-by: Carlos Chinea 
 >> Signed-off-by: Roman Tereshonkov 
 >> Reviewed-on: http://review.gluster.org/12927
 >> Smoke: Gluster Build System 
 >> NetBSD-regression: NetBSD Build System 
 >> CentOS-regression: Gluster Build System 
 >> Reviewed-by: Jeff Darcy 

Change-Id: Id67aa794c84969830ca7ea8c2374f80c64d7a639
BUG: 1311377
Signed-off-by: Gaurav Kumar Garg 
(cherry picked from commit e38bf1bdeda3c7a89be3193ad62a72b9139358dd)
Reviewed-on: http://review.gluster.org/13503
Smoke: Gluster Build System 
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

fuse: Add a new mount option capability

2016-03-10T03:09:43+00:00

Originally all security.* xattrs were forbidden if selinux is disabled,
which was causing Samba's acl_xattr module to not work, as it would
store the NTACL in security.NTACL. To fix this http://review.gluster.org/#/c/12826/
was sent, which forbid only security.selinux. This opened up a getxattr
call on security.capability before every write fop and others.

Capabilities can be used without selinux, hence if selinux is disabled,
security.capability cannot be forbidden. Hence adding a new mount
option called capability.

Only when "--capability" or "--selinux" mount option is used,
security.capability is sent to the brick, else it is forbidden.

Backport of : http://review.gluster.org/#/c/13540/ &
              http://review.gluster.org/#/c/13653/

BUG: 1309462
Change-Id: Ib8d4f32d9f1458f4d71a05785f92b526aa7033ff
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13626
Tested-by: Vijay Bellur 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur

cli/ afr: op_ret for index heal launch

2016-02-17T09:49:28+00:00

Backport of http://review.gluster.org/#/c/13303/

Problem:
If index heal is launched when some of the bricks are down, glustershd of that
node sends a -1 op_ret to glusterd which eventually propagates it to the CLI.
Also, glusterd sometimes sends an err_str and sometimes not (depending on the
failure happening in the brick-op phase or commit-op phase). So the message that
gets displayed varies in each case:

"Launching heal operation to perform index self heal on volume testvol has been
unsuccessful"
                (OR)
"Commit failed on . Please check log file for details."

Fix:
1. Modify afr_xl_op() to return -1 even if index healing of atleast one brick
fails.
2. Ignore glusterd's error string in gf_cli_heal_volume_cbk and print a more
meaningful message.

The patch also fixes a bug in glusterfs_handle_translator_op() where if we
encounter an error in notify of one xlator, we break out of the loop instead of
sending the notify to other xlators.

Change-Id: I957f6c4b4d0a45453ffd5488e425cab5a3e0acca
BUG: 1306922
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13435
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

glusterfsd: destroy frame after rebalance callback has completed

2016-02-16T02:43:01+00:00

Rebalance after sending a status notification immediately
destroys the frame. Now in its callback the frame is corrupted.
Rebalance crashes when this corrupted frame is accessed. To avoid
this we must destroy the frame after the callback is completed.

> Backport of http://review.gluster.org/#/c/13262/

> Change-Id: If383017a61f09275256e51c44a1efa28feace87b
> BUG: 1300152
> Signed-off-by: Sakshi 

Change-Id: If383017a61f09275256e51c44a1efa28feace87b
BUG: 1302962
Signed-off-by: Sakshi 
Reviewed-on: http://review.gluster.org/13317
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

glusterfsd: To support volfile-server-transport type "unix"

2015-12-03T11:39:58+00:00

glusterfsd fails if the glusterd is bind to specific-IP address.
This patch helps glusterfsd to get the volfile using Unix domain socket.
glusterfs -s  --volfile-server-transport unix
          --volfile-id  
The patch checks if the volfile-server-transport is of type "unix",
If It is then uses rpc_transport_unix_options_build to get the volfile.

Signed-off-by: Mohamed Ashiq 
Signed-off-by: Humble Devassy Chirammal 

>Change-Id: I81b881e7ac5a3a4f2ac83c789c385cf547f0d53e
>BUG: 1279484
>Signed-off-by: Mohamed Ashiq 
>Signed-off-by: Humble Devassy Chirammal 
>Reviewed-on: http://review.gluster.org/12556
>Reviewed-by: Atin Mukherjee 
>Reviewed-by: Niels de Vos 
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 

Change-Id: Ic3eeacc78aca3df3c07e5a9228043f3d7cdf094c
BUG: 1285961
Reviewed-on: http://review.gluster.org/12769
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Atin Mukherjee

glusterfsd: Initialize ctx, cmd_args

2015-11-25T03:14:36+00:00

        Backport of http://review.gluster.org/12311

Change-Id: I9c71ae264665b7bba609c7f86cf42a52a6b47260
BUG: 1269702
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/12312
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur