glusterfs.git/glusterfsd/src, branch v3.8rc1

glusterd: add defence mechanism to avoid brick port clashes

2016-05-10T07:11:33+00:00

Intro:
Currently glusterd maintain the portmap registry which contains ports that
are free to use between 49152 - 65535, this registry is initialized
once, and updated accordingly as an then when glusterd sees they are been
used.

Glusterd first checks for a port within the portmap registry and gets a FREE
port marked in it, then checks if that port is currently free using a connect()
function then passes it to brick process which have to bind on it.

Problem:
We see that there is a time gap between glusterd checking the port with
connect() and brick process actually binding on it. In this time gap it could
be so possible that any process would have occupied this port because of which
brick will fail to bind and exit.

Case 1:
To avoid the gluster client process occupying the port supplied by glusterd :

we have separated the client port map range with brick port map range more @
http://review.gluster.org/#/c/13998/

Case 2: (Handled by this patch)
To avoid the other foreign process occupying the port supplied by glusterd :

To handle above situation this patch implements a mechanism to return EADDRINUSE
error code to glusterd, upon which a new port is allocated and try to restart
the brick process with the newly allocated port.

Note: Incase of glusterd restarts i.e. runner_run_nowait() there is no way to
handle Case 2, becuase runner_run_nowait() will not wait to get the return/exit
code of the executed command (brick process). Hence as of now in such case,
we cannot know with what error the brick has failed to connect.

This patch also fix the runner_end() to perform some cleanup w.r.t
return values.

Backport of:
> Change-Id: Iec52e7f5d87ce938d173f8ef16aa77fd573f2c5e
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14043
> Tested-by: Prasanna Kumar Kalever 
> Reviewed-by: Atin Mukherjee 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Id7d8351a0082b44310177e714edc0571ad0f7195
BUG: 1333711
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14235
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterfsd: fix to return actual exit status on mount process

2016-04-29T13:23:59+00:00

Problem:
Currently, we always exit mount process with the pid as the exit number
which is return value of the waitpid(), it is not the exit value of the
child process

Solution:
Extract the actual exit code/status in case if the child terminated normally,
that is, by calling exit(3) or _exit(2), or by returning from main()

Change-Id: Iefec6e27b5a5a98a22f016e49967978853662e37
BUG: 1331042
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14094
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Jeff Darcy

glusterfsd: coverity fix

2016-04-12T16:11:54+00:00

CID 1124846: string overflow
CID 1124363: checked return value
CID 1210982: unsigned compare

Change-Id: I5995d98c07750615657668535fcc23ac30b3523b
BUG: 789278
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/9608
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

glusterd-client: switch volfile server incase existing connection breaks

2016-04-12T12:14:22+00:00

Problem:
Currently, say we have 10 Node gluster volume, and mounted it using
Node 1 (N1) as volfile server and the rest as backup volfile servers

$ mount -t glusterfs -obackup-volfile-servers=::...: :/vol /mnt

if N1 goes down we still be able to access the same mount point,
but the problem is that if we add or remove bricks to the volume
whoes volfile server is down in our case N1, that info will not be
passed to client, because connection between glusterfs and glusterd (of N1)
will be disconnected due to which we cannot store files to the newly
added bricks until N1 comes back

Solution:
If N1 goes down iterate through the nodes specified in
backup-volfile-servers list and try to establish the connection between
glusterfs and glsuterd, hence we don't really have to wait
until N1 comes back to store files in newly added bricks that are
successfully added when N1 was down

Change-Id: I653c9f081a84667630608091bc243ffc3859d5cd
BUG: 1289916
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/13002
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Poornima G 
Reviewed-by: Jeff Darcy

fuse: Address the review comments in the backport

2016-03-10T03:15:33+00:00

Backport @ http://review.gluster.org/#/c/13626/3

Fix a typo error, consolidate the selinux and capability
check in getxattr and setxattr.

Change-Id: I4303de3d4dd00853169b07577311e03cbb912ed7
BUG: 1316327
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13653
NetBSD-regression: NetBSD Build System 
Tested-by: Vijay Bellur 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur

fuse: Add a new mount option capability

2016-03-08T05:57:28+00:00

Originally all security.* xattrs were forbidden if selinux is disabled,
which was causing Samba's acl_xattr module to not work, as it would
store the NTACL in security.NTACL. To fix this http://review.gluster.org/#/c/12826/
was sent, which forbid only security.selinux. This opened up a getxattr
call on security.capability before every write fop and others.

Capabilities can be used without selinux, hence if selinux is disabled,
security.capability cannot be forbidden. Hence adding a new mount
option called capability.

Only when "--capability" or "--selinux" mount option is used,
security.capability is sent to the brick, else it is forbidden.

Change-Id: I77f60e0fb541deaa416159e45c78dd2ae653105e
BUG: 1309462
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13540
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd: fixing few memory leak in glusterd

2016-02-24T04:19:05+00:00

Current glusterd code base having memory leak. This is because of
memory allocate by dict_allocate_and_serialize function in
"gd_syncop_mgmt_v3_lock" and "gd_syncop_mgmt_v3_unlock"
function is not freeing up meory upon exit.

Fix is to free the memory after exit of the above function.

Thanx Carlos and Roman for finding out the issue and fix.

Change-Id: Id67aa794c84969830ca7ea8c2374f80c64d7a639
BUG: 1287517
Signed-off-by: Gaurav Kumar Garg 
Signed-off-by: Carlos Chinea 
Signed-off-by: Roman Tereshonkov 
Reviewed-on: http://review.gluster.org/12927
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

glusterd: Bug fixes for IPv6 support

2016-02-20T17:16:42+00:00

Problem:
Glusterd not working using ipv6 transport. The idea is with proper glusterd.vol configuration,
1. glusterd needs to listen on default port (240007) as IPv6 TCP listner.
2. Volume creation/deletion/mounting/add-bricks/delete-bricks/peer-probe
   needs to work using ipv6 addresses.
3. Bricks needs to listen on ipv6 addresses.
All the above functionality is needed to say that glusterd supports ipv6 transport and this is broken.

Fix:
When "option transport.address-family inet6" option is present in glusterd.vol
file, it is made sure that glusterd creates listeners using ipv6 sockets only and also the same information is saved
inside brick volume files used by glusterfsd brick process when they are starting.

Tests Run:
Regression tests using ./run-tests.sh
    IPv4: Ran manually till tests/basic/rpm.t .
    IPv6: (Need to add the above mentioned config and also add an entry for "hostname ::1" in /etc/hosts)
        Started failing at ./tests/basic/glusterd/arbiter-volume-probe.t and ran successfully till here

Unit Tests using Ipv6
    peer probe
    add-bricks
    remove-bricks
    create volume
    replace-bricks
    start volume
    stop volume
    delete volume

Change-Id: Iebc96e6cce748b5924ce5da17b0114600ec70a6e
BUG: 1117886
Signed-off-by: Nithin D 
Reviewed-on: http://review.gluster.org/11988
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Jeff Darcy

glusterfsd: destroy frame after rebalance callback has completed

2016-02-16T02:40:22+00:00

Rebalance after sending a status notification immediately
destroys the frame. Now in its callback the frame is corrupted.
Rebalance crashes when this corrupted frame is accessed. To avoid
this we must destroy the frame after the callback is completed.

Change-Id: If383017a61f09275256e51c44a1efa28feace87b
BUG: 1300152
Signed-off-by: Sakshi 
Reviewed-on: http://review.gluster.org/13262
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

experimental: add fdl (Full Data Logging) translator

2016-02-13T13:13:07+00:00

NSR needs logging that is different than our existing changelog in
several ways:

 * Full data, not just metadata

 * Pre-op, not post-op

 * High performance

 * Supports the concept of time-bounded "terms"

Others (for example EC) might need the same thing.  This patch adds such
a translator.  It also adds code to dump the resulting journals, and to replay
them using syncops, plus (very rudimentary) tests for all of the above.

Change-Id: I29680a1b4e0a9e7d5a8497fef302c46434b86636
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/12450
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System