glusterfs.git/glusterfsd/src/glusterfsd.c, branch v3.7.14

glusterd: add defence mechanism to avoid brick port clashes

2016-05-05T03:23:19+00:00

Intro:
Currently glusterd maintain the portmap registry which contains ports that
are free to use between 49152 - 65535, this registry is initialized
once, and updated accordingly as an then when glusterd sees they are been
used.

Glusterd first checks for a port within the portmap registry and gets a FREE
port marked in it, then checks if that port is currently free using a connect()
function then passes it to brick process which have to bind on it.

Problem:
We see that there is a time gap between glusterd checking the port with
connect() and brick process actually binding on it. In this time gap it could
be so possible that any process would have occupied this port because of which
brick will fail to bind and exit.

Case 1:
To avoid the gluster client process occupying the port supplied by glusterd :

we have separated the client port map range with brick port map range more @
http://review.gluster.org/#/c/13998/

Case 2: (Handled by this patch)
To avoid the other foreign process occupying the port supplied by glusterd :

To handle above situation this patch implements a mechanism to return EADDRINUSE
error code to glusterd, upon which a new port is allocated and try to restart
the brick process with the newly allocated port.

Note: Incase of glusterd restarts i.e. runner_run_nowait() there is no way to
handle Case 2, becuase runner_run_nowait() will not wait to get the return/exit
code of the executed command (brick process). Hence as of now in such case,
we cannot know with what error the brick has failed to connect.

This patch also fix the runner_end() to perform some cleanup w.r.t
return values.

Backport of:
> Change-Id: Iec52e7f5d87ce938d173f8ef16aa77fd573f2c5e
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14043
> Tested-by: Prasanna Kumar Kalever 
> Reviewed-by: Atin Mukherjee 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Signed-off-by: Prasanna Kumar Kalever 

Change-Id: Ief247b4d4538c1ca03e73aa31beb5fa99853afd6
BUG: 1323564
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14208
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterfsd: fix to return actual exit status on mount process

2016-05-02T11:30:10+00:00

Problem:
Currently, we always exit mount process with the pid as the exit number
which is return value of the waitpid(), it is not the exit value of the
child process

Solution:
Extract the actual exit code/status in case if the child terminated normally,
that is, by calling exit(3) or _exit(2), or by returning from main()

Backport of:
> Change-Id: Iefec6e27b5a5a98a22f016e49967978853662e37
> BUG: 1331042
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14094
> Tested-by: Prasanna Kumar Kalever 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Jeff Darcy 

Change-Id: Ib9fd878461b0b409c9d55d526cf60d7276c31775
BUG: 1331938
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14129
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra G

glusterd: fixing few memory leak in glusterd

2016-03-10T15:20:04+00:00

This patch is backport of: http://review.gluster.org/#/c/12927/

Current glusterd code base having memory leak. This is because of
memory allocate by dict_allocate_and_serialize function in
"gd_syncop_mgmt_v3_lock" and "gd_syncop_mgmt_v3_unlock"
function is not freeing up memory upon exit.

Fix is to free the memory after exit of the above function.

Thanks Carlos and Roman for finding out the issue and fix.

 >> Change-Id: Id67aa794c84969830ca7ea8c2374f80c64d7a639
 >> BUG: 1287517
 >> Signed-off-by: Gaurav Kumar Garg 
 >> Signed-off-by: Carlos Chinea 
 >> Signed-off-by: Roman Tereshonkov 
 >> Reviewed-on: http://review.gluster.org/12927
 >> Smoke: Gluster Build System 
 >> NetBSD-regression: NetBSD Build System 
 >> CentOS-regression: Gluster Build System 
 >> Reviewed-by: Jeff Darcy 

Change-Id: Id67aa794c84969830ca7ea8c2374f80c64d7a639
BUG: 1311377
Signed-off-by: Gaurav Kumar Garg 
(cherry picked from commit e38bf1bdeda3c7a89be3193ad62a72b9139358dd)
Reviewed-on: http://review.gluster.org/13503
Smoke: Gluster Build System 
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

fuse: Add a new mount option capability

2016-03-10T03:09:43+00:00

Originally all security.* xattrs were forbidden if selinux is disabled,
which was causing Samba's acl_xattr module to not work, as it would
store the NTACL in security.NTACL. To fix this http://review.gluster.org/#/c/12826/
was sent, which forbid only security.selinux. This opened up a getxattr
call on security.capability before every write fop and others.

Capabilities can be used without selinux, hence if selinux is disabled,
security.capability cannot be forbidden. Hence adding a new mount
option called capability.

Only when "--capability" or "--selinux" mount option is used,
security.capability is sent to the brick, else it is forbidden.

Backport of : http://review.gluster.org/#/c/13540/ &
              http://review.gluster.org/#/c/13653/

BUG: 1309462
Change-Id: Ib8d4f32d9f1458f4d71a05785f92b526aa7033ff
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13626
Tested-by: Vijay Bellur 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur

glusterfsd: To support volfile-server-transport type "unix"

2015-12-03T11:39:58+00:00

glusterfsd fails if the glusterd is bind to specific-IP address.
This patch helps glusterfsd to get the volfile using Unix domain socket.
glusterfs -s  --volfile-server-transport unix
          --volfile-id  
The patch checks if the volfile-server-transport is of type "unix",
If It is then uses rpc_transport_unix_options_build to get the volfile.

Signed-off-by: Mohamed Ashiq 
Signed-off-by: Humble Devassy Chirammal 

>Change-Id: I81b881e7ac5a3a4f2ac83c789c385cf547f0d53e
>BUG: 1279484
>Signed-off-by: Mohamed Ashiq 
>Signed-off-by: Humble Devassy Chirammal 
>Reviewed-on: http://review.gluster.org/12556
>Reviewed-by: Atin Mukherjee 
>Reviewed-by: Niels de Vos 
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 

Change-Id: Ic3eeacc78aca3df3c07e5a9228043f3d7cdf094c
BUG: 1285961
Reviewed-on: http://review.gluster.org/12769
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Atin Mukherjee

fuse: add "resolve-gids" mount option to overcome 32-groups limit

2015-09-28T09:51:08+00:00

Add a --resolve-gids commandline option to the glusterfs binary. This
option gets set when executing "mount -t glusterfs -o resolve-gids ...".

This option is most useful in combination with the "acl" mount option.
POSIX ACL permission checking is done on the FUSE-client side to improve
performance (in addition to the checking on the bricks).

The fuse-bridge reads /proc/$PID/status by default, and this file
contains maximum 32 groups. Any local (client-side) permission checking
that requires more than the first 32 groups will fail.

By enabling the "resolve-gids" option, the fuse-bridge will call
getgrouplist() to retrieve all the groups from the user accessing the
mountpoint. This is comparable to how "nfs.server-aux-gids" works.

Note that when a user belongs to more than ~93 groups, the volume option
server.manage-gids needs to be enabled too. Without this option, the
RPC-layer will need to reduce the number of groups to make them fit in
the RPC-header.

Cherry picked from commit 64a5bf3749c67fcc00773a2716d0c7b61b0b4417:
> Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
> BUG: 1246275
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/11732
> Tested-by: NetBSD Build System 
> Reviewed-by: Ravishankar N 
> Tested-by: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
BUG: 1246397
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11875
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: Enabling the fini() in cleanup_and_exit()

2015-06-17T04:15:11+00:00

Problem 1 : glusterd was crashing due to race between clean up thread and rpc event thread.

Scenario:
As we can observed, X thread is in the process of exiting the process. It has already
run the exit handlers, which cleanup things that require cleaning up. This includes
liburcu resources. By the time Y thread  called rcu_bp_register(), the liburcu resources
have been cleaned up. rcu_bp_register() tries to access these non-existent resources,
which leads to the segmentation fault.

Note1:
Crash happen when the process is almost at the point of stopping(exiting), it doesn't have any
serious impact to functionality apart from creating the core dump file and the log message.

Fix .Do proper clean up before calling exit().

Note2: Other xlator have clean up issues,so only glusterd clean up function invoked.

Note3: This patch also solve the selinux issue.

Problem 2 : glusterd runs as rpm_script_t when it's executed from the rpm scriptlet,files created
in this context are set as rpm_script_t, so glusterd unable to access these files when it runs
in glusterd_t context.

Fix: Fini clean up the files while glusterd exiting, so files are recreated by glusterd while
starting with proper SElinux context label.

Backport of :
>Change-Id: Idcfd087f51c18a729bdf44a146f9d294e2fca5e2
>BUG: 1209461
>Signed-off-by: anand 
>Reviewed-on: http://review.gluster.org/10894
>Tested-by: Gluster Build System 
>Reviewed-by: Atin Mukherjee 
>Tested-by: NetBSD Build System 
>Reviewed-by: Raghavendra Bhat 
>Reviewed-by: Vijay Bellur 

Change-Id: I59579e675bd73d7a19f7b965bd2c3c0fcd95d241
BUG: 1230026
Signed-off-by: anand 
Reviewed-on: http://review.gluster.org/11155
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Atin Mukherjee

core: Global timer-wheel

2015-05-10T12:27:40+00:00

Instantiate a process wide global instance of the timer wheel
data structure. Spawning glusterfs* process with option arg
"--global-timer-wheel" instantiates a global instance of
timer-wheel under global context (->ctx).

Translators can make use of this process wide instance [via a
call to glusterfs_global_timer_wheel()] instead of maintaining
an instance of their own and possibly consuming more memory.
Linux kernel too has a single instance of timer wheel where
subsystems such as IO, networking, etc.. make use of.

Bitrot daemon would be early consumers of this: bitrot translator
instances for multiple volumes would track objects belonging to
their respective bricks in this global expiry tracking data
structure. This is also a first step to move GlusterFS timer
mechanism to use timer-wheel.

> Change-Id: Ie882df607e07acaced846ea269ebf1ece306d6ae
> BUG: 1170075
> Signed-off-by: Venky Shankar 
> Reviewed-on: http://review.gluster.org/10380
> Tested-by: NetBSD Build System
> Reviewed-by: Vijay Bellur 
> Tested-by: Gluster Build System 

Change-Id: I35c840daa9996a059699f8ea5af54c76ede7e09c
Signed-off-by: Venky Shankar 
Signed-off-by: Gaurav Kumar Garg 
BUG: 1220041
Reviewed-on: http://review.gluster.org/10716
Tested-by: Gluster Build System

protocol: increase default group-cache-timeout to 300 seconds

2015-05-06T17:34:22+00:00

sssd uses 300 seconds by default too. There is no need to overload sssd
with requests that it would have cached.

Cherry picked from commit 34833364e9839f0036bccd58ec0a8a963e69263e:
> BUG: 1215187
> Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/10371
> Tested-by: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Vijay Bellur 

Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34
BUG: 1215189
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10523
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

glusterfsd: add "print-netgroups" and "print-exports" command

2015-03-19T05:16:34+00:00

NFS now has the ability to use a separate file for "netgroups" and
"exports". An administrator should have the ability to check the
validity of the files before applying the configuration.

The "glusterfsd" command now has the following additional arguments that
can be used to check the configuration:

   --print-netgroups: Validate the netgroups file and print it out
   --print-exports: Validate the exports file and print it out

BUG: 1143880
Change-Id: I24c40d50110d49d8290f9fd916742f7e4d0df85f
URL: http://www.gluster.org/community/documentation/index.php/Features/Exports_Netgroups_Authentication
Original-author: Shreyas Siravara 
CC: Richard Wareing 
CC: Jiffin Tony Thottan 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/9365
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur