glusterfs.git/glusterfsd/src/glusterfsd.c, branch v3.7.9

glusterd: fixing few memory leak in glusterd

2016-03-10T15:20:04+00:00

This patch is backport of: http://review.gluster.org/#/c/12927/

Current glusterd code base having memory leak. This is because of
memory allocate by dict_allocate_and_serialize function in
"gd_syncop_mgmt_v3_lock" and "gd_syncop_mgmt_v3_unlock"
function is not freeing up memory upon exit.

Fix is to free the memory after exit of the above function.

Thanks Carlos and Roman for finding out the issue and fix.

 >> Change-Id: Id67aa794c84969830ca7ea8c2374f80c64d7a639
 >> BUG: 1287517
 >> Signed-off-by: Gaurav Kumar Garg 
 >> Signed-off-by: Carlos Chinea 
 >> Signed-off-by: Roman Tereshonkov 
 >> Reviewed-on: http://review.gluster.org/12927
 >> Smoke: Gluster Build System 
 >> NetBSD-regression: NetBSD Build System 
 >> CentOS-regression: Gluster Build System 
 >> Reviewed-by: Jeff Darcy 

Change-Id: Id67aa794c84969830ca7ea8c2374f80c64d7a639
BUG: 1311377
Signed-off-by: Gaurav Kumar Garg 
(cherry picked from commit e38bf1bdeda3c7a89be3193ad62a72b9139358dd)
Reviewed-on: http://review.gluster.org/13503
Smoke: Gluster Build System 
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

fuse: Add a new mount option capability

2016-03-10T03:09:43+00:00

Originally all security.* xattrs were forbidden if selinux is disabled,
which was causing Samba's acl_xattr module to not work, as it would
store the NTACL in security.NTACL. To fix this http://review.gluster.org/#/c/12826/
was sent, which forbid only security.selinux. This opened up a getxattr
call on security.capability before every write fop and others.

Capabilities can be used without selinux, hence if selinux is disabled,
security.capability cannot be forbidden. Hence adding a new mount
option called capability.

Only when "--capability" or "--selinux" mount option is used,
security.capability is sent to the brick, else it is forbidden.

Backport of : http://review.gluster.org/#/c/13540/ &
              http://review.gluster.org/#/c/13653/

BUG: 1309462
Change-Id: Ib8d4f32d9f1458f4d71a05785f92b526aa7033ff
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13626
Tested-by: Vijay Bellur 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur

glusterfsd: To support volfile-server-transport type "unix"

2015-12-03T11:39:58+00:00

glusterfsd fails if the glusterd is bind to specific-IP address.
This patch helps glusterfsd to get the volfile using Unix domain socket.
glusterfs -s  --volfile-server-transport unix
          --volfile-id  
The patch checks if the volfile-server-transport is of type "unix",
If It is then uses rpc_transport_unix_options_build to get the volfile.

Signed-off-by: Mohamed Ashiq 
Signed-off-by: Humble Devassy Chirammal 

>Change-Id: I81b881e7ac5a3a4f2ac83c789c385cf547f0d53e
>BUG: 1279484
>Signed-off-by: Mohamed Ashiq 
>Signed-off-by: Humble Devassy Chirammal 
>Reviewed-on: http://review.gluster.org/12556
>Reviewed-by: Atin Mukherjee 
>Reviewed-by: Niels de Vos 
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 

Change-Id: Ic3eeacc78aca3df3c07e5a9228043f3d7cdf094c
BUG: 1285961
Reviewed-on: http://review.gluster.org/12769
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Atin Mukherjee

fuse: add "resolve-gids" mount option to overcome 32-groups limit

2015-09-28T09:51:08+00:00

Add a --resolve-gids commandline option to the glusterfs binary. This
option gets set when executing "mount -t glusterfs -o resolve-gids ...".

This option is most useful in combination with the "acl" mount option.
POSIX ACL permission checking is done on the FUSE-client side to improve
performance (in addition to the checking on the bricks).

The fuse-bridge reads /proc/$PID/status by default, and this file
contains maximum 32 groups. Any local (client-side) permission checking
that requires more than the first 32 groups will fail.

By enabling the "resolve-gids" option, the fuse-bridge will call
getgrouplist() to retrieve all the groups from the user accessing the
mountpoint. This is comparable to how "nfs.server-aux-gids" works.

Note that when a user belongs to more than ~93 groups, the volume option
server.manage-gids needs to be enabled too. Without this option, the
RPC-layer will need to reduce the number of groups to make them fit in
the RPC-header.

Cherry picked from commit 64a5bf3749c67fcc00773a2716d0c7b61b0b4417:
> Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
> BUG: 1246275
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/11732
> Tested-by: NetBSD Build System 
> Reviewed-by: Ravishankar N 
> Tested-by: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
BUG: 1246397
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11875
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: Enabling the fini() in cleanup_and_exit()

2015-06-17T04:15:11+00:00

Problem 1 : glusterd was crashing due to race between clean up thread and rpc event thread.

Scenario:
As we can observed, X thread is in the process of exiting the process. It has already
run the exit handlers, which cleanup things that require cleaning up. This includes
liburcu resources. By the time Y thread  called rcu_bp_register(), the liburcu resources
have been cleaned up. rcu_bp_register() tries to access these non-existent resources,
which leads to the segmentation fault.

Note1:
Crash happen when the process is almost at the point of stopping(exiting), it doesn't have any
serious impact to functionality apart from creating the core dump file and the log message.

Fix .Do proper clean up before calling exit().

Note2: Other xlator have clean up issues,so only glusterd clean up function invoked.

Note3: This patch also solve the selinux issue.

Problem 2 : glusterd runs as rpm_script_t when it's executed from the rpm scriptlet,files created
in this context are set as rpm_script_t, so glusterd unable to access these files when it runs
in glusterd_t context.

Fix: Fini clean up the files while glusterd exiting, so files are recreated by glusterd while
starting with proper SElinux context label.

Backport of :
>Change-Id: Idcfd087f51c18a729bdf44a146f9d294e2fca5e2
>BUG: 1209461
>Signed-off-by: anand 
>Reviewed-on: http://review.gluster.org/10894
>Tested-by: Gluster Build System 
>Reviewed-by: Atin Mukherjee 
>Tested-by: NetBSD Build System 
>Reviewed-by: Raghavendra Bhat 
>Reviewed-by: Vijay Bellur 

Change-Id: I59579e675bd73d7a19f7b965bd2c3c0fcd95d241
BUG: 1230026
Signed-off-by: anand 
Reviewed-on: http://review.gluster.org/11155
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Atin Mukherjee

core: Global timer-wheel

2015-05-10T12:27:40+00:00

Instantiate a process wide global instance of the timer wheel
data structure. Spawning glusterfs* process with option arg
"--global-timer-wheel" instantiates a global instance of
timer-wheel under global context (->ctx).

Translators can make use of this process wide instance [via a
call to glusterfs_global_timer_wheel()] instead of maintaining
an instance of their own and possibly consuming more memory.
Linux kernel too has a single instance of timer wheel where
subsystems such as IO, networking, etc.. make use of.

Bitrot daemon would be early consumers of this: bitrot translator
instances for multiple volumes would track objects belonging to
their respective bricks in this global expiry tracking data
structure. This is also a first step to move GlusterFS timer
mechanism to use timer-wheel.

> Change-Id: Ie882df607e07acaced846ea269ebf1ece306d6ae
> BUG: 1170075
> Signed-off-by: Venky Shankar 
> Reviewed-on: http://review.gluster.org/10380
> Tested-by: NetBSD Build System
> Reviewed-by: Vijay Bellur 
> Tested-by: Gluster Build System 

Change-Id: I35c840daa9996a059699f8ea5af54c76ede7e09c
Signed-off-by: Venky Shankar 
Signed-off-by: Gaurav Kumar Garg 
BUG: 1220041
Reviewed-on: http://review.gluster.org/10716
Tested-by: Gluster Build System

protocol: increase default group-cache-timeout to 300 seconds

2015-05-06T17:34:22+00:00

sssd uses 300 seconds by default too. There is no need to overload sssd
with requests that it would have cached.

Cherry picked from commit 34833364e9839f0036bccd58ec0a8a963e69263e:
> BUG: 1215187
> Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/10371
> Tested-by: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Vijay Bellur 

Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34
BUG: 1215189
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10523
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

glusterfsd: add "print-netgroups" and "print-exports" command

2015-03-19T05:16:34+00:00

NFS now has the ability to use a separate file for "netgroups" and
"exports". An administrator should have the ability to check the
validity of the files before applying the configuration.

The "glusterfsd" command now has the following additional arguments that
can be used to check the configuration:

   --print-netgroups: Validate the netgroups file and print it out
   --print-exports: Validate the exports file and print it out

BUG: 1143880
Change-Id: I24c40d50110d49d8290f9fd916742f7e4d0df85f
URL: http://www.gluster.org/community/documentation/index.php/Features/Exports_Netgroups_Authentication
Original-author: Shreyas Siravara 
CC: Richard Wareing 
CC: Jiffin Tony Thottan 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/9365
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

protocol/client: sequence CHILD_UP, CHILD_DOWN etc notifications

2015-02-07T21:25:26+00:00

... from all bricks in the volume

This patch is important in the context of MT epoll. With MT epoll,
notification events from client xlators could reach cluster xlators like
afr, dht, ec, stripe etc. in different orders.

For e.g, In a distributed replicate volume of 2 bricks, namely Brick1
and Brick2, the following network events are observed by a mount
process.

- connection to Brick1 is broken.
- connection to Brick1 has been restored.

- connection to Brick2 is broken.
- connection to Brick2 has been restored.

Without establishing a total ordering of events, we can't guarantee that
cluster xlators like afr, dht perceive them in the same order.  While we
would expect afr (say) to perceive it as only one of Brick1 and Brick2
going down at any given time, it is possible for the notification of
Brick2 going offline to race with the notification of Brick1 coming back
online.

Change-Id: I78f5a52bfb05593335d0e9ad53ebfff98995593d
BUG: 1104462
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/9591
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

epoll: Adding the ability to configure epoll threads

2015-02-07T21:23:03+00:00

Add the ability to configure the number of event threads
for various gluster services.

Currently with the multi thread epoll patch, it is possible
to have more than one thread waiting on socket activity and
processing the same. This thread count is currently static,
which this commit makes dynamic.

The current services which use IO path, i.e brick processes,
any client process (nfs, FUSE, gfapi, heal,
rebalance, etc.a), gain 2 set parameters to control the number
of threads that are processing events. These settings are,
  - client.event-threads 
  - server.event-threads 

The client setting affects the client graph consumers, and the
server setting affects the brick processes. These are processed
and inited/reconfigured using the client/server protocol xlators.

Other services (say glusterd) would need to extend similar
configuration settings to take advantage of multi threaded event
processing.

At present glusterd is not enabled with this commit, as it does not
stand to gain from this multi-threading (as I understand it).

Change-Id: Id8422fc57a9f95a135158eb6477ccf9d3c9ea4d9
BUG: 1104462
Signed-off-by: Shyam 
Reviewed-on: http://review.gluster.org/9488
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur