glusterfs.git/xlators/protocol, branch v3.7.12rc1

glusterd: add defence mechanism to avoid brick port clashes

2016-05-05T03:23:19+00:00

Intro:
Currently glusterd maintain the portmap registry which contains ports that
are free to use between 49152 - 65535, this registry is initialized
once, and updated accordingly as an then when glusterd sees they are been
used.

Glusterd first checks for a port within the portmap registry and gets a FREE
port marked in it, then checks if that port is currently free using a connect()
function then passes it to brick process which have to bind on it.

Problem:
We see that there is a time gap between glusterd checking the port with
connect() and brick process actually binding on it. In this time gap it could
be so possible that any process would have occupied this port because of which
brick will fail to bind and exit.

Case 1:
To avoid the gluster client process occupying the port supplied by glusterd :

we have separated the client port map range with brick port map range more @
http://review.gluster.org/#/c/13998/

Case 2: (Handled by this patch)
To avoid the other foreign process occupying the port supplied by glusterd :

To handle above situation this patch implements a mechanism to return EADDRINUSE
error code to glusterd, upon which a new port is allocated and try to restart
the brick process with the newly allocated port.

Note: Incase of glusterd restarts i.e. runner_run_nowait() there is no way to
handle Case 2, becuase runner_run_nowait() will not wait to get the return/exit
code of the executed command (brick process). Hence as of now in such case,
we cannot know with what error the brick has failed to connect.

This patch also fix the runner_end() to perform some cleanup w.r.t
return values.

Backport of:
> Change-Id: Iec52e7f5d87ce938d173f8ef16aa77fd573f2c5e
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14043
> Tested-by: Prasanna Kumar Kalever 
> Reviewed-by: Atin Mukherjee 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Signed-off-by: Prasanna Kumar Kalever 

Change-Id: Ief247b4d4538c1ca03e73aa31beb5fa99853afd6
BUG: 1323564
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14208
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

protocol/server: Do not log ENOENT/ESTALE in fd based fops

2016-05-03T06:51:17+00:00

When fd-fops come on anon-fds there is a chance to log ENOENT/ESTALE for them.
Log it as DEBUG.

 >Change-Id: I8ae53c29d6a66f6a65081c281a9a5c205f53766b
 >BUG: 1315168
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/13621
 >CentOS-regression: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Ashish Pandey 
 >Reviewed-by: Jeff Darcy 
 >Smoke: Gluster Build System 

BUG: 1322523
Change-Id: I4265bcb7dbdd70c3d0c7550434f743675313edf7
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14091
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

server: send lookup on root inode when itable is created

2016-03-31T12:15:32+00:00

This is a backport of http://review.gluster.org/#/c/13837

 * xlators like quota, marker, posix_acl can cause problems
   if inode-ctx are not created.
   sometime these xlarors may not get lookup on root inode
   with below cases
   1) client may not send lookup on root inode (like NSR leader)
   2) if the xlators on one of the bricks are not up,
      and client sending lookup during this time: brick
      can miss the lookup
   It is always better to make sure that there is one lookup
   on root. So send a first lookup when the inode table is created

 * When sending lookup on root, new inode is created, we need to
   use itable->root instead

> Change-Id: Iff2eeaa1a89795328833a7761789ef588f11218f
> BUG: 1320818
> Signed-off-by: vmallika 
> Reviewed-on: http://review.gluster.org/13837
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

Change-Id: I0abf45444c21b3bc77b5a75ab9a2049a411048d3
BUG: 1320892
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/13862
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

cluster/ec: Mark self-heal fops as internal

2016-03-05T01:26:21+00:00

 >Change-Id: I8ae7af266d3e00460f0cfdc9389a926e5f2fee36
 >BUG: 1282761
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/12598
 >Tested-by: Gluster Build System 
 >Tested-by: NetBSD Build System 
 >Reviewed-by: Xavier Hernandez 

BUG: 1283757
Change-Id: Ic20d4ee031265305db1a6ed2cf591ce94b7d0749
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/12668
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

protocol/client: Don't change op_ret when xdata_rsp is present

2016-03-02T07:22:16+00:00

 >Change-Id: Ia69cc420ad7b5766d513ea2715bbca50d8d57132
 >BUG: 1312226
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/13530
 >Smoke: Gluster Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Raghavendra Talur 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Jeff Darcy 
 >(cherry picked from commit 1f673d141fb06282583175357348a7a2fc19e604)

Change-Id: I2befb42ffff7410550f2373ec2ed7c251b911ae5
BUG: 1313448
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13568
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

dht: remove 'null' for directory/file path from brick log for many operations

2016-02-23T08:28:27+00:00

Backport of http://review.gluster.org/#/c/11206/

> Change-Id: I39cd2089240c0ad62b749f176847cc5337e57f73
> BUG: 1231264
> Signed-off-by: Sakshi 
> Reviewed-on: http://review.gluster.org/11206
> Tested-by: NetBSD Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> Reviewed-by: Raghavendra G 

BUG: 1308415
Change-Id: I161844d3b419a8b5fb0300969e54f2ac9182534d
Signed-off-by: Sakshi 
Reviewed-on: http://review.gluster.org/13445
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

protocol/server: Race between server_reconfigure and server_setvolume

2016-01-26T06:28:17+00:00

During server_reconfigure we authenticate each connected clients
against the current options. To do this authentication we store
previous values in a dictionary during the connection establishment
phase (server_setvolume). If the authentication fails during
reconfigure then we will disconnect the transport.

Here it introduce a race between server_setvolume and reconfugure.
If a reconfigure called before doing a setvolume, the transport
will be disconnected

Backport of>
>Change-Id: Icce2c28a171481327a06efd3901f8a5ee67b05ab
>BUG: 1300564
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/13271
>Smoke: Gluster Build System 
>Reviewed-by: Raghavendra G 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Raghavendra Talur 
>NetBSD-regression: NetBSD Build System 

(cherry picked from commit 16f6579cb3e1214b1386fb530b8e16c8cbfdef33)

Change-Id: I8ab7ad4bd5d2d70bcdae7bcbc233930bcfbeb411
BUG: 1300978
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13280
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

upcall: free the xdr* allocations

2016-01-24T17:13:08+00:00

Free the xdr string allocations after decoding the upcall
cache_invalidation request.

This is backport of the below fix
   - http://review.gluster.org/13232

Change-Id: I0ffc64f587d6c8566cba76cf08148f937a735926
BUG: 1300924
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/13232
Reviewed-by: Niels de Vos 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.org/13277
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

protocol/client: prevent use-after-free of frame->root

2015-12-03T00:03:02+00:00

A regression failure generated a coredump on the glusterfs-client side:

  (gdb) f 0
  #0  0x00007fba6cd76432 in client_submit_request (this=0x7fba68006fc0,
                  req=0x7fba6579aa70, frame=0x7fba5c0058cc,
                  prog=0x7fba6cfb53c0 , procnum=41,
                  cbkfn=0x7fba6cd9206d ,
                  iobref=0x0, rsphdr=0x0, rsphdr_count=0,
                  rsp_payload=0x0, rsp_payload_count=0, rsp_iobref=0x0,
                  xdrproc=0x7fba79801075 ) at
  /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/xlators/protocol/client/src/client.c:324
  324                   frame->root->ngrps = ngroups;
  (gdb) l
  319                   gf_msg_debug (this->name, 0, "rpc_clnt_submit failed");
  320           }
  321
  322           if (!conf->send_gids) {
  323                   /* restore previous values */
  324                   frame->root->ngrps = ngroups;
  325                   if (ngroups <= SMALL_GROUP_COUNT)
  326                           frame->root->groups_small[0] = gid;
  327           }
  328
  (gdb) p *frame->root
  Cannot access memory at address 0x64185df000000000

After looking at this in more detail, the flow is like this:

  client_submit_request()
    |
    '- rpc_clnt_submit() // on line 314
         |
         '- cbkfn() // = client3_3_release_cbk
              |
              :- STACK_DESTROY (frame->root);
         .----'
    .----'
    |
    :- frame->root->ngrps = ngroups; // on line 324
    '

So, there is a use-after-free, and it is not needed to restore the
previous groups in frame->root.

Cherry picked from commit dc3aa7524e4974f9d02465e2e5dd6ed9b6d319e1:
> Change-Id: I9e7d712183692ed92cfc2f75cd3c2781a9db20e2
> BUG: 1281285 (was incorrect in original patch)
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/12575
> Reviewed-by: Dan Lambright 
> Tested-by: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

Change-Id: I9e7d712183692ed92cfc2f75cd3c2781a9db20e2
BUG: 1283138
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/12639
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Vijay Bellur

geo-rep: Don't log geo-rep safe errors in mount logs

2015-11-23T19:10:41+00:00

ENOENT is a safe error for geo-replication in case of
rm -rf. RMDIR is recorded in changelog of each brick,
geo-rep processes all changelogs among which one will
succeed and rest will get ENOENT which can be ignored.
Similarly ENOENT can also be ignored in case of all
unlink operation during changelog replay that can
happen when worker goes down and comes back.

Change-Id: I6756f8f4c3fce7a159751a2bfce891ff16ad31a4
BUG: 1283473
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/11833
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Aravinda VK 
Reviewed-by: Milind Changire 
Reviewed-by: Saravanakumar Arumugam 
Reviewed-by: Venky Shankar 
 (cherry picked from commit a52fd2cb7fa3aaff74461f58f32f4ff0b8e0904d)
Reviewed-on: http://review.gluster.org/12651
Reviewed-by: Vijay Bellur