glusterfs.git/tests/basic, branch release-3.8-fb

glusterd: fix missing/renamed options

2017-10-02T19:23:14+00:00

Change-Id: I2ca0298ee9d166f58b8730256ea76a04e547ce5d
Signed-off-by: Jeff Darcy

self-heal: fix automatic split-brain resolution options

2017-09-27T23:03:35+00:00

Differential Revision: https://phabricator.intern.facebook.com/D5927193

Change-Id: Ife04c8738b9ee721e7be9bc843b2f6d54bbb468e

Replace namespace/io-stats/io-threads with 3.6-fb versions

2017-09-15T20:47:01+00:00

This rolls up multiple patches related to namespace identificaton and
throttling/QoS.  This primarily includes the following, all by Michael
Goulet .

  io-threads: Add weighted round robin queueing by namespace
  https://phabricator.facebook.com/D5615269

  io-threads: Add per-namespaces queue sizes to IO_THREADS_QUEUE_SIZE_KEY
  https://phabricator.facebook.com/D5683162

  io-threads: Implement better slot allocation algorithm
  https://phabricator.facebook.com/D5683186

  io-threads: Only enable weighted queueing on bricks
  https://phabricator.facebook.com/D5700062

  io-threads: Update queue sizes on drain
  https://phabricator.facebook.com/D5704832

  Fix parsing (-1) as default NS weight
  https://phabricator.facebook.com/D5723383

Parts of the following patches have also been applied to satisfy
dependencies.

  io-throttling: Calculate moving averages and throttle offending hosts
  https://phabricator.fb.com/D2516161
  Shreyas Siravara 

  Hook up ODS logging for FUSE clients.
  https://phabricator.facebook.com/D3963376
  Kevin Vigor 

  Add the flag --skip-nfsd-start to skip the NFS daemon stating, even if
  it is enabled
  https://phabricator.facebook.com/D4575368
  Alex Lorca 

There are also some "standard" changes: dealing with code that moved,
reindenting to comply with Gluster coding standards, gf_uuid_xxx, etc.

This patch *does* revert some changes which have occurred upstream since
3.6; these will be re-applied as apppropriate on top of this new base.

Change-Id: I69024115da7a60811e5b86beae781d602bdb558d
Signed-off-by: Jeff Darcy

Disable brick daemon from incorrect brick directory

2017-09-13T17:50:36+00:00

Summary:
Currently the bricks can open any mount directory from the given volume. This patch adds a provision to prevent
bricks from opening brick directories that aren't created for them. This will help with operating gluster on large
scale.

We add a new xattr GF_XATTR_BRICK_NAME to the brick directory. When we start a brick daemon, we make sure the path on
disk matches with the config provided. For backward compatibility, we ignore if there is no value for
GF_XATTR_BRICK_NAME and set the current brick daemon's path as value.

We ignore GF_XATTR_BRICK_NAME during healing and reset GF_XATTR_BRICK_NAME on brick replace.

Test Plan: Run fb-smoke

Reviewers: jdarcy, sshreyas

Reviewed By: sshreyas

Differential Revision: https://phabricator.intern.facebook.com/D5448921

Porting note: disabled some checks to deal with the snapshot case

Change-Id: I98e62033dfd07f30ad3b99ac003ce94c8d935e5f
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18275
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

nfs: Enable multi-core epoll support in gNFSd

2017-09-12T15:28:24+00:00

Summary:
- Enables multi-core epoll support in the nfs daemon.
- Option can be turned on using:
  gluster volume set  nfs.event-threads 

Test Plan: Prove test!

Reviewers: kvigor, rwareing

Reviewed By: rwareing

Subscribers: dld, moox, dph

Differential Revision: https://phabricator.fb.com/D3117966

Change-Id: Ie8a7b1ba04b0e83f5ec7a09f9d181fe59be479ca
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18266
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

event: Idle connection management

2017-09-12T15:12:21+00:00

Summary:
- This diff adds support for detecting and tracking idle client connections.
- It allows *service translators* (server, nfs) to opt-in to detect and close idle client connections.
- Right now it explicitly restricts the service to NFS as a safety.

Here are the debug logs when a client connection gets closed:

  [2016-03-29 17:27:06.154232] W [socket.c:2426:socket_timeout_handler] 0-socket: Shutting down idle client connection (idle=20s,fd=20,conn=[2401:db00:11:d0af:face:0:3:0:957]->[2401:db00:11:d0af:face:0:3:0:2049])!
  [2016-03-29 17:27:06.154292] D [event-epoll.c:655:__event_epoll_timeout_slot] 0-epoll: Connection on slot->fd=9 was idle for 20 seconds!
  [2016-03-29 17:27:06.163282] D [socket.c:629:__socket_rwv] 0-socket.nfs-server: EOF on socket
  [2016-03-29 17:27:06.163298] D [socket.c:2474:socket_event_handler] 0-transport: disconnecting now
  [2016-03-29 17:27:06.163316] D [event-epoll.c:614:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=4 to slot->gen=5, fd=20, slot->fd=20

Test Plan: - Used stuck NFS mounts to create idle clients and unstuck them.

Reviewers: kvigor, rwareing

Reviewed By: rwareing

Subscribers: dld, moox, dph

Differential Revision: https://phabricator.fb.com/D3112099

Change-Id: Ic06c89e03f87daabab7f07f892390edd1a1fcc20
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18265
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

debug/io-stats: Track outstanding requests

2017-09-08T04:14:24+00:00

Summary:
Add outstanding-req field to track requests that have been sent
down the stack and haven't come back.
This is a port of D4908836 to 3.8

Reviewers: sshreyas

Change-Id: I5870f63008d553416109c1808a434f526f5a633d
Reviewed-on: https://review.gluster.org/18236
Reviewed-by: Shreyas Siravara 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

[io-cache] New volume options for read sizes

2017-09-08T00:19:53+00:00

Summary:
Two new volume options that control reads.

performance.io-cache.read-size
- Tells gluster how much it should try to read on each posix_readv call

performance.io-cache.min-cached-read-size
- Tells gluster the smallest files it should start caching, anything smaller is not cached

This is a port of D4844662 to 3.8

Change-Id: I5ba891906f97e514e7365cc34374619379434766
Reviewed-on: https://review.gluster.org/18235
Reviewed-by: Shreyas Siravara 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

storage/posix: Add limit to number of hard links.

2017-09-08T00:13:22+00:00

Summary:
Too may hard links blow up btrfs by exceeding max xattr size (recordign
pgfid for each hardlink). Add a limit to prevent this explosion.

This is a port D4682329 to 3.8

Reviewed By: sshreyas

Change-Id: I614a247834fb8f2b2743c0c67d11cefafff0dbaa
Reviewed-on: https://review.gluster.org/18232
Reviewed-by: Shreyas Siravara 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

cluster/afr: Set AFR UP message as soon as quorum is obtained.

2017-09-07T23:55:31+00:00

Summary:
AFR currently waits for all children to respond before sending an UP
message. This means that one dead host cal cause us to wait a TCP
timeout (2 mins!) before declaring the volume up.

Now we send an UP as soon as quorum is obtained.

This is a port of D4701919 to 3.8.

Reviewed By: sshreyas

Change-Id: I642d4eb7dc7e0b289e89b7a16abf99a3f98aa8b3
Reviewed-on: https://review.gluster.org/18231
Smoke: Gluster Build System 
Reviewed-by: Shreyas Siravara 
CentOS-regression: Gluster Build System