glusterfs.git/tests/basic/afr, branch release-3.8-fb

self-heal: fix automatic split-brain resolution options

2017-09-27T23:03:35+00:00

Differential Revision: https://phabricator.intern.facebook.com/D5927193

Change-Id: Ife04c8738b9ee721e7be9bc843b2f6d54bbb468e

Replace namespace/io-stats/io-threads with 3.6-fb versions

2017-09-15T20:47:01+00:00

This rolls up multiple patches related to namespace identificaton and
throttling/QoS.  This primarily includes the following, all by Michael
Goulet .

  io-threads: Add weighted round robin queueing by namespace
  https://phabricator.facebook.com/D5615269

  io-threads: Add per-namespaces queue sizes to IO_THREADS_QUEUE_SIZE_KEY
  https://phabricator.facebook.com/D5683162

  io-threads: Implement better slot allocation algorithm
  https://phabricator.facebook.com/D5683186

  io-threads: Only enable weighted queueing on bricks
  https://phabricator.facebook.com/D5700062

  io-threads: Update queue sizes on drain
  https://phabricator.facebook.com/D5704832

  Fix parsing (-1) as default NS weight
  https://phabricator.facebook.com/D5723383

Parts of the following patches have also been applied to satisfy
dependencies.

  io-throttling: Calculate moving averages and throttle offending hosts
  https://phabricator.fb.com/D2516161
  Shreyas Siravara 

  Hook up ODS logging for FUSE clients.
  https://phabricator.facebook.com/D3963376
  Kevin Vigor 

  Add the flag --skip-nfsd-start to skip the NFS daemon stating, even if
  it is enabled
  https://phabricator.facebook.com/D4575368
  Alex Lorca 

There are also some "standard" changes: dealing with code that moved,
reindenting to comply with Gluster coding standards, gf_uuid_xxx, etc.

This patch *does* revert some changes which have occurred upstream since
3.6; these will be re-applied as apppropriate on top of this new base.

Change-Id: I69024115da7a60811e5b86beae781d602bdb558d
Signed-off-by: Jeff Darcy

cluster/afr: Fix gfid unsplit code when renamed filename exceeds NAME_MAX

2017-09-03T04:30:49+00:00

Summary:
- We noticed some folks name their files all the way up to NAME_MAX (usually 255) and when split-brain is encountered, we fail to heal the file.
- This diff puts an upper bound on the number of bytes we will snprintf into the buffer so that we do not fail the rename.
- This is a port of D3646254 to 3.8

Test Plan: Prove test -- can show it fails without patch as well.

Reviewers: #posix_storage, rwareing

Reviewed By: rwareing

Change-Id: I51c6b28374d4a3f21e29044cb727b4b1da7b69e1
Reviewed-on: https://review.gluster.org/18194
Reviewed-by: Shreyas Siravara 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

tests: fix timing issues from the previous merge

2017-08-31T20:13:38+00:00

Change-Id: Ic77287c1b96ae426b927b4bf6f2826d6f3a3b17d
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18175
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

cluster/afr: Fix case in PGFID healing where NOOP was not being honored

2017-08-01T11:48:43+00:00

Summary:
- PGFID healing should not be triggered in the case where there is
  nothing to do (ret = 2).  Instead this return code should be returned
  to the heal daemon to trigger the reap of the entry.
- Reworked shd-pgfid-heal.t to queue up heal naturally instead of
  synthetically

Test Plan: - Run tests/basic/afr/shd-pgfid-heal.t

Differential Revision: https://phabricator.fb.com/D2748578

Change-Id: I74300de2b4dce23867f4111548de35f58bf77453
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/17936
Smoke: Gluster Build System 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

afr/cluster: PGFID heal support

2017-07-31T21:23:56+00:00

Summary:
  PGFID healing enables heals which might otherwise fail due
  due to the lack of a entry heal to succeed by performing
  the entry healing within the same heal flow.

  It does this by leveraging the PGFID tracking feature of
  the POSIX xlator, and examining lookup replies for the
  PGFID attribute.  If detected, the pgfid will be decoded
  and stored for later use in case the heal fails for whatever
  reason.  Cascading heal failures are handled through
  recursion.

  This feature is critical for a couple reasons:
      1. General healing predictability - When the SHD
         attempts to heal a given GFID, it should be able
         to do so without having to wait for some other
         dependent heal to take place.
      2. Reliability - In some cases the parent directory
         may require healing, but the req'd entry in the
         indices/xattrop directory may not exist
         (e.g. bugs/crashes etc).  Prior to PGFID heal support
         some sort of external script would be required to
         queue up these heals by using FS specific utilities
         to lookup the parent directory by hardlink or
         worse...do a costly full heal to clean them up.
      3. Performance - In combination with multi-threaded SHD
         this feature will make SHD healing _much_ faster as
         directories with large amount of files to be healed
         will no longer have to wait for an entry heal to
         come along, the first file in that directory queued
         for healing will trigger an entry heal for the directory
         and this will allow the other files in that directory
         to be (immediatelly) healed in parallel.

Test Plan:
- run prove tests/basic/afr/shd_pgfid_heal.t
- run prove tests/basic/afr/shd*.t
- run prove tests/basic/afr/gfid*.t

Differential Revision: https://phabricator.fb.com/D2546133

Change-Id: I25f586047f8bcafa900c0cc9ee8f0e2128688c73
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/17929
Smoke: Gluster Build System 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

tests: forward-porting test fixes from 3.6 branch

2017-07-05T17:15:14+00:00

Change-Id: I4074e7cce8f6782860f849780ab6d0458e92a2ce
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/17708
Smoke: Gluster Build System 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

tests: avoid symlink between brick and mountpoint

2017-05-13T15:57:26+00:00

These were causing hung tests when the bricks were restarted, in some
cases even accompanied by kernel stack traces involving fuse.  They're
unnecessary to the purpose of the test.

Change-Id: I3c6c485324e2ee9418eb54929015b5bae436e9ea
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/17284
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Kevin Vigor

tests: Fix split-brain-favorite-child-policy.t failures

2017-01-05T11:41:48+00:00

Problem:
In CentOS-7, the file was receving an extra removexattr(security.ima)
FOP which changed its ctime, breaking the assumption that a particular brick
had the latest ctime based on the writevs done in the .t

Fix:
1. Compare the ctime of both files in the backend and pick the one with
the latest ctime for the fav-child policy. Also unmount the volume
before comparing, to avoid any further FOPS on the file that
can possibly modify the timestamps.

2. Added floating point handling in stat function. Thanks to Pranith for
the helping debugging the regex.

> Reviewed-on: http://review.gluster.org/16288
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit 76fff8cb2a164b596ca67e65c99623f5b68361fd)

Change-Id: I06041a0f39a29d2593b867af8685d65c7cd99150
BUG: 1410073
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16324
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

glfsheal: Explicitly enable self-heal xlator options

2016-12-16T01:34:03+00:00

Enable data, metadata and entry self-heal as xlator-options so that glfs-heal.c
can heal split-brain files even if they are disabled on the volume via volume
set commands.

> Reviewed-on: http://review.gluster.org/11333
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> Tested-by: Pranith Kumar Karampuri 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 209c2d447be874047cb98d86492b03fa807d1832)

Change-Id: Ic191a1017131db1ded94d97c932079d7bfd79457
BUG: 1405130
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16144
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System