diff options
author | Jeff Darcy <jdarcy@fb.com> | 2017-07-21 06:56:00 -0700 |
---|---|---|
committer | Jeff Darcy <jeff@pl.atyp.us> | 2017-09-13 14:46:59 +0000 |
commit | e6245ebc0bb373623e624d8a75569137d580df46 (patch) | |
tree | ddb930501f7c1bb20f37938bb7130f6a98a64d8f | |
parent | 5c30bda609f99e3360e11dc3e6ac2c727a11171a (diff) |
Wait for a brick's local filesystem to be mounted before starting.
Summary: By far the most common reason why a brick's directory might not exist is that the local filesystem on which it lives hasn't finished mounting yet. This is unlike other checks we do, such as for a volume ID and GFID. Some of these are normal conditions when a brick is firstcreated; others are often the result of operator/script error. In the singular case of the directory being absent, wait a little while to see if it comes up.
Test Plan:
Create a volume.
Start/stop a volume once so everything gets initialized.
Move a brick directory out of place.
Try to start the volume. This should pause.
Immediately move the brick directory back into place. This should break the pause.
Reviewers: #posix_storage, sshreyas
Reviewed By: sshreyas
Subscribers: shreyas, sshreyas, ventullo, moox
Differential Revision: https://phabricator.intern.facebook.com/D5063515
Tags: gluster
Change-Id: Ied7b07b1a60f54856a67d4cdbad35bfce9e196e4
Signed-off-by: Jeff Darcy <jdarcy@fb.com>
Reviewed-on: https://review.gluster.org/18274
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Tested-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd-volume-ops.c | 17 |
1 files changed, 13 insertions, 4 deletions
diff --git a/xlators/mgmt/glusterd/src/glusterd-volume-ops.c b/xlators/mgmt/glusterd/src/glusterd-volume-ops.c index 8f2a23a898a..97fb5605715 100644 --- a/xlators/mgmt/glusterd/src/glusterd-volume-ops.c +++ b/xlators/mgmt/glusterd/src/glusterd-volume-ops.c @@ -1482,6 +1482,7 @@ glusterd_op_stage_start_volume (dict_t *dict, char **op_errstr, char volid[50] = {0,}; char xattr_volid[50] = {0,}; int caps = 0; + uint32_t i; this = THIS; GF_ASSERT (this); @@ -1556,10 +1557,18 @@ glusterd_op_stage_start_volume (dict_t *dict, char **op_errstr, (brickinfo->snap_status == -1)) continue; - ret = gf_lstat_dir (brickinfo->path, NULL); - if (ret && (flags & GF_CLI_FLAG_OP_FORCE)) { - continue; - } else if (ret) { + for (i = 0; i < 30; ++i) { + ret = gf_lstat_dir (brickinfo->path, NULL); + if (ret == 0) { + break; + } + gf_log (brickinfo->path, GF_LOG_INFO, "not ready yet; sleeping"); + sleep (1); + } + if (ret != 0) { + if (flags & GF_CLI_FLAG_OP_FORCE) { + continue; + } snprintf (msg, sizeof (msg), "Failed to find " "brick directory %s for volume %s. " "Reason : %s", brickinfo->path, |