summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorJeff Darcy <jdarcy@fb.com>2017-07-21 06:56:00 -0700
committerJeff Darcy <jeff@pl.atyp.us>2017-09-13 14:46:59 +0000
commite6245ebc0bb373623e624d8a75569137d580df46 (patch)
treeddb930501f7c1bb20f37938bb7130f6a98a64d8f
parent5c30bda609f99e3360e11dc3e6ac2c727a11171a (diff)
Wait for a brick's local filesystem to be mounted before starting.
Summary: By far the most common reason why a brick's directory might not exist is that the local filesystem on which it lives hasn't finished mounting yet. This is unlike other checks we do, such as for a volume ID and GFID. Some of these are normal conditions when a brick is firstcreated; others are often the result of operator/script error. In the singular case of the directory being absent, wait a little while to see if it comes up. Test Plan: Create a volume. Start/stop a volume once so everything gets initialized. Move a brick directory out of place. Try to start the volume. This should pause. Immediately move the brick directory back into place. This should break the pause. Reviewers: #posix_storage, sshreyas Reviewed By: sshreyas Subscribers: shreyas, sshreyas, ventullo, moox Differential Revision: https://phabricator.intern.facebook.com/D5063515 Tags: gluster Change-Id: Ied7b07b1a60f54856a67d4cdbad35bfce9e196e4 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/18274 Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
-rw-r--r--xlators/mgmt/glusterd/src/glusterd-volume-ops.c17
1 files changed, 13 insertions, 4 deletions
diff --git a/xlators/mgmt/glusterd/src/glusterd-volume-ops.c b/xlators/mgmt/glusterd/src/glusterd-volume-ops.c
index 8f2a23a898a..97fb5605715 100644
--- a/xlators/mgmt/glusterd/src/glusterd-volume-ops.c
+++ b/xlators/mgmt/glusterd/src/glusterd-volume-ops.c
@@ -1482,6 +1482,7 @@ glusterd_op_stage_start_volume (dict_t *dict, char **op_errstr,
char volid[50] = {0,};
char xattr_volid[50] = {0,};
int caps = 0;
+ uint32_t i;
this = THIS;
GF_ASSERT (this);
@@ -1556,10 +1557,18 @@ glusterd_op_stage_start_volume (dict_t *dict, char **op_errstr,
(brickinfo->snap_status == -1))
continue;
- ret = gf_lstat_dir (brickinfo->path, NULL);
- if (ret && (flags & GF_CLI_FLAG_OP_FORCE)) {
- continue;
- } else if (ret) {
+ for (i = 0; i < 30; ++i) {
+ ret = gf_lstat_dir (brickinfo->path, NULL);
+ if (ret == 0) {
+ break;
+ }
+ gf_log (brickinfo->path, GF_LOG_INFO, "not ready yet; sleeping");
+ sleep (1);
+ }
+ if (ret != 0) {
+ if (flags & GF_CLI_FLAG_OP_FORCE) {
+ continue;
+ }
snprintf (msg, sizeof (msg), "Failed to find "
"brick directory %s for volume %s. "
"Reason : %s", brickinfo->path,