glusterfs.git/heal, branch v3.10dev glusterd : Introduce reset brick 2016-08-30T02:55:53+00:00 Anuradha Talur atalur@redhat.com 2016-08-22T17:22:03+00:00 936f8aeac3252951e7fa0cdaa5d260fad3bd5ea0 The command basically allows replace brick with src and dst bricks as same. Usage: gluster v reset-brick <volname> <hostname:brick-path> start This command kills the brick to be reset. Once this command is run, admin can do other manual operations that they need to do, like configuring some options for the brick. Once this is done, resetting the brick can be continued with the following options. gluster v reset-brick <vname> <hostname:brick> <hostname:brick> commit {force} Does the job of resetting the brick. 'force' option should be used when the brick already contains volinfo id. Problem: On doing a disk-replacement of a brick in a replicate volume the following 2 scenarios may occur : a) there is a chance that reads are served from this replaced-disk brick, which leads to empty reads. b) potential data loss if next writes succeed only on replaced brick, and heal is done to other bricks from this one. Solution: After disk-replacement, make sure that reset-brick command is run for that brick so that pending markers are set for the brick and it is not chosen as source for reads and heal. But, as of now replace-brick for the same brick-path is not allowed. In order to fix the above mentioned problem, same brick-path replace-brick is needed. With this patch reset-brick commit {force} will be allowed even when source and destination <hostname:brickpath> are identical as long as 1) destination brick is not alive 2) source and destination brick have the same brick uuid and path. Also, the destination brick after replace-brick will use the same port as the source brick. Change-Id: I440b9e892ffb781ea4b8563688c3f85c7a7c89de BUG: 1266876 Signed-off-by: Anuradha Talur <atalur@redhat.com> Reviewed-on: http://review.gluster.org/12250 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Ashish Pandey <aspandey@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> 
The command basically allows replace brick with src and
dst bricks as same.

Usage:
gluster v reset-brick <volname> <hostname:brick-path> start
This command kills the brick to be reset. Once this command is run,
admin can do other manual operations that they need to do,
like configuring some options for the brick. Once this is done,
resetting the brick can be continued with the following options.

gluster v reset-brick <vname> <hostname:brick> <hostname:brick> commit {force}

Does the job of resetting the brick. 'force' option should be used
when the brick already contains volinfo id.

Problem: On doing a disk-replacement of a brick in a replicate volume
the following 2 scenarios may occur :

a) there is a chance that reads are served from this replaced-disk brick,
which leads to empty reads. b) potential data loss if next writes succeed
only on replaced brick, and heal is done to other bricks from this one.

Solution: After disk-replacement, make sure that reset-brick command is
run for that brick so that pending markers are set for the brick and it
is not chosen as source for reads and heal. But, as of now replace-brick
for the same brick-path is not allowed. In order to fix the above
mentioned problem, same brick-path replace-brick is needed.
With this patch reset-brick commit {force} will be allowed even when
source and destination <hostname:brickpath> are identical as long as
1) destination brick is not alive
2) source and destination brick have the same brick uuid and path.
Also, the destination brick after replace-brick will use the same port
as the source brick.

Change-Id: I440b9e892ffb781ea4b8563688c3f85c7a7c89de
BUG: 1266876
Signed-off-by: Anuradha Talur <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/12250
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
glfsheal: print brick name and path even when brick is down 2016-08-12T06:25:30+00:00 Ravishankar N ravishankar@redhat.com 2016-08-11T10:10:25+00:00 5ef32c57f327e1dd4e9d227b9c8fd4b6f6fb4970 The xml variant of heal info command does not display brick name when the brick is down due to a failure to fetch the hostUUID. But the non xml variant does. So fixed the xml variant to print the remote_host and remote_subvol even when the brick is down. Change-Id: I16347eb4455b9bcc7a9b0127f8783140b6016578 BUG: 1366222 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/15146 Reviewed-by: Anuradha Talur <atalur@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> 
The xml variant of heal info command does not display brick name when
the brick is down due to a failure to fetch the hostUUID. But the non
xml variant does. So fixed the xml variant to print the remote_host
and remote_subvol even when the brick is down.

Change-Id: I16347eb4455b9bcc7a9b0127f8783140b6016578
BUG: 1366222
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/15146
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
heal/xml : xml implementation of heal info and splitbrain info 2016-05-08T12:19:50+00:00 hari gowtham hgowtham@redhat.com 2016-04-28T06:18:23+00:00 f0ade919006b2581ae192f997a8ae5bacc2892af Current output: heal didn't have xml output. expected output with patch: gluster v heal v1 info --xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <cliOutput> <healInfo> <bricks> <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c"> <name>10.70.42.183:/home/bricks/b1</name> <status>Connected</status> <numberOfEntries>0</numberOfEntries> </brick> <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c"> <name>10.70.42.183:/home/bricks/b2</name> <status>Connected</status> <numberOfEntries>0</numberOfEntries> </brick> </bricks> </healInfo> <opRet>0</opRet> <opErrno>0</opErrno> <opErrstr/> </cliOutput> gluster v heal v1 info split-brain --xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <cliOutput> <healInfo> <bricks> <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c"> <name>10.70.42.183:/home/bricks/b1</name> <status>Connected</status> <numberOfEntries>0</numberOfEntries> </brick> <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c"> <name>10.70.42.183:/home/bricks/b2</name> <status>Connected</status> <numberOfEntries>0</numberOfEntries> </brick> </bricks> </healInfo> <opRet>0</opRet> <opErrno>0</opErrno> <opErrstr/> </cliOutput> Change-Id: I20ad5eb7950d4721364ab46089e3af515162db82 BUG: 1331287 Signed-off-by: hari gowtham <hgowtham@redhat.com> Reviewed-on: http://review.gluster.org/14100 Reviewed-by: Ravishankar N <ravishankar@redhat.com> Tested-by: Ravishankar N <ravishankar@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> 
Current output:
heal didn't have xml output.

expected output with patch:
gluster v heal v1 info --xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cliOutput>
  <healInfo>
    <bricks>
      <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c">
        <name>10.70.42.183:/home/bricks/b1</name>
        <status>Connected</status>
        <numberOfEntries>0</numberOfEntries>
      </brick>
      <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c">
        <name>10.70.42.183:/home/bricks/b2</name>
        <status>Connected</status>
        <numberOfEntries>0</numberOfEntries>
      </brick>
    </bricks>
  </healInfo>
  <opRet>0</opRet>
  <opErrno>0</opErrno>
  <opErrstr/>
</cliOutput>

gluster v heal v1 info split-brain --xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cliOutput>
  <healInfo>
    <bricks>
      <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c">
        <name>10.70.42.183:/home/bricks/b1</name>
        <status>Connected</status>
        <numberOfEntries>0</numberOfEntries>
      </brick>
      <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c">
        <name>10.70.42.183:/home/bricks/b2</name>
        <status>Connected</status>
        <numberOfEntries>0</numberOfEntries>
      </brick>
    </bricks>
  </healInfo>
  <opRet>0</opRet>
  <opErrno>0</opErrno>
  <opErrstr/>
</cliOutput>

Change-Id: I20ad5eb7950d4721364ab46089e3af515162db82
BUG: 1331287
Signed-off-by: hari gowtham <hgowtham@redhat.com>
Reviewed-on: http://review.gluster.org/14100
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: Ravishankar N <ravishankar@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
heal: Fix incorrect heal info output 2016-05-05T01:46:23+00:00 Anuradha Talur atalur@redhat.com 2016-05-04T06:25:43+00:00 2326f8c2a775f6536d95e2e4676866f1b5dfc368 Problem: In heterogenous clusters, heal info gives incorrect outout as "Failed to process entries completely. Number of entries so far: 0". This happens when the getxattr on virtual xattr for <brickpath>/.glusterfs/indices/dirty fails on older bricks as they do not recognize the xattr. Fix: Ignore that error so that heal info doesn't incorrectly report failure. Change-Id: I0d3541ed0594d67c3c3b8568a89a975a100bf6dd BUG: 1332798 Signed-off-by: Anuradha Talur <atalur@redhat.com> Reviewed-on: http://review.gluster.org/14199 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> 
Problem:
In heterogenous clusters, heal info gives incorrect
outout as "Failed to process entries completely.
Number of entries so far: 0".

This happens when the getxattr on virtual xattr
for <brickpath>/.glusterfs/indices/dirty fails
on older bricks as they do not recognize the xattr.

Fix:
Ignore that error so that heal info doesn't
incorrectly report failure.

Change-Id: I0d3541ed0594d67c3c3b8568a89a975a100bf6dd
BUG: 1332798
Signed-off-by: Anuradha Talur <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/14199
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
afr: add mtime based split-brain resolution to CLI 2016-03-29T19:08:35+00:00 Ravishankar N ravishankar@redhat.com 2016-03-25T13:18:30+00:00 08d18ef9257067fac510af408665360019566000 Extended the CLI to include support for split-brain resolution based on mtime. The command syntax is: $:gluster volume heal <VOLNAME> split-brain latest-mtime <FILE> where <FILE> can be either the full file name as seen from the root of the volume (or) the gfid-string representation of the file. Change-Id: I7a16f72ff1a4495aa69f43f22758a9404e958b4f BUG: 1321322 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/13828 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> 
Extended the CLI to include support for split-brain resolution based on
mtime. The command syntax is:

$:gluster volume heal <VOLNAME> split-brain latest-mtime <FILE>

where <FILE> can be either the full file name as seen from the root of the
volume (or) the gfid-string representation of the file.

Change-Id: I7a16f72ff1a4495aa69f43f22758a9404e958b4f
BUG: 1321322
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/13828
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
glfs/heal: Use encrypted connection in shd 2016-03-24T23:55:00+00:00 Ashish Pandey aspandey@redhat.com 2016-03-23T05:56:49+00:00 351ec36e3146b7605334cb658927b447b1dbc796 When management encryption is enabled, GlusterD only allows encrypted connections for port 24007. SHD is trying to fetch it's volfile using an unencrypted connection. If /var/lib/glusterd/secure-access is present , i.e. if management ssl is enabled, use encrypted connection fecth info from glusterd. Change-Id: Ia41dcde1a69375379409889ec5b3a70cb95a4fd0 BUG: 1320388 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: http://review.gluster.org/13815 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> 
When management encryption is enabled, GlusterD
only allows encrypted connections for port 24007.
SHD is trying to fetch it's volfile using an
unencrypted connection.

If /var/lib/glusterd/secure-access is present , i.e.
if management ssl is enabled, use encrypted connection
fecth info from glusterd.

Change-Id: Ia41dcde1a69375379409889ec5b3a70cb95a4fd0
BUG: 1320388
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: http://review.gluster.org/13815
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
heal: Remove sleep() 2016-02-11T17:05:46+00:00 Pranith Kumar K pkarampu@redhat.com 2016-02-10T09:58:39+00:00 320779d53ae013147d5e2556d2946c73e45734ab I wrote this program from a sample gfapi program which had sleep. I am not sure why this sleep was needed. So removing it now. Changed tests/bugs/replicate/bug-1190069-afr-stale-index-entries.t to execute count_sh_entries every second, instead of comparing same value over and over. Change-Id: I7b89d6cab3e50bb7bf4d40a6064f2d8734155bea BUG: 1306199 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/13421 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> 
I wrote this program from a sample gfapi program which had sleep.
I am not sure why this sleep was needed. So removing it now.

Changed tests/bugs/replicate/bug-1190069-afr-stale-index-entries.t
to execute count_sh_entries every second, instead of comparing
same value over and over.

Change-Id: I7b89d6cab3e50bb7bf4d40a6064f2d8734155bea
BUG: 1306199
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/13421
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
heal: Have fixed number of fields in heal info output 2016-02-02T10:25:14+00:00 Pranith Kumar K pkarampu@redhat.com 2015-12-21T12:25:49+00:00 ac3183a5012bfed26fa0aead7f359f5d5b00e23e Sample output: ⚡ gluster v heal r2 info Brick localhost.localdomain:/home/gfs/r2_0 /a / Status: Connected Number of entries: 2 Brick localhost.localdomain:/home/gfs/r2_1 Status: Transport endpoint is not connected Number of entries: - Brick localhost.localdomain:/home/gfs/r2_2 Status: Connected Number of entries: 0 Brick localhost.localdomain:/home/gfs/r2_3 Status: Connected Number of entries: 0 Partial failure will print: ... <gfid:3efb51cf-5fa5-4bed-817b-afd600e9449e> Status: Failed to process entries completely. (Transport endpoint is not connected) Number of entries: 372 ... BUG: 1297897 Change-Id: I79131cbf255acd98f11eedafb53246c465f6665d Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/13231 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-by: Sahina Bose <sabose@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> 
Sample output:
⚡ gluster v heal r2 info
Brick localhost.localdomain:/home/gfs/r2_0
/a
/
Status: Connected
Number of entries: 2

Brick localhost.localdomain:/home/gfs/r2_1
Status: Transport endpoint is not connected
Number of entries: -

Brick localhost.localdomain:/home/gfs/r2_2
Status: Connected
Number of entries: 0

Brick localhost.localdomain:/home/gfs/r2_3
Status: Connected
Number of entries: 0

Partial failure will print:
...
<gfid:3efb51cf-5fa5-4bed-817b-afd600e9449e>
Status: Failed to process entries completely. (Transport endpoint is not connected)
Number of entries: 372
...

BUG: 1297897
Change-Id: I79131cbf255acd98f11eedafb53246c465f6665d
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/13231
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Sahina Bose <sabose@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
heal : Do not print heal count on ENOTCONN 2015-12-21T04:49:52+00:00 Anuradha Talur atalur@redhat.com 2015-12-09T09:20:41+00:00 a99bbccbb5e3087b9b8b687ee2be8887eeac0d08 When a brick is not reachable due to ENOTCONN, there are no entries gathered and this information should not be printed. It is a bug intorduced due to commit 9c378026e9561595586a817fee0b439e2c863a22, fixing it. Change-Id: I45559a9560c297854ea6b4177f86e0be30dc6b78 BUG: 1250803 Signed-off-by: Anuradha Talur <atalur@redhat.com> Reviewed-on: http://review.gluster.org/12919 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> 
When a brick is not reachable due to ENOTCONN,
there are no entries gathered and this information
should not be printed.
It is a bug intorduced due to commit
9c378026e9561595586a817fee0b439e2c863a22, fixing it.

Change-Id: I45559a9560c297854ea6b4177f86e0be30dc6b78
BUG: 1250803
Signed-off-by: Anuradha Talur <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/12919
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
heal : Changed heal info to process all indices directories 2015-12-03T00:00:16+00:00 Anuradha Talur atalur@redhat.com 2015-11-23T10:26:28+00:00 9c378026e9561595586a817fee0b439e2c863a22 Change-Id: Ida863844e14309b6526c1b8434273fbf05c410d2 BUG: 1250803 Signed-off-by: Anuradha Talur <atalur@redhat.com> Reviewed-on: http://review.gluster.org/12658 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
Change-Id: Ida863844e14309b6526c1b8434273fbf05c410d2
BUG: 1250803
Signed-off-by: Anuradha Talur <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/12658
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>