glusterfs.git/xlators/features/bit-rot/src/bitd, branch v3.9.1

feature/bitrot: Ondemand scrub option for bitrot

2016-08-25T21:39:38+00:00

The bitrot scrubber takes 'hourly/daily/biweekly/monthly'
as the values for 'scrub-frequency'. There is no way
to schedule the scrubbing when the admin wants it.

Ondemand scrubbing brings in the new option 'ondemand'
with which the admin can start scrubbing ondemand.
It starts the scrubbing immediately.

Ondemand scrubbing is successful only if the scrubber
is in 'Active (Idle)' (waiting for it's next frequency
cycle to start scrubbing). It is not entertained when
the scrubber is in 'Paused' or already running.

Here is the command line syntax.

gluster volume bitrot  scrub ondemand

Change-Id: I84c28904367eed827a7dae8d6a535c14b28e9f4d
BUG: 1366195
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/15111
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Venky Shankar

eventsapi: Fix disable-events issue

2016-08-24T08:23:06+00:00

Events related sources are not loaded in libglusterfs when
configure is run with --disable-events option. Due to this
every call of gf_event should be guarded with USE_EVENTS macro.

To prevent this, USE_EVENTS macro was included in events.c
itself(Patch #15054)

Instead of disabling building entire directory "events", selectively
disabled the code. So that constants and empty function gf_event is
exposed. Code will not fail even if gf_event is called when events is
disabled.

BUG: 1368042
Change-Id: Ia6abfe9c1e46a7640c4d8ff5ccf0e9c30c87f928
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15198
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

eventsapi: Bitrot events

2016-08-19T11:14:43+00:00

Following Bitrot Events are added

BITROT_ENABLE/BITROT_DISABLE
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": EVENT_TYPE,
    "message": {
       "name": VOLUME_NAME,
    }
}

BITROT_SCRUB_THROTTLE/BITROT_SCRUB_FREQ/BITROT_SCRUB
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": EVENT_TYPE,
    "message": {
       "name": VOLUME_NAME,
       "value": OPTION_VALUE
    }
}

EVENT_BITROT_BAD_FILE
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": EVENT_TYPE,
    "message": {
       "gfid": GFID_OF_FILE,
       "brick": BRICK_ROOT,
       "path":  FILE_PATH_FROM_BRICK_ROOT (if available)
    }
}

Change-Id: I8c37b0e9db9f4f0f3d05d8f78b5521c7db0e2237
BUG: 1367815
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/15190
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

feature/bitrot: Ignore files with sticky bit set

2016-07-21T13:59:49+00:00

Scrubber scrubs entries in backend. It is scrubbing files with sticky
bit as well. This might include linkfiles which should be skipped.  This
patch adds the check to ignore linkfiles during scrub.

Change-Id: Ic21367b37770d391326c55c659491a1e5a82335b
BUG: 1355706
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14903
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
CentOS-regression: NetBSD Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

feature/bitrot: Fix scrub status with sharded volume

2016-07-19T14:41:09+00:00

Bitrot scrubs each shard entries separately. Scrub
statistics was counting each shard entry which is
incorrect. This patch skips the statistics count
for sharded entries.

Change-Id: I184c315a4bc7f2cccabc506eef083ee926ec26d3
BUG: 1356851
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14927
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Vijay Bellur

features/bitrot: Move throttling code to libglusterfs

2016-07-18T12:03:17+00:00

Since throttling is a separate feature by itself,
move throttling code to libglusterfs.

Change-Id: If9b99885ceb46e5b1865a4af18b2a2caecf59972
BUG: 1352019
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14846
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Jeff Darcy

feature/bitrot: Show whether scrub is in progress/idle

2016-07-11T13:32:04+00:00

Bitrot scrub status shows whether the scrub is paused
or active. It doesn't show whether the scrubber is
actually scrubbing or waiting in the timer wheel
for the next schedule. This patch shows this status
with "In Progress" and "Idle" respectively.

Change-Id: I995d8553d1ff166503ae1e7b46282fc3ba961f0b
BUG: 1352871
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14864
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

features/bitrot: Option to set scrub interval to a minute

2016-07-05T11:46:14+00:00

Bitrot scrub-frequency supports "hourly|daily|weekly|biweekly|monthly".
But it is painful for testing as minimum  scrub-interval is an hour
Hence introducing a scrub interval of minute to ease testing.
It is intentionally not exposed in bitrot command help as it is
only for testing.

e.g.,
gluster vol bitrot  scrub-frequency minute

Change-Id: I155a65298d3fad5ae9e529d9c7d4b0d25fa297c0
BUG: 1351537
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14836
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

features/bitrot: Fix Compilation Warning!!!

2016-05-02T18:04:30+00:00

Earlier the lock  was using glusterfs macros
LOCK/UNLOCK/LOCK_INIT/LOCK_DESTROY. The patch
http://review.gluster.org/#/c/14140/ used 'pthread_cleanup_push'
interface for the same lock which was giving "initialization
discards qualifiers from pointer target type". It's strange that
the build succeeded in master branch with no warnings but fails for
the backport http://review.gluster.org/#/c/14140/ in 3.7 branch 
treating this warning as error.

Change-Id: I75c8a65a2bfb1147fe9a84cfd8f09a97c089ae70
BUG: 1332134
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14146
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Jeff Darcy

features/bitrot: Introduce scrubber monitor thread

2016-05-02T04:33:33+00:00

The patch does following changes.

 1. Introduce scrubber monitor thread.
 2. Move scrub status related APIs to separate file
    and make part of libbitrot library.

Problem:
     Earlier, each child of the scrubber was maintaining
the state machine and hence there was no way to track
the start and end time of scrubbing as each brick has
it's own start and end time. Also each brick was maintaining
it's own timer wheel instance. It was also not possible
to get scrubbed files count per session as we could not
get last child which finishes scrubbing to reset it to
zero.

Solution:
    Introduce scrubber monitor thread. It does following.

 1. Maintains the scrubber state machine. Earlier each
    child had it's own state machine. Now, only monitor
    maintains on behalf of all it's children.
 2. Maintains the timer wheel instance. Earlier each
    child had it's own timer wheel instance. Now, only
    monitor maintains on behalf of all it's children.

As a result, we can track the scrub statistics easily
and correctly.

Change-Id: Ic6e34ffa57984bd7a5ee81f4e263342bc1d9b302
BUG: 1329211
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14044
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Venky Shankar