glusterfs.git/xlators/features/changelog/src/changelog-misc.h, branch v3.12dev

core: use syscall wrappers instead of direct syscalls

2015-10-28T23:34:45+00:00

various xlators and other components are invoking system calls
directly instead of using the libglusterfs/syscall.[ch] wrappers.

If not using the system call wrappers there should be a comment
in the source explaining why the wrapper isn't used.

Change-Id: I8ef94c48728666465abf126c778b70c9e5c00e47
BUG: 1267967
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/12273
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

features/changelog: Fixing compilation issue

2015-05-08T15:18:30+00:00

This issue introduced due to manual rebase.

Change-Id: I0589f4a0a1270190340f419b8022d6483bcf853d
Signed-off-by: Aravinda VK 
BUG: 1219479
Reviewed-on: http://review.gluster.org/10685
Reviewed-by: Niels de Vos 
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Niels de Vos

features/changelog: Avoid creation of empty changelogs

2015-05-08T11:52:51+00:00

An empty changelog when rolled over gets unlinked and indexed with
a modified path-name in htime file. The modification is "changelog"
not "CHANGELOG" in basename of the empty changelog file.

Change-Id: I77fd0b48b5c33c245418f5ac7a9756f08ece24d9
BUG: 1208470
Signed-off-by: Ajeet Jha 
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/9572
Tested-by: NetBSD Build System
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Reviewed-by: Vijay Bellur

features/changelog: Version support for Changelog Parser

2015-05-07T06:01:51+00:00

As optional feature, during unlink, full path will be recorded.
Changelog Version number to be bumped up to 1.2.

With this patch, parser checks the version number before parsing
and handles accordingly.

Change-Id: Ic1ad98259c39e417029a08e26a1d4b467817e65a
BUG: 1214561
Signed-off-by: Aravinda VK 
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/10166
Reviewed-by: Saravanakumar Arumugam 
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar

feature/changelog: Capture path for deletes

2015-05-04T15:10:12+00:00

PROBLEM:
There is no way to get the path of deleted file if we
have gfid from changelog since the file is already deleted.

SOLUTION:
Do a recursive readlink on parent gfid in backend .glusterfs
path to get the complete path in I/O callpath in changelog
translator and capture it in callback.

The path captured is relative from the brick root. The field
separator used is '\0'.
e.g.,
......\0/bname\0\0

ADDITIONAL REQUIRED CHANGES:

1. The changelog translator option called "changelog.capture-del-path"
   is introduced to enable or disable the capturing of deleted entry
   path.
   e.g.,
   gluster vol set  changelog.capture-del-path on/off

   If capture-del-path is disabled, '\0' is captured instead of
   relative path.
   e.g.,
   ......\0/bname\0\0\0

2. The minor number in the version of changelog is bumped up from v1.1
   to v1.2.

3. If recursive readlink is failed for some reason, it will capture
   \0 in place of .
   e.g.,
   ......\0/bname\0\0\0
   (same as when caputre-del-path option is disabled)

4. If bname argument passed to "resolve_pargfid_to_path" function
   is NULL and pargfid is ROOT, "." is returned. This is not the
   case with changelog, where bname is always passed. This is
   applicable to other consumers of "resolve_pargfid_to_path"
   routine.

NOTE:
   Changelog parser should consider the above new changes
   and should parse accordingly.

Change-Id: I040ed429b5aa7d391033fc6a540edbf07fc37827
BUG: 1214561
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/10288
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: NetBSD Build System

features/changelog: Consider only changelog on/off as changelog breakage

2015-05-02T11:37:53+00:00

Earlier, both chagelog on/off and brick restart were considered
to be changelog breakage and treated as changelog not being
continuous. As a result, new HTIME.TSTAMP file was created on
both the above cases. Now the change is made such that only
on changelog enable/disable, the changelog is considered to be
discontinuous. New HTIME.TSTAMP file is not created on brick
restart, the changelogs files are appended to last HTIME.TSTAMP
file.

Treating changelog as continuous in above scenario is important
as changelog history API will fail otherwise. It can successfully
get changes between start and end timestamps only when changelog
is continuous (Changelogs in single HTIME.TSTAMP file are treated
as continuous). Without this change, changelog history API would
fail, and it would become necessary to fallback to other mechanisms
like xsync FSCrawl in case geo-rep to detect changes in this time
window. But Xsync FSCrawl would not be applicable to other
consumers like glusterfind.

Rationale:
1. In plain distributed volume, if brick goes down, no I/O can
   happen onto the brick. Hence changelog is intact with data
   on disk.
2. In distributed replicate volume, if brick goes down, since
   self-heal traffic is captured in changelog. Eventually,
   I/O happened whend brick down is captured in changelog.

Change-Id: I2eb66efe6ee9a9228fb1fcb38d6e7696b9559d5b
BUG: 1211327
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/10222
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

changelog: Fixing buffer overrun coverity issues.

2015-04-30T15:48:11+00:00

Coverity IDs:
1214630
1214631
1214633
1234643

Change-Id: I172c4f49bf651b2324522f9e661023f73ca05339
BUG: 789278
Signed-off-by: Nandaja Varma 
Reviewed-on: http://review.gluster.org/9557
Tested-by: Gluster Build System 
Reviewed-by: Sakshi Bansal
Reviewed-by: Venky Shankar

features/changelog: RPC'fy {libgf}changelog

2015-03-19T01:22:36+00:00

This patch introduces RPC based communication between the changelog
translator and libgfchangelog. It replaces the old pathetic stream
based interaction that existed earlier (due to time constraints :-/).

Changelog, upon initialization starts a RPC server (rpcsvc) allowing
clients to invoke a probe API as a bootup mechanism to request for
event notifications. During probe, clients can choose an event
filter specifying the type(s) of events they are interested in. As
of now there is no way to change the event notification set once
the probe RPC call is made, but that is easier to implement.

The actual event notifications is done on a separate RPC session.
The client (libgfchangelog) itself starts and RPC server which the
changelog translator "connects back" during probe. Notifications
are dispatched by a bunch of threads from the server (translator)
and the client optionally orders them if ordered notifications
are requried. FOPs fill in their respective event details in a
buffer (rot-buffs to be particular) and a bunch of threads
(consumers) swap the buffers out of roatation and dispatch them
via RPC. To avoid writer starvation, then number of dispatcher
threads is one less than the number of buffer list in rot-buffs.x

libgfchangelog becomes purely callback based -- upon event
notification from the server (and re-ordering them if required)
invoke a callback routine specified by consumer(s).

A major part of the patch is also aimed at providing backward
compatibility for geo-replication, which was one of the main
consumer of the stream based API. Also, this patch does not\
"turn on" event notifications for all fops, just a bunch which
is currently in requirement. Another pain point is that the
server does not filter events before dispatching it to the
clients. That load is taken up by the client itself (although
it's done at the library layer rather than making it hard on
the callback implementor). This needs improvement and care
needs to be taken to not load the server up with expensive
filtering mechanisms.

Change-Id: Ibf60a432b68f2dfa60c6f9add2bcfd37a9c41395
BUG: 1170075
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9708
Reviewed-by: Jeff Darcy 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/changelog: Crash consistency of changelog wrt snapshot

2014-08-26T10:16:53+00:00

This patch introduces call-path fop details logging for data operations
in CHANGELOG.SNAP. This feature is enabled with barrier-enable
notification and disabled with barrier-disable notification.

Change-Id: Ib548d34203eb99cea478a6baff402e82251c73a4
BUG: 1128093
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/8533
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/changelog : historical journal consumption.

2014-05-14T12:10:15+00:00

Facilitates Glusterfs with the ability to detect file-operations
happened in past by scanning the back-end(brick-level) glusterfs
journal (changelog).

Design:
  * List of changelogs produces in one perfectly running session are
    stored in htime file which also holds necessary information about
    the session start and end time.
  * Involves fixed sized seeks to identify N'th changelog in the list.
  * Requires O(log n), (where n is number of changelogs in the list),
    time to identify the end changelog for the given start-end time
    interval.

Currently the background processing of changelogs is sub optimal. BZ
1097041 tracks the development effort.

For complete design, refer the below link:
http://lists.nongnu.org/archive/html/gluster-devel/2014-02/msg00206.html

Change-Id: I27e49f75e492e843084d0ecaf9130224d08462a0
BUG: 1091961
Signed-off-by: Ajeet Jha 
Signed-off-by: Venky Shankar 
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/6930
Reviewed-by: Kotresh HR 
Tested-by: Gluster Build System