summaryrefslogtreecommitdiffstats
path: root/xlators/features/changelog/lib
Commit message (Collapse)AuthorAgeFilesLines
* all: reduce "inline" usageJeff Darcy2015-09-011-52/+1
| | | | | | | | | | | | | | | | | | | | | | | | | There are three kinds of inline functions: plain inline, extern inline, and static inline. All three have been removed from .c files, except those in "contrib" which aren't our problem. Inlines in .h files, which are overwhelmingly "static inline" already, have generally been left alone. Over time we should be able to "lower" these into .c files, but that has to be done in a case-by-case fashion requiring more manual effort. This part was easy to do automatically without (as far as I can tell) any ill effect. In the process, several pieces of dead code were flagged by the compiler, and were removed. Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155 BUG: 1245331 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/11769 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: Porting log messages to new logging frameworkAnusha Rao2015-08-118-117/+462
| | | | | | | | | | | Change-Id: Ic7f842acca52908fd88e0796dc90b82650405b25 BUG: 1194640 Signed-off-by: Anusha Rao <anusha91rao@gmail.com> Reviewed-on: http://review.gluster.org/10532 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* build: Mac OS X build issues, no spinlock, need sys_lgetxattrKaleb S. KEITHLEY2015-07-051-1/+1
| | | | | | | | | | | | use regular locks, use our syscall wrappers in libglusterfs Change-Id: I7e0d00956366806af041b69b65d1f169aa0d2ae2 BUG: 1238793 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/11515 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* libgfchangelog: Fix example programsKotresh HR2015-07-024-2/+21
| | | | | | | | | | | | | | Incorporate the new API 'gf_changelog_init' introduced in libgfchangelog which should be called before 'gf_changelog_register'. Change-Id: I10a9dd5ff3ecc64fd61dcb6b1aa53739e0bdeedd BUG: 1238063 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/11479 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* libgfchangelog: Fix crash in gf_changelog_processKotresh HR2015-06-172-8/+14
| | | | | | | | | | | | | | | | | | | | | | | Problem: Crash observed in gf_changelog_process and gf_changelog_callback_invoker. Cause: Assignments to arguments passed to thread is done post thread creation. If the thread created gets scheduled before the assignment and access these variables, it would crash with segmentation fault. Solution: Assignments to arguments are done prior to the thread creation. Change-Id: I6afc8ccedd050cf4b50b967fef8287a0c834177b BUG: 1232666 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/11273 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: Remove inline keyword to avoid warnings (gcc v5.1.1)Anoop C S2015-06-012-3/+3
| | | | | | | | | | | | | | | | | | | | | | | When compiled with gcc5, following warnings were displayed and volume start failed: changelog-helpers.h:499:1: warning: inline function 'changelog_dispatch_event' declared but never defined changelog_dispatch_event (xlator_t *, changelog_priv_t *, changelog_event_t *); gf-changelog-journal-handler.c:692:17: warning: 'list_add_tail' is static but used in inline function 'gf_changelog_queue_journal' which is not static list_add_tail (&entry->list, &jnl_proc->entries); Fix is to remove the keyword from function prototype and definitions. Change-Id: I188b35b7ca087a94d7a48a052b05a6d845e3b74b BUG: 1226307 Signed-off-by: Anoop C S <achiraya@redhat.com> Reviewed-on: http://review.gluster.org/11004 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: Avoid creation of empty changelogsSaravanakumar Arumugam2015-05-081-7/+38
| | | | | | | | | | | | | | | | | | An empty changelog when rolled over gets unlinked and indexed with a modified path-name in htime file. The modification is "changelog" not "CHANGELOG" in basename of the empty changelog file. Change-Id: I77fd0b48b5c33c245418f5ac7a9756f08ece24d9 BUG: 1208470 Signed-off-by: Ajeet Jha <ajha@redhat.com> Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/9572 Tested-by: NetBSD Build System Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: Version support for Changelog ParserAravinda VK2015-05-061-9/+66
| | | | | | | | | | | | | | | | | | As optional feature, during unlink, full path will be recorded. Changelog Version number to be bumped up to 1.2. With this patch, parser checks the version number before parsing and handles accordingly. Change-Id: Ic1ad98259c39e417029a08e26a1d4b467817e65a BUG: 1214561 Signed-off-by: Aravinda VK <avishwan@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10166 Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* build: make contrib/uuid dependency optionalNiels de Vos2015-04-103-12/+4
| | | | | | | | | | | | | | | | | | | On Linux systems we should use the libuuid from the distribution and not bundle and statically link the contrib/uuid/ bits. libglusterfs/src/compat-uuid.h has been introduced and should become an abstraction layer for different UUID APIs. Non-Linux operating systems should implement their compatibility layer there. Once all operating systems have an implementation in compat-uuid.h, we can remove contrib/uuid/ from the repository completely. Change-Id: I345e5357644be2521685e00358bb8c83c4ea0577 BUG: 1206587 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10129 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: fix possible illegal memory access (CID 1288819)Michael Adam2015-04-081-1/+2
| | | | | | | | | | | | | | | | Coverity CID 1288819 strncpy executed with a limit equal to the target array size potentially leaves the target string not null terminated. Make sure the copied string is a valid 0 terminated string. Change-Id: Ie2d2970f37840146aa18724be3b89e93194c8160 BUG: 789278 Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-on: http://review.gluster.org/10062 Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: fix possible illegal memory access (CID 1288821)Michael Adam2015-04-051-1/+2
| | | | | | | | | | | | | | | | | Coverity CID 1288821 strncpy executed with a limit equal to the target array size potentially leaves the target string not null terminated. Make sure the copied string is a valid 0 terminated string. Change-Id: I5ecc60549864cf50e3facbb8a2d9228cd67b57b5 BUG: 789278 Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-on: http://review.gluster.org/10061 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* libgfchangelog: Use correct 'this' pointer on new thread creationKotresh HR2015-04-034-4/+12
| | | | | | | | | | | | | | | | When libgfchangelog is linked with non xlator application, it should point to 'master' xlator which is initiated separately. When ever a new thread is created, 'THIS' points to the global xlator. 'THIS' should point to corresponding xlator even then. This patch adjusts the pointer accordingly. Change-Id: I2a199bb3c73146a0329540aedcbae697a00f6f0a BUG: 1207643 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10074 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: fix possible illegal mem access (CID 1288822)Michael Adam2015-04-021-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | Coverity CID 1288822 (#1 of 2) strncpy executed with a limit equal to the target array size potentially leaves the target string not null terminated. In this case the strncpy is not needed due to the snprintf with the same target buffer which follows immediately. This patch also removes the now unneeded scratch_dir argument to gf_changelog_init_history(), which is semantically correct, since scratch_dir has previously been filled into jnl->jnl_working_dir by the caller, and this is now used to fill hist_scratch_dir. Change-Id: Ib1ed3a1058e80e34191758921b49c29030d6c9db BUG: 789278 Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-on: http://review.gluster.org/10058 Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Xlators : Fixed typosManikandan Selvaganesh2015-04-021-2/+2
| | | | | | | | | | | Change-Id: I948f85cb369206ee8ce8b8cd5e48cae9adb971c9 BUG: 1075417 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/9529 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
* features/changelog: fix possible memory illegal access (2 of 2, coverity)Michael Adam2015-04-011-1/+2
| | | | | | | | | | | | | | | | | | Coverity CID 1288822 (#2 of 2) strncpy executed with a limit equal to the target array size potentially leaves the target string not null terminated. Make sure the copied string is a valid 0 terminated string. Change-Id: If283dd6f716912f65729edf6ea26a4331a697151 BUG: 789278 Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-on: http://review.gluster.org/10059 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Poornima G <pgurusid@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/bit-rot: Implementation of bit-rot xlatorVenky Shankar2015-03-242-118/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the "Signer" -- responsible for signing files with their checksums upon last file descriptor close (last release()). The event notification facility provided by the changelog xlator is made use of. Moreover, checksums are as of now SHA256 hash of the object data and is the only available hash at this point of time. Therefore, there is no special "what hash to use" type check, although it's does not take much to add various hashing algorithms to sign objects with. Signatures are stored in extended attributes of the objects along with the the type of hashing used to calculate the signature. This makes thing future proof when other hash types are added. The signature infrastructure is provided by bitrot stub: a little piece of code that sits over the POSIX xlator providing interfaces to "get or set" objects signature and it's staleness. Since objects are signed upon receiving release() notification, pre-existing data which are "never" modified would never be signed. To counter this, an initial crawler thread is spawned The crawler scans the entire brick for objects that are unsigned or "missed" signing due to the server going offline (node reboots, crashes, etc..) and triggers an explicit sign. This would also sign objects when bit-rot is enabled for a volume and/or after upgrade. Change-Id: I1d9a98bee6cad1c39c35c53c8fb0fc4bad2bf67b BUG: 1170075 Original-Author: Raghavendra Bhat <raghavendra@redhat.com> Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/9711 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Bitrot StubVenky Shankar2015-03-241-6/+16
| | | | | | | | | | | | | Bitrot stub implements object versioning required for identifying signature freshness. More details about versioning is explained as a part of the "bitrot feature documentation" patch. Change-Id: I2ad70d9eb109ba4a12148ab8d81336afda529ad9 BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/9709 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* libgfchangelog: cleanups on disconnectionVenky Shankar2015-03-245-140/+280
| | | | | | | | | | | | | | | | | | | | | | | [WIP patch as of now, just needs a little tweak] A pending TODO in the code caused regressions to fail as bitrot daemons are spawned during volume start (equivalent to enabling bitrot by default). The problematic part that casued such failures is during brick disconnections with unsafe handling of event data structured in the code. With this patch, data structures are properly cleaned up with care taken to cleanup all accessors first. This also fixes potential memory leaks which was bluntly ignored before. Change-Id: I70ed82cb1a0fb56c85ef390007e321a97a35c5ce BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> original-author: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/9959 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* libgfchangelog: Fix faulty reference to xlator contextVenky Shankar2015-03-244-55/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: libgfchangelog initializes global xlator on library load (via constructor: _ctor) and mangles it's xlator context thereby messing with certain important members of the command structure. On receiving an RPC disconnection event, if the point-of-execution was in libgfchangelogs context, accessing ->cmd_args during RPC notify resulted in a segfault. Fix: Since the libarary needs to be able to work with processes that have a notion of an xlator (THIS in particular) and without it, care needs to be taken to allocate the global xlator when needed. Moreover, the actual fix is to use the correct xlator context in both cases. A new API is introduces when needs to be invoked by the conusmer (although this could have been done during register() call, keeping it a separate API makes thing flexible and easy). Test: The issue is observed when a brick process goes offline. This is triggered when test cases (.t's) are run in bulk, since each test essestially spawns bricks processes (on volume start) and terminates them (volume stop). Since bitrot daemon, as of now, spawns upon volume start, the issue is much observed when the volume is taken offline at the end of each test case. With this fix, running the basic and core test cases along with building the linux kernel has passed without daemon segfaults. Thanks to Johnny (rabhat@) for helping in debugging the issue (and with the fix :)). Change-Id: I8d3022bf749590b2ee816504ed9b1dfccc65559a BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/9953 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: RPC'fy {libgf}changelogVenky Shankar2015-03-1815-710/+2104
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch introduces RPC based communication between the changelog translator and libgfchangelog. It replaces the old pathetic stream based interaction that existed earlier (due to time constraints :-/). Changelog, upon initialization starts a RPC server (rpcsvc) allowing clients to invoke a probe API as a bootup mechanism to request for event notifications. During probe, clients can choose an event filter specifying the type(s) of events they are interested in. As of now there is no way to change the event notification set once the probe RPC call is made, but that is easier to implement. The actual event notifications is done on a separate RPC session. The client (libgfchangelog) itself starts and RPC server which the changelog translator "connects back" during probe. Notifications are dispatched by a bunch of threads from the server (translator) and the client optionally orders them if ordered notifications are requried. FOPs fill in their respective event details in a buffer (rot-buffs to be particular) and a bunch of threads (consumers) swap the buffers out of roatation and dispatch them via RPC. To avoid writer starvation, then number of dispatcher threads is one less than the number of buffer list in rot-buffs.x libgfchangelog becomes purely callback based -- upon event notification from the server (and re-ordering them if required) invoke a callback routine specified by consumer(s). A major part of the patch is also aimed at providing backward compatibility for geo-replication, which was one of the main consumer of the stream based API. Also, this patch does not\ "turn on" event notifications for all fops, just a bunch which is currently in requirement. Another pain point is that the server does not filter events before dispatching it to the clients. That load is taken up by the client itself (although it's done at the library layer rather than making it hard on the callback implementor). This needs improvement and care needs to be taken to not load the server up with expensive filtering mechanisms. Change-Id: Ibf60a432b68f2dfa60c6f9add2bcfd37a9c41395 BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/9708 Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* changelog: Unchecked buffer fill in gf_history_changelog_next_changeNiels de Vos2015-03-041-0/+5
| | | | | | | | | | | | | | | | | | | A gf_history_changelog_next_change() calls gf_readline() to fill a buffer without checking buffer size. The size of maxlen is not verified to be less than the lenght of buffer. This could result in the over filling of buffer of maxlen is greater than PATH_MAX. Check the size of maxlen to be less than PATH_MAX and return a fail code as needed. BUG: 1174017 Change-Id: Ic53b1a6e25af69a339bc15fb2d233dc1e457910f Reported-by: Keith Schincke <kschinck@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/9275 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* feature/changelog: Fix changelog and history scan failure.Kotresh HR2015-01-271-2/+4
| | | | | | | | | | | | | Fixes bad file descriptor issue while cleaning up scratch directory during gf_changelog_register. Change-Id: Ia6aa8d55dcc2209144b48b6583681a155d919c42 BUG: 1162057 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/9495 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: Cleanup .processing and .current directoryAravinda VK2015-01-181-0/+21
| | | | | | | | | | | | | | | | | On changelog_register cleanup .processing, .history/.processing, .current and .history/.current from the working directory. Moved glusterd_recursive_rmdir and glusterd_for_each_entry to common place(libglusterfs) and renamed as recursive_rmdir and GF_FOR_EACH_ENTRY_IN_DIR respectively BUG: 1162057 Change-Id: I1f98468a344cead039026762a805437b2f9e507b Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9082 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* changelog: correct mmap() error checkingNiels de Vos2014-11-171-8/+6
| | | | | | | | | | | | | | | Upon failure, mmap() returns MAP_FAILED, which is not equal to NULL. This small correction makes sure that the potential error gets caught and handled. BUG: 1138621 Change-Id: I0a88b5187afa7187dcaa8f7d2cb0f9bb775c929d Reported-by: Santosh Pradhan <santosh.pradhan@gmail.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/9130 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* changelog: replace MAKE_HTIME_FILE_PATH with snprintf()Niels de Vos2014-10-281-7/+1
| | | | | | | | | | | | | | | | The used once MAKE_HTIME_FILE_PATH macro uses strcpy and strcat into a fixed buffer without checking the input lengths. Recommend replacing with a snprintf. Change-Id: Ia0245096774dc84be1b937e1d5750f3634fff034 BUG: 1099645 Reported-by: Keith Schincke <kschinck@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/8977 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* build: make GLUSTERD_WORKDIR rely on localstatedirHarshavardhana2014-08-071-2/+1
| | | | | | | | | | | | | | | | | | | | | | - Break-way from '/var/lib/glusterd' hard-coded previously, instead rely on 'configure' value from 'localstatedir' - Provide 's/lib/db' as default working directory for gluster management daemon for BSD and Darwin based installations - loff_t is really off_t on Darwin - fix-off the warnings generated by clang on FreeBSD/Darwin - Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all platforms. - Define proper environment for running tests, define correct PATH and LD_LIBRARY_PATH when running tests, so that the desired version of glusterfs is used, regardless where it is installed. (Thanks to manu@netbsd.org for this additional work) Change-Id: I2339a0d9275de5939ccad3e52b535598064a35e7 BUG: 1111774 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8246 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* geo-rep/libgfchangelog: Support of symlinks while creation of working dir.Kotresh H R2014-08-041-1/+1
| | | | | | | | | | | | | In gf_changelog_register, enable symlink support while creating working directory if its not already created. Change-Id: I4b81762d04f5276a65087d2ff94f2a36dc504b58 BUG: 1125843 Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8409 Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* geo-rep/libgfchangelog: Create working dir during changelog_register if not ↵Kotresh H R2014-08-011-6/+15
| | | | | | | | | | | | | | | | | | | present. Earlier, xysnc's register was being called first, which was creating working directory before calling changelog_register. Now it is history crawl first. Hence working directory would not have been created. Create it in gf_changelog_register itself if it is not already created. Change-Id: Ief3f2b87deaf5da16c135b64be1be42e0a7647f3 BUG: 1125843 Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8399 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* changelog: variable 'writen' is changedHumble Chirammal2014-07-143-7/+7
| | | | | | | | | | | | | This patch change the variable name to 'written' in changelog, along with few other typo fixing. Change-Id: I1d3cbea5c89fc0a15233bb6e61b4fbd4bfdd3a67 BUG: 1075417 Signed-off-by: Humble Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/8267 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: Fix for history api issue during upgradeAjeet Jha2014-06-221-27/+26
| | | | | | | | | | | | | | | On failure during history changelog availability check, there was an attempt to destory attribute (of consume thread) which was not initialised. Change-Id: Ica7a98ee70eb1303303884b01ef76ab71430a351 BUG: 1111169 Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/8118 Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: skipping empty changelogs in historyAjeet Jha2014-06-101-16/+6
| | | | | | | | | | | | | Empty changelgs are unlinked and htime-update is bypassed while publishing partially written changelog. Change-Id: I1533ad431e4fbf592d2f38837fce782932d01e9c BUG: 1105466 Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/7949 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: NULL termination after memcpy should be of dest pointerHarshavardhana2014-05-272-14/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | snippet code < ..memcpy (bufff, src, len - 1); ..*(src + len) = '\0'; ---> Wrong! > Source buffer lvalue() referencing with offset style NULL termination is wrong and unnecessary when we have a destination buffer, it is the destination buffer which should look to be NULL terminated Makes it more readable and also clearly logical. < ..memcpy (bufff, src, len - 1); ..bufff[len -1] = '\0'; ---> Correct! > Change-Id: I6d7f312aaa5c541f0345649ff1ef9f193892b674 BUG: 1099986 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/7836 Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog : historical journal consumption.Venky Shankar2014-05-144-14/+873
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Facilitates Glusterfs with the ability to detect file-operations happened in past by scanning the back-end(brick-level) glusterfs journal (changelog). Design: * List of changelogs produces in one perfectly running session are stored in htime file which also holds necessary information about the session start and end time. * Involves fixed sized seeks to identify N'th changelog in the list. * Requires O(log n), (where n is number of changelogs in the list), time to identify the end changelog for the given start-end time interval. Currently the background processing of changelogs is sub optimal. BZ 1097041 tracks the development effort. For complete design, refer the below link: http://lists.nongnu.org/archive/html/gluster-devel/2014-02/msg00206.html Change-Id: I27e49f75e492e843084d0ecaf9130224d08462a0 BUG: 1091961 Signed-off-by: Ajeet Jha <ajha@redhat.com> Signed-off-by: Venky Shankar <vshankar@redhat.com> Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/6930 Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* libgfchangelog: memory accounting intializationVenky Shankar2014-05-081-0/+3
| | | | | | | | | | | Needed as memory accounting turned ON by default now. Without this, GF_{CALLOC,MALLOC} calls assert. Change-Id: Ie54a6efb61871e70d4f7c796d9c86a867051e2e0 BUG: 1094708 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/7698 Reviewed-by: Kotresh HR <khiremat@redhat.com>
* features/libgfchangelog: APIs to process history changelogs.Kotresh H R2014-04-284-2/+336
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. Create directories in following fashion for history API's usage when consumer is registered with libgfchangelog shared library through gf_changelog_register. scratch_dir/.history scratch_dir/.history/.current scratch_dir/.history/.processed scratch_dir/.history/.processing 2. Added new file 'gf-history-changelog.c' and following APIs are provided for consumers to process history changelogs. 1. gf_history_changelog_scan: Move processed history changelog file from .processing to .processed 2. gf_history_changelog_next_change: Return the next history changelog file entry. Zero means all history chanelogs are consumed. 3. gf_history_changelog_done: Scan .processing directory and generate a list of change entries. 4. gf_history_changelog_start_fresh: For a set of changelogs, start from the begining. NOTE: Though this patch provides above funcationalities. It is considered functionally full with the patch (http://review.gluster.org/#/c/6930/). Change-Id: I200780c7278e0a6c008910d93faad5858a4b3e76 Original-author: Kotresh H R <khiremat@redhat.com> Signed-off-by: Kotresh H R <khiremat@redhat.com> Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/6998 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* changelog: When connection is unsuccessful, close socket too.Raghavendra Talur2014-02-071-0/+1
| | | | | | | | | | | | | Bug fix for Coverity CID: 1124791 Change-Id: I0362d45123ebc250290f3a5231f7fb113fa41212 BUG: 789278 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/6900 Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Santosh Pradhan <spradhan@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* changelog: Restrict length of unix socket files to UNIX_PATH_MAX.Raghavendra Talur2014-02-052-2/+2
| | | | | | | | | | | | Bug fix for Coverity-CID 1124847. Change-Id: I410ef8e06cbc491b1f72535298fae5e9bc77220d BUG: 789278 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/6870 Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* build: Start using library versioning for various librariesHarshavardhana2014-01-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to libtool three individual numbers stand for CURRENT:REVISION:AGE, or C:R:A for short. The libtool script typically tacks these three numbers onto the end of the name of the .so file it creates. The formula for calculating the file numbers on Linux and Solaris is /path/to/library/<library_name>.(C - A).(A).(R) As you release new versions of your library, you will update the library's C:R:A. Although the rules for changing these version numbers can quickly become confusing, a few simple tips should help keep you on track. The libtool documentation goes into greater depth. In essence, every time you make a change to the library and release it, the C:R:A should change. A new library should start with 0:0:0. Each time you change the public interface (i.e., your installed header files), you should increment the CURRENT number. This is called your interface number. The main use of this interface number is to tag successive revisions of your API. The AGE number is how many consecutive versions of the API the current implementation supports. Thus if the CURRENT library API is the sixth published version of the interface and it is also binary compatible with the fourth and fifth versions (i.e., the last two), the C:R:A might be 6:0:2. When you break binary compatibility, you need to set AGE to 0 and of course increment CURRENT. The REVISION marks a change in the source code of the library that doesn't affect the interface-for example, a minor bug fix. Anytime you increment CURRENT, you should set REVISION back to 0. Change-Id: Id72e74c1642c804fea6f93ec109135c7c16f1810 BUG: 862082 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/5645 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: more changelog fixes.Ajeet Jha2013-12-122-17/+64
| | | | | | | | | | | | | | -> log additional records. -> include FOP number for metadata. -> prevent crash if inode is not found in a fop. Change-Id: I9edd4b71819ebd68c6a2b4150ae279c471d129da BUG: 1036536 Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/6403 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@gmail.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* core: block unused signals in created threadsAnand Avati2013-09-251-2/+2
| | | | | | | | | | | | | | | Block all signal except those which are set for explicit handling in glusterfs_signals_setup(). Since thread spawning code in libglusterfs and xlators can get called from application threads when used through libgfapi, it is necessary to do this blocking. Change-Id: Ia320f80521a83d2edcda50b9ad414583a0175281 BUG: 1011662 Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/5995 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* log: set ident to openlogBala.FA2013-08-131-1/+2
| | | | | | | | | | | | | | | | | | | | at syslog side, log message is identified by its properties like programname, pid, etc. brick/mount processes need to be identified uniquely as they are different process of gluterfsd/glusterfs. At rsyslog side, log separated by programname/app-name with pid works but bit hard to identify them in long run which process is for what brick/mount. This patch fixes by setting identity string at openlog() which sets programname/app-name as similar to old style log file prefixed by gluster, glusterd, glusterfs or glusterfsd Change-Id: Ia05068943fa67ae1663aaded1444cf84ea648db8 BUG: 928648 Signed-off-by: Bala.FA <barumuga@redhat.com> Reviewed-on: http://review.gluster.org/5541 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/changelog: changelog translatorAvra Sengupta2013-07-2210-0/+1616
This is the initial version of the Changelog Translator. What is it ----------- Goal is to capture changes performed on a GlusterFS volume. The translator needs to be loaded on the server (bricks) and captures changes in a plain text file inside a configured directory path (controlled by "changelog-dir", should be somewhere in <export>/.glusterfs/changelog by default). Changes are classified into 3 types: - Data: : TYPE-I - Metadata : TYPE-II - Entry : TYPE-III Changelog file is rolled over after a certain time interval (defauls to 60 seconds) after which a changelog is started. The thing to be noted here is that for a time interval (time slice) multiple changes for an inode are recorded only once (ie. say for 100+ writes on an inode that happens within the time slice has only a single corresponding entry in the changelog file). That way we do not bloat up the changelog and also save lots of writes. Changelog Format ----------------- TYPE-I and TYPE-II changes have the gfid on the entity on which the operation happened. TYPE-III being a entry op requires the parent gfid and the basename. Changelog format has been kept to a minimal and it's upto the consumers to do the heavy loading of figuring out deletes, renames etc.. A single changelog file records all three types of changes, with each change starting with an identifier ("D": DATA, "M": METADATA and "E": ENTRY). Option is provided for the encoding type (See TUNABLES). Consumers ---------- The only consumer as of today would be geo-replication, although backup utilities, self-heal, bit-rot detection could be possible consumers in the future. CLI ---- By default, change-logging is disabled (the translator is present in the server graph but does nothing). When enabled (via cli) each brick starts to log the changes. There are a set of tunable that can be used to change the translators behaviour: - enable/disable changelog (disabled by default) gluster volume set <volume> changelog {on|off} - set the logging directory (<brick>/.glusterfs/changelogs is the default) gluster volume set <volume> changelog-dir /path/to/dir - select encoding type (binary (default) or ascii) gluster volume set <volume> encoding {binary|ascii} - change the rollover time for the logs (60 secs by default) gluster volume set <volume> rollover-time <secs> - when secs > 0, changelog file is not open()'d with O_SYNC flag - and fsync is trigerred periodically every <secs> seconds. gluster volume set <volume> fsync-interval <secs> features/changelog: changelog consumer library (libgfchangelog) A shared library is provided for the consumer of the changelogs for easy acess via APIs. Application can link against this library and request for changelog updates. Conversion of binary logs to human-readable ascii format is also taken care by the library which keeps a copy of the changelog in application provided working directory. Change-Id: I75575fb7f1c53d2bec3dba1a329ea7bb3c628497 BUG: 847839 Original Author: Venky Shankar <vshankar@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/5127 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-05 19:09:26 +0000