| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Thanks for Markiyan Kushnir <markiyan.kushnir@gmail.com> for
reporting this
Change-Id: Ia0272e51be4ddede1e6bc188dfb892979626a7cd
BUG: 1171524
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/9252
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Raghavendra Bhat <raghavendra@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: Valid SETATTR entries are missing in changelog when more
than one metadata operation happen on same inode within
changelog roll-over time.
Cause: Metadata entries with fop num being GF_FOP_NULL are logged
in changelog which is of no use. Since slice version
checking is done for metadata entries to avoid logging of
subsequent entries of same inode falling into same
changelog, if the entry with GF_FOP_NULL is logged first,
subsequent valid ones will be missed.
Solution: Have a boundary condition to log only those fops whose fop
number falls between GF_FOP_NULL and GF_FOP_MAXVALUE.
BUG: 1159213
Change-Id: I128cad323afba7d33f48df1ee5e78cb829536211
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8964
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9028
|
|
|
|
|
|
|
|
|
|
| |
BUG: 1138952
Change-Id: I594be0d09c6af2e4a34da3e819d1ab6fd85e34c4
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Reviewed-on: http://review.gluster.org/8542
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-on: http://review.gluster.org/8647
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
when a snapshot is taken, there are chances of entry creation fops
not being recorded either in changelog or through the recursive
ancestry xtime updation by marker. This causes consumers of changelog
(primarily geo-replication as of today) to not be aware of these entries
after a snapshot is restored. This can lead to inconsistencies. This patch
is an interim workaround to barrier creates till changelog becomes completely
crash consistent.
BUG: 1138952
Change-Id: Idd5e690a05fe2c7c5d32d1541a0d9b5132881ea7
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/8517
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: ajeet jha <ajha@redhat.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8646
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes changelog capturing internal FOPs in a cascaded
setup, where the intermediate master would record internal FOPs
(generated by DHT on link()/rename()). This is due to I/O happening
on the intermediate slave on geo-replication's auxillary mount with
client-pid -1. Currently, the internal FOP capturing logic depends
on client pid being non-negative and the presence of a special key
in dictionary. Due to this, internal FOPs on an inter-mediate master
would be recorded in the changelog. Checking client-pid being
non-negative was introduced to capture AFR self-heal traffic in
changelog, thereby breaking cascading setups. By coincidence,
AFR self-heal daemon uses -1 as frame->root->pid thereby making
is hard to differentiate b/w geo-rep's auxillary mount and self-heal
daemon.
BUG: 1138952
Change-Id: Ia08a2cfa3b02bb785f343794f5b2695d44398c4c
Original-Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8347
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/8638
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch introduces call-path fop details logging for data operations
in CHANGELOG.SNAP. This feature is enabled with barrier-enable
notification and disabled with barrier-disable notification.
BUG: 1138952
Change-Id: Ic418dd70b0a0b369202c5b79a6f7f96512821065
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Reviewed-on: http://review.gluster.org/8533
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/8648
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport from master branch - http://review.gluster.org/#/c/8246/
- Break-way from '/var/lib/glusterd' hard-coded previously,
instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
platforms.
- Define proper environment for running tests, define correct PATH
and LD_LIBRARY_PATH when running tests, so that the desired version
of glusterfs is used, regardless where it is installed.
(Thanks to manu@netbsd.org for this additional work)
Change-Id: I06e684ac4c26d1e74c9daf76753403ad15f79276
BUG: 1130308
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/8486
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch change the variable name to 'written' in changelog, along
with few other typo fixing.
Change-Id: I1d3cbea5c89fc0a15233bb6e61b4fbd4bfdd3a67
BUG: 1075417
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/8267
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
helper threads (fsync, rollover) wake up periodically and perform
their respective operation under a lock (crt->lock). These threads
are also subjected to cancellation under some circumstance such as
disabling changelog. This is inherently dangerous when funtions
which are cancellation points for pthread_cancel(3) are used
in the locked region.
Consider this
pthread_mutex_lock(&mutex);
{
/* ... */
ret = fsync (fd); <-- cancellation point
/* ... */
}
pthread_mutex_unlock(&mutex);
A pthread_cancel(3) by another thread just before fsync(3) but
after pthread_mutex_lock(3) would result in the thread getting
cancelled when fsync(3) is invoked, thereby never unlocking the
mutex. Moreover, in case of changelog translator, the locked
region (under crt->lock in changelog-rt.c) is also the code
path for fop changelog updation. Therefore, unlocking the
mutex in thread cleanup handler (pthread_cleanup_pop(3)) might
prematurely release the mutex during fop updation path.
This patch fixes such problems existing in fsync and rollover
threads. Fix is to enter the locked region with cancellation
disabled and enable it after mutex unlock. Also, test for a
cancellation request early on in case none of the functions
are cancellation points.
Change-Id: I1795627a12827609c1da659d07fc1457ffa033de
BUG: 1110917
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/8106
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
A few changelog files are missing at the backend
during snapshot with changelog enabled.
Cause:
Race between actual rollover and explicit rollover.
Changelog rollover can happen either due to actual
or the explict rollover due to snapshot. Actual
rollover is controlled by tuneable called rollover-time.
The minimum granularity for rollover-time is 1 second
Explicit rollover is asynchronous in nature and happens
during snapshot.
Basically, rollover renames the current CHANGELOG file
to CHANGELOG.TIMESTAMP after rollover-time. Let's assume,
at time 't1', actual and explicit rollover raced against
each other and actual rollover won the race renaming the
CHANGELOG file to CHANGELOG.t1 and opens a new
CHANGELOG file. An immediate explicit rollover at time
't1' happened with in the same second to rename
CHANGELOG file to CHANGELOG.t1 resulting in purging the
earlier CHANGELOG.t1 file created by actual rollover.
Solution:
Adding a delay of 1 sec guarantees unique CHANGELOG.TIMESTAMP
during explicit rollover.
Thanks Venky, for the all the help in root causing the issue.
Change-Id: I8958824e107e16f61be9f09a11d95f8645ecf34d
BUG: 1109692
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8196
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: Geo-rep fails to sync some files to slave as the
changelog entries are missing for those files.
Cause: Fops happened when the active brick is down and
self- healed later when it came up.
Solution: Capture self-heal fops as well in changelog so
those entries are not missed.
Change-Id: Ibc288779421b5156dd1695e529aba0b602a530e0
BUG: 1109692
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8070
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I37de39aec341fbde9516aa8f8c963b70bd808bac
BUG: 1108034
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8035
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed in this patch:
[glusterfs/extras/geo-rep/gsync-sync-gfid.c:105]: (error) Resource leak: fp
[glusterfs/libglusterfs/src/xlator.c:651]: (error) Uninitialized variable: gfid
[glusterfs/libglusterfs/src/xlator.c:652]: (error) Uninitialized variable: gfid
[glusterfs/xlators/cluster/ha/src/ha.c:2699]: (error) Possible null pointer dereference: priv
[glusterfs/xlators/features/changelog/src/changelog.c:1464]: (error) Possible null pointer dereference: priv
[glusterfs/xlators/mgmt/glusterd/src/glusterd-mgmt-handler.c:865]: (error) Possible null pointer dereference: ctx
[glusterfs/xlators/mgmt/glusterd/src/glusterd-mgmt-handler.c:194]: (error) Possible null pointer dereference: ctx
[glusterfs/xlators/mgmt/glusterd/src/glusterd-syncop.c:1408]: (error) Possible null pointer dereference: this
[glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:7002]: (error) Possible null pointer dereference: path_tokens
Fixed in 3.4 and 3.5 branch (http://review.gluster.org/#/c/7583/ ,
http://review.gluster.org/#/c/7605/ will be backported in a separate patch)
[glusterfs/xlators/mount/fuse/src/fuse-bridge.c:4688]: (error) Uninitialized variable: finh
[glusterfs/xlators/mount/fuse/src/fuse-bridge.c:3081]: (error) Possible null pointer dereference: state
[glusterfs/xlators/cluster/dht/src/dht-rebalance.c:1719]: (error) Possible null pointer dereference: ctx
[glusterfs/xlators/cluster/stripe/src/stripe.c:4940]: (error) Possible null pointer dereference: local
[glusterfs/xlators/mgmt/glusterd/src/glusterd-replace-brick.c:915]: (error) Resource leak: file
[glusterfs/xlators/mgmt/glusterd/src/glusterd-replace-brick.c:999]: (error) Resource leak: file
[glusterfs/xlators/mgmt/glusterd/src/glusterd-sm.c:248]: (error) Possible null pointer dereference: new_ev_ctx
[glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:5297]: (error) Possible null pointer dereference: this
[glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:6273]: (error) Possible null pointer dereference: this
[glusterfs/xlators/performance/quick-read/src/quick-read.c:586]: (error) Possible null pointer dereference: iobuf
[glusterfs/xlators/nfs/server/src/nfs-common.c:89]: (error) Dangerous usage of 'volname' (strncpy doesn't always null-terminate it).
False positives
[glusterfs/geo-replication/src/gsyncd.c:99]: (error) Memory leak: str
[glusterfs/geo-replication/src/gsyncd.c:395]: (error) Memory leak: argv
[glusterfs/xlators/nfs/server/src/nlm4.c:1199]: (error) Possible null pointer dereference: fde
[glusterfs/xlators/mgmt/glusterd/src/glusterd-geo-rep.c:1659]: (error) Possible null pointer dereference: command
[glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:7001]: (error) Possible null pointer dereference: path_tokens
Insignificant/Don't care
[glusterfs/contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 2) requires 'long *' but the argument type is 'unsigned long *'.
[glusterfs/contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 3) requires 'long *' but the argument type is 'unsigned long *'.
[glusterfs/extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments.
[glusterfs/xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr
[glusterfs/xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr
[glusterfs/xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr
Change-Id: I7696ed1a2a9553b79f9714e10210a8d563a5abd8
BUG: 1091677
Signed-off-by: Lalatendu Mohanty <lmohanty@redhat.com>
Reviewed-on: http://review.gluster.org/7693
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Empty changelgs are unlinked and htime-update is bypassed
while publishing partially written changelog.
Change-Id: I1533ad431e4fbf592d2f38837fce782932d01e9c
BUG: 1105466
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Reviewed-on: http://review.gluster.org/7949
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch introduces configurable 'changelog-barrier-timeout'
in changelog. The default changelog-barrier-timeout is set to
120 sec which is same as barrier-timeout of barrier translator.
On this timeout, all the barriered fops are unbarriered. It is
safe to have this internal timeout for changelog barrier to
prevent it from indefinite barrierring of fops if barrier
disable notification does not come from snapshot for some
reason.
Change-Id: Iad07807935f88222cdefcd46312b4b6579186dce
BUG: 1094226
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/7669
Reviewed-by: Kaushal M <kaushal@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Facilitates Glusterfs with the ability to detect file-operations
happened in past by scanning the back-end(brick-level) glusterfs
journal (changelog).
Design:
* List of changelogs produces in one perfectly running session are
stored in htime file which also holds necessary information about
the session start and end time.
* Involves fixed sized seeks to identify N'th changelog in the list.
* Requires O(log n), (where n is number of changelogs in the list),
time to identify the end changelog for the given start-end time
interval.
Currently the background processing of changelogs is sub optimal. BZ
1097041 tracks the development effort.
For complete design, refer the below link:
http://lists.nongnu.org/archive/html/gluster-devel/2014-02/msg00206.html
Change-Id: I27e49f75e492e843084d0ecaf9130224d08462a0
BUG: 1091961
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Reviewed-on: http://review.gluster.org/6930
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- C99 standard dictates that `inline` should be used only to hint
function implementation not as a definition
- http://clang.llvm.org/compatibility.html#inline
This also leads to run time issues as 'undefined symbols'
- Remember to use adding explicit headers when using an external
function
Change-Id: I66a9030da7ac38ded6e0b6a977182be977382fa2
BUG: 1089172
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/7655
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix warning message introduced in changelog
draining patch. It is not caught in gcc 4.4,
caught in gcc 4.8!!
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Change-Id: I01a8fda2102a409dd6e935805d4d48b41b615fe8
BUG: 1091817
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/7636
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Changelog barriers unlink, rename, rmdir fops on barrier 'on'
notification from glusterfsd mgmt layer and unbarriers the
same on barrier 'off' notification during snapshot.
Please see the following link for more details.
http://www.gluster.org/community/documentation/index.php/Changelog_Design_changes_for_snapshot
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Change-Id: Iea9c62fafc86242f9404e03679b1941aa9c88c9a
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/7415
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Varun Shastry <vshastry@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is required for Geo-rep to work with snapshots.
Following things are done in this patch.
1. Draining of in-transit fops during changelog rollover.
2. Explicit rollover of changelog when snapshot barrier
notification comes. During this, intransit fops are
drained and changelog is rolled over.
For more details on the purpose of the patch. Please
visit following link.
http://www.gluster.org/community/documentation/index.php/Changelog_Design_changes_for_snapshot
Change-Id: I22690131e19d3027f6d8957178bdc3431b9062f6
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/7216
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Varun Shastry <vshastry@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will change the following default configurables.
1. rollover-time: from 60 to 15.
2. fsync-interval: from 0 to 5.
Change-Id: I9c8db01376967c4f19547ec87f54833f8b139d29
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/7545
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
git@forge.gluster.org:~schafdog/glusterfs-core/osx-glusterfs
Working functionality on MacOSX
- GlusterD (management daemon)
- GlusterCLI (management cli)
- GlusterFS FUSE (using OSXFUSE)
- GlusterNFS (without NLM - issues with rpc.statd)
Change-Id: I20193d3f8904388e47344e523b3787dbeab044ac
BUG: 1089172
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Signed-off-by: Dennis Schafroth <dennis@schafroth.com>
Tested-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Dennis Schafroth <dennis@schafroth.com>
Reviewed-on: http://review.gluster.org/7503
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There has been a misspelled option in the Makefile.am files. The option
is called -avoid-version, and not -avoidversion.
It is not trivial to provide a test-case for this. One way would be to
check generated RPMs with a command like this (output should be empty):
$ rpm -qlp *.rpm | grep -E '/xlator/.+.so.0'
Change-Id: I2a6cc557eada4d098b73af5a254f8c75707543da
BUG: 1078365
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/7299
Reviewed-by: Lalatendu Mohanty <lmohanty@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix for Coverity-CID 1124847.
Change-Id: I410ef8e06cbc491b1f72535298fae5e9bc77220d
BUG: 789278
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/6870
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
-> log additional records.
-> include FOP number for metadata.
-> prevent crash if inode is not found in a fop.
Change-Id: I9edd4b71819ebd68c6a2b4150ae279c471d129da
BUG: 1036536
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Reviewed-on: http://review.gluster.org/6403
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
change in encoding method of changelog was critical section for
"fop dispatch thread", "roll-over thread" and "reconfigure dispatch thread".
In this patch the "encoding-method" is changed by the reconfigure dispatch thread
lazily during handle_change, which solves the concurrency among the racing
threads.
BUG: 1002940
Change-Id: I78c3e8887efa46d0fcc60755cdf4243031cfa3eb
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Reviewed-on: http://review.gluster.org/5844
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Block all signal except those which are set for explicit handling
in glusterfs_signals_setup(). Since thread spawning code in
libglusterfs and xlators can get called from application threads
when used through libgfapi, it is necessary to do this blocking.
Change-Id: Ia320f80521a83d2edcda50b9ad414583a0175281
BUG: 1011662
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5995
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch extends the persistent instrumentation work done by
Aravinda (@avishwa), by introducing a handfull of instrumentation
variables for crawl. These variables are "pulled up" by glusterd
in the event of a geo-replication status cli command and looks
something like below:
"Uptime=00:21:10;FilesSyned=2982;FilesPending=0;BytesPending=0;DeletesPending=0;"
"FilesPending", "BytesPending" and "DeletesPending" are short-lived
variables that are non-zero when a changelog is being processes (ie.
when an active sync in ongoing). After a successfull changelog process
"FilesPending" is summed up into "FilesSynced". The three short-lived
variabled are then reset to zero and the data is persisted
Additionally this patch also reverts some of the changes made for
BZ #986929 (those were not needed).
Change-Id: I948f1a0884ca71bc5e5bcfdc017d16c8c54fc30b
BUG: 990420
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5441
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A volume would fail to start if any one of the bricks fails staging or
fails to start, even with the 'force' option. With this patch, when the
'force' option is given for a volume start, glusterd will continue and
start other bricks even if one fails staging or starting.
Also did a small fix in changelog, to prevent it crashing when it fails
to init.
Change-Id: I7efbd9ab13d12d69b0335ae54143fa17586f8f98
BUG: 994375
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5510
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Other enhancements being:
* ignore fops made by rebalance
* ignore internally triggered fops
BUG: 987734
Change-Id: I7dd164ae3c209fdb8ec43a27e67b8846f937c93b
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5380
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
This is the initial version of the Changelog Translator.
What is it
-----------
Goal is to capture changes performed on a GlusterFS volume.
The translator needs to be loaded on the server (bricks) and
captures changes in a plain text file inside a configured
directory path (controlled by "changelog-dir", should be
somewhere in <export>/.glusterfs/changelog by default).
Changes are classified into 3 types:
- Data: : TYPE-I
- Metadata : TYPE-II
- Entry : TYPE-III
Changelog file is rolled over after a certain time interval
(defauls to 60 seconds) after which a changelog is started.
The thing to be noted here is that for a time interval
(time slice) multiple changes for an inode are recorded only
once (ie. say for 100+ writes on an inode that happens within
the time slice has only a single corresponding entry in the
changelog file). That way we do not bloat up the changelog
and also save lots of writes.
Changelog Format
-----------------
TYPE-I and TYPE-II changes have the gfid on the entity on
which the operation happened. TYPE-III being a entry op
requires the parent gfid and the basename. Changelog format
has been kept to a minimal and it's upto the consumers to
do the heavy loading of figuring out deletes, renames etc..
A single changelog file records all three types of changes,
with each change starting with an identifier ("D": DATA,
"M": METADATA and "E": ENTRY). Option is provided for the
encoding type (See TUNABLES).
Consumers
----------
The only consumer as of today would be geo-replication, although
backup utilities, self-heal, bit-rot detection could be possible
consumers in the future.
CLI
----
By default, change-logging is disabled (the translator is present
in the server graph but does nothing). When enabled (via cli) each
brick starts to log the changes. There are a set of tunable that
can be used to change the translators behaviour:
- enable/disable changelog (disabled by default)
gluster volume set <volume> changelog {on|off}
- set the logging directory (<brick>/.glusterfs/changelogs is the
default)
gluster volume set <volume> changelog-dir /path/to/dir
- select encoding type (binary (default) or ascii)
gluster volume set <volume> encoding {binary|ascii}
- change the rollover time for the logs (60 secs by default)
gluster volume set <volume> rollover-time <secs>
- when secs > 0, changelog file is not open()'d with O_SYNC flag
- and fsync is trigerred periodically every <secs> seconds.
gluster volume set <volume> fsync-interval <secs>
features/changelog: changelog consumer library (libgfchangelog)
A shared library is provided for the consumer of the changelogs
for easy acess via APIs. Application can link against this library
and request for changelog updates. Conversion of binary logs to
human-readable ascii format is also taken care by the library which
keeps a copy of the changelog in application provided working
directory.
Change-Id: I75575fb7f1c53d2bec3dba1a329ea7bb3c628497
BUG: 847839
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/5127
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|