| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
changelog-helpers.c:1911:17: warning: Size argument is greater than the free
space in the destination buffer strncat (result, pre_dir_name, PATH_MAX);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
changelog-helpers.c:1919:17: warning: Size argument is greater than the free
space in the destination buffer strncat (result, bname, PATH_MAX);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Change-Id: I60ca7fe762f07cb72fe7b69f0253835becaff7b9
BUG: 1222238
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-on: http://review.gluster.org/10802
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
use regular locks, use our syscall wrappers in libglusterfs
Change-Id: I7e0d00956366806af041b69b65d1f169aa0d2ae2
BUG: 1238793
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/11515
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Incorporate the new API 'gf_changelog_init'
introduced in libgfchangelog which should
be called before 'gf_changelog_register'.
Change-Id: I10a9dd5ff3ecc64fd61dcb6b1aa53739e0bdeedd
BUG: 1238063
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/11479
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Directory renames are being ignored as special renames. Special
renames can happen only on files. Hence always log directory
rename operations in changelog.
Change-Id: I4fbdb3e02e634a39a8846fb2f7a4c6cc2ba74400
BUG: 1234286
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/11356
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch uses "cleanup, v1" infrastrcuture to cleanup scrubber
(data structures, threads, timers, etc..) on brick disconnection.
Signer is not cleaned up yet: probably would be done as part of
another patch.
Change-Id: I78a92b8a7f02b2f39078aa9a5a6b101fc499fd70
BUG: 1231619
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/11148
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Crash observed in gf_changelog_process and
gf_changelog_callback_invoker.
Cause:
Assignments to arguments passed to thread is done
post thread creation. If the thread created gets
scheduled before the assignment and access these
variables, it would crash with segmentation fault.
Solution:
Assignments to arguments are done prior to the thread
creation.
Change-Id: I6afc8ccedd050cf4b50b967fef8287a0c834177b
BUG: 1232666
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/11273
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
HTIME_KEY marks the last changelog rolled over. The xattr is
maintained on .glusterfs/changelog/htime/HTIME.TSTAMP file.
On every rollover of the changelog file, the xattr is updated.
It is being updated with XATTR_REPLACE flag as xattr gets
created during changelog enable. But it is once found that
the xattrs on the file is cleared and is not reproduced later
on. This patch protects that case, if it happens by setting
xattr without XATTR_REPLACE flag in failure case.
The reason behind doing this in failure case is not to mask
the actual cause of xattrs getting cleared. This provides
the log message if the original issue still exists but the
consequential effects are fixed.
Also changed the log messages to depict the events happened
during changelog enable.
Change-Id: I699ed09a03667fd823d01d65c9c360fa7bc0e455
BUG: 1230015
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/11150
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
When a file is renamed and the (renamed)file's Hashing
falls into a different brick, DHT creates a special file(linkto file)
in the brick(Hashed subvolume) and carries out setattr operation
on that file.
Currently, Changelog records this(setattr) operation in Hashed
subvolume. glusterfind in turn records this operation
as MODIFY operation.
So, there is a NEW entry in Cached subvolume and MODIFY entry
in Hashed subvolume for the same file.
Solution:
Avoid logging setattr operation carried out, by
marking the operation as internal fop using xdata.
In changelog translator, check whether setattr is set
as internal fop and skip accordingly.
Change-Id: I21b09afb5a638b88a4ccb822442216680b7b74fd
BUG: 1230007
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: http://review.gluster.org/11137
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When compiled with gcc5, following warnings were displayed
and volume start failed:
changelog-helpers.h:499:1: warning: inline function 'changelog_dispatch_event'
declared but never defined
changelog_dispatch_event (xlator_t *, changelog_priv_t *, changelog_event_t *);
gf-changelog-journal-handler.c:692:17: warning: 'list_add_tail' is static but
used in inline function 'gf_changelog_queue_journal' which is not static
list_add_tail (&entry->list, &jnl_proc->entries);
Fix is to remove the keyword from function prototype and
definitions.
Change-Id: I188b35b7ca087a94d7a48a052b05a6d845e3b74b
BUG: 1226307
Signed-off-by: Anoop C S <achiraya@redhat.com>
Reviewed-on: http://review.gluster.org/11004
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During snapshot, changelog barrier is enabled and a
explicit rollover of changelog is initiated. During
rollover of changelog, if any error or changelog is
empty, the notification was not sent to reconfigure
and hence snapshot was failing because of timeout.
This patch addresses it by sending notification
irrespective of failures and sends error if any
back to barrier.
Change-Id: I898af624b44555281a9e43c69066077e0e121c17
BUG: 1225542
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10951
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).
When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.
BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Background:
Glusterfs changelogs are stored in each brick, which records the changes
happened in that brick. Georep will run in all the nodes of master and processes
changelogs "independently". Processing changelogs is in brick level, but
all the fops will be replayed on "slave mount" point.
Problem:
With a DHT volume, in changelog "internal fops" are NOT recorded.
For Rename case, Rename is recorded in "hashed" brick changelog.
(DHT's internal fops like creating linkto file, unlink is NOT recorded).
This lead us to inconsistent rename operations.
For example,
Distribute volume created with Two bricks B1, B2.
//Consider master volume mounted @ /mnt/master and following operations
executed:
cd /mnt/master
touch f1 // f1 falls on B1 Hash
mv f1 f2 // f2 falls on B2 Hash
// Here, Changelogs are recorded as below:
@B1
CREATE f1
@B2
RENAME f1 f2
Here, race exists between Brick B1 and B2, say B2 will get executed first.
Source file f1 itself is "NOT PRESENT", so it will go ahead and create
f2 (Current implementation).
We have this problem When rename falls in another brick and
file is unlinked in Master.
Similar kind of issue exists in following case too(multiple rename):
CREATE f1
RENAME f1 f2
RENAME f2 f1
Solution:
Instead of carrying out "changelogging" at "HASHED volume",
carry out at the "CACHED volume".
This way we have rename operations carried out where actual files are present.
So,Changelog recorded as :
@B1
CREATE f1
RENAME f1 f2
Note:
This patch is dependent on dht changes from this patch.
http://review.gluster.org/10410/
changelog related changes are separated out for review.
In changelog, xdata passed from DHT is considered as :
1. In case of unlink (internal operation as part of rename), xdata value
is set , it is considered as RENAME and recorded accordingly.
2. In case of rename (Hash and Cache different), xdata value is NOT
set, recording rename operation is SKIPPED.
BUG: 1141379
Change-Id: Ifca675e6d4ef8c4e3b7ef4a7ec85de8b3a38dc08
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: http://review.gluster.org/10220
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ie7e7c6028c7bffe47e60a2e93827e0e8767a3d66
BUG: 1219894
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10687
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
|
|
|
|
|
|
|
|
|
|
|
|
| |
This issue introduced due to manual rebase.
Change-Id: I0589f4a0a1270190340f419b8022d6483bcf853d
Signed-off-by: Aravinda VK <avishwan@redhat.com>
BUG: 1219479
Reviewed-on: http://review.gluster.org/10685
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An empty changelog when rolled over gets unlinked and indexed with
a modified path-name in htime file. The modification is "changelog"
not "CHANGELOG" in basename of the empty changelog file.
Change-Id: I77fd0b48b5c33c245418f5ac7a9756f08ece24d9
BUG: 1208470
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: http://review.gluster.org/9572
Tested-by: NetBSD Build System
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As optional feature, during unlink, full path will be recorded.
Changelog Version number to be bumped up to 1.2.
With this patch, parser checks the version number before parsing
and handles accordingly.
Change-Id: Ic1ad98259c39e417029a08e26a1d4b467817e65a
BUG: 1214561
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10166
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Tested-by: NetBSD Build System
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PROBLEM:
There is no way to get the path of deleted file if we
have gfid from changelog since the file is already deleted.
SOLUTION:
Do a recursive readlink on parent gfid in backend .glusterfs
path to get the complete path in I/O callpath in changelog
translator and capture it in callback.
The path captured is relative from the brick root. The field
separator used is '\0'.
e.g.,
......\0<pgfid>/bname\0<relative-path>\0<next-record>
ADDITIONAL REQUIRED CHANGES:
1. The changelog translator option called "changelog.capture-del-path"
is introduced to enable or disable the capturing of deleted entry
path.
e.g.,
gluster vol set <vol-name> changelog.capture-del-path on/off
If capture-del-path is disabled, '\0' is captured instead of
relative path.
e.g.,
......\0<pgfid>/bname\0\0\0<next-record>
2. The minor number in the version of changelog is bumped up from v1.1
to v1.2.
3. If recursive readlink is failed for some reason, it will capture
\0 in place of <relative path>.
e.g.,
......\0<pgfid>/bname\0\0\0<next-record>
(same as when caputre-del-path option is disabled)
4. If bname argument passed to "resolve_pargfid_to_path" function
is NULL and pargfid is ROOT, "." is returned. This is not the
case with changelog, where bname is always passed. This is
applicable to other consumers of "resolve_pargfid_to_path"
routine.
NOTE:
Changelog parser should consider the above new changes
and should parse accordingly.
Change-Id: I040ed429b5aa7d391033fc6a540edbf07fc37827
BUG: 1214561
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10288
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: NetBSD Build System
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The structure 'rpcsvc_state', which maintains rpc server
state had no separate pointer to track the translator.
It was using the mydata pointer itself. So callers were
forced to send xlator pointer as mydata which is opaque
(void pointer) by function prototype.
'rpcsvc_register_init' is setting svc->mydata with xlator
pointer. 'rpcsvc_register_notify' is overwriting svc->mydata
with mydata pointer. And rpc interprets svc->mydata as
xlator pointer internally. If someone passes non xlator
structure pointer to rpcsvc_register_notify as libgfchangelog
currently does, it might corrupt mydata. So interpreting opaque
mydata as xlator pointer is incorrect as it is caller's choice
to send mydata as any type of data to 'rpcsvc_register_notify'.
Maintaining two different pointers in 'rpcsvc_state' for xlator
and mydata solves the issue.
Change-Id: I7874933fefc68f3fe01d44f92016a8e4e9768378
BUG: 1215161
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10366
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Earlier, both chagelog on/off and brick restart were considered
to be changelog breakage and treated as changelog not being
continuous. As a result, new HTIME.TSTAMP file was created on
both the above cases. Now the change is made such that only
on changelog enable/disable, the changelog is considered to be
discontinuous. New HTIME.TSTAMP file is not created on brick
restart, the changelogs files are appended to last HTIME.TSTAMP
file.
Treating changelog as continuous in above scenario is important
as changelog history API will fail otherwise. It can successfully
get changes between start and end timestamps only when changelog
is continuous (Changelogs in single HTIME.TSTAMP file are treated
as continuous). Without this change, changelog history API would
fail, and it would become necessary to fallback to other mechanisms
like xsync FSCrawl in case geo-rep to detect changes in this time
window. But Xsync FSCrawl would not be applicable to other
consumers like glusterfind.
Rationale:
1. In plain distributed volume, if brick goes down, no I/O can
happen onto the brick. Hence changelog is intact with data
on disk.
2. In distributed replicate volume, if brick goes down, since
self-heal traffic is captured in changelog. Eventually,
I/O happened whend brick down is captured in changelog.
Change-Id: I2eb66efe6ee9a9228fb1fcb38d6e7696b9559d5b
BUG: 1211327
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10222
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity IDs:
1214630
1214631
1214633
1234643
Change-Id: I172c4f49bf651b2324522f9e661023f73ca05339
BUG: 789278
Signed-off-by: Nandaja Varma <nvarma@redhat.com>
Reviewed-on: http://review.gluster.org/9557
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Sakshi Bansal
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Changelog xlator was capturing bitrot-stub's fsetxattr sent
for versioning. Since it was using the same frame as of the
create fop, there was inconsistency in fop number and gfid
of capturing metadata. So fix is to mark fsetxattr used for
versioning as internal and add internal fop filter in
changelog_fsetxattr.
Change-Id: I51ff468995139838b22bf293a59a0713a92ee7a5
BUG: 1170075
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10148
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On Linux systems we should use the libuuid from the distribution and not
bundle and statically link the contrib/uuid/ bits.
libglusterfs/src/compat-uuid.h has been introduced and should become an
abstraction layer for different UUID APIs. Non-Linux operating systems
should implement their compatibility layer there.
Once all operating systems have an implementation in compat-uuid.h, we
can remove contrib/uuid/ from the repository completely.
Change-Id: I345e5357644be2521685e00358bb8c83c4ea0577
BUG: 1206587
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10129
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity CID 1288819
strncpy executed with a limit equal to the target array
size potentially leaves the target string not null terminated.
Make sure the copied string is a valid 0 terminated string.
Change-Id: Ie2d2970f37840146aa18724be3b89e93194c8160
BUG: 789278
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-on: http://review.gluster.org/10062
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity CID 1288821
strncpy executed with a limit equal to the target array
size potentially leaves the target string not null terminated.
Make sure the copied string is a valid 0 terminated string.
Change-Id: I5ecc60549864cf50e3facbb8a2d9228cd67b57b5
BUG: 789278
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-on: http://review.gluster.org/10061
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.
Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.
A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.
BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When libgfchangelog is linked with non xlator application,
it should point to 'master' xlator which is initiated separately.
When ever a new thread is created, 'THIS' points to the global
xlator. 'THIS' should point to corresponding xlator even then.
This patch adjusts the pointer accordingly.
Change-Id: I2a199bb3c73146a0329540aedcbae697a00f6f0a
BUG: 1207643
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10074
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity CID 1288822 (#1 of 2)
strncpy executed with a limit equal to the target array
size potentially leaves the target string not null terminated.
In this case the strncpy is not needed due to the snprintf
with the same target buffer which follows immediately.
This patch also removes the now unneeded scratch_dir
argument to gf_changelog_init_history(), which is semantically
correct, since scratch_dir has previously been filled into
jnl->jnl_working_dir by the caller, and this is now used to
fill hist_scratch_dir.
Change-Id: Ib1ed3a1058e80e34191758921b49c29030d6c9db
BUG: 789278
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-on: http://review.gluster.org/10058
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I948f85cb369206ee8ce8b8cd5e48cae9adb971c9
BUG: 1075417
Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Reviewed-on: http://review.gluster.org/9529
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity CID 1288822 (#2 of 2)
strncpy executed with a limit equal to the target array
size potentially leaves the target string not null terminated.
Make sure the copied string is a valid 0 terminated string.
Change-Id: If283dd6f716912f65729edf6ea26a4331a697151
BUG: 789278
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-on: http://review.gluster.org/10059
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Poornima G <pgurusid@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CID: 1124481
The goto label 'out' was trying to
dereference a pointer which might be
null, hence a check on not null is
made before the use of it.
Change-Id: I813e523207b8218a7a28c2b3ac36a8e472e36c89
BUG: 789278
Signed-off-by: arao <arao@redhat.com>
Reviewed-on: http://review.gluster.org/9617
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The changelog xlator was modified to use a poller thread, which uses
the same event pool as the xlator stack. Unfortunately, such threaded
usage of event pool is not supported by event-poll code, only
event-epoll supports it. As a result, platforms such as NetBSD that
lack epoll support got broken.
The fix is to remove the poller thread, which does not cause any
functionnality loss because the xlator stack event poll is
functionnal. That lets NetBSD pass AFR tests again.
BUG: 1129939
Change-Id: I3d73cf58e2ed8d92d9e0191f7abda3c37dea4159
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/10030
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is the "Signer" -- responsible for signing files with their
checksums upon last file descriptor close (last release()).
The event notification facility provided by the changelog xlator
is made use of.
Moreover, checksums are as of now SHA256 hash of the object data
and is the only available hash at this point of time. Therefore,
there is no special "what hash to use" type check, although it's
does not take much to add various hashing algorithms to sign
objects with. Signatures are stored in extended attributes of the
objects along with the the type of hashing used to calculate the
signature. This makes thing future proof when other hash types
are added. The signature infrastructure is provided by bitrot
stub: a little piece of code that sits over the POSIX xlator
providing interfaces to "get or set" objects signature and it's
staleness.
Since objects are signed upon receiving release() notification,
pre-existing data which are "never" modified would never be
signed. To counter this, an initial crawler thread is spawned
The crawler scans the entire brick for objects that are unsigned
or "missed" signing due to the server going offline (node reboots,
crashes, etc..) and triggers an explicit sign. This would also
sign objects when bit-rot is enabled for a volume and/or after
upgrade.
Change-Id: I1d9a98bee6cad1c39c35c53c8fb0fc4bad2bf67b
BUG: 1170075
Original-Author: Raghavendra Bhat <raghavendra@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9711
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bitrot stub implements object versioning required for identifying
signature freshness. More details about versioning is explained
as a part of the "bitrot feature documentation" patch.
Change-Id: I2ad70d9eb109ba4a12148ab8d81336afda529ad9
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9709
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Translators which wish to send event notifications can send
"down" an IPC FOP with op_type as GF_IPC_TARGET_CHANGELOG
and xdata carrying event structures (changelog_event_t).
Change-Id: I0e5f8c9170161c186f0e58d07105813e34e18786
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9775
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
[WIP patch as of now, just needs a little tweak]
A pending TODO in the code caused regressions to fail as
bitrot daemons are spawned during volume start (equivalent
to enabling bitrot by default). The problematic part that
casued such failures is during brick disconnections with
unsafe handling of event data structured in the code.
With this patch, data structures are properly cleaned up
with care taken to cleanup all accessors first. This also
fixes potential memory leaks which was bluntly ignored
before.
Change-Id: I70ed82cb1a0fb56c85ef390007e321a97a35c5ce
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
original-author: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9959
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
libgfchangelog initializes global xlator on library load (via
constructor: _ctor) and mangles it's xlator context thereby
messing with certain important members of the command structure.
On receiving an RPC disconnection event, if the point-of-execution
was in libgfchangelogs context, accessing ->cmd_args during RPC
notify resulted in a segfault.
Fix:
Since the libarary needs to be able to work with processes that
have a notion of an xlator (THIS in particular) and without it,
care needs to be taken to allocate the global xlator when needed.
Moreover, the actual fix is to use the correct xlator context
in both cases. A new API is introduces when needs to be invoked
by the conusmer (although this could have been done during
register() call, keeping it a separate API makes thing flexible
and easy).
Test:
The issue is observed when a brick process goes offline. This is
triggered when test cases (.t's) are run in bulk, since each
test essestially spawns bricks processes (on volume start) and
terminates them (volume stop). Since bitrot daemon, as of now,
spawns upon volume start, the issue is much observed when the
volume is taken offline at the end of each test case. With this
fix, running the basic and core test cases along with building
the linux kernel has passed without daemon segfaults.
Thanks to Johnny (rabhat@) for helping in debugging the issue
(and with the fix :)).
Change-Id: I8d3022bf749590b2ee816504ed9b1dfccc65559a
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9953
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch introduces RPC based communication between the changelog
translator and libgfchangelog. It replaces the old pathetic stream
based interaction that existed earlier (due to time constraints :-/).
Changelog, upon initialization starts a RPC server (rpcsvc) allowing
clients to invoke a probe API as a bootup mechanism to request for
event notifications. During probe, clients can choose an event
filter specifying the type(s) of events they are interested in. As
of now there is no way to change the event notification set once
the probe RPC call is made, but that is easier to implement.
The actual event notifications is done on a separate RPC session.
The client (libgfchangelog) itself starts and RPC server which the
changelog translator "connects back" during probe. Notifications
are dispatched by a bunch of threads from the server (translator)
and the client optionally orders them if ordered notifications
are requried. FOPs fill in their respective event details in a
buffer (rot-buffs to be particular) and a bunch of threads
(consumers) swap the buffers out of roatation and dispatch them
via RPC. To avoid writer starvation, then number of dispatcher
threads is one less than the number of buffer list in rot-buffs.x
libgfchangelog becomes purely callback based -- upon event
notification from the server (and re-ordering them if required)
invoke a callback routine specified by consumer(s).
A major part of the patch is also aimed at providing backward
compatibility for geo-replication, which was one of the main
consumer of the stream based API. Also, this patch does not\
"turn on" event notifications for all fops, just a bunch which
is currently in requirement. Another pain point is that the
server does not filter events before dispatching it to the
clients. That load is taken up by the client itself (although
it's done at the library layer rather than making it hard on
the callback implementor). This needs improvement and care
needs to be taken to not load the server up with expensive
filtering mechanisms.
Change-Id: Ibf60a432b68f2dfa60c6f9add2bcfd37a9c41395
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9708
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A gf_history_changelog_next_change() calls gf_readline() to fill a
buffer without checking buffer size. The size of maxlen is not verified
to be less than the lenght of buffer. This could result in the over
filling of buffer of maxlen is greater than PATH_MAX.
Check the size of maxlen to be less than PATH_MAX and return a fail code
as needed.
BUG: 1174017
Change-Id: Ic53b1a6e25af69a339bc15fb2d233dc1e457910f
Reported-by: Keith Schincke <kschinck@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/9275
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes bad file descriptor issue while cleaning up
scratch directory during gf_changelog_register.
Change-Id: Ia6aa8d55dcc2209144b48b6583681a155d919c42
BUG: 1162057
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/9495
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On changelog_register cleanup .processing, .history/.processing,
.current and .history/.current from the working directory.
Moved glusterd_recursive_rmdir and glusterd_for_each_entry to common
place(libglusterfs) and renamed as recursive_rmdir and
GF_FOR_EACH_ENTRY_IN_DIR respectively
BUG: 1162057
Change-Id: I1f98468a344cead039026762a805437b2f9e507b
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/9082
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is an improvement over the patch 'http://review.gluster.org/9337'
to trigger explicit geo-rep sync on regular files even if entry is not
present on the slave. An attempt is made to find the pargfid and
if available captures CREATE along with DATA in changelog.
CREATE is captured with default file permissions. Setting this virtual
setxattr on directories captures MKDIR in changelog. The value of
setxattr can be as follows.
If value = "1" : Both CREATE and DATA is captured in changelog if
pargfid is available, else on DATA is captured.
value = "any other: ENOTSUP is returned.
Usage:
setfattr -n glusterfs.geo-rep.trigger-sync -v "1" <file-path>
NOTE: This patch supports explicit record of entries only for
directories and regular files.
Change-Id: Iedde8b2c8bc3b78db524050d8c866ff664811d01
BUG: 1176934
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/9370
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A virtual xattr "glusterfs.geo-rep.trigger-sync" is provided
in glusterfs through changelog translator. Geo-rep triggers
a explicit data sync on setting this xattr on a file.
Changelog captures a DATA entry on file's gfid on setting this
virtual xattr on a file. This is supported only for files. It
doesn't support directories.
Usage: setfattr -n glusterfs.geo-rep.trigger-sync <file-path>
Change-Id: Ia689326ac2dcb31035ffbecad2c548eda4eb9245
BUG: 1176934
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/9337
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Thanks for Markiyan Kushnir <markiyan.kushnir@gmail.com> for
reporting this
Change-Id: I7f637295c7c2d54c33a4c16e29daf0b518874911
BUG: 1111774
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/9251
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Upon failure, mmap() returns MAP_FAILED, which is not equal to NULL.
This small correction makes sure that the potential error gets caught
and handled.
BUG: 1138621
Change-Id: I0a88b5187afa7187dcaa8f7d2cb0f9bb775c929d
Reported-by: Santosh Pradhan <santosh.pradhan@gmail.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/9130
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: Valid SETATTR entries are missing in changelog when more
than one metadata operation happen on same inode within
changelog roll-over time.
Cause: Metadata entries with fop num being GF_FOP_NULL are logged
in changelog which is of no use. Since slice version
checking is done for metadata entries to avoid logging of
subsequent entries of same inode falling into same
changelog, if the entry with GF_FOP_NULL is logged first,
subsequent valid ones will be missed.
Solution: Have a boundary condition to log only those fops whose fop
number falls between GF_FOP_NULL and GF_FOP_MAXVALUE.
Change-Id: Iff585ea573ac5e521a361541c6646225943f0b2d
BUG: 1104954
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8964
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The used once MAKE_HTIME_FILE_PATH macro uses strcpy and strcat into a
fixed buffer without checking the input lengths.
Recommend replacing with a snprintf.
Change-Id: Ia0245096774dc84be1b937e1d5750f3634fff034
BUG: 1099645
Reported-by: Keith Schincke <kschinck@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/8977
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ie5581579df63513e636fa3dd92dd90699ea7d999
BUG: 1128093
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Reviewed-on: http://review.gluster.org/8542
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch introduces call-path fop details logging for data operations
in CHANGELOG.SNAP. This feature is enabled with barrier-enable
notification and disabled with barrier-disable notification.
Change-Id: Ib548d34203eb99cea478a6baff402e82251c73a4
BUG: 1128093
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Reviewed-on: http://review.gluster.org/8533
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
when a snapshot is taken, there are chances of entry creation fops
not being recorded either in changelog or through the recursive
ancestry xtime updation by marker. This causes consumers of changelog
(primarily geo-replication as of today) to not be aware of these entries
after a snapshot is restored. This can lead to inconsistencies. This patch
is an interim workaround to barrier creates till changelog becomes completely
crash consistent.
BUG: 1128093
Change-Id: Ie0dbfd74beecb88df5c2ddf9fc680af91547c3f3
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/8517
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: ajeet jha <ajha@redhat.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Break-way from '/var/lib/glusterd' hard-coded previously,
instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
platforms.
- Define proper environment for running tests, define correct PATH
and LD_LIBRARY_PATH when running tests, so that the desired version
of glusterfs is used, regardless where it is installed.
(Thanks to manu@netbsd.org for this additional work)
Change-Id: I2339a0d9275de5939ccad3e52b535598064a35e7
BUG: 1111774
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/8246
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|