| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Afr does post-ops after write but the stat buffer it unwinds is at the
time of write, so if nfs client caches this, it will see different
ctime when it does stat on it after post-op is done. From NFS client's
perspective it thinks the file is changed. Tar which depends on this
to be correct keeps giving 'file changed as we read it' warning.
If Afr instead has to choose to unwind after post-op, eager-lock,
delayed-post-op will have to be disabled which will lead to bad
performance for all write usecases.
Fix:
Don't let client cache stat after write.
Change-Id: Ic6062acc6e5cdd97a9c83c56bd529ec83cee8a23
BUG: 1302948
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Signed-off-by: Anuradha Talur <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/13785
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
when bind-insecure is 'off', all the clients bind to secure ports,
if incase all the secure ports exhaust the client will no more bind
to secure ports and tries gets a random port which is obviously insecure.
we have seen the client obtaining a port number in the range 49152-65535
which are actually reserved as part of glusterd's pmap_registry for bricks,
hence this will lead to port clashes between client and brick processes.
Solution:
If we can define different port ranges for clients incase where secure ports
exhaust, we can avoid the maximum port clashes with in gluster processes.
Still we are prone to have clashes with other non-gluster processes, but
the chances being very low in the rhgs Env, but that's a different story
on its own, which will be handled in upcoming patches.
Change-Id: Ib5ce05991aa1290ccb17f6f04ffd65caf411feaf
BUG: 1322805
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-on: http://review.gluster.org/13998
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Recent changes done w.r.t handling of mkdir calls in posix translator
resulted in crashing the brick process from trash translator. This was
due to the changes made in posix translator to return EPERM for every
mkdir calls without 'gfid-req' set in dictionary. In order to avoid
gfid mismatches during directory creation from brick side trash
translator does not set 'gfid-req'. This patch is to have an exemption
for trash based on a special pid set for those mkdir calls originating
from trash translator and to reset it in callback.
This patch also includes a small optimization to the existing test case
for trash feature.
Change-Id: I59f084ac875e54342ecf2bffa6e43ebd84814153
BUG: 1317361
Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-on: http://review.gluster.org/13776
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Till now _gf_client_pid enum has been used to define special PIDs used
by glusterfs clients like shd, quotad etc. In order to have this enum
capable of holding all other special PIDs including the one used by
trash translator, _gf_client_pid is being renamed to _gf_special_pid.
Change-Id: Id123127771f18aa55d39f335801a54810848d7bc
BUG: 1330616
Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-on: http://review.gluster.org/14083
Reviewed-by: Joseph Fernandes
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I0b124e119d167817be2ae3eb52ac6c80fc7db5d1
BUG: 1320716
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/14000
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ia27d66b1061b0377857827515590eb89b18515c9
BUG: 1319992
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/11596
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Glusterd not working using ipv6 transport. The idea is with proper glusterd.vol configuration,
1. glusterd needs to listen on default port (240007) as IPv6 TCP listner.
2. Volume creation/deletion/mounting/add-bricks/delete-bricks/peer-probe
needs to work using ipv6 addresses.
3. Bricks needs to listen on ipv6 addresses.
All the above functionality is needed to say that glusterd supports ipv6 transport and this is broken.
Fix:
When "option transport.address-family inet6" option is present in glusterd.vol
file, it is made sure that glusterd creates listeners using ipv6 sockets only and also the same information is saved
inside brick volume files used by glusterfsd brick process when they are starting.
Tests Run:
Regression tests using ./run-tests.sh
IPv4: Ran manually till tests/basic/rpm.t .
IPv6: (Need to add the above mentioned config and also add an entry for "hostname ::1" in /etc/hosts)
Started failing at ./tests/basic/glusterd/arbiter-volume-probe.t and ran successfully till here
Unit Tests using Ipv6
peer probe
add-bricks
remove-bricks
create volume
replace-bricks
start volume
stop volume
delete volume
Change-Id: Iebc96e6cce748b5924ce5da17b0114600ec70a6e
BUG: 1117886
Signed-off-by: Nithin D <nithind1988@yahoo.in>
Reviewed-on: http://review.gluster.org/11988
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1. When sync fails, the cached-write is still preserved unless there
is a flush/fsync waiting on it.
2. When a sync fails and there is a flush/fsync waiting on the
cached-write, the cache is thrown away and no further retries will
be made. In other words flush/fsync act as barriers for all the
previous writes. The behaviour of fsync acting as a barrier is
controlled by an option (see below for details). All previous
writes are either successfully synced to backend or forgotten in
case of an error. Without such barrier fop (especially flush which
is issued prior to a close), we end up retrying for ever even after
fd is closed.
3. If a fop is waiting on cached-write and syncing to backend fails,
the waiting fop is failed.
4. sync failures when no fop is waiting are ignored and are not
propagated to application. For eg.,
a. first attempt of sync of a cached-write w1 fails
b. second attempt of sync of w1 succeeds
If there are no fops dependent on w1 are issued b/w a and b,
application won't know about failure encountered in a.
5. The effect of repeated sync failures is that, there will be no
cache for future writes and they cannot be written behind.
fsync as a barrier and resync of cached writes post fsync failure:
==================================================================
Whether to keep retrying failed syncs post fsync is controlled by an
option "resync-failed-syncs-after-fsync". By default, this option is
set to "off".
If sync of "cached-writes issued before fsync" (to backend) fails,
this option configures whether to retry syncing them after fsync or
forget them. If set to on, cached-writes are retried till a "flush"
fop (or a successful sync) on sync failures. fsync itself is failed
irrespective of the value of this option, when there is a sync failure
of any cached-writes issued before fsync.
Change-Id: I6097c9257bfb9ee5b15616fbe6a0576ae9af369a
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
BUG: 1279730
Reviewed-on: http://review.gluster.org/12594
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: For EC volume, If a file descriptor is open and
file has been unlinked, any further write on that fd will
fail. When a write request comes, EC internally reads some
blocks using anonymous fd. This read will fail as the file
has already been unlinked.
Solution: To solve this issue, we are using .unlink directory
to keep track of unlinked file. If a file is to be unlinked
while its fd is open, move this to .unlink directory and unlink
it from .glusterfs and real path. Once all the fd will be closed,
remove this entry form .unlink directory.
Change-Id: I8344edb0d340bdb883dc46458c16edbc336916b9
BUG: 1286029
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: http://review.gluster.org/12816
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
GEO-REP INTEROP WITH SHARD FEATURE
shard xlator updates size of the file using FXATTROP
or XATTROP. Hence record the same in changelog.
Change-Id: Ie0c21e9326da05ea78dc1ef3fd32a90ef38b4bb9
BUG: 1265148
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/12225
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I8ae7af266d3e00460f0cfdc9389a926e5f2fee36
BUG: 1282761
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/12598
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
tail, as in dog chasing its tail. These are the unwrapped
syscalls that have crept in (or were missed) in the previous
patches.
various xlators and other components are invoking system calls
directly instead of using the libglusterfs/syscall.[ch] wrappers.
If not using the system call wrappers there should be a comment
in the source explaining why the wrapper isn't used.
Change-Id: If183487de92fc7cbc47d4c5aa3f3e80eae50b84f
BUG: 1267967
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/12589
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
- Using sampling feature you can record details about every Nth FOP.
The fields in each sample are: FOP type, hostname, uid, gid, FOP priority,
port and time taken (latency) to fufill the request.
- Implemented using a ring buffer which is not (m/c) allocated in the IO path,
this should make the sampling process pretty cheap.
- DNS resolution done @ dump time not @ sample time for performance w/
cache
- Metrics can be used for both diagnostics, traffic/IO profiling as well
as P95/P99 calculations
- To control this feature there are two new volume options:
diagnostics.fop-sample-interval - The sampling interval, e.g. 1 means
sample every FOP, 100 means sample every 100th FOP
diagnostics.fop-sample-buf-size - The size (in bytes) of the ring
buffer used to store the samples. In the even more samples
are collected in the stats dump interval than can be held in this buffer,
the oldest samples shall be discarded. Samples are stored in the log
directory under /var/log/glusterfs/samples.
- Uses DNS cache written by sshreyas@fb.com (Thank-you!), the DNS cache
TTL is controlled by the diagnostics.stats-dnscache-ttl-sec option
and defaults to 24hrs.
Test Plan:
- Valgrind'd to ensure it's leak free
- Run prove test(s)
- Shadow testing on 100+ brick cluster
Change-Id: I9ee14c2fa18486b7efb38e59f70687249d3f96d8
BUG: 1271310
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/12210
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: The DB on the brick is been accessed by CTR, for write and
tier migrator, for read and write. The write from tier migrator is reseting
the heat counters after a cycle. Since we are using sqlite, two connections
trying to write would cause a db lock contention. As a result CTR used to fail
to update the db.
Solution: Using the same db connection of CTR for reseting the heat counters.
1) Introducted a new IPC FOP for CTR
2) After the query do a ipc syncop to the underlying client xlator associated
to the brick.
3) CTR in brick will catch the IPC FOP and cleat the heat counters.
Change-Id: I53306bfc08dcdba479deb4ccc154896521336150
BUG: 1260730
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/12031
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
problem:
When bind-insecure is turned on (which is the default now), it may happen
that brick is not able to bind to port assigned by Glusterd for example
49192-49195...
It seems to occur because the rpc_clnt connections are binding to ports in
the same range. so brick fails to bind to a port which is already used by
someone else
solution:
fix for now is to make rpc_clnt to get port numbers from 65535 in a descending
order, as a result port clash is minimized
other fixes:
previously rdma binds to port >= 1024 if it cannot find a free port < 1024,
even when bind insecure was turned off(ref to commit '0e3fd04e'), this patch
add's a check for bind-insecure in gf_rdma_client_bind function
This patch also re-enable bind-insecure and allow insecure by default which was
reverted (ref: commit cef1720) previously
Change-Id: Ia1cfa93c5454e2ae0ff57813689b75de282ebd07
BUG: 1238661
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-on: http://review.gluster.org/11512
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I8a0f40834da1151ddaef6139af3782bc076df57e
BUG: 1194640
Signed-off-by: Mohamed Ashiq Liyazudeen <mliyazud@redhat.com>
Reviewed-on: http://review.gluster.org/11464
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit b68f671b2b8a0aafef8f98145aee7044edaa907d from
http://review.gluster.org/11196 . The change depends on modifications to
the cluster xlators, but these are still partially under review.
Dropping this change now, it causes regression tests to fail.
Change-Id: If5ae4a519c9c6312cdb2e2a31acce4b1901f9442
BUG: 1231132
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/11452
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Changes to detect the list of upcall events enabled using GF_FOP_IPC
and return ENOTSUP to applications in case if they poll for any of
the events disabled.
Change-Id: Icc748054ef903598288119dbe99b1e337174662a
BUG: 1231132
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-on: http://review.gluster.org/11196
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
error message
Change-Id: I0a1c99ae7a2efc657e3465b21dd238e725ae236c
BUG: 1194640
Signed-off-by: Mohamed Ashiq Liyazudeen <mliyazud@redhat.com>
Reviewed-on: http://review.gluster.org/11400
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch uses "cleanup, v1" infrastrcuture to cleanup scrubber
(data structures, threads, timers, etc..) on brick disconnection.
Signer is not cleaned up yet: probably would be done as part of
another patch.
Change-Id: I78a92b8a7f02b2f39078aa9a5a6b101fc499fd70
BUG: 1231619
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/11148
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: Iee1c083774c988375a7261cfd6d510ed4c574de2
BUG: 1194640
Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com>
Reviewed-on: http://review.gluster.org/10824
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently bitrot using 120 second waiting time for object to be signed
after all fop's released. This signing waiting time value should be tunable.
Command for changing the signing waiting time will be
#gluster volume bitrot <VOLNAME> signing-time <waiting time value in second>
Change-Id: I89f3121564c1bbd0825f60aae6147413a2fbd798
BUG: 1228680
Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/11105
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).
When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.
BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Background:
Glusterfs changelogs are stored in each brick, which records the changes
happened in that brick. Georep will run in all the nodes of master and
processes changelogs "independently".
Processing changelogs is in brick level, but all the fops will be replayed
on "slave mount" point.
Problem:
With a DHT volume, in changelog "internal fops" are NOT recorded.
For Rename case, Rename is recorded in "hashed" brick changelog.
(DHT's internal fops like creating linkto file, unlink is NOT recorded).
This lead us to inconsistent rename operations.
For example,
Distribute volume created with Two bricks B1, B2.
//Consider master volume mounted @ /mnt/master
and following operations executed:
cd /mnt/master
touch f1 // f1 falls on B1 Hash
mv f1 f2 // f2 falls on B2 Hash
// Here, Changelogs are recorded as below:
@B1
CREATE f1
@B2
RENAME f1 f2
Here, race exists between Brick B1 and B2, say B2 will get executed first.
Source file f1 itself is "NOT PRESENT", so it will go ahead and create
f2 (Current implementation).
We have this problem When rename falls in another brick and
file is unlinked in Master.
Similar kind of issue exists in following case too(multiple rename):
CREATE f1
RENAME f1 f2
RENAME f2 f1
Solution:
Instead of carrying out "changelogging" at "HASHED volume",
carry out at the "CACHED volume".
This way we have rename operations carried out where actual files are present.
So,Changelog recorded as :
@B1
CREATE f1
RENAME f1 f2
credit: sarumuga@redhat.com
PS: Some of the races as the one below are _NOT_ fixed by this patch
* f1 and f2 exist. B1 and B2 are their respective cached subvols. For
both files hashed-subvol == cached-subvol
* mv f1 f2 on master.
* B1 has change-log entry of rename f1 f2
* rebalance migrates f2 from B1 and B2
* mv f2 f1 on master.
* B2 has change-log entry of rename f2 f1
Since changelog entries (rename f1 f2) and (rename f2 f1) are processed
independently by gsyncds, which of either f1 and f2 survives on slave
is subject to race. Note that on master its file f1 with name f1 which
survived. On slave it can be either file f1 with name f1 or file f2
with name f2 based on who wins the race of processing changelog.
Change-Id: Iebc222f582613924c3a7cba37fb6d3e2d8332eda
BUG: 1141379
Signed-off-by: Nithya Balachandran <nbalacha@redhat.com>
Reviewed-on: http://review.gluster.org/10410
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently when quota limit is set, corresponding gfid
is set in quota.conf. This patch supports storing
inode-quota limits in quota.conf and also stores
additional byte for each gfid to differentiate
between usage quota limit and inode quota limit.
Change-Id: I444d7399407594edd280e640681679a784d4c46a
BUG: 1202244
Signed-off-by: vmallika <vmallika@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/10069
Tested-by: NetBSD Build System
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
EEXIST and ENOENT are safe errors for geo-replication.
Since mkdir is captured in all the bricks of the changelog.
mkdir is tried multiple times as per the number of bricks.
The first one to process by gsyncd will succeed and all
others will get EEXIST. Hence EEXIST is a safe error
and can be ignored. Similarly ENOENT also in rm -rf case.
And also gsyncd validates these errors and log them in
master if it is genuine error. This is coming up with
the patch http://review.gluster.org/#/c/10048/
Hence ignoring above said safe errors.
Change-Id: I10ae86b11d49c7c3ba2be3110dace6b33daa509e
BUG: 1210562
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10184
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On Linux systems we should use the libuuid from the distribution and not
bundle and statically link the contrib/uuid/ bits.
libglusterfs/src/compat-uuid.h has been introduced and should become an
abstraction layer for different UUID APIs. Non-Linux operating systems
should implement their compatibility layer there.
Once all operating systems have an implementation in compat-uuid.h, we
can remove contrib/uuid/ from the repository completely.
Change-Id: I345e5357644be2521685e00358bb8c83c4ea0577
BUG: 1206587
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10129
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Quota hard-limit is supported only upto: 9223372036854775807 (int 64)
In CLI, it is allowed to set the value upto 16384PB (unsigned int 64),
this is not a valid value as the xattrop for quota accounting and
the quota enforcer operates on a signed int64 limit value.
This patches fixes the problem in CLI and allows user to set
the hard-limit value only from range 0 - 9223372036854775807
Change-Id: Ifce6e509e1832ef21d3278bacfa5bd71040c8cba
BUG: 1206432
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/10022
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This fix will solve the heating of the files during the promotion
or demotion.
Promotion:
~~~~~~~~~
When a file gets promoted it get the current time stamp
during creation only, but following writes or reads during the
migration wont heat the file.
Demotion:
~~~~~~~~
When a file gets demoted it get the wind/unwind time stamp is set to
zero. The following writes or reads during the migration wont heat
the file.
What is remaining ?
~~~~~~~~~~~~~~~~~
Bug 1209129 ( https://bugzilla.redhat.com/show_bug.cgi?id=1209129 )
Inspite of this fix there is still a issue remaining, i.e the heat of
the file is not keep intact during a internal rebalance activity i.e
a rebalance within a tier.
Change-Id: I01e82dc226355599732d40e699062cee7960b0a5
BUG: 1207867
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/10080
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.
Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.
A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.
BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I208289aae2423e4bb015cf33bafd2a961e1c3fc6
BUG: 1197593
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/9779
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Sachin Pandit <spandit@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Scrubber performs signature verification for objects that were
signed by signer. This is done by recalculating the signature
(using the hash algorithm the object was signed with) and
verifying it aginst the objects persisted signature. Since the
object could be undergoing IO opretaion at the time of hash
calculation, the signature may not match objects persisted
signature. Bitrot stub provides additional information about
the stalesness of an objects signature (determinted by it's
versioning mechanism). This additional bit of information is
used by scrubber to determine the staleness of the signature,
and in such cases the object is skipped verification (although
signature staleness is performed twice: once before initiation
of hash calculation and another after it (an object could be
modified after staleness checks).
The implmentation is a part of the bitrot xlator (signer) which
acts as a signer or scrubber based on a translator option. As
of now the scrub process is ever running (but has some form of
weak throttling mechanism during filesystem scan). Going forward,
there needs to be some form of scrub scheduling and IO throttling
(during hash calculation) tunables (via CLI).
Change-Id: I665ce90208f6074b98c5a1dd841ce776627cc6f9
BUG: 1170075
Original-Author: Raghavendra Bhat <rabhat@redhat.com>
Original-Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9914
Tested-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is the "Signer" -- responsible for signing files with their
checksums upon last file descriptor close (last release()).
The event notification facility provided by the changelog xlator
is made use of.
Moreover, checksums are as of now SHA256 hash of the object data
and is the only available hash at this point of time. Therefore,
there is no special "what hash to use" type check, although it's
does not take much to add various hashing algorithms to sign
objects with. Signatures are stored in extended attributes of the
objects along with the the type of hashing used to calculate the
signature. This makes thing future proof when other hash types
are added. The signature infrastructure is provided by bitrot
stub: a little piece of code that sits over the POSIX xlator
providing interfaces to "get or set" objects signature and it's
staleness.
Since objects are signed upon receiving release() notification,
pre-existing data which are "never" modified would never be
signed. To counter this, an initial crawler thread is spawned
The crawler scans the entire brick for objects that are unsigned
or "missed" signing due to the server going offline (node reboots,
crashes, etc..) and triggers an explicit sign. This would also
sign objects when bit-rot is enabled for a volume and/or after
upgrade.
Change-Id: I1d9a98bee6cad1c39c35c53c8fb0fc4bad2bf67b
BUG: 1170075
Original-Author: Raghavendra Bhat <raghavendra@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9711
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bitrot stub implements object versioning required for identifying
signature freshness. More details about versioning is explained
as a part of the "bitrot feature documentation" patch.
Change-Id: I2ad70d9eb109ba4a12148ab8d81336afda529ad9
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9709
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Translators which wish to send event notifications can send
"down" an IPC FOP with op_type as GF_IPC_TARGET_CHANGELOG
and xdata carrying event structures (changelog_event_t).
Change-Id: I0e5f8c9170161c186f0e58d07105813e34e18786
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9775
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Parses linux style export file/netgroups file into a structure that
can be lookedup.
* This parser turns each line into a structure called an "export
directory". Each of these has a dictionary of hosts and netgroups
which can be looked up during the mount authentication process.
(See Change-Id Ic060aac and I7e6aa6bc)
* A string beginning withan '@' is treated as a netgroup and a string
beginning without an @ is a host.
(See Change-Id Ie04800d)
* This parser does not currently support all the options in the man page
('man exports'), but we can easily add them.
BUG: 1143880
URL: http://www.gluster.org/community/documentation/index.php/Features/Exports_Netgroups_Authentication
Change-Id: I181e8c1814d6ef3cae5b4d88353622734f0c0f0b
Original-author: Shreyas Siravara <shreyas.siravara@gmail.com>
CC: Richard Wareing <rwareing@fb.com>
CC: Jiffin Tony Thottan <jthottan@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/8758
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The /etc/exports format for NFS-exports (see Change-Id I7e6aa6b) allows
a more fine grained control over the authentication. This change adds
the functions and structures that will be used in by Change-Id I181e8c1.
BUG: 1143880
Change-Id: Ic060aac7c52d91e08519b222ba46383c94665ce7
Original-author: Shreyas Siravara <shreyas.siravara@gmail.com>
CC: Richard Wareing <rwareing@fb.com>
CC: Jiffin Tony Thottan <jthottan@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/9362
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This generic parser will get used for parsing the netgroups and exports
files for the Gluster/NFS server. The parsing of netgroups shows how the
parser can be used (see Change-Id Ie04800d4).
BUG: 1143880
Change-Id: Id4cf2b0189ef5799c06868d211d3fcd9c8608c08
Original-author: Shreyas Siravara <shreyas.siravara@gmail.com>
CC: Richard Wareing <rwareing@fb.com>
CC: Jiffin Tony Thottan <jthottan@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/9359
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
These will be used by both afr and ec. Moved syncop_dirfd, syncop_ftw,
syncop_dir_scan functions also into syncop-utils.c
Change-Id: I467253c74a346e1e292d36a8c1a035775c3aa670
BUG: 1177601
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/9740
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PROBLEM:
Previously gluster accepting input value as a percentage which is out of range
[0-100] and accepting input value as a size (unit is byte) which is fractional
for option cluster.min-free-disk.
FIX:
Now with this change it will refer to correct validation function
and it will accept value that is in range [0-100] for input value as a
percentage and unsigned integer value for input as a size (unit in byte)
for option cluster.min-free-disk.
Change-Id: Iee1962a100542e146276cfc8a4068abddee2bf2d
BUG: 1163108
Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
Reviewed-on: http://review.gluster.org/9104
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For volumes with replicate, disperse xlators, self-heal daemon should do
healing. This patch provides enable/disable functionality for the xlators to be
part of self-heal-daemon. Replicate already had this functionality with
'gluster volume set cluster.self-heal-daemon on/off'. But this patch makes it
uniform for both types of volumes. Internally it still does 'volume set' based
on the volume type.
Change-Id: Ie0f3799b74c2afef9ac658ef3d50dce3e8072b29
BUG: 1177601
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/9358
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On changelog_register cleanup .processing, .history/.processing,
.current and .history/.current from the working directory.
Moved glusterd_recursive_rmdir and glusterd_for_each_entry to common
place(libglusterfs) and renamed as recursive_rmdir and
GF_FOR_EACH_ENTRY_IN_DIR respectively
BUG: 1162057
Change-Id: I1f98468a344cead039026762a805437b2f9e507b
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/9082
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
POSIX says that an offset obtained from telldir() can only be used
on the same DIR *. Linux is abls to reuse the offset accross
closedir()/opendir() for a given directory, but this is not portable
and such a behavior should be fixed.
An incomplete fix for the posix xlator was merged in
http://review.gluster.com/8926
This change set completes it.
- Perform the same fix index xlator.
- Use appropriate casts and variable types so that 32 bit signed
offsets obtained by telldir() do not get clobbered when copied into
64 bit signed types.
- modify glfs-heal.c and afr-self-heald.c so that they do not use
anonymous fd, since this will cause closedir()/opendir() between
each syncop_readdir(). On failure we fallback to anonymous fs
only for Linux so that we can cope with updated client vs not
updated brick.
- Avoid sending an EINVAL when the client request for the EOF offset.
Here we fix an error in previous fix for posix xlator: since we
fill each directory entry with the offset of the next entry, we
must consider as EOF the offset of the last entry, and not the
value of telldir() after we read it.
- Add checks in regression tests that we do not hit cases where
offsets fed to seekdir() are wrong. Introduce log_newer() shell
function to check for messages produced by the current script.
This fix gather changes from http://review.gluster.org/9047
and http://review.gluster.org/8936 making them obsolete.
BUG: 1129939
Change-Id: I59fb7f06a872c4f98987105792d648141c258c6a
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/9071
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Raghavendra Bhat <raghavendra@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
gf_time_fmt() has existed since 3.3; it provides consistent timestamps
(i.e. UTC times) throughout the implementation. (BTW, the other name for UTC
is GMT.)
N.B. many (all?) commercial storage solutions use UTC time for logging.
This makes for easier debugging across geographically distributed systems.
Also adding a "%s" fmt for portably printing time as simple numeric
value on systems regardless of whether 32-bit or 64-bit time_t. Plus a
minor tweak to return a ptr to the dest-string to allow gf_time_fmt()
to be passed as a param in a *printf().
Someday we should pick the "one true" timestamp format and revise all
calls to gf_time_fmt() to use it instead of the five or six different
formats.
Change-Id: I78202ae14b7246fa424efeea56bf2463e14abfb0
BUG: 1109917
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/8085
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We are using the function to export all sub-directories in a gluster volume via nfs.
For real directories it works fine but if we have a symbolic link which points to the
directory, it is not possible to mount that directory via nfs and the nameof the link.
Kernel nfs resolves symlink handle to directoryhandle , similar gluster nfs should
resolve the symbolic link handle into directory handle.
Change-Id: I8bd07534ba9474f0b863f2335b2fd222ab625dba
BUG: 1157223
Signed-off-by: jiffin tony thottan <jthottan@redhat.com>
Reviewed-on: http://review.gluster.org/9052
Reviewed-by: soumya k <skoduri@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Just after replace-brick the mount logs are filled with ENOENT/ESTALE
warning logs because the file is yet to be self-healed now that the
brick is new.
Fix:
Do conditional logging for the logs. ENOENT/ESTALE will be logged at
lower log level. Only when debug logs are enabled, these logs will
be written to the logfile.
Change-Id: If203d09e2479e8c2415ebc14fb79d4fbb81dfc95
BUG: 1151303
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/8918
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also, moved the backtrace fetching logic to a separate function.
Modified the backtrace fetching logic able to work under memory pressure
conditions.
Change-Id: Ie38bea425a085770f41831314aeda95595177ece
BUG: 1138503
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/8584
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Geo-replicatoin does a full xsync crawl after snapshot
restoration of slave and master. It does not do history crawl.
Analysis:
Marker creates 'marker.tstamp' file when geo-rep is started
for the first time. The virtual extended attribute
'trusted.glusterfs.volume-mark' is maintained and whenever
it is queried on gluster mount point, marker fills it on
the fly and returns the combination of uuid, ctime of
marker.tstamp and others. So ctime of marker.tstamp, in other
sense 'volume-mark' marks the geo-rep start time when the
session is freshly created.
From the above, after the first filesystem crawl(xsync) is
done during first geo-rep start, stime should always be less
than 'volume-mark'. So whenever stime is less than volume-mark,
it does full filesystem crawl (xsync).
Root Cause:
When snapshot is restored, marker.tstamp file is freshly
created losing the timestamps, it was originally created with.
Solution:
1. Change is made to depend on mtime instead of ctime.
2. mtime and atime of marker.tstamp is restored back when
snapshot is created and restored.
Change-Id: I4891b112f4aedc50cfae402832c50c5145807d7a
BUG: 1125918
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8401
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes changelog capturing internal FOPs in a cascaded
setup, where the intermediate master would record internal FOPs
(generated by DHT on link()/rename()). This is due to I/O happening
on the intermediate slave on geo-replication's auxillary mount with
client-pid -1. Currently, the internal FOP capturing logic depends
on client pid being non-negative and the presence of a special key
in dictionary. Due to this, internal FOPs on an inter-mediate master
would be recorded in the changelog. Checking client-pid being
non-negative was introduced to capture AFR self-heal traffic in
changelog, thereby breaking cascading setups. By coincidence,
AFR self-heal daemon uses -1 as frame->root->pid thereby making
is hard to differentiate b/w geo-rep's auxillary mount and self-heal
daemon.
Change-Id: Ib7bd71e80dd1856770391edb621ba9819cab7056
BUG: 1122037
Original-Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8347
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch improves the peer identification mechanism in glusterd and
lays down the framework for further improvements, including better multi
network support in glusterd.
This patch mainly does two things,
1. Extend the peerinfo object to store a list of addresses instead of a
single hostname as it does now. This also includes changes to make the
peer update behaviour of 'peer probe' to add to the list.
2. Improve glusterd_friend_find_by_hostname() to perform better matching
of hostnames. glusterd_friend_find_by_hostname() now does and initial
quick string compare against all the peer addresses known to glusterd,
after which it tries a more thorough search using address resolution and
matching the struc sockaddr's.
The above two changes together improve the peer identification situation
in glusterd a lot.
More information regarding the problem this patch attempts to resolve
and the approach chosen can be found at
http://www.gluster.org/community/documentation/index.php/Features/Better_peer_identification
This commit is a squashed commit of the following changes, the
development branch of which can be viewed at,
https://github.com/kshlm/glusterfs/tree/better-peer-identification or,
https://forge.gluster.org/~kshlm/glusterfs-core/kshlms-glusterfs/commits/better-peer-identification
commit 198f86e60fab74faf082eaa02657a4d8f60b92f0
Author: Kaushal M <kaushal@redhat.com>
Date: Tue Jul 15 14:34:06 2014 +0530
Update gluster.8
commit 35d597f3a6b3248373e727f7b7e889c92554d56c
Author: Kaushal M <kaushal@redhat.com>
Date: Tue Jul 15 09:01:01 2014 +0530
Address review comments
https://review.gluster.org/#/c/8238/3
commit 47b5331e17304477322bd2daed5bbed503c34ca1
Merge: c71b12c 78128af
Author: Kaushal M <kaushal@redhat.com>
Date: Tue Jul 15 08:41:39 2014 +0530
Merge branch 'master' into better-peer-identification
commit c71b12c164330e8d19d1df4734ab34ef9a8caad2
Merge: 57bc9de 0f5719a
Author: Kaushal M <kaushal@redhat.com>
Date: Thu Jul 10 19:50:19 2014 +0530
Merge branch 'master' into better-peer-identification
commit 57bc9de9e4f49ff2b1620df9906cda50a3527a25
Author: Kaushal M <kaushal@redhat.com>
Date: Thu Jul 10 19:49:08 2014 +0530
More fixes to review comments
commit 5482cc363a687a9e246a0780ec88acd53e218501
Author: Kaushal M <kaushal@redhat.com>
Date: Thu Jul 10 18:36:40 2014 +0530
Code refactoring in peer-utils based on review comments
https://review.gluster.org/#/c/8238/2/xlators/mgmt/glusterd/src/glusterd-peer-utils.c
commit 89b22c34757178f64d5fbaffa31e6302f841c060
Author: Kaushal M <kaushal@redhat.com>
Date: Thu Jul 10 12:30:00 2014 +0530
Hostnames in peer status
commit 63ebf9485cf50d736cf640238a1ab241671fcaf1
Merge: c8c8fdd f5f9721
Author: Kaushal M <kaushal@redhat.com>
Date: Thu Jul 10 12:06:33 2014 +0530
Merge remote-tracking branch 'origin/master' into better-peer-identification
commit c8c8fdd2104b5b6b8a1af739b1dd952b74e6dd66
Author: Kaushal M <kaushal@redhat.com>
Date: Wed Jul 9 18:35:27 2014 +0530
Hostnames in xml output
commit 732a92a0167ad7b1d70edbc35ebd8307c2766ae1
Author: Kaushal M <kaushal@redhat.com>
Date: Wed Jul 9 15:12:10 2014 +0530
Add hostnames to cli rsp dict during list-friends
commit fcf43e3e317508f0c225024738a988a4af8e9205
Merge: c0e2624 72d96e2
Author: Kaushal M <kaushal@redhat.com>
Date: Wed Jul 9 12:53:03 2014 +0530
Merge branch 'master' into better-peer-identification
commit c0e262416728a3c536a8347a216e471eb2251535
Author: Kaushal M <kaushal@redhat.com>
Date: Mon Jul 7 16:11:19 2014 +0530
Use list_for_each_entry_safe when cleaning peer hostnames
commit 6132e60224eb592f3657e535a12a3e72c772da42
Author: Kaushal M <kaushal@redhat.com>
Date: Mon Jul 7 15:52:19 2014 +0530
Fix crash in gd_add_friend_to_dict
commit 88ffa9a508fd5aac0b2a76e6e76487ce0cab786a
Author: Kaushal M <kaushal@redhat.com>
Date: Mon Jul 7 13:19:44 2014 +0530
gd_peerinfo_destroy -> glusterd_peerinfo_destroy
commit 4b36930a715b1e13cd1a77d136ef1cf78a06d574
Author: Kaushal M <kaushal@redhat.com>
Date: Mon Jul 7 12:50:12 2014 +0530
More refactoring
commit ee559b081d608c6501c10ae22166f26eeb65690e
Author: Kaushal M <kaushal@redhat.com>
Date: Mon Jul 7 12:14:40 2014 +0530
Major refactoring of code based on review comments at
https://review.gluster.org/#/c/8238/1/xlators/mgmt/glusterd/src/glusterd-peer-utils.h
commit e96dbc7bbb05fad2a9c424de41a394b8023fe48d
Merge: 2613d1d 83c09b7
Author: Kaushal M <kaushal@redhat.com>
Date: Mon Jul 7 09:47:05 2014 +0530
Merge remote-tracking branch 'origin/master' into better-peer-identification
commit 2613d1daebff0c56812de821c06ed4c16bb9d447
Merge: b242cf6 9a50211
Author: Kaushal M <kaushal@redhat.com>
Date: Fri Jul 4 15:28:57 2014 +0530
Merge remote-tracking branch 'origin/master' into better-peer-identification
commit b242cf66d95dd3dd5e3975aa430baa6bd74b8a29
Author: Kaushal M <kaushal@redhat.com>
Date: Fri Jul 4 15:08:18 2014 +0530
Fix a silly mistake, if (ctx->req) => if (ctx->req == NULL)
commit c835ed26433830ceed57289143f596cf60421558
Author: Kaushal M <kaushal@redhat.com>
Date: Fri Jul 4 14:58:23 2014 +0530
Fix reverse probe.
commit 9ede17f9329b854b02e8ad159f173244789fd08c
Author: Kaushal M <kaushal@redhat.com>
Date: Fri Jul 4 13:31:32 2014 +0530
Fix friend import for existing peers
commit 891bf74c7350064dfb008d1b7294bcec28d680fd
Author: Kaushal M <kaushal@redhat.com>
Date: Fri Jul 4 13:08:36 2014 +0530
Set first hostname in peerinfo->hostnames to peerinfo->hostname
commit 9421d6a217381a7427a7d84f369280883ca4297a
Author: Kaushal M <kaushal@redhat.com>
Date: Fri Jul 4 12:21:40 2014 +0530
Fix gf_asprintf return val check in glusterd_store_peer_write
commit defac978c1d94011ce8195e311839b9ffce057e7
Author: Kaushal M <kaushal@redhat.com>
Date: Fri Jul 4 11:16:13 2014 +0530
Fix store_retrieve_peers to correctly cleanup.
commit 00a799f5de1121b0cb7421da8285f9407063e1bd
Author: Kaushal M <kaushal@redhat.com>
Date: Fri Jul 4 10:52:11 2014 +0530
Update address list in glusterd_probe_cbk only when needed.
commit 7a628e8a9c562d85709c69cfa13fb1774c521b75
Merge: d191985 dc46d5e
Author: Kaushal M <kaushal@redhat.com>
Date: Fri Jul 4 09:24:12 2014 +0530
Merge remote-tracking branch 'origin/master' into better-peer-identification
commit d1919858e6639d2b54d716a61f662d9752ec5ff1
Author: Kaushal M <kaushal@redhat.com>
Date: Tue Jul 1 18:59:49 2014 +0530
gf_compare_addrinfo -> gf_compare_sockaddr
commit 31d8ef730d408f8d9ba8f504fa648f7dcd59da87
Merge: 93bbede 86ee233
Author: Kaushal M <kaushal@redhat.com>
Date: Tue Jul 1 18:16:13 2014 +0530
Merge remote-tracking branch 'origin/master' into better-peer-identification
commit 93bbedeac5181e29f59b2acd08f638146812ec41
Author: Kaushal M <kaushal@redhat.com>
Date: Tue Jul 1 18:15:16 2014 +0530
Improve glusterd_friend_find_by_hostname
glusterd_friend_find_by_hostname will now do an initial quick search for
the peerinfo performing string comparisions on the given host string. It
follows it with a more thorough match, by resolving the addresses and
comparing addrinfos instead of strings.
commit 2542cdbc45aa9cfcaf1f174686158d5565cdd07b
Author: Kaushal M <kaushal@redhat.com>
Date: Tue Jul 1 17:21:10 2014 +0530
New utility gf_compare_addrinfo
commit 338676e8389a44bd91136eebd110197429c2566c
Author: Kaushal M <kaushal@redhat.com>
Date: Tue Jul 1 14:55:56 2014 +0530
Use gd_peer_has_address instead of strcmp
commit 28d45be51f594328741c44455bd80ac9d64ca501
Merge: 728266e 991dd5e
Author: Kaushal M <kaushal@redhat.com>
Date: Tue Jul 1 14:54:40 2014 +0530
Merge branch 'master' into better-peer-identification
commit 728266eb16d5f5a4bf36266044425ae164337f99
Merge: 7d9b87b 2417de9
Author: Kaushal M <kaushal@redhat.com>
Date: Tue Jul 1 09:55:13 2014 +0530
Merge remote-tracking branch 'origin/master' into better-peer-identification
commit 7d9b87b84955ec17daeaf88a3e7462914039430f
Merge: b890625 e02275c
Author: Kaushal M <kshlmster@gmail.com>
Date: Tue Jul 1 08:41:40 2014 +0530
Merge pull request #4 from vpshastry/better-peer-identification
Better peer identification
commit e02275c52fb83c72ad082c098fd3e432c2b9c526
Merge: 75ee90d b890625
Author: Varun Shastry <vshastry@redhat.com>
Date: Mon Jun 30 16:44:29 2014 +0530
Merge branch 'better-peer-identification' of https://github.com/kshlm/glusterfs into better-peer-identification-kaushal-github
commit 75ee90d2f272e49b94d24c9ca4571e89a83055ff
Author: Varun Shastry <vshastry@redhat.com>
Date: Mon Jun 30 15:36:10 2014 +0530
glusterd: add to the list if the probed uuid pre-exists
Signed-off-by: Varun Shastry <vshastry@redhat.com>
commit b890625d8164c660695daef3285c67979eef723e
Merge: 04c5d60 187a7a9
Author: Kaushal M <kaushal@redhat.com>
Date: Mon Jun 30 11:44:13 2014 +0530
Merge remote-tracking branch 'origin/master' into better-peer-identification
commit 04c5d60cb938c8d94b214689580b40abb1b0ffcd
Merge: 3a5bfa1 e01edb6
Author: Kaushal M <kshlmster@gmail.com>
Date: Sat Jun 28 19:23:33 2014 +0530
Merge pull request #3 from vpshastry/better-peer-identification
glusterd: search through the list of hostnames in the peerinfo
commit 0c64f3346a977f9165ac55a84a1e03c40a7573a7
Merge: e01edb6 3a5bfa1
Author: Varun Shastry <vshastry@redhat.com>
Date: Sat Jun 28 10:43:29 2014 +0530
Merge branch 'better-peer-identification' of https://github.com/kshlm/glusterfs into better-peer-identification-kaushal-github
commit e01edb63153a1008db70b8fa76ae5b535e099326
Author: Varun Shastry <vshastry@redhat.com>
Date: Fri Jun 27 12:29:36 2014 +0530
glusterd: search through the list of hostnames in the peerinfo
Signed-off-by: Varun Shastry <vshastry@redhat.com>
commit 3a5bfa15855e660db2bfde644727371dd2d618cc
Merge: cda6d31 371ea35
Author: Kaushal M <kshlmster@gmail.com>
Date: Fri Jun 27 11:31:17 2014 +0530
Merge pull request #1 from vpshastry/better-peer-identification
glusterd: Add hostname to list instead of replaceing upon update
commit 371ea354f198b4182382d5403c5960c0b2add6b6
Author: Varun Shastry <vshastry@redhat.com>
Date: Fri Jun 27 11:24:54 2014 +0530
glusterd: Add hostname to list instead of replaceing upon update
Signed-off-by: Varun Shastry <vshastry@redhat.com>
commit cda6d3152886623ecbf46baf0048ebe0119b30b6
Author: Kaushal M <kaushal@redhat.com>
Date: Thu Jun 26 19:52:52 2014 +0530
Import address lists
commit 6649b54aa0440130c08e827e0a1d1bbfb840eca9
Author: Kaushal M <kaushal@redhat.com>
Date: Thu Jun 26 19:15:37 2014 +0530
Implement export address list
commit 55990034eead92bc9b936240029e460a4bf152d5
Author: Kaushal M <kaushal@redhat.com>
Date: Thu Jun 26 18:11:59 2014 +0530
Use first address in list to when setting up the peer RPC.
commit a35fde8d19b9988eb04c652fb3a5e4f84d90ad00
Author: Kaushal M <kaushal@redhat.com>
Date: Thu Jun 26 18:03:04 2014 +0530
Properly free addresses on glusterd_peer_destroy
commit 1988081db09ac9205f3dc7268cef8be267f3ce8b
Author: Kaushal M <kaushal@redhat.com>
Date: Thu Jun 26 17:52:35 2014 +0530
Restore peerinfo with address list implemented.
commit 66f524d5749a12f4910dd6b06c9d91f37e1d831e
Author: Kaushal M <kaushal@redhat.com>
Date: Mon Jun 23 13:02:23 2014 +0530
Move out all peer related utilities from glusterd-utils to glusterd-peer-utils
commit 14a2a326a4dff11b55490dca2a14f39320931340
Author: Kaushal M <kaushal@redhat.com>
Date: Tue May 27 12:16:41 2014 +0530
Compilation fix
commit c59cd351d0a102d0d5f3ea9001fd33c4edcb262f
Author: Kaushal M <kaushal@redhat.com>
Date: Mon May 5 12:51:11 2014 +0530
Add store support for hostname list
commit b70325f0beb884ad12645ef40185f0bf6cedd741
Author: Kaushal M <kaushal@redhat.com>
Date: Fri May 2 15:58:07 2014 +0530
Add a hostnames list to glusterd_peerinfo_t
glusterd_peerinfo_new will now init this list and add the given hostname
as the lists first member.
Signed-off-by: Kaushal M <kaushal@redhat.com>
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Change-Id: Ief3c5d6d6f16571ee2fab0a45e638b9d6506a06e
BUG: 1119547
Reviewed-on: http://review.gluster.org/8238
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
|