| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When freeing memory, our memory-accounting code expects to be able to
dereference from the (previously) allocated block to its owning
translator. However, as we have already found once in option
validation and twice in logging, that translator might itself have
been freed and the dereference attempt causes on of our daemons to
crash with SIGSEGV. This patch attempts to fix that as follows:
* We no longer embed a struct mem_acct directly in a struct xlator,
but instead allocate it separately.
* Allocated memory blocks now contain a pointer to the mem_acct
instead of the xlator.
* The mem_acct structure contains a reference count, manipulated in
both the normal and translator allocate/free code using atomic
increments and decrements.
* Because it's now a separate structure, we can defer freeing the
mem_acct until its reference count reaches zero (either way).
* Some unit tests were disabled, because they embedded their own
copies of the implementation for what they were supposedly testing.
Life's too short to spend time fixing tests that seem designed to
impede progress by requiring a certain implementation as well as
behavior.
Change-Id: Id929b11387927136f78626901729296b6c0d0fd7
BUG: 1219026
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/10417
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/10723
Tested-by: NetBSD Build System
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
back port of http://review.gluster.org/10284
Fix for handling cli output for attach-tier and
detach-tier
>Change-Id: I4d17f4b09612754fe1b8cec6c2e14927029b9678
>BUG: 1211562
>Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
>Reviewed-on: http://review.gluster.org/10284
>Reviewed-by: Dan Lambright <dlambrig@redhat.com>
>Tested-by: Gluster Build System <jenkins@build.gluster.com>
>Tested-by: NetBSD Build System
>Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Change-Id: Ic30c8f0a104d89bf98f5d0069937a34674ee3f2d
BUG: 1220052
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/10713
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I80175e06efc12fd2dab126d6e0143d63caf11209
BUG: 1220059
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/10726
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Back port of http://review.gluster.org/10303
gluster detach-tier help should show the usage of
detach-tier command properly.
Also, instead of force option, if any other was given,
then the command should fail with a usage message
>Change-Id: Ie801529176db067c98fd1bfada056cbc647973fb
>BUG: 1211570
>Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
>Reviewed-on: http://review.gluster.org/10303
>Tested-by: NetBSD Build System
>Tested-by: Gluster Build System <jenkins@build.gluster.com>
>Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
>Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Change-Id: I51ef7c3450c219b2b24724592270461570436010
BUG: 1220050
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/10709
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Back port of http://review.gluster.org/10108
These commands work in a manner analagous to rebalancing when removing a
brick. The existing migration daemon detects "detach start" and switches
to moving data off the hot tier. While in this state all lookups are
directed to the cold tier.
gluster v detach-tier <vol> start
gluster v detach-tier <vol> commit
The status and stop cli commands shall be submitted separately.
>Change-Id: I24fda5cc3ba74f5fb8aa9a3234ad51f18b80a8a0
>BUG: 1205540
>Signed-off-by: Dan Lambright <dlambrig@redhat.com>
>Signed-off-by: root <root@localhost.localdomain>
>Signed-off-by: Dan Lambright <dlambrig@redhat.com>
>Reviewed-on: http://review.gluster.org/10108
>Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Change-Id: I212d748d077fb5870ee84b316c653acbafbea3f7
BUG: 1220047
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/10708
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of http://review.gluster.org/#/c/9797
Storing and restoring THIS:
When the APIs exposed by libgfapi are called by other xlators like
snapview server etc. the THIS value is overwritten to contain the
THIS of libgfapi(viz libgfapi master xlator). Hence using 'THIS'
in any xlator after calling libgfapi API will lead to issues.
One such issue was uncovered in snapview and the patch
http://review.gluster.org/#/c/9469/ was sent to workaround this issue.
Hence, storing and restoring THIS, at the entry and exit of every API
exposed by libgfapi.
Change-Id: I6f330dde25e7700fb26339d667a7ccd193ec6ba0
BUG: 1210934
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/9797
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: soumya k <skoduri@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
(cherry picked from commit 1162bb36108ab8dba8303b86927a99835b791d79)
Reviewed-on: http://review.gluster.org/10414
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently glupy files resides in gluster namespace of python site packages.
The other projects like libgfapi-python ..etc are evolving and need to share
the gluster namespace. The current structure makes things difficult as all
subpackages have its own __init__ files and other files.
One subpackage can not any more own gluster namespace.
The attempt is to make below structure for gluster namespace so that
it is more portable and scalable for future use.
<sitepackages>/gluster/
|
-- __init__.py
|
|
-- glupy
|
-- __init__.py
-- glupy.py
-- ........
|
|
-- gfapi
|
-- __init__.py
-- gfapi.py
-- ........
By above structure clients can import:
>>> from gluster import glupy
>>> from gluster import gfapi
libgfapi-python project has been moved to this structure via
http://review.gluster.org/#/c/9668/
Cherry picked from commit 40df2ed4d098d4cd2c6abbed23e497ac3e2e5804:
> Change-Id: I54886200ddb6a4153a74d9e187aeca7cad79ef9e
> BUG: 1211900
> Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
> Reviewed-on: http://review.gluster.org/10248
> Tested-by: Gluster Build System <jenkins@build.gluster.com>
> Reviewed-by: Niels de Vos <ndevos@redhat.com>
> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
This backport is really minimal, because commit 44036808 removed the
need for the changes in the glusterfs.spec.in file.
Change-Id: I54886200ddb6a4153a74d9e187aeca7cad79ef9e
BUG: 1220022
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10706
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of : http://review.gluster.org/10281
A few log messages in dht directory self heal at log level INFO are useful
only for developers and these logs tend to casue excessive logs in our
log files. Hence moving the log level of such logs to DEBUG.
Change-Id: I8a543f4ddeb5c20b2978a0f7b18d8baccc935a54
BUG: 1217949
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/10281
Reviewed-by: N Balachandran <nbalacha@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-on: http://review.gluster.org/10704
Tested-by: NetBSD Build System
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is optional and enabled by default, if one node fails Glusterfind
will not fail to return list of files from other nodes. This behavior
can be changed using --disable-partial
Now session is maintained in each nodes as well as in initiator node.
Every pre command will pick the status file from respective node and
start collecting list of changes happened after the status time.
--reset-session-time, new option to force reset the session time.
Next incremental run will start from this time.
Change-detector argument is removed since Changelog mode is required
to detect deletes and Renames.
BUG: 1219467
Change-Id: I1d0a0629facc3d26780200ccdf39b221ab4037c4
Original-Author: Aravinda VK <avishwan@redhat.com>
Reviewed-On: http://review.gluster.org/#/c/10320
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10639
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When gsyncd fails with Python traceback, glusterd fails
parsing gsyncd output and shows error.
BUG: 1219938
Change-Id: Ic32fd897c49a5325294a6588351b539c6e124338
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/10694
Reviewed-on: http://review.gluster.org/10695
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of http://review.gluster.org/#/c/10619/
Snippet of the existing test:
TEST kill_glusterd 2;
TEST $glusterd_2;
sleep 5
EXPECT_WITHIN $PROCESS_UP_TIMEOUT 'Started' volinfo_field ${V0}_clone 'Status';
EXPECT_WITHIN $PROCESS_UP_TIMEOUT 'Started' volinfo_field ${V1}_clone 'Status';
stop_force_volumes 2
The above EXPECT_WITHINs will not wait as the volumes were never stopped which
results into immediate trigger of volume stop where handshaking is not
completed. A workaround would be to check whether handshaking is completed and
then proceed.
Change-Id: Id3468b56f801212fc8e19ff196bc99e5e2358471
BUG: 1220012
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/10619
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Justin Clift <justin@gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 0950e6c29fc51ddcc5bb7e1e4187d54ff7c171d2)
Reviewed-on: http://review.gluster.org/10700
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Background:
Glusterfs changelogs are stored in each brick, which records the changes
happened in that brick. Georep will run in all the nodes of master and processes
changelogs "independently". Processing changelogs is in brick level, but
all the fops will be replayed on "slave mount" point.
Problem:
With a DHT volume, in changelog "internal fops" are NOT recorded.
For Rename case, Rename is recorded in "hashed" brick changelog.
(DHT's internal fops like creating linkto file, unlink is NOT recorded).
This lead us to inconsistent rename operations.
For example,
Distribute volume created with Two bricks B1, B2.
//Consider master volume mounted @ /mnt/master and following operations
executed:
cd /mnt/master
touch f1 // f1 falls on B1 Hash
mv f1 f2 // f2 falls on B2 Hash
// Here, Changelogs are recorded as below:
@B1
CREATE f1
@B2
RENAME f1 f2
Here, race exists between Brick B1 and B2, say B2 will get executed first.
Source file f1 itself is "NOT PRESENT", so it will go ahead and create
f2 (Current implementation).
We have this problem When rename falls in another brick and
file is unlinked in Master.
Similar kind of issue exists in following case too(multiple rename):
CREATE f1
RENAME f1 f2
RENAME f2 f1
Solution:
Instead of carrying out "changelogging" at "HASHED volume",
carry out at the "CACHED volume".
This way we have rename operations carried out where actual files are present.
So,Changelog recorded as :
@B1
CREATE f1
RENAME f1 f2
Note:
This patch is dependent on dht changes from this patch.
http://review.gluster.org/10410/
changelog related changes are separated out for review.
In changelog, xdata passed from DHT is considered as:
1. In case of unlink (internal operation as part of rename), xdata value
is set , it is considered as RENAME and recorded accordingly.
2. In case of rename (Hash and Cache different), xdata value is NOT
set, recording rename operation is SKIPPED.
BUG: 1219412
Change-Id: I7691166c84991482b2cfe073df64e2317c935b13
Reviewed-On: http://review.gluster.org/#/c/10220/
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: http://review.gluster.org/10633
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of: http://review.gluster.org/#/c/10134/
1) Provided setfattr command to set timeout for split-brain
choice.
2) If split-brain inspection/resolution is being done
from the mount for a file, ref the inode when
split-brain-choice is set.
This inode will be unconditionally unref-ed after timeout
seconds set by the user/default otherwise.
3) Updated the doc and testcase to reflect the changes.
Change-Id: I15c9037dee28855f21e680e7e3632e1f48dba4e1
BUG: 1219388
Reviewed-on: http://review.gluster.org/10134
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Signed-off-by: Anuradha <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/10679
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ia8d061de5be1343cc10a945f6cf011686a770d33
BUG: 1220020
Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/10144
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
(cherry picked from commit f6e24fc54453669f3943e50f44cce9bb6070a71a)
Reviewed-on: http://review.gluster.org/10674
Reviewed-by: Kaushal M <kaushal@redhat.com>
Tested-by: Kaushal M <kaushal@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Modified the calls to open in fops-sanity.c to pass in
the mode as well if flags includes O_CREAT (as per man page).
The missing mode randomly caused T files to be created causing DHT
to treat them as linkto files and fail the fop.
Modified 2 other files where the mode was not being provided.
Change-Id: I047573d43655b4957d0703f7df36238f7e729c1f
BUG: 1218940
Signed-off-by: Nithya Balachandran <nbalacha@redhat.com>
Reviewed-on: http://review.gluster.org/10590
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
(cherry picked from commit a661f7f54cef34aa39894818568a2c1b462e8cbc)
Signed-off-by: Nithya Balachandran <nbalacha@redhat.com>
Reviewed-on: http://review.gluster.org/10627
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1) convert to md format
2) Add info about ctr and libgfdb
Change-Id: I531d8a0bff8195f759302c5e613c7af2113729eb
BUG: 1218638
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/10665
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Tested-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
(cherry picked from commit 9c21c5a632ba22a6f46d037bd4fa4d825b24d07f)
Reviewed-on: http://review.gluster.org/10669
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Restructuring the RPM packages in order to reduce the dependencies that
get installed for glusterfs-api (mainly on request for Qemu hosts).
The dependencies of these packages look roughly like this:
.------------------.
| glusterfs-server |
'-------+----------'
|\
| \
| '---------------+--------------------.
| | |
| v v
| .----------------. .---------------.
| | glusterfs-fuse | | glusterfs-api |
| '--------+-------' '------+--------'
| /| /|
| / | / |
| / | / |
| / | / |
| .-----------+----=---------------' |
|/ | |
v v v
.-----------. .--------------------------.
| glusterfs | | glusterfs-client-xlators |
'-----------' '--------------------------'
With this structure, users can install glusterfs-server, glusterfs-fuse
or glusterfs-api (libgfapi) without getting any unneeded xlators or
other scripts/binaries.
The "glusterfs" and "glusterfsd" binary (symlinked) is now part of the
glusterfs-fuse package (moved from glusterfs). This does not make a
difference for glusterfs-server installations, because a server
installation always needs the glusterfs-fuse package for doing internal
mounts.
The advantage is that glusterfs-api does not pull in any executables
that get into the $PATH. This has caused confusion before when people
tried to remove the (wrongly assumed) server-only "glusterfsd" binaries.
Cherry picked from commit fd8b8a81c243776b20e6d48993524ee3f1351ca6:
> URL: http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10643
> Change-Id: Id03f1a634ea3c62ab7008345be92e01ccf43b1a6
> BUG: 1195947
> Signed-off-by: Niels de Vos <ndevos@redhat.com>
> Reviewed-on: http://review.gluster.org/10554
> Tested-by: Gluster Build System <jenkins@build.gluster.com>
> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
> Tested-by: NetBSD Build System
> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Change-Id: Id03f1a634ea3c62ab7008345be92e01ccf43b1a6
BUG: 1219089
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10697
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
BUG: 1219823
Change-Id: I3a1c7b7742671847ed3fec13e06d861c3d09f7a9
Reviewed-On: http://review.gluster.org/#/c/10687/
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10688
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- With this change, the xattr will represent if the file needs to be healed or
not. It will have different values for data/entry and metadata changes.
- inode ref leaks and dict_set_dynstr related leaks fixed
- Added support for trylock/lock based on heal-cmd execution or not
in data heal.
- Made fixes to pass regression runs
Change-Id: I9d8def4c2badde18a76b7898816fecfac113737a
BUG: 1216303
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/10385
Reviewed-on: http://review.gluster.org/10693
Tested-by: NetBSD Build System
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Data self-heal:
1) Take inode lock in domain 'this->name:self-heal' on 0-0 range (full file),
So that no other processes try to do self-heal at the same time.
2) Take inode lock in domain 'this->name' on 0-0 range (full file),
3) perform fxattrop+fstat and get the xattrs on all the bricks
3) Choose the brick with ec->fragment number of same version as source
4) Truncate sinks
5) Unlock lock taken in 2)
5) For each block take full file lock, Read from sources write to the sinks, Unlock
6) Take full file lock and see if the file is still sane copy i.e. File didn't become unusable while the bricks are offline.
Update mtime to before healing
7) xattrop with -ve values of 'dirty' and difference of highest and its own
version values for version xattr
8) unlock lock acquired in 6)
9) unlock lock acquired in 1)
Change-Id: I6f4d42cd5423c767262c9d7bb5ca7767adb3e5fd
BUG: 1216303
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/10384
Reviewed-on: http://review.gluster.org/10692
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Metadata self-heal:
1) Take inode lock in domain 'this->name' on 0-0 range (full file)
2) perform lookup and get the xattrs on all the bricks
3) Choose the brick with highest version as source
4) Setattr uid/gid/permissions
5) removexattr stale xattrs
6) Setxattr existing/new xattrs
7) xattrop with -ve values of 'dirty' and difference of highest and its own
version values for version xattr
8) unlock lock acquired in 1)
Entry self-heal:
1) take directory lock in domain 'this->name:self-heal' on 'NULL' to prevent
more than one self-heal
2) we take directory lock in domain 'this->name' on 'NULL'
3) Perform lookup on version, dirty and remember the values
4) unlock lock acquired in 2)
5) readdir on all the bricks and trigger name heals
6) xattrop with -ve values of 'dirty' and difference of highest and its own
version values for version xattr
7) unlock lock acquired in 1)
Name heal:
1) Take 'name' lock in 'this->name' on 'NULL'
2) Perform lookup on 'name' and get stat and xattr structures
3) Build gfid_db where for each gfid we know what subvolumes/bricks have
a file with 'name'
4) Delete all the stale files i.e. the file does not exist on more than
ec->redundancy number of bricks
5) On all the subvolumes/bricks with missing entry create 'name' with same
type,gfid,permissions etc.
6) Unlock lock acquired in 1)
Known limitation: At the moment with present design, it conservatively
preserves the 'name' in case it can not decide whether to delete it. this can
happen in the following scenario:
1) we have 3=2+1 (bricks: A, B, C) ec volume and 1 brick is down (Lets say A)
2) rename d1/f1 -> d2/f2 is performed but the rename is successful only on one
of the bricks (Lets say B)
3) Now name self-heal on d1 and d2 would re-create the file on both d1 and d2
resulting in d1/f1 and d2/f2.
Because we wanted to prevent data loss in the case above, the following
scenario is not healable, i.e. it needs manual intervention:
1) we have 3=2+1 (bricks: A, B, C) ec volume and 1 brick is down (Lets say A)
2) We have two hard links: d1/a, d2/b and another file d3/c even before the
brick went down
3) rename d3/c -> d2/b is performed
4) Now name self-heal on d2/b doesn't heal because d2/b with older gfid will
not be deleted. One could think why not delete the link if there is
more than 1 hardlink, but that leads to similar data loss issue I described
earlier:
Scenario:
1) we have 3=2+1 (bricks: A, B, C) ec volume and 1 brick is down (Lets say A)
2) We have two hard links: d1/a, d2/b
3) rename d1/a -> d3/c, d2/b -> d4/d is performed and both the operations are
successful only on one of the bricks (Lets say B)
4) Now name self-heal on the 'names' above which can happen in parallel can
decide to delete the file thinking it has 2 links but after all the
self-heals do unlinks we are left with data loss.
Change-Id: I3a68218a47bb726bd684604efea63cf11cfd11be
BUG: 1216303
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/10298
Reviewed-on: http://review.gluster.org/10691
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If both dicts are NULL then equal. If one of the dicts is NULL but the other
has only ignorable keys then also they are equal. If both dicts are non-null
then check if for each non-ignorable key, values are same or not. value_ignore
function is used to skip comparing values for the keys which must be present in
both the dictionaries but the value could be different.
geo-rep's stime xattr doesn't need to be present in list xattr but when
getxattr comes on stime xattr even if there aren't enough responses with the
xattr we should still give out an answer which is maximum of the stimes
available.
Change-Id: I8de2ceaa2db785b797f302f585d88e73b154167d
BUG: 1216303
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/10078
Reviewed-on: http://review.gluster.org/10690
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently the gluster namespace/module is owned by glusterfs-api rpm.
Because of this, other packages (extra-xlators or libgfapi-python)
have to depend on glusterfs-api to get the gluster namespace. The gluster
namespace should be owned by a core package.This patch introduce new package
called python-gluster and moves the ownership of gluster module
to python-gluster.
Cherry picked from commit cf1c53343d4b6fe834833b76c50c61fd8a312de0:
> Change-Id: Ib75ab922736f3bea6355da4d8dbec3a08b2844d5
> BUG: 1211848
> Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
> Reviewed-on: http://review.gluster.org/10244
> Tested-by: Gluster Build System <jenkins@build.gluster.com>
> Reviewed-by: Niels de Vos <ndevos@redhat.com>
> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Change-Id: Ib75ab922736f3bea6355da4d8dbec3a08b2844d5
BUG: 1219787
Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/10672
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
minor typo in glusterfs.spec.in prevents building RPMS
Change-Id: I876943e256d393921009d75cab84944f420632ba
BUG: 1219787
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/10689
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On BSD systems, ls(1) reports dot-files when it is run as root, while
Linux ls(1) does not do that. In ec-readdir.t, this means the directory
count is different because of .trashcan directory.
Fix this by using ls $M0/* instead of ls $M0
Backport of: Ia337c8a8e68a163ed527c0e4cbe313072ab1672b
BUG: 1212676
Change-Id: Ic3eb43f22a7c12a51e955b459af5159afc9835aa
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/10658
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adding 64 bits in "version" key of extended attributes. First 64 bits (Left)
represents Data version. Last 64 bits (right) represents Meta Data version.
Note: 3.7 and 3.6 version ec can't co-exist with this change because xattrop in
3.6 will fail with ERANGE as the buffer passed to it will be '8' bytes where as
the value will be 16 bytes in 3.7. Where as 3.7 version clients can work with
old version files. For upgrades we need to tell users to complete heals and
then upgrade
BUG: 1215265
Change-Id: Ib85114680cb7e75b8371c984d9f7b6401c1ffb93
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: http://review.gluster.org/10312
Reviewed-on: http://review.gluster.org/10626
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ia7d43cb3b222db34ecb0e35424f1766715ed8e6a
BUG: 1219358
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: http://review.gluster.org/10176
Reviewed-on: http://review.gluster.org/10625
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This issue introduced due to manual rebase.
Change-Id: I0589f4a0a1270190340f419b8022d6483bcf853d
Signed-off-by: Aravinda VK <avishwan@redhat.com>
BUG: 1219479
Reviewed-on: http://review.gluster.org/10685
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10686
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An empty changelog when rolled over gets unlinked and indexed with
a modified path-name in htime file. The modification is "changelog"
not "CHANGELOG" in basename of the empty changelog file.
BUG: 1219479
Change-Id: Ib5b825ab563fa34d8dcf4368cf6cbf4b25d78a6d
Original-Author: Ajeet Jha <ajha@redhat.com>
Original-Author: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-On: http://review.gluster.org/#/c/9572/
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10642
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
backport of http://review.gluster.org/#/c/10660/
The fix that introduced this test has been reverted as part of design change.
Hence marking it as a bad test.
Change-Id: I811b6050edbf4bbb40b9ef518604d21d386f6d5c
BUG: 1219782
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/10660
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
(cherry picked from commit a5d124728640f58fa3f90007c149ad69809b6ca4)
Reviewed-on: http://review.gluster.org/10663
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
inode quota is a new feature implemented in glusterfs-3.7
if quota is enabled in the older version and is upgraded
to a new version, we can hit setxattr spike during self-heal
of inode quotas. So, when a quota is enabled, turn off
inode-quotas with a xlator option.
With this patch, we still account for inode quotas but only
when a write operation is performed for a particular file.
User will be able to query inode quotas once the Inode-quota
xlator option is enabled.
Change-Id: I52fb28bf7024989ce7bb08ac63a303bf3ec1ec9a
BUG: 1218243
Signed-off-by: vmallika <vmallika@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/10152
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/10621
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a backport of fix 10435 to Gluster 3.7.
When we attach a tier, the hot tier becomes the hashed
subvolume. But directories may not yet have been replicated by
the fix layout process. Hence lookups to those directories
will fail on the hot subvolume. We should only go to the hashed
subvolume once the layout has been fixed. This is known if the
layout for the parent directory does not have an error. If
there is an error, the cold tier is considered the hashed
subvolume. The exception to this rules is ENOCON, in which
case we do not know where the file is and must abort.
Note we may revalidate a lookup for a directory even if the
inode has not yet been populated by FUSE. This case can
happen in tiering (where one tier has completed a lookup
but the other has not, in which case we revalidate one tier
when we call lookup the second time). Such inodes are
still invalid and should not be consulted for validation.
> http://review.gluster.org/#/c/10435/
> Change-Id: Ia2bc62e1d807bd70590bd2a8300496264d73c523
> BUG: 1214289
> Signed-off-by: Dan Lambright <dlambrig@redhat.com>
> Reviewed-on: http://review.gluster.org/10435
> Tested-by: Gluster Build System <jenkins@build.gluster.com>
> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
> Reviewed-by: N Balachandran <nbalacha@redhat.com>
> Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Change-Id: Ia2bc62e1d807bd70590bd2a8300496264d73c523
BUG: 1219547
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-on: http://review.gluster.org/10649
Tested-by: NetBSD Build System
Reviewed-by: Joseph Fernandes
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of: http://review.gluster.org/10631
To-Do:
* Make ftruncate work even in the absence of path
* Aggregate and update ia_blocks appropriately when a file is
truncated to a lower size.
Change-Id: Icd424430066233ba61a030e72fdddf692d2b3f22
BUG: 1214247
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/10638
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Background:
Glusterfs changelogs are stored in each brick, which records the changes
happened in that brick. Georep will run in all the nodes of master and
processes changelogs "independently".
Processing changelogs is in brick level, but all the fops will be replayed
on "slave mount" point.
Problem:
With a DHT volume, in changelog "internal fops" are NOT recorded.
For Rename case, Rename is recorded in "hashed" brick changelog.
(DHT's internal fops like creating linkto file, unlink is NOT recorded).
This lead us to inconsistent rename operations.
For example,
Distribute volume created with Two bricks B1, B2.
//Consider master volume mounted @ /mnt/master
and following operations executed:
cd /mnt/master
touch f1 // f1 falls on B1 Hash
mv f1 f2 // f2 falls on B2 Hash
// Here, Changelogs are recorded as below:
@B1
CREATE f1
@B2
RENAME f1 f2
Here, race exists between Brick B1 and B2, say B2 will get executed first.
Source file f1 itself is "NOT PRESENT", so it will go ahead and create
f2 (Current implementation).
We have this problem When rename falls in another brick and
file is unlinked in Master.
Similar kind of issue exists in following case too(multiple rename):
CREATE f1
RENAME f1 f2
RENAME f2 f1
Solution:
Instead of carrying out "changelogging" at "HASHED volume",
carry out at the "CACHED volume".
This way we have rename operations carried out where actual files are present.
So,Changelog recorded as :
@B1
CREATE f1
RENAME f1 f2
credit: sarumuga@redhat.com
PS: Some of the races as the one below are _NOT_ fixed by this patch
* f1 and f2 exist. B1 and B2 are their respective cached subvols. For
both files hashed-subvol == cached-subvol
* mv f1 f2 on master.
* B1 has change-log entry of rename f1 f2
* rebalance migrates f2 from B1 and B2
* mv f2 f1 on master.
* B2 has change-log entry of rename f2 f1
Since changelog entries (rename f1 f2) and (rename f2 f1) are processed
independently by gsyncds, which of either f1 and f2 survives on slave
is subject to race. Note that on master its file f1 with name f1 which
survived. On slave it can be either file f1 with name f1 or file f2
with name f2 based on who wins the race of processing changelog.
BUG: 1219412
Change-Id: I43725d69635e2ce065135691ef629014e8df7d50
Original-Author: Nithya Balachandran <nbalacha@redhat.com>
Reviewed-on: http://review.gluster.org/10410
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: http://review.gluster.org/10628
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As optional feature, during unlink, full path will be recorded.
Changelog Version number to be bumped up to 1.2.
With this patch, parser checks the version number before parsing
and handles accordingly.
Change-Id: Ic1ad98259c39e417029a08e26a1d4b467817e65a
BUG: 1218383
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10166
Reviewed-on: http://review.gluster.org/10620
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of http://review.gluster.org/9664
If a snap is activated or deactivated, when a node is down,
it is not retrieving the data properly during the handshake
of glusterd
With this patch, a version check will made when a glusterd
is started running. If there is a mismach in version, then
peers will exchange the healed data.
Change-Id: I8bd2a347723db2194d3fa73295878b4dd2e9be5d
BUG: 1219744
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/9664
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/10661
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Argparse Python library is available as standard library in Python 2.7
In rhel, we need to install python-argparse.
Also added pyxattr dependency to the spec file.
BUG: 1219457
Change-Id: I2949d43b68731498130dbd27bf03c87f101141f0
Reviewed-on: http://review.gluster.org/10321
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10637
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Default Values for last_synced, checkpoint_time and
checkpoint_completion_time was zero instead of 'N/A'
BUG: 1218586
Change-Id: I8a96688588dd6e22edf8bc06ee0f3afe12a9e0f5
Reviewed-on: http://review.gluster.org/10580
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10636
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When rsync is executed using Python subprocess, by default
stdout of subprocess will be None. With the log rsync performance
patch stdout is assigned to PIPE. Rsync writes to that PIPE
whenever it syncs files. If log_rsync_performance is disabled
then nobody will consume stdout and that gets full. Rsync hangs
if PIPE is full.
log_rsync_performance option is introduced with patch 10070
With this patch stdout=PIPE only if log_rsync_performance is
enabled. Also removed -v option from Rsync.
Thanks Venky and Kotresh for RCA.
BUG: 1219444
Change-Id: I4b06ca2ebdcb93ac8319f60bc637182fb3d38091
Original-Author: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/10556
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10634
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a backport of fix 10324 to Gluster 3.7.
If a read IO occurs against a file that has reached rebalance
phase 2, we redirect the IO to the destination. For tiered
volumes, when we try to reopen the file (on the destination),
the lower level DHT receives the open call and fails; it does
not have a "cached subvol". Fix is to "teach" the lower level
DHT of the new location by sending a locate before the open.
> http://review.gluster.org/#/c/10324/
> Change-Id: Ia4acb0035ff1da15f6a8f9ed54f43c76e8b98f5f
> BUG: 1214048
> Signed-off-by: Dan Lambright <dlambrig@redhat.com>
> Signed-off-by: root <root@gprfs018.sbu.lab.eng.bos.redhat.com>
> Signed-off-by: Dan Lambright <dlambrig@redhat.com>
> Reviewed-on: http://review.gluster.org/10324
> Tested-by: NetBSD Build System
> Tested-by: Gluster Build System <jenkins@build.gluster.com>
> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
> Tested-by: Raghavendra G <rgowdapp@redhat.com>
> Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Change-Id: Ia4acb0035ff1da15f6a8f9ed54f43c76e8b98f5f
BUG: 1219608
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-on: http://review.gluster.org/10654
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Joseph Fernandes
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: The CTR xlator records file meta (heat/hardlinks)
into the data. This works fine for files which are created
after ctr xlator is switched ON. But for files which were
created before CTR xlator is ON, CTR xlator is not able to
record either of the meta i.e heat or hardlinks. Thus making
those files immune to promotions/demotions.
Solution: The solution that is implemented in this patch is
do ctr-db heal of all those pre-existent files, using named lookup.
For this purpose we use the inode-xlator context variable option
in gluster.
The inode-xlator context variable for ctr xlator will have the
following,
a. A Lock for the context variable
b. A hardlink list: This list represents the successful looked
up hardlinks.
These are the scenarios when the hardlink list is updated:
1) Named-Lookup: Whenever a named lookup happens on a file, in the
wind path we copy all required hardlink and inode information to
ctr_db_record structure, which resides in the frame->local variable.
We dont update the database in wind. During the unwind, we read the
information from the ctr_db_record and ,
Check if the inode context variable is created, if not we create it.
Check if the hard link is there in the hardlink list.
If its not there we add it to the list and send a update to the
database using libgfdb.
Please note: The database transaction can fail(and we ignore) as there
already might be a record in the db. This update to the db is to heal
if its not there.
If its there in the list we ignore it.
2) Inode Forget: Whenever an inode forget hits we clear the hardlink list in
the inode context variable and delete the inode context variable.
Please note: An inode forget may happen for two reason,
a. when the inode is delete.
b. the in-memory inode is evicted from the inode table due to cache limits.
3) create: whenever a create happens we create the inode context variable and
add the hardlink. The database updation is done as usual by ctr.
4) link: whenever a hardlink is created for the inode, we create the inode context
variable, if not present, and add the hardlink to the list.
5) unlink: whenever a unlink happens we delete the hardlink from the list.
6) mknod: same as create.
7) rename: whenever a rename happens we update the hardlink in list. if the hardlink
was not present for updation, we add the hardlink to the list.
What is pending:
1) This solution will only work for named lookups.
2) We dont track afr-self-heal/dht-rebalancer traffic for healing.
> http://review.gluster.org/#/c/10370/
> Cherry picked from commit cb11dd91a6cc296e4a3808364077f4eacb810e48
> Change-Id: Ia4bbaf84128ad6ce8c3ddd70bcfa82894c79585f
> BUG: 1212037
> Signed-off-by: Joseph Fernandes <josferna@redhat.com>
> Signed-off-by: Dan Lambright <dlambrig@redhat.com>
> Reviewed-on: http://review.gluster.org/10370
> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
> Tested-by: Gluster Build System <jenkins@build.gluster.com>
> Tested-by: NetBSD Build System
> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Change-Id: I367aa46c3f4b8f912248fb8be75866507f2538df
BUG: 1219075
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-on: http://review.gluster.org/10370
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/10615
Tested-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently when quota limit is set, corresponding gfid
is set in quota.conf. This patch supports storing
inode-quota limits in quota.conf and also stores
additional byte for each gfid to differentiate
between usage quota limit and inode quota limit.
Change-Id: I444d7399407594edd280e640681679a784d4c46a
BUG: 1218170
Signed-off-by: vmallika <vmallika@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/10069
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/10524
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
backport of http://review.gluster.org/#/c/9971/
Anonymous fd's are floating fd assigned to a glusterfs client
without a explicit file open. Here either it will create a new
anonymous fd or existing anonymous fd in the client stack for
requested file.The anonymous fd's are mainly used for IO's.
This patch introduces two api's glfs_h_anonymous_read and
glfs_h_anonymous_write which performs read and write respectively
cherry-picked as fa0ad231745846918b2625d0e1a89c0a5c3c24dc
>Change-Id: Id646f2220e8387b2f8bb244c848dc1db6761444f
>BUG: 1204651
>Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
>Reviewed-by: Niels de Vos <ndevos@redhat.com>
>Reviewed-on: http://review.gluster.org/9971
>Tested-by: Gluster Build System <jenkins@build.gluster.com>
>Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
Change-Id: I6b01d88f92ad045e48debee23aa79f4517c6bdc2
BUG: 1218857
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
Reviewed-on: http://review.gluster.org/10635
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The generation number for each peerinfo object is unique. It can be used
to find the exact peerinfo object, which is required for peer RPC
notifications.
Using hostname and uuid matching to find peerinfos can return incorrect
peerinfos to be returned in certain cases like multi network peer probe.
This could cause updates to happen to incorrect peerinfos.
Change-Id: Ia0aada8214fd6d43381e5afd282e08d53a277251
BUG: 1215018
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/10495
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
(cherry picked from commit 02583099a219ce327aac62af22b486c7b9fcb531)
Reviewed-on: http://review.gluster.org/10623
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Changelog processing is done in batch, for example if 10 changelogs
available for processing then process all at once. Collect Entry, Meta
and Data operations separately, All the entry operations like CREATE,
MKDIR, MKNOD, LINK, UNLINK will be executed first then rsync will be
triggered for whole batch. Stime will get updated once the complete
batch is complete.
In case of large number of Changelogs in a batch, If geo-rep fails after
Entry operations, but before rsync then on restart, it again starts from the
beginning since stime is not updated. It has to process all the changelogs
again. While processing same changelogs again, all CREATE will get EEXIST
since all the files created in previous run. Big hit for performance.
With this patch, Geo-rep limits number of changelogs per batch based on
Changelog file size. So that when geo-rep fails it has to retry only last batch
changelogs since stime gets updated after each batch.
BUG: 1217930
Change-Id: I844448c4cdcce38a3a2e2cca7c9a50db8f5a9062
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/10202
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/10499
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On Linux, kernel cache can be flushed using
echo 3 > /proc/sys/vm/drop_caches
This non-portable approach can be replaced by an on-purpose
failed attempt to unmount: if the mount point is the current
directory and umount is called, the kernel will flush inodes
until it realize it cannot complete the operation because
root of filesystem is busy:
( cd $M0 ; umount $M0 )
Unfortunately this does not flush everything. Entries may
still be present in the kenrel FUSE cache. Using $GFS to
mount the filesystem ensure --entry-timeout=0 and clears
this problem.
Some stall information may also remain in glusterfs caches,
and that may have to be adressed by appropriate volume option.
For instance tests/bugs/rpc/bug-954057.t needs to disable
performance.stat-prefetch. Qtherwise, root's new credentials
are not evaluated after root-quash is enabled. The test could
also be done with performance.stat-prefetch enabled using
various tricks: copying the file to read, creating a hard link
on it, or just waiting long enough for metadata cache to expire.
Backport of: I54929e899d55c04dcd9d947809133549f01fd0e1
BUG: 1212676
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Change-Id: I2849a27acaa0334ef30aae3b852019b5a6eeb419
Reviewed-on: http://review.gluster.org/10648
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ut
1) The glupy.so xlator should embed the runtime search path for
the python libraries. Unfortunately, python-config does not
gives the appprioate flags, therefore we need to also use
pkg-config to obtain them
2) Fix the glupy python module directory layout so that python
can import the module without problem
That two fixes seems to let glupy.t pass on NetBSD again.
Backport of: I397aa726ab8bf7d91fa0d6d870a30910a5f4a5d9
BUG: 1212676
Change-Id: Ie48916d71f0f1a357d65c3c22b5e7d7276d720db
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/10650
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It seems possible that auth_cache->cache_dict is not always allocated
before it is accessed. Instead of allocating the dict upon the 1st
access, just create it in auth_cache_init().
Cherry picked from commit eb8847703b8560a045e7ed0336f895bcceda98ea:
> Change-Id: I00e60522478b433cb0aae0c1f0948eac544dfd2b
> URL: http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10710
> BUG: 1143880
> Signed-off-by: Niels de Vos <ndevos@redhat.com>
> Reviewed-on: http://review.gluster.org/10600
> Tested-by: Gluster Build System <jenkins@build.gluster.com>
> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
> Tested-by: NetBSD Build System
Change-Id: I00e60522478b433cb0aae0c1f0948eac544dfd2b
BUG: 1212182
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10655
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
NFS-Ganesha related config files have to be copied over to the new node
and NFS-Ganesha service has to be started.
Similary NFS-Ganesha service has to be stopped when a node is
deleted from the HA cluster.
This is the backport of the patch merged on master,
http://review.gluster.org/#/c/10596/
Change-Id: I21f09e30a738d2ba01861ce480ecf906667d887b
BUG: 1218884
Signed-off-by: Meghana Madhusudhan <mmadhusu@redhat.com>
Reviewed-on: http://review.gluster.org/10632
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change broke the build on NetBSD, FreeBSD, and MacOS X:
http://review.gluster.org/10526/
We restore the build with two fixes:
- Use POSIX-compliant sysconf(_SC_NPROCESSORS_ONLN) to get the
number of processors, instead of Linux specific get_nprocs().
That let us remove Linux-specific #include <sys/sysinfo.h>
- Only define MAX() if it is not already defined. NetBSD defines
it in <sys/param.h> which is already included
Backport of: I62341c670598670e47ea2f69ab94864f96588b18
BUG: 1212676
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Change-Id: I0f098153e76954bb85b5dca3f054a069e31dd94c
Reviewed-on: http://review.gluster.org/10653
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
|