| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
uid is sent in place of gid while CREATE and
MKDIR.
BUG: 1159213
Change-Id: I711f383cef9de7467df9e60774d880eb5e1a642a
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7984
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9027
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When a slave node goes down, worker in master node
can connect to different slave node and resume the operation.
Existing georep was not checking the status of slave node before
worker restart. With this patch, geo-rep worker will check the
node status using `gluster volume status` when it goes faulty.
BUG: 1159209
Change-Id: I7ca2a66ff2a438435f297b7063313214c28f3d4b
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/8921
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9025
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When changelog is failed to process even after max
retries, files are skipped but were not logging.
This patch addresses this issue to log the skipped
files.
BUG: 1159198
Change-Id: I1f3d62cb61a940ee5e10e5a79e08fa65e1c15e97
Reviewed-on: http://review.gluster.org/7945
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/9022
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The rename of directories are captured in all distributed
brick changelogs. gsyncd processess these changelogs on
each brick parallellaly. The first changelog to get processed
will be successful. All subsequent ones will stat the 'src'
and if not present, tries to create freshly on slave. It
should be done only for files and not for directories.
Hence when this code path was hit, regular file's blob
is sent as directory's blob and gfid-access translator
was erroring out as 'Invalid blob length' with errno as
'ENOMEM'
Change-Id: I50545b02b98846464876795159d2446340155c82
BUG: 1147422
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8865
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/8880
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While processing RENAME in changelog, if the file is unlinked
in master, then geo-rep was sending UNLINK to slave instead of
RENAME.
If rsync job fails if one of the file failed to sync in the job.
This patch adds logic to remove GFID from data list if the same
changelog has UNLINK entry for it after the DATA. Or it removes
those GFIDs during retry of changelogs processing.
BUG: 1147420
Change-Id: I982dc976397cd0ab676bb912583f66a28f821926
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/8761
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/8879
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While identifying the file/dir to sync, xtime of the file was compared
with xsync_upper_limit as `xtime < xsync_upper_limit` After the sync,
xtime of parent directory is updated as stime. With the upper limit
condition, stime is updated as MIN(xtime_parent, xsync_upper_limit)
With this files will get missed if `xtime_of_file == xsync_upper_limit`
With this patch xtime_of_file is compared as
xtime_of_file <= xsync_upper_limit
BUG: 1138952
Change-Id: I469e8638ab6923e518022a539a19e2d040b60eb0
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/8439
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/8644
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If RMDIR is recorded in brick changelog which is due to
self heal traffic then it will not have UNLINK entries for
child files. Geo-rep hangs with ENOTEMPTY error on slave.
Now geo-rep recursively deletes the dir if it gets ENOTEMPTY.
BUG: 1138952
Change-Id: Ie79db90c52103b39fa795bb8a096b363d450b427
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/8477
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/8643
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For effective handling of deletes and renames use history crawl
as much as possible. History crawl will run in loop till it syncs
all data before live changelog time.
When it uses xsync crawl(fallback when changelog not available, or
very first crawl) it sets upper limit to crawl.
After completing History crawl, it checks actual end time returned
by history api to compare with register time, if actual end is less
than register time then run history crawl one more time.
If first turn history processing time is less than the CHANGELOG
ROLLOVER TIME then sleep for the difference, After sleep if it is
guaranteed that rollover will happen and switches to live changelog
consumption without switching to xsync. This sleep is only when
history processing completed < CHANGELOG_ROLLOVER_TIME and sleep
only after the first turn, So will not affect the performance.
BUG: 1138952
Change-Id: Ida024211d312f60f0e8190805e7469b2165f00e1
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/8151
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/8639
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport from master branch - http://review.gluster.org/#/c/8246/
- Break-way from '/var/lib/glusterd' hard-coded previously,
instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
platforms.
- Define proper environment for running tests, define correct PATH
and LD_LIBRARY_PATH when running tests, so that the desired version
of glusterfs is used, regardless where it is installed.
(Thanks to manu@netbsd.org for this additional work)
Change-Id: I06e684ac4c26d1e74c9daf76753403ad15f79276
BUG: 1130308
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/8486
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When A file/dir is identified for metadata sync, it was doing
duplicate stat to get the metadata to sync.
With this patch it avoids doing one additional stat call. Xsync
performance will improve.
rsync will copy files metadata, so no need to include for
processing.
BUG: 1111490
Change-Id: I79dad6375fa4742d9aaca7d9856993c184a744dc
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/8124
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Geo-rep does history crawl even if change-detector is set
to xsync.
This patch fixes by taking priority to user configured
change detector.
BUG: 1113525
Change-Id: Ic6c34e187c9cb6608c9ef8a010ea07015ba60a80
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/8183
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
worker and agent uses pipe to communicate, if worker dies
for some reason agent should get EOF and terminate.
Each worker-agent spawning is done in thread, Due to race
if multiple workers in same node retain the pipe refs of
other workers. Hence agent will not get EOF even if
worker dies.
BUG: 1114003
Change-Id: I36b9709b9392299483606bd3ef1db764fa3f2bff
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/8194
Tested-by: Justin Clift <justin@gluster.org>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On soft reboot, geo-rep monitor is writing 'faulty' into status
file. It should not do it if previous state is paused as
glusterd depend on the state file on node restart.
Change-Id: Idd45abf13350b087371935f1b4f6e1a346433d27
BUG: 1101410
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8097
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Monitor process spawns changelog agent and is not
wait on it, hence becoming zombie. When worker is
dies/killed, it respawns both worker and corresponding
agent leaving the earlier changelog agent in zombie
state. This patch addresses this issue by waiting
on agent process in montor process.
Change-Id: I571b7d6487133848edca67e7446f1caa70ae01c9
BUG: 1103643
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/7956
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On many linux distributions with iptables enabled, ICMP traffic
is usually dropped even when port 22 is open for SSH service
So practically `ping` is an unreliable command
~~~
root@rhs1:/var/log/glusterfs # gluster volume geo-replication geo-test \
17.16.10.1::geo-test-slave create push-pem force
172.16.10.1 not reachable.
geo-replication command failed
~~~
~~~
root@rhs1:/var/log/glusterfs # ping 172.16.10.1
PING rhs2.sjc.redhat.com (172.16.10.1) 56(84) bytes of data.
From rhs2.sjc.redhat.com (172.16.10.1) icmp_seq=1 Destination Host Prohibited
...
...
~~~
~~~
root@rhs2:/var/log/glusterfs # service iptables status | grep 22
4 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
root@rhs2:/var/log/glusterfs # service iptables status | grep icmp-host-prohibited
25 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
~~~
Change-Id: I33206ca071aa5d755c0762f7c486da222ec3c7db
BUG: 1105337
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/7997
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Replica failover check interval is hardcoded to 60 sec
by default. Now this option is made configurable and
defaulted to 1 sec.
To change the default value
gluster volume geo-replication <MASTERVOL> \
<SLAVEHOST>::<SLAVEVOL> config replica_failover_interval 15
Change-Id: Iada1b80d510452dcfedebd8a21bebd62394b0597
BUG: 1066410
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/8003
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Xattr.lsetxattr_l call will not raise OSError which
errno_wrap can handle, so we need to use Xattr.lsetxattr
to get proper OSError and errno_wrap ignores or retries
accordingly.
Change-Id: Ie0a777152ddbaf9ed80c977e4704974fec997bea
BUG: 1105083
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7972
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Upgrading "working_dir" on the fly is a bit unclean yet
(though it works) as currently config upgrade does not
support "old" values to be expanded by using configuration
variables.
Change-Id: I44ed65c281f2e0ce3b6b467addc5c1c88ac674e7
BUG: 1077516
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Reviewed-on: http://review.gluster.org/7375
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added "metadata" record for directory and file creations
during the intial crawl.
Change-Id: I811ae26e0144cadf7249cb64541ec354ab83fe66
BUG: 1106604
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/8018
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A new gsyncd options '--pause-on-start' is introduced. When node
reboots, if the status is paused, gsyncd is started with this
option. After gsyncd spawns worker and agent, worker will send
SIGSTOP to negative pid of monitor to enter pause mode.
Change-Id: I5aad82c9a9fc8c243f384940b77d25e26e520d6d
BUG: 1101410
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/7885
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also removing multiple mounts in gverify.sh
Change-Id: I1567a9f711222c5f571a12f7c4ab49cef32d60c9
BUG: 1101948
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/7911
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Required for unprivileged geo-replication sessions to
execute glusterd commands (which are however restricted).
Change-Id: Ib83b81defa061717f4465ffa665450d0f5d3d20d
BUG: 1077452
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/7833
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
unprivileged geo-replication session runs the slave gsyncd
process as unprivileged, thereby executing gluster cli commands
as an unprivileged user. By default, cli to glusterd
uses unix domain sockets, thereby restricting cli command
execution by non root users.
This patch introduces '--remote-host' cli option to force
cli to use INET socket. For this to work, the following
needs to be added in glusterd volfile
option rpc-auth-allow-insecure on
Change-Id: I84b1711281bbcbde156200f80ebdb065afb55488
BUG: 1077452
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/7820
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With an unprivileged geo-replication session, monitor
was using user@slave for --remote-host option for gluster
cli, thereby failing to sucessfully connect to the slave
glusterd.
This patch fixes the issue by selecting the hostname/IP
from the speicified slave endpoint url.
- For privileged geo-replication sessions, this patch
has no effect as the slave endpoint url is just the
hostname/IP.
Change-Id: I88f66c406a8d9a34db7fc626965f949075e3ceac
BUG: 1077452
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/7818
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also adding -P option to the usage of df for portability in gverify.sh
Change-Id: I0be19d26ea63769a934c6ccbfc04ef80768ebc9a
BUG: 1099041
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/7812
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
$HOME might not be set in the env variables, as is the case
when these scripts are executed using the runner framework.
Hence using getent passwd instead of $HOME
Change-Id: I99f6bcd788d727be534b3040600d66c8dbb7ee92
BUG: 1099041
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/7803
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Change-Id: If1d2cab3fcfe2391105551e54f0b9729a7c204e4
BUG: 1077452
Reviewed-on: http://review.gluster.org/7767
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
to operate for non-root privileged slave volume
Mounting the slave-volume on local node, to perform disk checks
in order to allow gverify.sh to operate for non-root privileged
slave volume
Allowing the hook script S56glusterd-geo-rep-create-post.sh
to operate for non-root privileged slave volume
Modified peer_add_secret_pub.in to accept username as argument
and add the pem keys to the users's_home_dir/.ssh/authorized_keys
Wrote set_geo_rep_pem_keys.sh which accepts username as argument
and copies the pem keys from the user's home directory to
$GLUSTERD_WORKING_DIR/geo-replication/ and then copies the keys
to other nodes in the cluster and add them to the respective
authorized keys. The script takes as argument the user name and
assumes that the user will be present in all the nodes in the
cluster. It is not needed for root.
To summarize:
For a privileged slave user, execute the following on master node as super user:
gluster system:: execute gsec_create
gluster volume geo-replication <master_vol> [root@]<slave_ip>::<slave_vol> create push_pem
For a non-privileged slave user execute the following on master node as super user:
gluster system:: execute gsec_create
gluster volume geo-replication <master_vol> <slave_user>@<slave_ip>::<slave_vol> create push_pem
then on the slave node execute the following as super user:
/usr/local/libexec/glusterfs/set_geo_rep_pem_keys.sh <slave_user>
BUG: 1077452
Change-Id: I88020968aa5b13a2c2ab86b1d6661b60071f6f5e
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/7744
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch enables geo-replication to be run as an unprivileged
user. As of now, this is just the partial support, but is very
close to achieve full functionality.
Current limitation
* Geo-replication executed Gluster CLI commands on the slave
via SSH. On a non-root setup, Gluster CLI would run as an
unprivileged user, failing to execute the command. As a
workaround (for testing), setuid(2) Gluster CLI executable
or use the glusterd option to accept commands by unprivileged
CLI process. The nature of cli commands are "system::"
commands (for key management) and remote volume info fetching.
Remote volume info fetching has been modified to use --remote-host
gluster cli option rather than ssh and remote cli execution.
Change-Id: Ica89e2ba9b7f48fd6e1c876c477d7822dc693617
BUG: 1077452
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/7658
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Additional argument added to API gf_history_changelog,
actual_end - The end time till where changelogs are available.
Added sort to history_get_changes API output.
BUG: 1091961
Change-Id: Id043409882a83cd0a7b9adc3d34d5147d17e532e
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7747
Reviewed-by: ajeet jha <ajha@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Changelog consumption/processing now happens in seperate process
group than monitor. When monitor process group gets SIGSTOP all
worker process, ssh, rsync will be paused except the changelog
processing. When it gets SIGCONT it resumes its operation.
Changelog agent runs as RepceServer, geo-rep worker communicates
with changelog agent using RepceClient.
Change-Id: I35c333e4d8b13d03a7808aed601960eef23cfa04
BUG: 1093602
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7322
|
|
|
|
|
|
|
|
|
|
|
| |
Number of parallel threads to process changelog history
is made configurable via sync_jobs
Change-Id: Idcd8e655d9df540cfa48648b9e98af941f95e9d0
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7660
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In source install, libgfchangelog is installed in /usr/local/lib
When glusterd runs /usr/local/libexec/glusterfs/python/gsyncd --version
it fails to find library without LD_LIBRARY_PATH.
This patch avoids loading library when it is run from glusterd
during start.
BUG: 1096026
Change-Id: I59912227ac27ff4877d947a7c8f1fe2e8c5be06e
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7713
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
commit 7fba3a8 enables memory accounting by default. Since
geo-replication binary (gsyncd) does not declare a memory
accounting function, call like GF_{CALLOC,MALLOC} result
int asserts.
This patch initializes default memory accounting.
Change-Id: I9ad1de8bca6745a9899a006a863a7cfeef73d52f
BUG: 1094708
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/7683
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Every time when geo-rep restarts it first does FS crawl using
XCrawl and then switches to Changelog Mode. This is because changelog
only had live API, that is we can get changes only after registering.
Now this(http://review.gluster.org/#/c/6930/) patch introduces History
API for changelogs. If history is available then geo-rep will use it
instead of FS Crawl.
History API returns TS till what time history is available for
given start and end time. If TS < endtime then switch to FS Crawl.
(History => FS Crawl => Live Changelog)
If TS >= endtime, then switch directly to Changelog mode
(History => Live Changelog)
Change-Id: I4922f62b9f899c40643bd35720e0c81c36b2f255
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/6938
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Sorry for the bug, which got introduced due to code refactoring.
http://review.gluster.org/#/c/7311/
Change-Id: Ide519ca114aa8a7d7624d7af99945c857f069069
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7417
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pep8 is a style guide for python.
http://legacy.python.org/dev/peps/pep-0008/
pep8 can be installed using, `pip install pep8`
Usage: `pep8 <python file>`, For example, `pep8 master.py`
will display all the coding standard errors.
flake8 is used to identify unused imports and other issues
in code.
pip install flake8
cd $GLUSTER_REPO/geo-replication/
flake8 syncdaemon
Updated license headers to each source file.
Change-Id: I01c7d0a6091d21bfa48720e9fb5624b77fa3db4a
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7311
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When a worker process not confirmed within 60 seconds of start
then monitor thread was terminated instead of stopping and restarting
the worker thread.
Before terminate monitor thread tries to add a signal handler for
SIGTERM to cleanup the stuff before terminate. Signal handling will
not work inside thread, so ValueError was raised.
This patch will not terminate monitor thread, instead only kills and
restarts the worker.
Change-Id: I14df26c0cc3097af29293c81536c13b86075e28f
BUG: 1078068
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7294
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remote xtime is required for failover/failback,
this patch is quick fix to avoid the OSError.
Code is masked out, this need to be resolved when
failover/failback is worked on.
Change-Id: If339d88a2ccd8ef18a3b3c015df765c93dcb020c
BUG: 1073844
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/7206
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When old config file is used with new geo-rep, config item like
'georep_session_working_dir' was missing in old config file.
With this patch geo-rep sets the default value for new items.
Following config options supported:
- georep_session_working_dir
- gluster_params
- ssh_command_tar
BUG: 1036539
Change-Id: I389c62e749f3b567f9ecf96d4b41367ef962c025
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/6934
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I851a50e0ebb7554b821fb591375bb6ebd6240ade
BUG: 1036539
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/6939
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
"commit ef4bb42382a5c9c3ca0907136669a659c2a20311" was incomplete since
reduction in sizes doesn't really have any affect on the problem when
disks sizes are similar.
Actual fix is to verify disk_size and then verify available_size
Change-Id: Idb57f606df229c0bdaeda8852c74e84d29109ffc
BUG: 1020154
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/6844
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ida4890abdc90d683a4a83127a1573bbb3829ea23
BUG: 1036539
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/6793
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ide927759c6a3d5301475eac9f6e785aa901d426e
BUG: 1036539
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/6792
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In cases (mostly upgrade) of unavailability of "stime" key and
availability of "xtime" (slave's xtime), introduce "stime" key
on the fly by setting it to the value to "xtime".
Change-Id: Iaa424662d838154c8abc2cf00830c7f9d6be45ac
BUG: 1036539
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/6791
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
A missing cleanup for the "tar" process (when tar+ssh
is used as the sync engine).
Change-Id: Ib9599b43e7ec606c70b7c5598793417142be3c0b
BUG: 1036539
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/6794
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When config.read is called it preserves the previous values
from the the previously opened config file.
For example
import ConfigParser
config = ConfigParser.RawConfigParser()
config.read("defaults.conf")
config.read("preferences.conf")
When change in config file is identified it will open new
instance of config to avoid getting old config values.
Change-Id: Iec677e61ebd2c59c95aea94481f569d78bd913e4
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/6747
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
in df command output, default block-size is 1K, but BUFFER_SIZE is
in bytes, so verify will succeed even if slave size is less than
master size.
Added -B1 to df command(that is --block-size=1) to get all values in
bytes.
BUG: 1056518
Change-Id: I2384d2a6f2505d967e0538b95fbf111c15b26f30
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/6746
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since xsync crawl generates new changelog when number of entries
reaches 8K or when directory is reached. If a directory has number
of files less than 8K then respective changelog file will have less
entries. Since xsync generated changelog files processed one after
the other, so syncjobs are underutilized. hence low bandwidth
utilization.
With this patch, changelog will be generated for 8K entries only,
but stime will be accumulated. Multiple dirs stime will be
updated together since the generated changelog will have entries
accross the dirs.
Change-Id: Ib0b40962a070f855f47f887d0840e412fb7928e1
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/6744
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
-> "threaded" hybrid crawl.
-> Enabling metatadata synchronization.
-> Handling EINVAL/ESTALE gracefully while syncing metadata.
-> Improvments to changelog crawl code.
-> Initial crawl changelog generation format.
-> No gsyncd restart when checkpoint updated.
-> Fix symlink handling in hybrid crawl.
-> Slave's xtime key is 'stime'.
-> tar+ssh as data synchronization.
-> Instead of 'raise', just log in warning level for xtime missing cases.
-> Fix for JSON object load failure
-> Get new config value after config value reset.
-> Skip already processed changelogs.
-> Saving status of each individual worker thread.
-> GFID fetch on slave for purges.
-> Add tar ssh keys and config options.
-> Fix nlink count when using backend.
-> Include "data" operation for hardlink.
-> Use changelog time prefix as slave's time.
-> Process changelogs in parallel.
Change-Id: I09fcbb2e2e418149a6d8435abd2ac6b2f015bb06
BUG: 1036539
Signed-off-by: Ajeet Jha <ajha@redhat.com>
Reviewed-on: http://review.gluster.org/6404
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|