summaryrefslogtreecommitdiffstats
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
* tests: Introduce a Vagrant VM based test environmentRaghavendra Talur2015-12-288-0/+155
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This introduces a mechanism using which a developer could easily test the Gluster code in a VM environment. Also, it will help bring uniformity in the environments used by various developers. How to use: 1. git checkout -b custom-branch-name 2. Make changes 3. Execute ./run-tests-in-vagrant.sh What happens in the background: 1. A new directory is created: tests/vagrant/vagrant-custom-branch-name It will serve as the Vagrant dir which has the Vagrantfile and related ansible playbooks. The VM is started using Vagrant and provisioned using ansible. 2. The source dir is recursively copied over to the VM under /home/vagrant/glusterfs. 3. Gluster is source installed in VM. What happens in the foreground: 1. run-tests.sh is executed in VM using ssh and output is displayed in the same terminal with option to use ctrl-c to interrupt the test midway. The VM would still persist and you could ssh into it. Also, you can checkout a different branch elsewhere and execute run-tests-in-vagrant.sh there to get another VM which would execute tests on that code. If you wish to make some changes in the code, you could: a. Change the code in host and run the script again to repeat the whole process. OR b. vagrant ssh into the VM and make the changes in the VM. Co-authored-by: Kaushal M <kaushal@redhat.com> Co-authored-by: Michael Adam <obnox@samba.org> Change-Id: Ic87801172c8b614cdecbdf2a765e1b3370a5faf7 BUG: 1291537 Signed-off-by: Michael Adam <obnox@samba.org> Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/12753 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* Tier: "tier start force" command implementationhari gowtham2015-12-221-0/+61
| | | | | | | | | | | | | | | | The start command doesnt restart the tier deamon if the deamon is running at one node. hence to bring up the tierd on the nodes where the deamon is down, the force command is implemented. It skips the check for tierd running. Change-Id: I0037d3e5ecfe56637d0da201a97903c435d26436 BUG: 1292112 Signed-off-by: hari gowtham <hgowtham@redhat.com> Reviewed-on: http://review.gluster.org/12983 Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* core: add preadv, pwritev, pread, pwrite syscall wrappersKaleb S KEITHLEY2015-12-221-4/+10
| | | | | | | | | | | | | add additional system calls plus pick up a couple missed unwrapped system calls that seem to have slipped into the master branch. Change-Id: If268ccd5e9a139ac3ffd38293c67cd2f62ea5b58 BUG: 1289258 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/12895 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* performance/write-behind: retry "failed syncs to backend"Raghavendra G2015-12-223-35/+162
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. When sync fails, the cached-write is still preserved unless there is a flush/fsync waiting on it. 2. When a sync fails and there is a flush/fsync waiting on the cached-write, the cache is thrown away and no further retries will be made. In other words flush/fsync act as barriers for all the previous writes. The behaviour of fsync acting as a barrier is controlled by an option (see below for details). All previous writes are either successfully synced to backend or forgotten in case of an error. Without such barrier fop (especially flush which is issued prior to a close), we end up retrying for ever even after fd is closed. 3. If a fop is waiting on cached-write and syncing to backend fails, the waiting fop is failed. 4. sync failures when no fop is waiting are ignored and are not propagated to application. For eg., a. first attempt of sync of a cached-write w1 fails b. second attempt of sync of w1 succeeds If there are no fops dependent on w1 are issued b/w a and b, application won't know about failure encountered in a. 5. The effect of repeated sync failures is that, there will be no cache for future writes and they cannot be written behind. fsync as a barrier and resync of cached writes post fsync failure: ================================================================== Whether to keep retrying failed syncs post fsync is controlled by an option "resync-failed-syncs-after-fsync". By default, this option is set to "off". If sync of "cached-writes issued before fsync" (to backend) fails, this option configures whether to retry syncing them after fsync or forget them. If set to on, cached-writes are retried till a "flush" fop (or a successful sync) on sync failures. fsync itself is failed irrespective of the value of this option, when there is a sync failure of any cached-writes issued before fsync. Change-Id: I6097c9257bfb9ee5b15616fbe6a0576ae9af369a Signed-off-by: Raghavendra G <rgowdapp@redhat.com> BUG: 1279730 Reviewed-on: http://review.gluster.org/12594
* cluster/afr: Fix data loss due to race between sh and ongoing writeKrutika Dhananjay2015-12-221-0/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: When IO is happening on a file and a brick goes down comes back up during this time, protocol/client translator attempts reopening of the fd on the gfid handle of the file. But if another client renames this file while a brick was down && writes were in progress on it, once this brick is back up, there can be a race between reopening of the fd and entry self-heal replaying the effect of the rename() on the sink brick. If the reopening of the fd happens first, the application's writes continue to go into the data blocks associated with the gfid. Now entry-self-heal deletes 'src' and creates 'dst' file on the sink, marking dst as a 'newentry'. Data self-heal is also completed on 'dst' as a result and self-heal terminates. If at this point the application is still writing to this fd, all writes on the file after self-heal would go into the data blocks associated with this fd, which would be lost once the fd is closed. The result - the 'dst' file on the source and sink are not the same and there is no pending heal on the file, leading to silent corruption on the sink. Fix: Leverage http://review.gluster.org/#/c/12816/ to ensure the gfid handle path gets saved in .glusterfs/unlink until the fd is closed on the file. During this time, when self-heal sends mknod() with gfid of the file, do the following: link() the gfid handle under .glusterfs/unlink to the new path to be created in mknod() and rename() the gfid handle to go back under .glusterfs/ab/cd/. Change-Id: I86ef1f97a76ffe11f32653bb995f575f7648f798 BUG: 1292379 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/13001 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* tier:unlink during migrationMohammed Rafi KC2015-12-161-0/+91
| | | | | | | | | | | | | | | | | | | | | | files deleted during promotion were not deleting as the files are moving from hashed to non-hashed. On deleting a file that is undergoing promotion, the unlink call is not sent to the dst file as the hashed subvol == cached subvol. This causes the file to reappear once the migration is complete. This patch also fixes a problem with stale linkfile deleting. Change-Id: I4b02a498218c9d8eeaa4556fa4219e91e7fa71e5 BUG: 1282390 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/12829 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/glusterd: Only positive values for freq-thresholdsJoseph Fernandes2015-12-161-0/+4
| | | | | | | | | | | | Fixed error handling for validation for freq-thresholds Change-Id: Ibe3a9752ac0b525b0c8c0d6c4b4e4d694bd91b88 BUG: 1291603 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12970 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* glusterd/afr: store afr pending xattrs as a volume optionRavishankar N2015-12-152-0/+2
| | | | | | | | | | | | | | | | | | | | | | | Problem: When AFR xlator initialises, it uses the name of the client xlators below it for storing the pending changelogs (xattrs). This can be problem when some other xlator is loaded in between AFR and the client. Though that is a trivial 'traverse-graph-till-the-client-and-use-the-name' fix in AFR's init(), there are other issues like when there's no client xlator at all when, say, AFR is moved to the server side. Fix: The client xlator names are currenly unique and stored as brickinfo->brick_ids. So persist these ids as comma separated values in AFR's volume_options and use them as xattr values during init(). Change-Id: Ie761ffeb3373a4c4d85ad05c84a768c4188aa90d BUG: 1285152 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/12738 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* tier/glusterd : making new tier detach command throw warninghari gowtham2015-12-151-0/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | For detach tier, the validation was done using the string "detach-tier" but the new commands used has the string "tier". Making the string use "tier" to compare, creates problem as the tier status and tier detach have the keyword "tier". So tier detach and tier status were separated. and strtok was used to prevent the condition from passing when the volume name has a substring of "tier". (only the second word from the string is got and checked if the feature is tier) Problem: new detach tier command doesnt throw warnings like "not a tier volume" or " detach tier not started" respectively instead it prints empty output. Fix: while validate the volume is checked if its a tiered volume if yes it is checked if the detach tier is started, else a warning is thrown respectively. Change-Id: I94246d53b18ab0e9406beaf459eaddb7c5b766c2 BUG: 1288517 Signed-off-by: hari gowtham <hgowtham@redhat.com> Reviewed-on: http://review.gluster.org/12883 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* storage/posix: Implement .unlink directoryAshish Pandey2015-12-152-0/+43
| | | | | | | | | | | | | | | | | | | | | | | | Problem: For EC volume, If a file descriptor is open and file has been unlinked, any further write on that fd will fail. When a write request comes, EC internally reads some blocks using anonymous fd. This read will fail as the file has already been unlinked. Solution: To solve this issue, we are using .unlink directory to keep track of unlinked file. If a file is to be unlinked while its fd is open, move this to .unlink directory and unlink it from .glusterfs and real path. Once all the fd will be closed, remove this entry form .unlink directory. Change-Id: I8344edb0d340bdb883dc46458c16edbc336916b9 BUG: 1286029 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: http://review.gluster.org/12816 Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* bitrot: getting correct value of scrub stat'sGaurav Kumar Garg2015-12-141-7/+0
| | | | | | | | | | | | | | | | | | | When user execute bitrot scrub status command then gluster is not giving correct value of Number of Scrubbed files, Number of Unsigned files, Last completed scrub time, Duration of last scrub. With this patch scrub status will give correct value for all the above fields. Change-Id: Ic966f76d22db5b0c889e6386a1c2219afbda1f49 BUG: 1285989 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12776 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* tests:bugs:fuse: add setup.sh and teardown.sh to facilitate manual testingMichael Adam2015-12-112-0/+25
| | | | | | | | | | Change-Id: Ia8fe402663bbdabdc10c18ab42a2063466eb42b6 BUG: 1286735 Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-on: http://review.gluster.org/12830 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaushal M <kaushal@redhat.com>
* tests:bugs:fuse: add test for bug #1283103 - selinux mount vs security xattrsMichael Adam2015-12-111-0/+59
| | | | | | | | | | BUG: 1283103 Change-Id: Ic4485d650275f67eb6b0b8382a92eb829c06e27c Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-on: http://review.gluster.org/12827 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* tests, shard: Remove dependency on strict-write-orderingKrutika Dhananjay2015-12-105-10/+9
| | | | | | | | | | Change-Id: I00171a77bdefb1c2e7e4610cb0ade5679bdb761f BUG: 1289840 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12915 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* tests: Fix arbiter-statfs.tRavishankar N2015-12-101-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ..and remove it from bad tests list. Problem: https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/12516/consoleFull ++ SETUP_LOOP /d/backends/brick1 ++ '[' 1 '!=' 1 ']' ++ backend=/d/backends/brick1 ++ case ${OSTYPE} in +++ awk -F: '/not in use/{print $1; exit}' +++ vnconfig -l vnconfig: VNDIOCGET: Bad file descriptor ++ vnd= ++ '[' x = x ']' ++ echo 'no more vnd' no more vnd ++ return 1 Fix: TEST the return value of SETUP_LOOP. Also added EXIT_EARLY to the test case because there is no point in continuing the test when setting the bricks fail. Change-Id: I933611c41f93ac646f1170b62db656314c801ef1 BUG: 1290125 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/12936 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* tests/tier: spurious failure in rename testN Balachandran2015-12-101-4/+8
| | | | | | | | | | | | | | | bug-1279376-rename-demoted-file.t fails sometimes The fix is based on the assumption that the test failed because the demotion happened too quickly. Change-Id: Ieccc736f387fcf6afaa72fa9918adb6dd34f2c8a BUG: 1289845 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/12926 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tests/bug-924726.t: fix grep pattern to get correct glusterfs pidRaghavendra G2015-12-091-1/+1
| | | | | | | | | Change-Id: Ia2444b1b3e45e3e224bcd59e780a0f38c492f133 BUG: 1289428 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: http://review.gluster.org/12906 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* tests: fix brick_up_statusKaushal M2015-12-094-5/+4
| | | | | | | | | | | | | | | The brick_up_status function wasn't correct after the introduction of the RDMA port into the `volume status` output. It has been fixed to use the XML brick status of a specific brick instead of normal CLI output. Change-Id: I5327e1a32b1c6f326bc3def735d0daa9ea320074 BUG: 1289584 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/12913 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* quota: copy quota_version value in func glusterd_volinfo_dupvmallika2015-12-081-0/+48
| | | | | | | | | | | | | | | | | | | | | quota_version is a new variable introduced for quota xattr versioning feature. quota_version was not copied when creating duplicate volinfo in function 'glusterd_volinfo_dup' so any feature like snapshot/tiering using glusterd_volinfo_dup will get the default value of quota_version instead of the correct number and can cause a problem Change-Id: I7b0f418002d49aa7210e2e741e65ee5b2593e6a6 BUG: 1288474 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12881 Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* glusterd/quota: quota-version conflict in export/import volinfovmallika2015-12-081-0/+24
| | | | | | | | | | | | | | | When exporting/importing voinfo during handshake, quota conf and quota xattr version were using same key 'quota-version' and updated wrong values when importing quota version values. Change-Id: If939d6f5bc4851d4114963877be72dda21834f0f BUG: 1287996 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12865 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* tier/glusterd: Check before starting tier daemon during volume startMohammed Rafi KC2015-12-081-0/+72
| | | | | | | | | | | | | | | | | We start tier daemon when volume is started without looking into the previous status. The problem with that if detach-tier is started and then volume force start is actually starting tier daemon. This is also fixes a problem where tier daemon is not starting after detach stop. Change-Id: I15b56a711e12f0e24f5ab123561258bd448621f7 BUG: 1286974 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/12833 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* cluster/tier: fix loading tier.so into glusterdN Balachandran2015-12-031-2/+2
| | | | | | | | | | | | | | | | | glusterd occasionally loads shared libraries of translators. This failed for tiering due to a reference to dht_methods which is defined as a global variable which is not necessary. The global variable has been removed and this is now a member of dht_conf and is now initialised in the *_init calls. Change-Id: Ifa0a21e3962b5cd8d9b927ef1d087d3b25312953 BUG: 1287842 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/12863 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* snapshot/clone : Fix tier pause failure for snapshot cloneAvra Sengupta2015-12-023-8/+37
| | | | | | | | | | | | | | | | | | | | On a tiered volume, snapshot clone fails while trying to pause tier, as we pass volname(snap) to the brick_op_phase module, which tries to look for the snap volume amongst regular volumes, and obviously doesn't find it and fail. Well as snapshot volumes are read only volume, and will not have tiering daemon acting upon them, there is really no need to pause tiereing while taking clone of snapshot volumes. Hence removing the code to pause and resume tiering during clone create. Change-Id: I2266aba589a830a13a806c0d8a56fd8855143ccd BUG: 1279327 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/12548 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
* tier/libgfdb/sql: Correcting logic in sql query for readJoseph Fernandes2015-12-021-6/+9
| | | | | | | | | | | | | | | | | | | | | | Correcting query in the where clause from "a & b | c" to "a & (b | c)" where "a" is the condition to join the gf_file_tb and gf_flink_tb through gfids "b" is the condition for the write heat and "c" is the condition for read heat Change-Id: I99226d82b0efb68fbef3a40f02b215bb2b4370d6 BUG: 1286656 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/12823 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* tier/glusterd : Validation for frequency thresholds and record-countersJoseph Fernandes2015-12-012-0/+79
| | | | | | | | | | | | | | | | | | | | | | | | | 1) if record-counters is set to off check if both the frequency thresholds are non-zero, then pop an error message, with volume set failed. 2) if record-counters is set to on check if both the frequency thresholds are zero, then pop an note, but volume set is not failed. 3) If any of the frequency thresholds are set to a non-zero value, switch record-counters on, if not already on 4) If both the frequency thresholds are set to zero, switch record-counters off, if not already off NOTE: In this fix we have 1) removed unnecessary ctr vol set options. 2) changed ctr_hardlink_heal_expire_period to ctr_lookupheal_link_timeout Change-Id: Ie7ccfd3f6e021056905a79de5a3d8f199312f315 BUG: 1286346 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/12780 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* cluster/ec: Create copy of dict for setting internal xattrsPranith Kumar K2015-12-011-0/+59
| | | | | | | | | | | | | | | | | | | | | | Problem: Ec takes a ref of the request xdata and sets trusted.ec.version/algo etc xattrs as part of it. But this request xdata could be using same dictionary to do the operation on multiple subvolumes, due to which other subvolumes will have internal xattrs of ec in it and will be created on subvols where they are not supposed to appear. Fix: Take a copy of the request xdata/dict to prevent this from happening. Most of the debugging work and test script is contributed by Nitya. BUG: 1286910 Change-Id: If146435dfb89656158dbed3862a3e9a0cda60581 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/12831 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
* Upcall: Read gfid from iatt in case of invalid inodeSoumya Koduri2015-12-012-0/+156
| | | | | | | | | | | | | | | | When any file/dir is looked upon for the first time, inode created shall be invalid till it gets linked to the inode table. In such cases, read the gfid from the iatt structure returned as part of such fops for UPCALL processing. Change-Id: Ie5eb2f3be18c34cf7ef172e126c9db5ef7a8512b BUG: 1283983 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/12773 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* glusterd: copy snapshot object during duplication of volfileMohammed Rafi KC2015-11-251-0/+48
| | | | | | | | | | | | | | | | When creating duplicate volfile for hot/cold tier, we need to copy the snapshot object in to volfile as it requires to generate snapshot brick volfile. Change-Id: I39ccfa20cd1c16ef2801901e3cd3a31c76f8995d BUG: 1284789 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/12734 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Avra Sengupta <asengupt@redhat.com>
* tier/test: Fixing tests/basic/tier/legacy-many.tJoseph Fernandes2015-11-251-1/+1
| | | | | | | | | | | Change-Id: Icbd83afdeac053aec5b3b8fa19665a2908e87d8e BUG: 1285483 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12751 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Correcting rename logicJoseph Fernandes2015-11-251-0/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: When a file with old_file_name and GFID_1 is renamed with a new_file_name which already exists and with GFID_2, this is what happens in linux internaly. a. "new_file_name" is unlinked for GFID_2 b. a hardlink "new_file_name" is created to GFID_1 c. "old_file_name" hardlink is unlinked for GFID_2. Well this is all internal to linux, and gluster just issues a rename system call at POSIX layer. But CTR Xlator doesn't delete the entries corresponding to the "new_file_name" and GFID_2. Thus leaving the stale entry in the DB. The following are the implications. a. Promotion are tried on these stale entries which will fail and show false results in the status of migration, b. GFID_2 Files with 2 hardlinks, which will have only one hardlink after the rename will not be promoted or demoted as the DB shows 2 entries. Solution: Delete the older database entry for the replaced hardlink Change-Id: I4eafa0872253e29ff1f0bec4283bcfc579ecf0e2 BUG: 1284090 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12711 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* glusterd/bitrot : Integration of bad files from bitd with scrub status commandGaurav Kumar Garg2015-11-232-0/+50
| | | | | | | | | | | | | | | | Currently scrub status command is not displaying list of all the bad files. All the bad files are avaliable in the bitd daemon. With this patch it will dispaly list of all the bad file's in the scrub status command. Change-Id: If09babafaf5d7cf158fa79119abbf5b986027748 BUG: 1207627 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/12720 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* cluster/tier: readdirp to cold tier onlyDan Lambright2015-11-232-10/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is possible a file would get migrated in the middle of a readdir operation. If there are four subvolumes A,B,C,D, and if readdir reads them in order and reaches subvol B, then, if a file is moved from D to A, it will not be included in the readdir output. This phenonema has pre-existed in DHT migration but is more apparent in tiering. When a file is moved off the hashed subvolume a T file is created. For tiering, we will make the cold subvolume the hashed subvolume. This will ensure the creation of a T file. Readdir will not skip T files in the tier translator. Making the cold subvolume the hashed subvolume ensures the T files created on promotions or creates will be less likely to fill the volume. Creates still put the data on the hot subvolume. Change-Id: Ifde557d3d0e94a4570ca9f115adee3db2ee75407 BUG: 1281598 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/12530 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/tier: Do not delete linkto file on demotionN Balachandran2015-11-181-0/+88
| | | | | | | | | | | | | | | | | The current DHT migration code will always delete the src linkto file after migration as dht always moves files to the hashed subvol. This is not the case in tiering. The lack of linkto files causes rename to fail leaving 2 files with the same name but different gfids on the volume. Modified to leave the linkto file behind if the source volume is the hashed subvolume. Change-Id: I2b99f7d34b4b719aee6232dc40c6a8f8ba88225d BUG: 1279376 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/12551 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* core: Add experimental xlator directoryShyam2015-11-181-0/+7
| | | | | | | | | | | | | | | | | | Added an experimental xlator directory under ./xlators/ The intent of this directory is presented in the README.md that accompanies this commit. This directory can be disabled from being compiled using, - configure --disable-experimental Change-Id: I047f380c91a082d111432f8bbdbd4d7bdcbaa809 Signed-off-by: Shyam <srangana@redhat.com> Reviewed-on: http://review.gluster.org/12321 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Avra Sengupta <asengupt@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* core: use syscall wrappers instead of direct syscalls - regression testKaleb S. KEITHLEY2015-11-172-0/+127
| | | | | | | | | | | | | | | | various xlators and other components are invoking system calls directly instead of using the libglusterfs/syscall.[ch] wrappers. If not using the system call wrappers there should be a comment in the source explaining why the wrapper isn't used. Change-Id: Id2207deb81a75e1af6f34bf857e74725f8bb532f BUG: 1267967 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/12410 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* cluster/tier make cache mode default for tiered volumesDan Lambright2015-11-176-5/+11
| | | | | | | | | | | | | The default mode for tiered volumes must be cache. The current test mode was for engineering and should ordinarily not be used by customers. Change-Id: I20583f54a9269ce75daade645be18ab8575b0b9b BUG: 1282076 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/12581 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com>
* quota: fix spurious failurevmallika2015-11-171-2/+1
| | | | | | | | | | Change-Id: I5d18533d66df3175752a73430f680dcdfdb3c12a BUG: 1278689 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12546 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* tier/ctr: Providing option to record or ignore metadata heatJoseph Fernandes2015-11-151-0/+101
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we heat up a file for both data and metadata write. Here we provide a ctr xlator option called "ctr-record-metadata-heat" were the admin can decide on recording metadata heat i.e heatup a file on metadata writes or not. Metadata data operation are a. setattr: explicit changing of atime/mtime using utimes, changing of posix permissions of the file b. rename: Renaming a file, c. unlink, link: adding or deleting hardlinks d. xattrs: setting or removal of xattrs. NOTE: atime, mtime and ctime change through writev, readv, truncate, mknod and create will not be considered here as these fops are data and primary metadata fops. Defaultly "ctr-record-metadata-heat" is off. Admin can switch it on using gluster volume set command. Change-Id: I91157509255dd5cb429cda2b6d4f64582e155e7b BUG: 1279166 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12540 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tests: fix spurious error in fops-during-migration-pause.tDan Lambright2015-11-111-2/+2
| | | | | | | | | | | | | | The test did not spend long enough time moving the file for the pause to occur simultaneously, leading to failure. Solution is to elongate that time by increasing the file size. Change-Id: I1727fa9e3f7a987dfa07dd5da44c68d3f17218d9 BUG: 1280428 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/12570 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Joseph Fernandes
* features/gfid-access: Fix entry creation via setxattr for geo-repKotresh HR2015-11-101-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GEO-REP INTEROP WITH SHARD FEATURE Problem: Geo-replication uses setxattr interface of gfid-access xlator to create entries and send explicit setattr after entry creation to set uid and gid. But between entry creation and setattr, the inode would not be linked. Hence operation which accesses inode structure during setattr by any the below xlator fails. Solution: Linking inode would seem the obvious solution but, gfid-access xlator cannot link inodes and maintain it as it would result in same inode pointing to two different paths one being virtual .gfid/<gfid> path and other being actual path. The solution is to set uid and gid in frame->root->uid and frame->root->gid respectively from which posix extracts and sets. Change-Id: Ic0749ee471432caeb8ded3152a07de6e64d8538d BUG: 1265148 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12206 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* tests: make mount-nfs-auth.t more stableNiels de Vos2015-11-091-15/+11
| | | | | | | | | | | | | | | | | | mount-nfs-auth.t has a funky way of restarting the Gluster/NFS service. It is a little racy and does not always work. Disabling and enabling the nfs.disable volume option triggers a restart of the Gluster/NFS service too, and is much simpler. Also adding a little more EXPECT_WITHIN statements to prevent the occasional failures. Change-Id: I6765e9f021abbe995dfac00fbfc67298e2ec769c BUG: 1278476 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/12542 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* geo-rep: Fix syncing chown in xsync crawlKotresh HR2015-11-083-22/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | GEO-REP INTEROP WITH SHARD FEATURE Problem: The sequence of entry creation and chown in master is recorded as creation of entry with resulted user:group in xsync changelog. During sync, entry creation is always split into two ops, MKNOD and SETATTR. Hence the issue is not being hit otherwise it would have failed with EPERM if parent is owned by different user. But with shard translator being enabled on slave, doing entry creation with MKNOD and SETATTR is not allowed, SETATTR fails as it accesses inode structure which is not linked. Solution: The sequence of entry creation and chown in master should be recorded as MKNOD and SETATTR separately always and do entry creation with single op in gfid-access xlator. The gfid-access patch will be sent separately. Change-Id: I93e554bf9342397a7660503f5128e9709f8a0cd8 BUG: 1265148 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12205 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* cluster/tier correct promotion cycle calculationDan Lambright2015-11-073-19/+30
| | | | | | | | | | | | | | | The tier translator should only choose candidate files for promotion from the most recent cycle, not a multiple of the most recent cycles. Otherwise user observed behavior can be inconsistent. Remove related test in tier.t that is subject to race condition. Change-Id: I9ad1523cac00f904097ce468efa6ddd515857024 BUG: 1275524 Signed-off-by: root <root@rhs-cli-15.gdev.lab.eng.bos.redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/12480 Reviewed-by: Joseph Fernandes Tested-by: Gluster Build System <jenkins@build.gluster.com>
* quota: fix for spurious failurevmallika2015-11-061-0/+3
| | | | | | | | | | | | | | Filed a bug# 1278689. For now marking the testcase tests/bugs/quota/bug-1235182.t' bad once the bug# 1278689, remove the testcase from bad list Change-Id: I224f907153d3e5f35834007a40b0050246d8787a BUG: 1278689 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12526 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/libgfdb: Replacing ASCII query file with binaryJoseph Fernandes2015-11-061-7/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Earlier, when the database was queried we used to save all the queried records in an ASCII format in the query file. This caused issues like filename having ASCII delimiter and used to take a lot of space. The tier.c file also had a lot of parsing code. Here we changed the format of the query file to binary. All the logic of serialization and formating of query record is done by libgfdb. Libgfdb provides API, gfdb_write_query_record() and gfdb_read_query_record(), which the user i.e tier migrator and CTR xlator can use to write to and read from query file. With this binary format we save on disk space i.e reduce to 50% atleast as we are saving GFID's in binary format 16 bytes and not the string format which takes 36 bytes + We are not saving path of the file + we are also saving on ASCII delimiters. The on disk format of query record is as follows, +---------------------------------------------------------------------------+ | Length of serialized query record | Serialized Query Record | +---------------------------------------------------------------------------+ 4 bytes Length of serialized query record | | -------------------------------------------------| | | V Serialized Query Record Format: +---------------------------------------------------------------------------+ | GFID | Link count | <LINK INFO> |..... | FOOTER | +---------------------------------------------------------------------------+ 16 B 4 B Link Length 4 B | | | | -----------------------------| | | | | | V | Each <Link Info> will be serialized as | +-----------------------------------------------+ | | PGID | BASE_NAME_LENGTH | BASE_NAME | | +-----------------------------------------------+ | 16 B 4 B BASE_NAME_LENGTH | | | ------------------------------------------------------------------------| | | V FOOTER is a magic number 0xBAADF00D indicating the end of the record. This also serves as a serialized schema validator. Change-Id: I9db7416fd421e118dd44eafab8b535caafe50d5a BUG: 1272207 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12354 Reviewed-by: N Balachandran <nbalacha@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* snapshot: Making bug-1275616.t more regression failure tolerantAvra Sengupta2015-11-061-0/+10
| | | | | | | | | | | | | | | | | | | snapshot clone creation fails 'spuriously' on the regression setup coz the brick rpc connect for snap3 in the testcase, happens way after the snap was created. So adding a EXPECT_WITHIN $PROCESS_UP_TIMEOUT check(read delay) to help the cause. But this isn't a 100% guaranteed fix, as on an even slower machine, even this check will fail followed by the subsequent failures that this patch is trying to fix in the first place Change-Id: I2f31558b717fd610111f14e451fe444c09f3f254 BUG: 1278418 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/12516 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* mgmt/glusterd: Store arbiter-count and restore itPranith Kumar K2015-11-043-0/+73
| | | | | | | | | | | | | | | | | | | | | | | | Problem: 1) Glusterd doesn't remember about arbiter information of replica volume in store. When glusterd goes down and comes backup, arbiter volumes will become replica volumes. 2) Glusterd doesn't import/export arbiter information to/from the other peers. 3) Volume info doesn't show any arbiter count in the output. Fix: 1) Persist arbiter information in glusterd-store 2) Import/Export arbiter information of the volume 3) Change volume info output to show arbiter count. Change-Id: I2db81e73d2694b01f7d07b08a17b41ad5a55c361 BUG: 1276675 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/12475 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* cluster/tier : Files skipped during tier query parsingN Balachandran2015-11-032-0/+85
| | | | | | | | | | | | | | | | | The tier query parsing code was using fscanf to read each record. As space is a delimiter for fscanf, filenames containing spaces caused the parsing to return unexpected values causing various issues in the tier process, including crashes due to buffer overflows. Change-Id: Ife602cb7ecb158fccbc2c89e4d2959bd97098a87 BUG: 1276562 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/12469 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* snapshot: Inherit snap-max-hard-limit from original volumeAvra Sengupta2015-11-022-0/+64
| | | | | | | | | | | | | | | | | | A snapshot should inherit snap-max-hard-limit from the original volume while being created and when being restored to, it should restore the same. Similarly a clone taken from a snapshot should inherit snap-max-hard-limit from the snapshot. Change-Id: If8e90e2ffc10e22086b803ac8e2638a16bcec968 BUG: 1275616 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/12437 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
* snapshot: Don't display snapshot's hard-limit and soft-limit in vol infoAvra Sengupta2015-11-021-5/+6
| | | | | | | | | | | | | | | | | | | | | | | The snap-max-hard-limit being displayed in the volume info currently is propagated from system's snap-max-hard-limit as that is a global option common for all volumes, and hence ends up showing the system's snap-max-hard-limit. We should not be displaying snap-max-hard-limit and snap-max-soft-limit in the volume info at all, as these are snap config options and should be set and displayed via snap config command. Modified bug-1113476.t to test the same behaviour. Change-Id: I90891f0cf7fb39fd686787297c7f7cd8c1e7daa1 BUG: 1276018 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/12443 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
generated by cgit (git 2.34.1) at 2024-05-13 18:53:16 +0000