summaryrefslogtreecommitdiffstats
path: root/tests/bitrot
Commit message (Collapse)AuthorAgeFilesLines
* features/bit-rot-stub: bring in optional versioningRaghavendra Bhat2017-04-183-0/+9
| | | | | | | | | | | | | | | | * As of now bit-rot-stub does versioning always. This leads lots of getxattr calls being made in lookups. So make object versioning optional. Change-Id: I83713e45ae59fb28004bb3cfa008f2d69edebbfa BUG: 1359599 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/14442 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* storage/posix: Set ret value correctly before exitingKrutika Dhananjay2017-03-011-3/+0
| | | | | | | | | | | Change-Id: I07c3a21c1c0625a517964693351356eead962571 BUG: 1427404 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/16792 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* tests: Mark tests/bitrot/bug-1373520.t bad until fixedKrutika Dhananjay2017-02-281-0/+3
| | | | | | | | | | | | Change-Id: Ic0b5c93c6365e26a5742184dd9445354c0a57295 BUG: 1427404 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/16780 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* cluster/ec: Don't trigger data/metadata heal on LookupsPranith Kumar K2017-02-261-33/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem-1 If Lookup which doesn't take any locks observes version mismatch it can't be trusted. If we launch a heal based on this information it will lead to self-heals which will affect I/O performance in the cases where Lookup is wrong. Considering self-heal-daemon and operations on the inode from client which take locks can still trigger heal we can choose to not attempt a heal on Lookup. Problem-2: Fixed spurious failure of tests/bitrot/bug-1373520.t For the issues above, what was happening was that ec_heal_inspect() is preventing 'name' heal to happen Problem-3: tests/basic/ec/ec-background-heals.t To be honest I don't know what the problem was, while fixing the 2 problems above, I made some changes to ec_heal_inspect() and ec_need_heal() after which when I tried to recreate the spurious failure it just didn't happen even after a long time. BUG: 1414287 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Change-Id: Ife2535e1d0b267712973673f6d474e288f3c6834 Reviewed-on: https://review.gluster.org/16468 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Ashish Pandey <aspandey@redhat.com>
* core: run many bricks within one glusterfsd processJeff Darcy2017-01-301-11/+31
| | | | | | | | | | | | | | | | | | | | | | | This patch adds support for multiple brick translator stacks running in a single brick server process. This reduces our per-brick memory usage by approximately 3x, and our appetite for TCP ports even more. It also creates potential to avoid process/thread thrashing, and to improve QoS by scheduling more carefully across the bricks, but realizing that potential will require further work. Multiplexing is controlled by the "cluster.brick-multiplex" global option. By default it's off, and bricks are started in separate processes as before. If multiplexing is enabled, then *compatible* bricks (mostly those with the same transport options) will be started in the same process. Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb BUG: 1385758 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/14763 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/bit-rot-stub: use the correct spelling of quarantine for bad objectsRaghavendra Bhat2017-01-302-8/+8
| | | | | | | | | | | | | | | | container The directory for containing the list of bad objects was named "quanrantine" instead of "quarantine" Change-Id: I8c20381ac637201d9d1a224f5223e8dfbed53f1e BUG: 1401571 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: https://review.gluster.org/16027 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* tests: Mark tests/bitrot/bug-1373520.t badAtin Mukherjee2017-01-301-0/+4
| | | | | | | | | | | | | Change-Id: Ief8014dd9faa012c7f3c5347f597a155873a8f92 BUG: 1417540 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/16479 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* feature/bitrot: Fix recovery of corrupted hardlinkKotresh HR2016-09-082-5/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: When a file with hardlink is corrupted in ec volume, the recovery steps mentioned was not working. Only name and metadata was healing but not the data. Cause: The bad file marker in the inode context is not removed. Hence when self heal tries to open the file for data healing, it fails with EIO. Background: The bitrot deletes inode context during forget. Briefly, the recovery steps involves following steps. 1. Delete the entry marked with bad file xattr from backend. Delete all the hardlinks including .glusters hardlink as well. 2. Access the each hardlink of the file including original from the mount. The step 2 will send lookup to the brick where the files are deleted from backend and returns with ENOENT. On ENOENT, server xlator forgets the inode if there are no dentries associated with it. But in case hardlinks, the forget won't be called as dentries (other hardlink files) are associated with the inode. Hence bitrot stube won't delete it's context failing the data self heal. Fix: Bitrot-stub should delete the inode context on getting ENOENT during lookup. Change-Id: Ice6adc18625799e7afd842ab33b3517c2be264c1 BUG: 1373520 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/15408 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
* feature/bitrot: Ondemand scrub option for bitrotKotresh HR2016-08-251-2/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The bitrot scrubber takes 'hourly/daily/biweekly/monthly' as the values for 'scrub-frequency'. There is no way to schedule the scrubbing when the admin wants it. Ondemand scrubbing brings in the new option 'ondemand' with which the admin can start scrubbing ondemand. It starts the scrubbing immediately. Ondemand scrubbing is successful only if the scrubber is in 'Active (Idle)' (waiting for it's next frequency cycle to start scrubbing). It is not entertained when the scrubber is in 'Paused' or already running. Here is the command line syntax. gluster volume bitrot <vol name> scrub ondemand Change-Id: I84c28904367eed827a7dae8d6a535c14b28e9f4d BUG: 1366195 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/15111 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* tests: Fix tests/bitrot/bug-1244613.tKotresh HR2016-07-271-0/+3
| | | | | | | | | | | | | | Wait for gluster nfs to initialize before attempting the nfs mount. Change-Id: I4bd9579ad5368935cf62632a5d612f89fce5979f BUG: 1360682 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/15028 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* tests: Fix spurious failure of br-stub.tKotresh HR2016-07-211-0/+3
| | | | | | | | | | | | | | | | | | | | | | The nfs mount fails occasionally in ./tests/bitrot/br-stub.t. The reason being nfs mount is attempted before the gluster nfs has come up. It is a race and hence happens occasionally. The patch fixes it by waiting for nfs server to come up before mount. Thanks skoduri@redhat.com for root causing it. Change-Id: I3adbf2363514635785c02b1478733095ad0b74cf BUG: 1358114 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/14960 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Poornima G <pgurusid@redhat.com> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* tests: assign bug id for br-state-check known issueRaghavendra Talur2016-05-041-1/+1
| | | | | | | | | | Change-Id: I13a1477830a023f40cb62097f04c22cd2d910021 BUG: 1332473 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/14198 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* tests: Move tests/bitrot/br-state-check.t to bad testKotresh HR2016-05-031-0/+1
| | | | | | | | | | | Change-Id: I9a1578ef4aeb8ce559b3af5092544b3023b14f87 BUG: 1332473 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/14186 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
* glusterd: default value of nfs.disable, change from false to trueKaleb S KEITHLEY2016-04-272-1/+2
| | | | | | | | | | | | | | | | | | Next step in eventual deprecation of glusterfs nfs server in favor of ganesha.nfsd. Also replace several open-coded strings with constant. Change-Id: If52f5e880191a14fd38e69b70a32b0300dd93a50 BUG: 1092414 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/13738 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Tested-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* features/bitrot: do not remove the quarantine handle in forgetRaghavendra Bhat2016-02-291-6/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If an object is marked as bad, then an entry is corresponding to the bad object is created in the .glusterfs/quarantine directory to help scrub status. The entry name is the gfid of the corrupted object. The quarantine handle is removed in below 2 cases. 1) When protocol/server revceives the -ve lookup on an entry whose inode is there in the inode table (it can happen when the corrupted object is deleted directly from the backend for recovery purpose) it sends a forget on the inode and bit-rot-stub removes the quarantine handle in upon getting the forget. refer to the below commit f853ed9c61bf65cb39f859470a8ffe8973818868: http://review.gluster.org/12743) 2) When bit-rot-stub itself realizes that lookup on a corrupted object has failed with ENOENT. But with step1, there is a problem when the bit-rot-stub receives forget due to lru limit exceeding in the inode table. In such cases, though the corrupted object is not deleted (either from the mount point or from the backend), the handle in the quarantine directory is removed and that object is not shown in the bad objects list in the scrub status command. So it is better to follow only 2nd step (i.e. bit-rot-stub removing the handle from the quarantine directory in -ve lookups). Also the handle has to be removed when a corrupted object is unlinked from the mount point itself. Change-Id: Ibc3bbaf4bc8a5f8986085e87b729ab912cbf8cf9 BUG: 1308961 Original author: Raghavendra Bhat <raghavendra@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/13472 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* features/bitrot: Fail node-uuid getxattr if file is marked badKotresh HR2016-01-071-0/+68
| | | | | | | | | | | | | | | If xattr is node-uuid and the inode is marked bad, fail getxattr and fgetxattr with EIO. Returning EIO would result in AFR to choose correct node-uuid coresponding to the subvolume where the good copy of the file resides. Change-Id: I45a42ca38f8322d2b10f3c4c48dc504521162b42 BUG: 1294786 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/13116 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* bitrot: getting correct value of scrub stat'sGaurav Kumar Garg2015-12-141-7/+0
| | | | | | | | | | | | | | | | | | | When user execute bitrot scrub status command then gluster is not giving correct value of Number of Scrubbed files, Number of Unsigned files, Last completed scrub time, Duration of last scrub. With this patch scrub status will give correct value for all the above fields. Change-Id: Ic966f76d22db5b0c889e6386a1c2219afbda1f49 BUG: 1285989 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12776 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* glusterd/bitrot : Integration of bad files from bitd with scrub status commandGaurav Kumar Garg2015-11-231-0/+43
| | | | | | | | | | | | | | | | Currently scrub status command is not displaying list of all the bad files. All the bad files are avaliable in the bitd daemon. With this patch it will dispaly list of all the bad file's in the scrub status command. Change-Id: If09babafaf5d7cf158fa79119abbf5b986027748 BUG: 1207627 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/12720 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* features/bit-rot-stub: fail the fop if inode context get failsRaghavendra Bhat2015-08-111-0/+42
| | | | | | | | | | | | | | | In stub, for fops like readv, writev etc, if the the object is bad, then the fop is denied. But for checking if the object is bad inode context should be checked. Now, if the inode context is not there, then the fop is allowed to continue. This patch fixes it and the fop is unwound with an error, if the inode context is not found. Change-Id: I5ea4d4fc1a91387f7f9d13ca8cb43c88429f02b0 BUG: 1243391 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/11449 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* tests: set inode-lru-limit to 1 and check if bit-rot xattrs are wrongy createdRaghavendra Bhat2015-08-101-0/+87
| | | | | | | | | | | | | This test sets the lru limit of the inode table to 1 and checks if inode forgets and resolve cause any problem with bit-rot xattrs (especially bad-file xattr). Change-Id: I1fa25fa2d31dda8d26e8192562e896e5bddd0381 BUG: 1244613 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/11718 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* tests/bitrot: Scrub state change testsVenky Shankar2015-06-211-0/+81
| | | | | | | | | | | Change-Id: Ibb4b503e7d723c86ac381ad3747b1198334bd6ad BUG: 1231619 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/11290 Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/bit-rot-stub: implement mknod fopRaghavendra Bhat2015-05-312-15/+80
| | | | | | | | | | | | | | | | | | | With the absence of mknod() fop implementation in bitrot stub, further operations that trigger versioning resulted in crashes as they expect the inode context to be valid. Therefore, this patch implements mknod() following similar simantics to fops such as create(). Furthermore, bitrot stub test C program is fixed to stop lying and validate obj versions according to the versioning protocol. Change-Id: If76f252577445d1851d6c13c7e969e864e2183ef BUG: 1221914 Original-Author: Raghavendra Bhat <raghavendra@redhat.com> Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/10790 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Tests: avoid hang in br-stub.tEmmanuel Dreyfus2015-05-181-0/+2
| | | | | | | | | | | | | | | | | br-stub.t forgot to unmount the NFS mount, leading to the situation where cleanup code had to do it after glusterfs daemon were shut down. NetBSD has trouble dealing with this situation and may enter endless wait for the NFS server to come back. Fix is simply to unmount before the end og the script BUG: 1129939 Change-Id: I972d379172a75f29ad11abc66d700c2d31106194 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10748 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* features/bit-rot-stub: versioning of objects in write/truncate fop instead ↵Raghavendra Bhat2015-05-082-14/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | of open * This patch brings in the changes where object versioning is done in write and truncate fops instead of tracking them in open and create fops. This model works for both regular and anonymous fds. It also removes the race associated with open calls, create and lookups. This patch follows the below method for object versioning and notifications: Before sending writev on the fd, increase the ongoing version first. This makes anonymous fd write similar to the regular fd write by having the ongoing version increased before doing the write. Do following steps to do versioning: 1) For anonymous fds set the fd context (so that release is invoked) and add the fd context to the list maintained in the inode context. For regular fds the above think would have been done in open itself. 2) Increase the on-disk ongoing version 3) Increase the in memory ongoing version and mark inode as non-dirty 3) Once versioning is successfully done send write operation. If versioning fails, then fail the write fop. 5) In writev_cbk mark inode as modified. Change-Id: I7104391bbe076d8fc49b68745d2ec29a6e92476c BUG: 1207979 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/10233 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/bitrot: Follow xattr naming conventionsVenky Shankar2015-05-061-1/+1
| | | | | | | | | | | | | | | | | Instead of "trusted.glusterfs.bit-rot.*" use "trusted.bit-rot.*" NOTE: With this patch, data on existing volumes would be resigned (which should be OK as of now since we do not expect many users as of now :-)) Change-Id: I926c7bca266a9c8f2cb35d57c4d0359aa5cecfa0 BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/10181 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tests/bitrot-stub: Object versioning test(s)Venky Shankar2015-04-082-0/+191
This patch introduces basic object versioning test(s) which is required for bitrot detection to work correctly. Basic test(s) such as opening a file in read-only mode, single open, multiple open()s are covered on FUSE mount _only_ as stub does not support anonymous fds yet. For this reason, the test case disables open-behind. Actual verification is implemented as a C source which makes use of the same on-disk data structures as used by the stub code. The data structures are moved to separate header file which is included by the test script. Such modularization helps in future enhancements to keep the version "data type" opaque and provide handful of APIs version checking (equal/greater/etc..). [ This is just a start and should grow over time as stub is enhanced and codebase matures. ] Change-Id: Ibee20e65a15b56bbdd59fd2703f9305b115aec7a BUG: 1201724 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/10140 Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-17 17:24:59 +0000