summaryrefslogtreecommitdiffstats
path: root/tests/features
Commit message (Collapse)AuthorAgeFilesLines
* fuse: add --lru-limit optionAmar Tumballi2019-01-161-0/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The inode LRU mechanism is moot in fuse xlator (ie. there is no limit for the LRU list), as fuse inodes are referenced from kernel context, and thus they can only be dropped on request of the kernel. This might results in a high number of passive inodes which are useless for the glusterfs client, causing a significant memory overhead. This change tries to remedy this by extending the LRU semantics and allowing to set a finite limit on the fuse inode LRU. A brief history of problem: When gluster's inode table was designed, fuse didn't have any 'invalidate' method, which means, userspace application could never ask kernel to send a 'forget()' fop, instead had to wait for kernel to send it based on kernel's parameters. Inode table remembers the number of times kernel has cached the inode based on the 'nlookup' parameter. And 'nlookup' field is not used by no other entry points (like server-protocol, gfapi etc). Hence the inode_table of fuse module always has to have lru-limit as '0', which means no limit. GlusterFS always had to keep all inodes in memory as kernel would have had a reference to it. Again, the reason for this is, kernel's glusterfs inode reference was pointer of 'inode_t' structure in glusterfs. As it is a pointer, we could never free it (to prevent segfault, or memory corruption). Solution: In the inode table, handle the prune case of inodes with 'nlookup' differently, and call a 'invalidator' method, which in this case is fuse_invalidate(), and it sends the request to kernel for getting the forget request. When the kernel sends the forget, it means, it has dropped all the reference to the inode, and it will send the forget with the 'nlookup' parameter too. We just need to make sure to reduce the 'nlookup' value we have when we get forget. That automatically cause the relevant prune to happen. Credits: Csaba Henk, Xavier Hernandez, Raghavendra Gowdappa, Nithya B fixes: bz#1623107 Change-Id: Ifee0737b23b12b1426c224ec5b8f591f487d83a2 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* gfapi: Offload callback notifications to synctaskSoumya Koduri2018-12-122-0/+374
| | | | | | | | | | | | | | | | | | | | | | | Upcall notifications are received from server via epoll and same thread is used to forward these notifications to the application. This may lead to deadlock and hang in the following scenario. Consider if as part of handling these callbacks, application has to do some operations which involve sending I/Os to gfapi stack which inturn have to wait for epoll threads to receive repsonse. Thus this may lead to deadlock if all the epoll threads are waiting to complete these callback notifications. To address it, instead of using epoll thread itself, make use of synctask to send those notificaitons to the application. Change-Id: If614e0d09246e4279b9d1f40d883a32a39c8fd90 updates: bz#1651323 Signed-off-by: Soumya Koduri <skoduri@redhat.com> (cherry picked from commit ad35193718a99494ab1b852ca4cbdf054f73de88)
* lease: Treat unlk request as noop if lease not foundSoumya Koduri2018-12-121-2/+2
| | | | | | | | | | | | | | | | | When the glusterfs server recalls the lease, it expects client to flush data and unlock the lease. If not it sets a timer (starting from the time it sends RECALL request) and post timeout, it revokes it. Here we could have a race where in client did send UNLK lease request but because of network delay it may have reached after server revokes it. To handle such situations, treat such requests as noop and return sucesss. Change-Id: I166402d10273f4f115ff04030ecbc14676a01663 updates: bz#1651323 Signed-off-by: Soumya Koduri <skoduri@redhat.com> (cherry picked from commit c2e758b54d8a3f778e3e63db0000bb8b63de9b25)
* python3: assume python3 unless building _packages_ on sys without py3Kaleb S. KEITHLEY2018-09-271-1/+0
| | | | | | | | | | | | | | | The jenkins release-new job runs on a CentOS 7 box, which does not have python3. As a result it runs (autogen.sh and) configure before producing the dist tar file, converting all the python3 shebangs to python2 shebangs in the dist tar file. Then when that tar file is "carried" to, e.g. Fedora koji build system to build packages, the shebangs are incorrect, despite having originally been correct in the git repo. Change-Id: I5154baba3f6d29d3c4823bafc2b57abecbf90e5b updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* core: remove experimental xlators and associated testsShyamsundarR2018-09-173-175/+0
| | | | | | | | experimental xlators removed from 5.0 Change-Id: I47219d8b95efc3d5875ec9224d1e79f8371e9f76 Updates: bz#1628620 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* Land part 2 of clang-format changesGluster Ant2018-09-122-596/+623
| | | | | Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* core: python3Kaleb S. KEITHLEY2018-09-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, https://review.gluster.org/#/c/20364/, https://review.gluster.org/#/c/20441/, and https://review.gluster.org/#/c/20484 shebangs changed from /usr/bin/python2 to /usr/bin/python3. (Reminder, various distribution packaging guidelines require use of explicit python version and don't allow '#!/usr/bin/env python', regardless of how handy that idiom may be.) glusterfs.spec(.in) package python{2,3}-gluster and python2 or python3 dependencies as appropriate. configure(.ac): + test for and use python2 or python3 as appropriate. If build machine has python2 and python3, use python3. Override by setting PYTHON=/usr/bin/python2 when running configure. + PYTHONDEV_CPPFLAGS from python[23]-config --includes is a better match to the original python sysconfig.get_python_inc(). All those other extraneous flags breaks the build. + Only change the shebangs once. Changing them over and over again, e.g., during a `make glusterrpms` in extras/LinuxRPM just sends make (is it really make that's looping?) into an infinite loop. If you figure out why, let me know. + Oldest python2 is python2.6 on CentOS 6 and Debian 8 (Jessie). Everything else has 2.7 or 3.x + logic from https://review.gluster.org/c/glusterfs/+/21050, which needs to be removed/merged after that patch is merged. Builds on CentOS 6, CentOS 7, Fedora 28, Fedora rawhide, and the mysterious RHEL > 7. Change-Id: Idae21d3b6f58b32372e1daa0d234e491e563198f updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* All: run codespell on the code and fix issues.Yaniv Kaul2018-07-221-2/+2
| | | | | | | | | | | | Please review, it's not always just the comments that were fixed. I've had to revert of course all calls to creat() that were changed to create() ... Only compile-tested! Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-06-211-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, and https://review.gluster.org/#/c/20307/ This patch fixes more selected comma white space (ws_comma) as suggested by the 2to3 utility. Note: Fedora packaging guidelines and SUSE rpmlint require explicit shebangs, so popular practices like #!/usr/bin/env python and or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, has_key, idioms, map, numliterals, raise, set_literal, types, urllib, and zip have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: I60932030813484803f73733a9b2b7b23c7a843fd updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-05-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/ and https://review.gluster.org/#/c/19871/ Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, idioms, numliterals, set_literal, types, urllib, zip Note: these 2to3 fixes report no changes are necessary: exec, execfile, exitfunc, filter, getcwdu, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Any 2to3 fixes not in the above two lists have more extensive changes which will follow in separate patches. most unicode changes suggested by 2to3 will need to be applied at the same time as changing the shebangs from python2 to python3. Prashanth notes that unicode strings in py2 need 'u' prefix; unicode strings in py3 3.0, 3.1, and 3.2 a 'u' prefix will throw an error, but in py3 3.3+ it is legal (or just ignored). All Linux dists we care about have 3.3 or later so we can leave 'u' prefixes on unicode strings. Change-Id: I49bba2f328b0ee24b9a8115a7183be979981563e updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* tests: Add lease test casePoornima G2018-05-052-2/+704
| | | | | | | Updates: #350 Change-Id: Iee78ab4baf48c481de1e13ff2b0393bc106b7d0e Signed-off-by: Poornima G <pgurusid@redhat.com> Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-05-021-2/+3
| | | | | | | | | | see https://review.gluster.org/#/c/19788/ use print fn from __future__ Change-Id: If5075d8d9ca9641058fbc71df8a52aa35804cda4 updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* core/build/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-04-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note 1) we're not supposed to be using #!/usr/bin/env python, see https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines Note 2) we're also not supposed to be using "!/usr/bin/python, see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out The previous patch (https://review.gluster.org/19767) tried to do too much in one patch, so it was abandoned. This patch does two things: 1) minor cleanup of configure(.ac) to explicitly use python2 2) change all the shebang lines to #!/usr/bin/python2 and add them where they were missing based on warnings emitted during rpmbuild. In a follow-up patch python2 will eventually be changed to python3. Before that python2-isms (e.g. print, string.join(), etc.) need to be converted to python3. Some of those can be rewritten in version agnostic python. E.g. print statements become print() with "from __future_ import print_function". The python 2to3 utility will be used for some of those. Also Aravinda has given guidance in the comments to the first patch for changes. updates: #411 Change-Id: I471730962b2526022115a1fc33629fb078b74338 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* cluster/dht: enable lookup-optimize by defaultN Balachandran2018-04-041-1/+1
| | | | | | | | | | | | | | Lookup-optimize has been shown to improve create performance. The code has been in the project for several years and is considered stable. Enabling this by default in order to test this in the upstream regression runs. Change-Id: Iab792979ee34f0af4713931e0b5b399c23f65313 updates: bz#1557435 BUG: 1557435 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* python: Remove all uses of find_library. Fixes #1450593Niklas Hambüchen2018-03-241-8/+2
| | | | | | | | `find_library()` doesn't consider LD_LIBRARY_PATH on Python < 3.6. Change-Id: Iee26085cb5d14061001f19f032c2664d69a378a8 BUG: 1450593 Signed-off-by: Niklas Hambüchen <mail@nh2.me>
* hooks: add a script to stat the subdirs in add-brickAmar Tumballi2018-03-061-12/+4
| | | | | | | | | | | | | | | | | | | | | | The subdirectories are expected to be present for a subdir mount to be successful. If not, the client_handshake() itself fails to succeed. When a volume is about to get mounted first time, this is easier to handle, as if the directory is not present in one brick, then its mostly not present in any other brick. In case of add-brick, the directory is not present in new brick, and there is no chance of healing it from the subdirectory mount, as in those clients, the subdir itself will be 'root' ('/') of the filesystem. Hence we need a volume mount to heal the directory before connections can succeed. This patch does take care of that by healing the directories which are expected to be mounted as subdirectories from the volume level mount point. Change-Id: I2c2ac7b7567fe209aaa720006d09b68584d0dd14 BUG: 1549915 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* features/worm: new config option to manage deletion of Worm files.Vishal Pandey2017-11-201-0/+8
| | | | | | | | | | | | | | | | | | | | Add a new configuration option worm-files-deletable to file-level Worm in order to control behaviour of Worm files upon deletion. Steps to Test: 1. Add all the configuration options to a volume to activate file-level-worm 2. Option features.worm-files-deletable is set to 1 by default. 3. Create a new file and wait for the retention time to expire. 4. After retention time expires, do an truncate, rename, unlink, link or write to send the file in Worm state. 5. After that do `rm -f filename`. 6. The file is successfully removed. 7. Repeat from step 2 by setting features.worm-files-deletable 0. This time deletion should not be successful. Change-Id: Ibc89861ee296e065330b93a9f9606be5da40af31 BUG: 1508898 Signed-off-by: Vishal Pandey <vishpandey2014@gmail.com>
* write once read many: file appendable in worm stateVishal Pandey2017-10-301-0/+7
| | | | | | | | | | | | | | | | | | | | Issue: A new file is appendable even when file level worm is 1. Fix: Do a state transition in writev function in worm.c file. Steps To Test: 1- Activate file level worm. 2- Create a new file. 3- Leave file dormant for auto commit period. 4- Try and append some content to the file. 5- check the file if new content has been appended or not. 6- check if file has been transitioned to Worm Retention state. Change-Id: I52d50ad888cb0c39ad54be9352ccb07d48b8d71a BUG: 1505807 Signed-off-by: Vishal Pandey <vishpandey2014@gmail.com>
* protocol/client: handle the subdir handshake properly for add-brickAmar Tumballi2017-10-291-5/+26
| | | | | | | | | There should be different way we handle handshake in case of subdir mount for the first time, and in case of subsequent graph changes. Change-Id: I2a7ba836433bb0a0f4a861809e2bb0d7fbc4da54 BUG: 1505323 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* Disable failing NetBSD testsNigel Babu2017-10-131-0/+1
| | | | | BUG: 1501390 Change-Id: I9a04c094783ec33e617baeae3d0e0cbedb1d6c3b
* protocol-auth: use the proper validation methodAmar Tumballi2017-10-121-0/+4
| | | | | | | | | | | Currently, server protocol's init and glusterd's option validation methods are different, causing an issue. They should be same for having consistent behavior Updates #175 Change-Id: Ibbf9a18c7192b2d77f9b7675ae7da9b8d2fe5de4 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* features/worm: Adding check to newloc when doing renameluneo72017-09-181-0/+9
| | | | | | | | | | | | | | | | | | Problem: Since rename didn't check if newloc exists and it's retention state it was possible to rename a new file that wasn't in retention over a existing file that was in read-only state. Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229 BUG: 1484490 Signed-off-by: luneo7 <luneo7@gmail.com> Reviewed-on: https://review.gluster.org/18104 Tested-by: jiffin tony Thottan <jthottan@redhat.com> Tested-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Karthik U S <ksubrahm@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* debug/delay-gen: Implement delay-generation featurePranith Kumar K2017-08-311-0/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: I was working on a customer issue where the disks were responding some times after seconds. It was becoming very difficult to recreate the issues in our labs, so had to come up with this feature. Requirements: We need an xlator which can delay x% of ops for y micro seconds. We should be able to enable delays for specific fops. This feature is modeled after error-gen. Most of the logic is borrowed from that xlator. This is a minimum implementation of the feature which satisfied the requirements I had. May be in future with more requirements and understanding of the problem further we can improve upon this implementation. Here are the commands and what they do: Enable delay-gen: (This is similar to how err-gen is enabled on the brick side) - gluster volume set <volname> delay-gen posix Set the percentage of fops that need to be delayed - gluster volume set <volname> delay-gen.delay-percentage 50 Default is 10% Set the delay in micro seconds - gluster volume set <volname> delay-gen.delay-duration 500000 Default is 100000 Set comma separated fops to be delayed - gluster v set r2 delay-gen.enable read,write Default is all fops. Fixes #257 Change-Id: Ib547bd39cc024c9cdb63754d21e3aa62fc9d6473 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17591 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* cluster/dht: Reorder dir operations in gf_defrag_fix_layoutN Balachandran2017-08-201-3/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | Earlier, rebalance performed a fix-layout on a directory before healing its subdirectories. If there were a lot of subdirs, it could take a while before all subdirs were created on the newly added bricks. As dht_readdirp only lists dirs from their hashed subvol, those dirs which hashed to the newly added bricks but were not yet created on them were not listed. Now, the child dirs are listed and processed before the layout of the parent is fixed. This introduces a change in behaviour where files in subdirs are migrated before those in parent directories. Credit: Shyam <srangana@redhat.com> Github issue: #239 Change-Id: I8ae7f24a510754cd8d1b31e5d608bcf1928599e2 BUG: 1248393 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/18045 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* glusterfsd: allow subdir mountAmar Tumballi2017-08-041-0/+99
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changes: 1. Take subdir mount option in client (mount.gluster / glusterfsd) 2. Pass the subdir mount to server-handshake (from client-handshake) 3. Handle subdir-mount dir's lookup in server-first-lookup and handle all fops resolution accordingly with proper gfid of subdir 4. Change the auth/addr module to handle the multiple subdir entries in option, and valid parsing. How to use the feature: `# mount -t glusterfs $hostname:/$volname/$subdir /$mount_point` Or `# mount -t glusterfs $hostname:/$volname -osubdir_mount=$subdir /$mount_point` Option can be set like: `# gluster volume set <volname> auth.allow "/subdir1(192.168.1.*),/(192.168.10.*),/subdir2(192.168.8.*)"` Updates #175 Change-Id: I7ea57f76ddbe6c3862cfe02e13f89e8a39719e11 Signed-off-by: Amar Tumballi <amarts@redhat.com> Reviewed-on: https://review.gluster.org/17141 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* tests: disable IPC test-caseNiels de Vos2017-07-181-0/+3
| | | | | | | | | | | | | | | | | | | | | The IPC test-case is not correct, and can cause segfaults or hangs. The ipctest.py script calls glfs_ipc() with the `glfs_t` structure, but should do so with a `glfs_fd_t`. In addition, the test-case is written in a way that we do not suggest to use libgfapi. Python scripts are encouraged to use the bindings from the libgfapi-python project. It would be better to rewrite the test in C so that there is type-checking while compiling and no additional issues with portability (see `LD_PRELOAD` note in the `.t` file). Change-Id: Icb52b5b1585fbee98f2c694547c31df0aa2ba70b Updates: #269 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17786 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
* tests/lock_revocation: mark as badRaghavendra G2017-05-101-0/+1
| | | | | | | | | | | | | The test is failing in master. see gluster-devel for more details. Change-Id: I7a589ad2c54bd55d62f4e66fdf8037c19fc123ea BUG: 1448364 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: https://review.gluster.org/17234 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* worm: add check for internal processes in ftruncate()Amar Tumballi2017-04-181-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The patch fixes the recently seen issues with worm_sh.t test. RCA: - $ git log --oneline xlators/features/read-only/src/worm.c 1b01bdc worm: allow Self-heal-Daemon to perform some operations c5a4a77 features/worm: Adding implementation for ftruncate - These two patches were merged in reverse order of their submission, and hence the check added for internal processes got missed in new fop 'ftruncate()'. The worm_sh.t passed the tests as while that patch got submitted there was no ftruncate() in worm xlator. Change-Id: I81a8a45fa2679917a2c859c4f5224a2c3edbc784 BUG: 1423413 Signed-off-by: Amar Tumballi <amarts@redhat.com> Reviewed-on: https://review.gluster.org/17048 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-by: David Spisla <david.spisla@iternity.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cluster/dht: Make rebalance honor min-free-diskSusant Palai2017-04-131-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | test: Manual created files of size 1K on 2 brick(of size 1GB) setup . added a brick of size 16GB. set min-free-disk to 12GB(so that first two bricks won't receive any files). removed one of the 1st brick of size 1GB. Logs from test: [2017-04-12 08:52:08.196484] W [MSGID: 0] [dht-rebalance.c:895:__dht_check_free_space] 0-test1-dht: Write will cross min-free-disk for file - /tile32 on subvol - test1-client-1. Looking for new subvol. [2017-04-12 08:52:08.196904] I [MSGID: 0] [dht-rebalance.c:925:__dht_check_free_space] 0-test1-dht: new target found - test1-client-2 for file - /tile32 - Post migration we have two files. The new destination (/brick/1) has the data file [root@vm1 ~]# ll /brick/1/tile32 -rw-r--r--. 2 root root 0 Apr 12 14:22 /brick/1/tile32 - On the old target the linkto file is there with linkto xattr pointing to /brick/1 [root@vm1 ~]# ll /tmp/2/tile32 ---------T. 2 root root 1000 Apr 12 14:22 /tmp/2/tile32 [root@vm1 ~]# getfattr -m . -de text /tmp/2/tile32 getfattr: Removing leading '/' from absolute path names security.selinux="unconfined_u:object_r:user_tmp_t:s0" trusted.gfid="����:Aс�#�/'b2" trusted.glusterfs.dht.linkto="test1-client-2" Marking ./tests/features/worm_sh.t as bad test. Reason being, this patch failed on master branch as well and it has nothing to do with rebalance/remove-brick. BUG: 1441508 Change-Id: I90bae251cda3d957a49cdceda90cd08311a392fb Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: https://review.gluster.org/17034 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* worm: allow Self-heal-Daemon to perform some operationsDavid Spisla2017-04-121-0/+75
| | | | | | | | | | | | | | | | | | The Self-Heal-Daemon should be allowed to trigger unlink, link, trauncate, rename and write operation. The value of frame->root->pid can be used to detect internal (by SHD) operations. Change-Id: I7526148100bef1e2837d69df5c119dc97d91fffd BUG: 1423413 Signed-off-by: David Spisla <david.spisla@iternity.com> Reviewed-on: https://review.gluster.org/16661 Tested-by: jiffin tony Thottan <jthottan@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* tests: reenable trash.tJeff Darcy2017-02-101-3/+0
| | | | | | | | | | | | | | | Now that the underlying bug has been fixed (by d97e63d0) we can allow the test to run again. Change-Id: If9736d142f414bf9af5481659c2b2673ec797a4b BUG: 1420434 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/16584 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Anoop C S <anoopcs@redhat.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
* core: run many bricks within one glusterfsd processJeff Darcy2017-01-305-5/+12
| | | | | | | | | | | | | | | | | | | | | | | This patch adds support for multiple brick translator stacks running in a single brick server process. This reduces our per-brick memory usage by approximately 3x, and our appetite for TCP ports even more. It also creates potential to avoid process/thread thrashing, and to improve QoS by scheduling more carefully across the bricks, but realizing that potential will require further work. Multiplexing is controlled by the "cluster.brick-multiplex" global option. By default it's off, and bricks are started in separate processes as before. If multiplexing is enabled, then *compatible* bricks (mostly those with the same transport options) will be started in the same process. Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb BUG: 1385758 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/14763 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* jbr: Sending rollback from failed fop to fdlAvra Sengupta2016-11-083-23/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In case of a failed fop, the failure is detected by the leader in the jbr-server in two places. First during a quorum check of +ve responses when it receives responses from all the followers. At this point if the fop hasn't been successfully journaled at a quorum of followers (as in there is no merit in trying the fop in the leader as the quorum will never be met), then we fail the fop. Also if this quorum is met, then the fop is tried on the leader, and after the leader completes the fop a quorum check similar to the previous one is done again, this time including the leaders outcome. If quorum is not met, then we fail the fop. In both these cases, when the fop fails we send a -ve ack to the client. With this patch, now we will also send a rollback through a GF_FOP_IPC to all the followers(and also to the leader in the second case of failure). This rollback will contain the index and term number of the fop which failed. This will be recorded in the respective journals of the bricks and will be used to rollback the fop on that brick later. A subsequent write, and it's respective rollback would look something like the following in the journal. The trusted.jbr.term and trusted.jbr.index present in the dict of both the logs, relate them, and the presence of "rollback-fop" in the dict of IPC indicates that it is a rollback fop, and the value 13(stands for GF_FOP_WRITE) indicates what kind of rollback operation it is. === GF_FOP_WRITE fd = <gfid 77f12ea2-ca56-40e3-a46e-ba2308baa035> vector = <158 bytes> offset = 0 (0x0) flags = 32769 (0x8001) xdata = dict { trusted.jbr.term = 0 <2 bytes> trusted.jbr.index = 4 <2 bytes> } === GF_FOP_IPC xdata = dict { trusted.jbr.term = 0 <2 bytes> trusted.jbr.index = 4 <2 bytes> rollback-fop = 13 <3 bytes> } Change-Id: I70b6a143d20697153d58e2f719e34ecd1ed160a5 BUG: 1349385 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/14783 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* tests: disable lock_revocation.t on NetBSDRaghavendra Talur2016-08-311-0/+1
| | | | | | | | | | | | | | | This has been consistently causing hangs in NetBSD machines. I have not been able to debug the issue and we have merge deadline for 3.9. It would be better to disable this for now. Change-Id: I8c63940aa26f78dd9994bb63293a5757835ec52b BUG: 1369401 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/15374 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* glusterd: clean up old port and allocate new one on every restartAtin Mukherjee2016-08-031-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | GlusterD as of now was blindly assuming that the brick port which was already allocated would be available to be reused and that assumption is absolutely wrong. Solution : On first attempt, we thought GlusterD should check if the already allocated brick ports are free, if not allocate new port and pass it to the daemon. But with that approach there is a possibility that if PMAP_SIGNOUT is missed out, the stale port will be given back to the clients where connection will keep on failing. Now given the port allocation always start from base_port, if everytime a new port has to be allocated for the daemons, the port range will still be under control. So this fix tries to clean up old port using pmap_registry_remove () if any and then goes for pmap_registry_alloc () Change-Id: If54a055d01ab0cbc06589dc1191d8fc52eb2c84f BUG: 1221623 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/15005 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Avra Sengupta <asengupt@redhat.com>
* features/locks: Add lock revocation functionality to posix locks translatorRichard Wareing2016-07-181-0/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: - Motivation: Prevents cluster instability by mis-behaving clients causing bricks to OOM due to inode/entry lock pile-ups. - Adds option to strip clients of entry/inode locks after N seconds - Adds option to clear ALL locks should the revocation threshold get hit - Adds option to clear all or granted locks should the max-blocked threshold get hit (can be used in combination w/ revocation-clear-all). - Options are: features.locks-revocation-secs <integer; 0 to disable> features.locks-revocation-clear-all [on/off] features.locks-revocation-max-blocked <integer> - Adds monkey-locking option to ignore 1% of unlock requests (dev only) features.locks-monkey-unlocking [on/off] - Adds logging to indicate revocation event & reason Test Plan: First you will need TWO fuse mounts for this repro. Call them /mnt/patchy1 & /mnt/patchy2. 1. Enable monkey unlocking on the volume: gluster vol set patchy features.locks-monkey-unlocking on 2. From the "patchy1", use DD or some other utility to begin writing to a file, eventually the dd will hang due to the dropped unlocked requests. This now simulates the broken client. Run: for i in {1..1000};do dd if=/dev/zero of=/mnt/patchy1/testfile bs=1k count=10;done' ...this will eventually hang as the unlock request has been lost. 3. Goto another window and setup the mount "patchy2" @ /mnt/patchy2, and observe that 'echo "hello" >> /mnt/patchy2/testfile" will hang due to the inability of the client to take out the required lock. 4. Next, re-start the test this time enabling lock revocation; use a timeout of 2-5 seconds for testing: 'gluster vol set patchy features.locks-revocation-secs <2-5>' 5. Wait 2-5 seconds before executing step 3 above this time. Observe that this time the access to the file will succeed, and the writes on patchy1 will unblock until they hit another failed unlock request due to "monkey-unlocking". BUG: 1350867 Change-Id: I814b9f635fec53834a26db634d1300d9a61057d8 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/14816 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* api: use versioned symbols for minor ABI changeKaleb S KEITHLEY2016-06-141-2/+2
| | | | | | | | | | | | | | | | | | | | | | | No need to expand the API/ABI. E.g., see how glfs_lookupat changed between 3.7.0 and 3.7.4 IIRC (I originally argued against versioning the library. I wanted to just add new functions as they were needed, as was initially done for glfs_ipc and glfs_ipc_xd in the master branch for 4.0. But others strongly wanted versioning.) Having made the decision to use versioning, I believe we should continue. At least until we have a public decision that we're no longer going to use versioning. Change-Id: I0c3b2c1cbb297ae2b2864b647c224922987d74ad Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14717 Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* glusterd: volgen and volume set changes for leasesPoornima G2016-06-021-0/+22
| | | | | | | | | | | | | Add leases xlator in volgen and also add corresponding volume set options Change-Id: Ic5de50cdb87eaf6a833e739bc7e08fecbeca3de3 BUG: 1319992 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/11722 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* features/locks: Implement mandatory locksAnoop C S2016-05-022-0/+218
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Initial change to fix/enable the mandatory locking support in GlusterFS as per the following design: https://review.gluster.org/#/c/12014/ Accordingly 'locks.mandatory-locking' option is available as part of this change which will accept one among the following values: * off * file * forced * optimal See design doc for more details Change-Id: I14c489b3f8af5ebcbfa155a03f0c175e9558ac46 BUG: 762184 Signed-off-by: Anoop C S <anoopcs@redhat.com> Reviewed-on: http://review.gluster.org/9768 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Poornima G <pgurusid@redhat.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* WORM/Retention Translator: Implementation of file level WORMkarthik-us2016-05-011-0/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To activate the file level worm feature, the features.read-only and features.worm options should be switched "off" on the volume and the features.worm-file-level should be switched "on". Both read-only and worm or worm-file-level cannot be switched "on" together. The files which are created when the worm-file-level option is set on the volume will have their own retention profile. If both worm and worm-file-level are "on" at that time the worm which is the volume level worm will have priority over file level worm. If worm-file level is switched "off" after some time and the read-only option is switched "on" then read-only will have priority. The current implementation allows the users to manually transmit a file to a WORM-Retained state by removing all the write bits of the file using the chmod command. The file will have a retention profile which contains the state of the file, mode of retention, and the default retention time. The file will be made WORM-Retained for a default of 120 seconds during which it will be immutable and undeletable and it sets the atime of the file to the time till which it is retained. After that period if any fop request comes for that file, will make the transition from WORM-Retained state to WORM state, where the file will be immutable but deletable and, it will reset the atime to the actual atime of the file. If a WORM file needs to be made undeletable again, it can be done by using the chmod command with all the write bits removed. There are two modes of retention: 1. Relax: where the retention time of a WORM-Retained file can be increased or decreased. 2. Enterprise: where the retention time of a WORM-Retained file can be increased but not be decreased. Whenever a utime change(touch -a, -t, ...)request comes for a file it checks the mode of retention before setting the utimes. This is done only if the file is WORM-Retained but for a WORM file it will change the utimes. Lazy auto commit: Whenever a file gets created it will store the creation time of the file or if a file already exists then any of the next unlink, link, truncate or rename fops will set the current time as the start time in an xattr. The next rename/unlink/truncate/link call will check for the auto commit period and if is is expired, then it will automatically do the state transition. If it is a normal file then it gets converted to WORM-Retained state. If it is a WORM-Retained file and its retention period is expired, then it gets converted to WORM state. Added the volume set options for the WORM translator. It allows the users to change the default values of auto-commit-period, default-retention-period, retention-mode. To make use of the file-level WORM first we have to set the 'worm-file' option to 'on'. The files which are created when the worm-file option is set on the volume will get WORM-Retained. Other files will work as usual and will not be WORMed. The auto-commit-period, retention-mode, and the default-retention-period values for the file will be set to the values which are set on the volume when the file is created. Added the tests to check the basic functionalities of the WORM/Retention feature. Change-Id: I77bd9777f9395a944d76b5cc35a5b48a3c14d148 BUG: 1326308 Signed-off-by: karthik-us <ksubrahm@redhat.com> Reviewed-on: http://review.gluster.org/13429 Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* glusterd: volume set changes for lock migrationSusant Palai2016-05-011-0/+34
| | | | | | | | | | | Change-Id: I48c6f9cdda47503615ba65882acd5eedf0a70c89 BUG: 1326085 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/14024 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* features/trash: wind mkdir with special pidAnoop C S2016-04-271-14/+6
| | | | | | | | | | | | | | | | | | | | | | | Recent changes done w.r.t handling of mkdir calls in posix translator resulted in crashing the brick process from trash translator. This was due to the changes made in posix translator to return EPERM for every mkdir calls without 'gfid-req' set in dictionary. In order to avoid gfid mismatches during directory creation from brick side trash translator does not set 'gfid-req'. This patch is to have an exemption for trash based on a special pid set for those mkdir calls originating from trash translator and to reset it in callback. This patch also includes a small optimization to the existing test case for trash feature. Change-Id: I59f084ac875e54342ecf2bffa6e43ebd84814153 BUG: 1317361 Signed-off-by: Anoop C S <anoopcs@redhat.com> Reviewed-on: http://review.gluster.org/13776 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* Revert "glusterd: Allocate fresh port on brick (re)start"Gaurav Kumar Garg2016-04-141-5/+0
| | | | | | | | | | | | | | | | | | | | | This reverts commit 34899d7 Commit 34899d7 introduced a change, where restarting a volume or rebooting a node result into fresh allocation of brick port. In production environment generally administrator makes firewall configuration for a range of ports for a volume. With commit 34899d7, on rebooting of node or restarting a volume might result into volume start fail because firewall might block fresh allocated port of a brick and also it will be difficult in testing because of fresh allocation of port. Change-Id: I7a90f69e8c267a013dc906b5228ca76e819d84ad BUG: 1322805 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/13989 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* tests: use trap mechanism to ensure that proper cleanups happenJeff Darcy2016-04-121-1/+2
| | | | | | | | | | | | | | | | | | | | | | | This actually consists of several parts. * Added a generic cleanup-scheduling mechanism. Instead of calling "trap ... EXIT" directly, just call "push_trapfunc ..." instead and your cleanup function will be called along with any others. * Converted a few tests to use push_trapfunc. * Added "push_trapfunc cleanup_lvm" to snapshot.rc to address the particular problem that's driving this - snapshot tests not calling cleanup_lvm on their own and leaving bad state for the next test. Change-Id: I548a97a26328390992fc71ee1f03c0463703f9d7 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/13933 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
* dht: extend time for "nuke" test's janitor-cleanup checkJeff Darcy2016-04-121-1/+1
| | | | | | | | | | | | | The previous check worked out to 50 deletes per second. That might have seemed generous, but NetBSD regression tests were failing because it can't hit that figure reliably. Change-Id: Ifbd8f4547caf53a8a8d11ad586aa8051f77ddc40 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/13935 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* dht: add "nuke" functionality for efficient server-side deletionJeff Darcy2016-04-071-0/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This turns a special xattr into an rmdir with flags set. When that hits the posix translator on the server side, that causes the file/directory to be moved into the special "landfill" directory. From there, the posix janitor thread will take care of deleting it entirely on the server side - traversing it recursively if necessary. A couple of secondary issues were fixed to make this effective. * FUSE now ensures that setxattr values are NUL terminated. * The janitor thread now gets woken up immediately when something is placed in 'landfill' instead of only when file descriptors need to be closed. * The default landfill-emptying interval was reduced to 10s. To use the feature, issue a setxattr something like this: setfattr -n glusterfs.dht.nuke -v "" /mnt/glusterfs/vol/some_dir The value doesn't actually matter; the mere receipt of a request with this key is sufficient. Some day it might be useful to allow setting a required value as a sort of password, so that only those who know it can access the underlying special functionality. Change-Id: I8a343c2cdb40a76d5a06c707191fb67babb8514f Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/13878 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* glusterd: Allocate fresh port on brick (re)startAtin Mukherjee2016-04-011-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | There is no point of using the same port through the entire volume life cycle for a particular bricks process since there is no guarantee that the same port would be free and no other application wouldn't consume it in between the glusterd/volume restart. We hit a race where on glusterd restart the daemon services start followed by brick processes and the time brick process tries to bind with the port which was allocated by glusterd before a restart is been already consumed by some other client like NFS/SHD/... Note : This is a short term solution as here we reduce the race window but don't eliminate it completely. As a long term solution the port allocation has to be done by glusterfsd and the same should be communicated back to glusterd for book keeping Change-Id: Ibbd1e7ca87e51a7cd9cf216b1fe58ef7783aef24 BUG: 1322805 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/13865 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* experimental: add fdl (Full Data Logging) translatorJeff Darcy2016-02-133-0/+188
| | | | | | | | | | | | | | | | | | | | | | | | NSR needs logging that is different than our existing changelog in several ways: * Full data, not just metadata * Pre-op, not post-op * High performance * Supports the concept of time-bounded "terms" Others (for example EC) might need the same thing. This patch adds such a translator. It also adds code to dump the resulting journals, and to replay them using syncops, plus (very rudimentary) tests for all of the above. Change-Id: I29680a1b4e0a9e7d5a8497fef302c46434b86636 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/12450 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* SSL improvements: ECDH, DH, CRL, and accessible optionsEmmanuel Dreyfus2015-08-053-0/+250
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Introduce ssl.dh-param option to specify a file containinf DH parameters. If it is provided, EDH ciphers are available. - Introduce ssl.ec-curve option to specify an elliptic curve name. If unspecified, ECDH ciphers are available using the prime256v1 curve. - Introduce ssl.crl-path option to specify the directory where the CRL hash file can be found. Setting to NULL disable CRL checking, just like the default. - Make all ssl.* options accessible through gluster volume set. - In default cipher list, exclude weak ciphers instead of listing the strong ones. - Enforce server cipher preference. - introduce RPC_SET_OPT macro to factor repetitive code in glusterd-volgen.c - Add ssl-ciphers.t test to check all the features touched by this change. Change-Id: I7bfd433df6bbf176f4a58e770e06bcdbe22a101a BUG: 1247152 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/11735 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaushal M <kaushal@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* Tests portability: umount(8)Emmanuel Dreyfus2015-06-091-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1) Avoid hangs on unmounting NFS on NetBSD NetBSD umount(8) on a NFS mount whose server is gone will wait forever because umount(8) calls realpath(3) and tries to access the mount before it calls unmount(2). The non-portable, NetBSD-specific umount -R flag prevent that behavior. We therefore introduce UMOUNT_F, defined as "umount -f" on Linux and "umount -f -R" on NetBSD to take care of forced unmounts, especially in the NFS case. 2) Enforce usage of force_umount wrapper with timeout Whenever umount is used it should be wrapped in force_umount with tiemout handling. That saves us timing issues, and it handles the NetBSD NFS case. 3) Cleanup kernel cache flush. We used (cd $M0 && umount $M0 ) as a portable kernel cache flush trick, but it does not flush everything we need on Linux. Introduce a drop_cache() shell function that reverts to previously used echo 3 > /proc/sys/vm/drop_caches on Linux, and keeps (cd $M0 && umount $M0 ) on other systems. BUG: 1129939 Change-Id: Iab1f5a023405f1f7270c42b595573702ca1eb6f3 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/11114 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-17 23:17:39 +0000