summaryrefslogtreecommitdiffstats
path: root/tests/features
Commit message (Collapse)AuthorAgeFilesLines
* glusterd: clean up old port and allocate new one on every restartAtin Mukherjee2016-11-231-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/15005/9. GlusterD as of now was blindly assuming that the brick port which was already allocated would be available to be reused and that assumption is absolutely wrong. Solution : On first attempt, we thought GlusterD should check if the already allocated brick ports are free, if not allocate new port and pass it to the daemon. But with that approach there is a possibility that if PMAP_SIGNOUT is missed out, the stale port will be given back to the clients where connection will keep on failing. Now given the port allocation always start from base_port, if everytime a new port has to be allocated for the daemons, the port range will still be under control. So this fix tries to clean up old port using pmap_registry_remove () if any and then goes for pmap_registry_alloc () This patch is being ported to 3.8 branch because, the brick process blindly re-using old port, without registering with the pmap server, causes snapd daemon to not start properly, even though snapd registers with the pmap server. With this patch, all the brick processes and snapd will register with the pmap server to either get the same port, or a new port, and avoid port collision. > Reviewed-on: http://review.gluster.org/15005 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Avra Sengupta <asengupt@redhat.com> (cherry picked from commit c3dee6d35326c6495591eb5bbf7f52f64031e2c4) Change-Id: If54a055d01ab0cbc06589dc1191d8fc52eb2c84f BUG: 1369766 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/15308 Tested-by: Avra Sengupta <asengupt@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* glusterd: volgen and volume set changes for leasesPoornima G2016-06-131-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | Add leases xlator in volgen and also add corresponding volume set options Backport of http://review.gluster.org/#/c/11722 Cherry picked from commit bc525b01002d7649be0ac28c5761c8ce92b6a658: > Change-Id: Ic5de50cdb87eaf6a833e739bc7e08fecbeca3de3 > BUG: 1319992 > Signed-off-by: Poornima G <pgurusid@redhat.com> > Reviewed-on: http://review.gluster.org/11722 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Smoke: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Change-Id: Ic5de50cdb87eaf6a833e739bc7e08fecbeca3de3 BUG: 1342350 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/14626 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com>
* build: remove jbr and fdl, they is not ready for 3.8Niels de Vos2016-05-273-189/+0
| | | | | | | | | | | Change-Id: Ia24ad18c43d56a751988e562323ede26d7785848 BUG: 1317278 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/14519 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* features/locks: Implement mandatory locksAnoop C S2016-05-032-0/+218
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Initial change to fix/enable the mandatory locking support in GlusterFS as per the following design: https://review.gluster.org/#/c/12014/ Accordingly 'locks.mandatory-locking' option is available as part of this change which will accept one among the following values: * off * file * forced * optimal See design doc for more details > Reviewed-on: http://review.gluster.org/9768 > Smoke: Gluster Build System <jenkins@build.gluster.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Poornima G <pgurusid@redhat.com> > Reviewed-by: Raghavendra Talur <rtalur@redhat.com> > Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> > Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> (cherry picked from commit 4517bf8dd6de310950cc5a612955aa3a2fddb57e) Change-Id: I14c489b3f8af5ebcbfa155a03f0c175e9558ac46 BUG: 1332162 Signed-off-by: Anoop C S <anoopcs@redhat.com> Reviewed-on: http://review.gluster.org/14149 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* WORM/Retention Translator: Implementation of file level WORMkarthik-us2016-05-011-0/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To activate the file level worm feature, the features.read-only and features.worm options should be switched "off" on the volume and the features.worm-file-level should be switched "on". Both read-only and worm or worm-file-level cannot be switched "on" together. The files which are created when the worm-file-level option is set on the volume will have their own retention profile. If both worm and worm-file-level are "on" at that time the worm which is the volume level worm will have priority over file level worm. If worm-file level is switched "off" after some time and the read-only option is switched "on" then read-only will have priority. The current implementation allows the users to manually transmit a file to a WORM-Retained state by removing all the write bits of the file using the chmod command. The file will have a retention profile which contains the state of the file, mode of retention, and the default retention time. The file will be made WORM-Retained for a default of 120 seconds during which it will be immutable and undeletable and it sets the atime of the file to the time till which it is retained. After that period if any fop request comes for that file, will make the transition from WORM-Retained state to WORM state, where the file will be immutable but deletable and, it will reset the atime to the actual atime of the file. If a WORM file needs to be made undeletable again, it can be done by using the chmod command with all the write bits removed. There are two modes of retention: 1. Relax: where the retention time of a WORM-Retained file can be increased or decreased. 2. Enterprise: where the retention time of a WORM-Retained file can be increased but not be decreased. Whenever a utime change(touch -a, -t, ...)request comes for a file it checks the mode of retention before setting the utimes. This is done only if the file is WORM-Retained but for a WORM file it will change the utimes. Lazy auto commit: Whenever a file gets created it will store the creation time of the file or if a file already exists then any of the next unlink, link, truncate or rename fops will set the current time as the start time in an xattr. The next rename/unlink/truncate/link call will check for the auto commit period and if is is expired, then it will automatically do the state transition. If it is a normal file then it gets converted to WORM-Retained state. If it is a WORM-Retained file and its retention period is expired, then it gets converted to WORM state. Added the volume set options for the WORM translator. It allows the users to change the default values of auto-commit-period, default-retention-period, retention-mode. To make use of the file-level WORM first we have to set the 'worm-file' option to 'on'. The files which are created when the worm-file option is set on the volume will get WORM-Retained. Other files will work as usual and will not be WORMed. The auto-commit-period, retention-mode, and the default-retention-period values for the file will be set to the values which are set on the volume when the file is created. Added the tests to check the basic functionalities of the WORM/Retention feature. Change-Id: I77bd9777f9395a944d76b5cc35a5b48a3c14d148 BUG: 1326308 Signed-off-by: karthik-us <ksubrahm@redhat.com> Reviewed-on: http://review.gluster.org/13429 Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* glusterd: volume set changes for lock migrationSusant Palai2016-05-011-0/+34
| | | | | | | | | | | Change-Id: I48c6f9cdda47503615ba65882acd5eedf0a70c89 BUG: 1326085 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/14024 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* features/trash: wind mkdir with special pidAnoop C S2016-04-271-14/+6
| | | | | | | | | | | | | | | | | | | | | | | Recent changes done w.r.t handling of mkdir calls in posix translator resulted in crashing the brick process from trash translator. This was due to the changes made in posix translator to return EPERM for every mkdir calls without 'gfid-req' set in dictionary. In order to avoid gfid mismatches during directory creation from brick side trash translator does not set 'gfid-req'. This patch is to have an exemption for trash based on a special pid set for those mkdir calls originating from trash translator and to reset it in callback. This patch also includes a small optimization to the existing test case for trash feature. Change-Id: I59f084ac875e54342ecf2bffa6e43ebd84814153 BUG: 1317361 Signed-off-by: Anoop C S <anoopcs@redhat.com> Reviewed-on: http://review.gluster.org/13776 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* Revert "glusterd: Allocate fresh port on brick (re)start"Gaurav Kumar Garg2016-04-141-5/+0
| | | | | | | | | | | | | | | | | | | | | This reverts commit 34899d7 Commit 34899d7 introduced a change, where restarting a volume or rebooting a node result into fresh allocation of brick port. In production environment generally administrator makes firewall configuration for a range of ports for a volume. With commit 34899d7, on rebooting of node or restarting a volume might result into volume start fail because firewall might block fresh allocated port of a brick and also it will be difficult in testing because of fresh allocation of port. Change-Id: I7a90f69e8c267a013dc906b5228ca76e819d84ad BUG: 1322805 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/13989 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* tests: use trap mechanism to ensure that proper cleanups happenJeff Darcy2016-04-121-1/+2
| | | | | | | | | | | | | | | | | | | | | | | This actually consists of several parts. * Added a generic cleanup-scheduling mechanism. Instead of calling "trap ... EXIT" directly, just call "push_trapfunc ..." instead and your cleanup function will be called along with any others. * Converted a few tests to use push_trapfunc. * Added "push_trapfunc cleanup_lvm" to snapshot.rc to address the particular problem that's driving this - snapshot tests not calling cleanup_lvm on their own and leaving bad state for the next test. Change-Id: I548a97a26328390992fc71ee1f03c0463703f9d7 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/13933 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
* dht: extend time for "nuke" test's janitor-cleanup checkJeff Darcy2016-04-121-1/+1
| | | | | | | | | | | | | The previous check worked out to 50 deletes per second. That might have seemed generous, but NetBSD regression tests were failing because it can't hit that figure reliably. Change-Id: Ifbd8f4547caf53a8a8d11ad586aa8051f77ddc40 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/13935 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* dht: add "nuke" functionality for efficient server-side deletionJeff Darcy2016-04-071-0/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This turns a special xattr into an rmdir with flags set. When that hits the posix translator on the server side, that causes the file/directory to be moved into the special "landfill" directory. From there, the posix janitor thread will take care of deleting it entirely on the server side - traversing it recursively if necessary. A couple of secondary issues were fixed to make this effective. * FUSE now ensures that setxattr values are NUL terminated. * The janitor thread now gets woken up immediately when something is placed in 'landfill' instead of only when file descriptors need to be closed. * The default landfill-emptying interval was reduced to 10s. To use the feature, issue a setxattr something like this: setfattr -n glusterfs.dht.nuke -v "" /mnt/glusterfs/vol/some_dir The value doesn't actually matter; the mere receipt of a request with this key is sufficient. Some day it might be useful to allow setting a required value as a sort of password, so that only those who know it can access the underlying special functionality. Change-Id: I8a343c2cdb40a76d5a06c707191fb67babb8514f Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/13878 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* glusterd: Allocate fresh port on brick (re)startAtin Mukherjee2016-04-011-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | There is no point of using the same port through the entire volume life cycle for a particular bricks process since there is no guarantee that the same port would be free and no other application wouldn't consume it in between the glusterd/volume restart. We hit a race where on glusterd restart the daemon services start followed by brick processes and the time brick process tries to bind with the port which was allocated by glusterd before a restart is been already consumed by some other client like NFS/SHD/... Note : This is a short term solution as here we reduce the race window but don't eliminate it completely. As a long term solution the port allocation has to be done by glusterfsd and the same should be communicated back to glusterd for book keeping Change-Id: Ibbd1e7ca87e51a7cd9cf216b1fe58ef7783aef24 BUG: 1322805 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/13865 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* experimental: add fdl (Full Data Logging) translatorJeff Darcy2016-02-133-0/+188
| | | | | | | | | | | | | | | | | | | | | | | | NSR needs logging that is different than our existing changelog in several ways: * Full data, not just metadata * Pre-op, not post-op * High performance * Supports the concept of time-bounded "terms" Others (for example EC) might need the same thing. This patch adds such a translator. It also adds code to dump the resulting journals, and to replay them using syncops, plus (very rudimentary) tests for all of the above. Change-Id: I29680a1b4e0a9e7d5a8497fef302c46434b86636 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/12450 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* SSL improvements: ECDH, DH, CRL, and accessible optionsEmmanuel Dreyfus2015-08-053-0/+250
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Introduce ssl.dh-param option to specify a file containinf DH parameters. If it is provided, EDH ciphers are available. - Introduce ssl.ec-curve option to specify an elliptic curve name. If unspecified, ECDH ciphers are available using the prime256v1 curve. - Introduce ssl.crl-path option to specify the directory where the CRL hash file can be found. Setting to NULL disable CRL checking, just like the default. - Make all ssl.* options accessible through gluster volume set. - In default cipher list, exclude weak ciphers instead of listing the strong ones. - Enforce server cipher preference. - introduce RPC_SET_OPT macro to factor repetitive code in glusterd-volgen.c - Add ssl-ciphers.t test to check all the features touched by this change. Change-Id: I7bfd433df6bbf176f4a58e770e06bcdbe22a101a BUG: 1247152 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/11735 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaushal M <kaushal@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* Tests portability: umount(8)Emmanuel Dreyfus2015-06-091-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1) Avoid hangs on unmounting NFS on NetBSD NetBSD umount(8) on a NFS mount whose server is gone will wait forever because umount(8) calls realpath(3) and tries to access the mount before it calls unmount(2). The non-portable, NetBSD-specific umount -R flag prevent that behavior. We therefore introduce UMOUNT_F, defined as "umount -f" on Linux and "umount -f -R" on NetBSD to take care of forced unmounts, especially in the NFS case. 2) Enforce usage of force_umount wrapper with timeout Whenever umount is used it should be wrapped in force_umount with tiemout handling. That saves us timing issues, and it handles the NetBSD NFS case. 3) Cleanup kernel cache flush. We used (cd $M0 && umount $M0 ) as a portable kernel cache flush trick, but it does not flush everything we need on Linux. Introduce a drop_cache() shell function that reverts to previously used echo 3 > /proc/sys/vm/drop_caches on Linux, and keeps (cd $M0 && umount $M0 ) on other systems. BUG: 1129939 Change-Id: Iab1f5a023405f1f7270c42b595573702ca1eb6f3 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/11114 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* dht: Add lookup-optimize configuration option for DHTShyam2015-06-021-1/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently with commit 4eaaf5 a mixed version cluster would have issues if lookup-uhashed is set to auto, as older clients would fail to validate the layouts if newer clients (i.e 3.7 or upwards) create directories. Also, in a mixed version cluster rebalance daemon would set commit hash for some subvolumes and not for the others. This commit fixes this problem by moving the enabling of the functionality introduced in the above mentioned commit to a new dht option. This option also has a op_version of 3_7_1 thereby preventing it from being set in a mixed version cluster. It brings in the following changes, - Option can be set only if min version of the cluster is 3.7.1 or more - Rebalance and mkdir update the layout with the commit hashes only if this option is set, hence ensuring rebalance works in a mixed version cluster, and also directories created by newer clients do not cause layout errors when read by older clients - This option also supersedes lookup-unhased, to enable the optimization for lookups more deterministic and not conflict with lookup-unhashed settings. Option added is cluster.lookup-optimize, which is a boolean. Usage: # gluster volume set VOLNAME cluster.lookup-optimize on Change-Id: Ifd1d4ce3f6438fcbcd60ffbfdbfb647355ea1ae0 BUG: 1222126 Signed-off-by: Shyam <srangana@redhat.com> Reviewed-on: http://review.gluster.org/10797 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaushal M <kaushal@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* tests: Fix typo in ssl-authz.tDeepak C Shetty2015-05-271-1/+1
| | | | | | | | | | | | Fixes a typo that was incorrectly causing the ssl cipher list not to be set properly on the test volume. Change-Id: I7969988551aa0c76261e41ab2f6247b684dacd49 Signed-off-by: Deepak C Shetty <deepakcs@redhat.com> Reviewed-on: http://review.gluster.org/10914 Tested-by: NetBSD Build System Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tests: write log for glupy.t to the standard log directoryNiels de Vos2015-05-161-1/+1
| | | | | | | | | | | | | | | | | | There is (or was) a spurious test in glupy.t for which a separate log file was setup. The directory where the log is saved, is not available on NetBSD and this causes glupy.t to always fail, without a log. Instead of hard-coding the path to the log, use "gluster --print-logdir" to provide a LOGDIR environment variable. glupy.t now writes the log to an existing directory. BUG: 1163543 Change-Id: Ifa73198d06fa267856d0da9d25a4380329909124 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10801 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* dht: make lookup-unhashed=auto do something actually usefulJeff Darcy2015-05-101-0/+99
| | | | | | | | | | | | | | | | | | | | | | | | The key concept here is to determine whether a directory is "clean" by comparing its last-known-good topology to the current one for the volume. These are stored as "commit hashes" on the directory and the volume root respectively. The volume's commit hash changes whenever a brick is added or removed, and a fix-layout is done. A directory's commit hash changes only when a full rebalance (not just fix-layout) is done on it. If all bricks are present and have a directory commit hash that matches the volume commit hash, then we can assume that every file is in its "proper" place. Therefore, if we look for a file in that proper place and don't find it, we can assume it's not on any other subvolume and *safely* skip the global (broadcast to all) lookup. Change-Id: Id6ce4593ba1f7daffa74cfab591cb45960629ae3 BUG: 1219637 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Signed-off-by: Shyam <srangana@redhat.com> Reviewed-on: http://review.gluster.org/7702 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tests: Fix spurious failuresPranith Kumar K2015-05-031-1/+1
| | | | | | | | | | | | | - Use REBALANCE_TIMEOUT in EXPECT_WITHIN - Use fdatasync to prevent write-behind from giving success - Add logfile to glupy Change-Id: I51ab51644aaa4aa9d49f185e7b8959bb58be966b BUG: 1217766 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10487 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Tests: fix spurious failures in trash.tEmmanuel Dreyfus2015-04-261-46/+69
| | | | | | | | | | | | Fix timing problems that cause rare spurious failures in trash.t BUG: 1129939 Change-Id: I673e033b53b6b4bb993c22fadbdcee725b2c1e96 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10360 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Tests: trash.t portability fixesEmmanuel Dreyfus2015-04-221-47/+63
| | | | | | | | | | | | | | | | | | Fix various portability problems in trash.t - work around timing issues - avoid wildcard usage only supported by bash Original patch from Anoop C S and Jiffin Tony Thottan. Removed LONGER_HEAL_TIMEOUT from previous patch as it seems to run fine without it now. BUG: 1129939 Change-Id: I0f6f484209ef4db7e0a7b733b863927cb248e73e Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10327 Reviewed-by: Anoop C S <achiraya@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Tests: portability fixes for ipc.tEmmanuel Dreyfus2015-03-302-3/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This fixes portability problems in ipc.t so that it can run on NetBSD: 1) EOPNOTSUPP value is OS-dependent. Learn it from system headers instead of hard-coding it in the script 2) liglusterfs embbeds its own UUID implementation. The function name may be the same as in built(in implementation from libc, but with different prototype. In that case, we must make sure python will use libglusterfs's version, otherwise we will crash in libc's UUID code. Since dlopen() does not make any guarantee on what symbol will be used, me need to preload libglusterfs when loading python. This is done using LD_PRELOAD. 3) In python code we need to load with RTLD_GLOBAL global in order to have dependencies loaded 4) Python's ctypes.util.find_library does not lookup LD_LIBRARy_PATH and may therefore miss the library. On failure, retry with less portable but more reliable explicit name BUG: 1129939 Change-Id: I024cdfd03a5a42a8ec23de38a99e7349aba92ea8 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/9944 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Tests: portability fixesEmmanuel Dreyfus2015-03-301-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fix various portability problems in mount-nfs-auth.t, quota-ancestry-building.t and trash.t: - dd bs=1M is not portable, use dd bs=1024k instead - dd bs=1MB is not portable iether, use dd bs=1000000 instead - After restarting NFS service, wait for it to become available - After killing a process, wait for it to terminate - BSD awk does not accept a=b="", use a=""; b="" instead - NetBSD displays the original program name in paenthesis at the end of ps output. Strip it using sed 's/ *([^()]*)$//' is we want just the command - Do no use umount $N0, which leads to many troubles solved by EXPECT_WITHIN $UMOUNT_TIMEOUT "Y" umount_nfs $N0 - The -p option for mkdir must be before the directory name - du -b is not portable. Use ls -l instead. BUG: 1129939 Change-Id: I3d44a10a37d47ebb6a263c206566487e3ffb85d8 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10033 Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Anoop C S <achiraya@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* tests: fix rebalance_completedJeff Darcy2015-03-261-12/+1
| | | | | | | | | | | | This was causing spurious faiures in bug-884455.t and possibly elsewhere. Change-Id: Iad6b7515ca0c7c485300f79dcd2477efc76877f8 BUG: 1163543 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/9994 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* every/where: add GF_FOP_IPC for inter-translator communicationJeff Darcy2015-03-172-0/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Several features - e.g. encryption, erasure codes, or NSR - involve multiple cooperating translators which sometimes need a "private" means of communication amongst themselves. Historically we've used virtual or synthetic xattrs, but that's not very elegant and clutters up the getxattr/setxattr path which must also handle real xattr requests. This new fop should address that. The only argument is an int32_t "op" which should be recognized by the target translator. It is recommended that translators using these feature follow some convention regarding the ops that they define, to avoid conflicts. Using a hash of the target translator's type string as a base for a series of ops would probably be a good start. Any other information can be passed in both directions using xdata. The default behavior for this fop, as with any other, is to pass through to FIRST_CHILD. That makes use of this fop "transparent" to other translators that were written before it existed, but it also means that it only really works with pass-through translators. If a routing translator (such as DHT) or a fan-out translator (such as AFR) is involved, the IPC might not reach its intended destination unless those translators are modified to forward IPC fops along all paths. If an IPC gets all the way to storage/posix it is considered an error, much like an uncaught exception. We don't actually *do* anything in that case, but we do log it send back an EOPNOTSUPP error. This makes the "unrecognized opcode" condition distinguishable from the "no IPC support" condition (which would yield an RPC error instead) so clients can probe for the presence of a handler for their own favorite opcode and either use that or use old-school xattrs depending on the result. BUG: 1158628 Signed-off-by: Venky Shankar <vshankar@redhat.com> Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Change-Id: I84af1b17babe5b30ec03ecf027ae37d09b873968 Reviewed-on: http://review.gluster.org/8812 Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Features/trash : Combined patches for trash translatorAnoop C S2015-03-161-0/+218
| | | | | | | | | | | | | | | | | | | | | | | | | This is the combined patch set for supporting trash feature. http://www.gluster.org/community/documentation/index.php/Features/Trash Current patch includes the following features: * volume set options for enabling trash globally and exclusively for internal operations like self-heal and re-balance * volume set options for setting the eliminate path, trash directory path and maximum trashable file size. * test script for checking the functionality of the feature * brief documentation on different aspects of trash feature. Change-Id: Ic7486982dcd6e295d1eba0f4d5ee6d33bf1b4cb3 BUG: 1132465 Signed-off-by: Anoop C S <achiraya@redhat.com> Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: http://review.gluster.org/8312 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tests/features/ssl-authz.t: Fix spurious failuresEmmanuel Dreyfus2015-01-271-2/+9
| | | | | | | | | | | | | | | | | Fix two spurious failures in tests/features/ssl-authz.t 1) Wait for bricks to come online after starting a volume, so that the mount is usable without "socket not connected" error 2) For a mount that must fail, we may get the situation where there is no mount at all, which means creating a file will write to the mount point instead of failing. To cover that case, write the file and check it is absent from the brick. BUG: 1129939 Change-Id: If95e1d65ab23d11123f778c20f8110a3177b0e7f Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/9483 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* transport: fix default behavior for SSL authorizationJeff Darcy2015-01-091-1/+22
| | | | | | | | | | | | | | | | | | | | Previously, enabling SSL authentication/encryption but not authorization required explicitly setting ssl-allow=*. Now that same behavior is the default (i.e. when ssl-allow is not set). Also, there's no reason that a name used for *login* auth (typically a UUID for internal purposes or a human name when using SSL) should validate as an RFC-compliant host name or IP address. Therefore the validation only occurs when the auth type is "addr" (not "login" or anything else). Change-Id: I01485ff4f0ab37de4b182858235a5fb0cf4c3c7d BUG: 1179208 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/9397 Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* glupy: portability fixesEmmanuel Dreyfus2014-11-051-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes portability problems so that NetBSD passes tests/features/glupy.t - Use python-config to detect python build environment on all systems, not just Linux and Darwin. - Get the site-package directory from python and make sure we install glupy.py there, Previously we installed within glusterfs prefix, which caused a problem if it was different that python's prefix. - Set PYTHONPATH for tests so that the detected site-packages is used in python's search path. This should be useless, but let us have it just in case. - Pass glupy.so path from glusterfsd to glupy.py through an environment variable and use it in CDLL instead of "", as the later seems not portable (at least it fails on NetBSD). - Use gil_init_key pthread_getspecific to avoid deadlocks (that code was #ifdef out, perhaps because it was not needed on Linux, but it seems to be required for NetBSD. - Recover the error message from Python and send it to the logs to help debugging problems. BUG: 1129939 Change-Id: Icc71e77d6940f0759cc14c5c5cf7ca6fa431e0d2 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8978 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* Regression test portability: loopback devicesEmmanuel Dreyfus2014-10-301-8/+6
| | | | | | | | | | | | | | | | | | Introduce functions to deal with loopback devices setup, mount and umount. Remove test for xfsprogs for non Linux systems, as loopback devices can be populated with other filesystems (e.g.: FFS for NetBSD) While there, remove mount.nfs test for non Linux systems. At least NetBSD has it in base system as mount_nfs. BUG: 1129939 Change-Id: I816b36e1d3e6933f92acf19d9be8eeaaa333356e Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8914 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* socket: disallow CBC cipher modesJeff Darcy2014-10-271-0/+17
| | | | | | | | | | | | | | | | | | | | | | This is related to CVE-2014-3566 a.k.a. POODLE. http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-3566 POODLE is specific to CBC cipher modes in SSLv3. Because there is no way to prevent SSLv3 fallback on a system with an unpatched version of OpenSSL, users of such systems can only be protected by disallowing CBC modes. The default cipher-mode specification in our code has been changed accordingly. Users can still set their own cipher modes if they wish. To support them, the ssl-authz.t test script provides an example of how to combine the CBC exclusion with other criteria in a script. Change-Id: Ib1fa547082fbb7de9df94ffd182b1800d6e354e5 BUG: 1155328 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/8962 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Regression test portability: truncateEmmanuel Dreyfus2014-10-011-2/+2
| | | | | | | | | | | | | | Use truncate -s 1M instead of truncate --size=1m for portability sake BUG: 1129939 Change-Id: I5bf6ca1f9bb4fa3c91796a659a06bf368776b3e5 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8894 Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Harshavardhana <harsha@harshavardhana.net> Tested-by: Harshavardhana <harsha@harshavardhana.net>
* Sane default for SSL on OSXHarshavardhana2014-09-291-1/+1
| | | | | | | | | | | - /opt/local is not preferred anymore use /usr/local Change-Id: I30cad4cbd28850063f26121cace05371e13bb314 BUG: 1129939 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8872 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Use sane OS-dependent defaults for SSL configurationEmmanuel Dreyfus2014-09-261-3/+8
| | | | | | | | | | | | Current code assumes /etc/ssl exists, which may not be the case. Attempt to guess sane default for a few OS. BUG: 1129939 Change-Id: I0f3168f79b8f4275636581041740dfcaf25f3edd Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8790 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* porting: Provide setfattr/getfattr implementationHarshavardhana2014-09-053-22/+22
| | | | | | | | | | | | | | | | - Use 'getfattr' properly avoid redundant options during xattr query - Untabify certain parts of tests (remove tabs) - Avoid backtick evaluation for certain values to make code more portable. - Use awk on FreeBSD/Darwin, since 'wc' implementation is broken and adds spurious spaces in its output. Change-Id: I7dcc0b70874e43b4cda8c306ed18a31b7a3f990a BUG: 1131713 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8520 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Emmanuel Dreyfus <manu@netbsd.org> Tested-by: Emmanuel Dreyfus <manu@netbsd.org>
* porting: various fixes regression tests OSX/FreeBSDHarshavardhana2014-08-293-3/+3
| | | | | | | | | | | | | | | | | | | | - `wc -l` on OSX/FreeBSD adds spurious spaces, this clobbers up TAP output parsers - fix it. - `umount -l` doesn't exist on OSX/FreeBSD use 'umount -f' if available. - Add check for 'file' version, to handle mime type variations across versions - Converge 'glusterfs --attribute-timeout=0 --entry-timeout=0' into '$GFS' - Modify remaining 'mount -t nfs' to use 'mount_nfs' - Update sha1sum for OSX to use 'openssl sha1'. Change-Id: Id1012faa5d67a921513d220e7fa9cebafe830d34 BUG: 1131713 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8501 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* tests: weighted-rebalance.t shouldn't write to '/dev/tty'Harshavardhana2014-08-061-2/+2
| | | | | | | | | | | | | | | | | | | | On our jenkins instance "/dev/tty" doesn't exist, necessary output fails as below message ~~~ ./tests/features/weighted-rebalance.t: \ line 72: /dev/tty: No such device or address ~~~ Comment out the debugging code Change-Id: Iba29b80c8ba2dcaab3d6654d7c54332a915bffb8 BUG: 1114680 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8421 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* dht: support heterogeneous brick sizesJeff Darcy2014-07-121-0/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Calculation of layouts now considers the size of each brick, so that smaller bricks don't get an "unfair" share of allocations and start returning ENOSPC while the larger bricks still have plenty of space. The observation has been made that some clients might get ENOTCONN when trying to fetch disk-size information, and end up calculating layouts differently. The following meta-observations can be made. (1) This scenario is extremely unlikely in configurations with AFR. (2) The most likely consequence of this scenario is that some files will be placed sub-optimally by the client with the obsolete (non-weighted) layout. They'll still be found anyway, so this isn't a show stopper. (3) Without this patch it's *guaranteed* that some files will be placed sub-optimally, because any layout that fails to account for brick sizes is sub-optimal. (4) We shouldn't be doing fix-layout from two nodes simultaneously anyway. That's inefficient at best. Any instances of such behavior are separate bugs, which should be fixed separately. (5) In the most extreme edge case, two nodes doing weighted and non-weighted layout fixes could race and end up creating an internally inconsistent layout. This condition is still transient; it will be detected and repaired automatically the next time anyone fetches the layout. (If it's not that's also a preexisting bug that can show up in other contexts.) In conclusion, it's not the purpose of this patch to fix bugs elsewhere in DHT. Its purpose is to make life incrementally better for users who add new hardware with larger disks etc. than the older equipment. It's only one part of an ongoing process to improve layout management and repair, all the way up to support for multiple hash rings or tiering. Change-Id: I05eb6f9eface9cdaf8622e0260c8c7f29020447f BUG: 1114680 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/8093 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* TESTS : replace "umount" with "force_umount" giving 5 retries.Sachin Pandit2014-07-022-2/+4
| | | | | | | | | | | Change-Id: I89dd87b9843273c57843bb28706f284e32debc48 BUG: 1092850 Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/8167 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* rpc/auth: allow SSL identity to be used for authorizationJeff Darcy2014-07-021-0/+53
| | | | | | | | | | | | | | | | | | | Access to a volume is now controlled by the following options, based on whether SSL is enabled or not. * server.ssl-allow: get identity from certificate, no password needed * auth.allow: get identity and matching password from command line It is not possible to allow both simultaneously, since the connection itself is either using SSL or it isn't. Change-Id: I5a5be66520f56778563d62f4b3ab35c66cc41ac0 BUG: 1114604 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/3695 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* performance/readdir-ahead: introduce directory read-ahead translatorBrian Foster2013-09-041-0/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a translator to improve the performance of typical, sequential directory reads (i.e., ls). readdir-ahead begins preloading the contents of a directory on open and serves readdir requests from the preloaded content. readdir-ahead is currently implemented to only handle the single threaded directory read case. readdir-ahead is currently disabled by default. It can be enabled with the following command: gluster volume set <volname> readdir-ahead on The following are results of a getdents test on a single brick volume. Test info: - Single VM, gluster client/server. - Volume mounted with native client using --gid-timeout=2. - getdents on single directory with 100k 0-byte files. Test results: - !readdir-ahead read 3120080 bytes from offset 0 3 MiB, 4348 ops, 0:00:07.00 (416.590 KiB/sec and 594.4737 ops/sec) - readdir-ahead read 3120080 bytes from offset 0 3 MiB, 4348 ops, 0:00:03.00 (820.116 KiB/sec and 1170.3043 ops/sec) BUG: 980517 Change-Id: Ieceb9e1eb47d1d5b5af8da2bf03839537364653f Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-on: http://review.gluster.org/4519 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* glupy: Importing Jeff's glupy project into glusterfsRam Raja2013-05-101-0/+29
Change-Id: I3891ef6eaf6ede7c8cbedc3298ce2501a69b2b05 BUG: 961856 Original-author: Jeff Darcy <jdarcy@redhat.com> Signed-off-by: Ram Raja <rraja@redhat.com> Reviewed-on: http://review.gluster.org/4906 Reviewed-by: Justin Clift <jclift@redhat.com> Tested-by: Justin Clift <jclift@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-17 14:58:53 +0000