summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* tests: improve vagrant usabilityJeff Darcy2016-04-063-2/+9
| | | | | | | | | | | | | | | | | | | The first and most important change is to deal with the fact that "localhost" is an IPv6 address in the vagrant box we're using. This was non-obviously behind some of the NFS tests failing to run properly, so there's now an Ansible role etc. to make the necessary modifications in /etc/hosts. Also added "-j" to the make command, and changed the invocation of "run-tests.sh" so it passes through any arguments instead of running all tests every time. BUG: 1291537 Change-Id: I5e6125de043c41fdb2866875e999cb1fcd72ba52 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/13904 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com>
* marker: optimize mq_update_dirty_inode_taskvmallika2016-04-063-50/+58
| | | | | | | | | | | | | | | | In function mq_update_dirty_inode_task we do readdirp on a dirty directory and for entry we again do lookup to fecth the contribution xattr. We can fetch this contribution as part of readdirp Change-Id: I766593c0dba793f1ab3b43625acce1c7d9af8d7f BUG: 1320818 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13892 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com>
* marker: do mq_reduce_parent_size_txn in FG for unlink & rmdirvmallika2016-04-063-32/+90
| | | | | | | | | | | | | | | | | | | | | | | * If a "rm -rf" is performed by a client, we initiate a marker background operation mq_reduce_parent_size_txn for rmdir and unlink. mq_reduce_parent_size_txn can fail when updating size on the ancestor directories, if these directories are removed during the txn as the child-parent association removed in the dentry list. So execute mq_reduce_parent_size_txn in foreground and then do the UNWIND for rmdir and unlink FOP Change-Id: Iefcdced4c6ae0dbd43f92814d0ddcd1e33825864 BUG: 1322489 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13874 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* posix_acl: skip acl_permits for special clientsvmallika2016-04-061-11/+28
| | | | | | | | | | | Change-Id: I3f478b7e4ecab517200f50eb09f65a634c029437 BUG: 1320818 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13894 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* quota: check inode limits only when new file/dir is createdvmallika2016-04-064-65/+57
| | | | | | | | | | | | | | | When a inode limit is full, writes to any existing file fails with disk quota exceed even if usage limit is not set or usage limit is not full. BUG: 1323486 Change-Id: I9679fe26a2839ade0b1541fa7f0a2b71ac6dcc31 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13911 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com>
* marker: build_ancestry in markervmallika2016-04-063-19/+162
| | | | | | | | | | | | | | | | | | | | | | | * quota-enforcer doesn't execute build_ancestry in the below code path 1) Special client (PID < 0) 2) unlink 3) rename within the same directory 4) link within the same directory In these cases, marker accounting can fail as parent not found. We need to build_ancestry in marker if it doesn't find parent during update txn Change-Id: Idb7a2906500647baa6d183ba859b15e34769029c BUG: 1320818 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13857 CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com>
* dht: lock on subvols to prevent lookup vs rmdir raceSakshi2016-04-055-89/+435
| | | | | | | | | | | | | | | | | | | | | | There is a possibility that while an rmdir is completed on some non-hashed subvol and proceeding to others, a lookup selfheal can recreate the same directory on those subvols for which the rmdir had succeeded. Now the deletion of the parent directory will fail with an ENOTEMPTY. To fix this take blocking inodelk on the subvols before starting rmdir. Selfheal must also take blocking inodelk before creating the entry. Change-Id: I168a195c35ac1230ba7124d3b0ca157755b3df96 BUG: 1245065 Signed-off-by: Sakshi <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13528 CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* glusterd: fill real_path variable in brickinfo during volume importMohammed Rafi KC2016-04-052-0/+54
| | | | | | | | | | | | | | | | | | Variable "real_path" in brick info was used to store absolute path and using this we check the availability of the newly added bricks. But we were not populating the variable when we import a volume from peers. That caused to reset the real_path variable to zero, which resulted in validation failure for all new brick creation. Change-Id: I62be7bf452f0dcdf6aec3a4ec33c2e1fba2951ca BUG: 1323287 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13890 Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* quota/cli: display quota usage on path when limit not setvmallika2016-04-051-17/+0
| | | | | | | | | | | | | | | | When a quota limit is not set, 'quota list <path>' should still display the usage when a path parameter is specified. Change-Id: Ida12d9c5e348fbd98db4d68d9324c623cbdd3dea BUG: 1323360 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13893 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com>
* glusterd: build realpath post recreate of brick mount for snapshotAtin Mukherjee2016-04-052-16/+76
| | | | | | | | | | | | | | | | | | | | | | Commit a60c39d introduced a new field called real_path in brickinfo to hold the realpath() conversion. However at restore path for all snapshots and snapshot restored volumes the brickpath gets recreated post restoration of bricks which means the realpath () call will fail here for all the snapshots and cloned volumes. Fix is to store the realpath for snapshots and clones post recreating the brick mounts. For normal volume it would be done during retrieving the brick details from the store. Change-Id: Ia34853acddb28bcb7f0f70ca85fabcf73276ef13 BUG: 1322772 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/13869 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Avra Sengupta <asengupt@redhat.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com>
* cluster/afr: Use parallel dir scan functionalityPranith Kumar K2016-04-045-13/+70
| | | | | | | | | | | BUG: 1221737 Change-Id: I0ed71a72f0e33bd733723e00a01cf28378c5534e Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/13755 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* syncop: Add parallel dir scan functionalityPranith Kumar K2016-04-043-0/+246
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Most of this functionality's ideas are contributed by Richard Wareing, in his patch: https://bugzilla.redhat.com/show_bug.cgi?id=1221737#c1 VERY BIG thanks to him :-). After starting porting/testing the patch above, I found a few things we can improve in this patch based on the results we got in testing. 1) We are reading all the indices before we launch self-heals. In some customer cases I worked on there were almost 5million files/directories that needed heal. With such a big number self-heal daemon will be OOM killed if we go this route. So I modified this to launch heals based on a queue length limit. 2) We found that for directory hierarchies, multi-threaded self-heal patch was not giving better results compared to single-threaded self-heal because of the order problems. We improved index xlator to give gfid type to make sure that all directories in the indices are healed before the files that follow in that iteration of readdir output(http://review.gluster.org/13553). In our testing this lead to zero errors of self-heals as we were only doing self-heals in parallel for files and not directories. I think we can further improve self-heal speed for directories by doing name heals in parallel based on similar techniques Richard's patch showed. I think the best thing there would be to introduce synccond_t infra (pthread_cond_t kind of infra for syncops) which I am planning to implement for future releases. 3) Based on 1), 2) and the fact that afr already does retries of the indices in a loop I removed retries again in the threads. 4) After the refactor, the changes required to bring in multi-threaded self-heal for ec would just be ~10 lines, most of it will be about options initialization. Our tests found that we are able to easily saturate network :-). High level description of the final feature: Traditionally self-heal daemon reads the indices (gfids) that need to be healed from the brick and initiates heal one gfid at a time. Goal of this feature is to add parallelization to the way we do self-heals in a way we do not regress in any case but increase parallelization wherever we can. As part of this following knobs are introduced to improve parallelization: 1) We can launch 'max-jobs' number of heals in parallel. 2) We can keep reading indices as long as the wait-q for heals doesn't go over 'max-qlen' passed as arguments to multi-threaded dir_scan. As a first cut, we always do healing of directories in serial order one at a time but for files we launch heals in parallel. In future we can do name-heals of dir in parallel, but this is not implemented as of now. Reason for this is mentioned already in '2)' above. AFR/EC can introduce options like max-shd-threads/wait-qlength which can be set by users to increase the rate of heals when they want. Please note that the options will take effect only for the next crawl. BUG: 1221737 Change-Id: I8fc0afc334def87797f6d41e309cefc722a317d2 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/13569 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com>
* glusterd: clean dead initializationsPrasanna Kumar Kalever2016-04-044-15/+0
| | | | | | | | | | | | | | | | This patch cleans unused variable initialization as well as their declarations which are no where used in the code Change-Id: I784165fc6e91297758079699dd9583d5203b7793 BUG: 1253831 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/11929 Tested-by: Prasanna Kumar Kalever <pkalever@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* fdl: fix "used before set" warnings in libreconJeff Darcy2016-04-011-1/+23
| | | | | | | | | | | | Newer compilers were complaining about this in generated code, because cleanup code fragments were being added in the wrong order. Change-Id: I90ff6eccfa78dc012da6d8fd83443490a3cb276d Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/13891 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* fdl: fix make cleanPrasanna Kumar Kalever2016-04-011-1/+2
| | | | | | | | | | | | | | add fdl/src/librecon.c to CLEANFILES Change-Id: I91332ddda5b4157483389c32b74d90a6a4e1322a BUG: 1322323 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/13850 Tested-by: Prasanna Kumar Kalever <pkalever@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com>
* cli: Make CLI timeout configurableAtin Mukherjee2016-04-013-6/+14
| | | | | | | | | | | | | | | | When number of volumes go high in numbers command like gluster volume status may timeout from CLI since it follows a static timeout value of 120 secs. This patch introduces a flag '--timeout' which can pass a specific time out value for a particular command. Change-Id: I9795b709f27357fe5ee10ba19de8f4c11b63186f Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/13882 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* glusterd: Allocate fresh port on brick (re)startAtin Mukherjee2016-04-015-70/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | There is no point of using the same port through the entire volume life cycle for a particular bricks process since there is no guarantee that the same port would be free and no other application wouldn't consume it in between the glusterd/volume restart. We hit a race where on glusterd restart the daemon services start followed by brick processes and the time brick process tries to bind with the port which was allocated by glusterd before a restart is been already consumed by some other client like NFS/SHD/... Note : This is a short term solution as here we reduce the race window but don't eliminate it completely. As a long term solution the port allocation has to be done by glusterfsd and the same should be communicated back to glusterd for book keeping Change-Id: Ibbd1e7ca87e51a7cd9cf216b1fe58ef7783aef24 BUG: 1322805 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/13865 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* packaging: gluster-ganesha requires pacemaker, etc., on RHEL6Kaleb S KEITHLEY2016-04-011-0/+3
| | | | | | | | | | | | | | HA packages in RHEL6 are missing dependencies. (RHEL7 versions do.) Change-Id: I4e80b569ea89f97e8a110aee2f4554175aa0ec8b BUG: 1322801 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/13870 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: soumya k <skoduri@redhat.com>
* locks: fix build breakage from missing pl-messages.hJeff Darcy2016-04-011-1/+1
| | | | | | | | | Change-Id: I0f8af57dd96226b1e7a8a39237a82ce1cdd48be6 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/13876 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* tests: add open-behind.t to bad testsPrasanna Kumar Kalever2016-04-011-0/+1
| | | | | | | | | | | | | | | open-behind.t fails on NetBSD very frequently will be disabled until the problem is fixed Change-Id: I0655e4b0c8b3ad7fc9e2ccfcf679673832ff12a5 BUG: 1300253 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/13884 Tested-by: Prasanna Kumar Kalever <pkalever@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* glusterd/syncop: double free of frame stackMohammed Rafi KC2016-03-312-24/+65
| | | | | | | | | | | | | | | If rpc message from glusterd during brick op phase fails without sending, then frame was freed from the caller function and call back function. Change-Id: I63cb3be30074e9a074f6895faa25b3d091f5b6a5 BUG: 1322262 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13854 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* nsr: Introducing a happy path test caseAvra Sengupta2016-03-317-20/+248
| | | | | | | | | | | | | | | | | Write infra for nsr_server to not send a CHILD_UP before it gets a CHILD_UP from a quorum of it's children. Using the CHILD_UP received in the nsr client translator from the server, to decide the right time for starting the I/Os Change-Id: I9551638b306bdcbc6bae6aeda00316576ea832fe Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/13623 CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* cluster/afr: Don't lookup/forget inodesPranith Kumar K2016-03-315-43/+6
| | | | | | | | | | | | | | | | | | | | | | | | Problem: All inodes that are looked-up are always forgotten without fail in afr removing the benefits of them being in lru. This same code can cause crashes if between inode_lookup, inode_forget in afr if the top xlator does inode_forget(0). Fix: Don't use lookup/forget in afr. No benefits are there at the moment for keeping this code. It is impossible to prevent top xlators to do inode_forget(0). Found similar instances in ec and removed them even though those code paths are not going to be executed in any place other than heal-daemon. BUG: 1321554 Change-Id: Ia4cb236178f7f129cc898d53f0bbd26f494a2a8d Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/13834 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anuradha Talur <atalur@redhat.com>
* tools/glusterfind: add --full option to query commandMilind Changire2016-03-303-35/+81
| | | | | | | | | | | | | | | | | The --full option will use brickfind.py to list all files in the volume. The output file will contain url-encoded file names prefixed with the tag string to indicate that all files should be considered as new. The default tag string for the --full option is "NEW". This can be changed with the --tag-for-full-find command-line option. Change-Id: Ic85ba5db062e19df13ae9dc2de8a08eacb5c9792 BUG: 1286279 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/12779 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* features/changelog: Don't modify 'pargfid' in 'resolve_pargfid_to_path'Kotresh HR2016-03-304-3/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | If 'changelog' is enabled and 'changelog.capture-del-path' option is on it calls 'resolve_pargfid_to_path' which modifies 'pargfid' sent by caller. 'changelog_unlink' calls this routine directly with 'loc->pargfid' resulting it being modified and point to root instead of actual pargfid. This is a nasty bug and could cause the deletion of entry on root directory instead on actual parent when 'loc->path' is not present. Hence this fix to make 'pargfid' a const pointer and 'resolve_pargfid' to work on copy of pargfid. Glusterfind session creation enables these options by default to capture deleted entry path in changelog. Thanks Pranith for root causing this. Change-Id: I1d275a86f91c981b6889bedef93401c039d01d71 BUG: 1321955 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/13845 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Ravishankar N <ravishankar@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* glusterd: scan for open ports only in required rangeRaghavendra Talur2016-03-301-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It does not make sense to keep track of free ports from 0 to base_port if we are not going to use them. glusterd start times without this patch 2.622 2.478 2.455 2.590 2.400 glusterd start times with this patch 1.9 1.9 1.9 2.0 2.0 1.8 We save around half a second for every glusterd start. BUG: 1322237 Change-Id: I0456689d0afad50dd068f2325ebfca9bdeffe01a Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/13841 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* extras: allign glusterd stop response in init scriptAtin Mukherjee2016-03-301-0/+1
| | | | | | | | | | | | | | An echo is missing stop() function which results in bad allignment of glusterd stop output Change-Id: I63ac9571457db90574fb1bfbf4f18ba09d3df1af BUG: 1321762 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/13839 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* build: git ignore files generated by fdl xlatorPrasanna Kumar Kalever2016-03-301-0/+8
| | | | | | | | | | | | Change-Id: I7693624646cfab254c93afc0361b483a72c99841 BUG: 1322320 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/13851 Tested-by: Prasanna Kumar Kalever <pkalever@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* server: send lookup on root inode when itable is createdvmallika2016-03-308-31/+139
| | | | | | | | | | | | | | | | | | | | | | | | | * xlators like quota, marker, posix_acl can cause problems if inode-ctx are not created. sometime these xlarors may not get lookup on root inode with below cases 1) client may not send lookup on root inode (like NSR leader) 2) if the xlators on one of the bricks are not up, and client sending lookup during this time: brick can miss the lookup It is always better to make sure that there is one lookup on root. So send a first lookup when the inode table is created * When sending lookup on root, new inode is created, we need to use itable->root instead Change-Id: Iff2eeaa1a89795328833a7761789ef588f11218f BUG: 1320818 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13837 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* tools/glusterfind: Handling Unicode file namesAravinda VK2016-03-304-48/+34
| | | | | | | | | | | | | | | | | | | | Unicode filenames handled cleanly with this patch. Changelog files and output files are opened with utf-8 encoding using codecs.open. urllib.quote_plus and unquote_plus will not handle Unicode so, encode Unicode to 8-bit string version before calling unquote. urllib.quote_plus requires 8-bit string itself so do not decode to Unicode if we need to use quote_plus(when --no-encode=false). Decode to unicode in --no-encode is set. BUG: 1319717 Change-Id: If5561c749ab5529445650d322c831eb4da22b65a Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13798 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* cluster/ec: Rebalance hangs during renameAshish Pandey2016-03-308-50/+221
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: During the rename of a particular file (ec is holding blocking inodelk on the parent directory), if the rename of another file under the same directory comes. EC does not release the lock and goes ahead and renames the "new" file with the "already held lock". That causes rebalance process to be blocked on a lock which has been acquired by rename. Solution: While rename fop comes, ec takes blocking inodelk on old and new parent of the file. Before releasing, every lock held by ec, it waits for some "time" to see if that lock can be reused by the next fop. If within this "time" some other request comes, it releases this lock based on condition "lock count > 1" To get this "lock count" for rename fop, we have implemented "pl_rename" in feature/lock. Also, on ec side, changed the condition to release the lock based on the type of fop and old and new parent directories. Change-Id: I979dbab1185df962e8f305a6074ae1186ffe7db0 Bug: 1304988 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: http://review.gluster.org/13460 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
* libglusterfs: open cmd_history log file with O_APPEND and O_WRONLYAtin Mukherjee2016-03-291-1/+1
| | | | | | | | | | | | | | | | | Commit 8fdfa0c introduced a fix to ensure cmd_history file is log rotated properly. However with this fix fdopen() is called with mode "a" on a fd which was not opened with O_WRONLY & O_APPEND resulting into a fdopen() failure. Fix is to open cmd_history.log file with O_CREATE|O_WRONLY|O_APPEND mode Change-Id: I75ef350560aa6d5435c78c5fd83adfde1a73bfc3 BUG: 1286959 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/13829 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* afr: add mtime based split-brain resolution to CLIRavishankar N2016-03-298-18/+145
| | | | | | | | | | | | | | | | | | | Extended the CLI to include support for split-brain resolution based on mtime. The command syntax is: $:gluster volume heal <VOLNAME> split-brain latest-mtime <FILE> where <FILE> can be either the full file name as seen from the root of the volume (or) the gfid-string representation of the file. Change-Id: I7a16f72ff1a4495aa69f43f22758a9404e958b4f BUG: 1321322 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/13828 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* build : fixing dependency issue for glusterfs-ganesha in epel6Jiffin Tony Thottan2016-03-291-2/+1
| | | | | | | | | | | | | | | | | | glusterfs-ganesha requires "dbus-send" binary, but location of this binary is different in epel6 and epel7. For epel6 it is /bin/ and epel7 it is /usr/bin/. So the patch will add dependency based on package dbus not on the binary dbus-send Change-Id: I266af7b983791be648749f01a04b38479d6f7c03 BUG: 1321872 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: http://review.gluster.org/13842 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* tier/libgfdb: Ordering query results from libgfdbJoseph Fernandes2016-03-291-5/+23
| | | | | | | | | | | | | | | | | | | When querying we will order the query result to get the hotest or the coldest files in the queried list so that these files are migrated first. Now here we are giving priority to the write heat(time and counters), as it requires complex queries to have a composite ordering of write and read + it has it impact on performance. Change-Id: I2e0415dcfad4218b42c68fc5c2ed8d1f075ce9ea Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/13607 Smoke: Gluster Build System <jenkins@build.gluster.com> Tested-by: Joseph Fernandes NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* debug/trace: Print {acm}times as integersKrutika Dhananjay2016-03-291-13/+13
| | | | | | | | | | | | | This helps in debugging issues with mtime, ctime etc. Change-Id: I4a0335f17c6b40e6c627658e91f43989c3efca22 BUG: 1302948 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/13826 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* glusterd: Add a new event to handle multi-net probesKaushal M2016-03-284-4/+17
| | | | | | | | | | | | | | This allows GlusterD to send updates to all other nodes when attaching new addresses using multi-net peer probe. Change-Id: I62846be750ab3721912e7b49656594347ea61723 BUG: 1320458 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/13817 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* tier/dht : Attach tier fix layout to run in backgroundJoseph Fernandes2016-03-285-30/+266
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. Spawn a thread for background fix-layout for tier process. 2. Once the fix-layout is completed a marker xttr is set on the root of volume to mark the completion of the background fixlayout, so that even if the tier process is spawned again, fixlayout will not be issued, if it was completed last time. 3. Please note that promotion of legacy files will happen eventually as the ctr lookup heal in the fixlayout slowly heals the ctr db for legacy files OR the ctr lookup heal happend due to a name lookup. 4. When a detach tier is successful in evacuation data from hot tier, we remove the marker xattr is removed. So that next attach tier runs the background tier fixlayout. what is remaining ? 1. Instead of clearing the marker xattr of tiering fix layout at the end of detach start clear it during detach commit. But the issue is detach commit is a glusterd operation and the volume is not mounted in glusterd. The reason we want to do it in detach commit is that if the admin wants to attach the same tier again, then a background fixlayout will be triggered, which would not be needed. 2. Clearing the CTR DB of the cold bricks when there is a detach commit, as it will be having entries which will be stale when the volume is used, with ctr off (ctr is switched off only when we have detach commit.) Change-Id: Ibe343572e95865325cd0eef4d0b976b626a3c0c5 BUG: 1313228 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/13491 Smoke: Gluster Build System <jenkins@build.gluster.com> Tested-by: Joseph Fernandes NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* cluster/afr: Fix read-child selection in entry create fopPranith Kumar K2016-03-282-3/+46
| | | | | | | | | | | | | | | | | | When an entry is being created the inode is yet to be linked so args must be filled with gfid and ia_type for it to give consistent iatt. Also handle Dht sending fops on inode not yet linked. BUG: 1302948 Change-Id: I6969cacb437cad02f66716f3bf8ec004ffe7c691 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/13827 Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anuradha Talur <atalur@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* storage/posix: send proper iatt attributes for the root inodeRaghavendra Bhat2016-03-266-70/+149
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * changes in posix to send proper iatt attributes for the root directory when ancestry is built. Before posix was filling only the gfid and the inode type in the iatt structure keeping rest of the fields zeros. This was cached by posix-acl and used to send EACCES when some fops came on that object if the uid of the caller is same as the uid of the object on the disk. * getting and setting inode_ctx in function 'posix_acl_ctx_get' is not atomic and can lead to memory leak when there are multiple looups for an inode at same time. This patch fix this problem * Linking an inode in posix_build_ancestry, can cause a race in posix_acl. When parent inode is linked in posix_build_ancestry, and before it reaches posix_acl_readdirp_cbkc, reate/lookup can come on a leaf-inode, as parent-inode-ctx not yet updated in posix_acl_readdirp_cbk, create/lookup can fail with EACCESS. So do the inode linking in the quota xlator Change-Id: I3101eefb65551cc4162c4ff2963be1b73deacd6d BUG: 1320818 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/13730 Tested-by: Vijaikumar Mallikarjuna <vmallika@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* glfs/heal: Use encrypted connection in shdAshish Pandey2016-03-241-0/+5
| | | | | | | | | | | | | | | | | | | | When management encryption is enabled, GlusterD only allows encrypted connections for port 24007. SHD is trying to fetch it's volfile using an unencrypted connection. If /var/lib/glusterd/secure-access is present , i.e. if management ssl is enabled, use encrypted connection fecth info from glusterd. Change-Id: Ia41dcde1a69375379409889ec5b3a70cb95a4fd0 BUG: 1320388 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: http://review.gluster.org/13815 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* glfs-mgmt: fix connecting to multiple volfile transportsPrasanna Kumar Kalever2016-03-241-26/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: [ {"host":"1.2.3.4", "port":"24007", "transport":"tcp"}, {"host":"/var/run/glusterd.socket", "transport":"unix"} ] Consider the above case where we have two volfile servers, each has different transport type, we first try to connect to server1 which has transport type "tcp" let's say we found that host address is invalid, now we pick the next available server2, in our case it is of type "unix" and try to connect to it. but we fail to connect to it even we have right unix path, because multiple volfile servers with different transport type is not taken care currently. Solution: Every time we receive a RPC_CLNT_DISCONNECT event in mgmt_rpc_notify, we need to check the transport type of the volfile server in the list and set the keys "transport.socket.connect-path" or "remote-host" accordingly. Change-Id: I36bec46ef2e92e4642a7d7d64b423d0bc3ab269b BUG: 1320489 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/13819 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Tested-by: Prasanna Kumar Kalever <pkalever@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* dht2: Creation of experimental shell for dht2Shyam2016-03-2413-1/+242
| | | | | | | | | | Change-Id: I1aa3ea25f99d36fa2356edaa4c3132386adef303 Signed-off-by: Shyam <srangana@redhat.com> Reviewed-on: http://review.gluster.org/13397 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* debug/io-stats: Disable fop stats dump by defaultAnoop C S2016-03-241-1/+1
| | | | | | | | | | | | | | | | | Even though someone is not interested in seeing the fops statistics in client logs under DEBUG level we forcefully dump the same. This is due to default dump-interval configured to as 5 seconds. Since can use diagnostics.stats-dump-interval volume set option to change the default dump-interval its better to disable this huge dumping by default. Change-Id: Id14a38bcd92e47d75003279567a0f80acac1d86e BUG: 1320101 Signed-off-by: Anoop C S <anoopcs@redhat.com> Reviewed-on: http://review.gluster.org/13808 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* glusterd: remove unused large memory/buffer allocationsPrasanna Kumar Kalever2016-03-246-9/+0
| | | | | | | | | | | | | | This patch cleans dead buffer allocations which are never used in the code. Change-Id: I8c55837434fd5bf9dde9fb7b070fdefbf64cf4a3 BUG: 1253828 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/11928 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* dht: update attr information in refresh layout to avoidSakshi Bansal2016-03-231-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | stale timestamp Consider the scenario where an mkdir has just created the directory but has not healed it yet. A parallel lookup on this entry will find anomalies and trigger a selfheal which will sample the ctime of the directory after the mkdir phase. Meanwhile the mkdir has completed setting the layout and updated the ctime. The selfheal then sees the layout to be healed and returns with the ctime it got after the mkdir phase which has now become stale. However if the lookup happens to unwind before the mkdir then the inode associated with lookup will get linked in the inode table which has the stale ctime. To avoid this selfheal must do an iatt_merge in refresh layout to get the latest timestamp irrespective of whether it needs to heal the layout or not. Change-Id: I3634c3978bcc1710705f44b48f3876601682d33e BUG: 1302948 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13781 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* cli: Do not remove whitespace if key is user.* in volume setAtin Mukherjee2016-03-231-3/+6
| | | | | | | | | | | | | | setting any user.* option can have multiple words in the key and hence skip the process of trimming whitespaces. Change-Id: Ia295428d7cda55e5470fef668325bea497f2333d BUG: 1316499 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/13669 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaushal M <kaushal@redhat.com>
* gfapi: Fix the crashes caused by global_xlator and THISPoornima G2016-03-227-12/+218
| | | | | | | | | | | | | | | | | | | Issue: http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10922 The right fix for this is elaborate and intrusive, until it is in place, this patch provides a temperory fix. This fix is necessary, as without this libgfapi applications like qemu, samba, NFS ganesha are prone to crashes. This patch will be reverted completely, once the actual fix gets accepted. Change-Id: Ic975ab0bb03ba415cdf9bddba1534ba4d2d2820c BUG: 1319374 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/13784 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* extras:add gfid-to-dirname.shRavishankar N2016-03-221-0/+46
| | | | | | | | | | | | | | | | | | | Add a script that converts the gfid-string of a directory into absolute path name w.r.t the brick path. Example:gfid-to-dirname.sh /home/ravi/bricks/brick1 e9c81321-f009-41b0-a009-cd77888c217e Location of the directory corresponding to gfid:e9c81321-f009-41b0-a009-cd77888c217e is /home/ravi/bricks/brick1/d1/d2/d3/d4/d5/d6/d7/ Change-Id: I276ddf9c4f3d11a5b1932286d848b00b5ea1895d BUG: 1319706 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/13797 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: bipin kunal <kunalbipin@gmail.com> Tested-by: bipin kunal <kunalbipin@gmail.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* glusterd/ afr: Fix op-version for background client-side healsRavishankar N2016-03-222-3/+1
| | | | | | | | | | | | | | | | http://review.gluster.org/13207 tied cluster.heal-wait-queue-length to GD_OP_VERSION_3_7_9 but the patch will be merged in release-3.7 branch (http://review.gluster.org/#/c/13564/) only for 3.7.10. Hence change it on master also for uniformity. Change-Id: Id581695e58b0765f5652016cc2045f05e36b768f BUG: 1297172 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/13810 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-12 16:34:27 +0000