summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* tests/lock_revocation: mark as badRaghavendra G2017-05-101-0/+1
| | | | | | | | | | | | | The test is failing in master. see gluster-devel for more details. Change-Id: I7a589ad2c54bd55d62f4e66fdf8037c19fc123ea BUG: 1448364 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: https://review.gluster.org/17234 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* features/changelog: Add xattr as special type in changelog entryMohammed Rafi KC2017-05-102-11/+13
| | | | | | | | | | | | | | | | | When both SETATTR and SETXATTR fops are happening on gfid within the rollover time then, SETXATTR were not logged. In Which case we will miss the xattr fop in slave. This patch will be fix the same Change-Id: Ia75538ad1fd2797dbcf90d20dfa89f756009243d BUG: 1448914 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: https://review.gluster.org/17205 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* Fix 'flush' type, coverity warningMichael Scherer2017-05-101-1/+1
| | | | | | | | | | | | | | | | | | | As getchar returna a int, (which is 4 bytes on most common platforms, and at least 2 bytes), and char being often 1 byte, the conversion would lose information, hence the warning. Change-Id: I5b0b39c636ff49ef5c15d51a7f798e21635eca06 BUG: 789278 Signed-off-by: Michael Scherer <misc@redhat.com> Reviewed-on: https://review.gluster.org/17193 Tested-by: Michael Scherer <misc@fedoraproject.org> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* features/shard: Set size in inode ctx before size update for truncate tooKrutika Dhananjay2017-05-101-6/+8
| | | | | | | | | | | Change-Id: I7e984bb0f50c7d42764c0648e697d94d6c768dc7 BUG: 1448299 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/17184 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* nfs/nlm: log the caller_name if nlm_client_t can be foundNiels de Vos2017-05-101-2/+4
| | | | | | | | | | | | | | | | In order to help tracking possible misbehaving clients down, log the 'caller_name' (hostname of the NFS client) that does not have a matching nlm_client_t structure. Change-Id: Ib514a78d1809719a3d0274acc31ee632727d746d BUG: 1381970 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17186 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* nfs/nlm: ignore notify when there is no matching rpc requestNiels de Vos2017-05-101-1/+7
| | | | | | | | | | | | | | | | | In certain (unclear) occasions it seems to happen that there are notifications sent to the Gluster/NFS NLM service, but no call-state can be found. Instead of segfaulting, log an error but keep on running. Change-Id: I0f186e56e46a86ca40314d230c1cc7719c61f0b5 BUG: 1381970 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17185 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* build: add site.h as a place to put environment-specific definesJeff Darcy2017-05-103-2/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Most people consume Gluster in one of two ways: * From packages provided by their OS/distribution vendor * By building themselves from source For the first group it doesn't matter whether configuration is done in a configure script, via command-line options to that configure script, or in a header file. All of these end up as edits to some file under the packager's control, which is then run through their tools and process (e.g. rpmbuild) to create the packages that users will install. For the second group, convenience matters. Such users might not even have a script wrapped around the configure process, and editing one line in a header file is a lot easier than editing several in the configure script. This also prevents a messy profusion of configure options, dozens of which might need to be added to support a single such user's preferences. This comes back around as greater simplicity for packagers as well. This patch defines site.h as the header file for options and parameters that someone building the code for themselves might want to tweak. The project can ship one version to reflect the developers' guess at the best defaults for most users, and sophisticated users with unusual needs can override many options at once just by maintaining their own version of that file. Everybody wins. Further guidelines for how to determine whether an option should go in configure.ac or site.h are explained within site.h itself. Fixes #201 Change-Id: I5b8fb518d42450737423c4c1f43ebeb3130b4ff6 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17206 Tested-by: Jeff Darcy <jeff@pl.atyp.us> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* afr: fixes to quorum-type in afr_priv_dump()Ravishankar N2017-05-102-0/+49
| | | | | | | | | | | | | | | | | | Include the 'none' option as well in the output. This fixes the bug in commit 335555d256d444f4952ce239168f72b393370f01. Also added a test-case. This is a Signed-off-by: Ravishankar N <ravishankar@redhat.com> Change-Id: I479a14ae69ecae5a03e85e73ed50c19b483df603 BUG: 1448804 Reviewed-on: https://review.gluster.org/17215 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* mount/fuse: Handle racing notify on more than one graph properlyRaghavendra G2017-05-101-2/+6
| | | | | | | | | | | | | | | Make sure that we always use latest graph as a candidate for active-subvol. Change-Id: Ie37c818366f28ba6b1570d65a9eb17697d38a6c5 BUG: 1448364 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: https://review.gluster.org/17200 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* dht:Spacing issue in fix-layout o/pAnkitRaj2017-05-091-3/+3
| | | | | | | | | | | | | | | | There is a spacing issue in the status output of the rebalance fix-layout operations. If the local host name is big then we will have spacing issue. Change-Id: I2fcc4fd382723fb7e93cb4d4dad03dae682cc1a8 BUG: 1437748 Signed-off-by: AnkitRaj <anraj@redhat.com> Reviewed-on: https://review.gluster.org/17203 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: ankitraj NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* libglusterfs: include sys/time.h avoid compiling error on MacOSXKinglong Mee2017-05-091-0/+1
| | | | | | | | | | | | Change-Id: I3d30bacc3d5d085220dd85a3919207deef8bd1dd Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Reviewed-on: https://review.gluster.org/17114 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Tested-by: Prashanth Pai <ppai@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* libglusterfs/graph: fix potential endless loop in options validate workZhou Zhengping2017-05-091-1/+3
| | | | | | | | | | | | Change-Id: Icb71ded6051afe44e07480e0499d2a39f05fac71 BUG: 1447826 Signed-off-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-on: https://review.gluster.org/17171 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* performance/read-ahead: prevent stale data being returned to application.Raghavendra G2017-05-091-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Assume that fd is shared by two application threads/processes. T0 read is triggered from app-thread t1 and read call passes through write-behind. T1 app-thread t2 issues a write. The page on which read from t1 is waiting is marked stale T2 write-behind caches write and indicates to application as write complete. T3 app-thread t2 issues read to same region. Since, there is already a page for that region (created as part of read at T0), this read request waits on that page to be filled (though it is stale, which is a bug). T4 read (triggered at T0) completes from brick (with write still pending). Now both read requests from t1 and t2 are served this data (though data is stale from app-thread t2's perspective - which is a bug) T5 write is flushed to brick by write-behind. Fix is to not to serve data from a stale page, but instead initiate a fresh read to back-end. Change-Id: Id6af733464fa41bb4e81fd29c7451c73d06453fb BUG: 1414242 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: https://review.gluster.org/7447 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Csaba Henk <csaba@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* cluster/ec: fix incorrect answer check in seek fopXavier Hernandez2017-05-091-15/+8
| | | | | | | | | | | | | | | A bad check in the answer of a seek request caused a segmentation fault when seek reported an error. Change-Id: Ifb25ae8bf7cc4019d46171c431f7b09b376960e8 BUG: 1439068 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: https://review.gluster.org/16998 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* posix: Send SIGKILL in 2nd attemptAtin Mukherjee2017-05-091-2/+2
| | | | | | | | | | | | | | | Commit 21c7f7ba changed the signal from SIGKILL to SIGTERM for the 2nd attempt to terminate the brick process if SIGTERM fails. This patch fixes this problem. Change-Id: I856df607b7109a215f2a2a4827ba3ea42d8a9729 BUG: 1444596 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/17208 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* glusterd: socketfile & pidfile related fixes for brick multiplexing featureMohit Agrawal2017-05-0916-63/+307
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: While brick-muliplexing is on after restarting glusterd, CLI is not showing pid of all brick processes in all volumes. Solution: While brick-mux is on all local brick process communicated through one UNIX socket but as per current code (glusterd_brick_start) it is trying to communicate with separate UNIX socket for each volume which is populated based on brick-name and vol-name.Because of multiplexing design only one UNIX socket is opened so it is throwing poller error and not able to fetch correct status of brick process through cli process. To resolve the problem write a new function glusterd_set_socket_filepath_for_mux that will call by glusterd_brick_start to validate about the existence of socketpath. To avoid the continuous EPOLLERR erros in logs update socket_connect code. Test: To reproduce the issue followed below steps 1) Create two distributed volumes(dist1 and dist2) 2) Set cluster.brick-multiplex is on 3) kill glusterd 4) run command gluster v status After apply the patch it shows correct pid for all volumes BUG: 1444596 Change-Id: I5d10af69dea0d0ca19511f43870f34295a54a4d2 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: https://review.gluster.org/17101 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* cli: fix build error with --disable-xml-outputKinglong Mee2017-05-083-20/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ./configure --disable-xml-output --disable-georeplication make Making all in src CC cli.o In file included from cli.c:42:0: cli.h:440:24: error: unknown type name ‘xmlTextWriterPtr’ cli_xml_output_common (xmlTextWriterPtr writer, int op_ret, int op_errno, ^ cli.h:443:26: error: unknown type name ‘xmlTextWriterPtr’ cli_xml_snapshot_delete (xmlTextWriterPtr writer, xmlDocPtr doc, dict_t *dict, ^ cli.h:443:51: error: unknown type name ‘xmlDocPtr’ cli_xml_snapshot_delete (xmlTextWriterPtr writer, xmlDocPtr doc, dict_t *dict, ^ make[1]: *** [cli.o] Error 1 make: *** [all-recursive] Error 1 Change-Id: I36c2dfc11f89d774b62dfe6f50b156826bed5b66 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Reviewed-on: https://review.gluster.org/17136 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Tested-by: Prashanth Pai <ppai@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* libglusterfs: stop special casing "cache-size" in size_t validationCsaba Henk2017-05-083-18/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The original situation was as follows: The function that validates xlator options indicating a size, xlator_option_validate_sizet(), handles the case when the name of the option is "cache-size" in a special way. - Xlator options (things of type volume_option_t) has a min and max attribute of type double. - An xlator option is endowed with a gluster specific type (not C type). An instance of an xlator option goes through a validation process by a type specific validator function (which are collected in option.c). - Validators of numeric types - size being one of them - make use the min and max attributes to perform a range check, except in one case: if an option is defined with min = max = 0, then this option will be exempt of range checking. (Note: the volume_option_t definition features the following comments along the min, max fields: double min; /* 0 means no range */ double max; /* 0 means no range */ which is slightly misleading as it lets one to conclude that zeroing min or max buys exemption from low or high boundary check, which is not true -- only *both* being zero buys exemption.) - Besides this, the validator for options of size type, xlator_option_validate_sizet() special cases options named "cache-size" so that only min is enforced. (The only consequence of a value exceeding max is that glusterd logs a warning about it, but the cli user who makes such a setting gets no feedback on it.) - This was introduced because a hard coded limit is not useful for io-cache and quick-read. They rather use a runtime calculated upper limit. (See changes I7dd4d8c53051b89a293696abf1ee8dc237e39a20 I9c744b5ace10604d5a814e6218ca0d83c796db80 about the last two points.) - As an unintended consequence, the upper limit check of cache-size of write-behind, for which a conventional hard coded limit is specified, is defeated. What we do about it: - Remove the special casing clause for cache-size in xlator_option_validate_sizet. Thus the general range check policy (as described above) will apply to cache-size too. - To implement a lower bound only check by the validator for cache-size of io-cache and quick-read, change the max attribute of these options to INFINITY. The only behavioral difference is the omission of the warnings about cache-size of io-cache and quick-read exceeding the former max values. (They were rather heuristic anyway.) BUG: 1445609 Change-Id: I0bd8bd391fa7d926f76e214a2178833fe4673b4a Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: https://review.gluster.org/17125 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* nfs/nlm: unref rpc-client after nlm4svc_send_granted()Niels de Vos2017-05-081-1/+1
| | | | | | | | | | | | | | | | nlm4svc_send_granted() uses the rpc_clnt by getting it from the call-state structure. It is safer to unref the rpc_clnt after the function is done with it. Change-Id: I7cb7c4297801463d21259c58b50d7df7c57aec5e BUG: 1381970 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17187 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* debug/error-gen: Don't need to convert error string to int in every fopZhou Zhengping2017-05-082-5/+12
| | | | | | | | | | | | | | | | Every fop in xlator error-gen will call function error_gen, which will call function conv_error to convert pvt->error_no to int. But actually the function could only be called in init and reconfigure Change-Id: I96c9780574f369fc58eed10fea9d50c4cd7d4e8a BUG: 1446412 Signed-off-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-on: https://review.gluster.org/17132 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* Remove uneeded gotos, as it jump to the next lineMichael Scherer2017-05-081-70/+0
| | | | | | | | | | | | | | | Found by coverity Signed-off-by: Michael Scherer <misc@redhat.com> Change-Id: I1fef09a427d18bbeb9af0bf4f5c82c77aa3c0d3d BUG: 789278 Reviewed-on: https://review.gluster.org/17195 Tested-by: Michael Scherer <misc@fedoraproject.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Smoke: Gluster Build System <jenkins@build.gluster.org>
* glusterd: cleanup pidfile on pmap signoutAtin Mukherjee2017-05-084-6/+109
| | | | | | | | | | | | | | | | This patch ensures 1. brick pidfile is cleaned up on pmap signout 2. pmap signout evemt is sent for all the bricks when a brick process shuts down. Change-Id: I7606a60775b484651d4b9743b6037b40323931a2 BUG: 1444596 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/17168 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* afr: include quorum type and count when dumping afr privRavishankar N2017-05-081-0/+6
| | | | | | | | | | | | | | | | Dump the client quorum type ('auto' or 'fixed'). If it is 'fixed', also dump the quorum-count. This information will be available in the client statedump and in /<fuse_mount>/.meta/graphs/active/testvol-replicate-X/private. Change-Id: Idbd6e2acbd622d4e6cfabf511e649a6da0e42384 BUG: 1448804 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: https://review.gluster.org/17196 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* Fixes quota aux mount failureSanoj Unnikrishnan2017-05-0821-83/+151
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The aux mount is created on the first limit/remove_limit/list command and it remains until volume is stopped / deleted / (quota is disabled) , where we do a lazy unmount. If the process is uncleanly terminated, then the mount entry remains and we get (Transport disconnected) error on subsequent attempts to run quota list/limit-usage/remove commands. Second issue, There is also a risk of inadvertent rm -rf on the /var/run/gluster causing data loss for the user. Ideally, /var/run is a temp path for application use and should not cause any data loss to persistent storage. Solution: 1) unmount the aux mount after each use. 2) clean stale mount before mounting, if any. One caveat with doing mount/unmount on each command is that we cannot use same mount point for both list and limit commands. The reason for this is that list command needs mount to be accessible in cli after response from glusterd, So it could be unmounted by a limit command if executed in parallel (had we used same mount point) Hence we use separate mount points for list and limit commands. Change-Id: I4f9e39da2ac2b65941399bffb6440db8a6ba59d0 BUG: 1433906 Signed-off-by: Sanoj Unnikrishnan <sunnikri@redhat.com> Reviewed-on: https://review.gluster.org/16938 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Manikandan Selvaganesh <manikandancs333@gmail.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* snapview-server : Refresh the snapshot list during each reconnectMohammed Rafi KC2017-05-082-0/+33
| | | | | | | | | | | | | | | | | | | | Currently we are refreshing the snapshot list either when there is a request from glusterd or the very first initialization. But if anything changed after when glusterd is down then there is no mechanism to refresh the snashot dentries. This patch will refresh snapshot list during each reconnect Change-Id: I3ed655572d777f60d57dd479d190f75553591267 BUG: 1448150 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: https://review.gluster.org/17178 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
* cluster/dht: Fix ret checkN Balachandran2017-05-081-1/+1
| | | | | | | | | | | | | | Fixed an incorrect return code check in the rebalance code. Change-Id: I60804ff121cec7a2f0419e2ee70dd22ea7533c0c BUG: 1448640 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/17197 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* rpc: use GF_ATOMIC_INC to generate rpc_clnt's callidZhou Zhengping2017-05-082-17/+3
| | | | | | | | | | | | Change-Id: I57ad970411db1ccd3d2c56c504c7da9cc221051f BUG: 1448692 Signed-off-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-on: https://review.gluster.org/17198 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* Tier: Watermark check for hi and low value being equalhari gowtham2017-05-082-2/+4
| | | | | | | | | | | | | | | | | | | Problem: Both low and hi watermark can be set to same value as the check missed the case for being equal. Fix: Add the check to both the hi and low values being equal along with the low value being higher than hi value. Change-Id: Ia235163aeefdcb2a059e2e58a5cfd8fb7f1a4c64 BUG: 1447960 Signed-off-by: hari gowtham <hgowtham@redhat.com> Reviewed-on: https://review.gluster.org/17175 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: hari gowtham <hari.gowtham005@gmail.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Milind Changire <mchangir@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* Fix ressources leak on error pathMichael Scherer2017-05-051-1/+1
| | | | | | | | | | | | | | | | | | | | | Coverty found a path where blk_fop is not freed, thus leaking memory. The main cause is because we are using the wrong variable for errors. I do not think this is leaking anything in practice, since the error condition seems quite rare, but fixing this should remove around 15 coverty warning. Change-Id: Ida0ea59f501f0a21bd1d8119aa1a299ce655add8 BUG: 789278 Signed-off-by: Michael Scherer <misc@redhat.com> Reviewed-on: https://review.gluster.org/16715 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Tested-by: Michael Scherer <misc@fedoraproject.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* coreutils: use coreutils instead of duplicate codeZhou Zhengping2017-05-056-33/+16
| | | | | | | | | | | | | Change-Id: I0e442331d2bbb22ec18c37af87ab2a8852737c43 BUG: 1448265 Signed-off-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-on: https://review.gluster.org/16975 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* gfapi: Fix few severe coverity issuesSoumya Koduri2017-05-053-2/+9
| | | | | | | | | | | | | | | From https://download.gluster.org/pub/gluster/glusterfs/static-analysis/master/glusterfs-coverity/2017-05-03-6484558c/html/ fixes coverity#189, 248, 250, 488, 546 Change-Id: I7c4aa5bf6c367344602748dfd736de27f341cca2 BUG: 789278 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: https://review.gluster.org/17161 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* rpc: Remove accidental IPV6 changesKaushal M2017-05-0512-189/+1
| | | | | | | | | | | | | | They snuck in with the HALO patch (07cc8679c) Change-Id: I8ced6cbb0b49554fc9d348c453d4d5da00f981f6 BUG: 1447953 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: https://review.gluster.org/17174 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* build: 'make cscope' requires generated .c and .h XDR filesNiels de Vos2017-05-051-0/+4
| | | | | | | | | | | | | | | | | When running 'make cscope' on a clean configured tree (before building anything), it fails because of missing files unders rpc/xdr/src/. These files are symlinked from generated files in the rpc/xdr/gen/ directory. In order to create the symlinks for 'make cscope', a target needs to specify how these symlinks are created. Change-Id: I473c90e10d915ee438425cf0f806c0531b9f582a BUG: 1447966 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17176 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* glusterd: disallow rebalance & remove-brick on a sharded volumeAtin Mukherjee2017-05-042-0/+19
| | | | | | | | | | | | | Change-Id: Idfbdbc61ca18054fdbf7556f74e195a63cd8a554 BUG: 1447630 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/17160 Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* Remove double freeMichael Scherer2017-05-041-1/+0
| | | | | | | | | | | | | | | Since there is already GF_FREE (value) near the out label, this one is uneeded. Found by coverity. Change-Id: Ib4a4e1a6af186e14fc75e7578c79beb58de053ec BUG: 789278 Signed-off-by: Michael Scherer <misc@redhat.com> Reviewed-on: https://review.gluster.org/17165 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: Michael Scherer <misc@fedoraproject.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* fd: We should use dict_set_uint64 to store fd->pidZhou Zhengping2017-05-041-1/+1
| | | | | | | | | | | | Change-Id: I136372b9929d3ecf243649b6945571a67bfd80eb BUG: 1447828 Signed-off-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-on: https://review.gluster.org/17172 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Amar Tumballi <amarts@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* Refactor the code for posix_handle_pairMichael Scherer2017-05-041-20/+8
| | | | | | | | | | | | | | | | | | It was no longer compiling on Darwin, and triggered a cppcheck error: [xlators/storage/posix/src/posix-helpers.c:1097]: (error) Invalid number of character '{' when these macros are defined: 'GF_DARWIN_HOST_OS'. Change-Id: I986609cc6cfc13ddea0ed35d7776736171f40a41 BUG: 789278 Signed-off-by: Michael Scherer <misc@redhat.com> Reviewed-on: https://review.gluster.org/16729 Tested-by: Michael Scherer <misc@fedoraproject.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* mgmt: Fix null pointer derefMichael Scherer2017-05-041-1/+0
| | | | | | | | | | | | | | | | since "this" can be NULL, we should skip this validation, there is another call to GF_VALIDATE_OR_GOTO in the following lines. Found by cppcheck. Change-Id: I329f50b986a9eaf3315e09f851080ab41bea57c0 BUG: 789278 Signed-off-by: Michael Scherer <misc@redhat.com> Reviewed-on: https://review.gluster.org/16742 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Tested-by: Jeff Darcy <jeff@pl.atyp.us> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* Fix wrong operatorsMichael Scherer2017-05-042-4/+3
| | | | | | | | | | | | | | | | | | | | | | Coverty rightfully note that if we verify that A =! C or A != B, it will always be true. In one case, that prevent healing from continuing. In the other, that trigger useless logs. Fixing this bug also show that ENOSPC shouldn't abort the rebalance operation, as seen during the review of the first patch on https://review.gluster.org/#/c/16676/1/xlators/cluster/dht/src/dht-rebalance.c Change-Id: I93c4df43b880b211da202a7e49cef6b1ce7ab68f BUG: 1424817 Signed-off-by: Michael Scherer <misc@redhat.com> Reviewed-on: https://review.gluster.org/16676 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Tested-by: Jeff Darcy <jeff@pl.atyp.us> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* server: fd should be refed before put into fdtableZhou Zhengping2017-05-031-3/+3
| | | | | | | | | | | | | | | | | If a network error occurs after we put a fd to fdtable, fds in fdtable will be cleaned up by do_fd_cleanup which will unref the fd and release the fd if the fd hasn't be refed by user. Change-Id: Idbad0fafd005c59398518190845fe137cbf95c60 BUG: 1447609 Signed-off-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-on: https://review.gluster.org/17158 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Milind Changire <mchangir@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* rpc: fix transport add/remove race on port probingMilind Changire2017-05-031-165/+196
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Spurious __gf_free() assertion failures seen all over the place with header->magic being overwritten when running port probing tests with 'nmap' Solution: Fix sequence of: 1. add accept()ed socket connection fd to epoll set 2. add newly created rpc_transport_t object in RPCSVC service list Correct sequence is #2 followed by #1. Reason: Adding new fd returned by accept() to epoll set causes an epoll_wait() to return immediately with a POLLIN event. This races ahead to a readv() which returms with errno:104 (Connection reset by peer) during port probing using 'nmap'. The error is then handled by POLLERR code to remove the new transport object from RPCSVC service list and later unref and destroy the rpc transport object. socket_server_event_handler() then catches up with registering the unref'd/destroyed rpc transport object. This is later manifest as assertion failures in __gf_free() with the header->magic field botched due to invalid address references. All this does not result in a Segmentation Fault since the address space continues to be mapped into the process and pages still being referenced elsewhere. As a further note: This race happens only in accept() codepath. Only in this codepath, the notify will be referring to two transports: 1, listener transport and 2. newly accepted transport All other notify refer to only one transport i.e., the transport/socket on which the event is received. Since epoll is ONE_SHOT another event won't arrive on the same socket till the current event is processed. However, in the accept() codepath, the current event - ACCEPT - and the new event - POLLIN/POLLER - arrive on two different sockets: 1. ACCEPT on listener socket and 2. POLLIN/POLLERR on newly registered socket. Also, note that these two events are handled different thread contexts. Cleanup: Critical section in socket_server_event_handler() has been removed. Instead, an additional ref on new_trans has been used to avoid ref/unref race when notifying RPCSVC. Change-Id: I4417924bc9e6277d24bd1a1c5bcb7445bcb226a3 BUG: 1438966 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: https://review.gluster.org/17139 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Oleksandr Natalenko <oleksandr@natalenko.name> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* SELinux : implementation of SELinux translatorManikandan Selvaganesh2017-05-0313-7/+536
| | | | | | | | | | | | | | | | | | | | The patch implement a part of SELinux translator to support setting SELinux contexts on files in a glusterfs volume. URL: https://github.com/gluster/glusterfs-specs/blob/master/accepted/SELinux-client-support.md Change-Id: Id8916bd8e064ccf74ba86225ead95f86dc5a1a25 BUG: 1318100 Fixes : #55 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/13762 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Manikandan Selvaganesh <manikandancs333@gmail.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* gfapi/handleops: Introducing glfs_xreaddirplus_r() fop for handleopsSoumya Koduri2017-05-0212-4/+437
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Its known that readdirplus operation fetches stat as well for each of the dirents. But often applications may need extra information, like for eg., NFS-Ganesha which operates on handles needs handles for each of those dirents returned. So this would require extra calls to the backend, in this case LOOKUP (which is very expensive operation) resulting in very low readdir performance. To address that introducing this new API using which applications can make request for any extra information to be returned as part of readdirplus response. Currently this new api returns stat and handles as demanded by application. The synopsis of the API is noted in glfs.h. @todo: * Enhance test script using this new API Below were the perf results on single brick volume with and without these changes - Dataset used - 10*100 directories and each directory containing 100 empty files. I used NFS-Ganesha application to test these changes - >for i in {1..5}; do systemctl restart nfs-ganesha; sleep 10; mount -t nfs -o vers=4 localhost:/brick_vol /mnt; cd /mnt; echo "ITERATION$i"; date; find . > tmp-nfs.log; date; cd /; umount /mnt; sleep 2; done; Without these changes - ITERATION1 Mon Mar 20 17:22:26 IST 2017 Mon Mar 20 17:23:18 IST 2017 ITERATION2 Mon Mar 20 17:23:39 IST 2017 Mon Mar 20 17:24:28 IST 2017 ITERATION3 Mon Mar 20 17:24:49 IST 2017 Mon Mar 20 17:25:36 IST 2017 ITERATION4 Mon Mar 20 17:30:57 IST 2017 Mon Mar 20 17:31:37 IST 2017 ITERATION5 Mon Mar 20 17:31:57 IST 2017 Mon Mar 20 17:32:40 IST 2017 [root@dhcp35-197 /]# On an average ~46.2 sec With these changes applied - ITERATION1 Mon Mar 20 17:35:03 IST 2017 Mon Mar 20 17:35:15 IST 2017 ITERATION2 Mon Mar 20 17:35:36 IST 2017 Mon Mar 20 17:35:46 IST 2017 ITERATION3 Mon Mar 20 17:36:06 IST 2017 Mon Mar 20 17:36:17 IST 2017 ITERATION4 Mon Mar 20 17:41:38 IST 2017 Mon Mar 20 17:41:49 IST 2017 ITERATION5 Mon Mar 20 17:42:10 IST 2017 Mon Mar 20 17:42:20 IST 2017 On an average ~10.8 sec Updates #174 BUG: 1442950 Change-Id: I0f74f74dc62085ca4c4a23c38e3edc84bd850876 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: https://review.gluster.org/15663 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* glusterd: skip nfs svc reconfigure if nfs xlator is not installedAtin Mukherjee2017-05-021-0/+9
| | | | | | | | | | | | | | | | With 83abcba, nfs svc is not (re)started or stopped if NFS so file is not installed. However the same check was missing in nfs svc reconfigure which was causing all volume set command to fail. Change-Id: Ie87b5dba44ac59e890cbd60f85944f8e685ad52b BUG: 1326219 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/17149 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* nl-cache: free nlc_conf_t in fini()Niels de Vos2017-05-021-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The (xlator_t*)->private structure in negative-lookup-cache is allocated in the init() function of the xlator, but never free'd. Valgrind detected this as: 656 bytes in 1 blocks are definitely lost in loss record X of Y at 0x..+ calloc (/builddir/build/BUILD/valgrind-3.11.0/coregrind/m_replacemalloc/vg_replace_malloc.c:711) by 0x.. __gf_calloc (/usr/src/debug/glusterfs-3.11dev/libglusterfs/src/mem-pool.c:117) by 0x.. init (/usr/src/debug/glusterfs-3.11dev/xlators/performance/nl-cache/src/nl-cache.c:669) by 0x.. __xlator_init (/usr/src/debug/glusterfs-3.11dev/libglusterfs/src/xlator.c:472) by 0x.. xlator_init (/usr/src/debug/glusterfs-3.11dev/libglusterfs/src/xlator.c:498) by 0x.. glusterfs_graph_init (/usr/src/debug/glusterfs-3.11dev/libglusterfs/src/graph.c:321) by 0x.. glusterfs_graph_activate (/usr/src/debug/glusterfs-3.11dev/libglusterfs/src/graph.c:693) by 0x.. glfs_process_volfp (/usr/src/debug/glusterfs-3.11dev/api/src/glfs-mgmt.c:79) by 0x.. glfs_volumes_init (/usr/src/debug/glusterfs-3.11dev/api/src/glfs.c:160) by 0x.. glfs_init_common (/usr/src/debug/glusterfs-3.11dev/api/src/glfs.c:868) by 0x.. glfs_init@@GFAPI_3.4.0 (/usr/src/debug/glusterfs-3.11dev/api/src/glfs.c:913) by 0x.. main (/root/gluster-debug/gfapi-load-volfile/gfapi-load-volfile.c:54) When the xlators is unloaded, it should free the resources it allocated. This can easily be done in the fini() function. Change-Id: I079e78cc207145bc542e2282fc4cf2bb4dadc28a BUG: 1442569 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17143 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* api: fix typo of glfs_upcall_get_eventKinglong Mee2017-05-022-2/+2
| | | | | | | | | | | | | glfs_upcall_inode_get_event is not exist. Change-Id: I27bd7eb9bf99d3289ae12fe31405d3b46855152d BUG: 1447116 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Reviewed-on: https://review.gluster.org/17135 Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* Halo Replication feature for AFR translatorKevin Vigor2017-05-0238-142/+908
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Halo Geo-replication is a feature which allows Gluster or NFS clients to write locally to their region (as defined by a latency "halo" or threshold if you like), and have their writes asynchronously propagate from their origin to the rest of the cluster. Clients can also write synchronously to the cluster simply by specifying a halo-latency which is very large (e.g. 10seconds) which will include all bricks. In other words, it allows clients to decide at mount time if they desire synchronous or asynchronous IO into a cluster and the cluster can support both of these modes to any number of clients simultaneously. There are a few new volume options due to this feature: halo-shd-latency: The threshold below which self-heal daemons will consider children (bricks) connected. halo-nfsd-latency: The threshold below which NFS daemons will consider children (bricks) connected. halo-latency: The threshold below which all other clients will consider children (bricks) connected. halo-min-replicas: The minimum number of replicas which are to be enforced regardless of latency specified in the above 3 options. If the number of children falls below this threshold the next best (chosen by latency) shall be swapped in. New FUSE mount options: halo-latency & halo-min-replicas: As descripted above. This feature combined with multi-threaded SHD support (D1271745) results in some pretty cool geo-replication possibilities. Operational Notes: - Global consistency is gaurenteed for synchronous clients, this is provided by the existing entry-locking mechanism. - Asynchronous clients on the other hand and merely consistent to their region. Writes & deletes will be protected via entry-locks as usual preventing concurrent writes into files which are undergoing replication. Read operations on the other hand should never block. - Writes are allowed from _any_ region and propagated from the origin to all other regions. The take away from this is care should be taken to ensure multiple writers do not write the same files resulting in a gfid split-brain which will require resolution via split-brain policies (majority, mtime & size). Recommended method for preventing this is using the nfs-auth feature to define which region for each share has RW permissions, tiers not in the origin region should have RO perms. TODO: - Synchronous clients (including the SHD) should choose clients from their own region as preferred sources for reads. Most of the plumbing is in place for this via the child_latency array. - Better GFID split brain handling & better dent type split brain handling (i.e. create a trash can and move the offending files into it). - Tagging in addition to latency as a means of defining which children you wish to synchronously write to Test Plan: - The usual suspects, clang, gcc w/ address sanitizer & valgrind - Prove tests Reviewers: jackl, dph, cjh, meyering Reviewed By: meyering Subscribers: ethanr Differential Revision: https://phabricator.fb.com/D1272053 Tasks: 4117827 Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1 BUG: 1428061 Signed-off-by: Kevin Vigor <kvigor@fb.com> Reviewed-on: http://review.gluster.org/16099 Reviewed-on: https://review.gluster.org/16177 Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* Revert "contrib/timerwheel: probable bug on 32-bit, use __builtin_ffs()"Shyamsundar Ranganathan2017-05-011-12/+5
| | | | | | | | | | | | | | This reverts commit c92b8347aea8ce78ca3fbc49b88f5adadc98509b. Commit is not ready for a merge! Change-Id: I3b3b52f7bfb4781dd42160e2b1059b4cdeb17956 Reviewed-on: https://review.gluster.org/17147 Tested-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* contrib/timerwheel: probable bug on 32-bit, use __builtin_ffs()Kaleb S. KEITHLEY2017-05-011-5/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | Simply always defining BITS_PER_LONG as 64 seems like it's almost certainly wrong on 32-bit platforms and could potentially result in incorrect results. fls and, e.g., __builtin_ffs() return the same answer for any given input, making it seem like the name fls (find last set) is a misnomer and ffs (find first set, starting from the lsb) is the more accurate name. Using __builtin_ffs() causes the compiler (in intel) to emit code with the bsf (bit scan forward) insn, which is approx 3x faster than the code in ffs(), at least on the machine I tried it on. (Even so, it takes 10M+ iterations for the speed difference to be measurable. Choosing the "faster" implementation seems like a no-brainer, even if there may not be any significant gain by doing so.) Change-Id: I1616dda1a5b76f208ba737a713877c1673131e33 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: https://review.gluster.org/17142 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* core: make the per glusterfs_ctx_t timer-wheel refcountedNiels de Vos2017-05-0112-87/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | xlators can use a 'global' timer-wheel for scheduling events. This timer-wheel is managed per glusterfs_ctx_t, but does not need to be allocated for every graph. When an xlator wants to use the timer-wheel, it will be instanciated on demand, and provided to xlators that request it later on. By adding a reference counter to the glusterfs_ctx_t for the timer-wheel, the threads and structures can be cleaned up when the last xlator does not have a need for it anymore. In general, the xlators request the timer-wheel in init(), and they should return it in fini(). Because the timer-wheel is managed per glusterfs_ctx_t, the functions can be added to ctx.c and do not need to live in their very minimal tw.[ch] files. Change-Id: I19d225b39aaa272d9005ba7adc3104c3764f1572 BUG: 1442788 Reported-by: Poornima G <pgurusid@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17068 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-20 07:32:54 +0000