summaryrefslogtreecommitdiffstats
path: root/xlators
Commit message (Collapse)AuthorAgeFilesLines
* glusterd: unref brickinfo object on volume stopAtin Mukherjee2017-02-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | If brick multiplexing is enabled, on a volume stop glusterd was not unrefing the brickinfo rpc object which lead to a flood of stale rpc logs. >Reviewed-on: https://review.gluster.org/16699 >Smoke: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Jeff Darcy <jdarcy@redhat.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Samikshan Bairagya <samikshan@gmail.com> >(cherry picked from commit 9cdfbdced23cd43b8738636a3ed906c8d4267d67) Change-Id: I18fedcd6921042ef2e945605466194b7b53fe2f7 BUG: 1425556 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/16703 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* cluster/ec: Fallback to precompiled codeXavier Hernandez2017-02-213-70/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | When dynamic code generation fails for some reason, instead of causing a failure in encode/decode, fallback to the precompiled version. >Change-Id: I4f8a97d3033aa5885779722b19c6e611caa4ffea >BUG: 1421955 >Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> >Reviewed-on: https://review.gluster.org/16614 >Smoke: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Niels de Vos <ndevos@redhat.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> >(cherry picked from commit 9f9d1482868e8e1044790c8358893f4421d89692) Change-Id: I5e6221fce7a77782fd7d961e752a61dd59821a98 BUG: 1421956 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: https://review.gluster.org/16697 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* cluster/dht Fix error assignment in dht_*xattr2 functionsN Balachandran2017-02-201-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Corrected the op_errno assignments and NULL checks in the dht_sexattr2 and dht_removexattr2 functions. Earlier, they unwound with the default EINVAL op_errno if the file had been deleted. > Change-Id: Iaf837a473d769cea40132487a966c7f452990071 > BUG: 1421653 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: https://review.gluster.org/16610 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: MOHIT AGRAWAL <moagrawa@redhat.com> > Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> (cherry picked from commit 028626a86ea409f908783b9007c02877f20be43e) Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: Icc53ca52281a533fc0c8eeafb4c889c818c313e1 BUG: 1424921 Reviewed-on: https://review.gluster.org/16679 Tested-by: N Balachandran <nbalacha@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* posix: Fix creation of files with S_ISVTX on FreeBSDXavier Hernandez2017-02-203-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | On FreeBSD the S_ISVTX flag is completely ignored when creating a regular file. Since gluster needs to create files with this flag set, specialy for DHT link files, it's necessary to force the flag. This fix does this by calling fchmod() after creating a file that must have this flag set. > Change-Id: I51eecfe4642974df6106b9084a0b144835a4997a > BUG: 1411228 > Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> > Reviewed-on: https://review.gluster.org/16417 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Change-Id: I2087516383bd132c59bbab98eda8f2243a2163fe BUG: 1424973 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: https://review.gluster.org/16686 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* marker: Fix inode value in loc, in setxattr fopPoornima G2017-02-202-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On recieving a rename fop, marker_rename() stores the, oldloc and newloc in its 'local' struct, once the rename is done, the xtime marker(last updated time) is set on the file, but sending a setxattr fop. When upcall receives the setxattr fop, the loc->inode is NULL and it crashes. The loc->inode can be NULL only in one valid case, i.e. in rename case where the inode of new loc can be NULL. Hence, marker should have filled the inode of the new_loc before issuing a setxattr. marker_rename_cbk was already fixed in a previous commit. Fixing marker_rename_done to send valid inode in this commit. Also in upcall check for NULL inode so that there is no crash. >Reviewed-on: https://review.gluster.org/16633 >Reviewed-by: Kotresh HR <khiremat@redhat.com> >Smoke: Gluster Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: soumya k <skoduri@redhat.com> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Raghavendra G <rgowdapp@redhat.com> >(cherry picked from commit 73defab8be16b73241225bb1c2588a61e3e425d5) Change-Id: I3ed2a05118fed3367dfe3251ce4477310cb480d0 BUG: 1424937 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: https://review.gluster.org/16684 Tested-by: Atin Mukherjee <amukherj@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd, readdir-ahead: Fix backward incompatibilityPoornima G2017-02-192-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of https://review.gluster.org/#/c/16657/ Issue: Any opion is spcified in two places: In the options[] of xlator itself and glusterd-volume-set.c. The default value of this option can be specified in both the places. If its specified only in xlator then the volfile generated will not have the option and default value, it will be assigned during graph initialization. With patch [1] the option rda-request-size was changed from INT to SIZET type, and default was changed from 131072 to 128KB, but was specified only in the readdir-ahead.c. Thus with this patch alone the volfile entry for readdir-ahead looks like: volume patchy-readdir-ahead type performance/readdir-ahead subvolumes patchy-read-ahead end-volume With patch [2], the default of option rda-request-size was specified in glusterd-volume-set.c as well(as it was necessary fr parallel readdir). With this patch the readdir entry in the volfile will look like: volume patchy-readdir-ahead type performance/readdir-ahead option rda-cache-limit 10MB option rda-request-size 128KB option parallel-readdir off subvolumes patchy-read-ahead end-volume Now consider the server has both these patches and client doesn't. Server will generate a volfile with entry: The old clients which thought the option rda-request-size is of type INT will now recieve the value 128KB which it willn't understand, and hence fail the mount. The issue is seen only with the combination of [1] and [2]. Solution: Instead of specifying 128KB as default in glusterd we specify 131072 so that the old clients will interpret as INT and new ones as 128KB Credits: Raghavendra G > Reviewed-on: https://review.gluster.org/16657 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Change-Id: I0c269a5890957fd8a38e9a05bdec088645a6688a BUG: 1423412 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: https://review.gluster.org/16658 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* readdir-ahead: Remove unnecessary loggingPoornima G2017-02-171-6/+0
| | | | | | | | | | | | | | | | | | | | dict_get_int can return < 0 when key is not found is a valid case. Hence no need to log. > Reviewed-on: https://review.gluster.org/16654 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Change-Id: If0795b0f178adbb94b10efc563506993f7411962 BUG: 1423429 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: https://review.gluster.org/16659 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd : Fix for error mesage while detaching peersGaurav Yadav2017-02-171-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When peer is detached from a cluster, an error log is being generated in glusterd.log -"Failed to reconfigure all daemon services". This log is seen in the originator node where the detach is issued. This happens in two cases. Case 1: Detach peer with no volume been created in cluster. Case 2: Detach peer after deleting all the volumes which were created but never started. In any one of the above two cases, in glusterd_check_files_identical() GlusterD fails to retrieve nfs-server.vol file from /var/lib/glusterd/nfs which gets created only when a volume is in place and and is started. With this fix both the above cases have been handled by added validation to skip reconfigure if there is no volume in started state. > Reviewed-on: https://review.gluster.org/16607 > Smoke: Gluster Build System <jenkins@build.gluster.org> > Tested-by: Atin Mukherjee <amukherj@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit be44a1bd519af69b21acf682b0908d4d695f868e) Change-Id: I039c0840e3d61ab54575e1e00c6a6a00874d84c0 BUG: 1423406 Signed-off-by: Gaurav Yadav <gyadav@redhat.com> Reviewed-on: https://review.gluster.org/16645 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Samikshan Bairagya <samikshan@gmail.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* performance/decompounder: Have terminal value in options[]Pranith Kumar K2017-02-171-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | Absence of terminal values is leading to buffer-over-flow errors in address sanitizer. >BUG: 1422152 >Change-Id: I769c0e4b5bbb3ef2849b8d1097b9def522ae08d9 >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> >Reviewed-on: https://review.gluster.org/16615 >Smoke: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Jeff Darcy <jdarcy@redhat.com> >(cherry picked from commit fca31ae5faa15a02e64143b9efb7e19c3b907c2d) Change-Id: Ifec6f5b96cfe4c26a039f88649284bcdad5321f6 BUG: 1423070 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/16652 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* cluster/afr: Perform new entry mark before creating new entryPranith Kumar K2017-02-164-54/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a chance for the source brick to go down just after the new entry is created and before source brick is marked with necessary pending markers. If after this any I/O happens then new entry will become source and reverse heal will happen. To prevent this mark the pending xattrs before creating the new entry. >BUG: 1417466 >Change-Id: I233b87e694d32e5d734df5a83b4d2ca711c17503 >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> >Reviewed-on: https://review.gluster.org/16474 >Smoke: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Ravishankar N <ravishankar@redhat.com> >Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> BUG: 1422942 Change-Id: I460caaa2c6571c4256e50421e48857e3c2769b4d Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/16646 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* md-cache: initialize mdc_counter.lockNiels de Vos2017-02-161-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | add missed LOCK_INIT to fix INCREMENT_ATOMIC on conf->mdc_counter.lock when pthread_spin_* using Cherry picked from commit 22f02d8f1dcdf176744ab1536cb23a5fcd291243: > Change-Id: I680bd6f41e3b8a1852ed969bf6794cbf4c1ccdd4 > BUG: 1417913 > Signed-off-by: Vitaly Lipatov <lav@etersoft.ru> > Reviewed-on: https://review.gluster.org/16515 > Reviewed-by: Niels de Vos <ndevos@redhat.com> > Tested-by: Niels de Vos <ndevos@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Poornima G <pgurusid@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: I680bd6f41e3b8a1852ed969bf6794cbf4c1ccdd4 BUG: 1417915 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/16640 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Poornima G <pgurusid@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* rpcsvc: Add rpchdr and proghdr to iobref before submitting to transportPoornima G2017-02-162-9/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of https://review.gluster.org/16613 Issue: When fio is run on multiple clients (each client writes to its own files), and meanwhile the clients does a readdirp, thus the client which did a readdirp will now recieve the upcalls. In this scenario the client disconnects with rpc decode failed error. RCA: Upcall calls rpcsvc_request_submit to submit the request to socket: rpcsvc_request_submit currently: rpcsvc_request_submit () { iobuf = iobuf_new iov = iobuf->ptr fill iobuf to contain xdrised upcall content - proghdr rpcsvc_callback_submit (..iov..) ... if (iobuf) iobuf_unref (iobuf) } rpcsvc_callback_submit (... iov...) { ... iobuf = iobuf_new iov1 = iobuf->ptr fill iobuf to contain xdrised rpc header - rpchdr msg.rpchdr = iov1 msg.proghdr = iov ... rpc_transport_submit_request (msg) ... if (iobuf) iobuf_unref (iobuf) } rpcsvc_callback_submit assumes that once rpc_transport_submit_request() returns the msg is written on to socket and thus the buffers(rpchdr, proghdr) can be freed, which is not the case. In especially high workload, rpc_transport_submit_request() may not be able to write to socket immediately and hence adds it to its own queue and returns as successful. Thus, we have use after free, for rpchdr and proghdr. Hence the clients gets garbage rpchdr and proghdr and thus fails to decode the rpc, resulting in disconnect. To prevent this, we need to add the rpchdr and proghdr to a iobref and send it in msg: iobref_add (iobref, iobufs) msg.iobref = iobref; The socket layer takes a ref on msg.iobref, if it cannot write to socket and is adding to the queue. Thus we do not have use after free. Thank You for discussing, debugging and fixing along: Prashanth Pai <ppai@redhat.com> Raghavendra G <rgowdapp@redhat.com> Rajesh Joseph <rjoseph@redhat.com> Kotresh HR <khiremat@redhat.com> Mohammed Rafi KC <rkavunga@redhat.com> Soumya Koduri <skoduri@redhat.com> > Reviewed-on: https://review.gluster.org/16613 > Reviewed-by: Prashanth Pai <ppai@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: soumya k <skoduri@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: Ifa6bf6f4879141f42b46830a37c1574b21b37275 BUG: 1422363 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: https://review.gluster.org/16623 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* cluster/ec: Change log level of messages to DEBUGSunil Kumar Acharya2017-02-161-5/+7
| | | | | | | | | | | | | | | | | | | | | | | | | Heal failed or passed should not be logged as info. These can be observed from heal info if the heal is happening or not. If we require to debug a case where heal is not happening, we can set the level to DEBUG. >Change-Id: I062668eadd145ef809b25e818e6bca1094f54cd6 >BUG: 1420619 >Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com> >Reviewed-on: https://review.gluster.org/16580 >Smoke: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Ashish Pandey <aspandey@redhat.com> Change-Id: I96517f9cf39340ae6883115e464b62339577fc5d BUG: 1422766 Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com> Reviewed-on: https://review.gluster.org/16635 Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* gNFS: Keep the mountdict as long as the service is activeNiels de Vos2017-02-151-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We initialize and take ref once on mountdict during NFS/MNT3 server initialization but seem to be unref'in it for every UMNTALL request. This can lead to crash when there are multiple UMNTALL requests with >=1 active mount entry(/ies) in the mountlist. Since we take the ref only once, we should keep the mountdict through out the life of the process and dereference it only during unitialization of mnt3 service. Cherry picked from commit a88ae92de190af0956013780939ba6bdfd509ff8: > Change-Id: I3238a8df09b8972e56dd93fee426d866d40d9959 > BUG: 1421759 > Signed-off-by: Soumya Koduri <skoduri@redhat.com> > Reviewed-on: https://review.gluster.org/16611 > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> > Reviewed-by: Niels de Vos <ndevos@redhat.com> Change-Id: I3238a8df09b8972e56dd93fee426d866d40d9959 BUG: 1422391 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/16625 Reviewed-by: soumya k <skoduri@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* protocol/client: Fix double free of client fdctx destroyRavishankar N2017-02-153-34/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch fixes the race between fd re-open code and fd release code, both of which free the fd context due to a race in certain variable checks as explained below: 1. client process (shd in the case of this BZ) sends an opendir to its children (client xlators) which send the fop to the bricks to get a valid fd. 2. Client xlator loses connection to the brick. fdctx->remotefd is -1 3. Client re-establishes connection. After handshake, it reopens the dir and sets fdctx->remotefd to a valid fd in client3_3_reopendir_cbk(). 4. Meanwhile, shd sends a fd unref after it is done with the opendir. This triggers a releasedir (since fd->refcount becomes 0). 5. client3_3_releasedir() sees that fdctx-->remotefd is a valid number (i.e not -1), sets fdctx->released=1 and calls client_fdctx_destroy() 6. As a continuation of step3, client_reopen_done() is called by client3_3_reopendir_cbk(), which sees that fdctx->released==1 and again calls client_fdctx_destroy(). Depending on when step-5 does GF_FREE(fdctx), we may crash at any place in step-6 in client3_3_reopendir_cbk() when it tries to access fdctx->{whatever}. > Reviewed-on: https://review.gluster.org/16521 > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> (cherry picked from commit 25fc74f9d1f2b1e7bab76485a99f27abadd10b7b) Change-Id: Ia50873d11763e084e41d2a1f4d53715438e5e947 BUG: 1422350 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: https://review.gluster.org/16619 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* afr: all children of AFR must be up to resolve s-brainRavishankar N2017-02-153-15/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: The various split-brain resolution policies (favorite-child-policy based, CLI based and mount (get/setfattr) based) attempt to resolve split-brain even when not all bricks of replica are up. This can be a problem when say in a replica 3, the only good copy is down and the other 2 bricks are up and blame each other (i.e. split-brain). We end up healing the file in such a case and allow I/O on it. Fix: A decision on whether the file is in split-brain or not must be taken only if we are able to examine the afr xattrs of *all* bricks of a given replica. Signed-off-by: Ravishankar N <ravishankar@redhat.com> > Reviewed-on: https://review.gluster.org/16476 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> (cherry picked from commit 0e03336a9362e5717e561f76b0c543e5a197b31b) Change-Id: Icddb1268b380005799990f5379ef957d84639ef9 BUG: 1420982 Reviewed-on: https://review.gluster.org/16587 Tested-by: Ravishankar N <ravishankar@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd: add a cli command to trigger a statedump on a clientPoornima G2017-02-135-0/+125
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With this, we will be able to trigger statedumps on remote Gluster clients, mainly targetted for applications using libgfapi. Design: SIGUSR signal is the most comman way of taking a statedump in Gluster. But it cannot be used for libgfapi based processes, as the process loading the library might have already consumed SIGUSR signal. Hence going by the command way. One has to issue a Gluster command to initiate a statedump on the libgfapi based client. The command takes hostname and PID as an argument. All the glusterds in the cluster, check if they are connected to the specified hostname, and send an RPC request to all the connected clients from that hostname (via the mgmt connection). > URL: http://review.gluster.org/16357 > BUG: 1169302 > Signed-off-by: Poornima G <pgurusid@redhat.com> > [ndevos: minor fixes and split patch in smaller pieces] > Reviewed-on-master: https://review.gluster.org/9228 > Reviewed-by: Niels de Vos <ndevos@redhat.com> > Tested-by: Niels de Vos <ndevos@redhat.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> > Reviewed-by: Samikshan Bairagya <samikshan@gmail.com> BUG: 1418981 Change-Id: Icbe4d2f026b32a2c7d5535e1bfb2cdaaff042e91 Signed-off-by: Shyam <srangana@redhat.com> Reviewed-on: https://review.gluster.org/16601 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* glusterd: keep snapshot bricks separate from regular onesJeff Darcy2017-02-101-82/+124
| | | | | | | | | | | | | | | | | | | | | | | | | | | The problem here is that a volume's transport options can change, but any snapshots' bricks don't follow along even though they're now incompatible (with respect to multiplexing). This was causing the USS+SSL test to fail. By keeping the snapshot bricks separate (though still potentially multiplexed with other snapshot bricks including those for other volumes) we can ensure that they remain unaffected by changes to their parent volumes. Also fixed various issues with how the test waits (or more precisely didn't) for various events to complete before it continues. Backport of: > Change-Id: Iab4a8a44fac5760373fac36956a3bcc27cf969da > BUG: 1385758 > Reviewed-on: https://review.gluster.org/16544 ` Change-Id: I91c73e3fdf20d23bff15fbfcc03a8a1922acec27 BUG: 1418091 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/16597 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* libglusterfs+changetimerecorder: reduce log noiseJeff Darcy2017-02-102-2/+2
| | | | | | | | | | | | | | | | | | The logging about translator options is so verbose that it significantly slows down scalability tests - sometimes even to the point where it induces timing-related failures. Quiet, please. Backport of: > Change-Id: If0766e2a80746bba586e67e6019ff7084d68b425 > Reviewed-on: https://review.gluster.org/16569 Change-Id: I65117e69427ce1d6a2490832c5c9ab57ee29004e Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/16599 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd: put null check for mod_dict in build_shd_graphAtin Mukherjee2017-02-101-1/+2
| | | | | | | | | | | | | | | | | | | | | >Reviewed-on: https://review.gluster.org/16573 >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Ravishankar N <ravishankar@redhat.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Prashanth Pai <ppai@redhat.com> >Smoke: Gluster Build System <jenkins@build.gluster.org> >(cherry picked from commit 824a48fb5ce20e26ed78b26afd81ec876bc826ec) Change-Id: I849f5f7eb5932d2dc2a76f8abc61e41231f9883e BUG: 1420987 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/16590 Reviewed-by: Ravishankar N <ravishankar@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* trash: fix problem with trash feature under multiplexingJeff Darcy2017-02-102-7/+1
| | | | | | | | | | | | | | | | | | | | | | With multiplexing, the trash translator gets a reconfigure call before a notify(CHILD_UP). In this case, priv->trash_itable was not yet initialized, so the reconfigure would get a SEGV. Moving the itable allocation to init seems to fix it, so trash can be reenabled. Backport of: > Change-Id: I21ac2d7fc66bac1bc4ec70fbc8bae306d73ac565 > BUG: 1420434 > Reviewed-on: https://review.gluster.org/16567 Change-Id: I43a6de6ac5070848619c5f905f075e4a4099c1bd BUG: 1420808 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/16582 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Anoop C S <anoopcs@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd: set default GLUSTERD_QUORUM_RATIO_KEY value to 51Atin Mukherjee2017-02-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | The default value of GLUSTERD_QUORUM_RATIO_KEY is set to 0, while this doesn't harm any functionality as the actual quorum calculation logic always take the factor that if the key is not set in the dictionary the value is considered to be 51 but from an user perspective it gives an incorrect output through volume get where quorum-ratio is always shown as 0 until reconfigured. >Reviewed-on: https://review.gluster.org/16572 >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Smoke: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Change-Id: I087712fa4736055859f29cc3bdbb3b111cbe7224 BUG: 1420989 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/16592 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd: ignore return code of glusterd_restart_bricksAtin Mukherjee2017-02-101-9/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When GlusterD is restarted on a multi node cluster, while syncing the global options from other GlusterD, it checks for quorum and based on which it decides whether to stop/start a brick. However we handle the return code of this function in which case if we don't want to start any bricks the ret will be non zero and we will end up failing the import which is incorrect. Fix is just to ignore the ret code of glusterd_restart_bricks () >Reviewed-on: https://review.gluster.org/16574 >Smoke: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Samikshan Bairagya <samikshan@gmail.com> >Reviewed-by: Jeff Darcy <jdarcy@redhat.com> >(cherry picked from commit 55625293093d485623f3f3d98687cd1e2c594460) Change-Id: I37766b0bba138d2e61d3c6034bd00e93ba43e553 BUG: 1420991 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/16593 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Samikshan Bairagya <samikshan@gmail.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd: glusterd is crashed at the time of stop volumeMohit Agrawal2017-02-091-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: glusterd is crashed at the time of stop volume due to overflow of pidfile array after build rpm with default options. Solution: To avoid the crash update the pidfile array size. Test: To test the patch followed below procedure 1) Setup 1*2 environment and start the volume 2) Stop the volume Before apply the patch glusterd is crashed. Note: The crash is happened only after build rpm with rpmbuild -ba <spec> because _FORTIFY_SOURCE is enabled. This option tries to figure out possible overflow scenarios like the bug here and crash the process. > BUG: 1420202 > Change-Id: I58a006bc0727843a7ed02a10b4ebd5dca39eae67 > Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> > Reviewed-on: https://review.gluster.org/16560 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>1420606 > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Reviewed-by: Atin Mukherjee <amukherj@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > (cherry picked from commit 9ac193a19b0ca6d6548aeafa5c915b26396f8697) Change-Id: I4bcfe830e789a9b95fbcace4495bfda17dca0269 BUG: 1420606 Reviewed-on: https://review.gluster.org/16571 Tested-by: MOHIT AGRAWAL <moagrawa@redhat.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* Tier: remove warning related to the enumhari gowtham2017-02-083-11/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | back-port of : https://review.gluster.org/#/c/16539/ PROBLEM: In the tier as a service patch the enums for tier (from gf1_op_command and gf_defrag_command) are put into a single enum gf_defrag_command which causes a warning that will make the build fail. FIX: send both the enum and eliminate the warning. >Change-Id: I899ff622dfb07134e6459aa65f65ea7252765293 >BUG: 1418973 >Signed-off-by: hari gowtham <hgowtham@redhat.com> >Reviewed-on: https://review.gluster.org/16539 >Smoke: Gluster Build System <jenkins@build.gluster.org> >Tested-by: hari gowtham <hari.gowtham005@gmail.com> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Change-Id: I8d2ec89b8689066091ab58406d225e1058f435cf BUG: 1419846 Signed-off-by: hari gowtham <hgowtham@redhat.com> Reviewed-on: https://review.gluster.org/16553 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: hari gowtham <hari.gowtham005@gmail.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* cluster/dht: Don't update layout in rebalance_task_completionN Balachandran2017-02-071-24/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Updating the layout in the dht inode_ctx in rebalance_task_completion after the file is migrated is erroneous in case of files with hardlinks. This step can be skipped as the layout will be set in the syncop_lookup call post the migration in dht_migrate_file. > Change-Id: I24ac798a919585d91a117d6a207e6a31b88486c6 > BUG: 1415761 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: https://review.gluster.org/16457 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > Reviewed-by: Susant Palai <spalai@redhat.com> (cherry picked from commit ddf05f3d1e39cc920251c809e9ba42fe42b2c5f2) Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I46176f8605cfb782aa17c2bc9ffd7a5f0d07dd88 BUG: 1419855 Reviewed-on: https://review.gluster.org/16554 Tested-by: N Balachandran <nbalacha@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* cluster/disperse: Do not log fop failed for lockless fopsAshish Pandey2017-02-071-12/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Operation failed messages are getting logged based on the callbacks of lockless fop's. If a fop does not take a lock, it is possible that it will get some out of sync xattr, iatts. We can not depend on these callback to psay that the fop has failed. Solution: Print failed messages only for locked fops. However, heal would still be triggered. > Change-Id: I4427402c8c944c23f16073613caa03ea788bead3 > BUG: 1414287 > Signed-off-by: Ashish Pandey <aspandey@redhat.com> > Reviewed-on: http://review.gluster.org/16435 > Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Change-Id: I8728109d5cd93c315a5ada0a50b1f0f158493309 BUG: 1419824 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: https://review.gluster.org/16550 Tested-by: Xavier Hernandez <xhernandez@datalab.es> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* cluster/ec: Do not start heal on good file while IO is going onAshish Pandey2017-02-072-4/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Write on a file has been slowed down significantly after http://review.gluster.org/#/c/13733/ RC : When update fop starts on a file, it sets dirty flag at the start and remove it at the end which make an index entry in indices/xattrop. During IO, SHD scans this and finds out an index and starts heal even if all the fragments are healthy and up tp date. This heal takes inodelk for different types of heal. If the IO is for long time this will happen in every 60 seconds. Due to this extra, unneccessary locking, IO gets slowed down. Solution: Before starting any type of heal check if file needs heal or not. > Change-Id: Ib9519a43e7e4b2565d3f3153f9ca0fb92174fe51 > BUG: 1409191 > Signed-off-by: Ashish Pandey <aspandey@redhat.com> > Reviewed-on: http://review.gluster.org/16377 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> > Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Change-Id: I1a66aca626156164555a7a99a4f715c54c87e9a9 BUG: 1419825 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: https://review.gluster.org/16552 Tested-by: Xavier Hernandez <xhernandez@datalab.es> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* cluster/ec: fix selinux issues with mmap()Xavier Hernandez2017-02-0610-94/+272
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | EC uses mmap() to create a memory area for the dynamic code. Since the code is created on the fly and executed when needed, this region of memory needs to have write and execution privileges. This combination is not allowed by default by selinux. To solve the problem a file is used as a backend storage for the dynamic code and it's mapped into two distinct memory regions, one with write access and the other one with execution access. This approach is the recommended way to create dynamic code by a program in a more secure way, and selinux allows it. Additionally selinux requires that the backend file be stored in a directory marked with type bin_t to be able to map it in an executable area. To satisfy this condition, GLUSTERFS_LIBEXECDIR has been used. This fix also changes the error check for mmap(), that was done incorrectly (it checked against NULL instead of MAP_FAILED), and it also correctly propagates the error codes and makes sure they aren't silently ignored. > Change-Id: I71c2f88be4e4d795b6cfff96ab3799c362c54291 > BUG: 1402661 > Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> > Reviewed-on: https://review.gluster.org/16405 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Change-Id: I5c2dd51b1161505316c8f78b73e9a585d0c115d0 BUG: 1418650 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: https://review.gluster.org/16522 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd : do not load io-threads in client graph for replicate volumesAtin Mukherjee2017-02-064-4/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | client.io-threads has been turned on by default from release-3.9 onwards, however this has an adverse effects on replicate volumes due to the design limitations on replications, till that gets addressed through server side replication as a preventive measure it is wiser not to load io-threads in the client graph for replicate volumes. >Reviewed-on: https://review.gluster.org/16502 >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Smoke: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Prashanth Pai <ppai@redhat.com> >Reviewed-by: Ravishankar N <ravishankar@redhat.com> >Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Change-Id: Ibc576d4517da23fcdf55c6f4d17b90152a8817d7 BUG: 1419305 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/16545 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Ravishankar N <ravishankar@redhat.com>
* glusterd: double-check brick liveness for remove-brick validationJeff Darcy2017-02-031-4/+20
| | | | | | | | | | | | | | | | | | | | | Same problem as https://review.gluster.org/#/c/16509/ in a different place. Tests detach bricks without glusterd's knowledge, so glusterd's internal brick state is out of date and we have to re-check (via the brick's pidfile) as well. Backport of: > BUG: 1385758 > Change-Id: I169538c1c62d72a685a49d57ef65fb6c3db6eab2 > Reviewed-on: https://review.gluster.org/16529 BUG: 1418091 Change-Id: Id0b597bc60807ed090f6ecdba549c5cf3d758f98 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/16537 Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* glusterd: double-check whether brick is alive for statsJeff Darcy2017-02-021-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | With multiplexing, our tests detach bricks from their host processes without glusterd being involved. Thus, when we ask glusterd to fetch profile info, it will try to fetch from a brick that's actually not present any more. While it can handle the process being dead and its RPC connection being closed, it barfs if it gets a negative response from a live brick process. This is not a problem in normal use, because the brick can't disappear without glusterd seeing it. The fix is to double check that the brick is actually running, by looking for its pidfile which the tests *do* clean up as part of killing a brick. Backport of: > Change-Id: I098465b175ecf23538bd7207357c752a2bba8f4e > BUG: 1385758 > Reviewed-on: https://review.gluster.org/16509 BUG: 1418091 Change-Id: Ia61e273134520c8ccfa3371ee2370cb9a1920877 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/16532 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* performance/write-behind: access stub only if available duringRaghavendra G2017-02-021-12/+12
| | | | | | | | | | | | | | | | | | | | | | | statedump >Change-Id: Ia5dd718458a5e32138012f81f014d13fc6b28be2 >BUG: 1415115 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> >Reviewed-on: https://review.gluster.org/16440 >Reviewed-by: N Balachandran <nbalacha@redhat.com> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Smoke: Gluster Build System <jenkins@build.gluster.org> Change-Id: Ia5dd718458a5e32138012f81f014d13fc6b28be2 BUG: 1418623 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 85d7f1d1ee24ac400d4aa223478727643532693a) Reviewed-on: https://review.gluster.org/16519 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* libglusterfs+transport+io-threads: fix 256KB stack abuseJeff Darcy2017-02-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Some functions were allocating 64K booleans, which are (crazily) mapped to 4-byte ints, for a total of 256KB per call. Changed to use bitfields instead, so usage is now only 8KB per call. This was the impediment to changing the io-threads stack size, so that has been adjusted too. Backport of: > Change-Id: I8781c4f2c8f2b830f4535e366995fac8dd0a8653 > BUG: 1418095 > Reviewed-on: https://review.gluster.org/15745 Change-Id: Ia5dada61703e6bea95f2511da71feb573fc9a429 BUG: 1418536 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/16511 Reviewed-by: N Balachandran <nbalacha@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* core: run many bricks within one glusterfsd processJeff Darcy2017-02-0131-453/+1200
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds support for multiple brick translator stacks running in a single brick server process. This reduces our per-brick memory usage by approximately 3x, and our appetite for TCP ports even more. It also creates potential to avoid process/thread thrashing, and to improve QoS by scheduling more carefully across the bricks, but realizing that potential will require further work. Multiplexing is controlled by the "cluster.brick-multiplex" global option. By default it's off, and bricks are started in separate processes as before. If multiplexing is enabled, then *compatible* bricks (mostly those with the same transport options) will be started in the same process. Backport of: > Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb > BUG: 1385758 > Reviewed-on: https://review.gluster.org/14763 Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8 BUG: 1418091 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/16496 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd: regenerate volfiles on op-version bump upAtin Mukherjee2017-02-011-28/+59
| | | | | | | | | | | | | | | | | | | >Reviewed-on: https://review.gluster.org/16455 >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >Smoke: Gluster Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Prashanth Pai <ppai@redhat.com> >Reviewed-by: Kaushal M <kaushal@redhat.com> Change-Id: I2fe7a3ebea19492d52253ad5a1fdd67ac95c71c8 BUG: 1417521 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/16475 Reviewed-by: Prashanth Pai <ppai@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* debug/trace: Print entries' iatts in readdirp cbkKrutika Dhananjay2017-01-311-2/+16
| | | | | | | | | | | | | Backport of: http://review.gluster.org/16432 Change-Id: I09d0c34918299626f0a2c58c8f864f2a0dbd0ff2 BUG: 1417804 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/16484 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* md-cache: Cache security.ima xattrsPoornima G2017-01-302-2/+94
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/16296 From kernel version 3.X or greater, creating of a file results in removexattr call on security.ima xattr. But this xattr is not set on the file unless IMA feature is active. With this patch, removxattr call returns ENODATA if it is not found in the cache. > Change-Id: I8136096598a983aebc09901945eba1db1b2f93c9 > Signed-off-by: Poornima G <pgurusid@redhat.com> > Reviewed-on: http://review.gluster.org/16296 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > (cherry picked from commit ac629e574935a8aed6526936bc83b1c6d295ae67) Change-Id: I27abc23024c8fcf07389608df61ef6e64736d414 BUG: 1415918 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: https://review.gluster.org/16460 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/ec: Change level of messages to DEBUGAshish Pandey2017-01-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | Heal failed or passed should not be logged as warning. These can be observed from heal info if the heal is happening or not. If we require to debug a case where heal is not happening, we can set the level to DEBUG. >Change-Id: I347665c8c8b6223bb08a9f3dd5643a10ddc3b93e >BUG: 1417050 >Signed-off-by: Ashish Pandey <aspandey@redhat.com> >Reviewed-on: https://review.gluster.org/16473 >Smoke: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Signed-off-by: Ashish Pandey <aspandey@redhat.com> Change-Id: I347665c8c8b6223bb08a9f3dd5643a10ddc3b93e BUG: 1417135 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: https://review.gluster.org/16478 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
* glusterd: daemon restart logic should adhere server side quorumAtin Mukherjee2017-01-301-6/+7
| | | | | | | | | | | | | | | | | | | | | | | | Just like brick processes, other daemon services should also follow the same logic of quorum checks to see if a particular service needs to come up if glusterd is restarted or the incoming friend add/update request is received (in glusterd_restart_bricks () function) >Reviewed-on: https://review.gluster.org/15626 >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Smoke: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Prashanth Pai <ppai@redhat.com> Change-Id: I54a1fbdaa1571cc45eed627181b81463fead47a3 BUG: 1417042 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/16472 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Samikshan Bairagya <samikshan@gmail.com> Reviewed-by: Prashanth Pai <ppai@redhat.com>
* glusterd, rda: If parallel readdir is enabled, split the cache limitPoornima G2017-01-273-2/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With patch http://review.gluster.org/#/c/16072/ readdir-ahead can be loaded as a child of dht. i.e. there can be more than one instance of readdir-ahead in client process. In this case the rda-cache-size should be split among all the readdir-ahead instances. Also the value of rda-request-size is considered as the minimum cache size of any readdir-ahead instance. >Change-Id: Iea2fe6d4c46adc09dd2e9a252332a0fe3005f2b9 >BUG: 1401812 >Signed-off-by: Poornima G <pgurusid@redhat.com> >Reviewed-on: https://review.gluster.org/16424 >Smoke: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Kaushal M <kaushal@redhat.com> >Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: Iea2fe6d4c46adc09dd2e9a252332a0fe3005f2b9 BUG: 1417028 Signed-off-by: Poornima G <pgurusid@redhat.com> (cherry picked from commit f245dc568e3c22882e22ddd3e26a4207f5704e3b) Reviewed-on: https://review.gluster.org/16459 Tested-by: Raghavendra G <rgowdapp@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* Readdir-ahead : Honour readdir-optimise option of dhtPoornima G2017-01-274-1/+33
| | | | | | | | | | | | | | | | | | | | | | | >Change-Id: I9c5e65b32e316e6a2fc7e1f5c79fce79386b78e2 >BUG: 1401812 >Signed-off-by: Poornima G <pgurusid@redhat.com> >Reviewed-on: https://review.gluster.org/16071 >Smoke: Gluster Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I9c5e65b32e316e6a2fc7e1f5c79fce79386b78e2 BUG: 1417027 Signed-off-by: Poornima G <pgurusid@redhat.com> (cherry picked from commit 7c6538f6c8f9a015663b4fc57c640a7c451c87f7) Reviewed-on: https://review.gluster.org/16461 Tested-by: Raghavendra G <rgowdapp@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* core: remove experimental xlators and associated testsKaleb S. KEITHLEY2017-01-2450-5807/+1
| | | | | | | | | | | | | | experimental xlators not included in 3.10 Change-Id: I547480ee5e7912664784643e436feb198b6d16d0 BUG: 1415866 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: https://review.gluster.org/16447 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* dht/rebalance Estimate time to complete rebalanceN Balachandran2017-01-244-4/+148
| | | | | | | | | | | | | | | | | | | | | | | | | The estimates will be logged to the rebalance log on running gluster v rebalance <vol> status > Change-Id: I9d51b139cd4c8dfde1ff2c2050720ae606c13fc6 > BUG: 1396004 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/15893 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 2edd75ec8de17da89004859375844f60890a4df0) Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I58b0550210149443966798d9a26e73cb598eeb6a BUG: 1415915 Reviewed-on: https://review.gluster.org/16458 Tested-by: N Balachandran <nbalacha@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* features/trash: Create trash directory only when it is enabledJiffin Tony Thottan2017-01-232-401/+488
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously trash directory was being created as part of volume start operation. And also the user/admin could not delete this directory from volume even if it is not needed. This patch will fix the same. From now onwards creation and enforcement on trash directory will come into pictrure only when trash translator is enabled. Similarly exact same behaviour is reflected on internal-op directory inside trash directory. Upstream reference : >Change-Id: I3e58316a7b299a691885e458c960438bec2220fb >BUG: 1264849 >Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> >Reviewed-on: http://review.gluster.org/12256 >Smoke: Gluster Build System <jenkins@build.gluster.org> >Tested-by: Anoop C S <anoopcs@redhat.com> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Anoop C S <anoopcs@redhat.com> >Reviewed-by: Jeff Darcy <jdarcy@redhat.com> >(cherry picked from commit 07b9853ad0c92b341be33a6cd632013c416221c8) Change-Id: I3e58316a7b299a691885e458c960438bec2220fb BUG: 1415581 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: https://review.gluster.org/16454 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* features/changelog: Fix htime xattr during brick crashv3.11devKotresh HR2017-01-191-1/+30
| | | | | | | | | | | | | | | | | | | | | | The htime file contains the path of all the changelogs that is rolloved over till now. It also maintains xattr which tracks the latest changelog file rolloved over and the number of changelogs. The path and and xattr update happens in two different system calls. If the brick is crashed between them, the xattr value becomes stale and can lead to the failure of gf_history_changelog. To identify this, the total number of changelogs is being calculated based on htime file size and the record length. The above value is used in case of mismatch. Change-Id: Ia1c3efcfda7b74227805bb2eb933c9bd4305000b BUG: 1413967 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/16420 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* glusterd: bypass add-brick validation with forceAtin Mukherjee2017-01-181-3/+3
| | | | | | | | | | | | | | | | | | | | | Commit c916a2f added a validation to restrict add-brick operation if a replica configuration is changed and any of the bricks belonging to the volume is down. However we should bypass this validation with a force option if users really want to have add-brick to go through at the sake of the corner cases of data loss issue. The original problem of add-brick getting failed when layout is not set will still be a problem with a force option as the issue has to be taken care in the DHT layer. Change-Id: I0ed3df91ea712f77674eb8afc6fdfa577f25a7bb BUG: 1406411 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/16358 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Ravishankar N <ravishankar@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* geo-rep: Separate slave mount logs for each connectionKotresh HR2017-01-182-6/+6
| | | | | | | | | | | | | | | | | | | | | Geo-rep worker mounts the slave volume on the slave node. If multiple worker connects to same slave node, all workers share the same mount log file. This is very difficult to debug as logs are cluttered from different mounts. Hence creating separate mount log file for each connection from worker. Each connection from worker is identified uniquely using 'mastervol uuid', 'master host', 'master brickpath', 'salve vol'. The log file name will be combination of the above. Change-Id: I67871dc8e8ea5864e2ad55e2a82063be0138bf0c BUG: 1412689 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/16384 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* glusterd: Change the volfile to have readdir-ahead as a childPoornima G2017-01-174-18/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | of dht As mentioned in feature page http://review.gluster.org/#/c/16090/ readdir-ahead will be optionally placed below dht. There are two options: 1. performance.readdir-ahead 2. performance.parallel-readdir If only option is enabled, then readdir ahead is placed at its original place as an ancestor of dht. If both the options 1 and 2 are enabled then readdir ahead is placed as a child of dht. Also changes have been made to retain the rebalance, quotad, snapd vol files to remain unchanged. Change-Id: I0adf0b476fcbf91251f5a2fee2241786a3d8255a BUG: 1401812 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/16072 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* fuse: Fix a possible resource leak under GF_SOLARIS_HOST_OSSaurabh Badhwar2017-01-171-1/+3
| | | | | | | | | | | | | | in fuse-helpers.c Change-Id: Ie367a6dec2a0d5848631b19ebbe39ceafa954a60 BUG: 1412918 Signed-off-by: Saurabh Badhwar <sbsaurabhbadhwar9@gmail.com> Reviewed-on: http://review.gluster.org/16395 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-27 10:26:20 +0000