glusterfs.git/xlators/mgmt/glusterd/src/glusterd-handler.c, branch v7.0rc2 glusterd/svc: update pid of mux volumes from the shd process 2019-07-24T10:29:17+00:00 Mohammed Rafi KC rkavunga@redhat.com 2019-06-24T06:30:20+00:00 47fcbc4c055a7880d2926e918ae1e1f57c7db20d For a normal volume, we are updating the pid from a the process while we do a daemonization or at the end of the init if it is no-daemon mode. Along with updating the pid we also lock the file, to make sure that the process is running fine. With brick mux, we were updating the pidfile from gluterd after an attach/detach request. There are two problems with this approach. 1) We are not holding a pidlock for any file other than parent process. 2) There is a chance for possible race conditions with attach/detach. For example, shd start and a volume stop could race. Let's say we are starting an shd and it is attached to a volume. While we trying to link the pid file to the running process, this would have deleted by the thread that doing a volume stop. Backport of : https://review.gluster.org/#/c/glusterfs/+/22935/ >Change-Id: I29a00352102877ce09ea3f376ca52affceb5cf1a >Updates: bz#1722541 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Change-Id: I29a00352102877ce09ea3f376ca52affceb5cf1a Updates: bz#1732668 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> 
For a normal volume, we are updating the pid from a the
process while we do a daemonization or at the end of the
init if it is no-daemon mode. Along with updating the pid
we also lock the file, to make sure that the process is
running fine.

With brick mux, we were updating the pidfile from gluterd
after an attach/detach request.

There are two problems with this approach.
1) We are not holding a pidlock for any file other than parent
   process.
2) There is a chance for possible race conditions with attach/detach.
   For example, shd start and a volume stop could race. Let's say
   we are starting an shd and it is attached to a volume.
   While we trying to link the pid file to the running process,
   this would have deleted by the thread that doing a volume stop.

Backport of : https://review.gluster.org/#/c/glusterfs/+/22935/
>Change-Id: I29a00352102877ce09ea3f376ca52affceb5cf1a
>Updates: bz#1722541
>Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>

Change-Id: I29a00352102877ce09ea3f376ca52affceb5cf1a
Updates: bz#1732668
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
glusterd: do not mark skip_locking as true for geo-rep operations 2019-07-19T06:18:23+00:00 Sanju Rakonde srakonde@redhat.com 2019-07-12T10:58:04+00:00 c79e07f54e251ca2d4b2dabd5960b1ca6f1c3f37 We need to send the commit req to peers in case of geo-rep operations even though it is a no volname operation. In commit phase peers try to set the txn_opinfo which will fail because it is a no volname operation where we don't require a commit phase. We mark skip_locking as true for no volname operations, but we have to give an exception to geo-rep operations, so that they can set txn_opinfo in commit phase. Please refer to detailed RCA at the bug: 1730543 fixes: bz#1730543 Change-Id: I9f2478b12a281f6e052035c0563c40543493a3fc Signed-off-by: Sanju Rakonde <srakonde@redhat.com> (cherry picked from commit b917974ee922d7a2e079692ad7d6f61f900b37b2) 
We need to send the commit req to peers in case of geo-rep
operations even though it is a no volname operation. In commit
phase peers try to set the txn_opinfo which will fail because
it is a no volname operation where we don't require a commit
phase. We mark skip_locking as true for no volname operations,
but we have to give an exception to geo-rep operations, so that
they can set txn_opinfo in commit phase.

Please refer to detailed RCA at the bug: 1730543

fixes: bz#1730543

Change-Id: I9f2478b12a281f6e052035c0563c40543493a3fc
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
(cherry picked from commit b917974ee922d7a2e079692ad7d6f61f900b37b2)
glusterd: Show the correct brick status in get-state 2019-07-15T13:19:32+00:00 Mohit Agrawal moagrawal@redhat.com 2019-07-03T09:52:38+00:00 4d84f0afee639c2a9e5e3c83d3ae9d5b8692997c Problem: get-state does not show correct brick status if brick status is not Started, it always shows started if any value is set brickinfo->status Solution: Check the value of brickinfo->status to show correct status in get-state Change-Id: I12a79619024c2cf59f338220d144f2f034059b3b fixes: bz#1726905 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com> (cherry picked from commit af989db23d1db00e087f2b9d3dfc43b13ef17153) 
Problem: get-state does not show correct brick status if brick
         status is not Started, it always shows started if any value
         is set brickinfo->status

Solution: Check the value of brickinfo->status to show correct status
          in get-state

Change-Id: I12a79619024c2cf59f338220d144f2f034059b3b
fixes: bz#1726905
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
(cherry picked from commit af989db23d1db00e087f2b9d3dfc43b13ef17153)
glusterd/thin-arbiter: Thin-arbiter integration with GD1 2019-07-04T07:42:11+00:00 Vishal Pandey vpandey@redhat.com 2019-04-24T08:07:16+00:00 08c87ae4208b73f4f183f7b54ebcb373e8bc0ede gluster volume create <VOLNAME> replica 2 thin-arbiter 1 <host1>:<brick1> <host2>:<brick2> <thin-arbiter-host>:<path-to-store-replica-id-file> [force] The changes have been made in a way that the last brick in the bricks list will be treated as the thin-arbiter. GD1 will be manipulated to consider replica count to be as 2 and continue creating the volume like any other replica 2 volume but since thin-arbiter volumes need ta-brick client xlator entries for each subvolume in fuse volfile, volfile generation is modified in a way to inject these entries seperately in the volfile for every subvolume. Few more additions - 1- Save the volinfo with new fields ta_bricks list and thin_arbiter_count. 2- Introduce a new option client.ta-brick-port to add remote-port to ta-brick xlator entry in fuse volfiles. The option can be set using the following CLI syntax - gluster volume set <VOLNAME> client.ta-brick-port <PORTNO.> 3- Volume Info will contain a Thin-Arbiter-path entry to distinguish from other replicate volumes. Change-Id: Ib434e2313b29716f32476c6c211d282c4ef39406 Updates #687 Signed-off-by: Vishal Pandey <vpandey@redhat.com> (cherry picked from commit 9b223b15ab69fce4076de036ee162f36a058bcd2) 
gluster volume create <VOLNAME> replica 2 thin-arbiter 1 <host1>:<brick1> <host2>:<brick2>
<thin-arbiter-host>:<path-to-store-replica-id-file> [force]

The changes have been made in a way that the last brick in the bricks list
will be treated as the thin-arbiter.
GD1 will be manipulated to consider replica count to be as 2 and continue creating the
volume like any other replica 2 volume but since thin-arbiter volumes need ta-brick
client xlator entries for each subvolume in fuse volfile, volfile generation is
modified in a way to inject these entries seperately in the volfile for every subvolume.

Few more additions -
1- Save the volinfo with new fields ta_bricks list and thin_arbiter_count.
2- Introduce a new option client.ta-brick-port to add remote-port to ta-brick xlator entry
   in fuse volfiles. The option can be set using the following CLI syntax -
   gluster volume set <VOLNAME> client.ta-brick-port <PORTNO.>
3- Volume Info will contain a Thin-Arbiter-path entry to distinguish
   from other replicate volumes.

Change-Id: Ib434e2313b29716f32476c6c211d282c4ef39406
Updates #687
Signed-off-by: Vishal Pandey <vpandey@redhat.com>
(cherry picked from commit 9b223b15ab69fce4076de036ee162f36a058bcd2)
glusterd-volgen.c: remove BD xlator from the graph 2019-06-18T12:09:09+00:00 Yaniv Kaul ykaul@redhat.com 2019-05-26T08:18:05+00:00 2d278f0407ab7d29507dc697653b39d72ddee472 The BD xlator was removed some time ago. Remove it from the graph. We can also remove the caps settings - only the BD xlator was using it. Lastly, remove the caps (which only BD was using) and the document describing the translator. Change-Id: Id0adcb2952f4832a5dc6301e726874522e07935d updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> 
The BD xlator was removed some time ago. Remove it from the graph.
We can also remove the caps settings - only the BD xlator
was using it.

Lastly, remove the caps (which only BD was using) and the document
describing the translator.

Change-Id: Id0adcb2952f4832a5dc6301e726874522e07935d
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
glusterd/tier: remove tier related code from glusterd 2019-05-27T07:50:24+00:00 Hari Gowtham hgowtham@redhat.com 2019-05-02T13:03:34+00:00 e1cc4275583dfd8ae8d0433587f39854c1851794 The handler functions are pointed to dummy functions. The switch case handling for tier also have been moved to point default case to avoid issues, if reintroduced. The tier changes in DHT still remain as such. updates: bz#1693692 Change-Id: I80d80c9a3eb862b4440a36b31ae82b2e9d92e4dc Signed-off-by: Hari Gowtham <hgowtham@redhat.com> 
The handler functions are pointed to dummy functions.
The switch case handling for tier also have been moved to
point default case to avoid issues, if reintroduced.

The tier changes in DHT still remain as such.

updates: bz#1693692

Change-Id: I80d80c9a3eb862b4440a36b31ae82b2e9d92e4dc
Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
glusterd: Fix coverity defects & put coverity annotations 2019-05-02T08:06:06+00:00 Atin Mukherjee amukherj@redhat.com 2019-04-25T06:30:52+00:00 91c19c50a3c7b336177dda186500568be43626b8 Along with fixing few defect, put the required annotations for the defects which are marked ignore/false positive/intentional as per the coverity defect sheet. This should avoid the per component graph showing many defects as open in the coverity glusterfs web page. Updates: bz#789278 Change-Id: I19461dc3603a3bd8f88866a1ab3db43d783af8e4 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> 
Along with fixing few defect, put the required annotations for the defects which
are marked ignore/false positive/intentional as per the coverity defect sheet.
This should avoid the per component graph showing many defects as open in the
coverity glusterfs web page.

Updates: bz#789278
Change-Id: I19461dc3603a3bd8f88866a1ab3db43d783af8e4
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
glusterd: coverity fixes 2019-04-26T13:15:36+00:00 Atin Mukherjee amukherj@redhat.com 2019-04-26T03:17:12+00:00 13e1fe6dbd39ef88cadce1590b5d4c45b67387cb 1400775 - USE_AFTER_FREE 1400742 - Missing Unlock 1400736 - CHECKED_RETURN 1398470 - Missing Unlock Missing unlock is the tricky one, we have had annotation added, but coverity still continued to complaint. Added pthread_mutex_unlock to clean up the lock before destroying it to see if it makes coverity happy. Updates: bz#789278 Change-Id: I1d892612a17f805144d96c1b15004a85a1639414 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> 
1400775 - USE_AFTER_FREE
1400742 - Missing Unlock
1400736 - CHECKED_RETURN
1398470 - Missing Unlock

Missing unlock is the tricky one, we have had annotation added, but
coverity still continued to complaint. Added pthread_mutex_unlock to
clean up the lock before destroying it to see if it makes coverity
happy.

Updates: bz#789278
Change-Id: I1d892612a17f805144d96c1b15004a85a1639414
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
glusterd: coverity fixes 2019-04-25T06:39:16+00:00 Atin Mukherjee amukherj@redhat.com 2019-04-24T16:32:51+00:00 c827682e4df44ec6aaae3780c325568fb43053ff Addresses the following: * CID 1124776: Resource leaks (RESOURCE_LEAK) - Variable "aa" going out of scope leaks the storage it points to in glusterd-volgen.c * Bunch of CHECKED_RETURN defects in the callers of synctask_barrier_init * CID 1400755: Error handling issues (CHECKED_RETURN) - Calling "gf_is_service_running" without checking return value in xlators/mgmt/glusterd/src/glusterd-shd-svc.c: 671 in glusterd_shdsvc_stop() * CID 1400745: Memory - illegal accesses (USE_AFTER_FREE) - Dereferencing freed pointer "volinfo" in /xlators/mgmt/glusterd/src/glusterd-shd-svc.c: 460 in glusterd_shdsvc_start() * CID 1400742: Program hangs (LOCK) - adding annotation to fix this false positive Updates: bz#789278 Change-Id: I02f16e7eeb8c5cf72f7d0b29d00df4f03b3718b3 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> 
Addresses the following:

* CID 1124776:  Resource leaks  (RESOURCE_LEAK) - Variable "aa" going out
of scope leaks the storage it points to in glusterd-volgen.c

* Bunch of CHECKED_RETURN defects in the callers of synctask_barrier_init

* CID 1400755:  Error handling issues  (CHECKED_RETURN) - Calling
"gf_is_service_running" without checking return value in
xlators/mgmt/glusterd/src/glusterd-shd-svc.c: 671 in
glusterd_shdsvc_stop()

* CID 1400745:  Memory - illegal accesses  (USE_AFTER_FREE) - Dereferencing
freed pointer "volinfo" in /xlators/mgmt/glusterd/src/glusterd-shd-svc.c: 460 in glusterd_shdsvc_start()

* CID 1400742:  Program hangs  (LOCK) - adding annotation to fix this
false positive

Updates: bz#789278
Change-Id: I02f16e7eeb8c5cf72f7d0b29d00df4f03b3718b3
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
glusterd: provide a way to detach failed node 2019-04-11T15:15:37+00:00 Sanju Rakonde srakonde@redhat.com 2019-04-09T08:26:24+00:00 35dd17e222fe172866e15824e5ac9d242382178c When a gluster node in trusted storage pool has failed due to hardware issues, volume delete operation fails saying "Not all peers are up" and peer detach for failed node fails saying "Brick(s) with peer <peer_ip> exists in cluster". The idea here is to use either replace-brick or remove-brick command to remove all the bricks hosted by failed node and then re-attempting the peer detach. This change adds this trick in peer detach error message. fixes: bz#1697866 Change-Id: I0c58887479d31db603ad8d6535ea9d547880ccc8 Signed-off-by: Sanju Rakonde <srakonde@redhat.com> 
When a gluster node in trusted storage pool has failed
due to hardware issues, volume delete operation fails
saying "Not all peers are up" and peer detach for failed
node fails saying "Brick(s) with peer <peer_ip> exists
in cluster".

The idea here is to use either replace-brick or remove-brick
command to remove all the bricks hosted by failed node and
then re-attempting the peer detach. This change adds this
trick in peer detach error message.

fixes: bz#1697866

Change-Id: I0c58887479d31db603ad8d6535ea9d547880ccc8
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>