1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
|
Work In Progress
Author - Satheesaran Sundaramoorthi
<sasundar@redhat.com>
**Introduction**
----------------
Gluster volumes are used to host Virtual Machines Images. (i.e) Virtual
machines Images are stored on gluster volumes. This usecase is popularly
known as *virt-store* usecase.
This document explains more about,
1. Enabling gluster volumes for virt-store usecase
2. Common Pitfalls
3. FAQs
4. References
**Enabling gluster volumes for virt-store**
-------------------------------------------
This section describes how to enable gluster volumes for virt store
usecase
#### Volume Types
Ideally gluster volumes serving virt-store, should provide
high-availability for the VMs running on it. If the volume is not
avilable, the VMs may move in to unusable state. So, its best
recommended to use **replica** or **distribute-replicate** volume for
this usecase
*If you are new to GlusterFS, you can take a look at
[QuickStart](http://gluster.readthedocs.org/en/latest/Quick-Start-Guide/Quickstart/) guide or the [admin
guide](http://gluster.readthedocs.org/en/latest/Administrator%20Guide/README/)*
#### Tunables
The set of volume options are recommended for virt-store usecase, which
adds performance boost. Following are those options,
quick-read=off
read-ahead=off
io-cache=off
stat-prefetch=off
eager-lock=enable
remote-dio=enable
quorum-type=auto
server-quorum-type=server
- quick-read is meant for improving small-file read performance,which
is no longer reasonable for VM Image files
- read-ahead is turned off. VMs have their own way of doing that. This
is pretty usual to leave it to VM to determine the read-ahead
- io-cache is turned off
- stat-prefetch is turned off. stat-prefetch, caches the metadata
related to files and this is no longer a concern for VM Images (why
?)
- eager-lock is turned on (why?)
- remote-dio is turned on,so in open() and creat() calls, O\_DIRECT
flag will be filtered at the client protocol level so server will
still continue to cache the file.
- quorum-type is set to auto. This basically enables client side
quorum. When client side quorum is enabled, there exists the rule
such that atleast half of the bricks in the replica group should be
UP and running. If not, the replica group would become read-only
- server-quorum-type is set to server. This basically enables
server-side quorum. This lays a condition that in a cluster, atleast
half the number of nodes, should be UP. If not the bricks ( read as
brick processes) will be killed, and thereby volume goes offline
#### Applying the Tunables on the volume
There are number of ways to do it.
1. Make use of group-virt.example file
2. Copy & Paste
##### Make use of group-virt.example file
This is the method best suited and recommended.
*/etc/glusterfs/group-virt.example* has all options recommended for
virt-store as explained earlier. Copy this file,
*/etc/glusterfs/group-virt.example* to */var/lib/glusterd/groups/virt*
cp /etc/glusterfs/group-virt.example /var/lib/glusterd/groups/virt
Optimize the volume with all the options available in this *virt* file
in a single go
gluster volume set <vol-name> group virt
NOTE: No restart of the volume is required Verify the same with the
command,
gluster volume info
In forthcoming releases, this file will be automatically put in
*/var/lib/glusterd/groups/* and you can directly apply it on the volume
##### Copy & Paste
Copy all options from the above
section,[Virt-store-usecase\#Tunables](Virt-store-usecase#Tunables "wikilink")
and put in a file named *virt* in */var/lib/glusterd/groups/virt* Apply
all the options on the volume,
gluster volume set <vol-name> group virt
NOTE: This is not recommended, as the recommended volume options may/may
not change in future.Always stick to *virt* file available with the rpms
#### Adding Ownership to Volume
You can add uid:gid to the volume,
gluster volume set <vol-name> storage.owner-uid <number>
gluster volume set <vol-name> storage.owner-gid <number>
For example, when the volume would be accessed by qemu/kvm, you need to
add ownership as 107:107,
gluster volume set <vol-name> storage.owner-uid 107
gluster volume set <vol-name> storage.owner-gid 107
It would be 36:36 in the case of oVirt/RHEV, 165:165 in the case of
OpenStack Block Service (cinder),161:161 in case of OpenStack Image
Service (glance) is accessing this volume
NOTE: Not setting the correct ownership may lead to "Permission Denied"
errors when accessing the image files residing on the volume
**Common Pitfalls**
-------------------
**FAQs**
--------
**References**
--------------
|