summaryrefslogtreecommitdiffstats
path: root/doc/release-notes/4.0.0.md
blob: 5b7211a12aa361fbb4dbdb99445c372f290fc0b8 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
# Release notes for Gluster 4.0.0

** DRAFT DRAFT ! DRAFT !!!**

**Header for the release - TBD**

The most notable features and changes are documented on this page. A full list
of bugs that have been addressed is included further below.

- [Major changes and features](#major-changes-and-features)
- [Major issues](#major-issues)
- [Bugs addressed in the release](#bugs-addressed)

Further, as 3.13 is a short term maintenance release, features included
in that release are available with 4.0.0 as well, and could be of interest to
users upgrading to 4.0.0 from older than 3.13 releases. The 3.13 [release notes](http://docs.gluster.org/en/latest/release-notes/)
captures the list of features that were introduced with 3.13.

**NOTE:** As 3.13 was a short term maintenance release, it will reach end of
life (EOL) with the release of 4.0.0. ([reference](https://www.gluster.org/release-schedule/))

## Major changes and features

Features are categorized into the following sections,

- [Management](#management)
- [Monitoring](#monitoring)
- [Performance](#performance)
- [Geo-replication](#geo-replication)
- [Standalone](#standalone)
- [Developer related](#developer-related)

### Management

#### 1. GlusterD2
**Notes for users:**
- Need GD2 team to fill in enough links and information here covering,
    - What it is
    - Install and configuration steps
    - Future plans (what happens in 4.1.0 and further)

**Limitations:**

**Known Issues:**

### Monitoring

The lack of live monitoring support on top of GlusterFS till date was a
limiting factor for many users (and in many cases for developers too).
[Statedump](docs.gluster.org/en/latest/Troubleshooting/statedump/) did some work of helping during debugging, but was heavy for
live monitoring.

Further, the existence of `debug/io-stats` translator was not known to many and
`gluster volume profile` was not recommended as it took a hit on performance.

With this release, glusterfs's core infrastructure itself gets some mechanisms
to provide internal information, that avoids the heavy weight nature of prior
monitoring mechanisms.

#### 1. Metrics collection across every FOP in every xlator
**Notes for users:**
Gluster, with this release, has in-built latency measures in the xlator
abstraction, thus enabling capture of metrics and usage patterns across
workloads.

These measures are currently enabled by default.

**Limitations:**
This feature is auto-enabled and cannot be disabled.

Providing means to disable the same in future releases also may not be made
available, as the data generated is deemed critical to understand and
troubleshoot gluster usage patterns.

#### 2. Monitoring support
**Notes for users:**
Currently, only project which consumes the metrics and provides basic
monitoring is [glustermetrics](https://github.com/amarts/glustermetrics), which provides a good idea on how to
utilize the metrics dumped from the processes.

Users can send SIGUSR2 signal to the process to dump the metrics, in
`/var/run/gluster/metrics/` directory.

**Limitations:**
Currently core gluster stack and memory management systems dump metrics. For
other translators and other core components, framework to provide more metrics
exists, but additional metrics are not added in this release.

### Performance
#### 1. EC: Make metadata [F]GETXATTR operations faster
**Notes for users:**
Optimized getxattr and fgetxattr in disperse volumes to speed up the operation.
Disperse translator, now forwards the request to one of the bircks that is
deemed to have a good copy, instead of all, thus improving the efficiency of the
operation.

#### 2. Allow md-cache to serve nameless lookup from cache
With this enhancement, some more lookups will be served from the cache. The workloads
that will perform better with this change are:
- Directory listing in FUSE mount with ACL enabled, will be faster.
- NFS workloads also will be benifited.

**Notes for users:**
Issue: https://github.com/gluster/glusterfs/issues/232
There is no specific option to enable this feature, it the default behaviour
when metadata caching is enabled. To enable metadata caching please refet to
Performance Tuning section in Admin Guide.

**Limitations:**
None

**Known Issues:**
None

#### 3. md-cache: Allow runtime addition of xattrs to the list of xattrs that md-cache caches
   Md-cache caches the stat and xattrs on the client side. In all the previous releases, the
list of xattrs that can be cached in the client side was hardcoded in the libraries, user could
only enable or disable the prelisted xattrs cache. With this feature, we can add new xattrs to
the list of xattrs that can be cached on the client side.

**Notes for users:**
Issue: https://github.com/gluster/glusterfs/issues/297
To add xattrs to the cache list, execute the following command:
    ```
    # gluster volume set <volname> xattr-cache-list "comma separated xattr list"
    ```
The option like "cache-samba-metadata" "cache-swift-metadata" which also adds some xattrs
to the cache list, still works. The new option "xattr-cache-list" appends to the list
generated by the old options. Refer to the Performance Tuning section in the Admin Guide
for further information.

**Limitations:**
Currently, setting this option overwrites the previous value set for this option.
The append to the existing list of xattr is currently not available

**Known Issues:**
None

#### 4. Cache last stripe of an EC volume while write is going on
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/256
- Release notes:
    - Needs option documentation and also use-case details, as to when to enable this

**Limitations:**


**Known Issues:**

#### 5. tie-breaker logic for blocking inodelks/entrylk in SHD
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/354
- Release notes:
    - Internal feature to enable faster(?) self-heal, mention it as such

**Limitations:**


**Known Issues:**

#### 6. Independent eager-lock options for file and directory accesses
**Notes for users:**
A new option named 'disperse.other-eager-lock' has been added to make it
possible to have different settings for regular file accesses and accesses
to other types of files (like directories).

By default this option is enabled to keep the same behaviour as the previous
versions. If you have multiple clients creating, renaming or removing files
from the same directory, you can disable this option to improve the performance
for these users while still keeping best performance for file accesses.

### Geo-replication
#### 1. JSON output for Geo-rep status and config for Glusterd2 integration
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/361
- Added option --json to the gsyncd.py script
    - Are end users going to use the status command here?
    - Documentation update?
- Release notes:
    - If there is no end user impact, no specific notes are needed for the same

**Limitations:**


**Known Issues:**

#### 2. Enhance Geo-replication to use Volinfo from Config file
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/396
- Documentation needs to be updated with the changed conf file section
- Release notes:
    - Need documentation of explanation of the process in the notes, and also when to use the same

**Limitations:**


**Known Issues:**

#### 3. Geo-replication: Improve gverify.sh logs
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/395
- Release notes:
    - Needs documentation around new log file locations and possibly strings to check for mount failures etc.

**Limitations:**


**Known Issues:**

#### 4. Geo-rep: Cleanup stale (unusable) XSYNC changelogs.
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/376
- Release notes:
    - A note on change in behavior maybe needed

**Limitations:**


**Known Issues:**

#### 5. Improve gsyncd configuration and arguments handling
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/73
- Release notes:
    - Needs user facing documentation for newer options and such
    - There seems to be code improvement as well in the patches, so that may not be needed in the release notes

**Limitations:**


**Known Issues:**

### Standalone
#### 1. Ability to force permissions while creating files/directories on a volume
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/301
- Adds options,
    - "create-mask" and "create-directory-mask"
    - "force-create-mode" and "force-create-directory"
    - End user documentation is required
- Release notes:
    - Need a summary of what the option would enable and how to set the same
    - If relevant user documentation is added, we maybe able to point to the same as well

**Limitations:**


**Known Issues:**

#### 2. Replace MD5 usage to enable FIPS support
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/230
- Need a lot of context on options and versions before which this feature can be used
- Release notes:
    - Need help from authors to complete

**Limitations:**


**Known Issues:**

#### 3. Dentry fop serializer xlator on brick stack
**Notes for users:**
This feature strengthens consistency of the file system, trading it for some
performance and is strongly suggested for workloads where consistency is
required.

For use-cases that involve a large number of renames or frequent creations and
deletions, the meta-data about the files and directories shared across the
clients were not always consistent. They do eventually become consistent, but
a large proportion of applications are not built to handle eventual consistency.

This feature can be enabled as follows,
```
# gluster volume set <volname> features.sdfs enable
```

**Limitations:**
This feature is released as a preview, as performance implications are not known
completely.

#### 4. Add option to disable nftw() based deletes when purging the landfill directory
**Notes for users:**
The gluster brick processes use an optimized manner of deleting entire sub-trees
using the nftw call. With this release, an option is being added to toggle this
behavior in cases where this optimization is not desired.

This is not an exposed option, and needs to be controlled using the volume
graph. Adding the disable-landfill-purge option to the storage/posix translator
helps toggle this feature.

The default is always enabled, as in the older releases.

#### 5. Add option in POSIX to limit hardlinks per inode
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/370
- Release notes:
    - Use github description

**Limitations:**


**Known Issues:**

#### 6. Enhancements for directory listing in readdirp
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/239
- Potentially has some behavioral change in the way rebalance works, needs some documentation efforts there
- Release notes:
    - Note the changes in rebalance behavior, any performance gains?

**Limitations:**


**Known Issues:**

### Developer related
#### 1. xlators should not provide init(), fini() and others directly, but have class_methods
**Notes for developers:**
This release brings in a new unified manner of defining xlator methods. Which
avoids certain unwanted side-effects of the older method (like having to have
certain symbols being defined always), and helps a cleaner single point
registration mechanism for all xlator methods.

The new method, needs just a single symbol in the translator code to be exposed,
which is named xlator_api.

The elements of this structure is defined [here](https://github.com/gluster/glusterfs/blob/release-4.0/libglusterfs/src/xlator.h#L1055) and an example usage of the
same can be seen [here](https://github.com/gluster/glusterfs/commit/5b4b25c697f93d3dfe352013a442f2ec73fd3b8b).

The older mechanism is still supported, but not preferred.

#### 2. Framework for distributed testing
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/374
- Developer README already present
- Release notes:
    - Can be handled by the release team

#### 3. New API for acquiring mandatory locks
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/393
- Code comment for API exists, and release notes can point to that
- Release notes:
    - Can be handled by the release team

#### 4. New on-wire protocol (XDR) needed to support iattx and cleaner dictionary structure
**Notes for developers:**
With changes in the code to adapt to a newer iatt structure, and stricter data
format enforcement within dictionaries passed across the wire, and also as a
part of reducing technical debt around the RPC layer, this release introduces a
new RPC Gluster protocol version (4.0.0).

Typically this does not impact any development, other than to ensure that newer
RPCs that are added would need to be on the 4.0.0 version of the protocol and
dictionaries on the wire need to be better encoded.

The newer iatt structure can be viewed [here](https://github.com/gluster/glusterfs/blob/release-4.0/libglusterfs/src/iatt.h#L47).

An example of better encoding dictionary values for wire transfers can be seen
[here](https://github.com/gluster/glusterfs/blob/release-4.0/libglusterfs/src/dict.c#L2454).

[Here](https://github.com/gluster/glusterfs/blob/master/doc/developer-guide/rpc-for-glusterfs.new-versions.md) is some additional information on Gluster RPC programs for the inquisitive.

#### 5. Leases support on GlusterFS
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/350
- Developer documentation is available in the header file
- Release notes:
    - Can be handled by the release team

#### 6. The protocol xlators should prevent sending binary values in a dict over the networks
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/203
- Release notes:
    - Point to developer documentation if any

#### 7. Translator to handle 'global' options
**Notes for developers:**
GlusterFS process has around 50 command line arguments to itself. While many of
the options are initial settings, many others can change its value in volume
lifetime. Prior to this release there was no way to change a setting, other than
restarting the process for many of these options.

With the introduction of global option translator, it is now possible to handle
these options without restarts.

If contributing code that adds to the process options, strongly consider adding
the same to the global option translator. An example is provided [here](https://review.gluster.org/18316).

## Major issues

**TBD**

## Bugs addressed

Bugs addressed since release-3.13.0 are listed below.

**TBD**