glusterfs.git/xlators/cluster/dht, branch master

DHT - Removing commit hash and 'magical' return value from rebalance

2020-10-07T04:38:50+00:00

The order of operation in rebalance is as follows:
gf_defrag_fix_layout - > gf_defrag_process_dir - > gf_defrag_get_entry

gf_defrag_process_dir is passing to gf_defrag_get_entry a pointer to a
variable 'gf_defrag_get_entry', however this value is ignored (remains
unchanged in the method).
Based on the return value from gf_defrag_get_entry,
gf_defrag_process_dir may change it's return value to the 'magical'
number 2, however since the value of 'should_commit_hash' never changes,
this never happnes.
All of this is propagated back to gf_defrag_fix_layout and is now
removed from there as well.

Change-Id: Ibff297650cf84139bd26c830bfa44f81119b60d4
updates: #1002
Signed-off-by: Barak Sason Rofman

dht - Remove "tier" code (part 2)

2020-10-01T14:50:15+00:00

Part 1 of this patch https://review.gluster.org/#/c/glusterfs/+/24328/

Following part 1, this patch complety removes all traces of
"tier" feature in dht.

This is based in the work done in
https://review.gluster.org/#/c/glusterfs/+/23935/

Change-Id: I7fba1ab7249719301ca578b4a6f4acac748da145
updates: #1097
Signed-off-by: Barak Sason Rofman

DHT - Fixing memory allocation crash

2020-10-01T04:01:35+00:00

The allocation
dir_dfmeta = GF_CALLOC(1, sizeof(*dir_dfmeta), gf_common_mt_pointer);
seems to cause a crash.
From the logs:
[2020-09-24 13:10:13.225935 +0000] I [dht-rebalance.c:3273:gf_defrag_process_dir]
 0-dist-dht: migrate data called on /dir1
[2020-09-24 13:10:13.226587 +0000] E [mem-pool.c:61:gf_mem_set_acct_info]
(-->/usr/local/lib/glusterfs/9dev/xlator/cluster/distribute.so(+0x18e60)
[0x7f4b1f71ee60] -->/usr/local/lib/glusterfs/9dev/xlator/cluster/distribute.so(+0x173ab)
[0x7f4b1f71d3ab] -->/usr/local/lib/libglusterfs.so.0(+0x4d8e5) [0x7f4b357668e5] )
0-: Assertion failed: type <= mem_acct->num_types
[2020-09-24 13:10:13.226623 +0000] E [mem-pool.c:61:gf_mem_set_acct_info]
(-->/usr/local/lib/glusterfs/9dev/xlator/cluster/distribute.so(+0x18e60)
[0x7f4b1f71ee60] -->/usr/local/lib/glusterfs/9dev/xlator/cluster/distribute.so(+0x173d3)
[0x7f4b1f71d3d3] -->/usr/local/lib/libglusterfs.so.0(+0x4d8e5) [0x7f4b357668e5] )
0-: Assertion failed: type <= mem_acct->num_types

The following change fixes that crash.

fixes: #1511
Change-Id: Ibf605648981f7108e863c91a80370cf077ad7c4a
Signed-off-by: Barak Sason Rofman

dht: explicit null dereference

2020-09-29T14:32:09+00:00

Added a null check for uuid_list_copy, to avoid
null dereference in strtok_r() in case of strdup()
failure.

CID: 1325612
CID: 1274223

Updates: #1060

Change-Id: I641a5068cd76d7b2ed92eccf39e7f97d6f7b2480
Signed-off-by: nik-redhat

dht: Ongoing IO is failing on non-distribute volumes after just add-brick

2020-09-28T03:55:41+00:00

Problem: On a non-distributed volumes linux kernel untar is failed
         after running add-brick operation

Solution:
1) Save hashed subvol as a MDS in case while MDS has not been
   populated

Fixes: #1328
Change-Id: I9967e136da008c6367973a7346637617dfa8f934
Signed-off-by: Mohit Agrawal

DHT - fixing a bug in thread creation

2020-09-24T13:29:56+00:00

The 3rd argument of gf_thread_create is a pointer to a method, however
in this case, a pointer to a pointer to a method is passed (2 levels of
indirection instead of 1).

Change-Id: Ic2d4ea75aa54c6bc85a80bd0277a0efa5e5814ad
updates: #1002
Signed-off-by: Barak Sason Rofman

Revert "dht: optimize rebalance crawl path"

2020-09-03T01:53:40+00:00

Based on the discussion on the issue, it is decided that it is
better to not have this implementation of the feature.

This reverts commit 3af9443c770837abe4f54db399623380ab9767a7.

Change-Id: I4e3bf18fc376cdb0cf29f1d98a915deca17c3496
Updates: #1422
Signed-off-by: Pranith Kumar K

cluster/dht: simplify and cleanup internal time management

2020-08-21T14:05:26+00:00

Prefer time_t and gf_time() over 'struct timeval' and gettimeofday()
where microseconds are not really used, drop unneeded 'struct timeval'
to 'struct timespec' conversion in dht_file_counter_thread().

Change-Id: Ibd802f79b8848df3f6175ca1fd82e93532bba38d
Signed-off-by: Dmitry Antipov 
Updates: #1002

libglusterfs: add functions to calculate time difference

2020-08-21T03:56:52+00:00

Add gf_tvdiff() and gf_tsdiff() to calculate the difference
between 'struct timeval' and 'struct timespec' values, use
them where appropriate.

Change-Id: I172be06ee84e99a1da76847c15e5ea3fbc059338
Signed-off-by: Dmitry Antipov 
Updates: #1002

dht: optimize rebalance crawl path

2020-07-31T16:22:23+00:00

For distribute only volumes we can use the information for
local subvolumes to avoid syncop calls which goes through the
whole stack to fetch stat and entries.

A separate function gf_defrag_fix_layout_puredist is introduced.

TODO: A glusterd flag needs to be introduced in case we want to
fall back to run the old way.

Perf numbers:
DirSize - 1Million	Old	New	%diff
Depth 	- 100 (Run 1)	353	74	+377%
Depth 	- 100 (Run 2)	348	72	+377~%
Depth 	- 50		246	122	+100%
Depth 	- 3		174	114	+52%

Change-Id: I67cc136cebd34092fd775e69f74c2d5b33d3156d
Fixes: #1242
Signed-off-by: Susant Palai