<feed xmlns='http://www.w3.org/2005/Atom'>
<title>glusterfs.git/xlators/performance, branch v8.0rc0</title>
<subtitle></subtitle>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/'/>
<entry>
<title>md-cache: fix several NULL dereferences</title>
<updated>2020-05-31T05:56:08+00:00</updated>
<author>
<name>Xavi Hernandez</name>
<email>xhernandez@redhat.com</email>
</author>
<published>2020-04-17T10:10:05+00:00</published>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/commit/?id=18bd1bdaa6ea5d589b21865769d6183e4e201006'/>
<id>18bd1bdaa6ea5d589b21865769d6183e4e201006</id>
<content type='text'>
This patch includes the following CID from Coverity Scan:

  * 1425196
  * 1425197
  * 1425198
  * 1425199
  * 1525200

Change-Id: Iddcfea449d3dd56d4dfcc39f4c3c608518e611e4
Signed-off-by: Xavi Hernandez &lt;xhernandez@redhat.com&gt;
Updates: #1060
(cherry picked from commit b53ba17dbfd2d18c10e2c308b8899d36726ab440)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This patch includes the following CID from Coverity Scan:

  * 1425196
  * 1425197
  * 1425198
  * 1425199
  * 1525200

Change-Id: Iddcfea449d3dd56d4dfcc39f4c3c608518e611e4
Signed-off-by: Xavi Hernandez &lt;xhernandez@redhat.com&gt;
Updates: #1060
(cherry picked from commit b53ba17dbfd2d18c10e2c308b8899d36726ab440)
</pre>
</div>
</content>
</entry>
<entry>
<title>md-cache: avoid clearing cache when not necessary</title>
<updated>2020-04-16T11:15:09+00:00</updated>
<author>
<name>Xavi Hernandez</name>
<email>xhernandez@redhat.com</email>
</author>
<published>2020-03-30T09:09:39+00:00</published>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/commit/?id=f507c20bd11565c329d103075a15da11dba461d5'/>
<id>f507c20bd11565c329d103075a15da11dba461d5</id>
<content type='text'>
mdc_inode_xatt_set() blindly cleared current cache when dict was not
NULL, even if there was no xattr requested.

This patch fixes this by only calling mdc_inode_xatt_set() when we have
explicitly requested something to cache.

Change-Id: Idc91a4693f1ff39f7059acde26682ccc361b947d
Fixes: #1140
Signed-off-by: Xavi Hernandez &lt;xhernandez@redhat.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
mdc_inode_xatt_set() blindly cleared current cache when dict was not
NULL, even if there was no xattr requested.

This patch fixes this by only calling mdc_inode_xatt_set() when we have
explicitly requested something to cache.

Change-Id: Idc91a4693f1ff39f7059acde26682ccc361b947d
Fixes: #1140
Signed-off-by: Xavi Hernandez &lt;xhernandez@redhat.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>write-behind: fix data corruption</title>
<updated>2020-04-03T14:28:33+00:00</updated>
<author>
<name>Xavi Hernandez</name>
<email>xhernandez@redhat.com</email>
</author>
<published>2020-03-27T22:56:15+00:00</published>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/commit/?id=77889c34b6f169dfc3257e2b566afa2f4bad1697'/>
<id>77889c34b6f169dfc3257e2b566afa2f4bad1697</id>
<content type='text'>
There was a bug in write-behind that allowed a previous completed write
to overwrite the overlapping region of data from a future write.

Suppose we want to send three writes (W1, W2 and W3). W1 and W2 are
sequential, and W3 writes at the same offset of W2:

    W2.offset = W3.offset = W1.offset + W1.size

Both W1 and W2 are sent in parallel. W3 is only sent after W2 completes.
So W3 should *always* overwrite the overlapping part of W2.

Suppose write-behind processes the requests from 2 concurrent threads:

    Thread 1                    Thread 2

    &lt;received W1&gt;
                                &lt;received W2&gt;
    wb_enqueue_tempted(W1)
    /* W1 is assigned gen X */
                                wb_enqueue_tempted(W2)
                                /* W2 is assigned gen X */

                                wb_process_queue()
                                  __wb_preprocess_winds()
                                    /* W1 and W2 are sequential and all
                                     * other requisites are met to merge
                                     * both requests. */
                                    __wb_collapse_small_writes(W1, W2)
                                    __wb_fulfill_request(W2)

                                  __wb_pick_unwinds() -&gt; W2
                                  /* In this case, since the request is
                                   * already fulfilled, wb_inode-&gt;gen
                                   * is not updated. */

                                wb_do_unwinds()
                                  STACK_UNWIND(W2)

                                /* The application has received the
                                 * result of W2, so it can send W3. */
                                &lt;received W3&gt;

                                wb_enqueue_tempted(W3)
                                /* W3 is assigned gen X */

                                wb_process_queue()
                                  /* Here we have W1 (which contains
                                   * the conflicting W2) and W3 with
                                   * same gen, so they are interpreted
                                   * as concurrent writes that do not
                                   * conflict. */
                                  __wb_pick_winds() -&gt; W3

                                wb_do_winds()
                                  STACK_WIND(W3)

    wb_process_queue()
      /* Eventually W1 will be
       * ready to be sent */
      __wb_pick_winds() -&gt; W1
      __wb_pick_unwinds() -&gt; W1
        /* Here wb_inode-&gt;gen is
         * incremented. */

    wb_do_unwinds()
      STACK_UNWIND(W1)

    wb_do_winds()
      STACK_WIND(W1)

So, as we can see, W3 is sent before W1, which shouldn't happen.

The problem is that wb_inode-&gt;gen is only incremented for requests that
have not been fulfilled but, after a merge, the request is marked as
fulfilled even though it has not been sent to the brick. This allows
that future requests are assigned to the same generation, which could
be internally reordered.

Solution:

Increment wb_inode-&gt;gen before any unwind, even if it's for a fulfilled
request.

Special thanks to Stefan Ring for writing a reproducer that has been
crucial to identify the issue.

Change-Id: Id4ab0f294a09aca9a863ecaeef8856474662ab45
Signed-off-by: Xavi Hernandez &lt;xhernandez@redhat.com&gt;
Fixes: #884
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
There was a bug in write-behind that allowed a previous completed write
to overwrite the overlapping region of data from a future write.

Suppose we want to send three writes (W1, W2 and W3). W1 and W2 are
sequential, and W3 writes at the same offset of W2:

    W2.offset = W3.offset = W1.offset + W1.size

Both W1 and W2 are sent in parallel. W3 is only sent after W2 completes.
So W3 should *always* overwrite the overlapping part of W2.

Suppose write-behind processes the requests from 2 concurrent threads:

    Thread 1                    Thread 2

    &lt;received W1&gt;
                                &lt;received W2&gt;
    wb_enqueue_tempted(W1)
    /* W1 is assigned gen X */
                                wb_enqueue_tempted(W2)
                                /* W2 is assigned gen X */

                                wb_process_queue()
                                  __wb_preprocess_winds()
                                    /* W1 and W2 are sequential and all
                                     * other requisites are met to merge
                                     * both requests. */
                                    __wb_collapse_small_writes(W1, W2)
                                    __wb_fulfill_request(W2)

                                  __wb_pick_unwinds() -&gt; W2
                                  /* In this case, since the request is
                                   * already fulfilled, wb_inode-&gt;gen
                                   * is not updated. */

                                wb_do_unwinds()
                                  STACK_UNWIND(W2)

                                /* The application has received the
                                 * result of W2, so it can send W3. */
                                &lt;received W3&gt;

                                wb_enqueue_tempted(W3)
                                /* W3 is assigned gen X */

                                wb_process_queue()
                                  /* Here we have W1 (which contains
                                   * the conflicting W2) and W3 with
                                   * same gen, so they are interpreted
                                   * as concurrent writes that do not
                                   * conflict. */
                                  __wb_pick_winds() -&gt; W3

                                wb_do_winds()
                                  STACK_WIND(W3)

    wb_process_queue()
      /* Eventually W1 will be
       * ready to be sent */
      __wb_pick_winds() -&gt; W1
      __wb_pick_unwinds() -&gt; W1
        /* Here wb_inode-&gt;gen is
         * incremented. */

    wb_do_unwinds()
      STACK_UNWIND(W1)

    wb_do_winds()
      STACK_WIND(W1)

So, as we can see, W3 is sent before W1, which shouldn't happen.

The problem is that wb_inode-&gt;gen is only incremented for requests that
have not been fulfilled but, after a merge, the request is marked as
fulfilled even though it has not been sent to the brick. This allows
that future requests are assigned to the same generation, which could
be internally reordered.

Solution:

Increment wb_inode-&gt;gen before any unwind, even if it's for a fulfilled
request.

Special thanks to Stefan Ring for writing a reproducer that has been
crucial to identify the issue.

Change-Id: Id4ab0f294a09aca9a863ecaeef8856474662ab45
Signed-off-by: Xavi Hernandez &lt;xhernandez@redhat.com&gt;
Fixes: #884
</pre>
</div>
</content>
</entry>
<entry>
<title>open-behind: fix missing fd reference</title>
<updated>2020-03-17T04:23:30+00:00</updated>
<author>
<name>Xavi Hernandez</name>
<email>xhernandez@redhat.com</email>
</author>
<published>2020-03-08T17:36:45+00:00</published>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/commit/?id=b30f76980fc5d01b569d9409b68d24d809444ff8'/>
<id>b30f76980fc5d01b569d9409b68d24d809444ff8</id>
<content type='text'>
Open behind was not keeping any reference on fd's pending to be
opened. This makes it possible that a concurrent close and en entry
fop (unlink, rename, ...) caused destruction of the fd while it
was still being used.

Change-Id: Ie9e992902cf2cd7be4af1f8b4e57af9bd6afd8e9
Fixes: bz#1810934
Signed-off-by: Xavi Hernandez &lt;xhernandez@redhat.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Open behind was not keeping any reference on fd's pending to be
opened. This makes it possible that a concurrent close and en entry
fop (unlink, rename, ...) caused destruction of the fd while it
was still being used.

Change-Id: Ie9e992902cf2cd7be4af1f8b4e57af9bd6afd8e9
Fixes: bz#1810934
Signed-off-by: Xavi Hernandez &lt;xhernandez@redhat.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>multiple: fix bad type cast</title>
<updated>2020-01-10T00:58:19+00:00</updated>
<author>
<name>Xavi Hernandez</name>
<email>xhernandez@redhat.com</email>
</author>
<published>2019-12-20T13:14:32+00:00</published>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/commit/?id=00c090b093c147a95bfb8fce93f08303993e1995'/>
<id>00c090b093c147a95bfb8fce93f08303993e1995</id>
<content type='text'>
When using inode_ctx_get() or inode_ctx_set(), a 'uint64_t *' is expected.
In many cases, the value to retrieve or store is a pointer, which will be
of smaller size in some architectures (for example 32-bits). In this case,
directly passing the address of the pointer casted to an 'uint64_t *' is
wrong and can cause memory corruption.

Change-Id: Iae616da9dda528df6743fa2f65ae5cff5ad23258
Signed-off-by: Xavi Hernandez &lt;xhernandez@redhat.com&gt;
Fixes: bz#1785611
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When using inode_ctx_get() or inode_ctx_set(), a 'uint64_t *' is expected.
In many cases, the value to retrieve or store is a pointer, which will be
of smaller size in some architectures (for example 32-bits). In this case,
directly passing the address of the pointer casted to an 'uint64_t *' is
wrong and can cause memory corruption.

Change-Id: Iae616da9dda528df6743fa2f65ae5cff5ad23258
Signed-off-by: Xavi Hernandez &lt;xhernandez@redhat.com&gt;
Fixes: bz#1785611
</pre>
</div>
</content>
</entry>
<entry>
<title>xlator/performance/io-threads: structure logging</title>
<updated>2020-01-06T08:40:52+00:00</updated>
<author>
<name>yatipadia</name>
<email>ypadia@redhat.com</email>
</author>
<published>2019-12-17T07:25:02+00:00</published>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/commit/?id=26fe6106762f6aea910d477d1c37e645797884e1'/>
<id>26fe6106762f6aea910d477d1c37e645797884e1</id>
<content type='text'>
convert gf_msg() to gf_smsg()

Change-Id: I35c6f62c346a75ecb22cd3a4346ad4dc48f09a91
Updates: #657
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
convert gf_msg() to gf_smsg()

Change-Id: I35c6f62c346a75ecb22cd3a4346ad4dc48f09a91
Updates: #657
</pre>
</div>
</content>
</entry>
<entry>
<title>xlator/io-cache: structure logging</title>
<updated>2020-01-06T08:39:53+00:00</updated>
<author>
<name>yatipadia</name>
<email>ypadia@redhat.com</email>
</author>
<published>2019-12-18T11:55:51+00:00</published>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/commit/?id=7f98c105a3a8e2a9f4e9c75451a94115e270405e'/>
<id>7f98c105a3a8e2a9f4e9c75451a94115e270405e</id>
<content type='text'>
Convert all gf_msg() to gf_smsg()

Updates: #657

Change-Id: I72215b2518df78174dda8a7bc8de6f21fe1ba10f
Signed-off-by: yatipadia &lt;ypadia@redhat.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Convert all gf_msg() to gf_smsg()

Updates: #657

Change-Id: I72215b2518df78174dda8a7bc8de6f21fe1ba10f
Signed-off-by: yatipadia &lt;ypadia@redhat.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>md-cache.c: move cache-swift-metadata to off by default</title>
<updated>2019-12-20T12:30:39+00:00</updated>
<author>
<name>Yaniv Kaul</name>
<email>ykaul@redhat.com</email>
</author>
<published>2019-12-03T13:28:13+00:00</published>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/commit/?id=e879ff1d4a3a0d795da2575406414eaeb28975f4'/>
<id>e879ff1d4a3a0d795da2575406414eaeb28975f4</id>
<content type='text'>
This causes mdc_xattr_list_populate() NOT to add "user.swift.metadata"
as an xattr in the list of attrs we look at in some paths of the code.

This is documented @
https://github.com/gluster/glusterfs/issues/775

Change-Id: Ie3d676c74a2f333beeacc302e253efe9f9942d1a
updates: bz#1193929
Signed-off-by: Yaniv Kaul &lt;ykaul@redhat.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This causes mdc_xattr_list_populate() NOT to add "user.swift.metadata"
as an xattr in the list of attrs we look at in some paths of the code.

This is documented @
https://github.com/gluster/glusterfs/issues/775

Change-Id: Ie3d676c74a2f333beeacc302e253efe9f9942d1a
updates: bz#1193929
Signed-off-by: Yaniv Kaul &lt;ykaul@redhat.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>To fix readdir-ahead memory leak</title>
<updated>2019-12-10T05:00:54+00:00</updated>
<author>
<name>HuangShujun</name>
<email>549702281@qq.com</email>
</author>
<published>2019-12-05T08:07:10+00:00</published>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/commit/?id=99044a5cedcff9a9eec40a07ecb32bd66271cd02'/>
<id>99044a5cedcff9a9eec40a07ecb32bd66271cd02</id>
<content type='text'>
Glusterfs client process has memory leak if create serveral files under one folder, and delete the folder.
According to statedump, the ref counts of readdir-ahead is bigger than zero in the inode table. Readdir-ahead get parent inode by inode_parent in rda_mark_inode_dirty when each rda_writev_cbk,the inode ref count of parent folder will be increased in inode_parent, but readdir-ahead do not unref it later.
The correction is unref the parent inode at the end of rda_mark_inode_dirty

Fixes: bz#1779055
Signed-off-by: HuangShujun &lt;549702281@qq.com&gt;

Change-Id: Iee68ab1089cbc2fbc4185b93720fb1f66ee89524
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Glusterfs client process has memory leak if create serveral files under one folder, and delete the folder.
According to statedump, the ref counts of readdir-ahead is bigger than zero in the inode table. Readdir-ahead get parent inode by inode_parent in rda_mark_inode_dirty when each rda_writev_cbk,the inode ref count of parent folder will be increased in inode_parent, but readdir-ahead do not unref it later.
The correction is unref the parent inode at the end of rda_mark_inode_dirty

Fixes: bz#1779055
Signed-off-by: HuangShujun &lt;549702281@qq.com&gt;

Change-Id: Iee68ab1089cbc2fbc4185b93720fb1f66ee89524
</pre>
</div>
</content>
</entry>
<entry>
<title>performance/open-behind: seek fop should open_and_resume</title>
<updated>2019-10-11T06:05:07+00:00</updated>
<author>
<name>Pranith Kumar K</name>
<email>pkarampu@redhat.com</email>
</author>
<published>2019-10-10T05:20:59+00:00</published>
<link rel='alternate' type='text/html' href='http://dev.gluster.org/cgit/glusterfs.git/commit/?id=9dd57ff963d4d0dde7f3ad0ff230837ff2f4b1df'/>
<id>9dd57ff963d4d0dde7f3ad0ff230837ff2f4b1df</id>
<content type='text'>
fixes: bz#1760187
Change-Id: I4c6ad13194d4fc5c7705e35bf9a27fce504b51f9
Signed-off-by: Pranith Kumar K &lt;pkarampu@redhat.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
fixes: bz#1760187
Change-Id: I4c6ad13194d4fc5c7705e35bf9a27fce504b51f9
Signed-off-by: Pranith Kumar K &lt;pkarampu@redhat.com&gt;
</pre>
</div>
</content>
</entry>
</feed>
