glusterfs.git/extras/quota, branch master

quota_fsck.py fails with UnicodeDecodeError

2020-09-16T03:31:53+00:00

Issue: While decoding the byte characters the quota_fsck
script stumbled across a corner case wherein the file names
given by the getfattr dump will cause the decoding to UTF-8
to fail with UnicodeDecodeError.

Code Change:
On looking through the quota_fsck.py script, it seems like
the file path is actually not needed when decoding for the
xattr parsing, hence the code change reflects that. Also,
removed a comparison which previously existed to skip the
file names as that won't be required now.

Fixes: #1487
Change-Id: I8a13ab07be6c9cfafae996f17764fbb4a285bd8c
Signed-off-by: srijan-sivakumar

Quota quota_fsck.py, converting byte string to string

2020-08-19T09:00:56+00:00

Issue: The quota_fsck.py script throws an TypeError
due to the fact that the data is read as bytes and then
the string operations are applied on the. Now, in python3
string is unicode and hence we get the type error.

Code Changes:
Decoding the bytes value into utf-8 format.

Change-Id: Ia1ff52a821d664a371c8166692ff506ae39f6e40
Signed-off-by: srijan-sivakumar 
Fixes: #1401

scripts: quota_fsck script TypeError: %d format:not dict

2019-11-06T13:11:21+00:00

Problem: One of the prints in the script have been using
%i as the format for printing which doesn't work.

Fix: use %s as the format in the place of %i

Fixes: bz#1764129
Change-Id: I4480ede7bf62906ddedbe5f880a1e89c76946641
Signed-off-by: Hari Gowtham

Scripts: quota_fsck script KeyError: 'contri_size'

2019-10-22T23:55:48+00:00

Problem: In a certain code flow, we weren't handling the
unavailability of the contri value in the dict. Trying to print
without the value resulted in erroring out.

Fix: Have printed the whole of dictionary as the values will be
helpful in understanding the state of the file/dir

Fixes: bz#1764129
Change-Id: I99c538adb712f281ca10e4e0088f404f515b9725
Signed-off-by: hari gowtham

misc: fix misc. shebangs

2018-09-11T04:25:17+00:00

* One #!/usr/bin/env python and three #!/usr/bin/python were overlooked
  in all the other python fixups. Ugh.

* Two new python files missed the memo about #!/usr/bin/python3.

* One #!/usr/bin/env bash.
  Various distribution packaging policies have strong wording about
  the use of #!/usr/bin/env ...

  Note: this patch does not change the use of #!/usr/bin/env bash in
  the two files extras/{clang-checker.sh,check_goto.pl} as these are
  not included in any packages. (Although I'm not actually sure why
  anyone would ever use '/usr/bin/env {sh,bash}' as I'm not aware of
  any version-specific differences like there are with, e.g., python.)

* One #!/usr/bin/bash.
  On Fedora and CentOS > 6, /bin is a symlink to /usr/bin, so it
  makes little difference. But Debian & Ubuntu still have separate
  /bin and /usr/bin; and sh and bash are in /bin, not /usr/bin.
  (Historically, in BSD and SYSV Unix it was /bin/sh.)

  Note: Fedora and CentOS package build runs a script that converts
  all /bin/sh and /bin/bash to /usr/bin/sh and /usr/bin/bash.

Change-Id: I9171265829af78dd0cd7622c22b56d22179ff8a3
updates: bz#1193929
Signed-off-by: Kaleb S. KEITHLEY

core: python3

2018-09-03T09:14:44+00:00

see https://review.gluster.org/#/c/19788/,
    https://review.gluster.org/#/c/19871/,
    https://review.gluster.org/#/c/19952/,
    https://review.gluster.org/#/c/20104/,
    https://review.gluster.org/#/c/20162/,
    https://review.gluster.org/#/c/20185/,
    https://review.gluster.org/#/c/20207/,
    https://review.gluster.org/#/c/20227/,
    https://review.gluster.org/#/c/20307/,
    https://review.gluster.org/#/c/20320/,
    https://review.gluster.org/#/c/20332/,
    https://review.gluster.org/#/c/20364/,
    https://review.gluster.org/#/c/20441/, and
    https://review.gluster.org/#/c/20484

shebangs changed from /usr/bin/python2 to /usr/bin/python3.
(Reminder, various distribution packaging guidelines require use
of explicit python version and don't allow '#!/usr/bin/env python',
regardless of how handy that idiom may be.)

glusterfs.spec(.in) package python{2,3}-gluster and python2 or
python3 dependencies as appropriate.

configure(.ac):
+ test for and use python2 or python3 as appropriate. If build
  machine has python2 and python3, use python3. Override by
  setting PYTHON=/usr/bin/python2 when running configure.
+ PYTHONDEV_CPPFLAGS from python[23]-config --includes is a
  better match to the original python sysconfig.get_python_inc().
  All those other extraneous flags breaks the build.
+ Only change the shebangs once. Changing them over and over
  again, e.g., during a `make glusterrpms` in extras/LinuxRPM
  just sends make (is it really make that's looping?) into an
  infinite loop. If you figure out why, let me know.
+ Oldest python2 is python2.6 on CentOS 6 and Debian 8 (Jessie).
  Everything else has 2.7 or 3.x
+ logic from https://review.gluster.org/c/glusterfs/+/21050, which
  needs to be removed/merged after that patch is merged.

Builds on CentOS 6, CentOS 7, Fedora 28, Fedora rawhide, and the
mysterious RHEL > 7.

Change-Id: Idae21d3b6f58b32372e1daa0d234e491e563198f
updates: #411
Signed-off-by: Kaleb S. KEITHLEY

All: run codespell on the code and fix issues.

2018-07-22T14:40:16+00:00

Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...

Only compile-tested!

Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul

core/various: python3 compat, prepare for python2 -> python3

2018-06-21T10:09:25+00:00

see https://review.gluster.org/#/c/19788/,
    https://review.gluster.org/#/c/19871/,
    https://review.gluster.org/#/c/19952/,
    https://review.gluster.org/#/c/20104/,
    https://review.gluster.org/#/c/20162/,
    https://review.gluster.org/#/c/20185/,
    https://review.gluster.org/#/c/20207/,
    https://review.gluster.org/#/c/20227/,
    https://review.gluster.org/#/c/20307/, and
    https://review.gluster.org/#/c/20320/

This patch fixes more selected comma white space (ws_comma) as suggested
by the 2to3 utility. (Earlier attempts to fix all ws_comma in one
patch did not pass centos regression, hence multiple patches to identify
the change that causes the failure.)

Note: Fedora packaging guidelines and SUSE rpmlint require explicit
shebangs; popular practices like #!/usr/bin/env python and #!/usr/bin/python
are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3

Note: Selected small fixes from 2to3 utility. Specifically apply,
basestring, funcattrs, has_key, idioms, map, numliterals, raise,
set_literal, types, urllib, and zip have already been applied. Also
version agnostic imports for urllib, cpickle, socketserver, _thread,
queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1

Note: these 2to3 fixes report no changes are necessary: asserts, buffer,
exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern,
itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren,
raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw,
tuple_params, xreadlines.

Change-Id: I6e5a2408fa1fc81e00e66d6e4a7f9f6fa1d1ed15
updates: #411
Signed-off-by: Kaleb S. KEITHLEY

core/various: python3 compat, prepare for python2 -> python3

2018-06-07T16:37:43+00:00

see https://review.gluster.org/#/c/19788/,
    https://review.gluster.org/#/c/19871/,
    https://review.gluster.org/#/c/19952/, and
    https://review.gluster.org/#/c/20104/
    https://review.gluster.org/#/c/20162/

This patch changes uses of map() and raise(), and a few cases of print()
that were overlooked in the prior patch that fixed print.

Note: Fedora packaging guidelines require explicit shebangs, so popular
practices like #!/usr/bin/env python and #!/usr/bin/python are not
allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3

Note: Selected small fixes from 2to3 utility. Specifically apply,
basestring, funcattrs, idioms, numliterals, set_literal, types, urllib,
zip, map, and raise have already been applied. Also version agnostic
imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested
 by Aravinda in https://review.gluster.org/#/c/19767/1

Note: these 2to3 fixes report no changes are necessary: asserts, buffer,
exec, execfile, exitfunc, filter, getcwdu, intern, itertools, metaclass,
methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce,
reload, renames, repr, standarderror, sys_exc, throw, tuple_params,
xreadlines.

Change-Id: Id62ea491e4ab5dd390075c5c6d9d889cf6f9da27
updates: #411
Signed-off-by: Kaleb S. KEITHLEY

Quota fsck script

2018-05-29T04:41:40+00:00

script can be used to for 3 purposes
1) Extract marker xattr from the  backend with a crawl (for analysis)
2) Report anomalies in accounting by comparing stat based accounting
with marker translator's accounting
3) Fix anomalies in the FS by marking the anamolous directories dirty
and having them heal in a bottom up fashion over the directory tree.

Change-Id: Ib4d4a6d1886d7332c80d5a93f7afccaa0f52ab58
fixes: #390
Signed-off-by: Sanoj Unnikrishnan 
Signed-off-by: Hari Gowtham