glusterfs.git/tests/bugs/bug-1053579.t, branch release-7

tests: move all test-cases into component subdirectories

2015-01-06T11:24:24+00:00

There are around 300 regression tests, 250 being in tests/bugs. Running
partial set of tests/bugs is not easy because this is a flat directory
with almost all tests inside.

It would be valuable to make partial test/bugs easier, and allow the use
of mulitple build hosts for a single commit, each running a subset of
the tests for a quicker result.

Additional changes made:
- correct the include path for *.rc shell libraries and *.py utils
- make the testcases pass checkpatch
- arequal-checksum in afr/self-heal.t was never executed, now it is
- include.rc now complains loudly if it fails to find env.rc

Change-Id: I26ffd067e9853d3be1fd63b2f37d8aa0fd1b4fea
BUG: 1178685
Reported-by: Emmanuel Dreyfus 
Reported-by: Atin Mukherjee 
URL: http://www.gluster.org/pipermail/gluster-devel/2014-December/043414.html
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/9353
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Emmanuel Dreyfus 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Regression test portability: batch of bugs (volume 1)

2014-11-08T15:07:47+00:00

Fix various regression test portability in tests/bugs.

bug-1004744.t:
- Slower systems really requires an increased REBALANCE_TIMEOUT in include.rc

bug-1023974.t:
- use the -p option to mkdir before the path for portability sake.

bug-1051896.t and bug-847622.t:
- skip ACL test for NetBSD as it has no POSIX 1e ACL support.

bugs-1053579.t:
- Override the amount of secondary groups depending of the maximum
  the system supports.
- Specify seq(1) format to avoid having trailing .00
- Use more portable su -m USER -c COMMAND (also in bug-884597.t)
- NetBSD does not fail high GID with NFS

bug-1058663.c:
- Depending on architecture, NetBSD can have SIGSEGV instead of SIGBUS

bug-762989.t and bug-867253.t:
- For non Linux systems, skip tests on features that are Linux-specific

bug-765473.t;
- Fix a {1} that bash is supposed to expand in 1 but does not, replace by 1.
- Sync volume to make sure it imediatly knows a fd got bad

BUG: 1129939
Change-Id: I5405f94ccb8f20d35b3095096b0602c43719a1ae
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/9009
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Vijay Bellur

tests: Change umount with force_umount with 5 retries

2014-06-23T10:13:28+00:00

Change-Id: I0e2dbdfd34080328dfa6b4eebef0366f2b0fcb04
BUG: 1092850
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8117
Tested-by: Justin Clift 
Reviewed-by: Harshavardhana 
Tested-by: Harshavardhana 
Reviewed-by: Sachin Pandit 
Reviewed-by: Prashanth Pai 
Reviewed-by: Vijay Bellur

tests: Use uniform timeouts

2014-05-20T04:00:46+00:00

Change-Id: I479ab941b3b2da3b16f624400fbd300f08326268
BUG: 1092850
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/7799
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

tests: Wait for nfs export to be available

2014-05-16T10:28:16+00:00

Change-Id: I59a5e0cb78f2b670761a65272b8ab1d7bdb3668a
BUG: 1092850
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/7773
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

rpc: implement server.manage-gids for group resolving on the bricks

2014-05-09T19:22:39+00:00

The new volume option 'server.manage-gids' can be enabled in
environments where a user belongs to more than the current absolute
maximum of 93 groups. This option triggers the following behavior:

1. The AUTH_GLUSTERFS structure sent by GlusterFS clients (fuse, nfs or
   libgfapi) will contain only one (1) auxiliary group, instead of
   a full list. This reduces network usage and prevents problems in
   encoding the AUTH_GLUSTERFS structure which should fit in 400 bytes.
2. The single group in the RPC Calls received by the server is replaced
   by resolving the groups server-side. Permission checks and similar in
   lower xlators are applied against the full list of groups where the
   user belongs to, and not the single auxiliary group that the client
   sent.

Change-Id: I9e540de13e3022f8b63ff893ecba511129a47b91
BUG: 1053579
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/7501
Tested-by: Gluster Build System 
Reviewed-by: Santosh Pradhan 
Reviewed-by: Harshavardhana 
Reviewed-by: Anand Avati

rpc: warn and truncate grouplist if RPC/AUTH can not hold everything

2014-04-08T17:50:52+00:00

The GlusterFS protocol currently uses AUTH_GLUSTERFS_V2 in the RPC/AUTH
header. This header contains the uid, gid and auxiliary groups of the
user/process that accesses the Gluster Volume.

The AUTH_GLUSTERFS_V2 structure allows up to 65535 auxiliary groups to
be passed on. Unfortunately, the RPC/AUTH header is limited to 400 bytes
by the RPC specification: http://tools.ietf.org/html/rfc5531#section-8.2

In order to not cause complete failures on the client-side when trying
to encode a AUTH_GLUSTERFS_V2 that would result in more than 400 bytes,
we can calculate the expected size of the other elements:

    1 | pid
    1 | uid
    1 | gid
    1 | groups_len
   XX | groups_val (GF_MAX_AUX_GROUPS=65535)
    1 | lk_owner_len
   YY | lk_owner_val (GF_MAX_LOCK_OWNER_LEN=1024)
  ----+-------------------------------------------
    5 | total xdr-units

  one XDR-unit is defined as BYTES_PER_XDR_UNIT = 4 bytes
  MAX_AUTH_BYTES = 400 is the maximum, this is 100 xdr-units.
  XX + YY can be 95 to fill the 100 xdr-units.

  Note that the on-wire protocol has tighter requirements than the
  internal structures. It is possible for xlators to use more groups and
  a bigger lk_owner than that can be sent by a GlusterFS-client.

This change prevents overflows when allocating the RPC/AUTH header. Two
new macros are introduced to calculate the number of groups that fit in
the RPC/AUTH header, when taking the size of the lk_owner in account. In
case the list of groups exceeds the maximum possible, only the first
groups are passed over the RPC/GlusterFS protocol to the bricks.
A warning is added to the logs, so that most system administrators will
get informed.

The reducing of the number of groups is not a new inventions. The
RPC/AUTH header (AUTH_SYS or AUTH_UNIX) that NFS uses has a limit of 16
groups. Most, if not all, NFS-clients will reduce any bigger number of
groups to 16. (nfs.server-aux-gids can be used to workaround the limit
of 16 groups, but the Gluster NFS-server will be limited to a maximum of
93 groups, or fewer in case the lk_owner structure contains more items.)

Change-Id: I8410e59d0fd246d601b54b961d3ae9cb5a858c10
BUG: 1053579
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/7202
Tested-by: Gluster Build System 
Reviewed-by: Harshavardhana 
Reviewed-by: Santosh Pradhan 
Reviewed-by: Vijay Bellur