Skip to contentSkip to conversation

iotests: Remove 030 from the auto group

PATCH Applied Thomas Huth 4 participants 10 messages Sep 4, 2020 – Sep 24, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iotests: Remove 030 from the auto group

Test 030 is still occasionally failing in the CI ... so for the time being, let's disable it in the "auto" group. We can add it back once it got more stable.

---
I just saw the problem here:
https://cirrus-ci.com/task/5449330930745344?command=main#L6482
and Peter hit it a couple of weeks ago:
https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00136.html

 tests/qemu-iotests/group | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/qemu-iotests/group +1 -1
index 5cad015231..f084061a16 100644
@@ -51,7 +51,7 @@
 027 rw auto quick
 028 rw backing quick
 029 rw auto quick
-030 rw auto backing
+030 rw backing
 031 rw auto quick
 032 rw auto quick
 033 rw auto quick
Signature · 1 line
2.18.2
#1 lore raw

Am 04.09.2020 um 07:57 hat Thomas Huth geschrieben:

Quoted 5 lines

Test 030 is still occasionally failing in the CI ... so for the time being, let's disable it in the "auto" group. We can add it back once it got more stable.

I would rather just disable this one test function as 030 is a pretty important one that tends to catch bugs.

Quoted 4 lines

I just saw the problem here:
https://cirrus-ci.com/task/5449330930745344?command=main#L6482
and Peter hit it a couple of weeks ago:
https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00136.html

I wonder how this can still happen. The test should have more than enough time to complete now. Except if the throttling doesn't work as expected.

I can't seem to reproduce this even if I add rather long delays. After 40 seconds, all jobs have moved either by 512k (which is STREAM_CHUNK) or not at all.

What is interesting is that in both cases it's stream-node8, which is the job streaming from node6 to node8, and node8 is the top-level node. It's also the last job to be changed to full speed, so all others did succeed before.

Kevin

Applied
#8 lore raw
In reply to Thomas Huth
Quoted 5 lines · Thomas Huth

Test 030 is still occasionally failing in the CI ... so for the time being, let's disable it in the "auto" group. We can add it back once it got more stable.

Quoted 8 lines

---
I just saw the problem here:
https://cirrus-ci.com/task/5449330930745344?command=main#L6482
and Peter hit it a couple of weeks ago:
https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00136.html

 tests/qemu-iotests/group | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Thanks, applied to my block branch:

https://git.xanclic.moe/XanClic/qemu/commits/branch/block

#9 lore raw
Quoted 12 lines · Max Reitz
Quoted 11 lines · Thomas Huth

Test 030 is still occasionally failing in the CI ... so for the time being, let's disable it in the "auto" group. We can add it back once it got more stable.

---
I just saw the problem here:
https://cirrus-ci.com/task/5449330930745344?command=main#L6482
and Peter hit it a couple of weeks ago:
https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00136.html

Quoted 6 lines
Quoted 2 lines
 tests/qemu-iotests/group | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Thanks, applied to my block branch:

https://git.xanclic.moe/XanClic/qemu/commits/branch/block

Or maybe not O:)

#2 lore raw
In reply to Kevin Wolf
Quoted 9 lines · Kevin Wolf

Am 04.09.2020 um 07:57 hat Thomas Huth geschrieben:

Quoted 5 lines

Test 030 is still occasionally failing in the CI ... so for the time being, let's disable it in the "auto" group. We can add it back once it got more stable.

I would rather just disable this one test function as 030 is a pretty important one that tends to catch bugs.

Ok, ... should it always get disabled, or shall we try to come up with some magic checks so that it only gets disabled in the CI pipelines (... though I don't have a clue how to check for Peter's merge test environment...)?

Thomas

#3 lore raw

Am 04.09.2020 um 12:14 hat Thomas Huth geschrieben:

Quoted 8 lines · Kevin Wolf
Quoted 7 lines

Am 04.09.2020 um 07:57 hat Thomas Huth geschrieben:

Quoted 5 lines

Test 030 is still occasionally failing in the CI ... so for the time being, let's disable it in the "auto" group. We can add it back once it got more stable.

Quoted 7 lines
Quoted 2 lines

I would rather just disable this one test function as 030 is a pretty important one that tends to catch bugs.

Ok, ... should it always get disabled, or shall we try to come up with some magic checks so that it only gets disabled in the CI pipelines (... though I don't have a clue how to check for Peter's merge test environment...)?

Maybe we can detect whether we're run as part of the "auto" group and skip the test then (as in QMPTestCase.case_skip)?

Kevin

#4 lore raw
In reply to Thomas Huth
Quoted 8 lines · Thomas Huth
Quoted 7 lines · Kevin Wolf

Am 04.09.2020 um 07:57 hat Thomas Huth geschrieben:

Quoted 5 lines

Test 030 is still occasionally failing in the CI ... so for the time being, let's disable it in the "auto" group. We can add it back once it got more stable.

Quoted 7 lines
Quoted 2 lines

I would rather just disable this one test function as 030 is a pretty important one that tends to catch bugs.

Ok, ... should it always get disabled, or shall we try to come up with some magic checks so that it only gets disabled in the CI pipelines (... though I don't have a clue how to check for Peter's merge test environment...)?

I suppose we could let check-block.sh set some environment variable.

Max

#5 lore raw
Quoted 12 lines · Max Reitz
Quoted 11 lines · Thomas Huth
Quoted 9 lines · Kevin Wolf

Am 04.09.2020 um 07:57 hat Thomas Huth geschrieben:

Quoted 5 lines

Test 030 is still occasionally failing in the CI ... so for the time being, let's disable it in the "auto" group. We can add it back once it got more stable.

I would rather just disable this one test function as 030 is a pretty important one that tends to catch bugs.

Quoted 6 lines
Quoted 4 lines

Ok, ... should it always get disabled, or shall we try to come up with some magic checks so that it only gets disabled in the CI pipelines (... though I don't have a clue how to check for Peter's merge test environment...)?

I suppose we could let check-block.sh set some environment variable.

Sounds like a plan! I'll try to cook a patch.

Thomas

#6 lore raw
In reply to Kevin Wolf
Quoted 14 lines · Kevin Wolf
Quoted 5 lines

Test 030 is still occasionally failing in the CI ... so for the time being, let's disable it in the "auto" group. We can add it back once it got more stable.

I would rather just disable this one test function as 030 is a pretty important one that tends to catch bugs.

Quoted 4 lines

I just saw the problem here:
https://cirrus-ci.com/task/5449330930745344?command=main#L6482
and Peter hit it a couple of weeks ago:
https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00136.html

Quoted 7 lines

I wonder how this can still happen. The test should have more than enough time to complete now. Except if the throttling doesn't work as expected.

I can't seem to reproduce this even if I add rather long delays. After 40 seconds, all jobs have moved either by 512k (which is STREAM_CHUNK) or not at all.

I also don't understand how this can fail... I assume the test is not running for that long in the cases when it fails, right?

Berto

#7 lore raw
Quoted 19 lines · Alberto Garcia
Quoted 18 lines · Kevin Wolf
Quoted 5 lines

Test 030 is still occasionally failing in the CI ... so for the time being, let's disable it in the "auto" group. We can add it back once it got more stable.

I would rather just disable this one test function as 030 is a pretty important one that tends to catch bugs.

Quoted 4 lines

I just saw the problem here:
https://cirrus-ci.com/task/5449330930745344?command=main#L6482
and Peter hit it a couple of weeks ago:
https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00136.html

I wonder how this can still happen. The test should have more than enough time to complete now. Except if the throttling doesn't work as expected.

Quoted 6 lines
Quoted 3 lines

I can't seem to reproduce this even if I add rather long delays. After 40 seconds, all jobs have moved either by 512k (which is STREAM_CHUNK) or not at all.

I also don't understand how this can fail... I assume the test is not running for that long in the cases when it fails, right?

Hard to say ... the problem only occurs occasionally, and I've never seen it happen "live", only in the CI logs after the job has failed. I guess you'd have to print timestamps in the code and then submit a lot of jobs to the CI systems that are sensitive to this problem (e.g. Cirrus and Travis) to find out...

Thomas