Skip to contentSkip to conversation

selftest: allow '-R' and '-r' be used together

PATCH In review Chen Qi 4 participants 11 messages Nov 24, 2022 – Dec 26, 2022

lore
Share & tools
lore

Chen Qi

author
#0 lore raw

PATCH 1/1 selftest: allow '-R' and '-r' be used together

For now, there's no easy way to skip some specified tests when using 'oe-selftest -r'. This is because '-R' and '-r' are in the mutually exclusive group. Looking at the codes, the codes have already allowed running specified tests and skipping specified tests at the same time. So move '-R' out of the mutually exclusive group and change its help message.

After this change, the following command could be used.
oe-selftest -r A -R A.B.C

This does introduce a behavior regression. Previous 'oe-selftest -R xxx' needs to now be changed to 'oe-selftest -a -R xxx'.

---
 meta/lib/oeqa/selftest/context.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/lib/oeqa/selftest/context.py +3 -3
index 78c7a467e2..c7dd03ce37 100644
@@ -154,9 +154,6 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
         group.add_argument('-a', '--run-all-tests', default=False,
                 action="store_true", dest="run_all_tests",
                 help='Run all (unhidden) tests')
-        group.add_argument('-R', '--skip-tests', required=False, action='store',
-                nargs='+', dest="skips", default=None,
-                help='Run all (unhidden) tests except the ones specified. Format should be <module>[.<class>[.<test_method>]]')
         group.add_argument('-r', '--run-tests', required=False, action='store',
                 nargs='+', dest="run_tests", default=None,
                 help='Select what tests to run (modules, classes or test methods). Format should be: <module>.<class>.<test_method>')
@@ -171,6 +168,9 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
                 action="store_true", default=False,
                 help='List all available tests.')
 
+        parser.add_argument('-R', '--skip-tests', required=False, action='store',
+                nargs='+', dest="skips", default=None,
+                help='Skip the tests specified. Format should be <module>[.<class>[.<test_method>]]')
         parser.add_argument('-j', '--num-processes', dest='processes', action='store',
                 type=int, help="number of processes to execute in parallel with")
 
Signature · 1 line
2.37.1
#1 lore raw

Hello,

Quoted 7 lines · Chen Qi

For now, there's no easy way to skip some specified tests when using 'oe-selftest -r'. This is because '-R' and '-r' are in the mutually exclusive group. Looking at the codes, the codes have already allowed running specified tests and skipping specified tests at the same time. So move '-R' out of the mutually exclusive group and change its help message.

Quoted 7 lines

After this change, the following command could be used.
oe-selftest -r A -R A.B.C

This does introduce a behavior regression. Previous 'oe-selftest -R xxx' needs to now be changed to 'oe-selftest -a -R xxx'.

This fails on the autobuilders which uses:

oe-selftest --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/4395/steps/14/logs/stdio

Quoted 42 lines
---
 meta/lib/oeqa/selftest/context.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/lib/oeqa/selftest/context.py +3 -3
index 78c7a467e2..c7dd03ce37 100644
@@ -154,9 +154,6 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
         group.add_argument('-a', '--run-all-tests', default=False,
                 action="store_true", dest="run_all_tests",
                 help='Run all (unhidden) tests')
-        group.add_argument('-R', '--skip-tests', required=False, action='store',
-                nargs='+', dest="skips", default=None,
-                help='Run all (unhidden) tests except the ones specified. Format should be <module>[.<class>[.<test_method>]]')
         group.add_argument('-r', '--run-tests', required=False, action='store',
                 nargs='+', dest="run_tests", default=None,
                 help='Select what tests to run (modules, classes or test methods). Format should be: <module>.<class>.<test_method>')
@@ -171,6 +168,9 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
                 action="store_true", default=False,
                 help='List all available tests.')
 
+        parser.add_argument('-R', '--skip-tests', required=False, action='store',
+                nargs='+', dest="skips", default=None,
+                help='Skip the tests specified. Format should be <module>[.<class>[.<test_method>]]')
         parser.add_argument('-j', '--num-processes', dest='processes', action='store',
                 type=int, help="number of processes to execute in parallel with")
 
Signature · 11 lines
2.37.1



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173732): https://lists.openembedded.org/g/openembedded-core/message/173732
Mute This Topic: https://lists.openembedded.org/mt/95231396/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
author
#2 lore raw

Hi Alexandre,

Yes, I'm aware of this behavior regression.
With this patch, 'oe-selftest -R xxx' needs to be changed to 'oe-selftest -a -R xxx'.

I'm not sure if this is useful for other people. Because how people use oe-selftest is unknown to me. For me, I want to run something like 'oe-selftest -r A -R A.B.C'.

Regards,
Qi

-----Original Message-----
From: Alexandre Belloni <[email protected]>
Sent: Thursday, November 24, 2022 7:35 PM
To: Chen, Qi <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core][PATCH] selftest: allow '-R' and '-r' be used together

Hello,

Quoted 7 lines · Chen Qi

For now, there's no easy way to skip some specified tests when using 'oe-selftest -r'. This is because '-R' and '-r' are in the mutually exclusive group. Looking at the codes, the codes have already allowed running specified tests and skipping specified tests at the same time. So move '-R' out of the mutually exclusive group and change its help message.

Quoted 7 lines

After this change, the following command could be used.
oe-selftest -r A -R A.B.C

This does introduce a behavior regression.
Previous 'oe-selftest -R xxx' needs to now be changed to 'oe-selftest -a -R xxx'.

This fails on the autobuilders which uses:

oe-selftest --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/4395/steps/14/logs/stdio

Quoted 45 lines
---
 meta/lib/oeqa/selftest/context.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/lib/oeqa/selftest/context.py +0 -3
index 78c7a467e2..c7dd03ce37 100644
@@ -154,9 +154,6 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
         group.add_argument('-a', '--run-all-tests', default=False,
                 action="store_true", dest="run_all_tests",
                 help='Run all (unhidden) tests')
-        group.add_argument('-R', '--skip-tests', required=False, action='store',
-                nargs='+', dest="skips", default=None,
-                help='Run all (unhidden) tests except the ones specified. Format should be <module>[.<class>[.<test_method>]]')
         group.add_argument('-r', '--run-tests', required=False, action='store',
                 nargs='+', dest="run_tests", default=None,
                 help='Select what tests to run (modules, classes or 

test methods). Format should be: <module>.<class>.<test_method>') @@ -171,6 +168,9 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
action="store_true", default=False,
help='List all available tests.')

meta/lib/oeqa/selftest/context.py +4 -0
+        parser.add_argument('-R', '--skip-tests', required=False, action='store',
+                nargs='+', dest="skips", default=None,
+                help='Skip the tests specified. Format should be 
+ <module>[.<class>[.<test_method>]]')
         parser.add_argument('-j', '--num-processes', dest='processes', action='store',
                 type=int, help="number of processes to execute in 

parallel with")

--
2.37.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173732): https://lists.openembedded.org/g/openembedded-core/message/173732
Mute This Topic: https://lists.openembedded.org/mt/95231396/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#3 lore raw
Quoted 5 lines · Chen Qi

Hi Alexandre,

Yes, I'm aware of this behavior regression.
With this patch, 'oe-selftest -R xxx' needs to be changed to 'oe-selftest -a -R xxx'.

Yes, my point is that there is no -R in the command line used by the AB

Quoted 92 lines

I'm not sure if this is useful for other people. Because how people use oe-selftest is unknown to me. For me, I want to run something like 'oe-selftest -r A -R A.B.C'.

Regards,
Qi

-----Original Message-----
From: Alexandre Belloni <[email protected]>
Sent: Thursday, November 24, 2022 7:35 PM
To: Chen, Qi <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core][PATCH] selftest: allow '-R' and '-r' be used together

Hello,

Quoted 7 lines · Chen Qi

For now, there's no easy way to skip some specified tests when using 'oe-selftest -r'. This is because '-R' and '-r' are in the mutually exclusive group. Looking at the codes, the codes have already allowed running specified tests and skipping specified tests at the same time. So move '-R' out of the mutually exclusive group and change its help message.

Quoted 7 lines

After this change, the following command could be used.
oe-selftest -r A -R A.B.C

This does introduce a behavior regression.
Previous 'oe-selftest -R xxx' needs to now be changed to 'oe-selftest -a -R xxx'.

This fails on the autobuilders which uses:

oe-selftest --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/4395/steps/14/logs/stdio

Quoted 39 lines
---
 meta/lib/oeqa/selftest/context.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/lib/oeqa/selftest/context.py +0 -3
index 78c7a467e2..c7dd03ce37 100644
@@ -154,9 +154,6 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
         group.add_argument('-a', '--run-all-tests', default=False,
                 action="store_true", dest="run_all_tests",
                 help='Run all (unhidden) tests')
-        group.add_argument('-R', '--skip-tests', required=False, action='store',
-                nargs='+', dest="skips", default=None,
-                help='Run all (unhidden) tests except the ones specified. Format should be <module>[.<class>[.<test_method>]]')
         group.add_argument('-r', '--run-tests', required=False, action='store',
                 nargs='+', dest="run_tests", default=None,
                 help='Select what tests to run (modules, classes or 

test methods). Format should be: <module>.<class>.<test_method>') @@ -171,6 +168,9 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
action="store_true", default=False,
help='List all available tests.')

meta/lib/oeqa/selftest/context.py +4 -0
+        parser.add_argument('-R', '--skip-tests', required=False, action='store',
+                nargs='+', dest="skips", default=None,
+                help='Skip the tests specified. Format should be 
+ <module>[.<class>[.<test_method>]]')
         parser.add_argument('-j', '--num-processes', dest='processes', action='store',
                 type=int, help="number of processes to execute in 

parallel with")

--
2.37.1

Signature · 12 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173743): https://lists.openembedded.org/g/openembedded-core/message/173743
Mute This Topic: https://lists.openembedded.org/mt/95231396/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
author
#4 lore raw

Hi Alexandre,

The command line is: oe-selftest --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15

The '--skip-tests' option is the long format of '-R'.

Regards,
Qi

-----Original Message-----
From: Alexandre Belloni <[email protected]>
Sent: Thursday, November 24, 2022 9:02 PM
To: Chen, Qi <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core][PATCH] selftest: allow '-R' and '-r' be used together

Quoted 5 lines · Chen Qi

Hi Alexandre,

Yes, I'm aware of this behavior regression.
With this patch, 'oe-selftest -R xxx' needs to be changed to 'oe-selftest -a -R xxx'.

Yes, my point is that there is no -R in the command line used by the AB

Quoted 87 lines

I'm not sure if this is useful for other people. Because how people use oe-selftest is unknown to me. For me, I want to run something like 'oe-selftest -r A -R A.B.C'.

Regards,
Qi

-----Original Message-----
From: Alexandre Belloni <[email protected]>
Sent: Thursday, November 24, 2022 7:35 PM
To: Chen, Qi <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core][PATCH] selftest: allow '-R' and '-r' be used together

Hello,

Quoted 7 lines · Chen Qi

For now, there's no easy way to skip some specified tests when using 'oe-selftest -r'. This is because '-R' and '-r' are in the mutually exclusive group. Looking at the codes, the codes have already allowed running specified tests and skipping specified tests at the same time. So move '-R' out of the mutually exclusive group and change its help message.

Quoted 7 lines

After this change, the following command could be used.
oe-selftest -r A -R A.B.C

This does introduce a behavior regression. Previous 'oe-selftest -R xxx' needs to now be changed to 'oe-selftest -a -R xxx'.

This fails on the autobuilders which uses:

oe-selftest --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/4395
/steps/14/logs/stdio

Quoted 32 lines
---
 meta/lib/oeqa/selftest/context.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/lib/oeqa/selftest/context.py +0 -3
index 78c7a467e2..c7dd03ce37 100644
@@ -154,9 +154,6 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
         group.add_argument('-a', '--run-all-tests', default=False,
                 action="store_true", dest="run_all_tests",
                 help='Run all (unhidden) tests')
-        group.add_argument('-R', '--skip-tests', required=False, action='store',
-                nargs='+', dest="skips", default=None,
-                help='Run all (unhidden) tests except the ones specified. Format should be <module>[.<class>[.<test_method>]]')
         group.add_argument('-r', '--run-tests', required=False, action='store',
                 nargs='+', dest="run_tests", default=None,
                 help='Select what tests to run (modules, classes or 

test methods). Format should be: <module>.<class>.<test_method>') @@ -171,6 +168,9 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
action="store_true", default=False,
help='List all available tests.')

meta/lib/oeqa/selftest/context.py +4 -0
+        parser.add_argument('-R', '--skip-tests', required=False, action='store',
+                nargs='+', dest="skips", default=None,
+                help='Skip the tests specified. Format should be
+ <module>[.<class>[.<test_method>]]')
         parser.add_argument('-j', '--num-processes', dest='processes', action='store',
                 type=int, help="number of processes to execute in 

parallel with")

Quoted 7 lines

--
2.37.1

--
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com

Quoted 10 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#173743):
https://lists.openembedded.org/g/openembedded-core/message/173743
Mute This Topic: https://lists.openembedded.org/mt/95231396/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

--
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com

#5 lore raw
Quoted 6 lines · Chen Qi

Hi Alexandre,

The command line is: oe-selftest --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15

The '--skip-tests' option is the long format of '-R'.

Right, sorry about that :) I actually checked we didn't have any -R before sending the patch to the AB... I'll discuss that with RP.

Quoted 122 lines

Regards,
Qi

-----Original Message-----
From: Alexandre Belloni <[email protected]>
Sent: Thursday, November 24, 2022 9:02 PM
To: Chen, Qi <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core][PATCH] selftest: allow '-R' and '-r' be used together

Quoted 5 lines · Chen Qi

Hi Alexandre,

Yes, I'm aware of this behavior regression.
With this patch, 'oe-selftest -R xxx' needs to be changed to 'oe-selftest -a -R xxx'.

Yes, my point is that there is no -R in the command line used by the AB

Quoted 82 lines

I'm not sure if this is useful for other people. Because how people use oe-selftest is unknown to me. For me, I want to run something like 'oe-selftest -r A -R A.B.C'.

Regards,
Qi

-----Original Message-----
From: Alexandre Belloni <[email protected]>
Sent: Thursday, November 24, 2022 7:35 PM
To: Chen, Qi <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core][PATCH] selftest: allow '-R' and '-r' be used together

Hello,

Quoted 7 lines · Chen Qi

For now, there's no easy way to skip some specified tests when using 'oe-selftest -r'. This is because '-R' and '-r' are in the mutually exclusive group. Looking at the codes, the codes have already allowed running specified tests and skipping specified tests at the same time. So move '-R' out of the mutually exclusive group and change its help message.

Quoted 7 lines

After this change, the following command could be used.
oe-selftest -r A -R A.B.C

This does introduce a behavior regression. Previous 'oe-selftest -R xxx' needs to now be changed to 'oe-selftest -a -R xxx'.

This fails on the autobuilders which uses:

oe-selftest --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/4395
/steps/14/logs/stdio

Quoted 32 lines
---
 meta/lib/oeqa/selftest/context.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/lib/oeqa/selftest/context.py +0 -3
index 78c7a467e2..c7dd03ce37 100644
@@ -154,9 +154,6 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
         group.add_argument('-a', '--run-all-tests', default=False,
                 action="store_true", dest="run_all_tests",
                 help='Run all (unhidden) tests')
-        group.add_argument('-R', '--skip-tests', required=False, action='store',
-                nargs='+', dest="skips", default=None,
-                help='Run all (unhidden) tests except the ones specified. Format should be <module>[.<class>[.<test_method>]]')
         group.add_argument('-r', '--run-tests', required=False, action='store',
                 nargs='+', dest="run_tests", default=None,
                 help='Select what tests to run (modules, classes or 

test methods). Format should be: <module>.<class>.<test_method>') @@ -171,6 +168,9 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
action="store_true", default=False,
help='List all available tests.')

meta/lib/oeqa/selftest/context.py +4 -0
+        parser.add_argument('-R', '--skip-tests', required=False, action='store',
+                nargs='+', dest="skips", default=None,
+                help='Skip the tests specified. Format should be
+ <module>[.<class>[.<test_method>]]')
         parser.add_argument('-j', '--num-processes', dest='processes', action='store',
                 type=int, help="number of processes to execute in 

parallel with")

Quoted 7 lines

--
2.37.1

Quoted 7 lines

--
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com

--
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173748): https://lists.openembedded.org/g/openembedded-core/message/173748
Mute This Topic: https://lists.openembedded.org/mt/95231396/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#6 lore raw
Quoted 10 lines · Alexandre Belloni via lists.openembedded.org
Quoted 6 lines · Chen Qi

Hi Alexandre,

The command line is: oe-selftest --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15

The '--skip-tests' option is the long format of '-R'.

Right, sorry about that :) I actually checked we didn't have any -R before sending the patch to the AB... I'll discuss that with RP.

The autobuilder is not running "all" tests though, it is running two groups of tests, -T machine and -T toolchain-user. Adding -a here doesn't really make sense.

Cheers,

Richard

author
#7 lore raw

Hi Richard,

The AB is actually running 'all' tests.

The '--skip-tests (-R)' option means 'Run all (unhidden) tests except the ones specified.', according to its help message. This is also its actual effect for now. You can see there's an implicit 'run all tests' meaning in this option. The '-T (--exclude-tag)' options means 'Exclude all (unhidden) tests that match any of the specified tag(s)', according to its help message.

So the AB's oe-selftest command means: execute all tests except the ones tagged with 'machine' and 'toolchain-user', and also skip 'distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible'.

Regards,
Qi

-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: Thursday, November 24, 2022 9:47 PM
To: [email protected]; Chen, Qi <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core][PATCH] selftest: allow '-R' and '-r' be used together

Quoted 8 lines · Alexandre Belloni via lists.openembedded.org
Quoted 7 lines · Chen Qi

Hi Alexandre,

The command line is: oe-selftest --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15

Quoted 5 lines
Quoted 2 lines

The '--skip-tests' option is the long format of '-R'.

Right, sorry about that :) I actually checked we didn't have any -R before sending the patch to the AB... I'll discuss that with RP.

The autobuilder is not running "all" tests though, it is running two groups of tests, -T machine and -T toolchain-user. Adding -a here doesn't really make sense.

Cheers,

Richard

#8 lore raw

Hi Qi,

Quoted 10 lines · Chen Qi

The AB is actually running 'all' tests.

The '--skip-tests (-R)' option means 'Run all (unhidden) tests except the ones specified.', according to its help message. This is also its actual effect for now. You can see there's an implicit 'run all tests' meaning in this option.
The '-T (--exclude-tag)' options means 'Exclude all (unhidden) tests that match any of the specified tag(s)', according to its help message.

Quoted 4 lines

So the AB's oe-selftest command means: execute all tests except the ones tagged with 'machine' and 'toolchain-user', and also skip 'distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible'.

You're right, a different build has -t machine and I'm confusing the two, sorry.

I don't really like making changes which require lockstep changes to the autobuilder configuration but I can see why we could do with doing so here.

I was thinking there should be symettry between -r and -R like there is with -T and -t but that also probably doesn't make sense when you think about it more.

Alex: We'll probably have to work out how to make this change...

Cheers,

Richard

#9 lore raw
Quoted 25 lines · Richard Purdie via lists.openembedded.org

Hi Qi,

Quoted 10 lines · Chen Qi

The AB is actually running 'all' tests.

The '--skip-tests (-R)' option means 'Run all (unhidden) tests except the ones specified.', according to its help message. This is also its actual effect for now. You can see there's an implicit 'run all tests' meaning in this option.
The '-T (--exclude-tag)' options means 'Exclude all (unhidden) tests that match any of the specified tag(s)', according to its help message.

Quoted 4 lines

So the AB's oe-selftest command means: execute all tests except the ones tagged with 'machine' and 'toolchain-user', and also skip 'distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible'.

You're right, a different build has -t machine and I'm confusing the two, sorry.

I don't really like making changes which require lockstep changes to the autobuilder configuration but I can see why we could do with doing so here.

Quoted 5 lines

I was thinking there should be symettry between -r and -R like there is with -T and -t but that also probably doesn't make sense when you think about it more.

Alex: We'll probably have to work out how to make this change...

Nudge and Happy Holidays.

../Randy

Quoted 15 lines

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173766): https://lists.openembedded.org/g/openembedded-core/message/173766
Mute This Topic: https://lists.openembedded.org/mt/95231396/3616765
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 2 lines
# Randy MacLeod
# Wind River Linux
#10 lore raw
Quoted 26 lines · Randy MacLeod
Quoted 25 lines · Richard Purdie via lists.openembedded.org

Hi Qi,

Quoted 10 lines · Chen Qi

The AB is actually running 'all' tests.

The '--skip-tests (-R)' option means 'Run all (unhidden) tests except the ones specified.', according to its help message. This is also its actual effect for now. You can see there's an implicit 'run all tests' meaning in this option.
The '-T (--exclude-tag)' options means 'Exclude all (unhidden) tests that match any of the specified tag(s)', according to its help message.

Quoted 4 lines

So the AB's oe-selftest command means: execute all tests except the ones tagged with 'machine' and 'toolchain-user', and also skip 'distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible'.

You're right, a different build has -t machine and I'm confusing the two, sorry.

I don't really like making changes which require lockstep changes to the autobuilder configuration but I can see why we could do with doing so here.

Quoted 7 lines
Quoted 5 lines

I was thinking there should be symettry between -r and -R like there is with -T and -t but that also probably doesn't make sense when you think about it more.

Alex: We'll probably have to work out how to make this change...

Nudge and Happy Holidays.

Yes, this is still on my radar.

Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com