Skip to contentSkip to conversation

Combined test-runner fixes

PATCH In review James Prestwood 2 participants 15 messages Mar 28, 2022

lore
Share & tools
lore

James Prestwood

author
#0 lore raw

v4 cover Combined test-runner fixes

This combines the individual test fixes with the memory fixes

James Prestwood (13):
test-runner: add memory usage for debugging
test-runner: fix OOM issues (hopefully)
auto-t: remove waitstatus_to_exitcode for better compatibility
auto-t: iwd.py: fix full_scan argument
auto-t: fixup testRoamRetry for better timing reliability
auto-t: use wait_for_event in testBSSBlacklist
auto-t: use wait_for_event in testSAE-roam
auto-t: do full scan for testSAE
auto-t: use wait_for_event in testPSK-roam
auto-t: do a full scan when getting network
dpp: fail if disconnected during DPP protocol
dpp: set new_freq when configuring as initiator
auto-t: don't wait for wpa_supplicant to connect

 autotests/testBSSBlacklist/connection_test.py | 18 ++--
 autotests/testDPP/connection_test.py          |  5 +-
 autotests/testNetconfig/connection_test.py    |  5 +-
 .../testOWE-transition/connection_test.py     | 13 +--
 autotests/testPSK-roam/connection_test.py     | 29 +-----
 autotests/testRoamRetry/fast_retry_test.py    | 13 ++-
 autotests/testRoamRetry/stop_retry_test.py    |  7 +-
 autotests/testSAE-roam/connection_test.py     | 11 ++-
 autotests/testSAE/connection_test.py          | 14 +--
 autotests/util/iwd.py                         | 23 ++---
 src/dpp.c                                     | 15 +++-
 tools/test-runner                             | 89 +++++++++++++++++--
 12 files changed, 142 insertions(+), 100 deletions(-)
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH v4 1/13 test-runner: add memory usage for debugging

In debug mode the test context is printed before each test. This adds some additional information in there:

Available Memory: /proc/meminfo: MemAvailable
Last Test Delta: Change in usage between current and last test Per-test Usage: Graph of usage relative to all past tests. This is
useful for seeing a trend down/up of usage.

---
 tools/test-runner | 78 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 77 insertions(+), 1 deletion(-)
tools/test-runner +77 -1
index b0ec87a2..73a2cf61 100755
@@ -807,6 +807,47 @@ class Namespace:
 
 		return ret
 
+class BarChart():
+	def __init__(self, height=10, max_width=80):
+		self._height = height
+		self._max_width = max_width
+		self._values = []
+		self._max_value = 0
+		self._min_value = 0
+
+	def add_value(self, value):
+		if len(self._values) == 0:
+			self._max_value = int(1.01 * value)
+			self._min_value = int(0.99 * value)
+		elif value > self._max_value:
+			self._max_value = int(1.01 * value)
+		elif value < self._min_value:
+			self._min_value = int(0.99 * value)
+
+		self._values.append(value)
+
+	def _value_to_stars(self, value):
+		# Need to scale value (range of min_value -> max_value) to
+		# a range of 0 -> height
+		#
+		# Scaled = ((value - min_value) / ( max_value - min_value)) * (Height - 0) + 0
+
+		return int(((value - self._min_value) /
+			(self._max_value - self._min_value)) * self._height)
+
+	def __str__(self):
+		# Need to map value from range 0 - self._height
+		ret = ''
+
+		for i, value in enumerate(self._values):
+			stars = self._value_to_stars(value)
+			ret += '[%3u] ' % i + '%-10s' % ('*' * stars) + '\t\t\t%d\n' % value
+
+		ret += '\n'
+
+		return ret
+
+
 class TestContext(Namespace):
 	'''
 		Contains all information for a given set of tests being run
@@ -825,6 +866,8 @@ class TestContext(Namespace):
 		self.results = {}
 		self.mainloop = GLib.MainLoop()
 		self.namespaces = []
+		self._last_mem_available = 0
+		self._mem_chart = BarChart()
 
 	def start_dbus_monitor(self):
 		if not self.is_verbose('dbus-monitor'):
@@ -1036,6 +1079,23 @@ class TestContext(Namespace):
 
 		self.reset()
 
+	def meminfo_to_dict(self):
+		def removesuffix(string, suffix):
+			if string.endswith(suffix):
+				return string[:-len(suffix)]
+			return string
+
+		ret = {}
+
+		with open('/proc/meminfo', 'r') as f:
+			data = f.read().strip().split('\n')
+
+		for l in data:
+			entry = l.split(':')
+			ret[entry[0]] = int(removesuffix(entry[1], 'kB'))
+
+		return ret
+
 	def __str__(self):
 		ret = 'Arguments:\n'
 		for arg in vars(self.args):
@@ -1048,6 +1108,16 @@ class TestContext(Namespace):
 		else:
 			ret += '\tNo Hostapd instances\n'
 
+		info = self.meminfo_to_dict()
+		self._mem_chart.add_value(info['MemAvailable'])
+
+		ret += 'Available Memory: %u kB\n' % info['MemAvailable']
+		ret += 'Last Test Delta: %+d kB\n' % (info['MemAvailable'] - self._last_mem_available)
+		ret += 'Per-test Usage:\n'
+		ret += str(self._mem_chart)
+
+		self._last_mem_available = info['MemAvailable']
+
 		ret += super().__str__()
 
 		for n in self.namespaces:
@@ -1171,7 +1241,13 @@ def start_test(ctx, subtests, rqueue):
 	#
 	for s in subtests:
 		loader = unittest.TestLoader()
-		module = importlib.import_module(os.path.splitext(s)[0])
+		try:
+			module = importlib.import_module(os.path.splitext(s)[0])
+		except OSError as e:
+			dbg(subprocess.check_output("dmesg | tail -80", shell=True).decode('utf-8'))
+			print(ctx)
+			raise e
+
 		subtest = loader.loadTestsFromModule(module)
 
 		# The test suite is being (ab)used to get a bit more granularity
Signature · 1 line
2.34.1
author
#2 lore raw

PATCH v4 2/13 test-runner: fix OOM issues (hopefully)

In reply to James Prestwood

For quite a while test-runner has run into frequent OOM exceptions when running many tests in a row. Its not completely known exactly why, but seems to point to the 9p driver which is used for sharing the root fs between the test-runner VM and the host.

With debugging enabled (-d) one can see the available memory available relatively stable. If a test fails it may spike ~3-4kb but this quickly recovers as python garbage collects.

At some point the kernel faults failing to allocate which (usually) is shown by a python OOM exception. At this point there is plenty of available memory.

Dumping the kernel trace its seen that the 9p driver is involved:

[  248.962949] test-runner: page allocation failure: order:7, mode:0x40dc0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null),cpuset=/,mems_allowed=0
[  248.962958] CPU: 2 PID: 477 Comm: test-runner Not tainted 5.16.0 #91
[  248.962960] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.14.0-4.fc34 04/01/2014
[  248.962961] Call Trace:
[  248.962964]  <TASK>
[  248.962965]  dump_stack_lvl+0x34/0x44
[  248.962971]  warn_alloc.cold+0x78/0xdc
[  248.962975]  ? __alloc_pages_direct_compact+0x14c/0x1e0
[  248.962979]  __alloc_pages_slowpath.constprop.0+0xbfe/0xc60
[  248.962982]  __alloc_pages+0x2d5/0x2f0
[  248.962984]  kmalloc_order+0x23/0x80
[  248.962988]  kmalloc_order_trace+0x14/0x80
[  248.962990]  v9fs_alloc_rdir_buf.isra.0+0x1f/0x30
[  248.962994]  v9fs_dir_readdir+0x51/0x1d0
[  248.962996]  ? __handle_mm_fault+0x6e0/0xb40
[  248.962999]  ? inode_security+0x1d/0x50
[  248.963009]  ? selinux_file_permission+0xff/0x140
[  248.963011]  iterate_dir+0x16f/0x1c0
[  248.963014]  __x64_sys_getdents64+0x7b/0x120
[  248.963016]  ? compat_fillonedir+0x150/0x150
[  248.963019]  do_syscall_64+0x3b/0x90
[  248.963021]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  248.963024] RIP: 0033:0x7fedd7c6d8c7
[  248.963026] Code: 00 00 0f 05 eb b7 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 f3 0f 1e fa b8 ff ff ff 7f 48 39 c2 48 0f 47 d0 b8 d9 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 01 c3 48 8b 15 81 a5 0f 00 f7 d8 64 89 02 48
[  248.963028] RSP: 002b:00007ffd06cd87e8 EFLAGS: 00000293 ORIG_RAX: 00000000000000d9
[  248.963031] RAX: ffffffffffffffda RBX: 000056090d87dd20 RCX: 00007fedd7c6d8c7
[  248.963032] RDX: 0000000000080000 RSI: 000056090d87dd50 RDI: 000000000000000f
[  248.963033] RBP: 000056090d87dd50 R08: 0000000000000030 R09: 00007fedc7d37af0
[  248.963035] R10: 00007fedc7d7d730 R11: 0000000000000293 R12: ffffffffffffff88
[  248.963038] R13: 000056090d87dd24 R14: 0000000000000000 R15: 000056090d0485e8

Here its seen an allocation of 512k is being requested (order:7), but faults. In this run it there was ~35MB of available memory on the system.

Available Memory: 35268 kB
Last Test Delta: -2624 kB
Per-test Usage:

[  0] **        			37016
[  1] ********* 			41584
[  2] *         			36280
[  3] ********* 			41452
[  4] ********  			40940
[  5] ******    			39284
[  6] ****      			38348
[  7] ***       			37496
[  8] ****      			37892
[  9]           			35268

This can be reproduced by running all autotests (changing the ram down to ~128MB helps trigger it faster):

./tools/test-runner -k <kernel> -d

After many attempts to fix this it was finally found that simply removing the explicit 9p2000.u version from the kernel command line 'fixed' the problem. This even allows decreasing the RAM down to 256MB from 384MB and so far no OOM's have been seen.

---
 tools/test-runner | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
tools/test-runner +3 -8
index 73a2cf61..9bc66419 100755
@@ -1244,6 +1244,7 @@ def start_test(ctx, subtests, rqueue):
 		try:
 			module = importlib.import_module(os.path.splitext(s)[0])
 		except OSError as e:
+			dbg(subprocess.check_output("cat /proc/buddyinfo", shell=True).decode('utf-8'))
 			dbg(subprocess.check_output("dmesg | tail -80", shell=True).decode('utf-8'))
 			print(ctx)
 			raise e
@@ -1891,13 +1892,7 @@ class Main:
 		else:
 			smp = int(nproc / 2)
 
-		#
-		# Increase RAM if valgrind is being used
-		#
-		if self.args.valgrind:
-			ram = 512
-		else:
-			ram = 384
+		ram = 256
 
 		print("Using %d cores, %d RAM for VM" % (smp, ram))
 
@@ -1936,7 +1931,7 @@ class Main:
 			'-append',
 			'console=ttyS0,115200n8 earlyprintk=serial \
 				rootfstype=9p root=/dev/root \
-				rootflags=trans=virtio,msize=1048576,version=9p2000.u \
+				rootflags=trans=virtio \
 				acpi=off pci=noacpi %s ro \
 				mac80211_hwsim.radios=0 %s' % (kern_log, options),
 			'-smp', str(smp)
Signature · 1 line
2.34.1
author
#3 lore raw

PATCH v4 3/13 auto-t: remove waitstatus_to_exitcode for better compatibility

In reply to James Prestwood

This was added in Python 3.9 which isn't always supported. Instead use the subprocess module which automatically checks the command output.

---
 autotests/testNetconfig/connection_test.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
autotests/testNetconfig/connection_test.py +3 -2
index 8eeffbbd..91c6eb6b 100644
@@ -12,6 +12,7 @@ from hostapd import HostapdCLI
 import testutil
 from config import ctx
 import os, time
+import subprocess
 
 class Test(unittest.TestCase):
 
@@ -40,8 +41,8 @@ class Test(unittest.TestCase):
         testutil.test_ifaces_connected()
 
         time.sleep(2)
-        ret = os.system('ip addr show ' + device.name + ' | grep \'inet6 3ffe:501:ffff:100::\'')
-        self.assertEqual(os.waitstatus_to_exitcode(ret), 0)
+        subprocess.check_output('ip addr show ' + device.name + \
+                            ' | grep \'inet6 3ffe:501:ffff:100::\'', shell=True)
 
         device.disconnect()
 
Signature · 1 line
2.34.1
author
#4 lore raw

PATCH v4 4/13 auto-t: iwd.py: fix full_scan argument

In reply to James Prestwood

This was not being properly honored when existing networks were already populated. This poses an issue for any test which uses full_scan after setting radio values such as signal strength.

---
 autotests/util/iwd.py | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)
autotests/util/iwd.py +12 -11
index d6620da5..253fe25e 100755
@@ -492,17 +492,18 @@ class Device(IWDDBusAbstract):
            main sorting factor.
         '''
         ordered_networks = []
-        for bus_obj in self._station.GetOrderedNetworks():
-            ordered_network = OrderedNetwork(bus_obj, self._bus, self._namespace)
-            ordered_networks.append(ordered_network)
-
-        names = [x.name for x in ordered_networks]
-
-        # all() will always return true if 'list' is empty
-        if all(x in names for x in list) and len(names) > 0:
-            return ordered_networks
-        elif not scan_if_needed:
-            return None
+        if not full_scan:
+            for bus_obj in self._station.GetOrderedNetworks():
+                ordered_network = OrderedNetwork(bus_obj, self._bus, self._namespace)
+                ordered_networks.append(ordered_network)
+
+            names = [x.name for x in ordered_networks]
+
+            # all() will always return true if 'list' is empty
+            if all(x in names for x in list) and len(names) > 0:
+                return ordered_networks
+            elif not scan_if_needed:
+                return None
 
         condition = 'not obj.scanning'
         IWD._wait_for_object_condition(self, condition)
Signature · 1 line
2.34.1
author
#5 lore raw

PATCH v4 5/13 auto-t: fixup testRoamRetry for better timing reliability

In reply to James Prestwood

Change a few critical checks that were failing sometimes:
- A few asserts were changed to wait_for_object_condition
- A 15 second timeout was removed (default used instead)
- Do a full scan at beginning of each test to clear any cached BSS's. The second test run was getting stale results and the RSSI values were not expected.

---
 autotests/testRoamRetry/fast_retry_test.py | 13 ++++++-------
 autotests/testRoamRetry/stop_retry_test.py |  7 +++----
 2 files changed, 9 insertions(+), 11 deletions(-)
autotests/testRoamRetry/fast_retry_test.py +6 -7
index c7de5055..b45daada 100644
@@ -63,7 +63,7 @@ class Test(unittest.TestCase):
         devices = wd.list_devices(1)
         device = devices[0]
 
-        ordered_network = device.get_ordered_network('TestRoamRetry')
+        ordered_network = device.get_ordered_network('TestRoamRetry', full_scan=True)
 
         self.assertEqual(ordered_network.type, NetworkType.psk)
 
@@ -75,18 +75,18 @@ class Test(unittest.TestCase):
         condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_condition(device, condition)
 
-        self.assertTrue(bss_hostapd[0].list_sta())
-        self.assertFalse(bss_hostapd[1].list_sta())
+        bss_hostapd[0].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         wd.wait(5)
 
         # Now push the signal LOW, wait for iwd to attempt a roam, fail, and
-        # schedule another attempt for 60 seconds later
+        # schedule another attempt
         rule0.signal = -8000
 
         device.wait_for_event('no-roam-candidates')
 
         self.assertEqual(device.state, iwd.DeviceState.connected)
+
         self.assertTrue(bss_hostapd[0].list_sta())
         self.assertFalse(bss_hostapd[1].list_sta())
 
@@ -101,12 +101,11 @@ class Test(unittest.TestCase):
         wd.wait(1)
 
         # Assert low signal for BSS 0, check that iwd starts transition to BSS 1
-        # in less than 10 seconds.
         rule0.signal = -8000
         rule1.signal = -2000
 
         condition = 'obj.state == DeviceState.roaming'
-        wd.wait_for_object_condition(device, condition, max_wait=15)
+        wd.wait_for_object_condition(device, condition)
 
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
@@ -114,7 +113,7 @@ class Test(unittest.TestCase):
         to_condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_change(device, from_condition, to_condition)
 
-        self.assertTrue(bss_hostapd[1].list_sta())
+        bss_hostapd[1].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         device.disconnect()
 
autotests/testRoamRetry/stop_retry_test.py +3 -4
index f2f95990..3957d12d 100644
@@ -62,7 +62,7 @@ class Test(unittest.TestCase):
         devices = wd.list_devices(1)
         device = devices[0]
 
-        ordered_network = device.get_ordered_network('TestRoamRetry')
+        ordered_network = device.get_ordered_network('TestRoamRetry', full_scan=True)
 
         self.assertEqual(ordered_network.type, NetworkType.psk)
 
@@ -74,8 +74,7 @@ class Test(unittest.TestCase):
         condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_condition(device, condition)
 
-        self.assertTrue(bss_hostapd[0].list_sta())
-        self.assertFalse(bss_hostapd[1].list_sta())
+        bss_hostapd[0].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         # Now push the signal below the RSSI threshold and check that iwd
         # connects to BSS 1
@@ -90,7 +89,7 @@ class Test(unittest.TestCase):
         to_condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_change(device, from_condition, to_condition)
 
-        self.assertTrue(bss_hostapd[1].list_sta())
+        bss_hostapd[1].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         # Now make sure that we don't roam anymore. In order to catch this via
         # DeviceState, a suitable roaming target needs to be available. So jack
Signature · 1 line
2.34.1
author
#6 lore raw

PATCH v4 6/13 auto-t: use wait_for_event in testBSSBlacklist

In reply to James Prestwood

Depending on timing hostapd may not show a station with list_sta(). Instead use wait_for_event.

---
 autotests/testBSSBlacklist/connection_test.py | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
autotests/testBSSBlacklist/connection_test.py +9 -9
index 6a12174d..df3bd507 100644
@@ -33,7 +33,7 @@ class Test(unittest.TestCase):
 
         dev1, dev2 = wd.list_devices(2)
 
-        ordered_network = dev1.get_ordered_network("TestBlacklist")
+        ordered_network = dev1.get_ordered_network("TestBlacklist", full_scan=True)
 
         self.assertEqual(ordered_network.type, NetworkType.psk)
 
@@ -45,12 +45,12 @@ class Test(unittest.TestCase):
         condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_condition(dev1, condition)
 
-        self.assertIn(dev1.address, bss_hostapd[2].list_sta())
+        bss_hostapd[2].wait_for_event('AP-STA-CONNECTED %s' % dev1.address)
 
         # dev1 now connected, this should max out the first AP, causing the next
         # connection to fail to this AP.
 
-        ordered_network = dev2.get_ordered_network("TestBlacklist")
+        ordered_network = dev2.get_ordered_network("TestBlacklist", full_scan=True)
 
         self.assertEqual(ordered_network.type, NetworkType.psk)
 
@@ -64,7 +64,7 @@ class Test(unittest.TestCase):
 
         # We should have temporarily blacklisted the first BSS, and connected
         # to this one.
-        self.assertIn(dev2.address, bss_hostapd[1].list_sta())
+        bss_hostapd[1].wait_for_event('AP-STA-CONNECTED %s' % dev2.address)
 
         # Now check that the first BSS is still not blacklisted. We can
         # disconnect dev1, opening up the AP for more connections
@@ -83,7 +83,7 @@ class Test(unittest.TestCase):
         condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_condition(dev2, condition)
 
-        self.assertIn(dev2.address, bss_hostapd[2].list_sta())
+        bss_hostapd[2].wait_for_event('AP-STA-CONNECTED %s' % dev2.address)
 
         wd.unregister_psk_agent(psk_agent)
 
@@ -134,7 +134,7 @@ class Test(unittest.TestCase):
         condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_condition(device, condition)
 
-        self.assertIn(device.address, bss_hostapd[0].list_sta())
+        bss_hostapd[0].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         wd.unregister_psk_agent(psk_agent)
 
@@ -214,7 +214,7 @@ class Test(unittest.TestCase):
         # should automatically try the next BSS in the list, which is
         # bss_hostapd[1]
         self.assertNotIn(device.address, bss_hostapd[0].list_sta())
-        self.assertIn(device.address, bss_hostapd[1].list_sta())
+        bss_hostapd[1].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         rule0.drop = False
 
@@ -234,7 +234,7 @@ class Test(unittest.TestCase):
 
         # Same as before, make sure we didn't connect to the blacklisted AP.
         self.assertNotIn(device.address, bss_hostapd[0].list_sta())
-        self.assertIn(device.address, bss_hostapd[1].list_sta())
+        bss_hostapd[1].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         # Wait for the blacklist to expire (10 seconds)
         elapsed = time.time() - start
@@ -264,7 +264,7 @@ class Test(unittest.TestCase):
         condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_condition(device, condition)
 
-        self.assertIn(device.address, bss_hostapd[0].list_sta())
+        bss_hostapd[0].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         self.wd.unregister_psk_agent(psk_agent)
 
Signature · 1 line
2.34.1
author
#7 lore raw

PATCH v4 7/13 auto-t: use wait_for_event in testSAE-roam

In reply to James Prestwood

Depending on timing hostapd may not show a station with list_sta(). Instead use wait_for_event.

---
 autotests/testSAE-roam/connection_test.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
autotests/testSAE-roam/connection_test.py +7 -4
index 2409e0d1..4c49eaf8 100644
@@ -35,7 +35,8 @@ class Test(unittest.TestCase):
         condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_condition(device, condition)
 
-        self.assertTrue(self.bss_hostapd[0].list_sta())
+        self.bss_hostapd[0].wait_for_event('AP-STA-CONNECTED %s' % device.address)
+
         self.assertFalse(self.bss_hostapd[1].list_sta())
 
         testutil.test_iface_operstate(device.name)
@@ -54,7 +55,7 @@ class Test(unittest.TestCase):
         to_condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_change(device, from_condition, to_condition)
 
-        self.assertTrue(self.bss_hostapd[1].list_sta())
+        self.bss_hostapd[1].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         testutil.test_iface_operstate(device.name)
         testutil.test_ifaces_connected(self.bss_hostapd[1].ifname, device.name)
@@ -72,8 +73,10 @@ class Test(unittest.TestCase):
         condition = 'obj.state != DeviceState.roaming'
         wd.wait_for_object_condition(device, condition)
 
-        self.assertEqual(device.state, iwd.DeviceState.connected)
-        self.assertTrue(self.bss_hostapd[2].list_sta())
+        condition = 'obj.state == DeviceState.connected'
+        wd.wait_for_object_condition(device, condition)
+
+        self.bss_hostapd[2].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         testutil.test_iface_operstate(device.name)
         testutil.test_ifaces_connected(self.bss_hostapd[2].ifname, device.name)
Signature · 1 line
2.34.1
author
#8 lore raw

PATCH v4 8/13 auto-t: do full scan for testSAE

In reply to James Prestwood

This will ensure the IWD scans until the network is found.

---
 autotests/testSAE/connection_test.py | 14 +-------------
 1 file changed, 1 insertion(+), 13 deletions(-)
autotests/testSAE/connection_test.py +1 -13
index 75f5c12d..c67f8c83 100644
@@ -23,19 +23,7 @@ class Test(unittest.TestCase):
 
         device.disconnect()
 
-        wd.wait_for_object_condition(device, 'not obj.scanning')
-
-        device.debug_scan([2412])
-
-        wd.wait_for_object_condition(device, 'obj.scanning')
-        wd.wait_for_object_condition(device, 'not obj.scanning')
-
-        #
-        # An explicit scan was done prior due to hostapd options changing.
-        # Because of this scan_if_needed is set to False to avoid a redundant
-        # scan
-        #
-        network = device.get_ordered_network('ssidSAE', scan_if_needed=False)
+        network = device.get_ordered_network('ssidSAE', full_scan=True)
 
         self.assertEqual(network.type, NetworkType.psk)
 
Signature · 1 line
2.34.1
author
#9 lore raw

PATCH v4 9/13 auto-t: use wait_for_event in testPSK-roam

In reply to James Prestwood

Depending on timing hostapd may not show a station with list_sta() Instead use wait_for_event

---
 autotests/testPSK-roam/connection_test.py | 29 ++++-------------------
 1 file changed, 4 insertions(+), 25 deletions(-)
autotests/testPSK-roam/connection_test.py +4 -25
index 92fdeb04..f1b43130 100644
@@ -16,20 +16,7 @@ class Test(unittest.TestCase):
     def validate_connection(self, wd, over_ds=False):
         device = wd.list_devices(1)[0]
 
-        condition = 'not obj.scanning'
-        wd.wait_for_object_condition(device, condition)
-
-        # Scanning is unavoidable in this case since several hostapd
-        # configurations are tested and changed mid-test
-        device.scan()
-
-        condition = 'obj.scanning'
-        wd.wait_for_object_condition(device, condition)
-
-        condition = 'not obj.scanning'
-        wd.wait_for_object_condition(device, condition)
-
-        ordered_network = device.get_ordered_network('TestFT')
+        ordered_network = device.get_ordered_network('TestFT', full_scan=True)
 
         self.assertEqual(ordered_network.type, NetworkType.psk)
 
@@ -44,15 +31,7 @@ class Test(unittest.TestCase):
         condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_condition(device, condition)
 
-        self.assertTrue(self.bss_hostapd[0].list_sta())
-
-        # list_sta actually reports any authenticated stations. Due to the
-        # nature of FT-over-DS IWD should authenticate to all stations with
-        # the same mobility domain. This means both APs should show our station.
-        if over_ds:
-            self.assertTrue(self.bss_hostapd[1].list_sta())
-        else:
-            self.assertFalse(self.bss_hostapd[1].list_sta())
+        self.bss_hostapd[0].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         testutil.test_iface_operstate(device.name)
         testutil.test_ifaces_connected(self.bss_hostapd[0].ifname, device.name)
@@ -73,7 +52,7 @@ class Test(unittest.TestCase):
         to_condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_change(device, from_condition, to_condition)
 
-        self.assertTrue(self.bss_hostapd[1].list_sta())
+        self.bss_hostapd[1].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         testutil.test_iface_operstate(device.name)
         testutil.test_ifaces_connected(self.bss_hostapd[1].ifname, device.name)
@@ -93,7 +72,7 @@ class Test(unittest.TestCase):
         to_condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_change(device, from_condition, to_condition)
 
-        self.assertTrue(self.bss_hostapd[0].list_sta())
+        self.bss_hostapd[0].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         testutil.test_iface_operstate(device.name)
         testutil.test_ifaces_connected(self.bss_hostapd[0].ifname, device.name)
Signature · 1 line
2.34.1
author
#10 lore raw

PATCH v4 10/13 auto-t: do a full scan when getting network

In reply to James Prestwood

This ensures the network will be scanned for again if not found.

---
 autotests/testOWE-transition/connection_test.py | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)
autotests/testOWE-transition/connection_test.py +1 -12
index aeaea70a..ff5ea811 100644
@@ -20,18 +20,7 @@ class Test(unittest.TestCase):
 
         if not autoconnect:
             if not connect_hidden:
-                condition = 'not obj.scanning'
-                wd.wait_for_object_condition(device, condition)
-
-                device.scan()
-
-                condition = 'obj.scanning'
-                wd.wait_for_object_condition(device, condition)
-
-                condition = 'not obj.scanning'
-                wd.wait_for_object_condition(device, condition)
-
-                network = device.get_ordered_network('transition', scan_if_needed=False)
+                network = device.get_ordered_network('transition', full_scan=True)
 
                 network.network_object.connect()
             else:
Signature · 1 line
2.34.1
author
#11 lore raw

PATCH v4 11/13 dpp: fail if disconnected during DPP protocol

In reply to James Prestwood

If the connected BSS is NULL IWD was disconnected and this should trigger a protocol termination.

---
 src/dpp.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
src/dpp.c +11 -2
index f2421b69..de8e6e14 100644
@@ -1294,7 +1294,7 @@ static void dpp_presence_announce(struct dpp_sm *dpp)
 	dpp_send_frame(dpp, iov, 2, dpp->current_freq);
 }
 
-static void dpp_send_authenticate_request(struct dpp_sm *dpp)
+static bool dpp_send_authenticate_request(struct dpp_sm *dpp)
 {
 	uint8_t hdr[32];
 	uint8_t attrs[256];
@@ -1305,6 +1305,12 @@ static void dpp_send_authenticate_request(struct dpp_sm *dpp)
 	struct station *station = station_find(netdev_get_ifindex(dpp->netdev));
 	struct scan_bss *bss = station_get_connected_bss(station);
 
+	/* Got disconnected by the time the peer was discovered */
+	if (!bss) {
+		dpp_reset(dpp);
+		return false;
+	}
+
 	l_ecc_point_get_data(dpp->own_proto_public, i_proto_key,
 				sizeof(i_proto_key));
 
@@ -1337,6 +1343,8 @@ static void dpp_send_authenticate_request(struct dpp_sm *dpp)
 	iov[1].iov_len = ptr - attrs;
 
 	dpp_send_frame(dpp, iov, 2, dpp->current_freq);
+
+	return true;
 }
 
 static void dpp_roc_started(void *user_data)
@@ -1948,7 +1956,8 @@ static void dpp_handle_presence_announcement(struct dpp_sm *dpp,
 
 	dpp->state = DPP_STATE_AUTHENTICATING;
 
-	dpp_send_authenticate_request(dpp);
+	if (!dpp_send_authenticate_request(dpp))
+		return;
 
 	/*
 	 * Should we wait for an ACK then go offchannel?
Signature · 1 line
2.34.1
author
#12 lore raw

PATCH v4 12/13 dpp: set new_freq when configuring as initiator

In reply to James Prestwood

This is used to hold the current BSS frequency which will be used after IWD receives a presence announcement. Since this was not being set, the logic was always thinking there was a channel mismatch (0 != current_freq) and attempting to go offchannel to '0' which resulted in -EINVAL, and ultimately protocol termination.

---
 src/dpp.c | 2 ++
 1 file changed, 2 insertions(+)
src/dpp.c +2 -0
index de8e6e14..1719a89d 100644
@@ -2436,6 +2436,8 @@ static struct l_dbus_message *dpp_start_configurator_common(
 
 		if (!dpp->mcast_support)
 			dpp->state = DPP_STATE_AUTHENTICATING;
+
+		dpp->new_freq = bss->frequency;
 	} else
 		dpp->current_freq = bss->frequency;
 
Signature · 1 line
2.34.1
author
#13 lore raw

PATCH v4 13/13 auto-t: don't wait for wpa_supplicant to connect

In reply to James Prestwood

When configuring wpa_supplicant all we care about is that it received the configuration object. wpa_supplicant takes quite a bit of time to connect in some cases so waiting for that is unneeded.

This also increases the DPP timeout which may be required on slower systems or if the timing is particularly unlucky when receiving frames.

---
 autotests/testDPP/connection_test.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
autotests/testDPP/connection_test.py +3 -2
index b58620b8..ab3cbf95 100644
@@ -80,7 +80,7 @@ class Test(unittest.TestCase):
 
         self.wpas.dpp_enrollee_start(uri)
 
-        self.wpas.wait_for_event('DPP-CONF-RECEIVED')
+        self.wpas.wait_for_event('DPP-CONF-RECEIVED', timeout=30)
 
     def test_iwd_as_configurator_initiator(self):
         self.hapd.reload()
@@ -96,7 +96,7 @@ class Test(unittest.TestCase):
 
         self.device.dpp_start_configurator(uri)
 
-        self.hapd.wait_for_event('AP-STA-CONNECTED 42:00:00:00:00:00')
+        self.wpas.wait_for_event('DPP-CONF-RECEIVED', timeout=30)
 
     def setUp(self):
         self.wd = IWD(True)
@@ -118,6 +118,7 @@ class Test(unittest.TestCase):
         self.device.disconnect()
         self.device.dpp_stop()
         self.wpas.dpp_configurator_remove()
+        self.wpas.clean_up()
 
         self.wd = None
         self.device = None
Signature · 1 line
2.34.1
#14 lore raw
In reply to James Prestwood

Hi James,

Quoted 17 lines · James Prestwood

This combines the individual test fixes with the memory fixes

James Prestwood (13):
test-runner: add memory usage for debugging
test-runner: fix OOM issues (hopefully)
auto-t: remove waitstatus_to_exitcode for better compatibility
auto-t: iwd.py: fix full_scan argument
auto-t: fixup testRoamRetry for better timing reliability
auto-t: use wait_for_event in testBSSBlacklist
auto-t: use wait_for_event in testSAE-roam
auto-t: do full scan for testSAE
auto-t: use wait_for_event in testPSK-roam
auto-t: do a full scan when getting network
dpp: fail if disconnected during DPP protocol
dpp: set new_freq when configuring as initiator
auto-t: don't wait for wpa_supplicant to connect

Quoted 14 lines
  autotests/testBSSBlacklist/connection_test.py | 18 ++--
  autotests/testDPP/connection_test.py          |  5 +-
  autotests/testNetconfig/connection_test.py    |  5 +-
  .../testOWE-transition/connection_test.py     | 13 +--
  autotests/testPSK-roam/connection_test.py     | 29 +-----
  autotests/testRoamRetry/fast_retry_test.py    | 13 ++-
  autotests/testRoamRetry/stop_retry_test.py    |  7 +-
  autotests/testSAE-roam/connection_test.py     | 11 ++-
  autotests/testSAE/connection_test.py          | 14 +--
  autotests/util/iwd.py                         | 23 ++---
  src/dpp.c                                     | 15 +++-
  tools/test-runner                             | 89 +++++++++++++++++--
  12 files changed, 142 insertions(+), 100 deletions(-)

All applied, thanks.

Regards,
-Denis