Skip to contentSkip to conversation

auto-t: fix unreliable behavior in testPSK-roam

PATCH In review James Prestwood 2 participants 7 messages Aug 22, 2022

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/6 auto-t: fix unreliable behavior in testPSK-roam

The packet loss test had a few problems. First being that the RSSI for the original BSS was not low enough to change the rank. This meant any roam was just lucky that the intended BSS was first in the results.

The second problem is timing related, and only happens on UML. Disabling the rules after the roaming condition sometimes allows IWD to fully roam and connect before the next state change checks.

---
 autotests/testPSK-roam/connection_test.py | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
autotests/testPSK-roam/connection_test.py +5 -5
index c926e255..364f75c2 100644
@@ -55,16 +55,16 @@ class Test(unittest.TestCase):
         condition = 'obj.state == DeviceState.roaming'
         wd.wait_for_object_condition(device, condition)
 
-        if pkt_loss:
-            self.rule1.enabled = False
-            self.rule2.enabled = False
-
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
         from_condition = 'obj.state == DeviceState.roaming'
         to_condition = 'obj.state == DeviceState.connected'
         wd.wait_for_object_change(device, from_condition, to_condition)
 
+        if pkt_loss:
+            self.rule1.enabled = False
+            self.rule2.enabled = False
+
         self.bss_hostapd[1].wait_for_event('AP-STA-CONNECTED %s' % device.address)
 
         testutil.test_iface_operstate(device.name)
@@ -197,7 +197,7 @@ class Test(unittest.TestCase):
 
         cls.rule2 = hwsim.rules.create()
         cls.rule2.source = rad0.addresses[0]
-        cls.rule2.signal = -4000
+        cls.rule2.signal = -7000
 
         # Set interface addresses to those expected by hostapd config files
         os.system('ip link set dev "' + cls.bss_hostapd[0].ifname + '" down')
Signature · 1 line
2.34.3
#1 lore raw

PATCH v4 2/6 test-runner: double RAM for --valgrind

It seems 256MB was right on the edge if valgrind was being used and sometimes the test would fail with OOM exceptions.

---
 tools/runner.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/runner.py +5 -1
index a96627de..004bf46d 100644
@@ -370,6 +370,7 @@ class QemuRunner(RunnerAbstract):
 
 		usb_adapters = None
 		pci_adapters = None
+		ram = 256
 
 		super().__init__(args)
 
@@ -400,11 +401,14 @@ class QemuRunner(RunnerAbstract):
 
 		kern_log = "ignore_loglevel" if "kernel" in args.verbose else "quiet"
 
+		if args.valgrind:
+			ram *= 2
+
 		qemu_cmdline = [
 			'qemu-system-x86_64',
 			'-machine', 'type=q35,accel=kvm:tcg',
 			'-nodefaults', '-no-user-config', '-monitor', 'none',
-			'-display', 'none', '-m', '256M', '-nographic', '-vga',
+			'-display', 'none', '-m', '%dM' % ram, '-nographic', '-vga',
 			'none', '-no-acpi', '-no-hpet',
 			'-no-reboot', '-fsdev',
 			'local,id=fsdev-root,path=/,readonly=on,security_model=none,multidevs=remap',
Signature · 1 line
2.34.3
#2 lore raw

PATCH v4 3/6 auto-t: hostapd: add set_address/group_neighbors

In reply to James Prestwood

This adds a few utilities for setting up an FT environment. All the roaming tests basically copy/paste the same code for setting up the hostapd instances and this can cause problems if not done correctly.

set_address() sets the MAC address on the device, and restarts hostapd group_neighbors() takes a list of HostapdCLI objects and makes each a
neighbor to the others.

The neighbor report element requires the operating class which isn't advertised by hostapd. For this we assume operating class 81 but this can be set explicitly if it differs. Currently no roaming tests use 5/6GHz frequencies, and just in case an exception will be thrown if the channel is greater than 14 and the op_class didn't change.

---
 autotests/util/hostapd.py | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
autotests/util/hostapd.py +31 -0
index 3ae8ff89..d6c13e2b 100644
@@ -281,3 +281,34 @@ class HostapdCLI(object):
     @property
     def enabled(self):
         return self._get_status()['state'] == 'ENABLED'
+
+    def set_address(self, mac):
+        os.system('ip link set dev %s down' % self.ifname)
+        os.system('ip link set dev %s addr %s up' % (self.ifname, mac))
+
+        self.reload()
+        self.wait_for_event("AP-ENABLED")
+
+    def _add_neighbors(self, *args, op_class=81):
+        for hapd in args:
+            status = hapd._get_status()
+
+            ssid = status['ssid[0]']
+            bssid = status['bssid[0]']
+            channel = int(status['channel'])
+
+            if (channel > 14 and op_class == 81):
+                raise Exception("default add_neighbors assumes opclass 0x51!")
+
+            channel = '{:02x}'.format(channel)
+            oper_class = '{:02x}'.format(op_class)
+
+            self.set_neighbor(bssid, ssid, '%s8f000000%s%s060603000000' %
+                                (bssid.replace(':', ''), oper_class, channel))
+
+    @classmethod
+    def group_neighbors(cls, *args):
+        for hapd in args:
+            others = [h for h in args if h != hapd]
+
+            hapd._add_neighbors(*others)
Signature · 1 line
2.34.3
#3 lore raw

PATCH v4 4/6 auto-t: update tests to use set_address/group_neighbors

In reply to James Prestwood

This reduces the big blocks of copied code with a few function calls.

---
 .../testFT-8021x-roam/connection_test.py      | 26 ++----------
 autotests/testFT-FILS/connection_test.py      | 24 ++---------
 autotests/testPSK-roam/connection_test.py     | 24 ++---------
 .../testPSK-roam/roam_ap_disconnect_test.py   | 12 ++----
 autotests/testPreauth-roam/connection_test.py | 13 +-----
 autotests/testRoamRetry/fast_retry_test.py    | 15 +------
 autotests/testRoamRetry/stop_retry_test.py    | 15 +------
 autotests/testSAE-roam/connection_test.py     | 42 +++----------------
 8 files changed, 23 insertions(+), 148 deletions(-)
autotests/testFT-8021x-roam/connection_test.py +4 -22
index 6853bdb0..d3f8e2c6 100644
@@ -66,28 +66,10 @@ class Test(unittest.TestCase):
         cls.bss_hostapd = [ HostapdCLI(config='ft-eap-ccmp-1.conf'),
                             HostapdCLI(config='ft-eap-ccmp-2.conf') ]
 
-        # Set interface addresses to those expected by hostapd config files
-        os.system('ip link set dev "' + cls.bss_hostapd[0].ifname + '" down')
-        os.system('ip link set dev "' + cls.bss_hostapd[0].ifname + \
-                  '" address 12:00:00:00:00:01 up')
-        os.system('ip link set dev "' + cls.bss_hostapd[1].ifname + '" down')
-        os.system('ip link set dev "' + cls.bss_hostapd[1].ifname + \
-                  '" address 12:00:00:00:00:02 up')
-
-        cls.bss_hostapd[0].reload()
-        cls.bss_hostapd[0].wait_for_event("AP-ENABLED")
-        cls.bss_hostapd[1].reload()
-        cls.bss_hostapd[1].wait_for_event("AP-ENABLED")
-
-        # Fill in the neighbor AP tables in both BSSes.  By default each
-        # instance knows only about current BSS, even inside one hostapd
-        # process.
-        # FT still works without the neighbor AP table but neighbor reports
-        # have to be disabled in the .conf files
-        cls.bss_hostapd[0].set_neighbor('12:00:00:00:00:02', 'TestFT',
-                '1200000000028f0000005102060603000000')
-        cls.bss_hostapd[1].set_neighbor('12:00:00:00:00:01', 'TestFT',
-                '1200000000018f0000005101060603000000')
+        cls.bss_hostapd[0].set_address('12:00:00:00:00:01')
+        cls.bss_hostapd[1].set_address('12:00:00:00:00:02')
+
+        HostapdCLI.group_neighbors(*cls.bss_hostapd)
 
     @classmethod
     def tearDownClass(cls):
autotests/testFT-FILS/connection_test.py +4 -20
index c1deb77c..b1ec42e0 100644
@@ -128,26 +128,10 @@ class Test(unittest.TestCase):
         cls.bss_hostapd = [ HostapdCLI(config='ft-eap-ccmp-1.conf'),
                             HostapdCLI(config='ft-eap-ccmp-2.conf') ]
 
-        # Set interface addresses to those expected by hostapd config files
-        os.system('ip link set dev "' + cls.bss_hostapd[0].ifname + '" down')
-        os.system('ip link set dev "' + cls.bss_hostapd[0].ifname + '" addr 12:00:00:00:00:01 up')
-        os.system('ip link set dev "' + cls.bss_hostapd[1].ifname + '" down')
-        os.system('ip link set dev "' + cls.bss_hostapd[1].ifname + '" addr 12:00:00:00:00:02 up')
-
-        cls.bss_hostapd[0].reload()
-        cls.bss_hostapd[0].wait_for_event("AP-ENABLED")
-        cls.bss_hostapd[1].reload()
-        cls.bss_hostapd[1].wait_for_event("AP-ENABLED")
-
-        # Fill in the neighbor AP tables in both BSSes.  By default each
-        # instance knows only about current BSS, even inside one hostapd
-        # process.
-        # FT still works without the neighbor AP table but neighbor reports
-        # have to be disabled in the .conf files
-        cls.bss_hostapd[0].set_neighbor('12:00:00:00:00:02', 'TestFT',
-                '1200000000028f0000005102060603000000')
-        cls.bss_hostapd[1].set_neighbor('12:00:00:00:00:01', 'TestFT',
-                '1200000000018f0000005101060603000000')
+        cls.bss_hostapd[0].set_address('12:00:00:00:00:01')
+        cls.bss_hostapd[1].set_address('12:00:00:00:00:02')
+
+        HostapdCLI.group_neighbors(*cls.bss_hostapd)
 
     @classmethod
     def tearDownClass(cls):
autotests/testPSK-roam/connection_test.py +4 -20
index 364f75c2..2206f120 100644
@@ -199,26 +199,10 @@ class Test(unittest.TestCase):
         cls.rule2.source = rad0.addresses[0]
         cls.rule2.signal = -7000
 
-        # Set interface addresses to those expected by hostapd config files
-        os.system('ip link set dev "' + cls.bss_hostapd[0].ifname + '" down')
-        os.system('ip link set dev "' + cls.bss_hostapd[0].ifname + '" addr 12:00:00:00:00:01 up')
-        os.system('ip link set dev "' + cls.bss_hostapd[1].ifname + '" down')
-        os.system('ip link set dev "' + cls.bss_hostapd[1].ifname + '" addr 12:00:00:00:00:02 up')
-
-        cls.bss_hostapd[0].reload()
-        cls.bss_hostapd[0].wait_for_event("AP-ENABLED")
-        cls.bss_hostapd[1].reload()
-        cls.bss_hostapd[1].wait_for_event("AP-ENABLED")
-
-        # Fill in the neighbor AP tables in both BSSes.  By default each
-        # instance knows only about current BSS, even inside one hostapd
-        # process.
-        # FT still works without the neighbor AP table but neighbor reports
-        # have to be disabled in the .conf files
-        cls.bss_hostapd[0].set_neighbor('12:00:00:00:00:02', 'TestFT',
-                '1200000000028f0000005102060603000000')
-        cls.bss_hostapd[1].set_neighbor('12:00:00:00:00:01', 'TestFT',
-                '1200000000018f0000005101060603000000')
+        cls.bss_hostapd[0].set_address('12:00:00:00:00:01')
+        cls.bss_hostapd[1].set_address('12:00:00:00:00:02')
+
+        HostapdCLI.group_neighbors(*cls.bss_hostapd)
 
     @classmethod
     def tearDownClass(cls):
autotests/testPSK-roam/roam_ap_disconnect_test.py +3 -9
index 760c25fa..416541ce 100644
@@ -46,10 +46,7 @@ class Test(unittest.TestCase):
         self.validate()
 
     def test_ap_disconnect_neighbors(self):
-        self.bss_hostapd[0].set_neighbor('12:00:00:00:00:02', 'TestFT',
-                '1200000000028f0000005102060603000000')
-        self.bss_hostapd[1].set_neighbor('12:00:00:00:00:01', 'TestFT',
-                '1200000000018f0000005101060603000000')
+        HostapdCLI.group_neighbors(*self.bss_hostapd)
 
         self.validate()
 
@@ -85,11 +82,8 @@ class Test(unittest.TestCase):
         cls.rule0.signal = -8000
         cls.rule0.enabled = True
 
-        # Set interface addresses to those expected by hostapd config files
-        os.system('ip link set dev "' + cls.bss_hostapd[0].ifname + '" down')
-        os.system('ip link set dev "' + cls.bss_hostapd[0].ifname + '" addr 12:00:00:00:00:01 up')
-        os.system('ip link set dev "' + cls.bss_hostapd[1].ifname + '" down')
-        os.system('ip link set dev "' + cls.bss_hostapd[1].ifname + '" addr 12:00:00:00:00:02 up')
+        cls.bss_hostapd[0].set_address('12:00:00:00:00:01')
+        cls.bss_hostapd[1].set_address('12:00:00:00:00:02')
 
     @classmethod
     def tearDownClass(cls):
autotests/testPreauth-roam/connection_test.py +1 -12
index 26475698..3c8fa0bb 100644
@@ -17,18 +17,7 @@ class Test(unittest.TestCase):
         bss0_addr = bss_hostapd[0].bssid
         bss1_addr = bss_hostapd[1].bssid
 
-        # Fill in the neighbor AP tables in both BSSes.  By default each
-        # instance knows only about current BSS, even inside one hostapd
-        # process.
-        # Roaming still works without the neighbor AP table but neighbor
-        # reports have to be disabled in the .conf files
-        bss0_nr = ''.join(bss0_addr.split(':')) + \
-                '8f0000005101060603000000'
-        bss1_nr = ''.join(bss1_addr.split(':')) + \
-                '8f0000005102060603000000'
-
-        bss_hostapd[0].set_neighbor(bss1_addr, 'TestPreauth', bss1_nr)
-        bss_hostapd[1].set_neighbor(bss0_addr, 'TestPreauth', bss0_nr)
+        HostapdCLI.group_neighbors(*bss_hostapd)
 
         wd = IWD(True)
 
autotests/testRoamRetry/fast_retry_test.py +1 -14
index b45daada..373f3d45 100644
@@ -36,20 +36,7 @@ class Test(unittest.TestCase):
         rule1.bidirectional = True
         rule1.enabled = True
 
-        # Fill in the neighbor AP tables in both BSSes.  By default each
-        # instance knows only about current BSS, even inside one hostapd
-        # process.
-        # Roaming still works without the neighbor AP table but neighbor
-        # reports have to be disabled in the .conf files
-        bss0_nr = ''.join(bss_radio[0].addresses[0].split(':')) + \
-                '8f0000005101060603000000'
-        bss1_nr = ''.join(bss_radio[1].addresses[0].split(':')) + \
-                '8f0000005102060603000000'
-
-        bss_hostapd[0].set_neighbor(bss_radio[1].addresses[0], 'TestRoamRetry',
-                bss1_nr)
-        bss_hostapd[1].set_neighbor(bss_radio[0].addresses[0], 'TestRoamRetry',
-                bss0_nr)
+        HostapdCLI.group_neighbors(*bss_hostapd)
 
         # Start in the vicinity of BSS 0, check that iwd connects to BSS 0
         rule0.signal = -2000
autotests/testRoamRetry/stop_retry_test.py +1 -14
index 3957d12d..b5eb6f77 100644
@@ -35,20 +35,7 @@ class Test(unittest.TestCase):
         rule1.bidirectional = True
         rule1.enabled = True
 
-        # Fill in the neighbor AP tables in both BSSes.  By default each
-        # instance knows only about current BSS, even inside one hostapd
-        # process.
-        # Roaming still works without the neighbor AP table but neighbor
-        # reports have to be disabled in the .conf files
-        bss0_nr = ''.join(bss_radio[0].addresses[0].split(':')) + \
-                '8f0000005101060603000000'
-        bss1_nr = ''.join(bss_radio[1].addresses[0].split(':')) + \
-                '8f0000005102060603000000'
-
-        bss_hostapd[0].set_neighbor(bss_radio[1].addresses[0], 'TestRoamRetry',
-                bss1_nr)
-        bss_hostapd[1].set_neighbor(bss_radio[0].addresses[0], 'TestRoamRetry',
-                bss0_nr)
+        HostapdCLI.group_neighbors(*bss_hostapd)
 
         # Start in the vicinity of BSS 0, check that iwd connects to BSS 0
         rule0.signal = -2000
autotests/testSAE-roam/connection_test.py +5 -37

48 lines not on this page · raw message

Signature · 1 line
2.34.3
#4 lore raw

PATCH v4 5/6 auto-t: remove bringing up lo interface

In reply to James Prestwood

Several tests were setting lo up which is already done by test-runner.

---
 autotests/testFILS/fils_256_test.py           | 2 --
 autotests/testFILS/fils_384_test.py           | 2 --
 autotests/testFT-FILS/connection_test.py      | 1 -
 autotests/testPreauth-roam/connection_test.py | 2 --
 tools/run-tests                               | 3 ---
 5 files changed, 10 deletions(-)
autotests/testFILS/fils_256_test.py +0 -2
index 5fa371e1..7018f0f2 100644
@@ -62,8 +62,6 @@ class Test(unittest.TestCase):
     @classmethod
     def setUpClass(cls):
         IWD.copy_to_storage('ssidFILS-256.8021x')
-        os.system('ip link set lo up')
-        pass
 
     @classmethod
     def tearDownClass(cls):
autotests/testFILS/fils_384_test.py +0 -2
index b7174418..ce8904df 100644
@@ -62,8 +62,6 @@ class Test(unittest.TestCase):
     @classmethod
     def setUpClass(cls):
         IWD.copy_to_storage('ssidFILS-384.8021x')
-        os.system('ip link set lo up')
-        pass
 
     @classmethod
     def tearDownClass(cls):
autotests/testFT-FILS/connection_test.py +0 -1
index b1ec42e0..5d60bf4d 100644
@@ -122,7 +122,6 @@ class Test(unittest.TestCase):
 
     @classmethod
     def setUpClass(cls):
-        os.system('ip link set lo up')
         IWD.copy_to_storage('TestFT.8021x')
 
         cls.bss_hostapd = [ HostapdCLI(config='ft-eap-ccmp-1.conf'),
autotests/testPreauth-roam/connection_test.py +0 -2
index 3c8fa0bb..02d03361 100644
@@ -75,8 +75,6 @@ class Test(unittest.TestCase):
     def setUpClass(cls):
         IWD.copy_to_storage('TestPreauth.8021x')
 
-        os.system('ip link set lo up')
-
     @classmethod
     def tearDownClass(cls):
         IWD.clear_storage()
tools/run-tests +0 -3
index e45ffe46..9620ed1f 100755
@@ -487,8 +487,6 @@ class TestContext(Namespace):
 			print("Ofono or Phonesim not found, skipping test")
 			return
 
-		Process(['ip', 'link', 'set', 'lo', 'up']).wait()
-
 		os.environ['OFONO_PHONESIM_CONFIG'] = '/tmp/phonesim.conf'
 
 		phonesim_args = ['phonesim', '-p', '12345', '/usr/share/phonesim/default.xml']
@@ -889,7 +887,6 @@ def post_test(ctx, to_copy):
 			elif os.path.exists('/tmp/' + f):
 				os.remove('/tmp/' + f)
 
-		Process(['ip', 'link', 'set', 'lo', 'down']).wait()
 	except Exception as e:
 		print("Exception thrown in post_test")
 	finally:
Signature · 1 line
2.34.3
#5 lore raw

PATCH v4 6/6 auto-t: fix testNetconfig static test

In reply to James Prestwood

This test was updated to add namespaces which actually 'fixed' dhcpd's address conflict detection since it uses ICMP echo requests. Part of this test assumed the DHCP server would not detect a conflict which isn't true for dhcpd (and any decent DHCP server I would guess). This check now fails since dhcpd detects a conflict (the static address) and hands out the next available address.

Testing conflict detection like this is really only testing dhcpd behavior, and however IWD behaves here isn't important since the network is doomed from the start.

---
 autotests/testNetconfig/static_test.py | 28 --------------------------
 1 file changed, 28 deletions(-)
autotests/testNetconfig/static_test.py +0 -28
index 01d694ca..aac2adcc 100644
@@ -20,17 +20,10 @@ class Test(unittest.TestCase):
         # Use a non-default storage_dir for one of the instances, the default for the other one
         wd = IWD(True, iwd_storage_dir='/tmp/storage')
 
-        ns0 = ctx.get_namespace('ns0')
-
-        wd_ns0 = IWD(True, namespace=ns0)
-
         psk_agent = PSKAgent("secret123")
-        psk_agent_ns0 = PSKAgent("secret123", namespace=ns0)
         wd.register_psk_agent(psk_agent)
-        wd_ns0.register_psk_agent(psk_agent_ns0)
 
         dev1 = wd.list_devices(1)[0]
-        dev2 = wd_ns0.list_devices(1)[0]
 
         ordered_network = dev1.get_ordered_network('ap-main')
 
@@ -80,28 +73,7 @@ class Test(unittest.TestCase):
         # of the log since we care about the end result here.
         self.assertEqual(expected_rclog, entries[-3:])
 
-        ordered_network = dev2.get_ordered_network('ap-main')
-
-        condition = 'not obj.connected'
-        wd_ns0.wait_for_object_condition(ordered_network.network_object, condition)
-
-        # Connect to the same network from a dynamically configured client.  The
-        # DHCP server doesn't know (even though dev1 announced itself) that
-        # 192.168.1.10 is already in use and if it assigns dev2 the lowest
-        # available address, that's going to be 192.168.1.10.  dev1's ACD
-        # implementation should then stop using this address.
-        ordered_network.network_object.connect()
-
-        condition = 'obj.state == DeviceState.connected'
-        wd_ns0.wait_for_object_condition(dev2, condition)
-
-        wd.wait(1)
-        # Check dev1 is now disconnected or without its IPv4 address
-        if dev1.state == iwd.DeviceState.connected:
-            testutil.test_ip_address_match(dev1.name, None)
-
         dev1.disconnect()
-        dev2.disconnect()
 
         condition = 'not obj.connected'
         wd.wait_for_object_condition(ordered_network.network_object, condition)
Signature · 1 line
2.34.3
#6 lore raw
In reply to James Prestwood

Hi James,

Quoted 10 lines · James Prestwood

The packet loss test had a few problems. First being that the RSSI for the original BSS was not low enough to change the rank. This meant any roam was just lucky that the intended BSS was first in the results.

The second problem is timing related, and only happens on UML. Disabling the rules after the roaming condition sometimes allows IWD to fully roam and connect before the next state change checks.

---
  autotests/testPSK-roam/connection_test.py | 10 +++++-----
  1 file changed, 5 insertions(+), 5 deletions(-)

I went ahead and applied patches 1-5 in this series. Andrew still has some comments pending on patch 6.

Regards,
-Denis