Skip to contentSkip to conversation

rt-tests: hwlatdetect: Update to integer division

PATCH In review Leah Leshchinsky 2 participants 2 messages Nov 10, 2022 – Nov 11, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 rt-tests: hwlatdetect: Update to integer division

In Python 3, "/" is a float division operator, as opposed to Python 2, which defaults to integer division. This results in an error when calculating width, which assumes an integer.

Update width division to integer division with the "//" operator.

src/hwlatdetect/hwlatdetect.py +3 -2
index 7b1ae646577a..929107f9c252 100755
@@ -454,9 +454,10 @@ if __name__ == '__main__':
 
     if args.window:
         w = microseconds(args.window)
+        width = w//2
         if w < int(detect.get("width")):
-            debug(f"shrinking width to {w//2} for new window of {w}")
-            detect.set("width", w/2)
+            debug(f"shrinking width to {width} for new window of {w}")
+            detect.set("width", width)
         debug(f"window parameter = {w}")
         detect.set("window", w)
         debug(f"window for sampling set to {w}us")
Signature · 1 line
2.31.1
#1 lore raw
Quoted 3 lines · Leah Leshchinsky

In Python 3, "/" is a float division operator, as opposed to Python 2, which defaults to integer division. This results in an error when calculating width, which assumes an integer.

What kind of error? Is there a traceback, or merely an incorrect value?

Quoted 13 lines

Update width division to integer division with the "//" operator.

src/hwlatdetect/hwlatdetect.py +0 -0
index 7b1ae646577a..929107f9c252 100755
@@ -454,9 +454,10 @@ if __name__ == '__main__':
 
     if args.window:

src/hwlatdetect/hwlatdetect.py if __name__ == '__main__': · line 463

Quoted 10 lines
src/hwlatdetect/hwlatdetect.py +3 -2
         w = microseconds(args.window)
+        width = w//2
         if w < int(detect.get("width")):
-            debug(f"shrinking width to {w//2} for new window of {w}")
-            detect.set("width", w/2)
+            debug(f"shrinking width to {width} for new window of {w}")
+            detect.set("width", width)
         debug(f"window parameter = {w}")
         detect.set("window", w)
         debug(f"window for sampling set to {w}us")