Skip to contentSkip to conversation

hwlatdetect: disable/enable c-state transitions during detection

PATCH In review Clark Williams 2 participants 2 messages May 6, 2019 – May 9, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 hwlatdetect: disable/enable c-state transitions during detection

Recent performance tuning problems led me to realize that just running at fifo:99 and turning off interrupts isn't enough while looking for BIOS induced latencies. Power savings logic is built into most modern cpus and so must be disabled while looking for BIOS induced (SMI/NMI) latencies.

Use the /dev/cpu_dma_latency mechanism to disable c-state transitions while running the hardware latency detector. Open the file /dev/cpu_dma_latency and write a 32-bit zero to it, which will prevent c-state transitions while the file is open.

---
 src/hwlatdetect/hwlatdetect.py | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)
src/hwlatdetect/hwlatdetect.py +19 -1
index 2c8f9f160419..368079a158b1 100755
@@ -1,6 +1,6 @@
 #!/usr/bin/python3
 
-# (C) 2018 Clark Williams <[email protected]>
+# (C) 2018,2019 Clark Williams <[email protected]>
 # (C) 2015,2016 Clark Williams <[email protected]>
 # (C) 2009 Clark Williams <[email protected]>
 #
@@ -213,6 +213,22 @@ watch = False
             counts = [ int(x.strip()) for x in p.stdout.readlines()]
         return counts
 
+    # methods for preventing/enabling c-state transitions
+    # openinging /dev/cpu_dma_latency and writeing a 32-bit zero to that file will prevent
+    # c-state transitions while the file descriptor is open.
+    # use c_states_off() to disable c-state transitions
+    # use c_states_on() to close the file descriptor and re-enable c-states
+    #
+    def c_states_off(self):
+        self.dma_latency_handle = os.open("/dev/cpu_dma_latency", os.O_WRONLY)
+        os.write(self.dma_latency_handle, b'\x00\x00\x00\x00')
+        debug("c-states disabled")
+
+    def c_states_on(self):
+        if self.dma_latency_handle:
+            os.close(self.dma_latency_handle)
+        debug("c-states enabled")
+
     def cleanup(self):
         raise RuntimeError("must override base method 'cleanup'!")
 
@@ -235,6 +251,7 @@ watch = False
     def start(self):
         count = 0
         threshold = int(self.get("threshold"))
+        self.c_states_off()
         debug("enabling detector module (threshold: %d)" % threshold)
         self.set("enable", 1)
         while self.get("enable") == 0:
@@ -258,6 +275,7 @@ watch = False
             time.sleep(0.1)
             debug("retrying disable of detector module(%d)" % count)
             self.set("enable", 0)
+        self.c_states_on()
         debug("detector module disabled")
 
     def detect(self):
Signature · 1 line
2.21.0
#1 lore raw
Quoted 71 lines · Clark Williams

Recent performance tuning problems led me to realize that just running at fifo:99 and turning off interrupts isn't enough while looking for BIOS induced latencies. Power savings logic is built into most modern cpus and so must be disabled while looking for BIOS induced (SMI/NMI) latencies.

Use the /dev/cpu_dma_latency mechanism to disable c-state transitions while running the hardware latency detector. Open the file /dev/cpu_dma_latency and write a 32-bit zero to it, which will prevent c-state transitions while the file is open.

---
 src/hwlatdetect/hwlatdetect.py | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)
src/hwlatdetect/hwlatdetect.py +19 -1
index 2c8f9f160419..368079a158b1 100755
@@ -1,6 +1,6 @@
 #!/usr/bin/python3
 
-# (C) 2018 Clark Williams <[email protected]>
+# (C) 2018,2019 Clark Williams <[email protected]>
 # (C) 2015,2016 Clark Williams <[email protected]>
 # (C) 2009 Clark Williams <[email protected]>
 #
@@ -213,6 +213,22 @@ watch = False
             counts = [ int(x.strip()) for x in p.stdout.readlines()]
         return counts
 
+    # methods for preventing/enabling c-state transitions
+    # openinging /dev/cpu_dma_latency and writeing a 32-bit zero to that file will prevent
+    # c-state transitions while the file descriptor is open.
+    # use c_states_off() to disable c-state transitions
+    # use c_states_on() to close the file descriptor and re-enable c-states
+    #
+    def c_states_off(self):
+        self.dma_latency_handle = os.open("/dev/cpu_dma_latency", os.O_WRONLY)
+        os.write(self.dma_latency_handle, b'\x00\x00\x00\x00')
+        debug("c-states disabled")
+
+    def c_states_on(self):
+        if self.dma_latency_handle:
+            os.close(self.dma_latency_handle)
+        debug("c-states enabled")
+
     def cleanup(self):
         raise RuntimeError("must override base method 'cleanup'!")
 
@@ -235,6 +251,7 @@ watch = False
     def start(self):
         count = 0
         threshold = int(self.get("threshold"))
+        self.c_states_off()
         debug("enabling detector module (threshold: %d)" % threshold)
         self.set("enable", 1)
         while self.get("enable") == 0:
@@ -258,6 +275,7 @@ watch = False
             time.sleep(0.1)
             debug("retrying disable of detector module(%d)" % count)
             self.set("enable", 0)
+        self.c_states_on()
         debug("detector module disabled")
 
     def detect(self):
Signature · 1 line
2.21.0