Skip to contentSkip to conversation

xen: debugtrace cleanup and per-cpu buffer support

PATCH Superseded Juergen Gross 1 participant 5 messages Aug 1, 2019

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v2 cover xen: debugtrace cleanup and per-cpu buffer support

Another debugtrace enhancement I needed for core scheduling debugging, plus some cleanup.

Changes in V2:
- added new patch 1 (preparing the move of debugtrace coding)
- patch 4 (v1 patch 3): avoid leaking buffer

Juergen Gross (4):
xen: use common output function in debugtrace
xen: move debugtrace coding to common/debugtrace.c
xen: refactor debugtrace data
xen: add per-cpu buffer option to debugtrace

 docs/misc/xen-command-line.pandoc |   7 +-
 xen/common/Makefile               |   1 +
 xen/common/debugtrace.c           | 262 ++++++++++++++++++++++++++++++++++++++
 xen/drivers/char/console.c        | 182 +-------------------------
 xen/include/xen/console.h         |   3 +
 5 files changed, 275 insertions(+), 180 deletions(-)
 create mode 100644 xen/common/debugtrace.c
Signature · 7 lines
2.16.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH v2 1/4 xen: use common output function in debugtrace

Today dumping the debugtrace buffers is done via sercon_puts(), while direct printing of trace entries (after toggling output to the console) is using serial_puts().

Use sercon_puts() in both cases, as the difference between both is not really making sense.

In order to prepare moving debugtrace functionality to an own source file rename sercon_puts() to console_serial_puts() and make it globally visible.

---
 xen/drivers/char/console.c | 16 ++++++++--------
 xen/include/xen/console.h  |  3 +++
 2 files changed, 11 insertions(+), 8 deletions(-)
xen/drivers/char/console.c +8 -8
index d728e737d1..49fbbed21a 100644
@@ -346,7 +346,7 @@ void console_giveback(int id)
         serial_steal_fn = NULL;
 }
 
-static void sercon_puts(const char *s)
+void console_serial_puts(const char *s)
 {
     if ( serial_steal_fn != NULL )
         (*serial_steal_fn)(s);
@@ -389,7 +389,7 @@ static void dump_console_ring_key(unsigned char key)
     }
     buf[sofar] = '\0';
 
-    sercon_puts(buf);
+    console_serial_puts(buf);
     video_puts(buf);
 
     free_xenheap_pages(buf, order);
@@ -548,7 +548,7 @@ static long guest_console_write(XEN_GUEST_HANDLE_PARAM(char) buffer, int count)
             /* Use direct console output as it could be interactive */
             spin_lock_irq(&console_lock);
 
-            sercon_puts(kbuf);
+            console_serial_puts(kbuf);
             video_puts(kbuf);
 
 #ifdef CONFIG_X86
@@ -669,7 +669,7 @@ static void __putstr(const char *str)
 {
     ASSERT(spin_is_locked(&console_lock));
 
-    sercon_puts(str);
+    console_serial_puts(str);
     video_puts(str);
 
 #ifdef CONFIG_X86
@@ -1182,11 +1182,11 @@ static void debugtrace_dump_worker(void)
 
     /* Print oldest portion of the ring. */
     ASSERT(debugtrace_buf[debugtrace_bytes - 1] == 0);
-    sercon_puts(&debugtrace_buf[debugtrace_prd]);
+    console_serial_puts(&debugtrace_buf[debugtrace_prd]);
 
     /* Print youngest portion of the ring. */
     debugtrace_buf[debugtrace_prd] = '\0';
-    sercon_puts(&debugtrace_buf[0]);
+    console_serial_puts(&debugtrace_buf[0]);
 
     memset(debugtrace_buf, '\0', debugtrace_bytes);
 
@@ -1267,8 +1267,8 @@ void debugtrace_printk(const char *fmt, ...)
     if ( debugtrace_send_to_console )
     {
         snprintf(cntbuf, sizeof(cntbuf), "%u ", ++count);
-        serial_puts(sercon_handle, cntbuf);
-        serial_puts(sercon_handle, buf);
+        console_serial_puts(cntbuf);
+        console_serial_puts(buf);
     }
     else
     {
xen/include/xen/console.h +3 -0
index b4f9463936..041522692f 100644
@@ -46,6 +46,9 @@ void console_giveback(int id);
 int console_suspend(void);
 int console_resume(void);
 
+/* Emit a string via the serial console. */
+void console_serial_puts(const char *s);
+
 extern int8_t opt_console_xen;
 
 #endif /* __CONSOLE_H__ */
Signature · 7 lines
2.16.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#2 lore raw

PATCH v2 2/4 xen: move debugtrace coding to common/debugtrace.c

In reply to Juergen Gross

Instead of living in drivers/char/console.c move the debugtrace related coding to a new file common/debugtrace.c

No functional change, code movement only.

---
 xen/common/Makefile        |   1 +
 xen/common/debugtrace.c    | 177 +++++++++++++++++++++++++++++++++++++++++++++
 xen/drivers/char/console.c | 174 +-------------------------------------------
 3 files changed, 179 insertions(+), 173 deletions(-)
 create mode 100644 xen/common/debugtrace.c
xen/common/Makefile +1 -0
index eddda5daa6..62b34e69e9 100644
@@ -4,6 +4,7 @@ obj-y += bsearch.o
 obj-$(CONFIG_CORE_PARKING) += core_parking.o
 obj-y += cpu.o
 obj-y += cpupool.o
+obj-$(CONFIG_DEBUG_TRACE) += debugtrace.o
 obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o
 obj-y += domctl.o
 obj-y += domain.o
xen/common/debugtrace.c new file +177 -0

177 lines not on this page · raw message

xen/drivers/char/console.c +1 -173

180 lines not on this page · raw message

Signature · 7 lines
2.16.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#3 lore raw

PATCH v2 3/4 xen: refactor debugtrace data

In reply to Juergen Gross

As a preparation for per-cpu buffers do a little refactoring of the debugtrace data: put the needed buffer admin data into the buffer as it will be needed for each buffer.

While at it switch debugtrace_send_to_console and debugtrace_used to bool and delete an empty line.

---
 xen/common/debugtrace.c | 58 ++++++++++++++++++++++++++++---------------------
 1 file changed, 33 insertions(+), 25 deletions(-)
xen/common/debugtrace.c +33 -25
index eede28d4d4..c0189267dd 100644
@@ -15,31 +15,37 @@
 #include <xen/watchdog.h>
 
 /* Send output direct to console, or buffer it? */
-static volatile int debugtrace_send_to_console;
+static volatile bool debugtrace_send_to_console;
 
-static char        *debugtrace_buf; /* Debug-trace buffer */
-static unsigned int debugtrace_prd; /* Producer index     */
-static unsigned int debugtrace_kilobytes = 128, debugtrace_bytes;
-static unsigned int debugtrace_used;
+struct debugtrace_data_s {
+    unsigned long bytes; /* Size of buffer. */
+    unsigned long prd;   /* Producer index. */
+    char          buf[];
+};
+
+static struct debugtrace_data_s *debtr_data;
+
+static unsigned int debugtrace_kilobytes = 128;
+static bool debugtrace_used;
 static DEFINE_SPINLOCK(debugtrace_lock);
 integer_param("debugtrace", debugtrace_kilobytes);
 
 static void debugtrace_dump_worker(void)
 {
-    if ( (debugtrace_bytes == 0) || !debugtrace_used )
+    if ( !debtr_data || !debugtrace_used )
         return;
 
     printk("debugtrace_dump() starting\n");
 
     /* Print oldest portion of the ring. */
-    ASSERT(debugtrace_buf[debugtrace_bytes - 1] == 0);
-    console_serial_puts(&debugtrace_buf[debugtrace_prd]);
+    ASSERT(debtr_data->buf[debtr_data->bytes - 1] == 0);
+    console_serial_puts(&debtr_data->buf[debtr_data->prd]);
 
     /* Print youngest portion of the ring. */
-    debugtrace_buf[debugtrace_prd] = '\0';
-    console_serial_puts(&debugtrace_buf[0]);
+    debtr_data->buf[debtr_data->prd] = '\0';
+    console_serial_puts(&debtr_data->buf[0]);
 
-    memset(debugtrace_buf, '\0', debugtrace_bytes);
+    memset(debtr_data->buf, '\0', debtr_data->bytes);
 
     printk("debugtrace_dump() finished\n");
 }
@@ -64,7 +70,6 @@ static void debugtrace_toggle(void)
 
     spin_unlock_irqrestore(&debugtrace_lock, flags);
     watchdog_enable();
-
 }
 
 void debugtrace_dump(void)
@@ -86,10 +91,10 @@ static void debugtrace_add_to_buf(char *buf)
 
     for ( p = buf; *p != '\0'; p++ )
     {
-        debugtrace_buf[debugtrace_prd++] = *p;
+        debtr_data->buf[debtr_data->prd++] = *p;
         /* Always leave a nul byte at the end of the buffer. */
-        if ( debugtrace_prd == (debugtrace_bytes - 1) )
-            debugtrace_prd = 0;
+        if ( debtr_data->prd == (debtr_data->bytes - 1) )
+            debtr_data->prd = 0;
     }
 }
 
@@ -102,14 +107,14 @@ void debugtrace_printk(const char *fmt, ...)
     va_list       args;
     unsigned long flags;
 
-    if ( debugtrace_bytes == 0 )
+    if ( !debtr_data )
         return;
 
-    debugtrace_used = 1;
+    debugtrace_used = true;
 
     spin_lock_irqsave(&debugtrace_lock, flags);
 
-    ASSERT(debugtrace_buf[debugtrace_bytes - 1] == 0);
+    ASSERT(debtr_data->buf[debtr_data->bytes - 1] == 0);
 
     va_start(args, fmt);
     vsnprintf(buf, sizeof(buf), fmt, args);
@@ -125,14 +130,14 @@ void debugtrace_printk(const char *fmt, ...)
     {
         if ( strcmp(buf, last_buf) )
         {
-            last_prd = debugtrace_prd;
+            last_prd = debtr_data->prd;
             last_count = ++count;
             safe_strcpy(last_buf, buf);
             snprintf(cntbuf, sizeof(cntbuf), "%u ", count);
         }
         else
         {
-            debugtrace_prd = last_prd;
+            debtr_data->prd = last_prd;
             snprintf(cntbuf, sizeof(cntbuf), "%u-%u ", last_count, ++count);
         }
         debugtrace_add_to_buf(cntbuf);
@@ -150,7 +155,8 @@ static void debugtrace_key(unsigned char key)
 static int __init debugtrace_init(void)
 {
     int order;
-    unsigned int kbytes, bytes;
+    unsigned long kbytes, bytes;
+    struct debugtrace_data_s *data;
 
     /* Round size down to next power of two. */
     while ( (kbytes = (debugtrace_kilobytes & (debugtrace_kilobytes-1))) != 0 )
@@ -161,12 +167,14 @@ static int __init debugtrace_init(void)
         return 0;
 
     order = get_order_from_bytes(bytes);
-    debugtrace_buf = alloc_xenheap_pages(order, 0);
-    ASSERT(debugtrace_buf != NULL);
+    data = alloc_xenheap_pages(order, 0);
+    if ( !data )
+        return -ENOMEM;
 
-    memset(debugtrace_buf, '\0', bytes);
+    memset(data, '\0', bytes);
 
-    debugtrace_bytes = bytes;
+    data->bytes = bytes - sizeof(*data);
+    debtr_data = data;
 
     register_keyhandler('T', debugtrace_key,
                         "toggle debugtrace to console/buffer", 0);
Signature · 7 lines
2.16.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#4 lore raw

PATCH v2 4/4 xen: add per-cpu buffer option to debugtrace

In reply to Juergen Gross

debugtrace is normally writing trace entries into a single trace buffer. There are cases where this is not optimal, e.g. when hunting a bug which requires writing lots of trace entries and one cpu is stuck. This will result in other cpus filling the trace buffer and finally overwriting the interesting trace entries of the hanging cpu.

In order to be able to debug such situations add the capability to use per-cpu trace buffers. This can be selected by specifying the debugtrace boot parameter with the modifier "cpu:", like:

debugtrace=cpu:16

At the same time switch the parsing function to accept size modifiers (e.g. 4M or 1G).

Printing out the trace entries is done for each buffer in order to minimize the effort needed during printing. As each entry is prefixed with its sequence number sorting the entries can easily be done when analyzing them.

---
V2: only allocate buffer if not already done so

---
 docs/misc/xen-command-line.pandoc |   7 +-
 xen/common/debugtrace.c           | 145 +++++++++++++++++++++++++++++---------
 2 files changed, 115 insertions(+), 37 deletions(-)
docs/misc/xen-command-line.pandoc +4 -3
index 7c72e31032..832797e2e2 100644
@@ -644,12 +644,13 @@ over the PCI busses sequentially) or by PCI device (must be on segment 0).
 Limits the number lines printed in Xen stack traces.
 
 ### debugtrace
-> `= <integer>`
+> `= [cpu:]<size>`
 
 > Default: `128`
 
-Specify the size of the console debug trace buffer in KiB. The debug
-trace feature is only enabled in debugging builds of Xen.
+Specify the size of the console debug trace buffer. By specifying `cpu:`
+additionally a trace buffer of the specified size is allocated per cpu.
+The debug trace feature is only enabled in debugging builds of Xen.
 
 ### dma_bits
 > `= <integer>`
xen/common/debugtrace.c +111 -34

230 lines not on this page · raw message

Signature · 7 lines
2.16.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel