Skip to contentSkip to conversation

dma-debug: implement dynamic entry allocation

PATCH Superseded Robin Murphy 4 participants 20 messages Dec 3, 2018 – Dec 4, 2018 DMA mapping helpers

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Robin Murphy

author reviewer
#0 lore raw

cover dma-debug: implement dynamic entry allocation

The HNS_ENET discussion got me thinking, why not just make DMA debug cleverer so that (in terms of basic functionality at least) we don't need to worry about driver- or arch-specific configuration at all?

Patches #2 and #3 are the real meat here - #1 is just some preparatory cleanup motivated by moving printks around, and could be split out if desired; I kept #4 separate as a possible nice-to-have depending on what people think.

Robin.

Robin Murphy (4):
dma-debug: Use pr_fmt()
dma-debug: Refactor dma_debug_entry allocation
dma-debug: Dynamically expand the dma_debug_entry pool
dma-debug: Make leak-like behaviour apparent

 kernel/dma/debug.c | 198 ++++++++++++++++++++++++---------------------
 1 file changed, 105 insertions(+), 93 deletions(-)
Signature · 1 line
2.19.1.dirty
author reviewer
#1 lore raw

PATCH 1/4 dma-debug: Use pr_fmt()

Use pr_fmt() to generate the "DMA-API: " prefix consistently. This results in it being added to a couple of pr_*() messages which were missing it before, and for the err_printk() calls moves it to the actual start of the message instead of somewhere in the middle.

Signed-off-by Robin Murphy <[email protected]>

---

I chose not to refactor the existing split strings for minimal churn here.

 kernel/dma/debug.c | 74 ++++++++++++++++++++++++----------------------
 1 file changed, 38 insertions(+), 36 deletions(-)
kernel/dma/debug.c +38 -36
index 231ca4628062..91b84140e4a5 100644
@@ -17,6 +17,8 @@
  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA
  */
 
+#define pr_fmt(fmt)	"DMA-API: " fmt
+
 #include <linux/sched/task_stack.h>
 #include <linux/scatterlist.h>
 #include <linux/dma-mapping.h>

@@ -234,7 +236,7 @@ static bool driver_filter(struct device *dev)
 		error_count += 1;					\
 		if (driver_filter(dev) &&				\
 		    (show_all_errors || show_num_errors > 0)) {		\
-			WARN(1, "%s %s: " format,			\
+			WARN(1, pr_fmt("%s %s: ") format,		\
 			     dev ? dev_driver_string(dev) : "NULL",	\
 			     dev ? dev_name(dev) : "NULL", ## arg);	\
 			dump_entry_trace(entry);			\

@@ -519,7 +521,7 @@ static void active_cacheline_inc_overlap(phys_addr_t cln)
 	 * prematurely.
 	 */
 	WARN_ONCE(overlap > ACTIVE_CACHELINE_MAX_OVERLAP,
-		  "DMA-API: exceeded %d overlapping mappings of cacheline %pa\n",
+		  pr_fmt("exceeded %d overlapping mappings of cacheline %pa\n"),
 		  ACTIVE_CACHELINE_MAX_OVERLAP, &cln);
 }
 

@@ -614,7 +616,7 @@ void debug_dma_assert_idle(struct page *page)
 
 	cln = to_cacheline_number(entry);
 	err_printk(entry->dev, entry,
-		   "DMA-API: cpu touching an active dma mapped cacheline [cln=%pa]\n",
+		   "cpu touching an active dma mapped cacheline [cln=%pa]\n",
 		   &cln);
 }
 

@@ -634,7 +636,7 @@ static void add_dma_entry(struct dma_debug_entry *entry)
 
 	rc = active_cacheline_insert(entry);
 	if (rc == -ENOMEM) {
-		pr_err("DMA-API: cacheline tracking ENOMEM, dma-debug disabled\n");
+		pr_err("cacheline tracking ENOMEM, dma-debug disabled\n");
 		global_disable = true;
 	}
 

@@ -673,7 +675,7 @@ static struct dma_debug_entry *dma_entry_alloc(void)
 	if (list_empty(&free_entries)) {
 		global_disable = true;
 		spin_unlock_irqrestore(&free_entries_lock, flags);
-		pr_err("DMA-API: debugging out of memory - disabling\n");
+		pr_err("debugging out of memory - disabling\n");
 		return NULL;
 	}
 

@@ -777,7 +779,7 @@ static int prealloc_memory(u32 num_entries)
 	num_free_entries = num_entries;
 	min_free_entries = num_entries;
 
-	pr_info("DMA-API: preallocated %d debug entries\n", num_entries);
+	pr_info("preallocated %d debug entries\n", num_entries);
 
 	return 0;
 

@@ -850,7 +852,7 @@ static ssize_t filter_write(struct file *file, const char __user *userbuf,
 		 * switched off.
 		 */
 		if (current_driver_name[0])
-			pr_info("DMA-API: switching off dma-debug driver filter\n");
+			pr_info("switching off dma-debug driver filter\n");
 		current_driver_name[0] = 0;
 		current_driver = NULL;
 		goto out_unlock;

@@ -868,7 +870,7 @@ static ssize_t filter_write(struct file *file, const char __user *userbuf,
 	current_driver_name[i] = 0;
 	current_driver = NULL;
 
-	pr_info("DMA-API: enable driver filter for driver [%s]\n",
+	pr_info("enable driver filter for driver [%s]\n",
 		current_driver_name);
 
 out_unlock:

@@ -887,7 +889,7 @@ static int dma_debug_fs_init(void)
 {
 	dma_debug_dent = debugfs_create_dir("dma-api", NULL);
 	if (!dma_debug_dent) {
-		pr_err("DMA-API: can not create debugfs directory\n");
+		pr_err("can not create debugfs directory\n");
 		return -ENOMEM;
 	}
 

@@ -973,7 +975,7 @@ static int dma_debug_device_change(struct notifier_block *nb, unsigned long acti
 		count = device_dma_allocations(dev, &entry);
 		if (count == 0)
 			break;
-		err_printk(dev, entry, "DMA-API: device driver has pending "
+		err_printk(dev, entry, "device driver has pending "
 				"DMA allocations while released from device "
 				"[count=%d]\n"
 				"One of leaked entries details: "

@@ -1023,14 +1025,14 @@ static int dma_debug_init(void)
 	}
 
 	if (dma_debug_fs_init() != 0) {
-		pr_err("DMA-API: error creating debugfs entries - disabling\n");
+		pr_err("error creating debugfs entries - disabling\n");
 		global_disable = true;
 
 		return 0;
 	}
 
 	if (prealloc_memory(nr_prealloc_entries) != 0) {
-		pr_err("DMA-API: debugging out of memory error - disabled\n");
+		pr_err("debugging out of memory error - disabled\n");
 		global_disable = true;
 
 		return 0;

@@ -1040,7 +1042,7 @@ static int dma_debug_init(void)
 
 	dma_debug_initialized = true;
 
-	pr_info("DMA-API: debugging enabled by kernel config\n");
+	pr_info("debugging enabled by kernel config\n");
 	return 0;
 }
 core_initcall(dma_debug_init);

@@ -1051,7 +1053,7 @@ static __init int dma_debug_cmdline(char *str)
 		return -EINVAL;
 
 	if (strncmp(str, "off", 3) == 0) {
-		pr_info("DMA-API: debugging disabled on kernel command line\n");
+		pr_info("debugging disabled on kernel command line\n");
 		global_disable = true;
 	}
 

@@ -1085,11 +1087,11 @@ static void check_unmap(struct dma_debug_entry *ref)
 
 		if (dma_mapping_error(ref->dev, ref->dev_addr)) {
 			err_printk(ref->dev, NULL,
-				   "DMA-API: device driver tries to free an "
+				   "device driver tries to free an "
 				   "invalid DMA memory address\n");
 		} else {
 			err_printk(ref->dev, NULL,
-				   "DMA-API: device driver tries to free DMA "
+				   "device driver tries to free DMA "
 				   "memory it has not allocated [device "
 				   "address=0x%016llx] [size=%llu bytes]\n",
 				   ref->dev_addr, ref->size);

@@ -1098,7 +1100,7 @@ static void check_unmap(struct dma_debug_entry *ref)
 	}
 
 	if (ref->size != entry->size) {
-		err_printk(ref->dev, entry, "DMA-API: device driver frees "
+		err_printk(ref->dev, entry, "device driver frees "
 			   "DMA memory with different size "
 			   "[device address=0x%016llx] [map size=%llu bytes] "
 			   "[unmap size=%llu bytes]\n",

@@ -1106,7 +1108,7 @@ static void check_unmap(struct dma_debug_entry *ref)
 	}
 
 	if (ref->type != entry->type) {
-		err_printk(ref->dev, entry, "DMA-API: device driver frees "
+		err_printk(ref->dev, entry, "device driver frees "
 			   "DMA memory with wrong function "
 			   "[device address=0x%016llx] [size=%llu bytes] "
 			   "[mapped as %s] [unmapped as %s]\n",

@@ -1114,7 +1116,7 @@ static void check_unmap(struct dma_debug_entry *ref)
 			   type2name[entry->type], type2name[ref->type]);
 	} else if ((entry->type == dma_debug_coherent) &&
 		   (phys_addr(ref) != phys_addr(entry))) {
-		err_printk(ref->dev, entry, "DMA-API: device driver frees "
+		err_printk(ref->dev, entry, "device driver frees "
 			   "DMA memory with different CPU address "
 			   "[device address=0x%016llx] [size=%llu bytes] "
 			   "[cpu alloc address=0x%016llx] "

@@ -1126,7 +1128,7 @@ static void check_unmap(struct dma_debug_entry *ref)
 
 	if (ref->sg_call_ents && ref->type == dma_debug_sg &&
 	    ref->sg_call_ents != entry->sg_call_ents) {
-		err_printk(ref->dev, entry, "DMA-API: device driver frees "
+		err_printk(ref->dev, entry, "device driver frees "
 			   "DMA sg list with different entry count "
 			   "[map count=%d] [unmap count=%d]\n",
 			   entry->sg_call_ents, ref->sg_call_ents);

@@ -1137,7 +1139,7 @@ static void check_unmap(struct dma_debug_entry *ref)
 	 * DMA API don't handle this properly, so check for it here
 	 */
 	if (ref->direction != entry->direction) {
-		err_printk(ref->dev, entry, "DMA-API: device driver frees "
+		err_printk(ref->dev, entry, "device driver frees "
 			   "DMA memory with different direction "
 			   "[device address=0x%016llx] [size=%llu bytes] "
 			   "[mapped with %s] [unmapped with %s]\n",

@@ -1153,7 +1155,7 @@ static void check_unmap(struct dma_debug_entry *ref)
 	 */
 	if (entry->map_err_type == MAP_ERR_NOT_CHECKED) {
 		err_printk(ref->dev, entry,
-			   "DMA-API: device driver failed to check map error"
+			   "device driver failed to check map error"
 			   "[device address=0x%016llx] [size=%llu bytes] "
 			   "[mapped as %s]",
 			   ref->dev_addr, ref->size,

@@ -1178,7 +1180,7 @@ static void check_for_stack(struct device *dev,
 			return;
 		addr = page_address(page) + offset;
 		if (object_is_on_stack(addr))
-			err_printk(dev, NULL, "DMA-API: device driver maps memory from stack [addr=%p]\n", addr);
+			err_printk(dev, NULL, "device driver maps memory from stack [addr=%p]\n", addr);
 	} else {
 		/* Stack is vmalloced. */
 		int i;

@@ -1188,7 +1190,7 @@ static void check_for_stack(struct device *dev,
 				continue;
 
 			addr = (u8 *)current->stack + i * PAGE_SIZE + offset;
-			err_printk(dev, NULL, "DMA-API: device driver maps memory from stack [probable addr=%p]\n", addr);
+			err_printk(dev, NULL, "device driver maps memory from stack [probable addr=%p]\n", addr);
 			break;
 		}
 	}

@@ -1208,7 +1210,7 @@ static void check_for_illegal_area(struct device *dev, void *addr, unsigned long
 {
 	if (overlap(addr, len, _stext, _etext) ||
 	    overlap(addr, len, __start_rodata, __end_rodata))
-		err_printk(dev, NULL, "DMA-API: device driver maps memory from kernel text or rodata [addr=%p] [len=%lu]\n", addr, len);
+		err_printk(dev, NULL, "device driver maps memory from kernel text or rodata [addr=%p] [len=%lu]\n", addr, len);
 }
 
 static void check_sync(struct device *dev,

@@ -1224,7 +1226,7 @@ static void check_sync(struct device *dev,
 	entry = bucket_find_contain(&bucket, ref, &flags);
 
 	if (!entry) {
-		err_printk(dev, NULL, "DMA-API: device driver tries "
+		err_printk(dev, NULL, "device driver tries "
 				"to sync DMA memory it has not allocated "
 				"[device address=0x%016llx] [size=%llu bytes]\n",
 				(unsigned long long)ref->dev_addr, ref->size);

@@ -1232,7 +1234,7 @@ static void check_sync(struct device *dev,
 	}
 
 	if (ref->size > entry->size) {
-		err_printk(dev, entry, "DMA-API: device driver syncs"
+		err_printk(dev, entry, "device driver syncs"
 				" DMA memory outside allocated range "
 				"[device address=0x%016llx] "
 				"[allocation size=%llu bytes] "

@@ -1245,7 +1247,7 @@ static void check_sync(struct device *dev,
 		goto out;
 
 	if (ref->direction != entry->direction) {
-		err_printk(dev, entry, "DMA-API: device driver syncs "
+		err_printk(dev, entry, "device driver syncs "
 				"DMA memory with different direction "
 				"[device address=0x%016llx] [size=%llu bytes] "
 				"[mapped with %s] [synced with %s]\n",

@@ -1256,7 +1258,7 @@ static void check_sync(struct device *dev,
 
 	if (to_cpu && !(entry->direction == DMA_FROM_DEVICE) &&
 		      !(ref->direction == DMA_TO_DEVICE))
-		err_printk(dev, entry, "DMA-API: device driver syncs "
+		err_printk(dev, entry, "device driver syncs "
 				"device read-only DMA memory for cpu "
 				"[device address=0x%016llx] [size=%llu bytes] "
 				"[mapped with %s] [synced with %s]\n",

@@ -1266,7 +1268,7 @@ static void check_sync(struct device *dev,
 
 	if (!to_cpu && !(entry->direction == DMA_TO_DEVICE) &&
 		       !(ref->direction == DMA_FROM_DEVICE))
-		err_printk(dev, entry, "DMA-API: device driver syncs "
+		err_printk(dev, entry, "device driver syncs "
 				"device write-only DMA memory to device "
 				"[device address=0x%016llx] [size=%llu bytes] "
 				"[mapped with %s] [synced with %s]\n",

@@ -1276,7 +1278,7 @@ static void check_sync(struct device *dev,
 
 	if (ref->sg_call_ents && ref->type == dma_debug_sg &&
 	    ref->sg_call_ents != entry->sg_call_ents) {
-		err_printk(ref->dev, entry, "DMA-API: device driver syncs "
+		err_printk(ref->dev, entry, "device driver syncs "
 			   "DMA sg list with different entry count "
 			   "[map count=%d] [sync count=%d]\n",
 			   entry->sg_call_ents, ref->sg_call_ents);

@@ -1297,7 +1299,7 @@ static void check_sg_segment(struct device *dev, struct scatterlist *sg)
 	 * whoever generated the list forgot to check them.
 	 */
 	if (sg->length > max_seg)
-		err_printk(dev, NULL, "DMA-API: mapping sg segment longer than device claims to support [len=%u] [max=%u]\n",
+		err_printk(dev, NULL, "mapping sg segment longer than device claims to support [len=%u] [max=%u]\n",
 			   sg->length, max_seg);
 	/*
 	 * In some cases this could potentially be the DMA API

@@ -1307,7 +1309,7 @@ static void check_sg_segment(struct device *dev, struct scatterlist *sg)
 	start = sg_dma_address(sg);
 	end = start + sg_dma_len(sg) - 1;
 	if ((start ^ end) & ~boundary)
-		err_printk(dev, NULL, "DMA-API: mapping sg segment across boundary [start=0x%016llx] [end=0x%016llx] [boundary=0x%016llx]\n",
+		err_printk(dev, NULL, "mapping sg segment across boundary [start=0x%016llx] [end=0x%016llx] [boundary=0x%016llx]\n",
 			   start, end, boundary);
 #endif
 }

@@ -1319,11 +1321,11 @@ void debug_dma_map_single(struct device *dev, const void *addr,
 		return;
 
 	if (!virt_addr_valid(addr))
-		err_printk(dev, NULL, "DMA-API: device driver maps memory from invalid area [addr=%p] [len=%lu]\n",
+		err_printk(dev, NULL, "device driver maps memory from invalid area [addr=%p] [len=%lu]\n",
 			   addr, len);
 
 	if (is_vmalloc_addr(addr))
-		err_printk(dev, NULL, "DMA-API: device driver maps memory from vmalloc area [addr=%p] [len=%lu]\n",
+		err_printk(dev, NULL, "device driver maps memory from vmalloc area [addr=%p] [len=%lu]\n",
 			   addr, len);
 }
 EXPORT_SYMBOL(debug_dma_map_single);

@@ -1780,7 +1782,7 @@ static int __init dma_debug_driver_setup(char *str)
 	}
 
 	if (current_driver_name[0])
-		pr_info("DMA-API: enable driver filter for driver [%s]\n",
+		pr_info("enable driver filter for driver [%s]\n",
 			current_driver_name);
Signature · 1 line
2.19.1.dirty
#3 lore raw

Re: [PATCH 1/4] dma-debug: Use pr_fmt()

In reply to Robin Murphy
Quoted 27 lines · Robin Murphy

Use pr_fmt() to generate the "DMA-API: " prefix consistently. This results in it being added to a couple of pr_*() messages which were missing it before, and for the err_printk() calls moves it to the actual start of the message instead of somewhere in the middle.

Signed-off-by Robin Murphy <[email protected]>

---

I chose not to refactor the existing split strings for minimal churn here.

 kernel/dma/debug.c | 74 ++++++++++++++++++++++++----------------------
 1 file changed, 38 insertions(+), 36 deletions(-)
kernel/dma/debug.c +2 -0
index 231ca4628062..91b84140e4a5 100644
@@ -17,6 +17,8 @@
  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA
  */
 
+#define pr_fmt(fmt)	"DMA-API: " fmt
+
 #include <linux/sched/task_stack.h>
 #include <linux/scatterlist.h>
 #include <linux/dma-mapping.h>

kernel/dma/debug.c static bool driver_filter(struct device *dev) · line 242

Quoted 8 lines
kernel/dma/debug.c +1 -1
@@ -234,7 +236,7 @@ static bool driver_filter(struct device *dev)
 		error_count += 1;					\
 		if (driver_filter(dev) &&				\
 		    (show_all_errors || show_num_errors > 0)) {		\
-			WARN(1, "%s %s: " format,			\
+			WARN(1, pr_fmt("%s %s: ") format,		\
 			     dev ? dev_driver_string(dev) : "NULL",	\
 			     dev ? dev_name(dev) : "NULL", ## arg);	\
 			dump_entry_trace(entry);			\

I think converting this WARN to

			dev_err(dev, format, ##__VA_ARGS__);
			dump_stack();

would look better and be more intelligible.

Perhaps add a #define for dev_fmt if really necessary.

author reviewer
#4 lore raw

PATCH 2/4 dma-debug: Refactor dma_debug_entry allocation

In reply to Robin Murphy

Make prealloc_memory() a little more general and robust so that it serves for runtime reallocations too. The first thing we can do with that is clean up dma_debug_resize_entries() quite a bit.

Signed-off-by Robin Murphy <[email protected]>
---
 kernel/dma/debug.c | 95 +++++++++++++++++++---------------------------
 1 file changed, 40 insertions(+), 55 deletions(-)
kernel/dma/debug.c +40 -55
index 91b84140e4a5..de5db800dbfc 100644
@@ -645,6 +645,39 @@ static void add_dma_entry(struct dma_debug_entry *entry)
 	 */
 }
 
+static int prealloc_memory(u32 num_entries)
+{
+	struct dma_debug_entry *entry, *next_entry;
+	unsigned long flags;
+	LIST_HEAD(tmp);
+	int i;
+
+	for (i = 0; i < num_entries; ++i) {
+		entry = kzalloc(sizeof(*entry), GFP_KERNEL);
+		if (!entry)
+			goto out_err;
+
+		list_add_tail(&entry->list, &tmp);
+	}
+
+	spin_lock_irqsave(&free_entries_lock, flags);
+	list_splice(&tmp, &free_entries);
+	num_free_entries += num_entries;
+	nr_total_entries += num_entries;
+	spin_unlock_irqrestore(&free_entries_lock, flags);
+
+	return 0;
+
+out_err:
+
+	list_for_each_entry_safe(entry, next_entry, &tmp, list) {
+		list_del(&entry->list);
+		kfree(entry);
+	}
+
+	return -ENOMEM;
+}
+
 static struct dma_debug_entry *__dma_entry_alloc(void)
 {
 	struct dma_debug_entry *entry;

@@ -714,44 +747,25 @@ int dma_debug_resize_entries(u32 num_entries)
 	int i, delta, ret = 0;
 	unsigned long flags;
 	struct dma_debug_entry *entry;
-	LIST_HEAD(tmp);
-
-	spin_lock_irqsave(&free_entries_lock, flags);
 
 	if (nr_total_entries < num_entries) {
 		delta = num_entries - nr_total_entries;
 
-		spin_unlock_irqrestore(&free_entries_lock, flags);
-
-		for (i = 0; i < delta; i++) {
-			entry = kzalloc(sizeof(*entry), GFP_KERNEL);
-			if (!entry)
-				break;
-
-			list_add_tail(&entry->list, &tmp);
-		}
-
-		spin_lock_irqsave(&free_entries_lock, flags);
-
-		list_splice(&tmp, &free_entries);
-		nr_total_entries += i;
-		num_free_entries += i;
+		ret = prealloc_memory(delta);
 	} else {
 		delta = nr_total_entries - num_entries;
 
+		spin_lock_irqsave(&free_entries_lock, flags);
 		for (i = 0; i < delta && !list_empty(&free_entries); i++) {
 			entry = __dma_entry_alloc();
 			kfree(entry);
 		}
+		spin_unlock_irqrestore(&free_entries_lock, flags);
 
 		nr_total_entries -= i;
+		if (nr_total_entries != num_entries)
+			ret = -EBUSY;
 	}
-
-	if (nr_total_entries != num_entries)
-		ret = 1;
-
-	spin_unlock_irqrestore(&free_entries_lock, flags);
-
 	return ret;
 }
 

@@ -763,36 +777,6 @@ int dma_debug_resize_entries(u32 num_entries)
  *   2. Preallocate a given number of dma_debug_entry structs
  */
 
-static int prealloc_memory(u32 num_entries)
-{
-	struct dma_debug_entry *entry, *next_entry;
-	int i;
-
-	for (i = 0; i < num_entries; ++i) {
-		entry = kzalloc(sizeof(*entry), GFP_KERNEL);
-		if (!entry)
-			goto out_err;
-
-		list_add_tail(&entry->list, &free_entries);
-	}
-
-	num_free_entries = num_entries;
-	min_free_entries = num_entries;
-
-	pr_info("preallocated %d debug entries\n", num_entries);
-
-	return 0;
-
-out_err:
-
-	list_for_each_entry_safe(entry, next_entry, &free_entries, list) {
-		list_del(&entry->list);
-		kfree(entry);
-	}
-
-	return -ENOMEM;
-}
-
 static ssize_t filter_read(struct file *file, char __user *user_buf,
 			   size_t count, loff_t *ppos)
 {

@@ -1038,7 +1022,8 @@ static int dma_debug_init(void)
 		return 0;
 	}
 
-	nr_total_entries = num_free_entries;
+	min_free_entries = num_free_entries;
+	pr_info("preallocated %d debug entries\n", nr_total_entries);
 
 	dma_debug_initialized = true;
Signature · 1 line
2.19.1.dirty
#5 lore raw

Re: [PATCH 2/4] dma-debug: Refactor dma_debug_entry allocation

Quoted 3 lines · Robin Murphy

Make prealloc_memory() a little more general and robust so that it serves for runtime reallocations too. The first thing we can do with that is clean up dma_debug_resize_entries() quite a bit.

Maybe also renamed it to dma_debug_alloc_entries or something like that?

Otherwise this looks fine to me.

author reviewer
#6 lore raw

Re: [PATCH 2/4] dma-debug: Refactor dma_debug_entry allocation

Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Robin Murphy

Make prealloc_memory() a little more general and robust so that it serves for runtime reallocations too. The first thing we can do with that is clean up dma_debug_resize_entries() quite a bit.

Maybe also renamed it to dma_debug_alloc_entries or something like that?

Yes, that's definitely nicer.

Quoted 1 line

Otherwise this looks fine to me.

Thanks (and for the review on #1)

Robin.

author reviewer
#7 lore raw

PATCH 3/4 dma-debug: Dynamically expand the dma_debug_entry pool

In reply to Robin Murphy

Certain drivers such as large multi-queue network adapters can use pools of mapped DMA buffers larger than the default dma_debug_entry pool of 65536 entries, with the result that merely probing such a device can cause DMA debug to disable itself during boot unless explicitly given an appropriate "dma_debug_entries=..." option.

Developers trying to debug some other driver on such a system may not be immediately aware of this, and at worst it can hide bugs if they fail to realise that dma-debug has already disabled itself unexpectedly by the time the code of interest gets to run. Even once they do realise, it can be a bit of a pain to emprirically determine a suitable number of preallocated entries to configure without massively over-allocating.

There's really no need for such a static limit, though, since we can quite easily expand the pool at runtime in those rare cases that the preallocated entries are insufficient, which is arguably the least surprising and most useful behaviour.

Signed-off-by Robin Murphy <[email protected]>
---
 kernel/dma/debug.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
kernel/dma/debug.c +15 -3
index de5db800dbfc..46cc075aec99 100644
@@ -47,6 +47,9 @@
 #ifndef PREALLOC_DMA_DEBUG_ENTRIES
 #define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16)
 #endif
+/* If the pool runs out, try this many times to allocate this many new entries */
+#define DMA_DEBUG_DYNAMIC_ENTRIES 256
+#define DMA_DEBUG_DYNAMIC_RETRIES 2
 
 enum {
 	dma_debug_single,

@@ -702,12 +705,21 @@ static struct dma_debug_entry *dma_entry_alloc(void)
 {
 	struct dma_debug_entry *entry;
 	unsigned long flags;
+	int retry_count;
 
-	spin_lock_irqsave(&free_entries_lock, flags);
+	for (retry_count = 0; ; retry_count++) {
+		spin_lock_irqsave(&free_entries_lock, flags);
+
+		if (num_free_entries > 0)
+			break;
 
-	if (list_empty(&free_entries)) {
-		global_disable = true;
 		spin_unlock_irqrestore(&free_entries_lock, flags);
+
+		if (retry_count < DMA_DEBUG_DYNAMIC_RETRIES &&
+		    !prealloc_memory(DMA_DEBUG_DYNAMIC_ENTRIES))
+			continue;
+
+		global_disable = true;
 		pr_err("debugging out of memory - disabling\n");
 		return NULL;
 	}
Signature · 1 line
2.19.1.dirty
#8 lore raw

Re: [PATCH 3/4] dma-debug: Dynamically expand the dma_debug_entry pool

Quoted 13 lines · Robin Murphy

Certain drivers such as large multi-queue network adapters can use pools of mapped DMA buffers larger than the default dma_debug_entry pool of 65536 entries, with the result that merely probing such a device can cause DMA debug to disable itself during boot unless explicitly given an appropriate "dma_debug_entries=..." option.

Developers trying to debug some other driver on such a system may not be immediately aware of this, and at worst it can hide bugs if they fail to realise that dma-debug has already disabled itself unexpectedly by the time the code of interest gets to run. Even once they do realise, it can be a bit of a pain to emprirically determine a suitable number of preallocated entries to configure without massively over-allocating.

Quoted 4 lines

There's really no need for such a static limit, though, since we can quite easily expand the pool at runtime in those rare cases that the preallocated entries are insufficient, which is arguably the least surprising and most useful behaviour.

Hi Robin,

Do you have an idea on shrinking the pool again when the culprit driver is removed, i.e. we have so many unused debug entries now available?

Thanks,
John

Quoted 46 lines
Signed-off-by Robin Murphy <[email protected]>
---
 kernel/dma/debug.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
kernel/dma/debug.c +15 -3
index de5db800dbfc..46cc075aec99 100644
@@ -47,6 +47,9 @@
 #ifndef PREALLOC_DMA_DEBUG_ENTRIES
 #define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16)
 #endif
+/* If the pool runs out, try this many times to allocate this many new entries */
+#define DMA_DEBUG_DYNAMIC_ENTRIES 256
+#define DMA_DEBUG_DYNAMIC_RETRIES 2
 
 enum {
 	dma_debug_single,

@@ -702,12 +705,21 @@ static struct dma_debug_entry *dma_entry_alloc(void)
 {
 	struct dma_debug_entry *entry;
 	unsigned long flags;
+	int retry_count;
 
-	spin_lock_irqsave(&free_entries_lock, flags);
+	for (retry_count = 0; ; retry_count++) {
+		spin_lock_irqsave(&free_entries_lock, flags);
+
+		if (num_free_entries > 0)
+			break;
 
-	if (list_empty(&free_entries)) {
-		global_disable = true;
 		spin_unlock_irqrestore(&free_entries_lock, flags);
+
+		if (retry_count < DMA_DEBUG_DYNAMIC_RETRIES &&
+		    !prealloc_memory(DMA_DEBUG_DYNAMIC_ENTRIES))
+			continue;
+
+		global_disable = true;
 		pr_err("debugging out of memory - disabling\n");
 		return NULL;
 	}
author reviewer
#9 lore raw

Re: [PATCH 3/4] dma-debug: Dynamically expand the dma_debug_entry pool

Hi John,

Quoted 14 lines · John Garry
Quoted 13 lines · Robin Murphy

Certain drivers such as large multi-queue network adapters can use pools of mapped DMA buffers larger than the default dma_debug_entry pool of 65536 entries, with the result that merely probing such a device can cause DMA debug to disable itself during boot unless explicitly given an appropriate "dma_debug_entries=..." option.

Developers trying to debug some other driver on such a system may not be immediately aware of this, and at worst it can hide bugs if they fail to realise that dma-debug has already disabled itself unexpectedly by the time the code of interest gets to run. Even once they do realise, it can be a bit of a pain to emprirically determine a suitable number of preallocated entries to configure without massively over-allocating.

Quoted 8 lines
Quoted 4 lines

There's really no need for such a static limit, though, since we can quite easily expand the pool at runtime in those rare cases that the preallocated entries are insufficient, which is arguably the least surprising and most useful behaviour.

Hi Robin,

Do you have an idea on shrinking the pool again when the culprit driver is removed, i.e. we have so many unused debug entries now available?

I honestly don't believe it's worth the complication. This is a development feature with significant overheads already, so there's not an awful lot to gain by trying to optimise memory usage. If a system can ever load a driver that makes hundreds of thousands of simultaneous mappings, it can almost certainly spare 20-odd megabytes of RAM for the corresponding debug entries in perpetuity. Sure, it does mean you'd need to reboot to recover memory from a major leak, but that's mostly true of the current behaviour too, and rebooting during driver development is hardly an unacceptable inconvenience.

In fact, having got this far in, what I'd quite like to do is to get rid of dma_debug_resize_entries() such that we never need to free things at all, since then we could allocate whole pages as blocks of entries to save on masses of individual slab allocations.

Robin.

Quoted 54 lines

Thanks,
John

Quoted 48 lines
Signed-off-by Robin Murphy <[email protected]>

---
 kernel/dma/debug.c | 18 +++++++++++++++---

 1 file changed, 15 insertions(+), 3 deletions(-)
kernel/dma/debug.c +1 -0
index de5db800dbfc..46cc075aec99 100644
@@ -47,6 +47,9 @@
  #ifndef PREALLOC_DMA_DEBUG_ENTRIES
  #define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16)
  #endif
+/* If the pool runs out, try this many times to allocate this many

new entries */

kernel/dma/debug.c +2 -0
+#define DMA_DEBUG_DYNAMIC_ENTRIES 256
+#define DMA_DEBUG_DYNAMIC_RETRIES 2

 enum {
     dma_debug_single,

kernel/dma/debug.c +12 -3
@@ -702,12 +705,21 @@ static struct dma_debug_entry 
 *dma_entry_alloc(void)
  {
      struct dma_debug_entry *entry;
      unsigned long flags;
+    int retry_count;
 
-    spin_lock_irqsave(&free_entries_lock, flags);
+    for (retry_count = 0; ; retry_count++) {
+        spin_lock_irqsave(&free_entries_lock, flags);
+
+        if (num_free_entries > 0)
+            break;
 
-    if (list_empty(&free_entries)) {
-        global_disable = true;
          spin_unlock_irqrestore(&free_entries_lock, flags);
+
+        if (retry_count < DMA_DEBUG_DYNAMIC_RETRIES &&
+            !prealloc_memory(DMA_DEBUG_DYNAMIC_ENTRIES))
+            continue;
+
+        global_disable = true;
          pr_err("debugging out of memory - disabling\n");
          return NULL;

     }

#10 lore raw

Re: [PATCH 3/4] dma-debug: Dynamically expand the dma_debug_entry pool

Quoted 4 lines · Robin Murphy

In fact, having got this far in, what I'd quite like to do is to get rid of dma_debug_resize_entries() such that we never need to free things at all, since then we could allocate whole pages as blocks of entries to save on masses of individual slab allocations.

Yes, we should defintively kill dma_debug_resize_entries. Allocating page batches might sound nice, but is that going to introduce additional complexity?

author reviewer
#11 lore raw

Re: [PATCH 3/4] dma-debug: Dynamically expand the dma_debug_entry pool

Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Robin Murphy

In fact, having got this far in, what I'd quite like to do is to get rid of dma_debug_resize_entries() such that we never need to free things at all, since then we could allocate whole pages as blocks of entries to save on masses of individual slab allocations.

Yes, we should defintively kill dma_debug_resize_entries. Allocating page batches might sound nice, but is that going to introduce additional complexity?

OK, looking at what the weird AMD GART code does I reckon it should be happy enough with on-demand expansion, and that no tears will be shed if it can no longer actually trim the pool to the size it thinks is necessary. I'll add a patch to clean that up.

Page-based allocation, at least the way I'm thinking of it, shouldn't do much more than add an extra loop in one place, which should be more than made up for by removing all the freeing code :)

Robin.

#12 lore raw

Re: [PATCH 3/4] dma-debug: Dynamically expand the dma_debug_entry pool

In reply to Robin Murphy
Quoted 28 lines · Robin Murphy

Hi John,

Quoted 19 lines · John Garry
Quoted 13 lines · Robin Murphy

Certain drivers such as large multi-queue network adapters can use pools of mapped DMA buffers larger than the default dma_debug_entry pool of 65536 entries, with the result that merely probing such a device can cause DMA debug to disable itself during boot unless explicitly given an appropriate "dma_debug_entries=..." option.

Developers trying to debug some other driver on such a system may not be immediately aware of this, and at worst it can hide bugs if they fail to realise that dma-debug has already disabled itself unexpectedly by the time the code of interest gets to run. Even once they do realise, it can be a bit of a pain to emprirically determine a suitable number of preallocated entries to configure without massively over-allocating.

Quoted 4 lines

There's really no need for such a static limit, though, since we can quite easily expand the pool at runtime in those rare cases that the preallocated entries are insufficient, which is arguably the least surprising and most useful behaviour.

Quoted 5 lines

Hi Robin,

Do you have an idea on shrinking the pool again when the culprit driver is removed, i.e. we have so many unused debug entries now available?

Quoted 10 lines

I honestly don't believe it's worth the complication. This is a development feature with significant overheads already, so there's not an awful lot to gain by trying to optimise memory usage. If a system can ever load a driver that makes hundreds of thousands of simultaneous mappings, it can almost certainly spare 20-odd megabytes of RAM for the corresponding debug entries in perpetuity. Sure, it does mean you'd need to reboot to recover memory from a major leak, but that's mostly true of the current behaviour too, and rebooting during driver development is hardly an unacceptable inconvenience.

ok, I just thought that it would not be too difficult to implement this on the dma entry free path.

Quoted 5 lines

In fact, having got this far in, what I'd quite like to do is to get rid of dma_debug_resize_entries() such that we never need to free things at all, since then we could allocate whole pages as blocks of entries to save on masses of individual slab allocations.

On a related topic, is it possible for the user to learn the total entries created at a given point in time? If not, could we add a file in the debugfs folder for this?

Thanks,
John

Quoted 59 lines

Robin.

Quoted 39 lines

Thanks,
John

Quoted 35 lines
Signed-off-by Robin Murphy <[email protected]>
---
 kernel/dma/debug.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
kernel/dma/debug.c +1 -0
index de5db800dbfc..46cc075aec99 100644
@@ -47,6 +47,9 @@
 #ifndef PREALLOC_DMA_DEBUG_ENTRIES
 #define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16)
 #endif
+/* If the pool runs out, try this many times to allocate this many

new entries */

kernel/dma/debug.c +8 -1
+#define DMA_DEBUG_DYNAMIC_ENTRIES 256
+#define DMA_DEBUG_DYNAMIC_RETRIES 2
 
 enum {
     dma_debug_single,

@@ -702,12 +705,21 @@ static struct dma_debug_entry
 *dma_entry_alloc(void)
 {
     struct dma_debug_entry *entry;
     unsigned long flags;
+    int retry_count;
 
-    spin_lock_irqsave(&free_entries_lock, flags);
+    for (retry_count = 0; ; retry_count++) {
+        spin_lock_irqsave(&free_entries_lock, flags);
+
+        if (num_free_entries > 0)
+            break;
Quoted 15 lines
Quoted 11 lines
kernel/dma/debug.c +6 -2
-    if (list_empty(&free_entries)) {
-        global_disable = true;
         spin_unlock_irqrestore(&free_entries_lock, flags);
+
+        if (retry_count < DMA_DEBUG_DYNAMIC_RETRIES &&
+            !prealloc_memory(DMA_DEBUG_DYNAMIC_ENTRIES))
+            continue;
+
+        global_disable = true;
         pr_err("debugging out of memory - disabling\n");
         return NULL;
Quoted 2 lines
     }

.

author reviewer
#13 lore raw

Re: [PATCH 3/4] dma-debug: Dynamically expand the dma_debug_entry pool

Quoted 40 lines · John Garry
Quoted 28 lines · Robin Murphy

Hi John,

Quoted 19 lines · John Garry
Quoted 13 lines · Robin Murphy

Certain drivers such as large multi-queue network adapters can use pools of mapped DMA buffers larger than the default dma_debug_entry pool of 65536 entries, with the result that merely probing such a device can cause DMA debug to disable itself during boot unless explicitly given an appropriate "dma_debug_entries=..." option.

Developers trying to debug some other driver on such a system may not be immediately aware of this, and at worst it can hide bugs if they fail to realise that dma-debug has already disabled itself unexpectedly by the time the code of interest gets to run. Even once they do realise, it can be a bit of a pain to emprirically determine a suitable number of preallocated entries to configure without massively over-allocating.

Quoted 4 lines

There's really no need for such a static limit, though, since we can quite easily expand the pool at runtime in those rare cases that the preallocated entries are insufficient, which is arguably the least surprising and most useful behaviour.

Quoted 5 lines

Hi Robin,

Do you have an idea on shrinking the pool again when the culprit driver is removed, i.e. we have so many unused debug entries now available?

Quoted 10 lines

I honestly don't believe it's worth the complication. This is a development feature with significant overheads already, so there's not an awful lot to gain by trying to optimise memory usage. If a system can ever load a driver that makes hundreds of thousands of simultaneous mappings, it can almost certainly spare 20-odd megabytes of RAM for the corresponding debug entries in perpetuity. Sure, it does mean you'd need to reboot to recover memory from a major leak, but that's mostly true of the current behaviour too, and rebooting during driver development is hardly an unacceptable inconvenience.

Quoted 1 line

ok, I just thought that it would not be too difficult to implement this on the dma entry free path.

True, in the current code it wouldn't be all that hard, but it feels more worthwhile to optimise for allocation rather than freeing, and as soon as we start allocating memory for multiple entries at once, trying to free anything becomes extremely challenging.

Quoted 7 lines
Quoted 5 lines

In fact, having got this far in, what I'd quite like to do is to get rid of dma_debug_resize_entries() such that we never need to free things at all, since then we could allocate whole pages as blocks of entries to save on masses of individual slab allocations.

On a related topic, is it possible for the user to learn the total entries created at a given point in time? If not, could we add a file in the debugfs folder for this?

I did get as far as pondering that you effectively lose track of utilisation once the low-water-mark of min_free_entries hits 0 and stays there - AFAICS it should be sufficient to just expose nr_total_entries as-is, since users can then calculate current and maximum occupancy based on *_free_entries. Does that sound reasonable to you?

That also indirectly reminds me that this lot is documented in DMA_API.txt, so I should be good and update that too...

Cheers,
Robin.

#14 lore raw

Re: [PATCH 3/4] dma-debug: Dynamically expand the dma_debug_entry pool

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

In fact, having got this far in, what I'd quite like to do is to get rid of dma_debug_resize_entries() such that we never need to free things at all, since then we could allocate whole pages as blocks of entries to save on masses of individual slab allocations.

Quoted 4 lines
Quoted 3 lines

On a related topic, is it possible for the user to learn the total entries created at a given point in time? If not, could we add a file in the debugfs folder for this?

Hi Robin,

Quoted 2 lines

I did get as far as pondering that you effectively lose track of utilisation once the low-water-mark of min_free_entries hits 0 and stays

I did try your patches and I noticed this, i.e I was hitting the point at which we start to alloc more entries.

Quoted 4 lines

there - AFAICS it should be sufficient to just expose nr_total_entries as-is, since users can then calculate current and maximum occupancy based on *_free_entries. Does that sound reasonable to you?

Sounds ok. I am just interested to know roughly how many DMA buffers we're using in our system.

Quoted 2 lines

That also indirectly reminds me that this lot is documented in DMA_API.txt, so I should be good and update that too...

Thanks,
John

Quoted 6 lines

Cheers,
Robin.

.

#15 lore raw

Re: [PATCH 3/4] dma-debug: Dynamically expand the dma_debug_entry pool

In reply to Robin Murphy

kernel/dma/debug.c static struct dma_debug_entry *dma_entry_alloc(void) · line 716

Quoted 7 lines
kernel/dma/debug.c +5 -0
+	for (retry_count = 0; ; retry_count++) {
+		spin_lock_irqsave(&free_entries_lock, flags);
+
+		if (num_free_entries > 0)
+			break;
 
 		spin_unlock_irqrestore(&free_entries_lock, flags);

Taking a spinlock just to read a single integer value doesn't really help anything.

kernel/dma/debug.c static struct dma_debug_entry *dma_entry_alloc(void) · line 719

Quoted 3 lines
kernel/dma/debug.c +3 -0
+
+		if (retry_count < DMA_DEBUG_DYNAMIC_RETRIES &&
+		    !prealloc_memory(DMA_DEBUG_DYNAMIC_ENTRIES))

Don't we need GFP_ATOMIC here? Also why do we need the retries?

author reviewer
#16 lore raw

Re: [PATCH 3/4] dma-debug: Dynamically expand the dma_debug_entry pool

Quoted 10 lines · Christoph Hellwig

kernel/dma/debug.c static struct dma_debug_entry *dma_entry_alloc(void) · line 716

Quoted 7 lines
kernel/dma/debug.c +5 -0
+	for (retry_count = 0; ; retry_count++) {
+		spin_lock_irqsave(&free_entries_lock, flags);
+
+		if (num_free_entries > 0)
+			break;
 
  		spin_unlock_irqrestore(&free_entries_lock, flags);

Taking a spinlock just to read a single integer value doesn't really help anything.

If the freelist is non-empty we break out with the lock still held in order to actually allocate our entry - only if there are no free entries left do we drop the lock in order to handle the failure. This much is just the original logic shuffled around a bit (with the tweak that testing num_free_entries seemed justifiably simpler than the original list_empty() check).

Quoted 5 lines

kernel/dma/debug.c static struct dma_debug_entry *dma_entry_alloc(void) · line 719

Quoted 3 lines
kernel/dma/debug.c +3 -0
+
+		if (retry_count < DMA_DEBUG_DYNAMIC_RETRIES &&
+		    !prealloc_memory(DMA_DEBUG_DYNAMIC_ENTRIES))

Don't we need GFP_ATOMIC here? Also why do we need the retries?

Ah, right, we may be outside our own spinlock, but of course the whole DMA API call which got us here might be under someone else's and/or in a non-sleeping context - I'll fix that.

The number of retries is just to bound the loop due to its inherent raciness - since we drop the lock to create more entries, under pathological conditions by the time we get back in to grab one they could have all gone. 2 retries (well, strictly it's 1 try and 1 retry) was an entirely arbitrary choice just to accommodate that happening very occasionally by chance.

However, if the dynamic allocations need GFP_ATOMIC for external reasons anyway, then I don't need the lock-juggling that invites that race in the first place, and the whole loop disappears again. Neat!

Robin.

author reviewer
#17 lore raw

RFC 4/4 dma-debug: Make leak-like behaviour apparent

In reply to Robin Murphy

Now that we can dynamically allocate DMA debug entries to cope with drivers maintaining excessively large numbers of live mappings, a driver which *does* actually have a bug leaking mappings (and is not unloaded) will no longer trigger the "DMA-API: debugging out of memory - disabling" message until it gets to actual kernel OOM conditions, which means it could go unnoticed for a while. To that end, let's inform the user each time the pool has grown to a multiple of its initial size, which should make it apparent that they either have a leak or might want to increase the preallocation size.

Signed-off-by Robin Murphy <[email protected]>

---

Tagging this one as RFC since people might think it's silly.

 kernel/dma/debug.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/dma/debug.c +14 -1
index 46cc075aec99..c4759dab0f8c 100644
@@ -696,6 +696,17 @@ static struct dma_debug_entry *__dma_entry_alloc(void)
 	return entry;
 }
 
+void __dma_entry_alloc_check_leak(void)
+{
+	u32 tmp = nr_total_entries % nr_prealloc_entries;
+
+	/* Shout each time we tick over some multiple of the initial pool */
+	if (tmp < DMA_DEBUG_DYNAMIC_ENTRIES) {
+		pr_info("dma_debug_entry pool grown to %u00%% - possible mapping leak?\n",
+			(nr_total_entries / nr_prealloc_entries));
+	}
+}
+
 /* struct dma_entry allocator
  *
  * The next two functions implement the allocator for

@@ -716,8 +727,10 @@ static struct dma_debug_entry *dma_entry_alloc(void)
 		spin_unlock_irqrestore(&free_entries_lock, flags);
 
 		if (retry_count < DMA_DEBUG_DYNAMIC_RETRIES &&
-		    !prealloc_memory(DMA_DEBUG_DYNAMIC_ENTRIES))
+		    !prealloc_memory(DMA_DEBUG_DYNAMIC_ENTRIES)) {
+			__dma_entry_alloc_check_leak();
 			continue;
+		}
 
 		global_disable = true;
 		pr_err("debugging out of memory - disabling\n");
Signature · 1 line
2.19.1.dirty
#18 lore raw

Re: [RFC 4/4] dma-debug: Make leak-like behaviour apparent

Quoted 10 lines · Robin Murphy

Now that we can dynamically allocate DMA debug entries to cope with drivers maintaining excessively large numbers of live mappings, a driver which *does* actually have a bug leaking mappings (and is not unloaded) will no longer trigger the "DMA-API: debugging out of memory - disabling" message until it gets to actual kernel OOM conditions, which means it could go unnoticed for a while. To that end, let's inform the user each time the pool has grown to a multiple of its initial size, which should make it apparent that they either have a leak or might want to increase the preallocation size.

Quoted 4 lines
Signed-off-by Robin Murphy <[email protected]>

---

Tagging this one as RFC since people might think it's silly.

I think finding out the numbers is useful, but I'm a little worried about claiming a possible leak. Maybe we just need to print a log message for each new power of 2 of entries reached?

#19 lore raw
In reply to Robin Murphy
Quoted 8 lines · Robin Murphy

The HNS_ENET discussion got me thinking, why not just make DMA debug cleverer so that (in terms of basic functionality at least) we don't need to worry about driver- or arch-specific configuration at all?

Patches #2 and #3 are the real meat here - #1 is just some preparatory cleanup motivated by moving printks around, and could be split out if desired; I kept #4 separate as a possible nice-to-have depending on what people think.

Hah, I just thought of this and suggested it a few seconds ago. So obviously I think it is a good idea, but let me look at the details.