Skip to contentSkip to conversation

Reading deterministic cache parameters and exporting it in /sysfs

PATCH In review Venkatesh Pallipadi 4 participants 7 messages Mar 15, 2005 – Mar 18, 2005

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Reading deterministic cache parameters and exporting it in /sysfs

The attached patch adds support for using cpuid(4) instead of cpuid(2), to get CPU cache information in a deterministic way for Intel CPUs, whenever supported. The details of cpuid(4) can be found here

IA-32 Intel Architecture Software Developer's Manual (vol 2a)
(http://developer.intel.com/design/pentium4/manuals/index_new.htm#sdm_vol2a)
and
Prescott New Instructions (PNI) Technology: Software Developer's Guide
(http://www.intel.com/cd/ids/developer/asmo-na/eng/events/43988.htm)

The advantage of using the cpuid(4) ('Deterministic Cache Parameters Leaf') are:
* It provides more information than the descriptors provided by cpuid(2)
* It is not table based as cpuid(2). So, we will not need changes to the kernel to support new cache descriptors in the descriptor table (as is the case with cpuid(2)).

The patch also adds a bunch of interfaces under /sys/devices/system/cpu/cpuX/cache, showing various information about the caches. Most useful field being shared_cpu_map, which says what caches are shared among which logical cpus.

The patch adds support for both i386 and x86-64.

include/asm-i386/processor.h renamed from include/asm-i386/processor.h.org +12 -0
@@ -147,6 +147,18 @@ static inline void cpuid(int op, int *ea
 		: "0" (op), "c"(0));
 }
 
+/* Some CPUID calls want 'count' to be placed in ecx */
+static inline void cpuid_count(int op, int count, int *eax, int *ebx, int *ecx,
+	       	int *edx)
+{
+	__asm__("cpuid"
+		: "=a" (*eax),
+		  "=b" (*ebx),
+		  "=c" (*ecx),
+		  "=d" (*edx)
+		: "0" (op), "c" (count));
+}
+
 /*
  * CPUID functions returning a single datum
  */
include/asm-x86_64/msr.h renamed from include/asm-x86_64/msr.h.org +12 -0
@@ -78,6 +78,18 @@ extern inline void cpuid(int op, unsigne
 		: "0" (op));
 }
 
+/* Some CPUID calls want 'count' to be placed in ecx */
+static inline void cpuid_count(int op, int count, int *eax, int *ebx, int *ecx,
+	       	int *edx)
+{
+	__asm__("cpuid"
+		: "=a" (*eax),
+		  "=b" (*ebx),
+		  "=c" (*ecx),
+		  "=d" (*edx)
+		: "0" (op), "c" (count));
+}
+
 /*
  * CPUID functions returning a single datum
  */
arch/i386/kernel/cpu/intel_cacheinfo.c renamed from arch/i386/kernel/cpu/intel_cacheinfo.c.org +451 -0

475 lines not on this page · raw message

#1 lore raw
Quoted 17 lines · Venkatesh Pallipadi

The attached patch adds support for using cpuid(4) instead of cpuid(2), to get CPU cache information in a deterministic way for Intel CPUs, whenever supported. The details of cpuid(4) can be found here

IA-32 Intel Architecture Software Developer's Manual (vol 2a)
(http://developer.intel.com/design/pentium4/manuals/index_new.htm#sdm_vol2a)
and
Prescott New Instructions (PNI) Technology: Software Developer's Guide
(http://www.intel.com/cd/ids/developer/asmo-na/eng/events/43988.htm)

The advantage of using the cpuid(4) ('Deterministic Cache Parameters Leaf') are:
* It provides more information than the descriptors provided by cpuid(2)
* It is not table based as cpuid(2). So, we will not need changes to the kernel to support new cache descriptors in the descriptor table (as is the case with cpuid(2)).

Quoted 3 lines

The patch also adds a bunch of interfaces under /sys/devices/system/cpu/cpuX/cache, showing various information about the caches.

Why does this need to be in kernel-space ? Is there some reason that prevents you from enhancing x86info for example ? I really want to live to see the death of /proc/cpuinfo one day, and reinventing it in sysfs seems pointless if it can all be done in userspace.

Quoted 2 lines

Most useful field being shared_cpu_map, which says what caches are shared among which logical cpus.

Given that the most useful field is of limited use to a majority of users, and those that are interested can read this from userspace, this has me very puzzled.

Dave

#2 lore raw
Quoted 18 lines · Dave Jones
Quoted 17 lines · Venkatesh Pallipadi

The attached patch adds support for using cpuid(4) instead of cpuid(2), to get CPU cache information in a deterministic way for Intel CPUs, whenever supported. The details of cpuid(4) can be found here

IA-32 Intel Architecture Software Developer's Manual (vol 2a)
(http://developer.intel.com/design/pentium4/manuals/index_new.htm#sdm_vol2a)
and
Prescott New Instructions (PNI) Technology: Software Developer's Guide
(http://www.intel.com/cd/ids/developer/asmo-na/eng/events/43988.htm)

The advantage of using the cpuid(4) ('Deterministic Cache Parameters Leaf') are:
* It provides more information than the descriptors provided by cpuid(2)
* It is not table based as cpuid(2). So, we will not need changes to the kernel to support new cache descriptors in the descriptor table (as is the case with cpuid(2)).

Quoted 5 lines
Quoted 3 lines

The patch also adds a bunch of interfaces under /sys/devices/system/cpu/cpuX/cache, showing various information about the caches.

Why does this need to be in kernel-space ?

Currently, the CPU cache information is printed as a part of kernel bootup messages and /proc/cpuinfo using cpuid(2). This patch is trying to use cpuid(4) to print the messages in these places. I think this part of the patch is required. Otherwise, we may end up printing 0 cache sizes on some CPUs. It will also reduce the zero_cache_size_complaints on lkml :-).

Quoted 6 lines

Is there some reason that prevents
you from enhancing x86info for example ? I really want to live to see the death of /proc/cpuinfo one day, and reinventing it in sysfs seems pointless if it can all be done in userspace.
Given that the most useful field is of limited use to a majority of users, and those that are interested can read this from userspace, this has me very puzzled.

Agreed. Exporting it in /sysfs is debatable. And some of the information like, 'Which CPUs are sharing what caches' may not be useful today. But, with CPUs with HT and multiple cores and combinations of it, sharing different caches, having this information will be useful inside the kernel as well. scheduler for example. We can setup some of the scheduler domain parameters based on whether L2 is shared or not.
Also, we felt, exporting this information to userspace in a consistent way will help userspace apps to do various things like binding to specific CPUs, using the working set size based on cache size, etc, to optimize the performance. Again, this can be done in userspace as well. But, if kernel is already doing it, it may be better to export it from the kernel space.

Thanks,
Venki

#3 lore raw
In reply to Dave Jones
Quoted 1 line · Dave Jones

I really want to live to see the death of /proc/cpuinfo one day,

Please don't. cpuinfo contains a vast amount of useful information for a quick inspection which cannot be determined usefully from userspace (think embedded devices) for very little code.

Servus,
Daniel

#4 lore raw
In reply to Venkatesh Pallipadi
Quoted 4 lines · Venkatesh Pallipadi

The attached patch adds support for using cpuid(4) instead of cpuid(2), to get CPU cache information in a deterministic way for Intel CPUs, whenever supported.

- find_num_cache_leaves can be marked __init

- Please look for other __init opportunities. That's quite a lot of code.

- Some functions have a space before the ( and some don't:

+static ssize_t show_size (struct _cpuid4_info *this_leaf, char *buf)

omitting the space is preferred.

- Don't cast the return value of kmalloc:

+ cpuid4_info[cpu] = (struct _cpuid4_info *)kmalloc(
+ sizeof(struct _cpuid4_info) * num_cache_leaves, GFP_KERNEL);

- Sometimes there's a space after an `if', sometimes not.

+ if(cpuid4_info[i])

a space is preferred.

- kfree(NULL) is permitted:

+ if(cpuid4_info[i])
+ kfree(cpuid4_info[i]);
+ if(cache_kobject[i])
+ kfree(cache_kobject[i]);
+ if(index_kobject[i])
+ kfree(index_kobject[i]);

(in several places)

Once you've worked through the design issues with davej, please upissue the patch, thanks.

#5 lore raw

Here is the updated patch.

I have seperated out the changes related to
(1) using new method to determine cache size in existing /proc/cpuinfo and kernel boot messages (All but last hunk below)
(2) code to look at sharedness of the caches and store these details for future uses inside kernel and also exporting the cache details in /sysfs (last hunk in the patch)

Dave: Do you still feel having the cache details exported in /sysfs is a bad idea? If yes, we can go ahead with the basic part of this patch (1 - above) and look at (2) sometime later, as and when required.

Thanks,
Venki

The attached patch adds support for using cpuid(4) instead of cpuid(2), to get CPU cache information in a deterministic way for Intel CPUs, whenever supported. The details of cpuid(4) can be found here

IA-32 Intel Architecture Software Developer's Manual (vol 2a)
(http://developer.intel.com/design/pentium4/manuals/index_new.htm#sdm_vol2a)
and
Prescott New Instructions (PNI) Technology: Software Developer's Guide
(http://www.intel.com/cd/ids/developer/asmo-na/eng/events/43988.htm)

The advantage of using the cpuid(4) ('Deterministic Cache Parameters Leaf') are:
* It provides more information than the descriptors provided by cpuid(2)
* It is not table based as cpuid(2). So, we will not need changes to the kernel to support new cache descriptors in the descriptor table (as is the case with cpuid(2)).

The patch also adds a bunch of interfaces under /sys/devices/system/cpu/cpuX/cache, showing various information about the caches. Most useful field being shared_cpu_map, which says what caches are shared among which logical cpus.

The patch adds support for both i386 and x86-64.

include/asm-i386/processor.h renamed from include/asm-i386/processor.h.org +12 -0
@@ -147,6 +147,18 @@ static inline void cpuid(int op, int *ea
 		: "0" (op), "c"(0));
 }
 
+/* Some CPUID calls want 'count' to be placed in ecx */
+static inline void cpuid_count(int op, int count, int *eax, int *ebx, int *ecx,
+	       	int *edx)
+{
+	__asm__("cpuid"
+		: "=a" (*eax),
+		  "=b" (*ebx),
+		  "=c" (*ecx),
+		  "=d" (*edx)
+		: "0" (op), "c" (count));
+}
+
 /*
  * CPUID functions returning a single datum
  */
include/asm-x86_64/msr.h renamed from include/asm-x86_64/msr.h.org +12 -0
@@ -78,6 +78,18 @@ extern inline void cpuid(int op, unsigne
 		: "0" (op));
 }
 
+/* Some CPUID calls want 'count' to be placed in ecx */
+static inline void cpuid_count(int op, int count, int *eax, int *ebx, int *ecx,
+	       	int *edx)
+{
+	__asm__("cpuid"
+		: "=a" (*eax),
+		  "=b" (*ebx),
+		  "=c" (*ecx),
+		  "=d" (*edx)
+		: "0" (op), "c" (count));
+}
+
 /*
  * CPUID functions returning a single datum
  */
arch/i386/kernel/cpu/intel_cacheinfo.c renamed from arch/i386/kernel/cpu/intel_cacheinfo.c.org +456 -0

480 lines not on this page · raw message

#6 lore raw
Quoted 10 lines · Venkatesh Pallipadi

Here is the updated patch.

I have seperated out the changes related to
(1) using new method to determine cache size in existing /proc/cpuinfo and kernel boot messages (All but last hunk below)
(2) code to look at sharedness of the caches and store these details for future uses inside kernel and also exporting the cache details in /sysfs (last hunk in the patch)

Quoted 3 lines

Dave: Do you still feel having the cache details exported in /sysfs is a bad idea? If yes, we can go ahead with the basic part of this patch (1 - above) and look at (2) sometime later, as and when required.

tbh I think its just bloat, but if no-one else has any objections I won't oppose it. The rest of the patch I have no problem with.

Dave