NUMA functions for accessing replicated areas

The linux kernel debugger (kdb) has hit a problem with setting software breakpoints on NUMA boxes with replicated kernel text. Instead of changing one kernel virtual address and affecting the only copy of the kernel, kdb has to loop over each replicated area and change each copy of the kernel individually, using some mapping that gives access to the data on other nodes.

I don't want code in kdb that knows about every NUMA platform, that is a maintenance nightmare. I want the NUMA platforms to provide the

following interfaces. Any objections or changes? I will do the the SGI version, other manufacturers will have to provide the equivalent functions if they want to use kdb.

BTW, is there a single NUMA mailing list? cc: me please, I am not on all lists.

CONFIG_NUMA_REPLICATE

Set to y if the platform supports the replication functions.

int numa_replicated(unsigned long address, size)

Return 0 if the area (address to address+size-1) is not replicated. Return 1 if it is replicated.
Return -EINVAL if the area is a mixture of replicated and non-replicated data.

int numa_replicate_loop(unsigned long address, (int *func)(void *numadata, void *funcdata), void *funcdata)

Call the function for each replication of the address. numadata identifies this instance of the replicated data, it is an opaque cookie to the function. funcdata is supplied by the caller of numa_replicate_loop(), it is an opaque cookie to the numa code.

Return 0 if the function returns 0 for all calls, otherwise 1. If any call to func() returns non-zero, numa_replicate_loop() continues with the other replicated areas. If func() wants to ignore subsequent areas it can set a flag in funcdata.

int numa_replicate_getarea_size(void *to, unsigned long from_xxx, size_t size, void *numadata)

Read size bytes from from_xxx (which must be a replicated address) to the user's data area.

Return 0 on success, -EFAULT for invalid from_xxx address or size.

int numa_replicate_putarea_size(unsigned long to_xxx, void *from, size_t size, void *numadata)

Write size bytes from the user's data area to to_xxx (which must be a replicated address).

Return 0 on success, -EFAULT for invalid to_xxx address or size.

Is that satisfactory? Once those routines exist, kdb can use them to set software breakpoints.

On Sat, Jan 19, 2002 at 11:25:17AM +1100, Keith Owens wrote:
The opaque void * spread across all the API looks rather scary. Furthmore I'm not convinced we need to pass any cookie to the api, all the calls knows the numadata info by the static status after boot.

So I think something like this should be enough:

	int numa_replicated(unsigned long address, int size);
	int numa_getarea(void *to, unsigned long from, int size);
	int numa_putarea(unsigned long to, void *from, int size);

Futhmore it may be even better to drop numa_replicated completly and to default using numa_getarea/numa_putarea always, this should make kdb even cleaner. Implementation of getarea/putarea for non numa case will be a simple memcpy.

There is one thing that occurred to me. Currently, on IA64, the replicated data is read-execute. The NUMA code known how to write it but I dont know if you want to teach kdb how to do it.