`io{re,un}map()` build error in s390 under `!CONFIG_HAS_IOMEM`
Subsystems (4)
Miguel Ojeda
maintainerHi all,
s390 does not provide ioremap()/iounmap() under !CONFIG_HAS_IOMEM:
In file included from rust/helpers/helpers.c:68:
rust/helpers/io.c:8:9: error: call to undeclared function 'ioremap'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
8 | return ioremap(offset, size);
| ^
rust/helpers/io.c:8:9: error: incompatible integer to pointer conversion returning 'int' from a function with result type 'void *' [-Wint-conversion]
8 | return ioremap(offset, size);
| ^~~~~~~~~~~~~~~~~~~~~
rust/helpers/io.c:19:2: error: call to undeclared function 'iounmap'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
19 | iounmap(addr);
| ^
./arch/s390/include/asm/io.h:31:17: note: expanded from macro 'iounmap'
31 | #define iounmap iounmap
| ^
rust/helpers/io.c:19:2: note: did you mean 'kunmap'?
./arch/s390/include/asm/io.h:31:17: note: expanded from macro 'iounmap'
31 | #define iounmap iounmap
| ^
./include/linux/highmem.h:46:20: note: 'kunmap' declared here
46 | static inline void kunmap(const struct page *page);
| ^
so e.g. an allnoconfig with Rust fails.
I assume we will need to #ifdef the helpers, plus the users; or stub at some layer to avoid changing callers.
I hope that helps & thanks!
Cheers,
Miguel
Arnd Bergmann
maintainerQuoted 7 lines · Miguel Ojeda
so e.g. an
allnoconfigwith Rust fails.I assume we will need to
#ifdefthe helpers, plus the users; or stub at some layer to avoid changing callers.I hope that helps & thanks!
I think in practice you'll find that all sensible configurations will have PCI enabled since that was introduced in the 2012 generation (zEC12), while the oldest supported machines in the kernel these days are from 2008 (z10) and Red Hat Enterprise Linux already needs a z14 from 2017.
In theory you should be able to use rust code without PCI MMIO support, but I can't see any practical downsides to making rust 'depends on HAS_MMIO' to avoid having to add those #ifdef.
Note that we already have 'config COMPILE_TEST depends on HAS_MMIO' to work around similar problems with random drivers expecting MMIO support.
Arnd
Danilo Krummrich
maintainerQuoted 3 lines · Arnd Bergmann
In theory you should be able to use rust code without PCI MMIO support, but I can't see any practical downsides to making rust 'depends on
HAS_MMIO' to avoid having to add those #ifdef.
I think the implications should be minor without making Rust depend on CONFIG_HAS_IOMEM.
I just sent out a fix [1]; the only annoying part is [2], but we should change those doc-tests anyway. For the one in rust/kernel/io.rs we already did in driver-core-next.
[1] https://lore.kernel.org/driver-core/[email protected]/
[2] https://lore.kernel.org/driver-core/[email protected]/
Heiko Carstens
maintainerQuoted 8 lines · Danilo Krummrich
Quoted 3 lines · Arnd Bergmann
In theory you should be able to use rust code without PCI MMIO support, but I can't see any practical downsides to making rust 'depends on
HAS_MMIO' to avoid having to add those #ifdef.I think the implications should be minor without making Rust depend on
CONFIG_HAS_IOMEM.
Quoted 6 lines
I just sent out a fix [1]; the only annoying part is [2], but we should change those doc-tests anyway. For the one in
rust/kernel/io.rswe already did in driver-core-next.[1] https://lore.kernel.org/driver-core/[email protected]/
[2] https://lore.kernel.org/driver-core/[email protected]/
I'm wondering if it would make sense to make HAS_IOMEM always available on s390, even though it doesn't make too much sense without PCI. But at least it would make s390 again a bit less special.
Niklas, Gerd?
Arnd Bergmann
maintainerQuoted 6 lines · Heiko Carstens
Quoted 5 lines · Danilo Krummrich
Quoted 3 lines · Arnd Bergmann
In theory you should be able to use rust code without PCI MMIO support, but I can't see any practical downsides to making rust 'depends on
HAS_MMIO' to avoid having to add those #ifdef.
Quoted 8 lines
Quoted 8 lines
I think the implications should be minor without making Rust depend on
CONFIG_HAS_IOMEM.I just sent out a fix [1]; the only annoying part is [2], but we should change those doc-tests anyway. For the one in
rust/kernel/io.rswe already did in driver-core-next.
This looks like you still provide the rust version of ioremap(), turning what is supposed to be a link failure into a runtime error.
Quoted 3 lines
I'm wondering if it would make sense to make
HAS_IOMEMalways available on s390, even though it doesn't make too much sense without PCI. But at least it would make s390 again a bit less special.
I see that with CONFIG_PCI=y, s390 already falls back to generic_ioremap_prot() and just maps any phys_addr_t into the page table as PAGE_KERNEL, regardless of whether this is an MMIO address or not.
The simple change below would just extend that behavior to !PCI and make that consistent with CONFIG_PCI=y on machines without actual PCI hardware. Of course any code that might rely on this is now a bug that likely never gets caught at build time.
This still relies on implementing the __raw_* helpers as nop to have the same behavior as the PCI=y version, as the generic version would just end up dereferencing the invalid pointers.
Arnd
arch/s390/Kconfig +1 -4
@@ -173,7 +173,7 @@ config S390 select GENERIC_ENTRY select GENERIC_GETTIMEOFDAY select GENERIC_SMP_IDLE_THREAD - select GENERIC_IOREMAP if PCI + select GENERIC_IOREMAP select GLOB select HAVE_ALIGNED_STRUCT_PAGE select HAVE_ARCH_AUDITSYSCALL
@@ -756,9 +756,6 @@ config PCI_NR_FUNCTIONS endif # PCI -config HAS_IOMEM - def_bool PCI - config CHSC_SCH def_tristate m prompt "Support for CHSC subchannels"
arch/s390/include/asm/io.h +46 -17
82 lines not on this page · raw message
init/Kconfig +0 -1
@@ -237,7 +237,6 @@ config INIT_ENV_ARG_LIMIT config COMPILE_TEST bool "Compile also drivers which will not load" - depends on HAS_IOMEM help Some drivers can be compiled on a different platform than they are intended to be run on. Despite they cannot be loaded there (or even
Danilo Krummrich
maintainerQuoted 14 lines · Arnd Bergmann
Quoted 13 lines · Heiko Carstens
Quoted 12 lines · Danilo Krummrich
Quoted 3 lines · Arnd Bergmann
In theory you should be able to use rust code without PCI MMIO support, but I can't see any practical downsides to making rust 'depends on
HAS_MMIO' to avoid having to add those #ifdef.I think the implications should be minor without making Rust depend on
CONFIG_HAS_IOMEM.I just sent out a fix [1]; the only annoying part is [2], but we should change those doc-tests anyway. For the one in
rust/kernel/io.rswe already did in driver-core-next.
Quoted 5 lines
Quoted 1 line
Quoted 1 line
This looks like you still provide the rust version of
ioremap(), turning what is supposed to be a link failure into a runtime error.
Which is the standard for many core APIs, such as [1]. However, I do agree that in this case the correct fix would be to have all architectures provide the stubs rather than the Rust code.
However, there's already a precedent for this in the kernel, e.g. in [2]. Of course, it would be better to clean this up, but depending on whether there's more architectures having this issue (I didn't check) that's separate from a fix.
[1] https://elixir.bootlin.com/linux/v7.1.5/source/include/linux/regmap.h#L1809
[2] https://elixir.bootlin.com/linux/v7.1.5/source/include/linux/device/devres.h#L115
Quoted 9 lines
Quoted 3 lines
I'm wondering if it would make sense to make
HAS_IOMEMalways available on s390, even though it doesn't make too much sense without PCI. But at least it would make s390 again a bit less special.I see that with
CONFIG_PCI=y, s390 already falls back togeneric_ioremap_prot()and just maps anyphys_addr_tinto the page table asPAGE_KERNEL, regardless of whether this is an MMIO address or not.
Quoted 8 lines
The simple change below would just extend that behavior to !PCI and make that consistent with
CONFIG_PCI=yon machines without actual PCI hardware. Of course any code that might rely on this is now a bug that likely never gets caught at build time.This still relies on implementing the __raw_* helpers as nop to have the same behavior as the PCI=y version, as the generic version would just end up dereferencing the invalid pointers.
As mentioned, I didn't check, but if this is the only architecture causing those issues that'd be the better fix of course.
However, IIUC, your patch below would make ioremap() and friends silenty succeed and only the accessors would prevent undefined behavior?
In this case I still think ioremap() should just fail.
Arnd Bergmann
maintainerQuoted 9 lines · Danilo Krummrich
Quoted 8 lines · Arnd Bergmann
Quoted 6 lines · Heiko Carstens
Quoted 6 lines
I just sent out a fix [1]; the only annoying part is [2], but we should change those doc-tests anyway. For the one in
rust/kernel/io.rswe already did in driver-core-next.
Quoted 7 lines
Quoted 3 lines
This looks like you still provide the rust version of
ioremap(), turning what is supposed to be a link failure into a runtime error.Which is the standard for many core APIs, such as [1]. However, I do agree that in this case the correct fix would be to have all architectures provide the stubs rather than the Rust code.
We have both types of interfaces in the kernel. For HAS_IOMEM and HAS_IOPORT, the link failure is intentional, as it helps identify drivers that need a Kconfig dependency and are either unusable or potentially harmful if loaded without this.
Having empty stubs only really makes sense for things like LED support where a driver calling the interfaces can continue to work correctly when the interface is compile-time disabled.
Quoted 4 lines
However, there's already a precedent for this in the kernel, e.g. in [2]. Of course, it would be better to clean this up, but depending on whether there's more architectures having this issue (I didn't check) that's separate from a fix.
arch/um is the only other one that does not always enable HAS_IOMEM, though most m68k targets don't have any support for ISA/PCI style MMIO or PIO and probably should not enable it in theory.
Right, we are definitely already inconsistent here.
Quoted 9 lines
Quoted 8 lines
The simple change below would just extend that behavior to !PCI and make that consistent with
CONFIG_PCI=yon machines without actual PCI hardware. Of course any code that might rely on this is now a bug that likely never gets caught at build time.This still relies on implementing the __raw_* helpers as nop to have the same behavior as the PCI=y version, as the generic version would just end up dereferencing the invalid pointers.
Quoted 7 lines
As mentioned, I didn't check, but if this is the only architecture causing those issues that'd be the better fix of course.
However, IIUC, your patch below would make
ioremap()and friends silenty succeed and only the accessors would prevent undefined behavior?In this case I still think
ioremap()should just fail.
In that case, it would make sense to also change the CONFIG_PCI=y version to fail the same way when the address points outside of the PCI memory space range. The current version in arch/s390/pci/pci.c just falls back to generic_ioremap_prot(), which is what I would use here directly:
void __iomem *ioremap_prot(phys_addr_t phys_addr, size_t size,
pgprot_t prot)
{
if (!static_branch_unlikely(&have_mio))
return (void __iomem *)phys_addr;
return generic_ioremap_prot(phys_addr, size, prot);
}The two methods here (generic_ioremap_prot() and the cast) are machine specific to refer to two different ways that PCI devices can be accessed if present, but there is no case for PCI being unavailable altogether.
Arnd
Gary Guo
reviewerQuoted 18 lines · Arnd Bergmann
Quoted 9 lines · Danilo Krummrich
Quoted 8 lines · Arnd Bergmann
Quoted 6 lines · Heiko Carstens
Quoted 6 lines
I just sent out a fix [1]; the only annoying part is [2], but we should change those doc-tests anyway. For the one in
rust/kernel/io.rswe already did in driver-core-next.Quoted 7 lines
Quoted 3 lines
This looks like you still provide the rust version of
ioremap(), turning what is supposed to be a link failure into a runtime error.Which is the standard for many core APIs, such as [1]. However, I do agree that in this case the correct fix would be to have all architectures provide the stubs rather than the Rust code.
Quoted 8 lines
We have both types of interfaces in the kernel. For
HAS_IOMEMandHAS_IOPORT, the link failure is intentional, as it helps identify drivers that need aKconfigdependency and are either unusable or potentially harmful if loaded without this.Having empty stubs only really makes sense for things like LED support where a driver calling the interfaces can continue to work correctly when the interface is compile-time disabled.
We can still have link failures if we always provide the signatures, just don't provide implementation?
Best,
Gary
Quoted 52 lines
Quoted 5 lines
However, there's already a precedent for this in the kernel, e.g. in [2]. Of course, it would be better to clean this up, but depending on whether there's more architectures having this issue (I didn't check) that's separate from a fix.
arch/umis the only other one that does not always enableHAS_IOMEM, though most m68k targets don't have any support for ISA/PCI style MMIO or PIO and probably should not enable it in theory.Right, we are definitely already inconsistent here.
Quoted 9 lines
Quoted 8 lines
The simple change below would just extend that behavior to !PCI and make that consistent with
CONFIG_PCI=yon machines without actual PCI hardware. Of course any code that might rely on this is now a bug that likely never gets caught at build time.This still relies on implementing the __raw_* helpers as nop to have the same behavior as the PCI=y version, as the generic version would just end up dereferencing the invalid pointers.
Quoted 7 lines
As mentioned, I didn't check, but if this is the only architecture causing those issues that'd be the better fix of course.
However, IIUC, your patch below would make
ioremap()and friends silenty succeed and only the accessors would prevent undefined behavior?In this case I still think
ioremap()should just fail.In that case, it would make sense to also change the
CONFIG_PCI=yversion to fail the same way when the address points outside of the PCI memory space range. The current version inarch/s390/pci/pci.cjust falls back togeneric_ioremap_prot(), which is what I would use here directly:void
__iomem*ioremap_prot(phys_addr_tphys_addr,size_tsize,
pgprot_tprot){ if (!static_branch_unlikely(&have_mio)) return (void __iomem *)phys_addr; return generic_ioremap_prot(phys_addr, size, prot); }The two methods here (
generic_ioremap_prot()and the cast) are machine specific to refer to two different ways that PCI devices can be accessed if present, but there is no case for PCI being unavailable altogether.Arnd
Danilo Krummrich
maintainerQuoted 14 lines · Arnd Bergmann
In that case, it would make sense to also change the
CONFIG_PCI=yversion to fail the same way when the address points outside of the PCI memory space range. The current version inarch/s390/pci/pci.cjust falls back togeneric_ioremap_prot(), which is what I would use here directly:void
__iomem*ioremap_prot(phys_addr_tphys_addr,size_tsize,
pgprot_tprot){ if (!static_branch_unlikely(&have_mio)) return (void __iomem *)phys_addr; return generic_ioremap_prot(phys_addr, size, prot); }
Quoted 4 lines
The two methods here (
generic_ioremap_prot()and the cast) are machine specific to refer to two different ways that PCI devices can be accessed if present, but there is no case for PCI being unavailable altogether.
I think we agree the correct solution is to fix things in s390. But the conversation seems to suggest that this rather results into merge window material, so I will go ahead and pull in the other fixes for -rc7.
Thanks,
Danilo
Quoted 3 lines · Arnd Bergmann
arch/umis the only other one that does not always enableHAS_IOMEM, though most m68k targets don't have any support for ISA/PCI style MMIO or PIO and probably should not enable it in theory.
While HAS_IOPORT is indeed a feature that has propagated from Intel x86
(or really 8080, etc.) to non-x86 systems through ISA/PCI what's been so special about ISA/PCI WRT HAS_IOMEM?
AFAICT HAS_IOMEM is all about just having peripherals located in the
CPU's regular memory addressing space. All the DEC MIPS platforms fall into this category, never seen PCI/ISA, some have no peripheral bus.
Maciej
Arnd Bergmann
maintainerQuoted 6 lines · Maciej W. Rozycki
Quoted 3 lines · Arnd Bergmann
arch/umis the only other one that does not always enableHAS_IOMEM, though most m68k targets don't have any support for ISA/PCI style MMIO or PIO and probably should not enable it in theory.
Quoted 7 lines
While
HAS_IOPORTis indeed a feature that has propagated from Intel x86
(or really 8080, etc.) to non-x86 systems through ISA/PCI what's been so special about ISA/PCI WRTHAS_IOMEM?AFAICT
HAS_IOMEMis all about just having peripherals located in the
CPU's regular memory addressing space. All the DEC MIPS platforms fall into this category, never seen PCI/ISA, some have no peripheral bus.
To me this is more about the driver model: On most architectures, you can use portable drivers that use IORESOURCE_MEM resources with ioremap() and the do readl()/writel() on those to access little-endian registers.
s390 and um only support this conceptually for PCI devices, and the HW implementation is different enough that the abstraction breaks down in some cases (like memremap()). m68k Amiga is similar to decstation I think, in that it has devices that are memory mapped, but they are always visible in the kernel address space and are accessed through their own abstraction instead of readl()/writel(). m68k q40 does have ISA devices with both port and memory accesses that can use portable drivers. I think other m68k platforms use a mix of ioremap()/readl()/writel() and their own lower-level accessors based on the driver, but none of those drivers are shared with another platform.
Arnd
Quoted 8 lines · Arnd Bergmann
Quoted 3 lines
AFAICT
HAS_IOMEMis all about just having peripherals located in the
CPU's regular memory addressing space. All the DEC MIPS platforms fall into this category, never seen PCI/ISA, some have no peripheral bus.To me this is more about the driver model: On most architectures, you can use portable drivers that use
IORESOURCE_MEMresources withioremap()and the do readl()/writel() on those to access little-endian registers.
Obviously I can't speak for the s390, but that's been my observation as
well -- it is a matter of coding style then rather than HAS_IOMEM being inherently PCI/ISA.
Quoted 11 lines
s390 and um only support this conceptually for PCI devices, and the HW implementation is different enough that the abstraction breaks down in some cases (like
memremap()). m68k Amiga is similar to decstation I think, in that it has devices that are memory mapped, but they are always visible in the kernel address space and are accessed through their own abstraction instead of readl()/writel(). m68k q40 does have ISA devices with both port and memory accesses that can use portable drivers. I think other m68k platforms use a mix of ioremap()/readl()/writel() and their own lower-level accessors based on the driver, but none of those drivers are shared with another platform.
On DECstation we actually have driver code shared between TURBOchannel
peripheral bus and PCI (and possibly EISA) and it obviously uses the standard MMIO accessors consistently regardless of the bus binding.
Then there is code shared between TURBOchannel and platform stuff, also
with the system models that lack a peripheral bus too. There is leftover code using non-standard accessors still there, but that's really a matter of resources needed to clean this up.
For instance drivers/net/ethernet/amd/declance.c has now been high on my
priority list to clean up and properly convert from its current mess to a regular TURBOchannel/platform device. The only issue outstanding are MMIO barriers, which have never been properly sorted out in a generic way; cf. arch/mips/include/asm/io.h and the iobarrier_*() stuff there.
Maciej
Quoted 34 lines · Arnd Bergmann
Quoted 6 lines · Heiko Carstens
Quoted 5 lines · Danilo Krummrich
Quoted 3 lines · Arnd Bergmann
In theory you should be able to use rust code without PCI MMIO support, but I can't see any practical downsides to making rust 'depends on
HAS_MMIO' to avoid having to add those #ifdef.Quoted 8 lines
Quoted 8 lines
I think the implications should be minor without making Rust depend on
CONFIG_HAS_IOMEM.I just sent out a fix [1]; the only annoying part is [2], but we should change those doc-tests anyway. For the one in
rust/kernel/io.rswe already did in driver-core-next.This looks like you still provide the rust version of
ioremap(), turning what is supposed to be a link failure into a runtime error.Quoted 3 lines
I'm wondering if it would make sense to make
HAS_IOMEMalways available on s390, even though it doesn't make too much sense without PCI. But at least it would make s390 again a bit less special.I see that with
CONFIG_PCI=y, s390 already falls back togeneric_ioremap_prot()and just maps anyphys_addr_tinto the page table asPAGE_KERNEL, regardless of whether this is an MMIO address or not.The simple change below would just extend that behavior to !PCI and make that consistent with
CONFIG_PCI=yon machines without actual PCI hardware. Of course any code that might rely on this is now a bug that likely never gets caught at build time.
Quoted 5 lines
This still relies on implementing the __raw_* helpers as nop to have the same behavior as the PCI=y version, as the generic version would just end up dereferencing the invalid pointers.
Arnd
I think there is also an interesting interaction with memremap(). That calls ioremap() under some circumstances and it's a known issue that on s390 memremap() will work if the PCI memory-I/O (MIO) instructions are enabled (e.g. on modern LPARs) because then ioremap() actually maps, but won't work if they aren't enabled. With your proposal it would also work for !PCI. Sadly, without working on the non-MIO one still really can't use memremap().
I actually have a prototype lying around where ioremap() always remaps by mapping the address cookies much like we do for user-space access via the s390 specific MMIO syscalls and then doing a page table walk in the accessors for the case where we don't have PCI MIO support. That would also allow us to implement the s390 MMIO syscall via generic_access_phys() getting rid of quite nasty inline assembly and over 300 lines removed in total. On the other hand it would cause overhead for non-MIO systems and sadly this currently includes all KVM and z/VM guests which is why it remains on my prototype pile. I haven't actually been able to measure the overhead but clearly more work is done.
Thanks,
Niklas
Arnd Bergmann
maintainerQuoted 7 lines · Niklas Schnelle
I think there is also an interesting interaction with
memremap(). That callsioremap()under some circumstances and it's a known issue that on s390memremap()will work if the PCI memory-I/O (MIO) instructions are enabled (e.g. on modern LPARs) because thenioremap()actually maps, but won't work if they aren't enabled. With your proposal it would also work for !PCI.
Right, that is an interesting one, as there are somewhat conflicting requirements:
- On PCI MMIO areas, memremap() and memremap_wt() should return a normal kernel pointer that can be dereferenced, e.g. for option ROM contents for a framebuffer console. This can't work on non-MIO guests but might work on MIO depending on which instructions are allowed those mappings (I can never quite remember how this part works on z, does this have to use pcistgi or does a normal aligned load/store work as well?).
- On areas that are backed by physical RAM but not part of the kernel memory (something like z/VM DCSS) should be mapped using memremap(), which would fail with the current !MIO implementation. On x86 and arm, this is done for mapping firmware blobs, but I don't think anything tries to do this on s390.
Right now, not having memremap() defined on !PCI configs is the one thing that lets you know if some non-PCI code ever start using it, other than crashing a non-MIO guest.
Quoted 11 lines
I actually have a prototype lying around where
ioremap()always remaps by mapping the address cookies much like we do for user-space access via the s390 specific MMIO syscalls and then doing a page table walk in the accessors for the case where we don't have PCI MIO support. That would also allow us to implement the s390 MMIO syscall viageneric_access_phys()getting rid of quite nasty inline assembly and over 300 lines removed in total. On the other hand it would cause overhead for non-MIO systems and sadly this currently includes all KVM and z/VM guests which is why it remains on my prototype pile. I haven't actually been able to measure the overhead but clearly more work is done.
This would not help with either memremap() or the !CONFIG_HAS_MMIO issue though, right?
Arnd
Quoted 12 lines · Arnd Bergmann
Quoted 7 lines · Niklas Schnelle
I think there is also an interesting interaction with
memremap(). That callsioremap()under some circumstances and it's a known issue that on s390memremap()will work if the PCI memory-I/O (MIO) instructions are enabled (e.g. on modern LPARs) because thenioremap()actually maps, but won't work if they aren't enabled. With your proposal it would also work for !PCI.Right, that is an interesting one, as there are somewhat conflicting requirements:
Quoted 7 lines
- On PCI MMIO areas,
memremap()andmemremap_wt()should return a normal kernel pointer that can be dereferenced, e.g. for option ROM contents for a framebuffer console. This can't work on non-MIO guests but might work on MIO depending on which instructions are allowed those mappings (I can never quite remember how this part works on z, does this have to use pcistgi or does a normal aligned load/store work as well?).
You still have to use pcistgi/pcilgi/pcistbi normal aligned loads/stores will fail as the physical address is beyond the memory limit. On the other hand the PCI instructions can't access normal memory. So you when using memremap() you'd have to know which kind of memory you're remapping and use the right accessors.
Quoted 23 lines
- On areas that are backed by physical RAM but not part of the kernel memory (something like z/VM DCSS) should be mapped using
memremap(), which would fail with the current !MIO implementation. On x86 and arm, this is done for mapping firmware blobs, but I don't think anything tries to do this on s390.
Right now, not havingmemremap()defined on !PCI configs is the one thing that lets you know if some non-PCI code ever start using it, other than crashing a non-MIO guest.Quoted 11 lines
I actually have a prototype lying around where
ioremap()always remaps by mapping the address cookies much like we do for user-space access via the s390 specific MMIO syscalls and then doing a page table walk in the accessors for the case where we don't have PCI MIO support. That would also allow us to implement the s390 MMIO syscall viageneric_access_phys()getting rid of quite nasty inline assembly and over 300 lines removed in total. On the other hand it would cause overhead for non-MIO systems and sadly this currently includes all KVM and z/VM guests which is why it remains on my prototype pile. I haven't actually been able to measure the overhead but clearly more work is done.
Quoted 4 lines
This would not help with either
memremap()or the !CONFIG_HAS_MMIO issue though, right?Arnd
I think it would help with memremap() because it would mean that when used on normal memory and accessed with normal loads/stores memremap() works and when used on PCI BAR / MIO addresses and accessed by I/O accessors it would also work. So it should behave the same no matter if memory-I/O is available.
Thanks,
Niklas
Arnd Bergmann
maintainerQuoted 10 lines · Niklas Schnelle
Quoted 8 lines · Arnd Bergmann
On Tue, Aug 4, 2026, at 14:21, Niklas Schnelle wrote:
- On PCI MMIO areas,memremap()andmemremap_wt()should return a normal kernel pointer that can be dereferenced, e.g. for option ROM contents for a framebuffer console. This can't work on non-MIO guests but might work on MIO depending on which instructions are allowed those mappings (I can never quite remember how this part works on z, does this have to use pcistgi or does a normal aligned load/store work as well?).
Quoted 5 lines
You still have to use pcistgi/pcilgi/pcistbi normal aligned loads/stores will fail as the physical address is beyond the memory limit. On the other hand the PCI instructions can't access normal memory. So you when using
memremap()you'd have to know which kind of memory you're remapping and use the right accessors.
Ok, so memremap() by definition cannot work on s390 for regular PCI devices, since you are not allowed to use readl() etc on a kernel pointer returned by memremap(), only on an __iomem token returned by ioremap().
One important exception seems to be virtio_fs, which uses devm_memremap_pages() to map a virtio_shm_region, which in turn can come from a virtio-pci device but is backed by actual cached memory in the host instead of an emulated PCI memory BAR.
Quoted 8 lines
Quoted 2 lines
This would not help with either
memremap()or the !CONFIG_HAS_MMIO issue though, right?I think it would help with
memremap()because it would mean that when used on normal memory and accessed with normal loads/storesmemremap()works and when used on PCI BAR / MIO addresses and accessed by I/O accessors it would also work. So it should behave the same no matter if memory-I/O is available.
Have you come across any users of memremap() that pass the kernel address into readl()/writel() rather than accessing them as pointers? This would of course work on arm and x86, but is still a violation of the linux/io.h interface definition and causes a compile-time warning with sparse. If these exist, I think we just need to change them to ioremap().
Arnd
Quoted 22 lines · Arnd Bergmann
Quoted 10 lines · Niklas Schnelle
Quoted 8 lines · Arnd Bergmann
On Tue, Aug 4, 2026, at 14:21, Niklas Schnelle wrote:
- On PCI MMIO areas,memremap()andmemremap_wt()should return a normal kernel pointer that can be dereferenced, e.g. for option ROM contents for a framebuffer console. This can't work on non-MIO guests but might work on MIO depending on which instructions are allowed those mappings (I can never quite remember how this part works on z, does this have to use pcistgi or does a normal aligned load/store work as well?).Quoted 5 lines
You still have to use pcistgi/pcilgi/pcistbi normal aligned loads/stores will fail as the physical address is beyond the memory limit. On the other hand the PCI instructions can't access normal memory. So you when using
memremap()you'd have to know which kind of memory you're remapping and use the right accessors.Ok, so
memremap()by definition cannot work on s390 for regular PCI devices, since you are not allowed to usereadl()etc on a kernel pointer returned bymemremap(), only on an__iomemtoken returned byioremap().
Quoted 5 lines
One important exception seems to be
virtio_fs, which usesdevm_memremap_pages()to map avirtio_shm_region, which in turn can come from a virtio-pci device but is backed by actual cached memory in the host instead of an emulated PCI memory BAR.
Interesting, do I see that right that devm_memremap_pages() and memremap_pages() unlike plain memremap() does not utilize ioremap() and thus isn't affected by our nopped ioreamp() when there is no-MIO?
Quoted 10 lines
Quoted 9 lines
Quoted 3 lines
This would not help with either
memremap()or the !CONFIG_HAS_MMIO issue though, right?I think it would help with
memremap()because it would mean that when used on normal memory and accessed with normal loads/storesmemremap()works and when used on PCI BAR / MIO addresses and accessed by I/O accessors it would also work. So it should behave the same no matter if memory-I/O is available.
Quoted 8 lines
Have you come across any users of
memremap()that pass the kernel address into readl()/writel() rather than accessing them as pointers? This would of course work on arm and x86, but is still a violation of thelinux/io.hinterface definition and causes a compile-time warning with sparse. If these exist, I think we just need to change them toioremap().Arnd
No, I'm not aware of any users of memremap() that would then use readl()/writel() and I agree they really should be using ioremap() and the use case doesn't really make sense I was just thinking about what would technically work with the prototype I have.
Thanks,
Niklas
Arnd Bergmann
maintainerQuoted 13 lines · Niklas Schnelle
Quoted 11 lines · Arnd Bergmann
On Wed, Aug 5, 2026, at 17:36, Niklas Schnelle wrote:
Ok, somemremap()by definition cannot work on s390 for regular PCI devices, since you are not allowed to usereadl()etc on a kernel pointer returned bymemremap(), only on an__iomemtoken returned byioremap().One important exception seems to be
virtio_fs, which usesdevm_memremap_pages()to map avirtio_shm_region, which in turn can come from a virtio-pci device but is backed by actual cached memory in the host instead of an emulated PCI memory BAR.
Quoted 3 lines
Interesting, do I see that right that
devm_memremap_pages()andmemremap_pages()unlike plainmemremap()does not utilizeioremap()and thus isn't affected by our noppedioreamp()when there is no-MIO?
Indeed, I had not noticed that before, but you are right that this goes directly to the page tables while the other variants of memremap() go through architecture specific ioremap_prot(), which may or may not create page table entries.
I think the reason is that there are a couple of architectures (including non-MIO s390) that take a shortcut for normal ioremap() that avoids actually mapping anything, e.g. mips and related architectures have a fixed uncached mapping of all MMIO space visible to kernel, while some alpha, m68k powerpc, and s390 do something other than a pointer dereference for MMIO.
Arnd
Hi Arnd,
Quoted 7 lines · Arnd Bergmann
I think the reason is that there are a couple of architectures (including non-MIO s390) that take a shortcut for normal
ioremap()that avoids actually mapping anything, e.g. mips and related architectures have a fixed uncached mapping of all MMIO space visible to kernel, while some alpha, m68k powerpc, and s390 do something other than a pointer dereference for MMIO.
FTR, several m68k platforms use the transparent translation registers to set up a mapping to the main system's MMIO space (like MIPS, but programmable), and thus just need to add an offset.
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Quoted 7 lines · Arnd Bergmann
I think the reason is that there are a couple of architectures (including non-MIO s390) that take a shortcut for normal
ioremap()that avoids actually mapping anything, e.g. mips and related architectures have a fixed uncached mapping of all MMIO space visible to kernel, while some alpha, m68k powerpc, and s390 do something other than a pointer dereference for MMIO.
FWIW 32-bit MIPS configurations only have low 512MiB of MMIO accessible
via a fixed mapping and arrange for a regular mapping via the page table for anything beyond.
Maciej