Skip to contentSkip to conversation

Allow deferred execution of iomem_get_mapping()

PATCH Superseded Krzysztof Wilczyński 4 participants 14 messages Jun 25, 2021 – Jun 28, 2021

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#1 lore raw

PATCH 1/2 sysfs: Invoke iomem_get_mapping() from the sysfs open callback

In reply to Krzysztof Wilczyński

Defer invocation of the iomem_get_mapping() to the sysfs open callback so that it can be executed as needed when the binary sysfs object has been accessed.

To do that, convert the "mapping" member of the struct bin_attribute from a pointer to the struct address_space into a function pointer with a signature that requires the same return type, and then updates the sysfs_kf_bin_open() to invoke provided function should the function pointer be valid.

Thus, this change removes the need for the fs_initcalls to complete before any other sub-system that uses the iomem_get_mapping() would be able to invoke it safely without leading to a failure and an Oops related to an invalid iomem_get_mapping() access.

---
 fs/sysfs/file.c       | 2 +-
 include/linux/sysfs.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
fs/sysfs/file.c +1 -1
index 9aefa7779b29..a3ee4c32a264 100644
@@ -175,7 +175,7 @@ static int sysfs_kf_bin_open(struct kernfs_open_file *of)
 	struct bin_attribute *battr = of->kn->priv;
 
 	if (battr->mapping)
-		of->file->f_mapping = battr->mapping;
+		of->file->f_mapping = battr->mapping();
 
 	return 0;
 }
include/linux/sysfs.h +1 -1
index d76a1ddf83a3..fbb7c7df545c 100644
@@ -170,7 +170,7 @@ struct bin_attribute {
 	struct attribute	attr;
 	size_t			size;
 	void			*private;
-	struct address_space	*mapping;
+	struct address_space *(*mapping)(void);
 	ssize_t (*read)(struct file *, struct kobject *, struct bin_attribute *,
 			char *, loff_t, size_t);
 	ssize_t (*write)(struct file *, struct kobject *, struct bin_attribute *,
Signature · 1 line
2.32.0
#0 lore raw

cover Allow deferred execution of iomem_get_mapping()

Hello,

At the moment, the dependency on iomem_get_mapping() that is currently used in the pci_create_resource_files() and pci_create_legacy_files() stops us from completely retiring the late_initcall() that is used to invoke pci_sysfs_init() when creating sysfs object for PCI devices.

This dependency on iomem_get_mapping() stops us from retiring the late_initcall at the moment as when we convert dynamically added sysfs objects, that are primarily added in the pci_create_resource_files() and pci_create_legacy_files(), as these attributes are added before the VFS completes its initialisation, and since most of the PCI devices are typically enumerated in subsys_initcall this leads to a failure and an Oops related to iomem_get_mapping() access.

See relevant conversations:
https://lore.kernel.org/linux-pci/[email protected]/
https://lore.kernel.org/linux-pci/20210313215747.GA2394467@bjorn-Precision-5520/

After some deliberation about the problem at hand, Dan Williams suggested a solution to the problem, see:
https://lore.kernel.org/linux-pci/CAPcyv4i0y_4cMGEpNVShLUyUk3nyWH203Ry3S87BqnDJE0Rmxg@mail.gmail.com/

The idea is to defer execution of the iomem_get_mapping() to only when the sysfs open callback is run, and thus removing the reliance of fs_initcalls to complete before any other sub-system that uses the iomem_get_mapping().

Currently, the PCI sub-system will benefit the most from this change allowing for it to complete the transition from dynamically created to static sysfs objects.

This series aims to take Dan Williams' idea through the finish line.

Related to:
https://lore.kernel.org/linux-pci/20210527205845.GA1421476@bjorn-Precision-5520/
https://lore.kernel.org/linux-pci/[email protected]/
https://lore.kernel.org/linux-pci/20200716110423.xtfyb3n6tn5ixedh@pali/

Krzysztof

Krzysztof Wilczyński (2):
sysfs: Invoke iomem_get_mapping() from the sysfs open callback PCI/sysfs: Pass iomem_get_mapping() as a function pointer

 drivers/pci/pci-sysfs.c | 6 +++---
 fs/sysfs/file.c         | 2 +-
 include/linux/sysfs.h   | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)
Signature · 1 line
2.32.0
#8 lore raw

PATCH 2/2 PCI/sysfs: Pass iomem_get_mapping() as a function pointer

The struct bin_attribute requires the "mapping" member to be a function pointer with a signature requiring the return type to be a pointer to the struct address_space.

Thus, convert every invocation of iomem_get_mapping() into a function pointer assignment, therefore allowing for the iomem_get_mapping() invocation to be deferred to when the sysfs open callback runs.

---
 drivers/pci/pci-sysfs.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/pci/pci-sysfs.c +3 -3
index beb8d1f4fafe..cff1c121eb08 100644
@@ -965,7 +965,7 @@ void pci_create_legacy_files(struct pci_bus *b)
 	b->legacy_io->read = pci_read_legacy_io;
 	b->legacy_io->write = pci_write_legacy_io;
 	b->legacy_io->mmap = pci_mmap_legacy_io;
-	b->legacy_io->mapping = iomem_get_mapping();
+	b->legacy_io->mapping = iomem_get_mapping;
 	pci_adjust_legacy_attr(b, pci_mmap_io);
 	error = device_create_bin_file(&b->dev, b->legacy_io);
 	if (error)
@@ -978,7 +978,7 @@ void pci_create_legacy_files(struct pci_bus *b)
 	b->legacy_mem->size = 1024*1024;
 	b->legacy_mem->attr.mode = 0600;
 	b->legacy_mem->mmap = pci_mmap_legacy_mem;
-	b->legacy_io->mapping = iomem_get_mapping();
+	b->legacy_io->mapping = iomem_get_mapping;
 	pci_adjust_legacy_attr(b, pci_mmap_mem);
 	error = device_create_bin_file(&b->dev, b->legacy_mem);
 	if (error)
@@ -1195,7 +1195,7 @@ static int pci_create_attr(struct pci_dev *pdev, int num, int write_combine)
 		}
 	}
 	if (res_attr->mmap)
-		res_attr->mapping = iomem_get_mapping();
+		res_attr->mapping = iomem_get_mapping;
 	res_attr->attr.name = res_attr_name;
 	res_attr->attr.mode = 0600;
 	res_attr->size = pci_resource_len(pdev, num);
Signature · 1 line
2.32.0
#2 lore raw

Re: [PATCH 1/2] sysfs: Invoke iomem_get_mapping() from the sysfs open callback

In reply to Krzysztof Wilczyński
Quoted 11 lines · Krzysztof Wilczyński

Defer invocation of the iomem_get_mapping() to the sysfs open callback so that it can be executed as needed when the binary sysfs object has been accessed.

To do that, convert the "mapping" member of the struct bin_attribute from a pointer to the struct address_space into a function pointer with a signature that requires the same return type, and then updates the sysfs_kf_bin_open() to invoke provided function should the function pointer be valid.

Quoted 6 lines

Thus, this change removes the need for the fs_initcalls to complete before any other sub-system that uses the iomem_get_mapping() would be able to invoke it safely without leading to a failure and an Oops related to an invalid iomem_get_mapping() access.

Go ahead and replace this with:

Reviewed-by on v1 2/2
#9 lore raw

Re: [PATCH 2/2] PCI/sysfs: Pass iomem_get_mapping() as a function pointer

In reply to Krzysztof Wilczyński
Quoted 9 lines · Krzysztof Wilczyński

The struct bin_attribute requires the "mapping" member to be a function pointer with a signature requiring the return type to be a pointer to the struct address_space.

Thus, convert every invocation of iomem_get_mapping() into a function pointer assignment, therefore allowing for the iomem_get_mapping() invocation to be deferred to when the sysfs open callback runs.

Looks good. Since I did not write any of the below go ahead and change the "Co-authored-by" to:

Thanks for picking this up and carrying it through.

#10 lore raw

Re: [PATCH 2/2] PCI/sysfs: Pass iomem_get_mapping() as a function pointer

Hi Dan,

[...]

Quoted 4 lines

Looks good. Since I did not write any of the below go ahead and change the "Co-authored-by" to:

Thank you!

Quoted 1 line

Thanks for picking this up and carrying it through.

Of course, any time! Also, thank you for suggesting how to solve the problem we've had and for sending the initial patch!

Krzysztof

#3 lore raw

Re: [PATCH 1/2] sysfs: Invoke iomem_get_mapping() from the sysfs open callback

In reply to Krzysztof Wilczyński

fs/sysfs/file.c static int sysfs_kf_bin_open(struct kernfs_open_file *of) · line 178

Quoted 3 lines · Krzysztof Wilczy??ski
fs/sysfs/file.c +1 -1
 	if (battr->mapping)
-		of->file->f_mapping = battr->mapping;
+		of->file->f_mapping = battr->mapping();

I think get_mapping() is a better name now. That being said this whole programming model looks a little weird.

Also, does this patch imply the mapping field of the sysfs bin attributes wasn't used before at all?

#12 lore raw

Re: [PATCH 2/2] PCI/sysfs: Pass iomem_get_mapping() as a function pointer

Hi Christoph,

Quoted 2 lines

Doesn't this need to be merged into the previous patch to prevent a compile failure after just the previous patch is applied?

Yes, it does. I kept it separate for the sake of review, since we have sysfs and PCI involved. I wasn't sure if Bjorn would prefer to have this done as separate patches or not, to be honest.

Bjorn said that he can squash this when applying, thus I left it as-is for now - which means that the entire series has to be applied for everything to build cleanly.

I will send v3 that merges first two patches. Sorry for troubles!

Krzysztof

#4 lore raw

Re: [PATCH 1/2] sysfs: Invoke iomem_get_mapping() from the sysfs open callback

In reply to Christoph Hellwig

Hi Christoph,

Quoted 6 lines

fs/sysfs/file.c static int sysfs_kf_bin_open(struct kernfs_open_file *of) · line 178

Quoted 3 lines
fs/sysfs/file.c +1 -1
 	if (battr->mapping)
-		of->file->f_mapping = battr->mapping;
+		of->file->f_mapping = battr->mapping();

I think get_mapping() is a better name now. That being said this whole programming model looks a little weird.

I would have to lean on Daniel and Dan here as they might have more context than I do, especially since in the PCI world we are only consumers of this new API. The related patches are:

commit 3234ac664a87 ("/dev/mem: Revoke mappings when a driver claims the region") commit 636b21b50152 ("PCI: Revoke mappings like devmem")

Quoted 2 lines

Also, does this patch imply the mapping field of the sysfs bin attributes wasn't used before at all?

No, everything worked as intended, thankfully. Changes here are meant to help us transition to use static sysfs objects when we add various PCI-related attributes a particular device. This in turn will allow us to remove the need for late_initcall() in the drivers/pci/pci-sysfs.c, and thus fix the race condition people noticed on some platforms when sysfs objects are being added while PCI sub-system and devices are initialised.

More details are captured in the following conversations:
https://lore.kernel.org/linux-pci/20200716110423.xtfyb3n6tn5ixedh@pali/
https://lore.kernel.org/linux-pci/20210527205845.GA1421476@bjorn-Precision-5520/
https://lore.kernel.org/linux-pci/20210313215747.GA2394467@bjorn-Precision-5520/

Dan's original patch:
https://lore.kernel.org/linux-pci/CAPcyv4i0y_4cMGEpNVShLUyUk3nyWH203Ry3S87BqnDJE0Rmxg@mail.gmail.com/

Krzysztof

#13 lore raw

Re: [PATCH 2/2] PCI/sysfs: Pass iomem_get_mapping() as a function pointer

In reply to Krzysztof Wilczy??ski
Quoted 9 lines · Krzysztof Wilczy??ski

Hi Christoph,

Quoted 2 lines

Doesn't this need to be merged into the previous patch to prevent a compile failure after just the previous patch is applied?

Yes, it does. I kept it separate for the sake of review, since we have sysfs and PCI involved. I wasn't sure if Bjorn would prefer to have this done as separate patches or not, to be honest.

Quoted 5 lines

Bjorn said that he can squash this when applying, thus I left it as-is for now - which means that the entire series has to be applied for everything to build cleanly.

I will send v3 that merges first two patches. Sorry for troubles!

This all needs to wait until after 5.14-rc1 is out anyway, so no rush.

greg k-h

#5 lore raw

Re: [PATCH 1/2] sysfs: Invoke iomem_get_mapping() from the sysfs open callback

In reply to Christoph Hellwig
Quoted 8 lines · Christoph Hellwig

fs/sysfs/file.c static int sysfs_kf_bin_open(struct kernfs_open_file *of) · line 178

Quoted 3 lines · Krzysztof Wilczy??ski
      if (battr->mapping)
fs/sysfs/file.c +1 -1
-             of->file->f_mapping = battr->mapping;
+             of->file->f_mapping = battr->mapping();

I think get_mapping() is a better name now. That being said this whole programming model looks a little weird.

I think both those points are fair.

Quoted 2 lines

Also, does this patch imply the mapping field of the sysfs bin attributes wasn't used before at all?

It defaulted to an address_space per file rather than a shared address space across all files that map physical addresses as file offsets.

#6 lore raw

Re: [PATCH 1/2] sysfs: Invoke iomem_get_mapping() from the sysfs open callback

Hi Dan,

Quoted 11 lines · Dan Williams
Quoted 8 lines · Christoph Hellwig

fs/sysfs/file.c static int sysfs_kf_bin_open(struct kernfs_open_file *of) · line 178

Quoted 3 lines · Krzysztof Wilczy??ski
      if (battr->mapping)
fs/sysfs/file.c +1 -1
-             of->file->f_mapping = battr->mapping;
+             of->file->f_mapping = battr->mapping();

I think get_mapping() is a better name now. That being said this whole programming model looks a little weird.

I think both those points are fair.

Anything for us to do?

Quoted 5 lines
Quoted 2 lines

Also, does this patch imply the mapping field of the sysfs bin attributes wasn't used before at all?

It defaulted to an address_space per file rather than a shared address space across all files that map physical addresses as file offsets.

I will include this in the commit message for v3 of the patch series, if you don't mind. Also, would this shared address space be a potential issue? Security, functional, etc.

Krzysztof

#7 lore raw

Re: [PATCH 1/2] sysfs: Invoke iomem_get_mapping() from the sysfs open callback

Quoted 21 lines · Krzysztof Wilczyński

Hi Dan,

Quoted 11 lines · Dan Williams
Quoted 8 lines · Christoph Hellwig

fs/sysfs/file.c static int sysfs_kf_bin_open(struct kernfs_open_file *of) · line 178

Quoted 3 lines · Krzysztof Wilczy??ski
      if (battr->mapping)
fs/sysfs/file.c +1 -1
-             of->file->f_mapping = battr->mapping;
+             of->file->f_mapping = battr->mapping();

I think get_mapping() is a better name now. That being said this whole programming model looks a little weird.

I think both those points are fair.

Anything for us to do?

Quoted 3 lines
Quoted 2 lines

Also, does this patch imply the mapping field of the sysfs bin attributes wasn't used before at all?

Quoted 6 lines
Quoted 2 lines

It defaulted to an address_space per file rather than a shared address space across all files that map physical addresses as file offsets.

I will include this in the commit message for v3 of the patch series, if you don't mind. Also, would this shared address space be a potential issue? Security, functional, etc.

The shared address_space arrangement is what allows for a "revoke" mechanism for /dev/mem and pci-resource-sysfs mappings. Without a shared address space there would need to be tracking for each 'inode' instance to run revoke_iomem(). Note that this shared address_space scheme is also deployed for block-device special files, see blkdev_open(). It's done for similar reasons of allowing all address_space operations to act on a common representation of the single block-device.