Skip to contentSkip to conversation

sysfs for my chips

PATCH In review Benjamin Herrenschmidt 3 participants 11 messages Oct 10, 2013 – Oct 11, 2013 Driver core

lore
Share & tools
lore
#0 lore raw

Hi Greg !

(random CC list of clueful people)

On some new powerpc platforms (non-hypervisor or rather linux is the hypervisor), I want to expose a bunch of stuff per "chip", the chips being currently the processor chips and the "centaurs" (think of them as the bottom half of the memory controllers).

Among other, I want a sysfs file in there to access "xscom" on the chip which is a sideband bus used for low level stuff (think jtag on steroid) which we can use, among others, for chip health monitoring, general debugging and diagnostics, etc...

I might add more such as VPD, model information, etc... later or at least a link to corresponding device-tree node.

How do you suggest I expose that ? So far I've been thinking about something like

/sys/chips/{processor,centaur}/chip#/files

or to avoid namespace clashes

/sys/firmware/chips/{processor,centaur}/chip#/files

Or maybe just

/sys/firmware/chips/chip#/files

(the chip type can be inferred from the chip#, they use the same space at least as far my firmware exposes them to Linux)

(the actual access to xscom goes via firmware tho it makes *some* sense)

But I could instead create platform devices corresponding to the device-tree representation of each of those chips ... and have the platform devices contain the magic attributes. That's a bit more convoluted though.

What's the current trend of the day for that sort of thing ? I'd rather avoid yet-another-chardev-with-ioctl's here ...

Cheers,
Ben.

#1 lore raw

PATCH 1/1 sysfs/bin: Fix size handling overflow for bin_attribute

While looking at the code, I noticed that bin_attribute read() and write() ops copy the inode size into an int for futher comparisons.

Some bin_attributes can be fairly large. For example, pci creates some for BARs set to the BAR size and giant BARs are around the corner, so this is going to break something somewhere eventually.

Let's use the right type.

---

I noticed that while messing around with my "xscom" file which I had originally set to be LONG_MAX :-)

I eventually decided to use an i_size of 0 instead which seems to be the way that sort of "special" file tend to be done (it's a bridge to a special sideband bus in the system which has a sparse addressing which is splattered all over 64 bits though I've somewhat "compressed" it to 63 for the sake of sysfs).

Note that I noticed that late and don't have a good test case for it, but code inspection didn't seem to show anything else cropping i_size.

fs/sysfs/bin.c +2 -2
index c590cab..373ddcf 100644
@@ -69,7 +69,7 @@ static ssize_t
 read(struct file *file, char __user *userbuf, size_t bytes, loff_t *off)
 {
 	struct bin_buffer *bb = file->private_data;
-	int size = file_inode(file)->i_size;
+	loff_t size = file_inode(file)->i_size;
 	loff_t offs = *off;
 	int count = min_t(size_t, bytes, PAGE_SIZE);
 	char *temp;
@@ -139,7 +139,7 @@ static ssize_t write(struct file *file, const char __user *userbuf,
 		     size_t bytes, loff_t *off)
 {
 	struct bin_buffer *bb = file->private_data;
-	int size = file_inode(file)->i_size;
+	loff_t size = file_inode(file)->i_size;
 	loff_t offs = *off;
 	int count = min_t(size_t, bytes, PAGE_SIZE);
 	char *temp;
maintainer
#2 lore raw

Re: [PATCH] sysfs/bin: Fix size handling overflow for bin_attribute

Quoted 10 lines · Benjamin Herrenschmidt

While looking at the code, I noticed that bin_attribute read() and write() ops copy the inode size into an int for futher comparisons.

Some bin_attributes can be fairly large. For example, pci creates some for BARs set to the BAR size and giant BARs are around the corner, so this is going to break something somewhere eventually.

Let's use the right type.

This file (fs/sysfs/bin.c) is no longer in linux-next as Tejun merged it with the other file handling code in sysfs, to remove a bunch of duplicated logic.

So, could you look at linux-next and see if this is still an issue there or not?

thanks,

greg k-h

maintainer
#3 lore raw

Re: [PATCH] sysfs/bin: Fix size handling overflow for bin_attribute

Quoted 10 lines · Greg KH
Quoted 9 lines · Benjamin Herrenschmidt

While looking at the code, I noticed that bin_attribute read() and write() ops copy the inode size into an int for futher comparisons.

Some bin_attributes can be fairly large. For example, pci creates some for BARs set to the BAR size and giant BARs are around the corner, so this is going to break something somewhere eventually.

Let's use the right type.

Quoted 8 lines
Quoted 1 line

This file (fs/sysfs/bin.c) is no longer in linux-next as Tejun merged it with the other file handling code in sysfs, to remove a bunch of duplicated logic.

So, could you look at linux-next and see if this is still an issue there or not?

Ah, nevermind, I just looked, and it still is an issue there, I'll hand-edit the patch to apply properly, thanks.

greg k-h

maintainer
#5 lore raw
In reply to Benjamin Herrenschmidt
Quoted 9 lines · Benjamin Herrenschmidt

Hi Greg !

(random CC list of clueful people)

On some new powerpc platforms (non-hypervisor or rather linux is the hypervisor), I want to expose a bunch of stuff per "chip", the chips being currently the processor chips and the "centaurs" (think of them as the bottom half of the memory controllers).

Quoted 7 lines

Among other, I want a sysfs file in there to access "xscom" on the chip which is a sideband bus used for low level stuff (think jtag on steroid) which we can use, among others, for chip health monitoring, general debugging and diagnostics, etc...

I might add more such as VPD, model information, etc... later or at least a link to corresponding device-tree node.

So a mixture of things that traditionally have been in /proc/cpuinfo?

I've always wanted to see the cpuinfo files turn into sysfs files, so that tools can parse them "properly" and not have to do different things on different arches, like the proc/cpuinfo file is today (a mess).

Quoted 16 lines

How do you suggest I expose that ? So far I've been thinking about something like

/sys/chips/{processor,centaur}/chip#/files

or to avoid namespace clashes

/sys/firmware/chips/{processor,centaur}/chip#/files

Or maybe just

/sys/firmware/chips/chip#/files

(the chip type can be inferred from the chip#, they use the same space at least as far my firmware exposes them to Linux)

Quoted 6 lines

(the actual access to xscom goes via firmware tho it makes *some* sense)

But I could instead create platform devices corresponding to the device-tree representation of each of those chips ... and have the platform devices contain the magic attributes. That's a bit more convoluted though.

We already create platform devices for the cpus in the system in /sys/devices/system/, so can you just hang the attribute files off of those platform devices?

And system devices seem like the correct thing for your "chips" that are not cpus, right?

thanks,

greg k-h

#6 lore raw
Quoted 15 lines · Greg KH
Quoted 14 lines · Benjamin Herrenschmidt

Hi Greg !

(random CC list of clueful people)

On some new powerpc platforms (non-hypervisor or rather linux is the hypervisor), I want to expose a bunch of stuff per "chip", the chips being currently the processor chips and the "centaurs" (think of them as the bottom half of the memory controllers).

Among other, I want a sysfs file in there to access "xscom" on the chip which is a sideband bus used for low level stuff (think jtag on steroid) which we can use, among others, for chip health monitoring, general debugging and diagnostics, etc...

Quoted 4 lines
Quoted 2 lines

I might add more such as VPD, model information, etc... later or at least a link to corresponding device-tree node.

So a mixture of things that traditionally have been in /proc/cpuinfo?

Not really no. /proc/cpuinfo is per-thread, and potentially might have a core number and a few global things. Here I'm talking mostly about

- A pseudo-file that gives access to a sideband HW bus to perform
read/write of system registers essentially, which is per physical "chip" (chip in this context is processor chip, which can have lots of cores/threads, but also can be our memory buffer chips).

- The other stuff is much more than the kind of one-liner than one
finds in cpuinfo. For now I'm just slapping the usual "devspec" file that contains the corresponding device-tree path and whatever additional info can be retrieved from there. Things like VPDs for example can be pretty big.

Quoted 22 lines

I've always wanted to see the cpuinfo files turn into sysfs files, so that tools can parse them "properly" and not have to do different things on different arches, like the proc/cpuinfo file is today (a mess).

Quoted 18 lines

How do you suggest I expose that ? So far I've been thinking about something like

/sys/chips/{processor,centaur}/chip#/files

or to avoid namespace clashes

/sys/firmware/chips/{processor,centaur}/chip#/files

Or maybe just

/sys/firmware/chips/chip#/files

(the chip type can be inferred from the chip#, they use the same space at least as far my firmware exposes them to Linux)

(the actual access to xscom goes via firmware tho it makes *some* sense)

Quoted 8 lines
Quoted 4 lines

But I could instead create platform devices corresponding to the device-tree representation of each of those chips ... and have the platform devices contain the magic attributes. That's a bit more convoluted though.

We already create platform devices for the cpus in the system in /sys/devices/system/, so can you just hang the attribute files off of those platform devices?

This is not CPUs. CPUs are threads really. Even if they were cores, that still wouldn't cut it. I'm looking at chips here and not all of them actually processor chips. The SCOM bus address space is global to a physical chip and allows to access various resources that are only remotely related to the cores on it.

Quoted 2 lines

And system devices seem like the correct thing for your "chips" that are not cpus, right?

Well, I could use system devices for everything I don't see much point, since those things won't have a "driver" per-se, I don't actually need any of the other infrastructure provided by a system device here. I've posted a patch (which I should have labelled RFC) there with my current experimental code:

http://patchwork.ozlabs.org/patch/282167/

Right now I do /sys/firmware/scom/<chip#>/{devspec,access} but of course I can easily change that.

Cheers,
Ben.

Quoted 8 lines

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#4 lore raw

Re: [PATCH] sysfs/bin: Fix size handling overflow for bin_attribute

In reply to Greg KH
Quoted 9 lines · Greg KH
Quoted 6 lines

This file (fs/sysfs/bin.c) is no longer in linux-next as Tejun merged it with the other file handling code in sysfs, to remove a bunch of duplicated logic.

So, could you look at linux-next and see if this is still an issue there or not?

Ah, nevermind, I just looked, and it still is an issue there, I'll hand-edit the patch to apply properly, thanks.

Ah sorry, nice to see that Tejun cleaned that stuff up, it took me a while to find my way around :-)

Cheers,
Ben.

#7 lore raw
In reply to Benjamin Herrenschmidt
Quoted 55 lines · Benjamin Herrenschmidt
Quoted 9 lines
Quoted 7 lines

Among other, I want a sysfs file in there to access "xscom" on the chip which is a sideband bus used for low level stuff (think jtag on steroid) which we can use, among others, for chip health monitoring, general debugging and diagnostics, etc...

I might add more such as VPD, model information, etc... later or at least a link to corresponding device-tree node.

So a mixture of things that traditionally have been in /proc/cpuinfo?

Not really no. /proc/cpuinfo is per-thread, and potentially might have a core number and a few global things. Here I'm talking mostly about

- A pseudo-file that gives access to a sideband HW bus to perform
read/write of system registers essentially, which is per physical "chip" (chip in this context is processor chip, which can have lots of cores/threads, but also can be our memory buffer chips).

- The other stuff is much more than the kind of one-liner than one
finds in cpuinfo. For now I'm just slapping the usual "devspec" file that contains the corresponding device-tree path and whatever additional info can be retrieved from there. Things like VPDs for example can be pretty big.

Quoted 22 lines

I've always wanted to see the cpuinfo files turn into sysfs files, so that tools can parse them "properly" and not have to do different things on different arches, like the proc/cpuinfo file is today (a mess).

Quoted 18 lines

How do you suggest I expose that ? So far I've been thinking about something like

/sys/chips/{processor,centaur}/chip#/files

or to avoid namespace clashes

/sys/firmware/chips/{processor,centaur}/chip#/files

Or maybe just

/sys/firmware/chips/chip#/files

(the chip type can be inferred from the chip#, they use the same space at least as far my firmware exposes them to Linux)

(the actual access to xscom goes via firmware tho it makes *some* sense)

Quoted 8 lines
Quoted 4 lines

But I could instead create platform devices corresponding to the device-tree representation of each of those chips ... and have the platform devices contain the magic attributes. That's a bit more convoluted though.

We already create platform devices for the cpus in the system in /sys/devices/system/, so can you just hang the attribute files off of those platform devices?

Quoted 5 lines

This is not CPUs. CPUs are threads really. Even if they were cores, that still wouldn't cut it. I'm looking at chips here and not all of them actually processor chips. The SCOM bus address space is global to a physical chip and allows to access various resources that are only remotely related to the cores on it.

What about a "bus" for the sideband bus? That allows to decouple it from cores, and allows to support non-processor chips, too?

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#8 lore raw
Quoted 9 lines · Geert Uytterhoeven
Quoted 1 line

This is not CPUs. CPUs are threads really. Even if they were cores,

that

Quoted 4 lines

still wouldn't cut it. I'm looking at chips here and not all of them actually processor chips. The SCOM bus address space is global to a physical chip and allows to access various resources that are only remotely related to the cores on it.

What about a "bus" for the sideband bus? That allows to decouple it from cores, and allows to support non-processor chips, too?

Not sure what you mean .... create a linux bus type with devices on it ?

That's really overkill I think... doable though.

I think sysdev, despite my previous qualms, might be the best way... we could have a "driver" in the form of the actual code that provide that sideband bus access. Right now the platform registers a single global "scom controller", we could make a sysdev instead with an instance per "chip" I suppose...

Though there's still some kind of namespace issue, if I start creating something like /sys/devices/system/chip/<id>/... but ...

That means doing something very platform specific in a location that is very "generic" with a name that's likely to clash with something else later on unless we start introducing a standard concept of "chip" with associated properties but what do that really mean ? With things like DCMs etc... the concept of "chip" is blurry at best.

Also what about other chips that do not participate in that xscom/scom sideband mechanism ? like PCIe device chips, or other random stuff on the mobo ?

I'm trying to solve a very specific problem here, which is to provide a userspace scom access interface. I was trying to avoid a /dev/scom because I would need 32-bit minors or do ioctls and because it seemed simpler to just put it in sysfs somewhere but maybe that won't fly.

A last option is to put it in /sys/firmware/opal (OPAL is our firmware name). Under there, I can put a lot of stuff that's essentially firmware/platform specific. However our "scom" infrastructure is somewhat generic and is used by at least one platform that isn't using OPAL (though arguably that platform is dead: wsp).

Cheers,
Ben.

#9 lore raw
Quoted 8 lines · Benjamin Herrenschmidt
Quoted 7 lines · Geert Uytterhoeven
Quoted 1 line

This is not CPUs. CPUs are threads really. Even if they were cores,

that

Quoted 4 lines

still wouldn't cut it. I'm looking at chips here and not all of them actually processor chips. The SCOM bus address space is global to a physical chip and allows to access various resources that are only remotely related to the cores on it.

Quoted 5 lines
Quoted 2 lines

What about a "bus" for the sideband bus? That allows to decouple it from cores, and allows to support non-processor chips, too?

Not sure what you mean .... create a linux bus type with devices on it ?

Yes, that's what I meant.

Quoted 1 line

From the nodes on that bus you can have symlinks in sysfs to e.g. the CPUs

in the rest of the sysfs tree.

It's a bit like smbus/i2c connecting various peripherals that may also be connected in some other way (e.g. media devices).

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#10 lore raw
Quoted 10 lines · Geert Uytterhoeven
Quoted 2 lines

Not sure what you mean .... create a linux bus type with devices on it ?

Yes, that's what I meant.

Quoted 1 line

From the nodes on that bus you can have symlinks in sysfs to e.g. the CPUs

in the rest of the sysfs tree.

It's a bit like smbus/i2c connecting various peripherals that may also be connected in some other way (e.g. media devices).

It's fairly overkill though ... we don't really plan to expose much of these things to Linux anyway, it's mostly buried in firmware, I just want scom access to userspace for debug/diagnostics.

I don't want it in debugfs however because we might want some "health monitoring" daemon running in userspace that uses it to check some of the built-in CE statistics etc... in the various chips and handle some of the repair work. I prefer having that stuff in userspace (it's fairly complex) than in firmware but it will get all the data it needs from the device-tree, there's no point trying to reproduce the whole chip hierarchy and sub hierarchy in Linux.

Cheers,
Ben.