Skip to contentSkip to conversation

OF_DYNAMIC usage

DISCUSSION Michal Simek 4 participants 12 messages Jul 5, 2012 – Jul 6, 2012 5 messages not in the archive

lore
Share & tools
lore
#0 lore raw

Hi,

can someone tell me usage of OF_DYNAMIC? How this can be used by user?

Is it possible to exchange device node? Add new one, delete one, etc?

Any user guide/log will be helpful to see what I can do with it.

Thanks,
Michal

Signature · 4 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian
#2 lore raw
In reply to a message not in the archive
Quoted 9 lines · Michal Simek

Hi,

can someone tell me usage of OF_DYNAMIC?
How this can be used by user?

Is it possible to exchange device node?
Add new one, delete one, etc?

Any user guide/log will be helpful to see what I can do with it.

2nd question on OF_DYNAMIC in a week... I haven't really looked at it. I would suggest looking at the existing users (only powerpc iseries and pseries). My concern using would be that the reference counting is not correct for many drivers as of_node_get/put is a nop without OF_DYNAMIC.

You can also activate/deactivate nodes with the status property.

Rob

#4 lore raw
In reply to a message not in the archive
Quoted 9 lines · Rob Herring
Quoted 8 lines · Michal Simek

Hi,

can someone tell me usage of OF_DYNAMIC?
How this can be used by user?

Is it possible to exchange device node?
Add new one, delete one, etc?

Quoted 9 lines
Quoted 1 line

Any user guide/log will be helpful to see what I can do with it.

2nd question on OF_DYNAMIC in a week... I haven't really looked at it. I would suggest looking at the existing users (only powerpc iseries and pseries). My concern using would be that the reference counting is not correct for many drivers as of_node_get/put is a nop without OF_DYNAMIC.

You can also activate/deactivate nodes with the status property.

No doc that I know of. It's used by pseries only (iseries is gone btw) for "DLPAR" (aka Dynamic Reconfiguration aka hotplug).

The way it works at the moment is that when something new is plugged in, the hypervisor talks to a proprietary crap daemon in userspace which talks to a special tool (that one we have the source code) which then obtains via some FW interfaces a "blob" of bits of device-tree to add (or to remove), using a phyp specific format, and echo that stuff into /proc/ppc64/ofdt.

It's pretty fugly...

Because nodes can come and go, we thus add refcounting. There's various places where the refcounting is not quite right but mostly things that don't get plugged/unplugged, actual vio devices or PCI devices tend to get it right.

Also we don't have good rules for properties lifetime, as it is, it's assumed to be the same as the node and there's no proper way to add/remove/modify a property "live". I would recommend at least something RCUish if you're going to try to be safe to delay the freeing of the old value in the dynamic case.

There's definitely room for improvements in that area...

Cheers,
Ben.

#16 lore raw
In reply to a message not in the archive
Quoted 9 lines · Rob Herring
Quoted 8 lines · Michal Simek

Hi,

can someone tell me usage of OF_DYNAMIC?
How this can be used by user?

Is it possible to exchange device node?
Add new one, delete one, etc?

Quoted 6 lines
Quoted 1 line

Any user guide/log will be helpful to see what I can do with it.

2nd question on OF_DYNAMIC in a week... I haven't really looked at it. I would suggest looking at the existing users (only powerpc iseries and pseries). My concern using would be that the reference counting is not correct for many drivers as of_node_get/put is a nop without OF_DYNAMIC.

Agree, I know that reference counting is not correct for many drivers.

Thanks,
Michal

Signature · 4 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian
#5 lore raw
In reply to Benjamin Herrenschmidt
Quoted 17 lines · Benjamin Herrenschmidt
Quoted 16 lines · Rob Herring
Quoted 9 lines · Michal Simek

Hi,

can someone tell me usage of OF_DYNAMIC?
How this can be used by user?

Is it possible to exchange device node?
Add new one, delete one, etc?

Any user guide/log will be helpful to see what I can do with it.

2nd question on OF_DYNAMIC in a week... I haven't really looked at it. I would suggest looking at the existing users (only powerpc iseries and pseries). My concern using would be that the reference counting is not correct for many drivers as of_node_get/put is a nop without OF_DYNAMIC.

Quoted 5 lines
Quoted 2 lines

You can also activate/deactivate nodes with the status property.

No doc that I know of. It's used by pseries only (iseries is gone btw) for "DLPAR" (aka Dynamic Reconfiguration aka hotplug).

ok. This is for Xilinx partial reconfiguration.

Quoted 7 lines

The way it works at the moment is that when something new is plugged in, the hypervisor talks to a proprietary crap daemon in userspace which talks to a special tool (that one we have the source code) which then obtains via some FW interfaces a "blob" of bits of device-tree to add (or to remove), using a phyp specific format, and echo that stuff into /proc/ppc64/ofdt.

Where is that source code for the special tool?

Can you point me to the "phyp specific format"?

From reconfig.c it looks like that there are some key words like add_node/remove_node/add_property... follow by space and node name + options which lookes like dtb format.

Quoted 2 lines

It's pretty fugly...

Yes, I see.

Quoted 5 lines

Because nodes can come and go, we thus add refcounting. There's various places where the refcounting is not quite right but mostly things that don't get plugged/unplugged, actual vio devices or PCI devices tend to get it right.

Some driver validation has to be made to get it work. I have done some correction for xilinx drivers in past.

Quoted 8 lines

Also we don't have good rules for properties lifetime, as it is, it's assumed to be the same as the node and there's no proper way to add/remove/modify a property "live". I would recommend at least something RCUish if you're going to try to be safe to delay the freeing of the old value in the dynamic case.

There's definitely room for improvements in that area...

:-)

Thanks,
Michal

Signature · 4 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian
#7 lore raw
In reply to a message not in the archive
Quoted 9 lines · Michal Simek
Quoted 7 lines

The way it works at the moment is that when something new is plugged in, the hypervisor talks to a proprietary crap daemon in userspace which talks to a special tool (that one we have the source code) which then obtains via some FW interfaces a "blob" of bits of device-tree to add (or to remove), using a phyp specific format, and echo that stuff into /proc/ppc64/ofdt.

Where is that source code for the special tool?

I can dig that for you, however ...

Quoted 1 line

Can you point me to the "phyp specific format"?

Same deal, I don't think there's a public doc, however..

Quoted 3 lines

From reconfig.c it looks like that there are some key words like
add_node/remove_node/add_property... follow by space and node name + options which lookes like dtb format.

Right, I would just recommend you don't do that.

The need to "hotplug" bits of device-tree is going to be generic enough that we should come up with something better and more generic than that pseries stuff.

IE. Some way to pass bits of ftb blob rather than this specific format to begin with, etc...

So I'd say just ignore the pseries stuff, I can dig the tool etc... if you -really- need them but I'd rather you don't base your stuff on it, just make up something better & more generic for you. It will be useful to others.

Cheers,
Ben.

#8 lore raw
Quoted 9 lines · Benjamin Herrenschmidt
Quoted 8 lines · Michal Simek
Quoted 7 lines

The way it works at the moment is that when something new is plugged in, the hypervisor talks to a proprietary crap daemon in userspace which talks to a special tool (that one we have the source code) which then obtains via some FW interfaces a "blob" of bits of device-tree to add (or to remove), using a phyp specific format, and echo that stuff into /proc/ppc64/ofdt.

Quoted 7 lines
Quoted 1 line

Where is that source code for the special tool?

I can dig that for you, however ...

Quoted 1 line

Can you point me to the "phyp specific format"?

Same deal, I don't think there's a public doc, however..

Why is it in the kernel something which does not have any public documentation? It seems like that it is more proprietary code.

Quoted 5 lines
Quoted 4 lines

From reconfig.c it looks like that there are some key words like
add_node/remove_node/add_property... follow by space and node name + options which lookes like dtb format.

Quoted 8 lines

Right, I would just recommend you don't do that.

The need to "hotplug" bits of device-tree is going to be generic enough that we should come up with something better and more generic than that pseries stuff.

IE. Some way to pass bits of ftb blob rather than this specific format to begin with, etc...

Can we discuss this a little bit more?

From my partial reconfiguration understanding is that you have partial bitstream which you pass through pcap(IP and driver which can do it) to specific location which they are known.

It means you have to unplug device which is in the same location, load partial bitstream via pcap
register driver and probe it.

I expect that pcap driver could be aware which driver is at that location and is able to plug and unplug it based on description.

I will ask Xilinx how this is exactly done and I hope I will get any example to be able to do some experiments.

Quoted 5 lines

So I'd say just ignore the pseries stuff, I can dig the tool etc... if you -really- need them but I'd rather you don't base your stuff on it, just make up something better& more generic for you. It will be useful to others.

My point here is just to try to understand current working version to get the better overview how it is done and probably working somehow.

Of course some ideas how this can/should be done will be helpful.

Thanks,
Michal

Signature · 4 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian
#10 lore raw
In reply to a message not in the archive
Quoted 12 lines · Michal Simek
Quoted 11 lines · Benjamin Herrenschmidt
Quoted 9 lines · Michal Simek
Quoted 7 lines

The way it works at the moment is that when something new is plugged in, the hypervisor talks to a proprietary crap daemon in userspace which talks to a special tool (that one we have the source code) which then obtains via some FW interfaces a "blob" of bits of device-tree to add (or to remove), using a phyp specific format, and echo that stuff into /proc/ppc64/ofdt.

Where is that source code for the special tool?

Quoted 8 lines
Quoted 5 lines

I can dig that for you, however ...

Quoted 1 line

Can you point me to the "phyp specific format"?

Same deal, I don't think there's a public doc, however..

Why is it in the kernel something which does not have any public documentation? It seems like that it is more proprietary code.

Don't ask me, it's been there since day 1 of the ppc64 port afaik, before I was involved in it. I've been wanting to deprecate that interface for a while, but haven't got to it yet.

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

From reconfig.c it looks like that there are some key words like
add_node/remove_node/add_property... follow by space and node name + options which lookes like dtb format.

Right, I would just recommend you don't do that.

Quoted 8 lines
Quoted 6 lines

The need to "hotplug" bits of device-tree is going to be generic enough that we should come up with something better and more generic than that pseries stuff.

IE. Some way to pass bits of ftb blob rather than this specific format to begin with, etc...

Can we discuss this a little bit more?

Sure :-)

Quoted 20 lines

From my partial reconfiguration understanding is that you have partial bitstream
which you pass through pcap(IP and driver which can do it) to specific location which they are known.

It means you have to unplug device which is in the same location, load partial bitstream via pcap
register driver and probe it.

I expect that pcap driver could be aware which driver is at that location and is able to plug and unplug it based on description.

I will ask Xilinx how this is exactly done and I hope I will get any example to be able to do some experiments.

Quoted 5 lines

So I'd say just ignore the pseries stuff, I can dig the tool etc... if you -really- need them but I'd rather you don't base your stuff on it, just make up something better& more generic for you. It will be useful to others.

Quoted 4 lines

My point here is just to try to understand current working version to get the better overview how it is done and probably working somehow.

Of course some ideas how this can/should be done will be helpful.

Well, I think the right way is to have some mechanism (TBD) to be able to inject into the kernel a bit of device-tree (or remove a bit of device-tree). Ideally by passing it an FDT blob segment which is a well known & documented format.

The kernel would then expand it and call some notifiers which we can use to create platform devices if needed etc...

Cheers,
Ben.

#11 lore raw
Quoted 17 lines · Benjamin Herrenschmidt
Quoted 16 lines · Michal Simek
Quoted 15 lines · Benjamin Herrenschmidt
Quoted 9 lines · Michal Simek
Quoted 7 lines

The way it works at the moment is that when something new is plugged in, the hypervisor talks to a proprietary crap daemon in userspace which talks to a special tool (that one we have the source code) which then obtains via some FW interfaces a "blob" of bits of device-tree to add (or to remove), using a phyp specific format, and echo that stuff into /proc/ppc64/ofdt.

Where is that source code for the special tool?

I can dig that for you, however ...

Quoted 1 line

Can you point me to the "phyp specific format"?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Same deal, I don't think there's a public doc, however..

Why is it in the kernel something which does not have any public documentation? It seems like that it is more proprietary code.

Don't ask me, it's been there since day 1 of the ppc64 port afaik, before I was involved in it. I've been wanting to deprecate that interface for a while, but haven't got to it yet.

:-)

Quoted 40 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines

From reconfig.c it looks like that there are some key words like
add_node/remove_node/add_property... follow by space and node name + options which lookes like dtb format.

Right, I would just recommend you don't do that.

Quoted 8 lines
Quoted 6 lines

The need to "hotplug" bits of device-tree is going to be generic enough that we should come up with something better and more generic than that pseries stuff.

IE. Some way to pass bits of ftb blob rather than this specific format to begin with, etc...

Can we discuss this a little bit more?

Sure :-)

Quoted 23 lines

From my partial reconfiguration understanding is that you have partial bitstream
which you pass through pcap(IP and driver which can do it) to specific location which they are known.

It means you have to unplug device which is in the same location, load partial bitstream via pcap
register driver and probe it.

I expect that pcap driver could be aware which driver is at that location and is able to plug and unplug it based on description.

I will ask Xilinx how this is exactly done and I hope I will get any example to be able to do some experiments.

Quoted 5 lines

So I'd say just ignore the pseries stuff, I can dig the tool etc... if you -really- need them but I'd rather you don't base your stuff on it,
just make up something better& more generic for you. It will be useful
to others.

My point here is just to try to understand current working version to get the better overview how it is done and probably working somehow.

Quoted 6 lines
Quoted 1 line

Of course some ideas how this can/should be done will be helpful.

Well, I think the right way is to have some mechanism (TBD) to be able to inject into the kernel a bit of device-tree (or remove a bit of device-tree). Ideally by passing it an FDT blob segment which is a well known& documented format.

ok. How that FDT blob segment should look like?
It can't be just one node because it also requires path where it is connected.

It means at least the part like below for injecting.

/dts-v1/;

/ {
	#address-cells = <1>;
	#size-cells = <1>;
	compatible = "xlnx,microblaze";

	mb_plb: plb@0 {
		#address-cells = <1>;
		#size-cells = <1>;
		compatible = "xlnx,plb-v46-1.00.a", "simple-bus";
		DIP_Switches_4Bit: gpio@81440000 {
			#gpio-cells = <2>;
			compatible = "xlnx,xps-gpio-1.00.a";
			gpio-controller ;
			reg = < 0x81440000 0x10000 >;
			/* ... */
		} ;
	} ;
};

Not sure if this can be used for removing. I mean if you want to remove node if make sense to pass the whole node.

Thanks,
Michal

Signature · 4 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian
#13 lore raw
In reply to a message not in the archive
Quoted 4 lines · Michal Simek

ok. How that FDT blob segment should look like?
It can't be just one node because it also requires path where it is connected.

It means at least the part like below for injecting.

No, my idea was to pass 3 arguments:

- action (enum)
- path (string)
- segment (blob)

action would be typically add/remove

* add: add node under <path> whose content is in <segment>

* remove: remove node <path> and all children

(or we could require children removal to be done explicitly).

Cheers,
Ben.

Quoted 28 lines

/dts-v1/;

/ {
	#address-cells = <1>;
	#size-cells = <1>;
	compatible = "xlnx,microblaze";

	mb_plb: plb@0 {
		#address-cells = <1>;
		#size-cells = <1>;
		compatible = "xlnx,plb-v46-1.00.a", "simple-bus";
		DIP_Switches_4Bit: gpio@81440000 {
			#gpio-cells = <2>;
			compatible = "xlnx,xps-gpio-1.00.a";
			gpio-controller ;
			reg = < 0x81440000 0x10000 >;
			/* ... */
		} ;
	} ;
};

Not sure if this can be used for removing. I mean if you want to remove node if make sense to pass the whole node.

Thanks,
Michal

#14 lore raw
Quoted 13 lines · Benjamin Herrenschmidt
Quoted 4 lines · Michal Simek

ok. How that FDT blob segment should look like?
It can't be just one node because it also requires path where it is connected.

It means at least the part like below for injecting.

No, my idea was to pass 3 arguments:

- action (enum)
- path (string)
- segment (blob)

Quoted 7 lines

action would be typically add/remove

* add: add node under<path> whose content is in<segment>

* remove: remove node<path> and all children

(or we could require children removal to be done explicitly).

ok.

Thanks,
Michal

Signature · 4 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian
#15 lore raw
In reply to Benjamin Herrenschmidt
Quoted 23 lines

-----Original Message-----
From: devicetree-discuss [mailto:devicetree-discuss- bounces+stephen.neuendorffer=xilinx.com-uLR06cmDAlY/[email protected]] On Behalf Of Benjamin Herrenschmidt
Sent: Friday, July 06, 2012 1:10 AM
To: [email protected]
Cc: devicetree-discuss-uLR06cmDAlY/[email protected]
Subject: Re: OF_DYNAMIC usage

Quoted 2 lines · Michal Simek

ok. How that FDT blob segment should look like?
It can't be just one node because it also requires path where it is

connected.

Quoted 2 lines

It means at least the part like below for injecting.

No, my idea was to pass 3 arguments:

- action (enum)
- path (string)
- segment (blob)

Quoted 7 lines

action would be typically add/remove

* add: add node under <path> whose content is in <segment>

* remove: remove node <path> and all children

(or we could require children removal to be done explicitly).

I've implemented something like Michal's suggestion, although I never worked out the details of removing. Primarily, I was interested in the slightly simpler case of a PCIe endpoint, whose contents are described by a device tree. This means that the device tree fragment has to be able to be independent of its context. Basically, I added a function:

+/**
+ * unflatten_partial_device_tree - create tree of device_nodes from flat blob
+ *
+ * unflattens the device-tree passed by the firmware, creating the
+ * tree of struct device_node. It also fills the "name" and "type"
+ * pointers of the nodes so the normal device-tree walking functions
+ * can be used.
+ */
+void unflatten_partial_device_tree(unsigned long *blob, struct device_node **mynodes)

That can be called by a device driver, like:

+ unflatten_partial_device_tree(drvdata->fw_entry->data,
+ &drvdata->child_nodes);
+
+ /* Generate child devices from the device tree */
+ of_platform_bus_probe(drvdata->child_nodes, NULL, &pdev->dev);

Part of the reason for implementing it this way is that it allows the result to work on x86, where
there may not be a toplevel device tree, so adding and deleting device nodes is handled at the device level, rather than having the device trees themselves be dynamic. The downside, of course it that it is harder to get references from one device tree fragment into another. However, in the case of either partial reconfiguration or PCIe, I think this makes more sense than the OF_DYNAMIC way.

We demonstrated this working in a PCIe system and posted the patches several years ago, but they were never mainlined, and the architecture-independent device tree pieces have gone through a lot of change since then. Maybe Michal can get them to compile and post updated versions for more comment?

Steve

Quoted 35 lines

Cheers,
Ben.

Quoted 20 lines

/dts-v1/;

/ {
	#address-cells = <1>;
	#size-cells = <1>;
	compatible = "xlnx,microblaze";

	mb_plb: plb@0 {
		#address-cells = <1>;
		#size-cells = <1>;
		compatible = "xlnx,plb-v46-1.00.a", "simple-bus";
		DIP_Switches_4Bit: gpio@81440000 {
			#gpio-cells = <2>;
			compatible = "xlnx,xps-gpio-1.00.a";
			gpio-controller ;
			reg = < 0x81440000 0x10000 >;
			/* ... */
		} ;
	} ;
};
Quoted 1 line

Not sure if this can be used for removing. I mean if you want to

remove node

Quoted 7 lines

if make sense to pass the whole node.

Thanks,
Michal

Quoted 4 lines

_______________________________________________
devicetree-discuss mailing list devicetree-discuss-uLR06cmDAlY/[email protected]
https://lists.ozlabs.org/listinfo/devicetree-discuss

This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.