Enable BCM2835 mailbox support
Subsystems (6)
RFC PATCH 1/3 Enable BCM2835 mailbox support
Cherry-picked selection of commits by Lubomir Rintel [1], Suman Anna and Jassi Brar [2] on which to base the implementation.
[1] http://lists.infradead.org/pipermail/linux-rpi-kernel/2013-April/000528.html
[2] http://lists.infradead.org/pipermail/linux-rpi-kernel/2013-May/000546.html
---
Documentation/devicetree/bindings/mailbox/brcm,bcm2835-mbox.txt new file +15 -0
@@ -0,0 +1,15 @@ +Broadcom BCM2835 VideoCore mailbox IPC + +Required properties: + +- compatible : should be "brcm,bcm2835-mbox" +- reg : Specifies base physical address and size of the registers. +- interrupts : the interrupt number + +Example: + +mailbox { + compatible = "brcm,bcm2835-mbox"; + reg = <0x7e00b800 0x400>; + interrupts = <0 1>; +};
arch/arm/boot/dts/bcm2835.dtsi +5 -0
@@ -29,6 +29,12 @@ #interrupt-cells = <2>; }; + mailbox { + compatible = "brcm,bcm2835-mbox"; + reg = <0x7e00b880 0x40>; + interrupts = <0 1>; + }; + watchdog { compatible = "brcm,bcm2835-pm-wdt"; reg = <0x7e100000 0x28>;
arch/arm/mach-bcm2835/Makefile +1 -1
@@ -1 +1 @@ -obj-y += bcm2835.o +obj-y += bcm2835.o mailbox.o
arch/arm/mach-bcm2835/mailbox.c new file +278 -0
278 lines not on this page · raw message
drivers/mailbox/Makefile +4 -0
@@ -1,3 +1,7 @@ +# Generic MAILBOX API + +obj-$(CONFIG_MAILBOX) += mailbox.o + obj-$(CONFIG_PL320_MBOX) += pl320-ipc.o obj-$(CONFIG_OMAP_MBOX) += omap-mailbox.o
drivers/mailbox/mailbox.c new file +474 -0
474 lines not on this page · raw message
drivers/mailbox/mailbox_internal.h new file +77 -0
77 lines not on this page · raw message
include/linux/mailbox.h +14 -13
@@ -1,17 +1,17 @@/* -* This program is free software; you can redistribute it and/or modify it -* under the terms and conditions of the GNU General Public License, -* version 2, as published by the Free Software Foundation. -* -* This program is distributed in the hope it will be useful, but WITHOUT -* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or -* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for -* more details. -* -* You should have received a copy of the GNU General Public License along with -* this program. If not, see <http://www.gnu.org/licenses/>. +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License version 2 as +* published by the Free Software Foundation. */ -int pl320_ipc_transmit(u32 *data); -int pl320_ipc_register_notifier (struct notifier_block *nb); -int pl320_ipc_unregister_notifier (struct notifier_block *nb); +#ifndef __MAILBOX_H +#define __MAILBOX_H + +enum xfer_result { +XFER_OK =0 , +XFER_ERR , +}; + +typedef unsigned request_token_t ; + +#endif /* __MAILBOX_H */
include/linux/mailbox_client.h new file +87 -0
87 lines not on this page · raw message
include/linux/mailbox_controller.h new file +102 -0
102 lines not on this page · raw message
RFC PATCH 1/3 Enable BCM2835 mailbox support
Quoted 9 lines · Craig McGeachie
Cherry-picked selection of commits by Lubomir Rintel [1], Suman Anna and Jassi Brar [2] on which to base the implementation.
Signed-off-by Suman Ana <[email protected]>Signed-off-by Jassi Brar <[email protected]>Signed-off-by Lubomir Rintel <[email protected]>[1] http://lists.infradead.org/pipermail/linux-rpi-kernel/2013-April/000528.html
[2] http://lists.infradead.org/pipermail/linux-rpi-kernel/2013-May/000546.html
There are two problems with the mailbox framework that I can find. The first is in ipc_send_message. On the BCM2835, the mailbox send result doesn't trigger an interrupt, but the pending reply does. When ipc_request_channel is called without blocking, the flow of events looks like this:
[ 0.521312] req - cf9845c0: 00000020 00000000 00030006 00000008 [ 0.527245] req - cf9845d0: 00000008 00000000 00000000 00000000 [ 0.533248] bcm2835-mbox 2000b880.mailbox: Request 0x0F9845C8 [ 0.539007] bcm2835-mbox 2000b880.mailbox: Reply 0x0F9845C8 [ 0.544582] bcm2835_thermal thermal.1: rxcb #1 @0f9845c0 [ 0.549939] bcm2835-mbox 2000b880.mailbox: Polling [ 0.554773] bcm2835_thermal thermal.1: txcb #1 @0f9845c0 [ 0.560102] rep - cf9845c0: 00000020 80000000 00030006 00000008 [ 0.566062] rep - cf9845d0: 80000008 00000000 000078da 00000000
I've never seen the transmit callback come first.
When ipc_request_channel is called with blocking, then the best result I've had a null pointer causing a kernel panic. Most of the time, the system just deadlocks. The fix is:
drivers/mailbox/mailbox.c +1 -1
@@ -202,6 +202,7 @@ request_token_t ipc_send_message(void *channel, void *mssg)if (!t)pr_err ("Try increasing MBOX_TX_QUEUE_LEN\n" ); +init_completion (&chan->tx_complete);_msg_submit (chan);if (chan->txdone_method ==TXDONE_BY_POLL )
@@ -209,7 +210,6 @@ request_token_t ipc_send_message(void *channel, void *mssg)if (chan->tx_block && chan->active_req) {int ret; -init_completion (&chan->tx_complete); ret =wait_for_completion_timeout (&chan->tx_complete, chan->tx_tout);if (ret ==0 ) {
When the init_completion is done after the _msg_submit, it might be executed after the complete in tx_tick. And for the BCM2835, it's pretty reliable that this will happen. When this happens, the calls are:
1. complete
2. init_completion
3. wait_for_completion_timeout
The complete call is missed.
With this fix in place, the flow of events now looks like:
[ 0.521214] req - cf9845c0: 00000020 00000000 00030006 00000008 [ 0.527149] req - cf9845d0: 00000008 00000000 00000000 00000000 [ 0.533149] bcm2835-mbox 2000b880.mailbox: Request 0x0F9845C8 [ 0.538908] bcm2835-mbox 2000b880.mailbox: Reply 0x0F9845C8 [ 0.544482] bcm2835_thermal thermal.1: rxcb #1 @0f9845c0 [ 0.549840] bcm2835-mbox 2000b880.mailbox: Polling [ 0.554686] rep - cf9845c0: 00000020 80000000 00030006 00000008 [ 0.560648] rep - cf9845d0: 80000008 00000000 0000835c 00000000
Just a comment about this common mailbox API with respect to the BCM2835. It makes more sense for the BCM285 to always send messages with a blocking transmit. The mailbox send is a single 32 bit IO register shared for all channels (the channel number is encoded into the send data). BCM2835 drivers would only want to use the mailbox API with an effective transmit queue depth of 1. This is fine. From what I can see, IPC calls via the mailbox should be very infrequent. Mostly only to set up some other inter-processor communication mechanism, or to query remote processor properties at system start or on behalf of a user process.
The second problem is in ipc_request_channel
list_for_each_entry(chan, &con->channels, node) {
if (!strcmp(con_name + len + 1, chan->name)
&& !chan->assigned) {
spin_lock_irqsave(&chan->lock, flags);
[...snip...]
chan->assigned = true;
[...snip...]
spin_unlock_irqrestore(&chan->lock, flags);
ret = 1;
break;
}
}I think this is a race condition. I can see no reason why two or more threads couldn't proceed through the if condition before the spin lock is acquired. If that happens, then after the first thread has claimed the channel by setting assigned true and released the spin lock, the second thread will proceed, and overwrite the channel state with its own values. Then wackiness ensues.
The fix would be to move the lock acquisition to before the if, but getting the unlock correct will become messy.
Cheers,
Craig.
RFC PATCH 1/3 Enable BCM2835 mailbox support
Quoted 63 lines · Craig McGeachie
Quoted 9 lines · Craig McGeachie
Cherry-picked selection of commits by Lubomir Rintel [1], Suman Anna and Jassi Brar [2] on which to base the implementation.
Signed-off-by Suman Ana <[email protected]>Signed-off-by Jassi Brar <[email protected]>Signed-off-by Lubomir Rintel <[email protected]>[1] http://lists.infradead.org/pipermail/linux-rpi-kernel/2013-April/000528.html
[2] http://lists.infradead.org/pipermail/linux-rpi-kernel/2013-May/000546.htmlThere are two problems with the mailbox framework that I can find. The first is in
ipc_send_message. On the BCM2835, the mailbox send result doesn't trigger an interrupt, but the pending reply does. Whenipc_request_channelis called without blocking, the flow of events looks like this:[ 0.521312] req - cf9845c0: 00000020 00000000 00030006 00000008 [ 0.527245] req - cf9845d0: 00000008 00000000 00000000 00000000 [ 0.533248] bcm2835-mbox 2000b880.mailbox: Request 0x0F9845C8 [ 0.539007] bcm2835-mbox 2000b880.mailbox: Reply 0x0F9845C8 [ 0.544582] bcm2835_thermal thermal.1: rxcb #1 @0f9845c0 [ 0.549939] bcm2835-mbox 2000b880.mailbox: Polling [ 0.554773] bcm2835_thermal thermal.1: txcb #1 @0f9845c0 [ 0.560102] rep - cf9845c0: 00000020 80000000 00030006 00000008 [ 0.566062] rep - cf9845d0: 80000008 00000000 000078da 00000000I've never seen the transmit callback come first.
When
ipc_request_channelis called with blocking, then the best result I've had a null pointer causing a kernel panic. Most of the time, the system just deadlocks. The fix is:drivers/mailbox/mailbox.c +1
-1@@ -202,6 +202,7 @@ request_token_t ipc_send_message(void *channel, void *mssg)if (!t)pr_err ("Try increasing MBOX_TX_QUEUE_LEN\n" ); +init_completion (&chan->tx_complete);_msg_submit (chan);if (chan->txdone_method ==TXDONE_BY_POLL )@@ -209,7 +210,6 @@ request_token_t ipc_send_message(void *channel, void *mssg)if (chan->tx_block && chan->active_req) {int ret; -init_completion (&chan->tx_complete); ret =wait_for_completion_timeout (&chan->tx_complete, chan->tx_tout);if (ret ==0 ) {When the
init_completionis done after the_msg_submit, it might be executed after the complete intx_tick. And for the BCM2835, it's pretty reliable that this will happen. When this happens, the calls are:
1. complete
2.init_completion
3.wait_for_completion_timeoutThe complete call is missed.
With this fix in place, the flow of events now looks like:
Quoted 9 lines
[ 0.521214] req - cf9845c0: 00000020 00000000 00030006 00000008 [ 0.527149] req - cf9845d0: 00000008 00000000 00000000 00000000 [ 0.533149] bcm2835-mbox 2000b880.mailbox: Request 0x0F9845C8 [ 0.538908] bcm2835-mbox 2000b880.mailbox: Reply 0x0F9845C8 [ 0.544482] bcm2835_thermal thermal.1: rxcb #1 @0f9845c0 [ 0.549840] bcm2835-mbox 2000b880.mailbox: Polling [ 0.554686] rep - cf9845c0: 00000020 80000000 00030006 00000008 [ 0.560648] rep - cf9845d0: 80000008 00000000 0000835c 00000000
LF.Tan already provided the same patch (and another) a few weeks ago. I have it locally but it should be pushed soon.
commit 28785f7c4af687026e601e0a7328414de43c84b3
Author: LeyFoon Tan <[email protected]> Date: Tue Aug 13 10:40:21 2013 +0530
mailbox: Fix deleteing poll timer
Try to delete the timer only if it was init/used.
drivers/mailbox/mailbox.c +2 -1
@@ -469,7 +469,8 @@ void ipc_links_unregister(struct ipc_controller *ipc)list_for_each_entry (chan, &con->channels, node)ipc_free_channel ((void *)chan); -del_timer_sync (&con->poll); +if (ipc->txdone_poll) +del_timer_sync (&con->poll);kfree (con); }
commit 8456ca4665da4a5bac3163eb1ddd98bcccd690c2
Author: LeyFoon Tan <[email protected]> Date: Mon Aug 12 22:02:10 2013 +0530
mailbox: Fix TX completion init
For fast TX the complete could be called before being initialized as follows
ipc_send_message --> poll_txdone --> tx_tick -->complete(&chan->tx_complete)
Init the completion early enough to fix the race.
drivers/mailbox/mailbox.c +3 -1
@@ -198,6 +198,9 @@ request_token_t ipc_send_message(void *channel, void *mssg)if (!chan || !chan->assigned)return 0 ; +if (chan->tx_block) +init_completion (&chan->tx_complete); + t =_add_to_rbuf (chan, mssg);if (!t)pr_err ("Try increasing MBOX_TX_QUEUE_LEN\n" );
@@ -209,7 +212,6 @@ request_token_t ipc_send_message(void *channel, void *mssg)if (chan->tx_block && chan->active_req) {int ret; -init_completion (&chan->tx_complete); ret =wait_for_completion_timeout (&chan->tx_complete, chan->tx_tout);if (ret ==0 ) {
Quoted 33 lines
Just a comment about this common mailbox API with respect to the BCM2835. It makes more sense for the BCM285 to always send messages with a blocking transmit. The mailbox send is a single 32 bit IO register shared for all channels (the channel number is encoded into the send data). BCM2835 drivers would only want to use the mailbox API with an effective transmit queue depth of 1. This is fine. From what I can see, IPC calls via the mailbox should be very infrequent. Mostly only to set up some other inter-processor communication mechanism, or to query remote processor properties at system start or on behalf of a user process.
The second problem is in
ipc_request_channellist_for_each_entry(chan, &con->channels, node) { if (!strcmp(con_name + len + 1, chan->name) && !chan->assigned) { spin_lock_irqsave(&chan->lock, flags); [...snip...] chan->assigned = true; [...snip...] spin_unlock_irqrestore(&chan->lock, flags); ret = 1; break; } }I think this is a race condition. I can see no reason why two or more threads couldn't proceed through the if condition before the spin lock is acquired. If that happens, then after the first thread has claimed the channel by setting assigned true and released the spin lock, the second thread will proceed, and overwrite the channel state with its own values. Then wackiness ensues.
Quoted 3 lines
The fix would be to move the lock acquisition to before the if, but getting the unlock correct will become messy.
Thanks for pointing out... the following fix should work.
drivers/mailbox/mailbox.c +7 -5
@@ -261,9 +261,10 @@ void *ipc_request_channel(struct ipc_client *cl) ret =0 ;list_for_each_entry (chan, &con->channels, node) { -if (!strcmp (con_name + len +1 , chan->name) - && !chan->assigned) { -spin_lock_irqsave (&chan->lock, flags); +if (strcmp (con_name + len +1 , chan->name)) +continue ; +spin_lock_irqsave (&chan->lock, flags); +if (!chan->assigned) { chan->msg_free =0 ; chan->msg_count =0 ; chan->active_req =NULL ;
@@ -279,10 +280,11 @@ void *ipc_request_channel(struct ipc_client *cl)if (chan->txdone_method ==TXDONE_BY_POLL && cl->knows_txdone) chan->txdone_method |=TXDONE_BY_ACK ; -spin_unlock_irqrestore (&chan->lock, flags); ret =1 ; -break ; } +spin_unlock_irqrestore (&chan->lock, flags); +if (ret) +break ; }if (!ret) {
RFC PATCH 1/3 Enable BCM2835 mailbox support
In reply to Craig McGeachieQuoted 9 lines · Craig McGeachie
Cherry-picked selection of commits by Lubomir Rintel [1], Suman Anna and Jassi Brar [2] on which to base the implementation.
Signed-off-by Suman Ana <[email protected]>Signed-off-by Jassi Brar <[email protected]>Signed-off-by Lubomir Rintel <[email protected]>[1] http://lists.infradead.org/pipermail/linux-rpi-kernel/2013-April/000528.html
[2] http://lists.infradead.org/pipermail/linux-rpi-kernel/2013-May/000546.html
This needs a proper patch description. It should describe what the mailbox HW is, what services it provides, etc.
There shouldn't be any text after the tag paragraph (i.e. the s-o-b lines) and the --- separator.
Are you able to generate patches using "git format-patch". That would included diffstat in the patch summary, which can be useful.
I'd prefer to see patch 1 and 2 squashed together, since the new mailbox driver should just be implemented in one shot, rather than added with one implementation, then reworked to be different in a separate patch.
That said, the patches need to be split up based on subsystem divisions. The squashed patch 1 and 2, and the separate patch 3, should be split into:
a) A patch that adds the driver. This would touch just drivers/mailbox/... (or drivers/thermal/...) and Documentation/devicetree/bindings/...
b) A patch that adds the necessary entries to *.dts/*.dtsi
c) A patch that enables any new entries in bcm2835_defconfig.
In my initial round of review, I'll only discuss the DT bindings, and only in response to patch 2 where they're complete rather than here. Once the bindings are finalized and the driver adjusted to implement the final bindings, then I'll look at the driver code.
Note that all new DT bindings should be CC'd to the DT bindings maintainers; see the following in MAINTAINERS:
OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS
M: Rob Herring <[email protected]> M: Pawel Moll <[email protected]> M: Mark Rutland <[email protected]> M: Stephen Warren <[email protected]> M: Ian Campbell <[email protected]> L: devicetree at vger.kernel.org S: Maintained F: Documentation/devicetree/ F: arch/*/boot/dts/ F: include/dt-bindings/
arch/arm/boot/dts/bcm2835.dtsi line 30
Quoted 7 lines
arch/arm/boot/dts/bcm2835.dtsi +0
-0@@ -29,6 +29,12 @@ #interrupt-cells = <2>; };
There needs to be a blank line here.
arch/arm/boot/dts/bcm2835.dtsi line 31
Quoted 1 line
arch/arm/boot/dts/bcm2835.dtsi +0
-0+ mailbox {
Patch corruption? + is indented!
RFC PATCH 1/3 Enable BCM2835 mailbox support
Quoted 11 lines · Stephen Warren
Quoted 9 lines · Craig McGeachie
Cherry-picked selection of commits by Lubomir Rintel [1], Suman Anna and Jassi Brar [2] on which to base the implementation.
Signed-off-by Suman Ana <[email protected]>Signed-off-by Jassi Brar <[email protected]>Signed-off-by Lubomir Rintel <[email protected]>[1] http://lists.infradead.org/pipermail/linux-rpi-kernel/2013-April/000528.html
[2] http://lists.infradead.org/pipermail/linux-rpi-kernel/2013-May/000546.html
Quoted 8 lines
This needs a proper patch description. It should describe what the mailbox HW is, what services it provides, etc.
There shouldn't be any text after the tag paragraph (i.e. the s-o-b lines) and the --- separator.
Are you able to generate patches using "git format-patch". That would included diffstat in the patch summary, which can be useful.
I can do all that. I'm rather new to all of this, and didn't get git format-patch and git send-email sorted out until after this. You really don't want to know what I did to get this sent.
Quoted 11 lines
I'd prefer to see patch 1 and 2 squashed together, since the new mailbox driver should just be implemented in one shot, rather than added with one implementation, then reworked to be different in a separate patch.
That said, the patches need to be split up based on subsystem divisions. The squashed patch 1 and 2, and the separate patch 3, should be split into:
a) A patch that adds the driver. This would touch just
drivers/mailbox/... (ordrivers/thermal/...) andDocumentation/devicetree/bindings/...
Quoted 8 lines
b) A patch that adds the necessary entries to *.dts/*.dtsi
c) A patch that enables any new entries in
bcm2835_defconfig.In my initial round of review, I'll only discuss the DT bindings, and only in response to patch 2 where they're complete rather than here. Once the bindings are finalized and the driver adjusted to implement the final bindings, then I'll look at the driver code.
I'll look at restructuring the patches. Are you really sure about drivers/mailbox/* and drivers/thermal/* as one patch?
The serious concern that I have is that Suman Anna and Jassi Brar are still working on the core mailbox subsystem. I've just captured their work to present a working patch that will build and show the mailbox working. That is why I marked the series as RFC. I'm not happy about submitting someone else's work that I know is still evolving. Especially when I know that there are a couple of fixes that Jassi has made but not published yet.
For all that, I want mailbox support for the BCM2835 as soon as possible. I see its lack as a serious blocker. The video driver needs it to get the videocore to map an agreed chunk of memory for a framebuffer. The DMA controller could use it to find which DMA channels are available for use by the ARM core. Power management of whole devices uses it (DMA can turn individual channels on and off but that's with iomem register). There may be more, but that's all I've researched. Anything I can do to hasten the general availability of mailbox support ... well, anything that doesn't involve stepping on toes.
Quoted 4 lines
Note that all new DT bindings should be CC'd to the DT bindings maintainers; see the following in
MAINTAINERS:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS
Quoted 10 lines
M: Rob Herring <[email protected]> M: Pawel Moll <[email protected]> M: Mark Rutland <[email protected]> M: Stephen Warren <[email protected]> M: Ian Campbell <[email protected]> L: devicetree at vger.kernel.org S: Maintained F: Documentation/devicetree/ F: arch/*/boot/dts/ F: include/dt-bindings/
Do you want me to CC this interim discussion as well, or wait until there is a more accpetable patch put together.
Quoted 3 lines
Quoted 1 line
+ mailbox {Patch corruption? + is indented!
As I said. Poor tool use.
Cheers,
Craig.