powerpc: ibmveth: Harden driver initilisation for kexec
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): 20060131042903.GF28896@krispykreme
PATCH 1/1 powerpc: ibmveth: Harden driver initilisation for kexec
From: Michael Ellerman <[email protected]>
After a kexec the veth driver will fail when trying to register with the Hypervisor because the previous kernel has not unregistered.
So if the registration fails, we unregister and then try again.
---
drivers/net/ibmveth.c | 32 ++++++++++++++++++++++++++------ 1 files changed, 26 insertions(+), 6 deletions(-)
Looks good to me, and has been around for a couple of months.
drivers/net/ibmveth.c +25 -0
@@ -436,6 +436,31 @@ static void ibmveth_cleanup(struct ibmveibmveth_free_buffer_pool (adapter, &adapter->rx_buff_pool[i]); } +static int ibmveth_register_logical_lan (struct ibmveth_adapter *adapter, +union ibmveth_buf_desc rxq_desc,u64 mac_address) +{ +int rc, try_again =1 ; + +/* After a kexec the adapter will still be open, so our attempt to +* open it will fail. So if we get a failure we free the adapter and +* try again, but only once. */ +retry: + rc =h_register_logical_lan (adapter->vdev->unit_address, + adapter->buffer_list_dma, rxq_desc.desc, + adapter->filter_list_dma, mac_address); + +if (rc != H_Success && try_again) { +do { + rc =h_free_logical_lan (adapter->vdev->unit_address); + }while (H_isLongBusy (rc) || (rc == H_Busy)); + + try_again =0 ; + goto retry; + } + +return rc; +} +static int ibmveth_open (struct net_device *netdev) {struct ibmveth_adapter *adapter = netdev->priv;
@@ -504,12 +529,7 @@ static int ibmveth_open(struct net_devicibmveth_debug_printk ("filter list @ 0x%p\n" , adapter->filter_list_addr); ibmveth_debug_printk("receive q @ 0x%p\n",
adapter->rx_queue.queue_addr);
-
- lpar_rc = h_register_logical_lan(adapter->vdev->unit_address,
- adapter->buffer_list_dma,
- rxq_desc.desc,
- adapter->filter_list_dma,
- mac_address);
+ lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
if(lpar_rc != H_Success) {
ibmveth_error_printk("h_register_logical_lan failed with %ld\n",lpar_rc);
Hi Jeff,
I realise it's late, but it'd be really good if you could send this up for 2.6.16, we're hosed without it.
cheers
Quoted 76 lines · Santiago Leon
From: Michael Ellerman <[email protected]>
After a kexec the veth driver will fail when trying to register with the Hypervisor because the previous kernel has not unregistered.
So if the registration fails, we unregister and then try again.
Signed-off-by Michael Ellerman <[email protected]>Acked-by Anton Blanchard <[email protected]>Signed-off-by Santiago Leon <[email protected]>---
drivers/net/ibmveth.c | 32 ++++++++++++++++++++++++++------ 1 files changed, 26 insertions(+), 6 deletions(-)Looks good to me, and has been around for a couple of months.
Index:
kexec/drivers/net/ibmveth.c
===================================================================drivers/net/ibmveth.c +26
-6@@ -436,6 +436,31 @@ static void ibmveth_cleanup(struct ibmveibmveth_free_buffer_pool (adapter, &adapter->rx_buff_pool[i]); } +static int ibmveth_register_logical_lan (struct ibmveth_adapter *adapter, +union ibmveth_buf_desc rxq_desc,u64 mac_address) +{ +int rc, try_again =1 ; + +/* After a kexec the adapter will still be open, so our attempt to +* open it will fail. So if we get a failure we free the adapter and +* try again, but only once. */ +retry: + rc =h_register_logical_lan (adapter->vdev->unit_address, + adapter->buffer_list_dma, rxq_desc.desc, + adapter->filter_list_dma, mac_address); + +if (rc != H_Success && try_again) { +do { + rc =h_free_logical_lan (adapter->vdev->unit_address); + }while (H_isLongBusy (rc) || (rc == H_Busy)); + + try_again =0 ; + goto retry; + } + +return rc; +} +static int ibmveth_open (struct net_device *netdev) {struct ibmveth_adapter *adapter = netdev->priv;@@ -504,12 +529,7 @@ static int ibmveth_open(struct net_devicibmveth_debug_printk ("filter list @ 0x%p\n" , adapter->filter_list_addr);ibmveth_debug_printk ("receive q @ 0x%p\n" , adapter->rx_queue.queue_addr); - - lpar_rc =h_register_logical_lan (adapter->vdev->unit_address, - adapter->buffer_list_dma, - rxq_desc.desc, - adapter->filter_list_dma, - mac_address); + lpar_rc =ibmveth_register_logical_lan (adapter, rxq_desc, mac_address);if (lpar_rc != H_Success) {ibmveth_error_printk("h_register_logical_lan failed with %ld\n",
lpar_rc);_______________________________________________
Linuxppc64-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc64-dev
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
_______________________________________________
Linuxppc64-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc64-dev
Quoted 4 lines · On Fri, 3 Mar 2006 11:22:45 +1100 Michael Ellerman wrote:
Hi Jeff,
I realise it's late, but it'd be really good if you could send this up for 2.6.16, we're hosed without it.
I'm wondering if this means that for every virtual/hypervisor situation, we have to modify any $interested_drivers. Why wouldn't we come up with a cleaner solution (in the long term)?
E.g., could the hypervisor know when one of it's virtual OSes dies or reboots and release its resources then?
This patch just looks like a short-term solution to me.
Quoted 75 lines
cheers
Quoted 72 lines · Santiago Leon
From: Michael Ellerman <[email protected]>
After a kexec the veth driver will fail when trying to register with the Hypervisor because the previous kernel has not unregistered.
So if the registration fails, we unregister and then try again.
Signed-off-by Michael Ellerman <[email protected]>Acked-by Anton Blanchard <[email protected]>Signed-off-by Santiago Leon <[email protected]>---
drivers/net/ibmveth.c | 32 ++++++++++++++++++++++++++------ 1 files changed, 26 insertions(+), 6 deletions(-)Looks good to me, and has been around for a couple of months.
Index:
kexec/drivers/net/ibmveth.c
===================================================================drivers/net/ibmveth.c +26
-6@@ -436,6 +436,31 @@ static void ibmveth_cleanup(struct ibmveibmveth_free_buffer_pool (adapter, &adapter->rx_buff_pool[i]); } +static int ibmveth_register_logical_lan (struct ibmveth_adapter *adapter, +union ibmveth_buf_desc rxq_desc,u64 mac_address) +{ +int rc, try_again =1 ; + +/* After a kexec the adapter will still be open, so our attempt to +* open it will fail. So if we get a failure we free the adapter and +* try again, but only once. */ +retry: + rc =h_register_logical_lan (adapter->vdev->unit_address, + adapter->buffer_list_dma, rxq_desc.desc, + adapter->filter_list_dma, mac_address); + +if (rc != H_Success && try_again) { +do { + rc =h_free_logical_lan (adapter->vdev->unit_address); + }while (H_isLongBusy (rc) || (rc == H_Busy)); + + try_again =0 ; + goto retry; + } + +return rc; +} +static int ibmveth_open (struct net_device *netdev) {struct ibmveth_adapter *adapter = netdev->priv;@@ -504,12 +529,7 @@ static int ibmveth_open(struct net_devicibmveth_debug_printk ("filter list @ 0x%p\n" , adapter->filter_list_addr);ibmveth_debug_printk ("receive q @ 0x%p\n" , adapter->rx_queue.queue_addr); - - lpar_rc =h_register_logical_lan (adapter->vdev->unit_address, - adapter->buffer_list_dma, - rxq_desc.desc, - adapter->filter_list_dma, - mac_address); + lpar_rc =ibmveth_register_logical_lan (adapter, rxq_desc, mac_address);if (lpar_rc != H_Success) {ibmveth_error_printk("h_register_logical_lan failed with %ld\n",
lpar_rc);
Quoted 8 lines
Quoted 4 lines
_______________________________________________
Linuxppc64-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc64-devSignature · 2 lines
Michael Ellerman IBM OzLabs
---
~Randy
Quoted 2 lines · Randy.Dunlap
E.g., could the hypervisor know when one of it's virtual OSes dies or reboots and release its resources then?
I think the point is that with kexec, the same virtual machine keeps running, so the hypervisor doesn't see the OS dying or rebooting.
Paul.
Quoted 6 lines · Randy.Dunlap
Quoted 4 lines · On Fri, 3 Mar 2006 11:22:45 +1100 Michael Ellerman wrote:
Hi Jeff,
I realise it's late, but it'd be really good if you could send this up for 2.6.16, we're hosed without it.
Quoted 6 lines
I'm wondering if this means that for every virtual/hypervisor situation, we have to modify any $interested_drivers. Why wouldn't we come up with a cleaner solution (in the long term)?
E.g., could the hypervisor know when one of it's virtual OSes dies or reboots and release its resources then?
It does exactly that for a regular reboot, but when we kexec we _don't_ die or reboot, as far as the Hypervisor is concerned it's all systems go.
It's something of a double-edged sword, we're totally in control which gives us lots of flexibility, and _fast_ reboot times, but we also have to do a bit of extra stuff (ie. this patch) to keep things sane.
cheers
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
_______________________________________________
Linuxppc64-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc64-dev
Quoted 14 lines · On Fri, 3 Mar 2006 12:10:47 +1100 Michael Ellerman wrote:
Quoted 6 lines · Randy.Dunlap
Quoted 4 lines · On Fri, 3 Mar 2006 11:22:45 +1100 Michael Ellerman wrote:
Hi Jeff,
I realise it's late, but it'd be really good if you could send this up for 2.6.16, we're hosed without it.
Quoted 6 lines
I'm wondering if this means that for every virtual/hypervisor situation, we have to modify any $interested_drivers. Why wouldn't we come up with a cleaner solution (in the long term)?
E.g., could the hypervisor know when one of it's virtual OSes dies or reboots and release its resources then?
Quoted 6 lines
It does exactly that for a regular reboot, but when we kexec we _don't_ die or reboot, as far as the Hypervisor is concerned it's all systems go.
It's something of a double-edged sword, we're totally in control which gives us lots of flexibility, and _fast_ reboot times, but we also have to do a bit of extra stuff (ie. this patch) to keep things sane.
s/this patch/some patch/
Yes, you have certainly thought about this more/longer than I have, so why is something more generic like this bad instead of good:
Somewhere early in start_kernel() (e.g.), do an hv call that says "free all assigned resources".
Maybe hv doesn't know "all assigned resources."
Maybe it's just that this patch is simpler than an hv change, although this (current) patch could leave some other drivers that need to be "fixed," while an hv change wouldn't do that.
So I'm not opposed to this current patch as a short-term solution, but I don't think it's the right long-term solution.
---
~Randy
Quoted 10 lines · Santiago Leon
From: Michael Ellerman <[email protected]>
After a kexec the veth driver will fail when trying to register with the Hypervisor because the previous kernel has not unregistered.
So if the registration fails, we unregister and then try again.
Signed-off-by Michael Ellerman <[email protected]>Acked-by Anton Blanchard <[email protected]>Signed-off-by Santiago Leon <[email protected]>
Quoted 6 lines
---
drivers/net/ibmveth.c | 32 ++++++++++++++++++++++++++------ 1 files changed, 26 insertions(+), 6 deletions(-)Looks good to me, and has been around for a couple of months.
This seems completely bonkers to me:
are resources available?
if no
free resources
try again
It makes resource checking pointless.
Jeff
Quoted 16 lines · Jeff Garzik
Quoted 15 lines · Santiago Leon
From: Michael Ellerman <[email protected]>
After a kexec the veth driver will fail when trying to register with the Hypervisor because the previous kernel has not unregistered.
So if the registration fails, we unregister and then try again.
Signed-off-by Michael Ellerman <[email protected]>Acked-by Anton Blanchard <[email protected]>Signed-off-by Santiago Leon <[email protected]>---
drivers/net/ibmveth.c | 32 ++++++++++++++++++++++++++------ 1 files changed, 26 insertions(+), 6 deletions(-)
Quoted 8 lines
Quoted 1 line
Looks good to me, and has been around for a couple of months.
This seems completely bonkers to me:
are resources available?
if no
free resources
try again
I'm not sure I follow, are you suggesting we do the h_free_logical_lan() unconditionally, followed by h_register_logical_lan() ??
If that's what you mean, I didn't do it that way because it would effect the normal code path. This patch only modifies the behaviour if we fail to register the adapter. I'm much more comfortable changing the failure case than the default.
cheers
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
_______________________________________________
Linuxppc64-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc64-dev
Quoted 7 lines · Santiago Leon
From: Michael Ellerman <[email protected]>
After a kexec the veth driver will fail when trying to register with the Hypervisor because the previous kernel has not unregistered.
So if the registration fails, we unregister and then try again.
Quoted 7 lines
Signed-off-by Michael Ellerman <[email protected]>Acked-by Anton Blanchard <[email protected]>Signed-off-by Santiago Leon <[email protected]>---
drivers/net/ibmveth.c | 32 ++++++++++++++++++++++++++------ 1 files changed, 26 insertions(+), 6 deletions(-)
Looks like this hit the floor. Any chance of getting it into to 2.6.17 Jeff? AFAICT it should still apply cleanly.
cheers
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
Quoted 13 lines · Michael Ellerman
Quoted 12 lines · Santiago Leon
From: Michael Ellerman <[email protected]>
After a kexec the veth driver will fail when trying to register with the Hypervisor because the previous kernel has not unregistered.
So if the registration fails, we unregister and then try again.
Signed-off-by Michael Ellerman <[email protected]>Acked-by Anton Blanchard <[email protected]>Signed-off-by Santiago Leon <[email protected]>---
Quoted 5 lines
Quoted 2 lines
drivers/net/ibmveth.c | 32 ++++++++++++++++++++++++++------ 1 files changed, 26 insertions(+), 6 deletions(-)Looks like this hit the floor. Any chance of getting it into to 2.6.17 Jeff? AFAICT it should still apply cleanly.
/me knocks politely
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
Santiago Leon
Acked-by Anton BlanchardQuoted 4 lines · Michael Ellerman
Quoted 2 lines
Looks like this hit the floor. Any chance of getting it into to 2.6.17 Jeff? AFAICT it should still apply cleanly.
/me knocks politely
Actually, it doesn't apply cleanly anymore. Here's a patch that does.
Signature · 3 lines
Santiago A. Leon Power Linux Development IBM Linux Technology Center
From: Michael Ellerman <[email protected]>
After a kexec the ibmveth driver will fail when trying to register with the Hypervisor because the previous kernel has not unregistered.
So if the registration fails, we unregister and then try again.
ibmveth.c | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-)
drivers/net/ibmveth.c +26 -5
@@ -436,6 +436,31 @@ static void ibmveth_cleanup(struct ibmveibmveth_free_buffer_pool (adapter, &adapter->rx_buff_pool[i]); } +static int ibmveth_register_logical_lan (struct ibmveth_adapter *adapter, +union ibmveth_buf_desc rxq_desc,u64 mac_address) +{ +int rc, try_again =1 ; + +/* After a kexec the adapter will still be open, so our attempt to +* open it will fail. So if we get a failure we free the adapter and +* try again, but only once. */ +retry: + rc =h_register_logical_lan (adapter->vdev->unit_address, + adapter->buffer_list_dma, rxq_desc.desc, + adapter->filter_list_dma, mac_address); + +if (rc !=H_SUCCESS && try_again) { +do { + rc =h_free_logical_lan (adapter->vdev->unit_address); + }while (H_IS_LONG_BUSY (rc) || (rc ==H_BUSY )); + + try_again =0 ; + goto retry; + } + +return rc; +} +static int ibmveth_open (struct net_device *netdev) {struct ibmveth_adapter *adapter = netdev->priv;
@@ -505,11 +530,7 @@ static int ibmveth_open(struct net_devicibmveth_debug_printk ("receive q @ 0x%p\n" , adapter->rx_queue.queue_addr); - lpar_rc =h_register_logical_lan (adapter->vdev->unit_address, - adapter->buffer_list_dma, - rxq_desc.desc, - adapter->filter_list_dma, - mac_address); + lpar_rc =ibmveth_register_logical_lan (adapter, rxq_desc, mac_address);if (lpar_rc !=H_SUCCESS ) {ibmveth_error_printk ("h_register_logical_lan failed with %ld\n" , lpar_rc);
Quoted 2 lines · Santiago Leon
Quoted 1 line · Michael Ellerman
Any chance of getting it into to 2.6.17 ...
2.6.19 perhaps?
cheers