partition suspend updates
v2 cover partition suspend updates
This series aims to improve the pseries-specific partition migration and hibernation implementation, part of which has been living in kernel/rtas.c. Most of that code is eliminated or moved to platforms/pseries, and the following major functional changes are made:
- Use stop_machine() instead of on_each_cpu() to avoid deadlock in the join/suspend sequence.
- Retry the join/suspend sequence on errors that are likely to be transient. This is a mitigation for the fact that drivers currently have no way to prepare for an impending partition suspension, sometimes resulting in a virtual adapter being in a state which causes the platform to fail the suspend call.
- Request cancellation of the migration via H_VASI_SIGNAL if Linux is going to error out of the suspend attempt. This allows the management console and other entities to promptly clean up their operations instead of relying on long timeouts to fail the migration.
- Little-endian users of ibm,suspend-me, ibm,update-nodes and ibm,update-properties via sys_rtas are blocked when CONFIG_PPC_RTAS_FILTERS is enabled.
- Legacy user space code (drmgr) historically has driven the migration process by using sys_rtas to separately call ibm,suspend-me, ibm,activate-firmware, and ibm,update-nodes/properties, in that order. With these changes, when sys_rtas() dispatches ibm,suspend-me, the kernel performs the device tree update and firmware activation before returning. This is more reliable, and drmgr does not seem bothered by it.
- If the H_VASI_STATE hcall is absent, the implementation proceeds with the suspend instead of erroring out. This allows us to exercise these code paths in QEMU.
Changes since v1:
- Drop "powerpc/rtas: move rtas_call_reentrant() out of pseries guards". rtas_call_reentrant() actually is pseries-specific and this broke builds without CONFIG_PPC_PSERIES set.
- Simplify polling logic in wait_for_vasi_session_suspending(). ("powerpc/pseries/mobility: extract VASI session polling logic")
- Use direct return instead of goto in pseries_migrate_partition(). ("powerpc/pseries/mobility: use stop_machine for join/suspend")
- Change dispatch of ibm,suspend-me in rtas syscall path to use conventional config symbol guards instead of a weak function. ("powerpc/rtas: dispatch partition migration requests to pseries")
- Fix refcount imbalance in add_dt_node() error path. ("powerpc/pseries/mobility: refactor node lookup during DT update")
Nathan Lynch (28):
powerpc/rtas: prevent suspend-related sys_rtas use on LE powerpc/rtas: complete ibm,suspend-me status codes powerpc/rtas: rtas_ibm_suspend_me -> rtas_ibm_suspend_me_unsafe powerpc/rtas: add rtas_ibm_suspend_me()
powerpc/rtas: add rtas_activate_firmware()
powerpc/hvcall: add token and codes for H_VASI_SIGNAL powerpc/pseries/mobility: don't error on absence of ibm,update-nodes powerpc/pseries/mobility: add missing break to default case powerpc/pseries/mobility: error message improvements powerpc/pseries/mobility: use rtas_activate_firmware() on resume powerpc/pseries/mobility: extract VASI session polling logic powerpc/pseries/mobility: use stop_machine for join/suspend powerpc/pseries/mobility: signal suspend cancellation to platform powerpc/pseries/mobility: retry partition suspend after error powerpc/rtas: dispatch partition migration requests to pseries powerpc/rtas: remove rtas_ibm_suspend_me_unsafe() powerpc/pseries/hibernation: drop pseries_suspend_begin() from suspend
ops
powerpc/pseries/hibernation: pass stream id via function arguments powerpc/pseries/hibernation: remove pseries_suspend_cpu() powerpc/machdep: remove suspend_disable_cpu() powerpc/rtas: remove rtas_suspend_cpu() powerpc/pseries/hibernation: switch to rtas_ibm_suspend_me() powerpc/rtas: remove unused rtas_suspend_last_cpu() powerpc/pseries/hibernation: remove redundant cacheinfo update powerpc/pseries/hibernation: perform post-suspend fixups later powerpc/pseries/hibernation: remove prepare_late() callback powerpc/rtas: remove unused rtas_suspend_me_data powerpc/pseries/mobility: refactor node lookup during DT update
arch/powerpc/include/asm/hvcall.h | 9 + arch/powerpc/include/asm/machdep.h | 1 - arch/powerpc/include/asm/rtas-types.h | 8 - arch/powerpc/include/asm/rtas.h | 17 +- arch/powerpc/kernel/rtas.c | 243 ++++++--------- arch/powerpc/platforms/pseries/mobility.c | 358 ++++++++++++++++++---- arch/powerpc/platforms/pseries/suspend.c | 79 +---- 7 files changed, 415 insertions(+), 300 deletions(-)
Signature · 1 line
2.28.0
PATCH v2 1/28 powerpc/rtas: prevent suspend-related sys_rtas use on LE
While drmgr has had work in some areas to make its RTAS syscall interactions endian-neutral, its code for performing partition migration via the syscall has never worked on LE. While it is able to complete ibm,suspend-me successfully, it crashes when attempting the subsequent ibm,update-nodes call.
drmgr is the only known (or plausible) user of ibm,suspend-me, ibm,update-nodes, and ibm,update-properties, so allow them only in big-endian configurations.
--- arch/powerpc/kernel/rtas.c | 2 ++ 1 file changed, 2 insertions(+)
arch/powerpc/kernel/rtas.c +2 -0
@@ -1050,9 +1050,11 @@ static struct rtas_filter rtas_filters[] __ro_after_init = { { "set-time-for-power-on", -1, -1, -1, -1, -1 }, { "ibm,set-system-parameter", -1, 1, -1, -1, -1 }, { "set-time-of-day", -1, -1, -1, -1, -1 }, +#ifdef CONFIG_CPU_BIG_ENDIAN { "ibm,suspend-me", -1, -1, -1, -1, -1 }, { "ibm,update-nodes", -1, 0, -1, -1, -1, 4096 }, { "ibm,update-properties", -1, 0, -1, -1, -1, 4096 }, +#endif { "ibm,physical-attestation", -1, 0, 1, -1, -1 }, };
Signature · 1 line
2.28.0
PATCH v2 2/28 powerpc/rtas: complete ibm,suspend-me status codes
In reply to Nathan LynchWe don't completely account for the possible return codes for ibm,suspend-me. Add definitions for these.
--- arch/powerpc/include/asm/rtas.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/rtas.h +6 -1
@@ -23,11 +23,16 @@#define RTAS_RMOBUF_MAX (64 * 1024)/* RTAS return status codes */ -#define RTAS_NOT_SUSPENDABLE -9004#define RTAS_BUSY -2/* RTAS Busy */ #define RTAS_EXTENDED_DELAY_MIN 9900#define RTAS_EXTENDED_DELAY_MAX 9905 +/* statuses specific to ibm,suspend-me */ +#define RTAS_SUSPEND_ABORTED 9000/* Suspension aborted */ +#define RTAS_NOT_SUSPENDABLE -9004/* Partition not suspendable */ +#define RTAS_THREADS_ACTIVE -9005/* Multiple processor threads active */ +#define RTAS_OUTSTANDING_COPROC -9006/* Outstanding coprocessor operations */ +/* * In general to call RTAS use rtas_token("string") to lookup * an RTAS token for the given string (e.g. "event-scan").
Signature · 1 line
2.28.0
PATCH v2 3/28 powerpc/rtas: rtas_ibm_suspend_me -> rtas_ibm_suspend_me_unsafe
In reply to Nathan LynchThe pseries partition suspend sequence requires that all active CPUs call H_JOIN, which suspends all but one of them with interrupts disabled. The "chosen" CPU is then to call ibm,suspend-me to complete the suspend. Upon returning from ibm,suspend-me, the chosen CPU is to use H_PROD to wake the joined CPUs.
Using on_each_cpu() for this, as rtas_ibm_suspend_me() does to implement partition migration, is susceptible to deadlock with other users of on_each_cpu() and with users of stop_machine APIs. The callback passed to on_each_cpu() is not allowed to synchronize with other CPUs in the way it is used here.
Complicating the fix is the fact that rtas_ibm_suspend_me() also occupies the function name that should be used to provide a more conventional wrapper for ibm,suspend-me. Rename rtas_ibm_suspend_me() to rtas_ibm_suspend_me_unsafe() to free up the name and indicate that it should not gain users.
--- arch/powerpc/include/asm/rtas.h | 2 +- arch/powerpc/kernel/rtas.c | 6 +++--- arch/powerpc/platforms/pseries/mobility.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-)
arch/powerpc/include/asm/rtas.h +1 -1
@@ -257,7 +257,7 @@ extern int rtas_set_indicator_fast(int indicator, int index, int new_value);extern void rtas_progress (char *s,unsigned short hex);extern int rtas_suspend_cpu (struct rtas_suspend_me_data *data);extern int rtas_suspend_last_cpu (struct rtas_suspend_me_data *data); -extern int rtas_ibm_suspend_me (u64 handle); +int rtas_ibm_suspend_me_unsafe (u64 handle);struct rtc_time ;extern time64_t rtas_get_boot_time (void );
arch/powerpc/kernel/rtas.c +3 -3
@@ -843,7 +843,7 @@ static void rtas_percpu_suspend_me(void *info)__rtas_suspend_cpu ((struct rtas_suspend_me_data *)info,1 ); } -int rtas_ibm_suspend_me (u64 handle) +int rtas_ibm_suspend_me_unsafe (u64 handle) {long state;long rc;
@@ -949,7 +949,7 @@ int rtas_call_reentrant(int token, int nargs, int nret, int *outputs, ...) }#else /* CONFIG_PPC_PSERIES */ -int rtas_ibm_suspend_me (u64 handle) +int rtas_ibm_suspend_me_unsafe (u64 handle) {return -ENOSYS ; }
@@ -1185,7 +1185,7 @@ SYSCALL_DEFINE1(rtas, struct rtas_args __user *, uargs)int rc =0 ;u64 handle = ((u64 )be32_to_cpu (args.args[0 ]) <<32 ) |be32_to_cpu (args.args[1 ]); - rc =rtas_ibm_suspend_me (handle); + rc =rtas_ibm_suspend_me_unsafe (handle);if (rc == -EAGAIN ) args.rets[0 ] =cpu_to_be32 (RTAS_NOT_SUSPENDABLE );else if (rc == -EIO )
arch/powerpc/platforms/pseries/mobility.c +1 -1
@@ -370,7 +370,7 @@ static ssize_t migration_store(struct class *class,return rc;do { - rc =rtas_ibm_suspend_me (streamid); + rc =rtas_ibm_suspend_me_unsafe (streamid);if (rc == -EAGAIN )ssleep (1 ); }while (rc == -EAGAIN );
Signature · 1 line
2.28.0
PATCH v2 4/28 powerpc/rtas: add rtas_ibm_suspend_me()
In reply to Nathan LynchNow that the name is available, provide a simple wrapper for ibm,suspend-me which returns both a Linux errno and optionally the actual RTAS status to the caller.
--- arch/powerpc/include/asm/rtas.h | 1 + arch/powerpc/kernel/rtas.c | 57 +++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+)
arch/powerpc/include/asm/rtas.h +1 -0
@@ -258,6 +258,7 @@ extern void rtas_progress(char *s, unsigned short hex);extern int rtas_suspend_cpu (struct rtas_suspend_me_data *data);extern int rtas_suspend_last_cpu (struct rtas_suspend_me_data *data);int rtas_ibm_suspend_me_unsafe (u64 handle); +int rtas_ibm_suspend_me (int *fw_status);struct rtc_time ;extern time64_t rtas_get_boot_time (void );
arch/powerpc/kernel/rtas.c +57 -0
63 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v2 5/28 powerpc/rtas: add rtas_activate_firmware()
In reply to Nathan LynchProvide a documented wrapper function for the ibm,activate-firmware service, which must be called after a partition migration or hibernation.
If the function is absent or the call fails, the OS will continue to run normally with the current firmware, so there is no need to perform any recovery. Just log it and continue.
--- arch/powerpc/include/asm/rtas.h | 1 + arch/powerpc/kernel/rtas.c | 30 ++++++++++++++++++++++++++++++ 2 files changed, 31 insertions(+)
arch/powerpc/include/asm/rtas.h +1 -0
@@ -247,6 +247,7 @@ extern void __noreturn rtas_restart(char *cmd);extern void rtas_power_off (void );extern void __noreturn rtas_halt (void );extern void rtas_os_term (char *str); +void rtas_activate_firmware (void );extern int rtas_get_sensor (int sensor,int index,int *state);extern int rtas_get_sensor_fast (int sensor,int index,int *state);extern int rtas_get_power_level (int powerdomain,int *level);
arch/powerpc/kernel/rtas.c +30 -0
@@ -798,6 +798,36 @@ void rtas_os_term(char *str)printk (KERN_EMERG "ibm,os-term call failed %d\n" , status); } +/** +* rtas_activate_firmware() - Activate a new version of firmware. +* +* Activate a new version of partition firmware. The OS must call this +* after resuming from a partition hibernation or migration in order +* to maintain the ability to perform live firmware updates. It's not +* catastrophic for this method to be absent or to fail; just log the +* condition in that case. +* +* Context: This function may sleep. +*/ +void rtas_activate_firmware (void ) +{ +int token; +int fwrc; + + token =rtas_token ("ibm,activate-firmware" ); +if (token ==RTAS_UNKNOWN_SERVICE ) { +pr_notice ("ibm,activate-firmware method unavailable\n" ); +return ; + } + +do { + fwrc =rtas_call (token,0 ,1 ,NULL ); + }while (rtas_busy_delay (fwrc)); + +if (fwrc) +pr_err ("ibm,activate-firmware failed (%i)\n" , fwrc); +} +static int ibm_suspend_me_token =RTAS_UNKNOWN_SERVICE ;#ifdef CONFIG_PPC_PSERIES static int __rtas_suspend_last_cpu (struct rtas_suspend_me_data *data,int wake_when_done)
Signature · 1 line
2.28.0
PATCH v2 6/28 powerpc/hvcall: add token and codes for H_VASI_SIGNAL
In reply to Nathan LynchH_VASI_SIGNAL can be used by a partition to request cancellation of its migration. To be used in future changes.
--- arch/powerpc/include/asm/hvcall.h | 9 +++++++++ 1 file changed, 9 insertions(+)
arch/powerpc/include/asm/hvcall.h +9 -0
@@ -155,6 +155,14 @@#define H_VASI_RESUMED 5#define H_VASI_COMPLETED 6 +/* VASI signal codes. Only the Cancel code is valid for H_VASI_SIGNAL. */ +#define H_VASI_SIGNAL_CANCEL 1 +#define H_VASI_SIGNAL_ABORT 2 +#define H_VASI_SIGNAL_SUSPEND 3 +#define H_VASI_SIGNAL_COMPLETE 4 +#define H_VASI_SIGNAL_ENABLE 5 +#define H_VASI_SIGNAL_FAILOVER 6 +/* Each control block has to be on a 4K boundary */ #define H_CB_ALIGNMENT 4096
@@ -261,6 +269,7 @@#define H_ADD_CONN 0x284#define H_DEL_CONN 0x288#define H_JOIN 0x298 +#define H_VASI_SIGNAL 0x2A0#define H_VASI_STATE 0x2A4#define H_VIOCTL 0x2A8#define H_ENABLE_CRQ 0x2B0
Signature · 1 line
2.28.0
PATCH v2 7/28 powerpc/pseries/mobility: don't error on absence of ibm, update-nodes
In reply to Nathan LynchTreat the absence of the ibm,update-nodes function as benign instead of reporting an error. If the platform does not provide that facility, it's not a problem for Linux.
--- arch/powerpc/platforms/pseries/mobility.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/platforms/pseries/mobility.c +1 -1
@@ -261,7 +261,7 @@ int pseries_devicetree_update(s32 scope) update_nodes_token =rtas_token ("ibm,update-nodes" );if (update_nodes_token ==RTAS_UNKNOWN_SERVICE ) -return -EINVAL ; +return 0 ; rtas_buf =kzalloc (RTAS_DATA_BUF_SIZE ,GFP_KERNEL );if (!rtas_buf)
Signature · 1 line
2.28.0
PATCH v2 8/28 powerpc/pseries/mobility: add missing break to default case
In reply to Nathan Lynchupdate_dt_node() has a switch statement where the default case lacks a break statement.
--- arch/powerpc/platforms/pseries/mobility.c | 1 + 1 file changed, 1 insertion(+)
arch/powerpc/platforms/pseries/mobility.c +1 -0
@@ -213,6 +213,7 @@ static int update_dt_node(__be32 phandle, s32 scope) } prop_data += vd; +break ; }cond_resched ();
Signature · 1 line
2.28.0
PATCH v2 9/28 powerpc/pseries/mobility: error message improvements
In reply to Nathan Lynch- Convert printk(KERN_ERR) to pr_err().
- Include errno in property update failure message.
- Remove reference to "Post-mobility" from device tree update message:
with pr_err() it will have a "mobility:" prefix.
--- arch/powerpc/platforms/pseries/mobility.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
arch/powerpc/platforms/pseries/mobility.c +3 -4
@@ -208,8 +208,8 @@ static int update_dt_node(__be32 phandle, s32 scope) rc =update_dt_property (dn, &prop, prop_name, vd, prop_data);if (rc) { -printk (KERN_ERR "Could not update %s" -" property\n" , prop_name); +pr_err ("updating %s property failed: %d\n" , + prop_name, rc); } prop_data += vd;
@@ -343,8 +343,7 @@ void post_mobility_fixup(void) rc =pseries_devicetree_update (MIGRATION_SCOPE );if (rc) -printk (KERN_ERR "Post-mobility device tree update " -"failed: %d\n" , rc); +pr_err ("device tree update failed: %d\n" , rc);cacheinfo_rebuild ();
Signature · 1 line
2.28.0
PATCH v2 10/28 powerpc/pseries/mobility: use rtas_activate_firmware() on resume
In reply to Nathan LynchIt's incorrect to abort post-suspend processing if ibm,activate-firmware isn't available. Use rtas_activate_firmware(), which logs this condition appropriately and allows us to proceed.
--- arch/powerpc/platforms/pseries/mobility.c | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-)
arch/powerpc/platforms/pseries/mobility.c +1 -14
@@ -312,21 +312,8 @@ int pseries_devicetree_update(s32 scope)void post_mobility_fixup (void ) {int rc; -int activate_fw_token; - activate_fw_token =rtas_token ("ibm,activate-firmware" ); -if (activate_fw_token ==RTAS_UNKNOWN_SERVICE ) { -printk (KERN_ERR "Could not make post-mobility " -"activate-fw call.\n" ); -return ; - } - -do { - rc =rtas_call (activate_fw_token,0 ,1 ,NULL ); - }while (rtas_busy_delay (rc)); - -if (rc) -printk (KERN_ERR "Post-mobility activate-fw failed: %d\n" , rc); +rtas_activate_firmware ();/* * We don't want CPUs to go online/offline while the device
Signature · 1 line
2.28.0
PATCH v2 11/28 powerpc/pseries/mobility: extract VASI session polling logic
In reply to Nathan LynchThe behavior of rtas_ibm_suspend_me_unsafe() is to return -EAGAIN to the caller until the specified VASI suspend session state makes the transition from H_VASI_ENABLED to H_VASI_SUSPENDING. In the interest of separating concerns to prepare for a new implementation of the join/suspend sequence, extract VASI session polling logic into a couple of local functions. Waiting for the session state to reach H_VASI_SUSPENDING before calling rtas_ibm_suspend_me_unsafe() ensures that we will never get an EAGAIN result necessitating a retry. No user-visible change in behavior is intended.
--- arch/powerpc/platforms/pseries/mobility.c | 69 +++++++++++++++++++++-- 1 file changed, 64 insertions(+), 5 deletions(-)
arch/powerpc/platforms/pseries/mobility.c +64 -5
@@ -345,6 +345,66 @@ void post_mobility_fixup(void)return ; } +static int poll_vasi_state (u64 handle,unsigned long *res) +{ +unsigned long retbuf[PLPAR_HCALL_BUFSIZE ]; +long hvrc; +int ret; + + hvrc =plpar_hcall (H_VASI_STATE , retbuf, handle); +switch (hvrc) { +case H_SUCCESS : + ret =0 ; + *res = retbuf[0 ]; +break ; +case H_PARAMETER : + ret = -EINVAL ; +break ; +case H_FUNCTION : + ret = -EOPNOTSUPP ; +break ; +case H_HARDWARE : +default : +pr_err ("unexpected H_VASI_STATE result %ld\n" , hvrc); + ret = -EIO ; +break ; + } +return ret; +} + +static int wait_for_vasi_session_suspending (u64 handle) +{ +unsigned long state; +int ret; + +/* +* Wait for transition from H_VASI_ENABLED to +* H_VASI_SUSPENDING. Treat anything else as an error. +*/ +while (true) { + ret =poll_vasi_state (handle, &state); + +if (ret !=0 || state ==H_VASI_SUSPENDING ) { +break ; + }else if (state ==H_VASI_ENABLED ) { +ssleep (1 ); + }else { +pr_err ("unexpected H_VASI_STATE result %lu\n" , state); + ret = -EIO ; +break ; + } + } + +/* +* Proceed even if H_VASI_STATE is unavailable. If H_JOIN or +* ibm,suspend-me are also unimplemented, we'll recover then. +*/ +if (ret == -EOPNOTSUPP ) + ret =0 ; + +return ret; +} +static ssize_t migration_store (struct class *class,struct class_attribute *attr,const char *buf,size_t count)
@@ -356,12 +416,11 @@ static ssize_t migration_store(struct class *class,if (rc)return rc; -do { - rc =rtas_ibm_suspend_me_unsafe (streamid); -if (rc == -EAGAIN ) -ssleep (1 ); - }while (rc == -EAGAIN ); + rc =wait_for_vasi_session_suspending (streamid); +if (rc) +return rc; + rc =rtas_ibm_suspend_me_unsafe (streamid);if (rc)return rc;
Signature · 1 line
2.28.0
PATCH v2 12/28 powerpc/pseries/mobility: use stop_machine for join/suspend
In reply to Nathan LynchThe partition suspend sequence as specified in the platform architecture requires that all active processor threads call H_JOIN, which:
- suspends the calling thread until it is the target of
an H_PROD; or
- immediately returns H_CONTINUE, if the calling thread is the last to call H_JOIN. This thread is expected to call ibm,suspend-me to completely suspend the partition.
Upon returning from ibm,suspend-me the calling thread must wake all others using H_PROD.
rtas_ibm_suspend_me_unsafe() uses on_each_cpu() to implement this protocol, but because of its synchronizing nature this is susceptible to deadlock versus users of stop_machine() or other callers of on_each_cpu().
Not only is stop_machine() intended for use cases like this, it handles error propagation and allows us to keep the data shared between CPUs minimal: a single atomic counter which ensures exactly one CPU will wake the others from their joined states.
Switch the migration code to use stop_machine() and a less complex local implementation of the H_JOIN/ibm,suspend-me logic, which carries additional benefits:
- more informative error reporting, appropriately ratelimited
- resets the lockup detector / watchdog on resume to prevent lockup warnings when the OS has been suspended for a time exceeding the threshold.
--- arch/powerpc/platforms/pseries/mobility.c | 132 ++++++++++++++++++++-- 1 file changed, 125 insertions(+), 7 deletions(-)
arch/powerpc/platforms/pseries/mobility.c +125 -7
@@ -12,9 +12,11 @@#include <linux/cpu.h> #include <linux/kernel.h> #include <linux/kobject.h> +#include <linux/nmi.h> #include <linux/sched.h> #include <linux/smp.h> #include <linux/stat.h> +#include <linux/stop_machine.h> #include <linux/completion.h> #include <linux/device.h> #include <linux/delay.h>
@@ -405,6 +407,128 @@ static int wait_for_vasi_session_suspending(u64 handle)return ret; } +static void prod_single (unsigned int target_cpu) +{ +long hvrc; +int hwid; + + hwid =get_hard_smp_processor_id (target_cpu); + hvrc =plpar_hcall_norets (H_PROD , hwid); +if (hvrc ==H_SUCCESS ) +return ; +pr_err_ratelimited ("H_PROD of CPU %u (hwid %d) error: %ld\n" , + target_cpu, hwid, hvrc); +} + +static void prod_others (void ) +{ +unsigned int cpu; + +for_each_online_cpu (cpu) { +if (cpu !=smp_processor_id ()) +prod_single (cpu); + } +} + +static u16 clamp_slb_size (void ) +{ +u16 prev = mmu_slb_size; + +slb_set_size (SLB_MIN_SIZE ); + +return prev; +} + +static int do_suspend (void ) +{ +u16 saved_slb_size; +int status; +int ret; + +pr_info ("calling ibm,suspend-me on CPU %i\n" ,smp_processor_id ()); + +/* +* The destination processor model may have fewer SLB entries +* than the source. We reduce mmu_slb_size to a safe minimum +* before suspending in order to minimize the possibility of +* programming non-existent entries on the destination. If +* suspend fails, we restore it before returning. On success +* the OF reconfig path will update it from the new device +* tree after resuming on the destination. +*/ + saved_slb_size =clamp_slb_size (); + + ret =rtas_ibm_suspend_me (&status); +if (ret !=0 ) { +pr_err ("ibm,suspend-me error: %d\n" , status); +slb_set_size (saved_slb_size); + } + +return ret; +} + +static int do_join (void *arg) +{ +atomic_t *counter = arg; +long hvrc; +int ret; + +/* Must ensure MSR.EE off for H_JOIN. */ +hard_irq_disable (); + hvrc =plpar_hcall_norets (H_JOIN ); + +switch (hvrc) { +case H_CONTINUE : +/* +* All other CPUs are offline or in H_JOIN. This CPU +* attempts the suspend. +*/ + ret =do_suspend (); +break ; +case H_SUCCESS : +/* +* The suspend is complete and this cpu has received a +* prod. +*/ + ret =0 ; +break ; +case H_BAD_MODE : +case H_HARDWARE : +default : + ret = -EIO ; +pr_err_ratelimited ("H_JOIN error %ld on CPU %i\n" , + hvrc,smp_processor_id ()); +break ; + } + +if (atomic_inc_return (counter) ==1 ) { +pr_info ("CPU %u waking all threads\n" ,smp_processor_id ()); +prod_others (); + } +/* +* Execution may have been suspended for several seconds, so +* reset the watchdog. +*/ +touch_nmi_watchdog (); +return ret; +} + +static int pseries_migrate_partition (u64 handle) +{ +atomic_t counter =ATOMIC_INIT (0 ); +int ret; + + ret =wait_for_vasi_session_suspending (handle); +if (ret) +return ret; + + ret =stop_machine (do_join, &counter, cpu_online_mask); +if (ret ==0 ) +post_mobility_fixup (); + +return ret; +} +static ssize_t migration_store (struct class *class,struct class_attribute *attr,const char *buf,size_t count)
@@ -416,16 +540,10 @@ static ssize_t migration_store(struct class *class,if (rc)return rc; - rc =wait_for_vasi_session_suspending (streamid); + rc =pseries_migrate_partition (streamid);if (rc)return rc; - rc =rtas_ibm_suspend_me_unsafe (streamid); -if (rc) -return rc; - -post_mobility_fixup (); -return count; }
Signature · 1 line
2.28.0
PATCH v2 13/28 powerpc/pseries/mobility: signal suspend cancellation to platform
In reply to Nathan LynchIf we're returning an error to user space, use H_VASI_SIGNAL to send a cancellation request to the platform. This isn't strictly required but it communicates that Linux will not attempt to complete the suspend, which allows the various entities involved to promptly end the operation in progress.
--- arch/powerpc/platforms/pseries/mobility.c | 31 +++++++++++++++++++++++ 1 file changed, 31 insertions(+)
arch/powerpc/platforms/pseries/mobility.c +31 -0
@@ -513,6 +513,35 @@ static int do_join(void *arg)return ret; } +/* +* Abort reason code byte 0. We use only the 'Migrating partition' value. +*/ +enum vasi_aborting_entity { +ORCHESTRATOR =1 , +VSP_SOURCE =2 , +PARTITION_FIRMWARE =3 , +PLATFORM_FIRMWARE =4 , +VSP_TARGET =5 , +MIGRATING_PARTITION =6 , +}; + +static void pseries_cancel_migration (u64 handle,int err) +{ +u32 reason_code; +u32 detail; +u8 entity; +long hvrc; + + entity =MIGRATING_PARTITION ; + detail =abs (err) &0xffffff ; + reason_code = (entity <<24 ) | detail; + + hvrc =plpar_hcall_norets (H_VASI_SIGNAL , handle, +H_VASI_SIGNAL_CANCEL , reason_code); +if (hvrc) +pr_err ("H_VASI_SIGNAL error: %ld\n" , hvrc); +} +static int pseries_migrate_partition (u64 handle) {atomic_t counter =ATOMIC_INIT (0 );
@@ -525,6 +554,8 @@ static int pseries_migrate_partition(u64 handle) ret =stop_machine (do_join, &counter, cpu_online_mask);if (ret ==0 )post_mobility_fixup (); +else +pseries_cancel_migration (handle, ret);return ret; }
Signature · 1 line
2.28.0
PATCH v2 14/28 powerpc/pseries/mobility: retry partition suspend after error
In reply to Nathan LynchThis is a mitigation for the relatively rare occurrence where a virtual IOA can be in a transient state that prevents the suspend/migration from succeeding, resulting in an error from ibm,suspend-me.
If the join/suspend sequence returns an error, it is acceptable to retry as long as the VASI suspend session state is still "Suspending" (i.e. the platform is still waiting for the OS to suspend).
Retry a few times on suspend failure while this condition holds, progressively increasing the delay between attempts. We don't want to retry indefinitey because firmware emits an error log event on each unsuccessful attempt.
--- arch/powerpc/platforms/pseries/mobility.c | 59 ++++++++++++++++++++++- 1 file changed, 57 insertions(+), 2 deletions(-)
arch/powerpc/platforms/pseries/mobility.c +57 -2
@@ -542,16 +542,71 @@ static void pseries_cancel_migration(u64 handle, int err)pr_err ("H_VASI_SIGNAL error: %ld\n" , hvrc); } +static int pseries_suspend (u64 handle) +{ +const unsigned int max_attempts =5 ; +unsigned int retry_interval_ms =1 ; +unsigned int attempt =1 ; +int ret; + +while (true) { +atomic_t counter =ATOMIC_INIT (0 ); +unsigned long vasi_state; +int vasi_err; + + ret =stop_machine (do_join, &counter, cpu_online_mask); +if (ret ==0 ) +break ; +/* +* Encountered an error. If the VASI stream is still +* in Suspending state, it's likely a transient +* condition related to some device in the partition +* and we can retry in the hope that the cause has +* cleared after some delay. +* +* A better design would allow drivers etc to prepare +* for the suspend and avoid conditions which prevent +* the suspend from succeeding. For now, we have this +* mitigation. +*/ +pr_notice ("Partition suspend attempt %u of %u error: %d\n" , + attempt, max_attempts, ret); + +if (attempt == max_attempts) +break ; + + vasi_err =poll_vasi_state (handle, &vasi_state); +if (vasi_err ==0 ) { +if (vasi_state !=H_VASI_SUSPENDING ) { +pr_notice ("VASI state %lu after failed suspend\n" , + vasi_state); +break ; + } + }else if (vasi_err != -EOPNOTSUPP ) { +pr_err ("VASI state poll error: %d" , vasi_err); +break ; + } + +pr_notice ("Will retry partition suspend after %u ms\n" , + retry_interval_ms); + +msleep (retry_interval_ms); + retry_interval_ms *=10 ; + attempt++; + } + +return ret; +} +static int pseries_migrate_partition (u64 handle) { -atomic_t counter =ATOMIC_INIT (0 );int ret; ret =wait_for_vasi_session_suspending (handle);if (ret)return ret; - ret =stop_machine (do_join, &counter, cpu_online_mask); + ret =pseries_suspend (handle);if (ret ==0 )post_mobility_fixup ();else
Signature · 1 line
2.28.0
PATCH v2 15/28 powerpc/rtas: dispatch partition migration requests to pseries
In reply to Nathan Lynchsys_rtas() cannot call ibm,suspend-me directly in the same way it handles other inputs. Instead it must dispatch the request to code that can first perform the H_JOIN sequence before any call to ibm,suspend-me can succeed. Over time kernel/rtas.c has accreted a fair amount of platform-specific code to implement this.
Since a different, more robust implementation of the suspend sequence is now in the pseries platform code, we want to dispatch the request there.
Note that invoking ibm,suspend-me via the RTAS syscall is all but deprecated; this change preserves ABI compatibility for old programs while providing to them the benefit of the new partition suspend implementation. This is a behavior change in that the kernel performs the device tree update and firmware activation before returning, but experimentation indicates this is tolerated fine by legacy user space.
--- arch/powerpc/include/asm/rtas.h | 5 +++++ arch/powerpc/kernel/rtas.c | 2 +- arch/powerpc/platforms/pseries/mobility.c | 5 +++++ 3 files changed, 11 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/rtas.h +5 -0
@@ -279,8 +279,13 @@ extern time64_t last_rtas_event;extern int clobbering_unread_rtas_event (void );extern int pseries_devicetree_update (s32 scope);extern void post_mobility_fixup (void ); +int rtas_syscall_dispatch_ibm_suspend_me (u64 handle);#else static inline int clobbering_unread_rtas_event (void ) {return 0 ; } +static inline int rtas_syscall_dispatch_ibm_suspend_me (u64 handle) +{ +return -EINVAL ; +}#endif #ifdef CONFIG_PPC_RTAS_DAEMON
arch/powerpc/kernel/rtas.c +1 -1
@@ -1272,7 +1272,7 @@ SYSCALL_DEFINE1(rtas, struct rtas_args __user *, uargs)int rc =0 ;u64 handle = ((u64 )be32_to_cpu (args.args[0 ]) <<32 ) |be32_to_cpu (args.args[1 ]); - rc =rtas_ibm_suspend_me_unsafe (handle); + rc =rtas_syscall_dispatch_ibm_suspend_me (handle);if (rc == -EAGAIN ) args.rets[0 ] =cpu_to_be32 (RTAS_NOT_SUSPENDABLE );else if (rc == -EIO )
arch/powerpc/platforms/pseries/mobility.c +5 -0
@@ -615,6 +615,11 @@ static int pseries_migrate_partition(u64 handle)return ret; } +int rtas_syscall_dispatch_ibm_suspend_me (u64 handle) +{ +return pseries_migrate_partition (handle); +} +static ssize_t migration_store (struct class *class,struct class_attribute *attr,const char *buf,size_t count)
Signature · 1 line
2.28.0
PATCH v2 16/28 powerpc/rtas: remove rtas_ibm_suspend_me_unsafe()
In reply to Nathan Lynchrtas_ibm_suspend_me_unsafe() is now unused; remove it and rtas_percpu_suspend_me() which becomes unused as a result.
--- arch/powerpc/include/asm/rtas.h | 1 - arch/powerpc/kernel/rtas.c | 67 +-------------------------------- 2 files changed, 1 insertion(+), 67 deletions(-)
arch/powerpc/include/asm/rtas.h +0 -1
@@ -258,7 +258,6 @@ extern int rtas_set_indicator_fast(int indicator, int index, int new_value);extern void rtas_progress (char *s,unsigned short hex);extern int rtas_suspend_cpu (struct rtas_suspend_me_data *data);extern int rtas_suspend_last_cpu (struct rtas_suspend_me_data *data); -int rtas_ibm_suspend_me_unsafe (u64 handle);int rtas_ibm_suspend_me (int *fw_status);struct rtc_time ;
arch/powerpc/kernel/rtas.c +1 -66
79 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v2 17/28 powerpc/pseries/hibernation: drop pseries_suspend_begin() from suspend ops
In reply to Nathan LynchThere are three ways pseries_suspend_begin() can be reached:
1. When "mem" is written to /sys/power/state:
kobj_attr_store()
-> state_store()
-> pm_suspend()
-> suspend_devices_and_enter()
-> pseries_suspend_begin()This never works because there is no way to supply a valid stream id using this interface, and H_VASI_STATE is called with a stream id of zero. So this call path is useless at best.
2. When a stream id is written to /sys/devices/system/power/hibernate. pseries_suspend_begin() is polled directly from store_hibernate() until the stream is in the "Suspending" state (i.e. the platform is ready for the OS to suspend execution):
dev_attr_store()
-> store_hibernate()
-> pseries_suspend_begin()
3. When a stream id is written to /sys/devices/system/power/hibernate (continued). After #2, pseries_suspend_begin() is called once again from the pm core:
dev_attr_store()
-> store_hibernate()
-> pm_suspend()
-> suspend_devices_and_enter()
-> pseries_suspend_begin()This is redundant because the VASI suspend state is already known to be Suspending.
The begin() callback of platform_suspend_ops is optional, so we can simply remove that assignment with no loss of function.
--- arch/powerpc/platforms/pseries/suspend.c | 1 - 1 file changed, 1 deletion(-)
arch/powerpc/platforms/pseries/suspend.c +0 -1
@@ -187,7 +187,6 @@ static struct bus_type suspend_subsys = {static const struct platform_suspend_ops pseries_suspend_ops = { .valid = suspend_valid_only_mem, - .begin = pseries_suspend_begin, .prepare_late = pseries_prepare_late, .enter = pseries_suspend_enter, };
Signature · 1 line
2.28.0
PATCH v2 18/28 powerpc/pseries/hibernation: pass stream id via function arguments
In reply to Nathan LynchThere is no need for the stream id to be a file-global variable; pass it from hibernate_store() to pseries_suspend_begin() for the H_VASI_STATE call.
--- arch/powerpc/platforms/pseries/suspend.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
arch/powerpc/platforms/pseries/suspend.c +3 -5
@@ -15,7 +15,6 @@#include <asm/topology.h> #include "../../kernel/cacheinfo.h" -static u64 stream_id;static struct device suspend_dev;static DECLARE_COMPLETION (suspend_work);static struct rtas_suspend_me_data suspend_data;
@@ -29,7 +28,7 @@ static atomic_t suspending;* Return value: * 0 on success / other on failure **/ -static int pseries_suspend_begin (suspend_state_t state) +static int pseries_suspend_begin (u64 stream_id) {long vasi_state, rc;unsigned long retbuf[PLPAR_HCALL_BUFSIZE ];
@@ -132,6 +131,7 @@ static ssize_t store_hibernate(struct device *dev,struct device_attribute *attr, const char *buf, size_t count) { + u64 stream_id;int rc;if (!capable (CAP_SYS_ADMIN ))
@@ -140,7 +140,7 @@ static ssize_t store_hibernate(struct device *dev, stream_id =simple_strtoul (buf,NULL ,16 );do { - rc =pseries_suspend_begin (PM_SUSPEND_MEM ); + rc =pseries_suspend_begin (stream_id);if (rc == -EAGAIN )ssleep (1 ); }while (rc == -EAGAIN );
@@ -148,8 +148,6 @@ static ssize_t store_hibernate(struct device *dev,if (!rc) rc =pm_suspend (PM_SUSPEND_MEM ); - stream_id =0 ; -if (!rc) rc = count;
Signature · 1 line
2.28.0
PATCH v2 19/28 powerpc/pseries/hibernation: remove pseries_suspend_cpu()
In reply to Nathan LynchSince commit 48f6e7f6d948 ("powerpc/pseries: remove cede offline state for CPUs"), ppc_md.suspend_disable_cpu() is no longer used and all CPUs (save one) are placed into true offline state as opposed to H_JOIN. So pseries_suspend_cpu() is effectively unused; remove it.
--- arch/powerpc/platforms/pseries/suspend.c | 15 --------------- 1 file changed, 15 deletions(-)
arch/powerpc/platforms/pseries/suspend.c +0 -15
@@ -48,20 +48,6 @@ static int pseries_suspend_begin(u64 stream_id) vasi_state);return -EIO ; } - -return 0 ; -} - -/** -* pseries_suspend_cpu - Suspend a single CPU -* -* Makes the H_JOIN call to suspend the CPU -* -**/ -static int pseries_suspend_cpu (void ) -{ -if (atomic_read (&suspending)) -return rtas_suspend_cpu (&suspend_data);return 0 ; }
@@ -235,7 +221,6 @@ static int __init pseries_suspend_init(void)if ((rc =pseries_suspend_sysfs_register (&suspend_dev)))return rc; - ppc_md.suspend_disable_cpu = pseries_suspend_cpu; ppc_md.suspend_enable_irqs = pseries_suspend_enable_irqs;suspend_set_ops (&pseries_suspend_ops);return 0 ;
Signature · 1 line
2.28.0
PATCH v2 20/28 powerpc/machdep: remove suspend_disable_cpu()
In reply to Nathan LynchThere are no users left of the suspend_disable_cpu() callback, remove it.
--- arch/powerpc/include/asm/machdep.h | 1 - 1 file changed, 1 deletion(-)
arch/powerpc/include/asm/machdep.h +0 -1
@@ -207,7 +207,6 @@ struct machdep_calls {void (*suspend_disable_irqs)(void );void (*suspend_enable_irqs)(void );#endif -int (*suspend_disable_cpu)(void );#ifdef CONFIG_ARCH_CPU_PROBE_RELEASE ssize_t (*cpu_probe)(const char *,size_t );
Signature · 1 line
2.28.0
PATCH v2 21/28 powerpc/rtas: remove rtas_suspend_cpu()
In reply to Nathan Lynchrtas_suspend_cpu() no longer has users; remove it and __rtas_suspend_cpu() which now becomes unused as well.
--- arch/powerpc/include/asm/rtas.h | 1 - arch/powerpc/kernel/rtas.c | 52 --------------------------------- 2 files changed, 53 deletions(-)
arch/powerpc/include/asm/rtas.h +0 -1
@@ -256,7 +256,6 @@ extern bool rtas_indicator_present(int token, int *maxindex);extern int rtas_set_indicator (int indicator,int index,int new_value);extern int rtas_set_indicator_fast (int indicator,int index,int new_value);extern void rtas_progress (char *s,unsigned short hex); -extern int rtas_suspend_cpu (struct rtas_suspend_me_data *data);extern int rtas_suspend_last_cpu (struct rtas_suspend_me_data *data);int rtas_ibm_suspend_me (int *fw_status);
arch/powerpc/kernel/rtas.c +0 -52
58 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v2 22/28 powerpc/pseries/hibernation: switch to rtas_ibm_suspend_me()
In reply to Nathan Lynchrtas_suspend_last_cpu() and related code perform a lot of work that isn't relevant to the hibernation workflow. All other CPUs are offline when called so there is no need to place them in H_JOIN or prod them on resume, nor is there need for retries or operations on shared state.
Call the rtas_ibm_suspend_me() wrapper function directly from pseries_suspend_enter() instead of using rtas_suspend_last_cpu().
--- arch/powerpc/platforms/pseries/suspend.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
arch/powerpc/platforms/pseries/suspend.c +1 -5
@@ -76,11 +76,7 @@ static void pseries_suspend_enable_irqs(void)**/ static int pseries_suspend_enter (suspend_state_t state) { -int rc =rtas_suspend_last_cpu (&suspend_data); - -atomic_set (&suspending,0 ); -atomic_set (&suspend_data.done,1 ); -return rc; +return rtas_ibm_suspend_me (NULL ); }/**
Signature · 1 line
2.28.0
PATCH v2 23/28 powerpc/rtas: remove unused rtas_suspend_last_cpu()
In reply to Nathan Lynchrtas_suspend_last_cpu() is now unused, remove it and __rtas_suspend_last_cpu() which also becomes unused.
--- arch/powerpc/include/asm/rtas.h | 1 - arch/powerpc/kernel/rtas.c | 43 --------------------------------- 2 files changed, 44 deletions(-)
arch/powerpc/include/asm/rtas.h +0 -1
@@ -256,7 +256,6 @@ extern bool rtas_indicator_present(int token, int *maxindex);extern int rtas_set_indicator (int indicator,int index,int new_value);extern int rtas_set_indicator_fast (int indicator,int index,int new_value);extern void rtas_progress (char *s,unsigned short hex); -extern int rtas_suspend_last_cpu (struct rtas_suspend_me_data *data);int rtas_ibm_suspend_me (int *fw_status);struct rtc_time ;
arch/powerpc/kernel/rtas.c +0 -43
49 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v2 24/28 powerpc/pseries/hibernation: remove redundant cacheinfo update
In reply to Nathan LynchPartitions with cache nodes in the device tree can encounter the following warning on resume:
CPU 0 already accounted in PowerPC,POWER9@0(Data)
WARNING: CPU: 0 PID: 3177 at arch/powerpc/kernel/cacheinfo.c:197 cacheinfo_cpu_online+0x640/0x820
These calls to cacheinfo_cpu_offline/online have been redundant since commit e610a466d16a ("powerpc/pseries/mobility: rebuild cacheinfo hierarchy post-migration").
--- arch/powerpc/platforms/pseries/suspend.c | 3 --- 1 file changed, 3 deletions(-)
arch/powerpc/platforms/pseries/suspend.c +0 -3
@@ -13,7 +13,6 @@#include <asm/mmu.h> #include <asm/rtas.h> #include <asm/topology.h> -#include "../../kernel/cacheinfo.h" static struct device suspend_dev;static DECLARE_COMPLETION (suspend_work);
@@ -63,9 +62,7 @@ static void pseries_suspend_enable_irqs(void)* Update configuration which can be modified based on device tree * changes during resume. */ -cacheinfo_cpu_offline (smp_processor_id ());post_mobility_fixup (); -cacheinfo_cpu_online (smp_processor_id ()); }/**
Signature · 1 line
2.28.0
PATCH v2 25/28 powerpc/pseries/hibernation: perform post-suspend fixups later
In reply to Nathan LynchThe pseries hibernate code calls post_mobility_fixup() which is sort of a dumping ground of fixups that need to run after resuming from suspend regardless of whether suspend was a hibernation or a migration. Calling post_mobility_fixup() from pseries_suspend_enable_irqs() runs this code early in resume with devices suspended and only one CPU up, while the much more commonly used migration case runs these fixups in a more typical process context.
Call post_mobility_fixup() after the suspend core returns a success status to the hibernate sysfs store method and remove pseries_suspend_enable_irqs().
--- arch/powerpc/platforms/pseries/suspend.c | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-)
arch/powerpc/platforms/pseries/suspend.c +4 -17
@@ -50,21 +50,6 @@ static int pseries_suspend_begin(u64 stream_id)return 0 ; } -/** -* pseries_suspend_enable_irqs -* -* Post suspend configuration updates -* -**/ -static void pseries_suspend_enable_irqs (void ) -{ -/* -* Update configuration which can be modified based on device tree -* changes during resume. -*/ -post_mobility_fixup (); -} -/** * pseries_suspend_enter - Final phase of hibernation *
@@ -127,8 +112,11 @@ static ssize_t store_hibernate(struct device *dev,if (!rc) rc =pm_suspend (PM_SUSPEND_MEM ); -if (!rc) +if (!rc) { rc = count; +post_mobility_fixup (); + } +return rc; }
@@ -214,7 +202,6 @@ static int __init pseries_suspend_init(void)if ((rc =pseries_suspend_sysfs_register (&suspend_dev)))return rc; - ppc_md.suspend_enable_irqs = pseries_suspend_enable_irqs;suspend_set_ops (&pseries_suspend_ops);return 0 ; }
Signature · 1 line
2.28.0
PATCH v2 26/28 powerpc/pseries/hibernation: remove prepare_late() callback
In reply to Nathan LynchThe pseries hibernate code no longer calls into the original join/suspend code in kernel/rtas.c, so pseries_prepare_late() and related code don't accomplish anything now.
--- arch/powerpc/platforms/pseries/suspend.c | 25 ------------------------ 1 file changed, 25 deletions(-)
arch/powerpc/platforms/pseries/suspend.c +0 -25
@@ -15,9 +15,6 @@#include <asm/topology.h> static struct device suspend_dev; -static DECLARE_COMPLETION (suspend_work); -static struct rtas_suspend_me_data suspend_data; -static atomic_t suspending;/** * pseries_suspend_begin - First phase of hibernation
@@ -61,23 +58,6 @@ static int pseries_suspend_enter(suspend_state_t state)return rtas_ibm_suspend_me (NULL ); } -/** -* pseries_prepare_late - Prepare to suspend all other CPUs -* -* Return value: -* 0 on success / other on failure -**/ -static int pseries_prepare_late (void ) -{ -atomic_set (&suspending,1 ); -atomic_set (&suspend_data.working,0 ); -atomic_set (&suspend_data.done,0 ); -atomic_set (&suspend_data.error,0 ); - suspend_data.complete = &suspend_work; -reinit_completion (&suspend_work); -return 0 ; -} -/** * store_hibernate - Initiate partition hibernation * @dev: subsys root device
@@ -152,7 +132,6 @@ static struct bus_type suspend_subsys = {static const struct platform_suspend_ops pseries_suspend_ops = { .valid = suspend_valid_only_mem, - .prepare_late = pseries_prepare_late, .enter = pseries_suspend_enter, };
@@ -195,10 +174,6 @@ static int __init pseries_suspend_init(void)if (!firmware_has_feature (FW_FEATURE_LPAR ))return 0 ; - suspend_data.token =rtas_token ("ibm,suspend-me" ); -if (suspend_data.token ==RTAS_UNKNOWN_SERVICE ) -return 0 ; -if ((rc =pseries_suspend_sysfs_register (&suspend_dev)))return rc;
Signature · 1 line
2.28.0
PATCH v2 27/28 powerpc/rtas: remove unused rtas_suspend_me_data
In reply to Nathan LynchAll code which used this type has been removed.
--- arch/powerpc/include/asm/rtas-types.h | 8 -------- 1 file changed, 8 deletions(-)
arch/powerpc/include/asm/rtas-types.h +0 -8
@@ -23,14 +23,6 @@ struct rtas_t {struct device_node *dev;/* virtual address pointer */ }; -struct rtas_suspend_me_data { -atomic_t working;/* number of cpus accessing this struct */ -atomic_t done; -int token;/* ibm,suspend-me */ -atomic_t error; -struct completion *complete;/* wait on this until working == 0 */ -}; -struct rtas_error_log {/* Byte 0 */ u8 byte0;/* Architectural version */
Signature · 1 line
2.28.0
PATCH v2 28/28 powerpc/pseries/mobility: refactor node lookup during DT update
In reply to Nathan LynchIn pseries_devicetree_update(), with each call to ibm,update-nodes the partition firmware communicates the node to be deleted or updated by placing its phandle in the work buffer. Each of delete_dt_node(), update_dt_node(), and add_dt_node() have duplicate lookups using the phandle value and corresponding refcount management.
Move the lookup and of_node_put() into pseries_devicetree_update(), and emit a warning on any failed lookups.
--- arch/powerpc/platforms/pseries/mobility.c | 49 ++++++++--------------- 1 file changed, 17 insertions(+), 32 deletions(-)
arch/powerpc/platforms/pseries/mobility.c +17 -32
@@ -61,18 +61,10 @@ static int mobility_rtas_call(int token, char *buf, s32 scope)return rc; } -static int delete_dt_node (__be32 phandle) +static int delete_dt_node (struct device_node *dn) { -struct device_node *dn; - - dn =of_find_node_by_phandle (be32_to_cpu (phandle)); -if (!dn) -return -ENOENT ; -pr_debug ("removing node %pOFfp\n" , dn); -dlpar_detach_node (dn); -of_node_put (dn);return 0 ; }
@@ -137,10 +129,9 @@ static int update_dt_property(struct device_node *dn, struct property **prop,return 0 ; } -static int update_dt_node (__be32 phandle,s32 scope) +static int update_dt_node (struct device_node *dn,s32 scope) {struct update_props_workarea *upwa; -struct device_node *dn;struct property *prop =NULL ;int i, rc, rtas_rc;char *prop_data;
@@ -157,14 +148,8 @@ static int update_dt_node(__be32 phandle, s32 scope)if (!rtas_buf)return -ENOMEM ; - dn =of_find_node_by_phandle (be32_to_cpu (phandle)); -if (!dn) { -kfree (rtas_buf); -return -ENOENT ; - } - upwa = (struct update_props_workarea *)&rtas_buf[0 ]; - upwa->phandle = phandle; + upwa->phandle =cpu_to_be32 (dn->phandle); do { rtas_rc = mobility_rtas_call(update_properties_token, rtas_buf,
@@ -224,26 +209,18 @@ static int update_dt_node(__be32 phandle, s32 scope)cond_resched (); }while (rtas_rc ==1 ); -of_node_put (dn);kfree (rtas_buf);return 0 ; } -static int add_dt_node (__be32 parent_phandle,__be32 drc_index) +static int add_dt_node (struct device_node *parent_dn,__be32 drc_index) {struct device_node *dn; -struct device_node *parent_dn;int rc; - parent_dn =of_find_node_by_phandle (be32_to_cpu (parent_phandle)); -if (!parent_dn) -return -ENOENT ; - dn =dlpar_configure_connector (drc_index, parent_dn); -if (!dn) { -of_node_put (parent_dn); +if (!dn)return -ENOENT ; - } rc =dlpar_attach_node (dn, parent_dn);if (rc)
@@ -251,7 +228,6 @@ static int add_dt_node(__be32 parent_phandle, __be32 drc_index)pr_debug ("added node %pOFfp\n" , dn); -of_node_put (parent_dn);return rc; }
@@ -284,22 +260,31 @@ int pseries_devicetree_update(s32 scope) data++;for (i =0 ; i < node_count; i++) { +struct device_node *np;__be32 phandle = *data++;__be32 drc_index; + np =of_find_node_by_phandle (be32_to_cpu (phandle)); +if (!np) { +pr_warn ("Failed lookup: phandle 0x%x for action 0x%x\n" , +be32_to_cpu (phandle), action); +continue ; + } +switch (action) {case DELETE_DT_NODE : -delete_dt_node (phandle); +delete_dt_node (np);break ;case UPDATE_DT_NODE : -update_dt_node (phandle, scope); +update_dt_node (np, scope);break ;case ADD_DT_NODE : drc_index = *data++; -add_dt_node (phandle, drc_index); +add_dt_node (np, drc_index);break ; } +of_node_put (np);cond_resched (); } }
Signature · 1 line
2.28.0
Michael Ellerman
AppliedQuoted 10 lines · Nathan Lynch
This series aims to improve the pseries-specific partition migration and hibernation implementation, part of which has been living in
kernel/rtas.c. Most of that code is eliminated or moved to platforms/pseries, and the following major functional changes are made:- Use
stop_machine()instead ofon_each_cpu()to avoid deadlock in the join/suspend sequence.[...]
Applied to powerpc/next.
[01/28] powerpc/rtas: prevent suspend-related sys_rtas use on LE
https://git.kernel.org/powerpc/c/de0f7349a0dd072e54b5fc04c305907b22d28a5f [02/28] powerpc/rtas: complete ibm,suspend-me status codes https://git.kernel.org/powerpc/c/970e453ea4ecdd7a16a46c229294547148d1c7b6 [03/28] powerpc/rtas: rtas_ibm_suspend_me -> rtas_ibm_suspend_me_unsafe https://git.kernel.org/powerpc/c/7049b288ea8c95f270ec8fe643e3c3187938d5af [04/28] powerpc/rtas: add rtas_ibm_suspend_me() https://git.kernel.org/powerpc/c/701ba68342412ae9be99a7c7f3badebf95271403 [05/28] powerpc/rtas: add rtas_activate_firmware() https://git.kernel.org/powerpc/c/5f485a66f4d0693a535e4ab38ffc3538716d2c2b [06/28] powerpc/hvcall: add token and codes for H_VASI_SIGNAL https://git.kernel.org/powerpc/c/9bae89f528c041f3117f0a6c21878dda5a55af60 [07/28] powerpc/pseries/mobility: don't error on absence of ibm, update-nodes https://git.kernel.org/powerpc/c/b06a6717873560e9dd1c07357781fc2b27545701 [08/28] powerpc/pseries/mobility: add missing break to default case https://git.kernel.org/powerpc/c/aa5e5c9b556a2e5f68a915e4b5dfa5c6bda47c64 [09/28] powerpc/pseries/mobility: error message improvements https://git.kernel.org/powerpc/c/2d5be6f16c4ba5c27d06704976daf55f3236a236 [10/28] powerpc/pseries/mobility: use rtas_activate_firmware() on resume https://git.kernel.org/powerpc/c/c3ae9781d5a64093f161e6cc5dfefb0773106ca9 [11/28] powerpc/pseries/mobility: extract VASI session polling logic https://git.kernel.org/powerpc/c/d9213319b84ee8393475c38361c84151d5c33415 [12/28] powerpc/pseries/mobility: use stop_machine for join/suspend https://git.kernel.org/powerpc/c/9327dc0aeef36a3cbb9d94f79b79cc4f91ff8a41 [13/28] powerpc/pseries/mobility: signal suspend cancellation to platform https://git.kernel.org/powerpc/c/37cddc7d6cf4568a7fb69aeff6f26e4c8a3bc0f7 [14/28] powerpc/pseries/mobility: retry partition suspend after error https://git.kernel.org/powerpc/c/aeca35b9a52b0e0d019a5244fbaab699f753b443 [15/28] powerpc/rtas: dispatch partition migration requests to pseries https://git.kernel.org/powerpc/c/4d756894ba75f1afe7945ccafe9afebff50484b6 [16/28] powerpc/rtas: remove rtas_ibm_suspend_me_unsafe() https://git.kernel.org/powerpc/c/5f6665e400569de479733677e77862542aebb6cc [17/28] powerpc/pseries/hibernation: drop pseries_suspend_begin() from suspend ops https://git.kernel.org/powerpc/c/52719fce3f4c7a8ac9eaa191e8d75a697f9fbcbc [18/28] powerpc/pseries/hibernation: pass stream id via function arguments https://git.kernel.org/powerpc/c/a10a5a17f4ac4f84fcc26162d43b53e2a4e1009a [19/28] powerpc/pseries/hibernation: remove pseries_suspend_cpu() https://git.kernel.org/powerpc/c/ed22bb8d39fa7f3980afc6e16d2a891847367d33 [20/28] powerpc/machdep: remove suspend_disable_cpu() https://git.kernel.org/powerpc/c/796f9247b4fa9bec320d6b47ffde2ecf86cc71c0 [21/28] powerpc/rtas: remove rtas_suspend_cpu() https://git.kernel.org/powerpc/c/395b2c090907975c627902ba8fda0bdb04c7cad3 [22/28] powerpc/pseries/hibernation: switch to rtas_ibm_suspend_me() https://git.kernel.org/powerpc/c/366fb13bf13b029c4d43bf19382f7aea69bfa4b7 [23/28] powerpc/rtas: remove unused rtas_suspend_last_cpu() https://git.kernel.org/powerpc/c/1b2488176ea56e299d2b084772daeb5ecbfc16d1 [24/28] powerpc/pseries/hibernation: remove redundant cacheinfo update https://git.kernel.org/powerpc/c/b866459489fe8ef0e92cde3cbd6bbb1af6c4e99b [25/28] powerpc/pseries/hibernation: perform post-suspend fixups later https://git.kernel.org/powerpc/c/fa53bcdb7413e7c40170106781f6b5bb9d74db84 [26/28] powerpc/pseries/hibernation: remove prepare_late() callback https://git.kernel.org/powerpc/c/d102f8312e1ea5e8bf84fceebf99186f22d16fc6 [27/28] powerpc/rtas: remove unused rtas_suspend_me_data https://git.kernel.org/powerpc/c/87b57ea7e109520d3c6dfb01671a0cb134d3ccff [28/28] powerpc/pseries/mobility: refactor node lookup during DT update https://git.kernel.org/powerpc/c/2efd7f6eb9b7107e469837d8452e750d7d080a5d
cheers