s390x/vfio: VFIO-AP interrupt control interception
RFC PATCH v3 1/1 s390x/vfio: VFIO-AP interrupt control interception
This QEMU patch is to be used in conjonction with the following patch series in LINUX.
vfio: ap: ioctl definitions for AP Queue Interrupt Control
The LINUX patch series handle all the interception in KVM and we do not need to take care of the interception in QEMU.
The goal of this patch is to define the apqi CPU feature which will be proposed by KVM when the AP Interrupt Control facility is to be available in the guest.
This feature must be turned on by QEMU to intercept AP instructions on the KVM guest.
The QEMU command line to turn this feature on looks something like:
qemu-system-s390x ... -cpu xxx,apqi=on ...
Pierre Morel (1):
s390x/cpumodel: Set up CPU model for AQIC interception
target/s390x/cpu_features.c | 1 + target/s390x/cpu_features_def.h | 1 + target/s390x/cpu_models.c | 1 + target/s390x/gen-features.c | 1 + 4 files changed, 4 insertions(+)
Signature · 1 line
2.7.4
PATCH v3 1/1 s390x/cpumodel: Set up CPU model for AQIC interception
A new CPU model facilities is introduced to support AP devices interruption interception for a KVM guest.
"APQI" for "AP-Queue Interruption" facility
The S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL, CPU facility indicates whether the PQAP instruction with the AQIC command is available to the guest.
This feature will be enabled only if the AP instructions are available on the linux host and AQIC facility is installed on the host.
This feature must be turned on from userspace to intercept AP instructions on the KVM guest. The QEMU command line to turn this feature on looks something like this:
qemu-system-s390x ... -cpu xxx,apqi=on ...
--- target/s390x/cpu_features.c | 1 + target/s390x/cpu_features_def.h | 1 + target/s390x/cpu_models.c | 1 + target/s390x/gen-features.c | 1 + 4 files changed, 4 insertions(+)
target/s390x/cpu_features.c +1 -0
@@ -84,6 +84,7 @@ static const S390FeatDef s390_features[] = {FEAT_INIT ("sema" ,S390_FEAT_TYPE_STFL ,59 ,"Semaphore-assist facility" ),FEAT_INIT ("tsi" ,S390_FEAT_TYPE_STFL ,60 ,"Time-slice Instrumentation facility" ),FEAT_INIT ("ri" ,S390_FEAT_TYPE_STFL ,64 ,"CPU runtime-instrumentation facility" ), +FEAT_INIT ("apqi" ,S390_FEAT_TYPE_STFL ,65 ,"AP-Queue interruption facility" ),FEAT_INIT ("zpci" ,S390_FEAT_TYPE_STFL ,69 ,"z/PCI facility" ),FEAT_INIT ("aen" ,S390_FEAT_TYPE_STFL ,71 ,"General-purpose-adapter-event-notification facility" ), FEAT_INIT("ais", S390_FEAT_TYPE_STFL, 72, "General-purpose-adapter-interruption-suppression facility"),
target/s390x/cpu_features_def.h +1 -0
@@ -72,6 +72,7 @@ typedef enum {S390_FEAT_SEMAPHORE_ASSIST ,S390_FEAT_TIME_SLICE_INSTRUMENTATION ,S390_FEAT_RUNTIME_INSTRUMENTATION , +S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL ,S390_FEAT_ZPCI ,S390_FEAT_ADAPTER_EVENT_NOTIFICATION , S390_FEAT_ADAPTER_INT_SUPPRESSION,
target/s390x/cpu_models.c +1 -0
@@ -788,6 +788,7 @@ static void check_consistency(const S390CPUModel *model) { S390_FEAT_SIE_KSS, S390_FEAT_SIE_F2 }, { S390_FEAT_AP_QUERY_CONFIG_INFO, S390_FEAT_AP }, { S390_FEAT_AP_FACILITIES_TEST, S390_FEAT_AP }, + { S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL, S390_FEAT_AP }, };int i;
target/s390x/gen-features.c +1 -0
@@ -448,6 +448,7 @@ static uint16_t full_GEN12_GA1[] = {S390_FEAT_EDAT_2 ,S390_FEAT_SIDE_EFFECT_ACCESS_ESOP2 ,S390_FEAT_AP_QUERY_CONFIG_INFO , +S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL ,S390_FEAT_AP_FACILITIES_TEST , S390_FEAT_AP, };
Signature · 1 line
2.7.4
Re: [Qemu-devel] [PATCH v3] s390x/cpumodel: Set up CPU model for AQIC interception
Quoted 12 lines · Pierre Morel
A new CPU model facilities is introduced to support AP devices interruption interception for a KVM guest.
"APQI" for "AP-Queue Interruption" facility
The
S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL, CPU facility indicates whether the PQAP instruction with the AQIC command is available to the guest.
This feature will be enabled only if the AP instructions are available on the linux host and AQIC facility is installed on the host.
Quoted 5 lines
This feature must be turned on from userspace to intercept AP instructions on the KVM guest. The QEMU command line to turn this feature on looks something like this:
qemu-system-s390x ... -cpu xxx,apqi=on ...
Hi,
I suddenly have a very big doubt on this QEMU patch.
Older Linux advertise PQAP/AQIC feature (65)
Older Qemu mask it for the guest
Now if we set aqic = on in the new QEMU,
QEMU advertises feature (65)
The guest gets an OPERATION EXCEPTION even we advertised the feature 65.
This seems wrong to me, we should verify that the vfio_driver is loaded in the host before starting the guest with something like:
kvm_vm_check_attr(kvm_state, KVM_S390_VM_ENABLE_AQIC)
Don't you think?
Regards,
Pierre
Quoted 61 lines
Signed-off-by Pierre Morel <[email protected]>Reviewed-by Tony Krowiak <[email protected]>Reviewed-by Christian Borntraeger <[email protected]>Reviewed-by Halil Pasic <[email protected]>--- target/s390x/cpu_features.c | 1 + target/s390x/cpu_features_def.h | 1 + target/s390x/cpu_models.c | 1 + target/s390x/gen-features.c | 1 + 4 files changed, 4 insertions(+)target/s390x/cpu_features.c +1
-0@@ -84,6 +84,7 @@ static const S390FeatDef s390_features[] = {FEAT_INIT ("sema" ,S390_FEAT_TYPE_STFL ,59 ,"Semaphore-assist facility" ),FEAT_INIT ("tsi" ,S390_FEAT_TYPE_STFL ,60 ,"Time-slice Instrumentation facility" ),FEAT_INIT ("ri" ,S390_FEAT_TYPE_STFL ,64 ,"CPU runtime-instrumentation facility" ), +FEAT_INIT ("apqi" ,S390_FEAT_TYPE_STFL ,65 ,"AP-Queue interruption facility" ),FEAT_INIT ("zpci" ,S390_FEAT_TYPE_STFL ,69 ,"z/PCI facility" ),FEAT_INIT ("aen" ,S390_FEAT_TYPE_STFL ,71 ,"General-purpose-adapter-event-notification facility" ), FEAT_INIT("ais", S390_FEAT_TYPE_STFL, 72, "General-purpose-adapter-interruption-suppression facility"),target/s390x/cpu_features_def.h +1
-0@@ -72,6 +72,7 @@ typedef enum {S390_FEAT_SEMAPHORE_ASSIST ,S390_FEAT_TIME_SLICE_INSTRUMENTATION ,S390_FEAT_RUNTIME_INSTRUMENTATION , +S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL ,S390_FEAT_ZPCI ,S390_FEAT_ADAPTER_EVENT_NOTIFICATION , S390_FEAT_ADAPTER_INT_SUPPRESSION,target/s390x/cpu_models.c +1
-0@@ -788,6 +788,7 @@ static void check_consistency(const S390CPUModel *model) { S390_FEAT_SIE_KSS, S390_FEAT_SIE_F2 }, { S390_FEAT_AP_QUERY_CONFIG_INFO, S390_FEAT_AP }, { S390_FEAT_AP_FACILITIES_TEST, S390_FEAT_AP }, + { S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL, S390_FEAT_AP }, };int i;target/s390x/gen-features.c +1
-0@@ -448,6 +448,7 @@ static uint16_t full_GEN12_GA1[] = {S390_FEAT_EDAT_2 ,S390_FEAT_SIDE_EFFECT_ACCESS_ESOP2 ,S390_FEAT_AP_QUERY_CONFIG_INFO , +S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL ,S390_FEAT_AP_FACILITIES_TEST , S390_FEAT_AP, };
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [Qemu-devel] [PATCH v3] s390x/cpumodel: Set up CPU model for AQIC interception
Quoted 21 lines · Pierre Morel
Quoted 12 lines · Pierre Morel
A new CPU model facilities is introduced to support AP devices interruption interception for a KVM guest.
"APQI" for "AP-Queue Interruption" facility
The
S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL, CPU facility indicates whether the PQAP instruction with the AQIC command is available to the guest.
This feature will be enabled only if the AP instructions are available on the linux host and AQIC facility is installed on the host.Quoted 5 lines
This feature must be turned on from userspace to intercept AP instructions on the KVM guest. The QEMU command line to turn this feature on looks something like this:
qemu-system-s390x ... -cpu xxx,apqi=on ...
Quoted 5 lines
Hi,
I suddenly have a very big doubt on this QEMU patch.
Older Linux advertise PQAP/AQIC feature (65)
Which Linuxes advertise PQAP/AQIC (stfle.65)?
This was never advertised by any previous Linux as far as I can tell.
Re: [Qemu-devel] [PATCH v3] s390x/cpumodel: Set up CPU model for AQIC interception
Quoted 24 lines · Christian Borntraeger
Quoted 21 lines · Pierre Morel
Quoted 17 lines · Pierre Morel
A new CPU model facilities is introduced to support AP devices interruption interception for a KVM guest.
"APQI" for "AP-Queue Interruption" facility
The
S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL, CPU facility indicates whether the PQAP instruction with the AQIC command is available to the guest.
This feature will be enabled only if the AP instructions are available on the linux host and AQIC facility is installed on the host.This feature must be turned on from userspace to intercept AP instructions on the KVM guest. The QEMU command line to turn this feature on looks something like this:
qemu-system-s390x ... -cpu xxx,apqi=on ...
Quoted 9 lines
Quoted 5 lines
Hi,
I suddenly have a very big doubt on this QEMU patch.
Older Linux advertise PQAP/AQIC feature (65)
Which Linuxes advertise PQAP/AQIC (stfle.65)?
This was never advertised by any previous Linux as far as I can tell.
OK, very sorry, you are right, I checked on a different machine and it is not. I must have left something in my system.
Sorry for the panic.
I will look at what made me experience this.
Regards,
Pierre
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [Qemu-devel] [PATCH v3] s390x/cpumodel: Set up CPU model for AQIC interception
In reply to Pierre MorelQuoted 2 lines · Pierre Morel
A new CPU model facilities is introduced to support AP devices interruption interception for a KVM guest.
This sentence is confusing. At first I thought you would introduce a new _fake_ facility.
"Let's add support for the AP-Queue interruption facility to the CPU model".
Quoted 69 lines
"APQI" for "AP-Queue Interruption" facility
The
S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL, CPU facility indicates whether the PQAP instruction with the AQIC command is available to the guest.
This feature will be enabled only if the AP instructions are available on the linux host and AQIC facility is installed on the host.This feature must be turned on from userspace to intercept AP instructions on the KVM guest. The QEMU command line to turn this feature on looks something like this:
qemu-system-s390x ... -cpu xxx,apqi=on ...Signed-off-by Pierre Morel <[email protected]>Reviewed-by Tony Krowiak <[email protected]>Reviewed-by Christian Borntraeger <[email protected]>Reviewed-by Halil Pasic <[email protected]>--- target/s390x/cpu_features.c | 1 + target/s390x/cpu_features_def.h | 1 + target/s390x/cpu_models.c | 1 + target/s390x/gen-features.c | 1 + 4 files changed, 4 insertions(+)target/s390x/cpu_features.c +1
-0@@ -84,6 +84,7 @@ static const S390FeatDef s390_features[] = {FEAT_INIT ("sema" ,S390_FEAT_TYPE_STFL ,59 ,"Semaphore-assist facility" ),FEAT_INIT ("tsi" ,S390_FEAT_TYPE_STFL ,60 ,"Time-slice Instrumentation facility" ),FEAT_INIT ("ri" ,S390_FEAT_TYPE_STFL ,64 ,"CPU runtime-instrumentation facility" ), +FEAT_INIT ("apqi" ,S390_FEAT_TYPE_STFL ,65 ,"AP-Queue interruption facility" ),FEAT_INIT ("zpci" ,S390_FEAT_TYPE_STFL ,69 ,"z/PCI facility" ),FEAT_INIT ("aen" ,S390_FEAT_TYPE_STFL ,71 ,"General-purpose-adapter-event-notification facility" ), FEAT_INIT("ais", S390_FEAT_TYPE_STFL, 72, "General-purpose-adapter-interruption-suppression facility"),target/s390x/cpu_features_def.h +1
-0@@ -72,6 +72,7 @@ typedef enum {S390_FEAT_SEMAPHORE_ASSIST ,S390_FEAT_TIME_SLICE_INSTRUMENTATION ,S390_FEAT_RUNTIME_INSTRUMENTATION , +S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL ,S390_FEAT_ZPCI ,S390_FEAT_ADAPTER_EVENT_NOTIFICATION , S390_FEAT_ADAPTER_INT_SUPPRESSION,target/s390x/cpu_models.c +1
-0@@ -788,6 +788,7 @@ static void check_consistency(const S390CPUModel *model) { S390_FEAT_SIE_KSS, S390_FEAT_SIE_F2 }, { S390_FEAT_AP_QUERY_CONFIG_INFO, S390_FEAT_AP }, { S390_FEAT_AP_FACILITIES_TEST, S390_FEAT_AP }, + { S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL, S390_FEAT_AP }, };int i;target/s390x/gen-features.c +0
-0
target/s390x/gen-features.c static uint16_t full_GEN12_GA1[] = { · line 454
Quoted 7 lines
target/s390x/gen-features.c +1
-0@@ -448,6 +448,7 @@ static uint16_t full_GEN12_GA1[] = {S390_FEAT_EDAT_2 ,S390_FEAT_SIDE_EFFECT_ACCESS_ESOP2 ,S390_FEAT_AP_QUERY_CONFIG_INFO , +S390_FEAT_AP_QUEUE_INTERRUPT_CONTROL ,S390_FEAT_AP_FACILITIES_TEST , S390_FEAT_AP, };
Looks good to me. In QEMU, we enable/indicate facilities only when all required parts were implemented on the QEMU side.
Is there anything we have to do regarding
- migration support unlocked with this facility
- functional changes we have to implement?
Or is this really a "kernel does it all, migration not relevant" ?
Signature · 3 lines
Thanks, David / dhildenb