KVM: arm64: vgic: Misc ITS fixes
v3 cover KVM: arm64: vgic: Misc ITS fixes
The purpose of this series is to help debugging failed ITS saves and restores. Failures can be due to misconfiguration on the guest side:
tables with bogus base addresses, or the guest overwriting L1 indirect tables. KVM can't do much in these cases, but one thing it can do to help is raising errors as soon as possible. Here are a couple of cases where KVM could do more:
- A command that adds an entry into an ITS table that is not in guest memory should fail, as any command should be treated as if it was actually saving entries in guest memory (KVM doesn't until saving). KVM does this check for collections and devices (using vgic_its_check_id), but it doesn't for the ITT (Interrupt Translation Table). Commit #1 adds the missing check.
- Restoring the ITS tables does some checks for corrupted tables, but not as many as in a save. For example, a device ID overflowing the table will be detected on save but not on restore. The consequence is that restoring a corrupted table won't be detected until the next save; including the ITS not working as expected after the restore. As an example, if the guest sets tables overlapping each other, this would most likely result in some corrupted table; and this is what we would see from the host point of view:
guest sets bogus baser addresses
save ioctl
restore ioctl
save ioctl (fails)
This failed save could happen many days after the first operation, so it would be hard to track down. Commit #2 adds some checks into restore:
like checking that the ITE entries are not repeated.
- Restoring a corrupted collection entry is being ignored. Commit #3 fixes this while trying to organize the code so to make the bug more obvious next time.
Finally, failed restores should clean up all intermediate state. Commit #4 takes care of cleaning up everything created until the restore was deemed a failure.
v2: https://lore.kernel.org/kvmarm/[email protected]/
v2 -> v3:
- collect RBs from Eric (Thanks!)
- reorder check in vgic_its_cmd_handle_mapi (commit 1) [Eric]
- removed some checks in vgic_its_restore_ite and vgic_its_restore_dte. [Eric]
- not skipping dummy end elements when restoring collection tables. [Eric]
v1: https://lore.kernel.org/kvmarm/[email protected]/
v1 -> v2:
- moved alloc_collection comment to its respective commit. [marc]
- refactored check_ite to reuse some code from check_id. [marc]
- rewrote all commit messages. [marc]
Tested with kvm-unit-tests ITS tests.
Ricardo Koller (4):
KVM: arm64: vgic: Check that new ITEs could be saved in guest memory
KVM: arm64: vgic: Add more checks when restoring ITS tables
KVM: arm64: vgic: Do not ignore vgic_its_restore_cte failures
KVM: arm64: vgic: Undo work in failed ITS restores
arch/arm64/kvm/vgic/vgic-its.c | 96 +++++++++++++++++++++++++++------- 1 file changed, 78 insertions(+), 18 deletions(-)
Signature · 6 lines
2.36.0.512.ge40c2bad7a-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 1/4 KVM: arm64: vgic: Check that new ITEs could be saved in guest memory
Try to improve the predictability of ITS save/restores by failing commands that would lead to failed saves. More specifically, fail any command that adds an entry into an ITS table that is not in guest memory, which would otherwise lead to a failed ITS save ioctl. There are already checks for collection and device entries, but not for ITEs. Add the corresponding check for the ITT when adding ITEs.
--- arch/arm64/kvm/vgic/vgic-its.c | 47 +++++++++++++++++++++++++--------- 1 file changed, 35 insertions(+), 12 deletions(-)
arch/arm64/kvm/vgic/vgic-its.c +35 -12
@@ -894,6 +894,18 @@ static int vgic_its_cmd_handle_movi(struct kvm *kvm, struct vgic_its *its,return update_affinity (ite->irq, vcpu); } +static bool __is_visible_gfn_locked (struct vgic_its *its,gpa_t gpa) +{ +gfn_t gfn = gpa >>PAGE_SHIFT ; +int idx; +bool ret; + + idx =srcu_read_lock (&its->dev->kvm->srcu); + ret =kvm_is_visible_gfn (its->dev->kvm, gfn); +srcu_read_unlock (&its->dev->kvm->srcu, idx); +return ret; +} +/* * Check whether an ID can be stored into the corresponding guest table. * For a direct table this is pretty easy, but gets a bit nasty for
@@ -908,9 +920,7 @@ static bool vgic_its_check_id(struct vgic_its *its, u64 baser, u32 id,u64 indirect_ptr, type =GITS_BASER_TYPE (baser);phys_addr_t base =GITS_BASER_ADDR_48_to_52 (baser);int esz =GITS_BASER_ENTRY_SIZE (baser); -int index, idx; -gfn_t gfn; -bool ret; +int index;switch (type) {case GITS_BASER_TYPE_DEVICE :
@@ -933,12 +943,11 @@ static bool vgic_its_check_id(struct vgic_its *its, u64 baser, u32 id,return false; addr = base + id * esz; - gfn = addr >>PAGE_SHIFT ;if (eaddr) *eaddr = addr; - goto out; +return __is_visible_gfn_locked (its, addr); }/* calculate and check the index into the 1st level */
@@ -964,16 +973,30 @@ static bool vgic_its_check_id(struct vgic_its *its, u64 baser, u32 id,/* Find the address of the actual entry */ index = id % (SZ_64K / esz); indirect_ptr += index * esz; - gfn = indirect_ptr >>PAGE_SHIFT ;if (eaddr) *eaddr = indirect_ptr; -out: - idx =srcu_read_lock (&its->dev->kvm->srcu); - ret =kvm_is_visible_gfn (its->dev->kvm, gfn); -srcu_read_unlock (&its->dev->kvm->srcu, idx); -return ret; +return __is_visible_gfn_locked (its, indirect_ptr); +} + +/* +* Check whether an event ID can be stored in the corresponding Interrupt +* Translation Table, which starts at device->itt_addr. +*/ +static bool vgic_its_check_event_id (struct vgic_its *its,struct its_device *device, +u32 event_id) +{ +const struct vgic_its_abi *abi =vgic_its_get_abi (its); +int ite_esz = abi->ite_esz; +gpa_t gpa; + +/* max table size is: BIT_ULL(device->num_eventid_bits) * ite_esz */ +if (event_id >=BIT_ULL (device->num_eventid_bits)) +return false; + + gpa = device->itt_addr + event_id * ite_esz; +return __is_visible_gfn_locked (its, gpa); } static int vgic_its_alloc_collection(struct vgic_its *its,
@@ -1061,7 +1084,7 @@ static int vgic_its_cmd_handle_mapi(struct kvm *kvm, struct vgic_its *its,if (!device)return E_ITS_MAPTI_UNMAPPED_DEVICE ; -if (event_id >=BIT_ULL (device->num_eventid_bits)) +if (!vgic_its_check_event_id (its, device, event_id))return E_ITS_MAPTI_ID_OOR ;if (its_cmd_get_command (its_cmd) ==GITS_CMD_MAPTI )
Signature · 6 lines
2.36.0.512.ge40c2bad7a-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 2/4 KVM: arm64: vgic: Add more checks when restoring ITS tables
In reply to Ricardo KollerTry to improve the predictability of ITS save/restores (and debuggability of failed ITS saves) by failing early on restore when trying to read corrupted tables.
Restoring the ITS tables does some checks for corrupted tables, but not as many as in a save: an overflowing device ID will be detected on save but not on restore. The consequence is that restoring a corrupted table won't be detected until the next save; including the ITS not working as expected after the restore. As an example, if the guest sets tables overlapping each other, which would most likely result in some corrupted table, this is what we would see from the host point of view:
guest sets base addresses that overlap each other save ioctl
restore ioctl
save ioctl (fails)
Ideally, we would like the first save to fail, but overlapping tables could actually be intended by the guest. So, let's at least fail on the restore with some checks: like checking that device and event IDs don't overflow their tables.
--- arch/arm64/kvm/vgic/vgic-its.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/arm64/kvm/vgic/vgic-its.c +7 -0
@@ -2198,6 +2198,9 @@ static int vgic_its_restore_ite(struct vgic_its *its, u32 event_id,if (!collection)return -EINVAL ; +if (!vgic_its_check_event_id (its, dev, event_id)) +return -EINVAL ; + ite =vgic_its_alloc_ite (dev, collection, event_id);if (IS_ERR (ite))return PTR_ERR (ite);
@@ -2319,6 +2322,7 @@ static int vgic_its_restore_dte(struct vgic_its *its, u32 id, void *ptr, void *opaque) { struct its_device *dev; +u64 baser = its->baser_device_table;gpa_t itt_addr;u8 num_eventid_bits;u64 entry = *(u64 *)ptr;
@@ -2339,6 +2343,9 @@ static int vgic_its_restore_dte(struct vgic_its *its, u32 id,/* dte entry is valid */ offset = (entry &KVM_ITS_DTE_NEXT_MASK ) >>KVM_ITS_DTE_NEXT_SHIFT ; +if (!vgic_its_check_id (its, baser, id,NULL )) +return -EINVAL ; + dev =vgic_its_alloc_device (its, id, itt_addr, num_eventid_bits);if (IS_ERR (dev))return PTR_ERR (dev);
Signature · 6 lines
2.36.0.512.ge40c2bad7a-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 3/4 KVM: arm64: vgic: Do not ignore vgic_its_restore_cte failures
In reply to Ricardo KollerRestoring a corrupted collection entry (like an out of range ID) is being ignored and treated as success. More specifically, a vgic_its_restore_cte failure is treated as success by vgic_its_restore_collection_table. vgic_its_restore_cte uses positive and negative numbers to return error, and +1 to return success. The caller then uses "ret > 0" to check for success.
Fix this by having vgic_its_restore_cte only return negative numbers on error. Do this by changing alloc_collection return codes to only return negative numbers on error.
--- arch/arm64/kvm/vgic/vgic-its.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-)
arch/arm64/kvm/vgic/vgic-its.c +23 -4
@@ -999,15 +999,16 @@ static bool vgic_its_check_event_id(struct vgic_its *its, struct its_device *devreturn __is_visible_gfn_locked (its, gpa); } +/* +* Add a new collection into the ITS collection table. +* Returns 0 on success, and a negative error value for generic errors. +*/ static int vgic_its_alloc_collection (struct vgic_its *its,struct its_collection **colp,u32 coll_id) {struct its_collection *collection; -if (!vgic_its_check_id (its, its->baser_coll_table, coll_id,NULL )) -return E_ITS_MAPC_COLLECTION_OOR ; - collection =kzalloc (sizeof (*collection),GFP_KERNEL_ACCOUNT );if (!collection)return -ENOMEM ;
@@ -1101,7 +1102,12 @@ static int vgic_its_cmd_handle_mapi(struct kvm *kvm, struct vgic_its *its, collection =find_collection (its, coll_id);if (!collection) { -int ret =vgic_its_alloc_collection (its, &collection, coll_id); +int ret; + +if (!vgic_its_check_id (its, its->baser_coll_table, coll_id,NULL )) +return E_ITS_MAPC_COLLECTION_OOR ; + + ret =vgic_its_alloc_collection (its, &collection, coll_id);if (ret)return ret; new_coll = collection;
@@ -1256,6 +1262,10 @@ static int vgic_its_cmd_handle_mapc(struct kvm *kvm, struct vgic_its *its,if (!collection) {int ret; +if (!vgic_its_check_id (its, its->baser_coll_table, + coll_id,NULL )) +return E_ITS_MAPC_COLLECTION_OOR ; + ret =vgic_its_alloc_collection (its, &collection, coll_id);if (ret)
@@ -2491,6 +2501,11 @@ static int vgic_its_save_cte(struct vgic_its *its,return kvm_write_guest_lock (its->dev->kvm, gpa, &val, esz); } +/* +* Restore a collection entry into the ITS collection table. +* Return +1 on success, 0 if the entry was invalid (which should be +* interpreted as end-of-table), and a negative error value for generic errors. +*/ static int vgic_its_restore_cte (struct vgic_its *its,gpa_t gpa,int esz) {struct its_collection *collection;
@@ -2517,6 +2532,10 @@ static int vgic_its_restore_cte(struct vgic_its *its, gpa_t gpa, int esz) collection =find_collection (its, coll_id);if (collection)return -EEXIST ; + +if (!vgic_its_check_id (its, its->baser_coll_table, coll_id,NULL )) +return -EINVAL ; + ret =vgic_its_alloc_collection (its, &collection, coll_id);if (ret)return ret;
Signature · 6 lines
2.36.0.512.ge40c2bad7a-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 4/4 KVM: arm64: vgic: Undo work in failed ITS restores
In reply to Ricardo KollerFailed ITS restores should clean up all state restored until the failure. There is some cleanup already present when failing to restore some tables, but it's not complete. Add the missing cleanup.
Note that this changes the behavior in case of a failed restore of the device tables.
restore ioctl:
1. restore collection tables
2. restore device tables
With this commit, failures in 2. clean up everything created so far, including state created by 1.
--- arch/arm64/kvm/vgic/vgic-its.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)
arch/arm64/kvm/vgic/vgic-its.c +13 -2
@@ -2219,8 +2219,10 @@ static int vgic_its_restore_ite(struct vgic_its *its, u32 event_id, vcpu =kvm_get_vcpu (kvm, collection->target_addr); irq =vgic_add_lpi (kvm, lpi_id, vcpu); -if (IS_ERR (irq)) +if (IS_ERR (irq)) { +its_free_ite (kvm, ite);return PTR_ERR (irq); + } ite->irq = irq;return offset;
@@ -2485,6 +2487,9 @@ static int vgic_its_restore_device_tables(struct vgic_its *its)if (ret >0 ) ret =0 ; +if (ret <0 ) +vgic_its_free_device_list (its->dev->kvm, its); +return ret; }
@@ -2615,6 +2620,9 @@ static int vgic_its_restore_collection_table(struct vgic_its *its)if (ret >0 )return 0 ; +if (ret <0 ) +vgic_its_free_collection_list (its->dev->kvm, its); +return ret; }
@@ -2646,7 +2654,10 @@ static int vgic_its_restore_tables_v0(struct vgic_its *its)if (ret)return ret; -return vgic_its_restore_device_tables (its); + ret =vgic_its_restore_device_tables (its); +if (ret) +vgic_its_free_collection_list (its->dev->kvm, its); +return ret; }static int vgic_its_commit_v0 (struct vgic_its *its)
Signature · 6 lines
2.36.0.512.ge40c2bad7a-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Hi Ricardo,
Quoted 5 lines · Ricardo Koller
The purpose of this series is to help debugging failed ITS saves and restores. Failures can be due to misconfiguration on the guest side:
tables with bogus base addresses, or the guest overwriting L1 indirect tables. KVM can't do much in these cases, but one thing it can do to help is raising errors as soon as possible.
For the series:
--
Thanks,
Oliver
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Quoted 8 lines · Ricardo Koller
The purpose of this series is to help debugging failed ITS saves and restores. Failures can be due to misconfiguration on the guest side:
tables with bogus base addresses, or the guest overwriting L1 indirect tables. KVM can't do much in these cases, but one thing it can do to help is raising errors as soon as possible. Here are a couple of cases where KVM could do more:[...]
Applied to next, thanks!
[1/4] KVM: arm64: vgic: Check that new ITEs could be saved in guest memory
commit: cafe7e544d4979da222eaff12141ecac07901b9c
[2/4] KVM: arm64: vgic: Add more checks when restoring ITS tables
commit: 243b1f6c8f0748bd7b03eab17323f1187e580771
[3/4] KVM: arm64: vgic: Do not ignore vgic_its_restore_cte failures
commit: a1ccfd6f6e06eceb632cc29c4f15a32860f05a7e
[4/4] KVM: arm64: vgic: Undo work in failed ITS restores
commit: 8c5e74c90bb522181dfb051fffff3dad702e704dCheers,
M.
Signature · 7 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm