Skip to contentSkip to conversation

GICv3 stride issue with many VCPUs

DISCUSSION Christoffer Dall 3 participants 5 messages Sep 3, 2018 KVM arm64

lore
Share & tools
lore
#0 lore raw

Hi,

I recently ran kvm-unit-tests on TX2, and I got an error from gicv3-ipi, with this in the log:

  % cat logs/gicv3-ipi.log
  timeout -k 1s --foreground 90s ../src/qemu/build/aarch64-softmmu/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 224 -machine gic-version=3 -append ipi # -initrd /tmp/tmp.Xy3T5zjn9n
  lib/arm/gic-v3.c:25: assert failed: 0
          STACK:

Which is an assert in gicv3_set_redist_base().

I suspect this is related to the number of vcpus (112) and our GICv3 configuration.

Reporting it here in case anyone else is seeing this and wondering what's going on.

Thanks,

Christoffer

#1 lore raw

Hi Christoffer,

Quoted 9 lines · Christoffer Dall

Hi,

I recently ran kvm-unit-tests on TX2, and I got an error from gicv3-ipi, with this in the log:

  % cat logs/gicv3-ipi.log
  timeout -k 1s --foreground 90s ../src/qemu/build/aarch64-softmmu/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 224 -machine gic-version=3 -append ipi # -initrd /tmp/tmp.Xy3T5zjn9n
  lib/arm/gic-v3.c:25: assert failed: 0
          STACK:

Quickly looking at gicv3_set_redist_base(), I noted a single contiguous redistributor region seems to be used: void *ptr = gicv3_data.redist_base[0];

Supporting more than 123 vcpus requires to use the 2 separate redistributor regions exposed by lastest QEMU mach-virt. I suspect the kvm-unit-test is not using both redist regions yet.

Thanks

Eric

Quoted 13 lines

Which is an assert in gicv3_set_redist_base().

I suspect this is related to the number of vcpus (112) and our GICv3 configuration.

Reporting it here in case anyone else is seeing this and wondering what's going on.

Thanks,

Christoffer

#2 lore raw
Quoted 13 lines · Auger Eric

Hi Christoffer,

Quoted 9 lines · Christoffer Dall

Hi,

I recently ran kvm-unit-tests on TX2, and I got an error from gicv3-ipi, with this in the log:

  % cat logs/gicv3-ipi.log
  timeout -k 1s --foreground 90s ../src/qemu/build/aarch64-softmmu/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 224 -machine gic-version=3 -append ipi # -initrd /tmp/tmp.Xy3T5zjn9n
  lib/arm/gic-v3.c:25: assert failed: 0
          STACK:
Quoted 7 lines

Quickly looking at gicv3_set_redist_base(), I noted a single contiguous redistributor region seems to be used: void *ptr = gicv3_data.redist_base[0];

Supporting more than 123 vcpus requires to use the 2 separate redistributor regions exposed by lastest QEMU mach-virt. I suspect the kvm-unit-test is not using both redist regions yet.

Nope. I didn't have any hardware to test that on yet :-) I can write a patch, though, and send it to Christoffer to test. Or, if Christoffer doesn't mind hacking kvm-unit-tests, I'd be happy to review his patch :-)

Thanks,
drew

Quoted 19 lines

Thanks

Eric

Quoted 13 lines

Which is an assert in gicv3_set_redist_base().

I suspect this is related to the number of vcpus (112) and our GICv3 configuration.

Reporting it here in case anyone else is seeing this and wondering what's going on.

Thanks,

Christoffer

#3 lore raw
Quoted 18 lines · Andrew Jones
Quoted 17 lines · Auger Eric

Hi Christoffer,

Quoted 9 lines · Christoffer Dall

Hi,

I recently ran kvm-unit-tests on TX2, and I got an error from gicv3-ipi, with this in the log:

  % cat logs/gicv3-ipi.log
  timeout -k 1s --foreground 90s ../src/qemu/build/aarch64-softmmu/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 224 -machine gic-version=3 -append ipi # -initrd /tmp/tmp.Xy3T5zjn9n
  lib/arm/gic-v3.c:25: assert failed: 0
          STACK:

Quickly looking at gicv3_set_redist_base(), I noted a single contiguous redistributor region seems to be used: void *ptr = gicv3_data.redist_base[0];

Quoted 8 lines
Quoted 3 lines

Supporting more than 123 vcpus requires to use the 2 separate redistributor regions exposed by lastest QEMU mach-virt. I suspect the kvm-unit-test is not using both redist regions yet.

Nope. I didn't have any hardware to test that on yet :-) I can write a patch, though, and send it to Christoffer to test. Or, if Christoffer doesn't mind hacking kvm-unit-tests, I'd be happy to review his patch :-)

I'm quite caugt up at the moment, so if you could write a patch, I'll be happy to test.

Thanks,

Christoffer

#4 lore raw
Quoted 23 lines · Christoffer Dall
Quoted 22 lines · Andrew Jones
Quoted 20 lines · Auger Eric

Hi Christoffer,

Quoted 9 lines · Christoffer Dall

Hi,

I recently ran kvm-unit-tests on TX2, and I got an error from gicv3-ipi, with this in the log:

  % cat logs/gicv3-ipi.log
  timeout -k 1s --foreground 90s ../src/qemu/build/aarch64-softmmu/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 224 -machine gic-version=3 -append ipi # -initrd /tmp/tmp.Xy3T5zjn9n
  lib/arm/gic-v3.c:25: assert failed: 0
          STACK:

Quickly looking at gicv3_set_redist_base(), I noted a single contiguous redistributor region seems to be used: void *ptr = gicv3_data.redist_base[0];

Supporting more than 123 vcpus requires to use the 2 separate redistributor regions exposed by lastest QEMU mach-virt. I suspect the kvm-unit-test is not using both redist regions yet.

Quoted 8 lines
Quoted 4 lines

Nope. I didn't have any hardware to test that on yet :-) I can write a patch, though, and send it to Christoffer to test. Or, if Christoffer doesn't mind hacking kvm-unit-tests, I'd be happy to review his patch :-)

I'm quite caugt up at the moment, so if you could write a patch, I'll be happy to test.

No problem. I'll try to pull it together soon.

Thanks,
drew