Skip to contentSkip to conversation

New: need to re-load driver in guest to make a hot-plug VF work

Bug 47451 [email protected] 1 participant 7 messages Sep 13, 2012 – Dec 10, 2012 KVM

lore
Share & tools
lore

[email protected]

author
#0 lore raw

https://bugzilla.kernel.org/show_bug.cgi?id=47451

Summary: need to re-load driver in guest to make a hot-plug VF
work
Product: Virtualization
Version: unspecified
Kernel Version: 3.5.0
Platform: All
OS/Version: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: kvm
AssignedTo: [email protected]
ReportedBy: [email protected]
Regression: Yes

Environment:
------------
Host OS (ia32/ia32e/IA64):ia32e
Guest OS (ia32/ia32e/IA64):ia32e
Guest OS Type (Linux/Windows):Linux (RHEL6u3)
kvm.git Commit:37e41afa97307a3e54b200a5c9179ada1632a844(master branch) qemu-kvm Commit:28c3a9b197900c88f27b14f8862a7a15c00dc7f0(master branch) Host Kernel Version:3.5.0-rc6 (Also exists in 3.6.0-rc3) Hardware:Romley-EP (SandyBridge system)

Bug detailed description:
--------------------------
After hot plugging a VF to a Linux guest (e.g.RHEL6.3) in qemu monitor, the VF cannot work in the guest by. I need to remove the VF driver (e.g. igbvf, ixgbevf) and probe it again, then the VF can work in guest.
NIC: Intel 82599 NIC, Intel 82576 NIC

It needn't reload VF driver in hot-plug case when using an old kernel. It's a regression in kernel. (commits are in kvm.git and qemu-kvm.git tree)
kvm + qemu-kvm =result
37e41afa + 28c3a9b1 =bad
322728e5 + 28c3a9b1 =good

Note:
1. When assigning a VF in qemu-kvm command line (not hot-plug), VF can work fine after boot-up.
2. It's easier to reproduce this in guest with 512/1024MB memory and 1/2 vCPUs.
3. Can't always reproduce with 2048MB and 2vCPUs. (Not very stable.)

Reproduce steps:
----------------
1.start up a host with kvm
2.qemu-system-x86_64 -m 512 smp 2 –net none –hda /root/rhel6u3.img 3.switch to qemu monitor (ctrl+Alt+2)
4.device_add pci-assign,host=02:10.0,id=mynic (02:10.0 is VF's BDF number.)
5.switch to guest (ctrl+Alt+1)
6.check network of the VF. (it can't work)
7. remove VF driver in guest ('rmmod igbvf')
8. re-probe VF driver in guest ('modprobe igbvf')
9. check network of the VF. (It should work this time.)

Current result:
----------------
The VF cannot work in the guest by default. Need to re-load VF driver in guest.

Expected result:
----------------
VF works well in the guest by default after hot-plug.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
author
#1 lore raw

[Bug 47451] need to re-load driver in guest to make a hot-plug VF work

https://bugzilla.kernel.org/show_bug.cgi?id=47451

Alex Williamson <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #1 from Alex Williamson <[email protected]> 2012-09-14 16:24:11 --- The steps to reproduce don't indicate what you've done in the host to prepare the device. Is igbvf loaded in the host? Is the vf attached to igbvf or pci-stub or nothing? Can we narrow down the kvm.git commit range at all? The one provided is over 12k commits covering v3.4-rc3 to v3.5-rc6. Thanks

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
author
#2 lore raw

[Bug 47451] need to re-load driver in guest to make a hot-plug VF work

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=47451

--- Comment #2 from Jay Ren <[email protected]> 2012-09-20 03:36:31 --- (In reply to comment #1)

Quoted 2 lines

The steps to reproduce don't indicate what you've done in the host to prepare the device. Is igbvf loaded in the host?

No, I don't load igbvf driver in host.

Quoted 1 line

Is the vf attached to igbvf or pci-stub or nothing?

VF is attached to 'pci-stub' in host.

Quoted 2 lines

Can we narrow down the kvm.git commit range at all? The
one provided is over 12k commits covering v3.4-rc3 to v3.5-rc6. Thanks

I did more testing.
Do you remember the bug #43328 ( VT-d/SR-IOV totally doesn't work in guest)? Just use your fix commit for that bug, I'll meet this hot-plug issue. Is there a chance your patch fixed one bug but introduced another one? :)

commit a76beb14123a69ca080f5a5425e28b786d62318d
Author: Alex Williamson <[email protected]>
Date: Mon Jul 9 10:53:22 2012 -0600

KVM: Fix device assignment threaded irq handler

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
author
#3 lore raw

[Bug 47451] need to re-load driver in guest to make a hot-plug VF work

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=47451

--- Comment #3 from Alex Williamson <[email protected]> 2012-09-20 04:36:50 --- (In reply to comment #2)

Quoted 8 lines

(In reply to comment #1)

Quoted 2 lines

Can we narrow down the kvm.git commit range at all? The
one provided is over 12k commits covering v3.4-rc3 to v3.5-rc6. Thanks

I did more testing.
Do you remember the bug #43328 ( VT-d/SR-IOV totally doesn't work in guest)? Just use your fix commit for that bug, I'll meet this hot-plug issue. Is there a chance your patch fixed one bug but introduced another one? :)

Quoted 5 lines

commit a76beb14123a69ca080f5a5425e28b786d62318d
Author: Alex Williamson <[email protected]>
Date: Mon Jul 9 10:53:22 2012 -0600

KVM: Fix device assignment threaded irq handler

Thanks for the narrowing it down. It looks like perhaps that patch was ineffective at trying to keep us out of using IRQF_ONESHOT due to irq_setup_forced_threading() re-enabling it. Does the problem go away if you change the two calls to request_threaded_irq() in that commit to use IRQF_NO_THREAD for the flag value in place of 0?

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
author
#4 lore raw

[Bug 47451] need to re-load driver in guest to make a hot-plug VF work

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=47451

--- Comment #4 from Jay Ren <[email protected]> 2012-09-28 06:07:50 --- (In reply to comment #3)

Quoted 13 lines

(In reply to comment #2)

Quoted 12 lines

(In reply to comment #1)

Quoted 2 lines

Can we narrow down the kvm.git commit range at all? The
one provided is over 12k commits covering v3.4-rc3 to v3.5-rc6. Thanks

I did more testing.
Do you remember the bug #43328 ( VT-d/SR-IOV totally doesn't work in guest)? Just use your fix commit for that bug, I'll meet this hot-plug issue. Is there a chance your patch fixed one bug but introduced another one? :)

commit a76beb14123a69ca080f5a5425e28b786d62318d
Author: Alex Williamson <[email protected]>
Date: Mon Jul 9 10:53:22 2012 -0600

Quoted 7 lines
Quoted 1 line

KVM: Fix device assignment threaded irq handler

Thanks for the narrowing it down. It looks like perhaps that patch was ineffective at trying to keep us out of using IRQF_ONESHOT due to irq_setup_forced_threading() re-enabling it. Does the problem go away if you change the two calls to request_threaded_irq() in that commit to use IRQF_NO_THREAD for the flag value in place of 0?

No, replacing flag value with 'IRQF_NO_THREAD' can't make PCIe NIC hot-plug work.
Can you try with your commit "a76beb14123a6" ?
BTW, sometimes, this bug is not so stable. Using '-m 512 -smp 2' option for qemu-kvm commandline to start a RHEL6.x guest will make it very easy to reproduce.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
author
#5 lore raw

[Bug 47451] need to re-load driver in guest to make a hot-plug VF work

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=47451

Jay Ren <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |RESOLVED
         Resolution|                            |UNREPRODUCIBLE

--- Comment #5 from Jay Ren <[email protected]> 2012-12-10 03:00:01 --- I re-tested this bug against kvm.git next branch (commit:e6c7d321,kernel 3.7) and qemu-kvm.git master branch (commit:4d9367b7). I can't reproduce it now. It should have been fixed by some patches, but I don't know which patch fixed it. As this bug is unstable to reproduce, it's hard for me to do a bisect to find out the exact fix.
close this bug as unreproducible.

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
author
#6 lore raw

[Bug 47451] need to re-load driver in guest to make a hot-plug VF work

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=47451

Jay Ren <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |CLOSED

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.