Skip to contentSkip to conversation

PCI: Linux kernel SR-IOV support

PATCH Superseded Yu Zhao 25 participants 229 messages Oct 22, 2008 – Apr 7, 2009 Page 2 of 2

Subsystems (5)
Documentation x86 KVM PCI +1 more subsystems

v6 of 12 Superseded Latest is v12 →

lore
Share & tools
lore
#192 lore raw
In reply to Greg KH
Quoted 27 lines · * Greg KH ([email protected]) wrote:
Quoted 18 lines · Matthew Wilcox
Quoted 17 lines · Greg KH
Quoted 5 lines · H L

I have not modified any existing drivers, but instead I threw together a bare-bones module enabling me to make a call to pci_iov_register() and then poke at an SR-IOV adapter's /sys entries for which no driver was loaded.

Quoted 7 lines

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver associated with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify() function. Essentially this suggests to me a model for the PF driver to perform any "global actions" or setup on behalf of VFs before enabling them after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the "pci_dev" level or on a higher one?

Quoted 7 lines
Quoted 2 lines

Will all drivers that want to bind to a "VF" device need to be rewritten?

The current model being implemented by my colleagues has separate drivers for the PF (aka native) and VF devices. I don't personally believe this is the correct path, but I'm reserving judgement until I see some code.

Quoted 8 lines

Hm, I would like to see that code before we can properly evaluate this interface. Especially as they are all tightly tied together.

Quoted 3 lines

I don't think we really know what the One True Usage model is for VF devices. Chris Wright has some ideas, I have some ideas and Yu Zhao has some ideas. I bet there's other people who have other ideas too.

I'd love to hear those ideas.

First there's the question of how to represent the VF on the host. Ideally (IMO) this would show up as a normal interface so that normal tools can configure the interface. This is not exactly how the first round of patches were designed.

Second there's the question of reserving the BDF on the host such that we don't have two drivers (one in the host and one in a guest) trying to drive the same device (an issue that shows up for device assignment as well as VF assignment).

Third there's the question of whether the VF can be used in the host at all.

Fourth there's the question of whether the VF and PF drivers are the same or separate.

The typical usecase is assigning the VF to the guest directly, so there's only enough functionality in the host side to allocate a VF, configure it, and assign it (and propagate AER). This is with separate PF and VF driver.

As Anthony mentioned, we are interested in allowing the host to use the VF. This could be useful for containers as well as dedicating a VF (a set of device resources) to a guest w/out passing it through.

thanks,
-chris

#193 lore raw
In reply to Greg KH
Quoted 27 lines · * Greg KH ([email protected]) wrote:
Quoted 18 lines · Matthew Wilcox
Quoted 17 lines · Greg KH
Quoted 5 lines · H L

I have not modified any existing drivers, but instead I threw together a bare-bones module enabling me to make a call to pci_iov_register() and then poke at an SR-IOV adapter's /sys entries for which no driver was loaded.

Quoted 7 lines

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver associated with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify() function. Essentially this suggests to me a model for the PF driver to perform any "global actions" or setup on behalf of VFs before enabling them after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the "pci_dev" level or on a higher one?

Quoted 7 lines
Quoted 2 lines

Will all drivers that want to bind to a "VF" device need to be rewritten?

The current model being implemented by my colleagues has separate drivers for the PF (aka native) and VF devices. I don't personally believe this is the correct path, but I'm reserving judgement until I see some code.

Quoted 8 lines

Hm, I would like to see that code before we can properly evaluate this interface. Especially as they are all tightly tied together.

Quoted 3 lines

I don't think we really know what the One True Usage model is for VF devices. Chris Wright has some ideas, I have some ideas and Yu Zhao has some ideas. I bet there's other people who have other ideas too.

I'd love to hear those ideas.

First there's the question of how to represent the VF on the host. Ideally (IMO) this would show up as a normal interface so that normal tools can configure the interface. This is not exactly how the first round of patches were designed.

Second there's the question of reserving the BDF on the host such that we don't have two drivers (one in the host and one in a guest) trying to drive the same device (an issue that shows up for device assignment as well as VF assignment).

Third there's the question of whether the VF can be used in the host at all.

Fourth there's the question of whether the VF and PF drivers are the same or separate.

The typical usecase is assigning the VF to the guest directly, so there's only enough functionality in the host side to allocate a VF, configure it, and assign it (and propagate AER). This is with separate PF and VF driver.

As Anthony mentioned, we are interested in allowing the host to use the VF. This could be useful for containers as well as dedicating a VF (a set of device resources) to a guest w/out passing it through.

thanks,
-chris

#194 lore raw
Quoted 53 lines · Chris Wright
Quoted 27 lines · * Greg KH ([email protected]) wrote:
Quoted 18 lines · Matthew Wilcox
Quoted 17 lines · Greg KH
Quoted 5 lines · H L

I have not modified any existing drivers, but instead I threw together a bare-bones module enabling me to make a call to pci_iov_register() and then poke at an SR-IOV adapter's /sys entries for which no driver was loaded.

Quoted 7 lines

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver associated with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify() function. Essentially this suggests to me a model for the PF driver to perform any "global actions" or setup on behalf of VFs before enabling them after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the "pci_dev" level or on a higher one?

Quoted 7 lines
Quoted 2 lines

Will all drivers that want to bind to a "VF" device need to be rewritten?

The current model being implemented by my colleagues has separate drivers for the PF (aka native) and VF devices. I don't personally believe this is the correct path, but I'm reserving judgement until I see some code.

Quoted 8 lines

Hm, I would like to see that code before we can properly evaluate this interface. Especially as they are all tightly tied together.

Quoted 3 lines

I don't think we really know what the One True Usage model is for VF devices. Chris Wright has some ideas, I have some ideas and Yu Zhao has some ideas. I bet there's other people who have other ideas too.

I'd love to hear those ideas.

First there's the question of how to represent the VF on the host. Ideally (IMO) this would show up as a normal interface so that normal tools can configure the interface. This is not exactly how the first round of patches were designed.

Second there's the question of reserving the BDF on the host such that we don't have two drivers (one in the host and one in a guest) trying to drive the same device (an issue that shows up for device assignment as well as VF assignment).

Third there's the question of whether the VF can be used in the host at all.

Fourth there's the question of whether the VF and PF drivers are the same or separate.

Quoted 8 lines

The typical usecase is assigning the VF to the guest directly, so there's only enough functionality in the host side to allocate a VF, configure it, and assign it (and propagate AER). This is with separate PF and VF driver.

As Anthony mentioned, we are interested in allowing the host to use the VF. This could be useful for containers as well as dedicating a VF (a set of device resources) to a guest w/out passing it through.

All of this looks great. So, with all of these questions, how does the current code pertain to these issues? It seems like we have a long way to go...

thanks,

greg k-h

#195 lore raw
In reply to Chris Wright
Quoted 53 lines · Chris Wright
Quoted 27 lines · * Greg KH ([email protected]) wrote:
Quoted 18 lines · Matthew Wilcox
Quoted 17 lines · Greg KH
Quoted 5 lines · H L

I have not modified any existing drivers, but instead I threw together a bare-bones module enabling me to make a call to pci_iov_register() and then poke at an SR-IOV adapter's /sys entries for which no driver was loaded.

Quoted 7 lines

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver associated with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify() function. Essentially this suggests to me a model for the PF driver to perform any "global actions" or setup on behalf of VFs before enabling them after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the "pci_dev" level or on a higher one?

Quoted 7 lines
Quoted 2 lines

Will all drivers that want to bind to a "VF" device need to be rewritten?

The current model being implemented by my colleagues has separate drivers for the PF (aka native) and VF devices. I don't personally believe this is the correct path, but I'm reserving judgement until I see some code.

Quoted 8 lines

Hm, I would like to see that code before we can properly evaluate this interface. Especially as they are all tightly tied together.

Quoted 3 lines

I don't think we really know what the One True Usage model is for VF devices. Chris Wright has some ideas, I have some ideas and Yu Zhao has some ideas. I bet there's other people who have other ideas too.

I'd love to hear those ideas.

First there's the question of how to represent the VF on the host. Ideally (IMO) this would show up as a normal interface so that normal tools can configure the interface. This is not exactly how the first round of patches were designed.

Second there's the question of reserving the BDF on the host such that we don't have two drivers (one in the host and one in a guest) trying to drive the same device (an issue that shows up for device assignment as well as VF assignment).

Third there's the question of whether the VF can be used in the host at all.

Fourth there's the question of whether the VF and PF drivers are the same or separate.

Quoted 8 lines

The typical usecase is assigning the VF to the guest directly, so there's only enough functionality in the host side to allocate a VF, configure it, and assign it (and propagate AER). This is with separate PF and VF driver.

As Anthony mentioned, we are interested in allowing the host to use the VF. This could be useful for containers as well as dedicating a VF (a set of device resources) to a guest w/out passing it through.

All of this looks great. So, with all of these questions, how does the current code pertain to these issues? It seems like we have a long way to go...

thanks,

greg k-h

author
#196 lore raw
In reply to Chris Wright
Quoted 31 lines · Chris Wright
Quoted 30 lines · * Greg KH ([email protected]) wrote:
Quoted 17 lines · Matthew Wilcox
Quoted 16 lines · Greg KH
Quoted 5 lines · H L

I have not modified any existing drivers, but instead I threw together a bare-bones module enabling me to make a call to pci_iov_register() and then poke at an SR-IOV adapter's /sys entries for which no driver was loaded.

Quoted 7 lines

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver associated with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify() function. Essentially this suggests to me a model for the PF driver to perform any "global actions" or setup on behalf of VFs before enabling them after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the "pci_dev" level or on a higher one?

Quoted 6 lines
Quoted 2 lines

Will all drivers that want to bind to a "VF" device need to be rewritten?

The current model being implemented by my colleagues has separate drivers for the PF (aka native) and VF devices. I don't personally believe this is the correct path, but I'm reserving judgement until I see some code.

Hm, I would like to see that code before we can properly evaluate this interface. Especially as they are all tightly tied together.

Quoted 3 lines

I don't think we really know what the One True Usage model is for VF devices. Chris Wright has some ideas, I have some ideas and Yu Zhao has some ideas. I bet there's other people who have other ideas too.

Quoted 6 lines
Quoted 1 line

I'd love to hear those ideas.

First there's the question of how to represent the VF on the host. Ideally (IMO) this would show up as a normal interface so that normal tools can configure the interface. This is not exactly how the first round of patches were designed.

Whether the VF can show up as a normal interface is decided by VF driver. VF is represented by 'pci_dev' at PCI level, so VF driver can be loaded as normal PCI device driver.

What the software representation (eth, framebuffer, etc.) created by VF driver is not controlled by SR-IOV framework.

So you definitely can use normal tool to configure the VF if its driver supports that :-)

Quoted 5 lines

Second there's the question of reserving the BDF on the host such that we don't have two drivers (one in the host and one in a guest) trying to drive the same device (an issue that shows up for device assignment as well as VF assignment).

If we don't reserve BDF for the device, they can't work neither in the host nor the guest.

Without BDF, we can't access the config space of the device, the device also can't do DMA.

Did I miss your point?

Quoted 3 lines

Third there's the question of whether the VF can be used in the host at all.

Why can't? My VFs work well in the host as normal PCI devices :-)

Quoted 3 lines

Fourth there's the question of whether the VF and PF drivers are the same or separate.

As I mentioned in another email of this thread. We can't predict how hardware vendor creates their SR-IOV device. PCI SIG doesn't define device specific logics.

So I think the answer of this question is up to the device driver developers. If PF and VF in a SR-IOV device have similar logics, then they can combine the driver. Otherwise, e.g., if PF doesn't have real functionality at all -- it only has registers to control internal resource allocation for VFs, then the drivers should be separate, right?

Quoted 9 lines

The typical usecase is assigning the VF to the guest directly, so there's only enough functionality in the host side to allocate a VF, configure it, and assign it (and propagate AER). This is with separate PF and VF driver.

As Anthony mentioned, we are interested in allowing the host to use the VF. This could be useful for containers as well as dedicating a VF (a set of device resources) to a guest w/out passing it through.

I've considered the container cases, we don't have problem with running VF driver in the host.

Thanks,
Yu

author
#197 lore raw
In reply to Chris Wright
Quoted 31 lines · Chris Wright
Quoted 30 lines · * Greg KH ([email protected]) wrote:
Quoted 17 lines · Matthew Wilcox
Quoted 16 lines · Greg KH
Quoted 5 lines · H L

I have not modified any existing drivers, but instead I threw together a bare-bones module enabling me to make a call to pci_iov_register() and then poke at an SR-IOV adapter's /sys entries for which no driver was loaded.

Quoted 7 lines

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver associated with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify() function. Essentially this suggests to me a model for the PF driver to perform any "global actions" or setup on behalf of VFs before enabling them after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the "pci_dev" level or on a higher one?

Quoted 6 lines
Quoted 2 lines

Will all drivers that want to bind to a "VF" device need to be rewritten?

The current model being implemented by my colleagues has separate drivers for the PF (aka native) and VF devices. I don't personally believe this is the correct path, but I'm reserving judgement until I see some code.

Hm, I would like to see that code before we can properly evaluate this interface. Especially as they are all tightly tied together.

Quoted 3 lines

I don't think we really know what the One True Usage model is for VF devices. Chris Wright has some ideas, I have some ideas and Yu Zhao has some ideas. I bet there's other people who have other ideas too.

Quoted 6 lines
Quoted 1 line

I'd love to hear those ideas.

First there's the question of how to represent the VF on the host. Ideally (IMO) this would show up as a normal interface so that normal tools can configure the interface. This is not exactly how the first round of patches were designed.

Whether the VF can show up as a normal interface is decided by VF driver. VF is represented by 'pci_dev' at PCI level, so VF driver can be loaded as normal PCI device driver.

What the software representation (eth, framebuffer, etc.) created by VF driver is not controlled by SR-IOV framework.

So you definitely can use normal tool to configure the VF if its driver supports that :-)

Quoted 5 lines

Second there's the question of reserving the BDF on the host such that we don't have two drivers (one in the host and one in a guest) trying to drive the same device (an issue that shows up for device assignment as well as VF assignment).

If we don't reserve BDF for the device, they can't work neither in the host nor the guest.

Without BDF, we can't access the config space of the device, the device also can't do DMA.

Did I miss your point?

Quoted 3 lines

Third there's the question of whether the VF can be used in the host at all.

Why can't? My VFs work well in the host as normal PCI devices :-)

Quoted 3 lines

Fourth there's the question of whether the VF and PF drivers are the same or separate.

As I mentioned in another email of this thread. We can't predict how hardware vendor creates their SR-IOV device. PCI SIG doesn't define device specific logics.

So I think the answer of this question is up to the device driver developers. If PF and VF in a SR-IOV device have similar logics, then they can combine the driver. Otherwise, e.g., if PF doesn't have real functionality at all -- it only has registers to control internal resource allocation for VFs, then the drivers should be separate, right?

Quoted 9 lines

The typical usecase is assigning the VF to the guest directly, so there's only enough functionality in the host side to allocate a VF, configure it, and assign it (and propagate AER). This is with separate PF and VF driver.

As Anthony mentioned, we are interested in allowing the host to use the VF. This could be useful for containers as well as dedicating a VF (a set of device resources) to a guest w/out passing it through.

I've considered the container cases, we don't have problem with running VF driver in the host.

Thanks,
Yu

#198 lore raw
Quoted 5 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf

Of

Quoted 4 lines

Zhao, Yu
Sent: Thursday, November 06, 2008 11:06 PM
To: Chris Wright
Cc: [email protected]; [email protected];

[email protected];

Quoted 1 line

Matthew Wilcox; Greg KH; [email protected];

[email protected];

Quoted 4 lines

[email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 0/16 v6] PCI: Linux kernel SR-IOV support

Quoted 8 lines
Quoted 5 lines · Chris Wright
Quoted 4 lines · * Greg KH ([email protected]) wrote:
Quoted 3 lines · Matthew Wilcox
Quoted 2 lines · Greg KH
Quoted 1 line · H L

I have not modified any existing drivers, but instead I threw

together

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

a bare-bones module enabling me to make a call to

pci_iov_register()

Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

and then poke at an SR-IOV adapter's /sys entries for which no

driver

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

was loaded.

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver

associated

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify()

function.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Essentially this suggests to me a model for the PF driver to

perform

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

any "global actions" or setup on behalf of VFs before enabling

them

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the

"pci_dev"

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

level or on a higher one?

Will all drivers that want to bind to a "VF" device need to be rewritten?

The current model being implemented by my colleagues has separate drivers for the PF (aka native) and VF devices. I don't

personally

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

believe this is the correct path, but I'm reserving judgement

until I

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

see some code.

Hm, I would like to see that code before we can properly evaluate

this

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

interface. Especially as they are all tightly tied together.

Quoted 1 line

I don't think we really know what the One True Usage model is for

VF

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

devices. Chris Wright has some ideas, I have some ideas and Yu

Zhao

Quoted 6 lines

has

Quoted 5 lines
Quoted 2 lines
Quoted 1 line

some ideas. I bet there's other people who have other ideas too.

I'd love to hear those ideas.

First there's the question of how to represent the VF on the host. Ideally (IMO) this would show up as a normal interface so that

normal

Quoted 2 lines

tools

Quoted 1 line

can configure the interface. This is not exactly how the first

round of

Quoted 4 lines
Quoted 1 line

patches were designed.

Whether the VF can show up as a normal interface is decided by VF driver. VF is represented by 'pci_dev' at PCI level, so VF driver can

be

Quoted 3 lines

loaded as normal PCI device driver.

What the software representation (eth, framebuffer, etc.) created by

VF

Quoted 3 lines

driver is not controlled by SR-IOV framework.

So you definitely can use normal tool to configure the VF if its

driver

Quoted 4 lines

supports that :-)

Quoted 2 lines

Second there's the question of reserving the BDF on the host such

that

Quoted 1 line
Quoted 1 line

we don't have two drivers (one in the host and one in a guest)

trying to

Quoted 1 line
Quoted 1 line

drive the same device (an issue that shows up for device assignment

as

Quoted 6 lines
Quoted 1 line

well as VF assignment).

If we don't reserve BDF for the device, they can't work neither in the host nor the guest.

Without BDF, we can't access the config space of the device, the

device

Quoted 6 lines

also can't do DMA.

Did I miss your point?

Quoted 2 lines

Third there's the question of whether the VF can be used in the host

at

Quoted 12 lines
Quoted 1 line

all.

Why can't? My VFs work well in the host as normal PCI devices :-)

Quoted 3 lines

Fourth there's the question of whether the VF and PF drivers are the same or separate.

As I mentioned in another email of this thread. We can't predict how hardware vendor creates their SR-IOV device. PCI SIG doesn't define device specific logics.

Quoted 5 lines

So I think the answer of this question is up to the device driver developers. If PF and VF in a SR-IOV device have similar logics, then they can combine the driver. Otherwise, e.g., if PF doesn't have real functionality at all -- it only has registers to control internal resource allocation for VFs, then the drivers should be separate,

right?

Right, this really depends upon the functionality behind a VF. If VF is done as a subset of netdev interface (for example, a queue pair), then a split VF/PF driver model and a proprietary communication channel is in order.

If each VF is done as a complete netdev interface (like in our 10GbE IOV controllers), then PF and VF drivers could be the same. Each VF can be independently driven by such "native" netdev driver; this includes the ability to run a native driver in a guest in passthru mode. A PF driver in a privileged domain doesn't even have to be present.

Quoted 5 lines
Quoted 5 lines

The typical usecase is assigning the VF to the guest directly, so there's only enough functionality in the host side to allocate a VF, configure it, and assign it (and propagate AER). This is with

separate

Quoted 3 lines
Quoted 3 lines

PF and VF driver.

As Anthony mentioned, we are interested in allowing the host to use

the

Quoted 1 line
Quoted 1 line

VF. This could be useful for containers as well as dedicating a VF

(a

Quoted 3 lines
Quoted 1 line

set of device resources) to a guest w/out passing it through.

I've considered the container cases, we don't have problem with

running

Quoted 8 lines

VF driver in the host.

Thanks,
Yu
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/virtualization

#199 lore raw
In reply to Zhao, Yu
Quoted 5 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf

Of

Quoted 4 lines

Zhao, Yu
Sent: Thursday, November 06, 2008 11:06 PM
To: Chris Wright
Cc: [email protected]; [email protected];

[email protected];

Quoted 1 line

Matthew Wilcox; Greg KH; [email protected];

[email protected];

Quoted 4 lines

[email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 0/16 v6] PCI: Linux kernel SR-IOV support

Quoted 8 lines
Quoted 5 lines · Chris Wright
Quoted 4 lines · * Greg KH ([email protected]) wrote:
Quoted 3 lines · Matthew Wilcox
Quoted 2 lines · Greg KH
Quoted 1 line · H L

I have not modified any existing drivers, but instead I threw

together

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

a bare-bones module enabling me to make a call to

pci_iov_register()

Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

and then poke at an SR-IOV adapter's /sys entries for which no

driver

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

was loaded.

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver

associated

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify()

function.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Essentially this suggests to me a model for the PF driver to

perform

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

any "global actions" or setup on behalf of VFs before enabling

them

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the

"pci_dev"

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

level or on a higher one?

Will all drivers that want to bind to a "VF" device need to be rewritten?

The current model being implemented by my colleagues has separate drivers for the PF (aka native) and VF devices. I don't

personally

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

believe this is the correct path, but I'm reserving judgement

until I

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

see some code.

Hm, I would like to see that code before we can properly evaluate

this

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

interface. Especially as they are all tightly tied together.

Quoted 1 line

I don't think we really know what the One True Usage model is for

VF

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

devices. Chris Wright has some ideas, I have some ideas and Yu

Zhao

Quoted 6 lines

has

Quoted 5 lines
Quoted 2 lines
Quoted 1 line

some ideas. I bet there's other people who have other ideas too.

I'd love to hear those ideas.

First there's the question of how to represent the VF on the host. Ideally (IMO) this would show up as a normal interface so that

normal

Quoted 2 lines

tools

Quoted 1 line

can configure the interface. This is not exactly how the first

round of

Quoted 4 lines
Quoted 1 line

patches were designed.

Whether the VF can show up as a normal interface is decided by VF driver. VF is represented by 'pci_dev' at PCI level, so VF driver can

be

Quoted 3 lines

loaded as normal PCI device driver.

What the software representation (eth, framebuffer, etc.) created by

VF

Quoted 3 lines

driver is not controlled by SR-IOV framework.

So you definitely can use normal tool to configure the VF if its

driver

Quoted 4 lines

supports that :-)

Quoted 2 lines

Second there's the question of reserving the BDF on the host such

that

Quoted 1 line
Quoted 1 line

we don't have two drivers (one in the host and one in a guest)

trying to

Quoted 1 line
Quoted 1 line

drive the same device (an issue that shows up for device assignment

as

Quoted 6 lines
Quoted 1 line

well as VF assignment).

If we don't reserve BDF for the device, they can't work neither in the host nor the guest.

Without BDF, we can't access the config space of the device, the

device

Quoted 6 lines

also can't do DMA.

Did I miss your point?

Quoted 2 lines

Third there's the question of whether the VF can be used in the host

at

Quoted 12 lines
Quoted 1 line

all.

Why can't? My VFs work well in the host as normal PCI devices :-)

Quoted 3 lines

Fourth there's the question of whether the VF and PF drivers are the same or separate.

As I mentioned in another email of this thread. We can't predict how hardware vendor creates their SR-IOV device. PCI SIG doesn't define device specific logics.

Quoted 5 lines

So I think the answer of this question is up to the device driver developers. If PF and VF in a SR-IOV device have similar logics, then they can combine the driver. Otherwise, e.g., if PF doesn't have real functionality at all -- it only has registers to control internal resource allocation for VFs, then the drivers should be separate,

right?

Right, this really depends upon the functionality behind a VF. If VF is done as a subset of netdev interface (for example, a queue pair), then a split VF/PF driver model and a proprietary communication channel is in order.

If each VF is done as a complete netdev interface (like in our 10GbE IOV controllers), then PF and VF drivers could be the same. Each VF can be independently driven by such "native" netdev driver; this includes the ability to run a native driver in a guest in passthru mode. A PF driver in a privileged domain doesn't even have to be present.

Quoted 5 lines
Quoted 5 lines

The typical usecase is assigning the VF to the guest directly, so there's only enough functionality in the host side to allocate a VF, configure it, and assign it (and propagate AER). This is with

separate

Quoted 3 lines
Quoted 3 lines

PF and VF driver.

As Anthony mentioned, we are interested in allowing the host to use

the

Quoted 1 line
Quoted 1 line

VF. This could be useful for containers as well as dedicating a VF

(a

Quoted 3 lines
Quoted 1 line

set of device resources) to a guest w/out passing it through.

I've considered the container cases, we don't have problem with

running

Quoted 8 lines

VF driver in the host.

Thanks,
Yu
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/virtualization

#200 lore raw
In reply to Matthew Wilcox
Quoted 18 lines · Matthew Wilcox
Quoted 17 lines · Greg KH
Quoted 5 lines · H L

I have not modified any existing drivers, but instead I threw together a bare-bones module enabling me to make a call to pci_iov_register() and then poke at an SR-IOV adapter's /sys entries for which no driver was loaded.

Quoted 7 lines

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver associated with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify() function. Essentially this suggests to me a model for the PF driver to perform any "global actions" or setup on behalf of VFs before enabling them after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the "pci_dev" level or on a higher one?

Quoted 7 lines
Quoted 2 lines

Will all drivers that want to bind to a "VF" device need to be rewritten?

The current model being implemented by my colleagues has separate drivers for the PF (aka native) and VF devices. I don't personally believe this is the correct path, but I'm reserving judgement until I see some code.

Hm, I would like to see that code before we can properly evaluate this interface. Especially as they are all tightly tied together.

Quoted 3 lines

I don't think we really know what the One True Usage model is for VF devices. Chris Wright has some ideas, I have some ideas and Yu Zhao has some ideas. I bet there's other people who have other ideas too.

I'd love to hear those ideas.

Rumor has it, there is some Xen code floating around to support this already, is that true?

thanks,

greg k-h

#201 lore raw
In reply to Greg KH
Quoted 14 lines · Greg KH
Quoted 5 lines · H L

I have not modified any existing drivers, but instead I threw together a bare-bones module enabling me to make a call to pci_iov_register() and then poke at an SR-IOV adapter's /sys entries for which no driver was loaded.

Quoted 7 lines

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver associated with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify() function. Essentially this suggests to me a model for the PF driver to perform any "global actions" or setup on behalf of VFs before enabling them after which VF drivers could be associated.

Quoted 5 lines

Where would the VF drivers have to be associated? On the "pci_dev" level or on a higher one?

Will all drivers that want to bind to a "VF" device need to be rewritten?

The current model being implemented by my colleagues has separate drivers for the PF (aka native) and VF devices. I don't personally believe this is the correct path, but I'm reserving judgement until I see some code.

I don't think we really know what the One True Usage model is for VF devices. Chris Wright has some ideas, I have some ideas and Yu Zhao has some ideas. I bet there's other people who have other ideas too.

Signature · 4 lines
Matthew Wilcox				Intel Open Source Technology Centre
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."
#202 lore raw
In reply to Greg KH
Quoted 18 lines · Greg KH
Quoted 1 line · H L

I have not modified any existing drivers, but instead

I threw together

Quoted 1 line

a bare-bones module enabling me to make a call to

pci_iov_register()

Quoted 1 line

and then poke at an SR-IOV adapter's /sys entries

for which no driver

Quoted 3 lines

was loaded.

It appears from my perusal thus far that drivers using

these new

Quoted 1 line

SR-IOV patches will require modification; i.e. the

driver associated

Quoted 1 line

with the Physical Function (PF) will be required to

make the

Quoted 1 line

pci_iov_register() call along with the requisite

notify() function.

Quoted 1 line

Essentially this suggests to me a model for the PF

Quoted 8 lines

driver to perform

Quoted 1 line

any "global actions" or setup on behalf of

VFs before enabling them

Quoted 1 line

after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the "pci_dev"
level or on a higher one?

I have not yet fully grocked Yu Zhao's model to answer this. That said, I would *hope* to find it on the "pci_dev" level.

Quoted 3 lines

Will all drivers that want to bind to a "VF" device need to be
rewritten?

Not necessarily, or perhaps minimally; depends on hardware/firmware and actions the driver wants to take. An example here might assist. Let's just say someone has created, oh, I don't know, maybe an SR-IOV NIC. Now, for 'general' I/O operations to pass network traffic back and forth there would ideally be no difference in the actions and therefore behavior of a PF driver and a VF driver. But, what do you do in the instance a VF wants to change link-speed? As that physical characteristic affects all VFs, how do you handle that? This is where the hardware/firmware implementation part comes to play. If a VF driver performs some actions to initiate the change in link speed, the logic in the adapter could be anything like:

1. Acknowledge the request as if it were really done, but effectively ignore it. The Independent Hardware Vendor (IHV) might dictate that if you want to change any "global" characteristics of an adapter, you may only do so via the PF driver. Granted, this, depending on the device class, may just not be acceptable.

2. Acknowledge the request and then trigger an interrupt to the PF driver to have it assist. The PF driver might then just set the new link-speed, or it could result in a PF driver communicating by some mechanism to all of the VF driver instances that this change of link-speed was requested.

3. Acknowledge the request and perform inner PF and VF communication of this event within the logic of the card (e.g. to "vote" on whether or not to perform this action) with interrupts and associated status delivered to all PF and VF drivers.

The list goes on.

Quoted 8 lines
Quoted 2 lines

I have so far only seen Yu Zhao's

"7-patch" set. I've not yet looked

Quoted 1 line

at his subsequently tendered "15-patch" set

so I don't know what has

Quoted 1 line

changed. The hardware/firmware implementation for

any given SR-IOV

Quoted 1 line

compatible device, will determine the extent of

Quoted 8 lines

differences required

Quoted 1 line

between a PF driver and a VF driver.

Yeah, that's what I'm worried/curious about. Without seeing the code
for such a driver, how can we properly evaluate if this infrastructure
is the correct one and the proper way to do all of this?

As the example above demonstrates, that's a tough question to answer. Ideally, in my view, there would only be one driver written per SR-IOV device and it would contain the logic to "do the right things" based on whether its running as a PF or VF with that determination easily accomplished by testing the existence of the SR-IOV extended capability. Then, in an effort to minimize (if not eliminate) the complexities of driver-to-driver actions for fielding "global events", contain as much of the logic as is possible within the adapter. Minimizing the efforts required for the device driver writers in my opinion paves the way to greater adoption of this technology.

#203 lore raw
Quoted 26 lines · H L
Quoted 23 lines · Greg KH
Quoted 1 line · H L

I have not modified any existing drivers, but instead

I threw together

Quoted 1 line

a bare-bones module enabling me to make a call to

pci_iov_register()

Quoted 1 line

and then poke at an SR-IOV adapter's /sys entries

for which no driver

Quoted 3 lines

was loaded.

It appears from my perusal thus far that drivers using

these new

Quoted 1 line

SR-IOV patches will require modification; i.e. the

driver associated

Quoted 1 line

with the Physical Function (PF) will be required to

make the

Quoted 1 line

pci_iov_register() call along with the requisite

notify() function.

Quoted 1 line

Essentially this suggests to me a model for the PF

driver to perform

Quoted 1 line

any "global actions" or setup on behalf of

VFs before enabling them

Quoted 1 line

after which VF drivers could be associated.

Quoted 7 lines
Quoted 3 lines

Where would the VF drivers have to be associated? On the "pci_dev"
level or on a higher one?

I have not yet fully grocked Yu Zhao's model to answer this. That said, I would *hope* to find it on the "pci_dev" level.

Me too.

Quoted 4 lines
Quoted 3 lines

Will all drivers that want to bind to a "VF" device need to be
rewritten?

Quoted 11 lines

Not necessarily, or perhaps minimally; depends on hardware/firmware and actions the driver wants to take. An example here might assist. Let's just say someone has created, oh, I don't know, maybe an SR-IOV NIC. Now, for 'general' I/O operations to pass network traffic back and forth there would ideally be no difference in the actions and therefore behavior of a PF driver and a VF driver. But, what do you do in the instance a VF wants to change link-speed? As that physical characteristic affects all VFs, how do you handle that? This is where the hardware/firmware implementation part comes to play. If a VF driver performs some actions to initiate the change in link speed, the logic in the adapter could be anything like:

<snip>

Yes, I agree that all of this needs to be done, somehow.

It's that "somehow" that I am interested in trying to see how it works out.

Quoted 18 lines
Quoted 8 lines
Quoted 2 lines

I have so far only seen Yu Zhao's

"7-patch" set. I've not yet looked

Quoted 1 line

at his subsequently tendered "15-patch" set

so I don't know what has

Quoted 1 line

changed. The hardware/firmware implementation for

any given SR-IOV

Quoted 1 line

compatible device, will determine the extent of

Quoted 8 lines

differences required

Quoted 1 line

between a PF driver and a VF driver.

Yeah, that's what I'm worried/curious about. Without seeing the code
for such a driver, how can we properly evaluate if this infrastructure
is the correct one and the proper way to do all of this?

Quoted 10 lines

As the example above demonstrates, that's a tough question to answer. Ideally, in my view, there would only be one driver written per SR-IOV device and it would contain the logic to "do the right things" based on whether its running as a PF or VF with that determination easily accomplished by testing the existence of the SR-IOV extended
capability. Then, in an effort to minimize (if not eliminate) the
complexities of driver-to-driver actions for fielding "global events", contain as much of the logic as is possible within the adapter. Minimizing the efforts required for the device driver writers in my opinion paves the way to greater adoption of this technology.

Yes, making things easier is the key here.

Perhaps some of this could be hidden with a new bus type for these kinds of devices? Or a "virtual" bus of pci devices that the original SR-IOV device creates that corrispond to the individual virtual PCI devices? If that were the case, then it might be a lot easier in the end.

thanks,

greg k-h

#204 lore raw
Quoted 95 lines

Subject: Re: [PATCH 0/16 v6] PCI: Linux kernel SR-IOV support

Quoted 26 lines · H L
Quoted 23 lines · Greg KH
Quoted 1 line · H L

I have not modified any existing drivers, but instead

I threw together

Quoted 1 line

a bare-bones module enabling me to make a call to

pci_iov_register()

Quoted 1 line

and then poke at an SR-IOV adapter's /sys entries

for which no driver

Quoted 3 lines

was loaded.

It appears from my perusal thus far that drivers using

these new

Quoted 1 line

SR-IOV patches will require modification; i.e. the

driver associated

Quoted 1 line

with the Physical Function (PF) will be required to

make the

Quoted 1 line

pci_iov_register() call along with the requisite

notify() function.

Quoted 1 line

Essentially this suggests to me a model for the PF

driver to perform

Quoted 1 line

any "global actions" or setup on behalf of

VFs before enabling them

Quoted 1 line

after which VF drivers could be associated.

Quoted 7 lines
Quoted 3 lines

Where would the VF drivers have to be associated? On the "pci_dev"
level or on a higher one?

I have not yet fully grocked Yu Zhao's model to answer this. That said, I would *hope* to find it on the "pci_dev" level.

Me too.

Quoted 4 lines
Quoted 3 lines

Will all drivers that want to bind to a "VF" device need to be
rewritten?

Quoted 8 lines

Not necessarily, or perhaps minimally; depends on hardware/firmware and actions the driver wants to take. An example here might assist. Let's just say someone has created, oh, I don't know, maybe an SR-IOV NIC. Now, for 'general' I/O operations to pass network traffic back and forth there would ideally be no difference in the actions and therefore behavior of a PF driver and a VF driver. But, what do you do in the instance a VF wants to change link-speed? As that physical characteristic affects all VFs, how do you handle that? This is

where

Quoted 2 lines

the hardware/firmware implementation part comes to play. If a VF driver performs some actions to initiate the change in link speed,

the

Quoted 1 line

logic in the adapter could be anything like:

<snip>

Yes, I agree that all of this needs to be done, somehow.

It's that "somehow" that I am interested in trying to see how it works out.

Quoted 18 lines
Quoted 8 lines
Quoted 2 lines

I have so far only seen Yu Zhao's

"7-patch" set. I've not yet looked

Quoted 1 line

at his subsequently tendered "15-patch" set

so I don't know what has

Quoted 1 line

changed. The hardware/firmware implementation for

any given SR-IOV

Quoted 1 line

compatible device, will determine the extent of

Quoted 8 lines

differences required

Quoted 1 line

between a PF driver and a VF driver.

Yeah, that's what I'm worried/curious about. Without seeing the code
for such a driver, how can we properly evaluate if this infrastructure
is the correct one and the proper way to do all of this?

Quoted 2 lines

As the example above demonstrates, that's a tough question to answer. Ideally, in my view, there would only be one driver written per SR-

IOV

Quoted 5 lines

device and it would contain the logic to "do the right things" based on whether its running as a PF or VF with that determination easily accomplished by testing the existence of the SR-IOV extended
capability. Then, in an effort to minimize (if not eliminate) the
complexities of driver-to-driver actions for fielding "global

events",

Quoted 3 lines

contain as much of the logic as is possible within the adapter. Minimizing the efforts required for the device driver writers in my opinion paves the way to greater adoption of this technology.

Quoted 7 lines

Yes, making things easier is the key here.

Perhaps some of this could be hidden with a new bus type for these kinds
of devices? Or a "virtual" bus of pci devices that the original SR-IOV device creates that corrispond to the individual virtual PCI devices? If that were the case, then it might be a lot easier in the end.

I think a standard communication channel in Linux for SR-IOV devices would be a good start, and help to adopt the technology. Something like the virtual bus you are describing. It means that vendors do not need to write their own communication channel in the drivers. It would need to have well defined APIs though, as I guess that devices will have very different capabilities and hardware implementations for PFs and VFs, and so they might have very different events and information to propagate.

Anna

#205 lore raw
Quoted 6 lines · Anna
Quoted 5 lines

Perhaps some of this could be hidden with a new bus type for these kinds
of devices? Or a "virtual" bus of pci devices that the original SR-IOV device creates that corrispond to the individual virtual PCI devices? If that were the case, then it might be a lot easier in the end.

Quoted 8 lines

I think a standard communication channel in Linux for SR-IOV devices would be a good start, and help to adopt the technology. Something like the virtual bus you are describing. It means that vendors do not need to write their own communication channel in the drivers. It would need to have well defined APIs though, as I guess that devices will have very different capabilities and hardware implementations for PFs and VFs, and so they might have very different events and information to propagate.

That would be good to standardize on. Have patches?

thanks,

greg k-h

author
#206 lore raw
In reply to Greg KH
Quoted 25 lines · Greg KH
Quoted 24 lines · H L
Quoted 22 lines · Greg KH
Quoted 1 line · H L

I have not modified any existing drivers, but instead

I threw together

Quoted 1 line

a bare-bones module enabling me to make a call to

pci_iov_register()

Quoted 1 line

and then poke at an SR-IOV adapter's /sys entries

for which no driver

Quoted 3 lines

was loaded.

It appears from my perusal thus far that drivers using

these new

Quoted 1 line

SR-IOV patches will require modification; i.e. the

driver associated

Quoted 1 line

with the Physical Function (PF) will be required to

make the

Quoted 1 line

pci_iov_register() call along with the requisite

notify() function.

Quoted 1 line

Essentially this suggests to me a model for the PF

driver to perform

Quoted 1 line

any "global actions" or setup on behalf of

VFs before enabling them

Quoted 1 line

after which VF drivers could be associated.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Where would the VF drivers have to be associated? On the "pci_dev"
level or on a higher one?

I have not yet fully grocked Yu Zhao's model to answer this. That said, I would *hope* to find it on the "pci_dev" level.

Me too.

VF is kind of lightweight PCI device, and it's represented by "struct pci_dev". VF driver bounds to the "pci_dev" and works in the same way as other drivers.

Quoted 16 lines
Quoted 4 lines
Quoted 4 lines

Will all drivers that want to bind to a "VF" device need to be
rewritten?

Quoted 11 lines

Not necessarily, or perhaps minimally; depends on hardware/firmware and actions the driver wants to take. An example here might assist. Let's just say someone has created, oh, I don't know, maybe an SR-IOV NIC. Now, for 'general' I/O operations to pass network traffic back and forth there would ideally be no difference in the actions and therefore behavior of a PF driver and a VF driver. But, what do you do in the instance a VF wants to change link-speed? As that physical characteristic affects all VFs, how do you handle that? This is where the hardware/firmware implementation part comes to play. If a VF driver performs some actions to initiate the change in link speed, the logic in the adapter could be anything like:

Quoted 6 lines

<snip>

Yes, I agree that all of this needs to be done, somehow.

It's that "somehow" that I am interested in trying to see how it works out.

This is device specific part. VF driver is free to do what it wants to do with device specific registers and resources, and wouldn't concern us as far as it behaves as PCI device driver.

Quoted 27 lines
Quoted 16 lines
Quoted 7 lines
Quoted 2 lines

I have so far only seen Yu Zhao's

"7-patch" set. I've not yet looked

Quoted 1 line

at his subsequently tendered "15-patch" set

so I don't know what has

Quoted 1 line

changed. The hardware/firmware implementation for

any given SR-IOV

Quoted 8 lines
Quoted 1 line

compatible device, will determine the extent of

differences required

Quoted 1 line

between a PF driver and a VF driver.

Yeah, that's what I'm worried/curious about. Without seeing the code
for such a driver, how can we properly evaluate if this infrastructure
is the correct one and the proper way to do all of this?

Quoted 10 lines

As the example above demonstrates, that's a tough question to answer. Ideally, in my view, there would only be one driver written per SR-IOV device and it would contain the logic to "do the right things" based on whether its running as a PF or VF with that determination easily accomplished by testing the existence of the SR-IOV extended
capability. Then, in an effort to minimize (if not eliminate) the
complexities of driver-to-driver actions for fielding "global events", contain as much of the logic as is possible within the adapter. Minimizing the efforts required for the device driver writers in my opinion paves the way to greater adoption of this technology.

Quoted 6 lines

Yes, making things easier is the key here.

Perhaps some of this could be hidden with a new bus type for these kinds of devices? Or a "virtual" bus of pci devices that the original SR-IOV device creates that corrispond to the individual virtual PCI devices? If that were the case, then it might be a lot easier in the end.

PCI SIG only defines SR-IOV at PCI level, we can't predict what the hardware vendors would implement at device specific logic level.

An example of SR-IOV NIC: PF may not have network functionality, it only controls VFs. Because people only want to use VFs in virtual machines, they don't need network functionality in the environment (e.g. hypervisor) where PF resides.

Thanks,
Yu

author
#207 lore raw
In reply to Greg KH
Quoted 25 lines · Greg KH
Quoted 24 lines · H L
Quoted 22 lines · Greg KH
Quoted 1 line · H L

I have not modified any existing drivers, but instead

I threw together

Quoted 1 line

a bare-bones module enabling me to make a call to

pci_iov_register()

Quoted 1 line

and then poke at an SR-IOV adapter's /sys entries

for which no driver

Quoted 3 lines

was loaded.

It appears from my perusal thus far that drivers using

these new

Quoted 1 line

SR-IOV patches will require modification; i.e. the

driver associated

Quoted 1 line

with the Physical Function (PF) will be required to

make the

Quoted 1 line

pci_iov_register() call along with the requisite

notify() function.

Quoted 1 line

Essentially this suggests to me a model for the PF

driver to perform

Quoted 1 line

any "global actions" or setup on behalf of

VFs before enabling them

Quoted 1 line

after which VF drivers could be associated.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Where would the VF drivers have to be associated? On the "pci_dev"
level or on a higher one?

I have not yet fully grocked Yu Zhao's model to answer this. That said, I would *hope* to find it on the "pci_dev" level.

Me too.

VF is kind of lightweight PCI device, and it's represented by "struct pci_dev". VF driver bounds to the "pci_dev" and works in the same way as other drivers.

Quoted 16 lines
Quoted 4 lines
Quoted 4 lines

Will all drivers that want to bind to a "VF"
device need to be
rewritten?

Quoted 11 lines

Not necessarily, or perhaps minimally; depends on hardware/firmware and actions the driver wants to take. An example here might assist. Let's just say someone has created, oh, I don't know, maybe an SR-IOV NIC. Now, for 'general' I/O operations to pass network traffic back and forth there would ideally be no difference in the actions and therefore behavior of a PF driver and a VF driver. But, what do you do in the instance a VF wants to change link-speed? As that physical characteristic affects all VFs, how do you handle that? This is where the hardware/firmware implementation part comes to play. If a VF driver performs some actions to initiate the change in link speed, the logic in the adapter could be anything like:

Quoted 6 lines

<snip>

Yes, I agree that all of this needs to be done, somehow.

It's that "somehow" that I am interested in trying to see how it works out.

This is device specific part. VF driver is free to do what it wants to do with device specific registers and resources, and wouldn't concern us as far as it behaves as PCI device driver.

Quoted 26 lines
Quoted 15 lines
Quoted 6 lines
Quoted 2 lines

I have so far only seen Yu Zhao's

"7-patch" set. I've not yet looked

Quoted 1 line

at his subsequently tendered "15-patch" set

so I don't know what has

Quoted 1 line

changed. The hardware/firmware implementation for

Quoted 8 lines

any given SR-IOV

Quoted 1 line

compatible device, will determine the extent of

differences required

Quoted 1 line

between a PF driver and a VF driver.

Yeah, that's what I'm worried/curious about. Without seeing the code for such a driver, how can we properly evaluate if this infrastructure
is the correct one and the proper way to do all of this?

Quoted 10 lines

As the example above demonstrates, that's a tough question to answer. Ideally, in my view, there would only be one driver written per SR-IOV device and it would contain the logic to "do the right things" based on whether its running as a PF or VF with that determination easily accomplished by testing the existence of the SR-IOV extended
capability. Then, in an effort to minimize (if not eliminate) the
complexities of driver-to-driver actions for fielding "global events", contain as much of the logic as is possible within the adapter. Minimizing the efforts required for the device driver writers in my opinion paves the way to greater adoption of this technology.

Quoted 6 lines

Yes, making things easier is the key here.

Perhaps some of this could be hidden with a new bus type for these kinds of devices? Or a "virtual" bus of pci devices that the original SR-IOV device creates that corrispond to the individual virtual PCI devices? If that were the case, then it might be a lot easier in the end.

PCI SIG only defines SR-IOV at PCI level, we can't predict what the hardware vendors would implement at device specific logic level.

An example of SR-IOV NIC: PF may not have network functionality, it only controls VFs. Because people only want to use VFs in virtual machines, they don't need network functionality in the environment (e.g. hypervisor) where PF resides.

Thanks,
Yu

#208 lore raw
In reply to H L
Quoted 26 lines · H L
Quoted 23 lines · Greg KH
Quoted 1 line · H L

I have not modified any existing drivers, but instead

I threw together

Quoted 1 line

a bare-bones module enabling me to make a call to

pci_iov_register()

Quoted 1 line

and then poke at an SR-IOV adapter's /sys entries

for which no driver

Quoted 3 lines

was loaded.

It appears from my perusal thus far that drivers using

these new

Quoted 1 line

SR-IOV patches will require modification; i.e. the

driver associated

Quoted 1 line

with the Physical Function (PF) will be required to

make the

Quoted 1 line

pci_iov_register() call along with the requisite

notify() function.

Quoted 1 line

Essentially this suggests to me a model for the PF

driver to perform

Quoted 1 line

any "global actions" or setup on behalf of

VFs before enabling them

Quoted 1 line

after which VF drivers could be associated.

Quoted 7 lines
Quoted 3 lines

Where would the VF drivers have to be associated? On the "pci_dev"
level or on a higher one?

I have not yet fully grocked Yu Zhao's model to answer this. That said, I would *hope* to find it on the "pci_dev" level.

Me too.

Quoted 4 lines
Quoted 3 lines

Will all drivers that want to bind to a "VF" device need to be
rewritten?

Quoted 11 lines

Not necessarily, or perhaps minimally; depends on hardware/firmware and actions the driver wants to take. An example here might assist. Let's just say someone has created, oh, I don't know, maybe an SR-IOV NIC. Now, for 'general' I/O operations to pass network traffic back and forth there would ideally be no difference in the actions and therefore behavior of a PF driver and a VF driver. But, what do you do in the instance a VF wants to change link-speed? As that physical characteristic affects all VFs, how do you handle that? This is where the hardware/firmware implementation part comes to play. If a VF driver performs some actions to initiate the change in link speed, the logic in the adapter could be anything like:

<snip>

Yes, I agree that all of this needs to be done, somehow.

It's that "somehow" that I am interested in trying to see how it works out.

Quoted 18 lines
Quoted 8 lines
Quoted 2 lines

I have so far only seen Yu Zhao's

"7-patch" set. I've not yet looked

Quoted 1 line

at his subsequently tendered "15-patch" set

so I don't know what has

Quoted 1 line

changed. The hardware/firmware implementation for

any given SR-IOV

Quoted 1 line

compatible device, will determine the extent of

Quoted 8 lines

differences required

Quoted 1 line

between a PF driver and a VF driver.

Yeah, that's what I'm worried/curious about. Without seeing the code
for such a driver, how can we properly evaluate if this infrastructure
is the correct one and the proper way to do all of this?

Quoted 10 lines

As the example above demonstrates, that's a tough question to answer. Ideally, in my view, there would only be one driver written per SR-IOV device and it would contain the logic to "do the right things" based on whether its running as a PF or VF with that determination easily accomplished by testing the existence of the SR-IOV extended
capability. Then, in an effort to minimize (if not eliminate) the
complexities of driver-to-driver actions for fielding "global events", contain as much of the logic as is possible within the adapter. Minimizing the efforts required for the device driver writers in my opinion paves the way to greater adoption of this technology.

Yes, making things easier is the key here.

Perhaps some of this could be hidden with a new bus type for these kinds of devices? Or a "virtual" bus of pci devices that the original SR-IOV device creates that corrispond to the individual virtual PCI devices? If that were the case, then it might be a lot easier in the end.

thanks,

greg k-h

#209 lore raw
In reply to Greg KH
Quoted 18 lines · Greg KH
Quoted 1 line · H L

I have not modified any existing drivers, but instead

I threw together

Quoted 1 line

a bare-bones module enabling me to make a call to

pci_iov_register()

Quoted 1 line

and then poke at an SR-IOV adapter's /sys entries

for which no driver

Quoted 3 lines

was loaded.

It appears from my perusal thus far that drivers using

these new

Quoted 1 line

SR-IOV patches will require modification; i.e. the

driver associated

Quoted 1 line

with the Physical Function (PF) will be required to

make the

Quoted 1 line

pci_iov_register() call along with the requisite

notify() function.

Quoted 1 line

Essentially this suggests to me a model for the PF

Quoted 8 lines

driver to perform

Quoted 1 line

any "global actions" or setup on behalf of

VFs before enabling them

Quoted 1 line

after which VF drivers could be associated.

Where would the VF drivers have to be associated? On the "pci_dev"
level or on a higher one?

I have not yet fully grocked Yu Zhao's model to answer this. That said, I would *hope* to find it on the "pci_dev" level.

Quoted 3 lines

Will all drivers that want to bind to a "VF" device need to be
rewritten?

Not necessarily, or perhaps minimally; depends on hardware/firmware and actions the driver wants to take. An example here might assist. Let's just say someone has created, oh, I don't know, maybe an SR-IOV NIC. Now, for 'general' I/O operations to pass network traffic back and forth there would ideally be no difference in the actions and therefore behavior of a PF driver and a VF driver. But, what do you do in the instance a VF wants to change link-speed? As that physical characteristic affects all VFs, how do you handle that? This is where the hardware/firmware implementation part comes to play. If a VF driver performs some actions to initiate the change in link speed, the logic in the adapter could be anything like:

1. Acknowledge the request as if it were really done, but effectively ignore it. The Independent Hardware Vendor (IHV) might dictate that if you want to change any "global" characteristics of an adapter, you may only do so via the PF driver. Granted, this, depending on the device class, may just not be acceptable.

2. Acknowledge the request and then trigger an interrupt to the PF driver to have it assist. The PF driver might then just set the new link-speed, or it could result in a PF driver communicating by some mechanism to all of the VF driver instances that this change of link-speed was requested.

3. Acknowledge the request and perform inner PF and VF communication of this event within the logic of the card (e.g. to "vote" on whether or not to perform this action) with interrupts and associated status delivered to all PF and VF drivers.

The list goes on.

Quoted 8 lines
Quoted 2 lines

I have so far only seen Yu Zhao's

"7-patch" set. I've not yet looked

Quoted 1 line

at his subsequently tendered "15-patch" set

so I don't know what has

Quoted 1 line

changed. The hardware/firmware implementation for

any given SR-IOV

Quoted 1 line

compatible device, will determine the extent of

Quoted 8 lines

differences required

Quoted 1 line

between a PF driver and a VF driver.

Yeah, that's what I'm worried/curious about. Without seeing the code
for such a driver, how can we properly evaluate if this infrastructure
is the correct one and the proper way to do all of this?

As the example above demonstrates, that's a tough question to answer. Ideally, in my view, there would only be one driver written per SR-IOV device and it would contain the logic to "do the right things" based on whether its running as a PF or VF with that determination easily accomplished by testing the existence of the SR-IOV extended capability. Then, in an effort to minimize (if not eliminate) the complexities of driver-to-driver actions for fielding "global events", contain as much of the logic as is possible within the adapter. Minimizing the efforts required for the device driver writers in my opinion paves the way to greater adoption of this technology.

#210 lore raw
In reply to Greg KH

I have not modified any existing drivers, but instead I threw together a bare-bones module enabling me to make a call to pci_iov_register() and then poke at an SR-IOV adapter's /sys entries for which no driver was loaded.

It appears from my perusal thus far that drivers using these new SR-IOV patches will require modification; i.e. the driver associated with the Physical Function (PF) will be required to make the pci_iov_register() call along with the requisite notify() function. Essentially this suggests to me a model for the PF driver to perform any "global actions" or setup on behalf of VFs before enabling them after which VF drivers could be associated.

I have so far only seen Yu Zhao's "7-patch" set. I've not yet looked at his subsequently tendered "15-patch" set so I don't know what has changed. The hardware/firmware implementation for any given SR-IOV compatible device, will determine the extent of differences required between a PF driver and a VF driver.

--
Lance Hartmann

Quoted 30 lines · Greg KH

Date: Thursday, November 6, 2008, 9:43 AM

Quoted 2 lines · H L

Greetings (from a new lurker to the list),

Welcome!

Quoted 1 line

To your question Greg, "yes" and "sort

of" ;-). I have started taking

Quoted 1 line

a look at these patches with a strong interest in

understanding how

Quoted 1 line

they work. I've built a kernel with them and

tried out a few things

Quoted 1 line

with real SR-IOV hardware.

Did you have to modify individual drivers to take advantage of this
code? It looks like the core code will run on this type of hardware,
but there seems to be no real advantage until a driver is modified to
use it, right?

Or am I missing some great advantage to having this code without
modified drivers?

thanks,

greg k-h

#211 lore raw

git repository for SR-IOV development?

In reply to Greg KH

Has anyone initiated or given consideration to the creation of a git repository (say, on kernel.org) for SR-IOV development?

--
Lance Hartmann

#212 lore raw

Re: git repository for SR-IOV development?

Quoted 3 lines · H L

Has anyone initiated or given consideration to the creation of a git repository (say, on kernel.org) for SR-IOV development?

Why? It's only a few patches, right? Why would it need a whole new git tree?

thanks,

greg k-h

#213 lore raw

Re: git repository for SR-IOV development?

Quoted 10 lines · Greg KH
Quoted 2 lines · H L

Has anyone initiated or given consideration to the

creation of a git

Quoted 1 line

repository (say, on kernel.org) for SR-IOV

development?

Why? It's only a few patches, right? Why would it need a whole new git
tree?

So as to minimize the time and effort patching a kernel, especially if the tree (and/or hash level) against which the patches were created fails to be specified on a mailing-list. Plus, there appears to be questions raised on how, precisely, the implementation should ultimately be modeled and especially given that, who knows at this point what number of patches will ultimately be submitted? I know I've built the "7-patch" one (painfully, by the way), and I'm aware there's another 15-patch set out there which I've not yet examined.

#214 lore raw

Re: git repository for SR-IOV development?

Quoted 14 lines · H L
Quoted 10 lines · Greg KH
Quoted 2 lines · H L

Has anyone initiated or given consideration to the

creation of a git

Quoted 1 line

repository (say, on kernel.org) for SR-IOV

development?

Why? It's only a few patches, right? Why would it need a whole new git
tree?

Quoted 8 lines

So as to minimize the time and effort patching a kernel, especially if the tree (and/or hash level) against which the patches were created fails to be specified on a mailing-list. Plus, there appears to be questions raised on how, precisely, the implementation should ultimately be modeled and especially given that, who knows at this point what number of patches will ultimately be submitted? I know I've built the "7-patch" one (painfully, by the way), and I'm aware there's another 15-patch set out there which I've not yet examined.

FWIW, the v6 patch series (this thread) applied to both 2.6.28-rc3 and the current Linus tree after a minor tweak to the first patch, as below.

Signature · 31 lines
Simon Horman
  VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
  H: www.vergenet.net/~horms/             W: www.valinux.co.jp/en

From: Yu Zhao <[email protected]>

[PATCH 1/16 v6] PCI: remove unnecessary arg of pci_update_resource()

This cleanup removes unnecessary argument 'struct resource *res' in
pci_update_resource(), so it takes same arguments as other companion
functions (pci_assign_resource(), etc.).

Cc: Alex Chiang <[email protected]>
Cc: Grant Grundler <[email protected]>
Cc: Greg KH <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Jesse Barnes <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Cc: Randy Dunlap <[email protected]>
Cc: Roland Dreier <[email protected]>
Signed-off-by: Yu Zhao <[email protected]>
Upported-by: Simon Horman <[email protected]>

---
 drivers/pci/pci.c       |    4 ++--
 drivers/pci/setup-res.c |    7 ++++---
 include/linux/pci.h     |    2 +-
 3 files changed, 7 insertions(+), 6 deletions(-)

* Fri, 07 Nov 2008 09:05:18 +1100, Simon Horman
  - Minor rediff of include/linux/pci.h section to apply to 2.6.28-rc3
drivers/pci/pci.c +2 -2
index 4db261e..ae62f01 100644
@@ -376,8 +376,8 @@ pci_restore_bars(struct pci_dev *dev)
 		return;
 	}
 
-	for (i = 0; i < numres; i ++)
-		pci_update_resource(dev, &dev->resource[i], i);
+	for (i = 0; i < numres; i++)
+		pci_update_resource(dev, i);
 }
 
 static struct pci_platform_pm_ops *pci_platform_pm;
drivers/pci/setup-res.c +4 -3
index 2dbd96c..b7ca679 100644
@@ -26,11 +26,12 @@
 #include "pci.h"
 
 
-void pci_update_resource(struct pci_dev *dev, struct resource *res, int resno)
+void pci_update_resource(struct pci_dev *dev, int resno)
 {
 	struct pci_bus_region region;
 	u32 new, check, mask;
 	int reg;
+	struct resource *res = dev->resource + resno;
 
 	/*
 	 * Ignore resources for unimplemented BARs and unused resource slots
@@ -162,7 +163,7 @@ int pci_assign_resource(struct pci_dev *dev, int resno)
 	} else {
 		res->flags &= ~IORESOURCE_STARTALIGN;
 		if (resno < PCI_BRIDGE_RESOURCES)
-			pci_update_resource(dev, res, resno);
+			pci_update_resource(dev, resno);
 	}
 
 	return ret;
@@ -197,7 +198,7 @@ int pci_assign_resource_fixed(struct pci_dev *dev, int resno)
 		dev_err(&dev->dev, "BAR %d: can't allocate %s resource %pR\n",
 			resno, res->flags & IORESOURCE_IO ? "I/O" : "mem", res);
 	} else if (resno < PCI_BRIDGE_RESOURCES) {
-		pci_update_resource(dev, res, resno);
+		pci_update_resource(dev, resno);
 	}
 
 	return ret;
include/linux/pci.h +1 -1
index 085187b..43e1fc1 100644
@@ -626,7 +626,7 @@ int pcix_get_mmrbc(struct pci_dev *dev);
 int pcie_set_readrq(struct pci_dev *dev, int rq);
 int pci_reset_function(struct pci_dev *dev);
 int pci_execute_reset_function(struct pci_dev *dev);
-void pci_update_resource(struct pci_dev *dev, struct resource *res, int resno);
+void pci_update_resource(struct pci_dev *dev, int resno);
 int __must_check pci_assign_resource(struct pci_dev *dev, int i);
 int pci_select_bars(struct pci_dev *dev, unsigned long flags);
 
Signature · 1 line
1.5.6.4
#215 lore raw

Re: git repository for SR-IOV development?

In reply to H L
Quoted 14 lines · H L
Quoted 10 lines · Greg KH
Quoted 2 lines · H L

Has anyone initiated or given consideration to the

creation of a git

Quoted 1 line

repository (say, on kernel.org) for SR-IOV

development?

Why? It's only a few patches, right? Why would it need a whole new git
tree?

Quoted 8 lines

So as to minimize the time and effort patching a kernel, especially if the tree (and/or hash level) against which the patches were created fails to be specified on a mailing-list. Plus, there appears to be questions raised on how, precisely, the implementation should ultimately be modeled and especially given that, who knows at this point what number of patches will ultimately be submitted? I know I've built the "7-patch" one (painfully, by the way), and I'm aware there's another 15-patch set out there which I've not yet examined.

FWIW, the v6 patch series (this thread) applied to both 2.6.28-rc3 and the current Linus tree after a minor tweak to the first patch, as below.

Signature · 31 lines
Simon Horman
  VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
  H: www.vergenet.net/~horms/             W: www.valinux.co.jp/en

From: Yu Zhao <[email protected]>

[PATCH 1/16 v6] PCI: remove unnecessary arg of pci_update_resource()

This cleanup removes unnecessary argument 'struct resource *res' in
pci_update_resource(), so it takes same arguments as other companion
functions (pci_assign_resource(), etc.).

Cc: Alex Chiang <[email protected]>
Cc: Grant Grundler <[email protected]>
Cc: Greg KH <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Jesse Barnes <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Cc: Randy Dunlap <[email protected]>
Cc: Roland Dreier <[email protected]>
Signed-off-by: Yu Zhao <[email protected]>
Upported-by: Simon Horman <[email protected]>

---
 drivers/pci/pci.c       |    4 ++--
 drivers/pci/setup-res.c |    7 ++++---
 include/linux/pci.h     |    2 +-
 3 files changed, 7 insertions(+), 6 deletions(-)

* Fri, 07 Nov 2008 09:05:18 +1100, Simon Horman
  - Minor rediff of include/linux/pci.h section to apply to 2.6.28-rc3
drivers/pci/pci.c +2 -2
index 4db261e..ae62f01 100644
@@ -376,8 +376,8 @@ pci_restore_bars(struct pci_dev *dev)
 		return;
 	}
 
-	for (i = 0; i < numres; i ++)
-		pci_update_resource(dev, &dev->resource[i], i);
+	for (i = 0; i < numres; i++)
+		pci_update_resource(dev, i);
 }
 
 static struct pci_platform_pm_ops *pci_platform_pm;
drivers/pci/setup-res.c +4 -3
index 2dbd96c..b7ca679 100644
@@ -26,11 +26,12 @@
 #include "pci.h"
 
 
-void pci_update_resource(struct pci_dev *dev, struct resource *res, int resno)
+void pci_update_resource(struct pci_dev *dev, int resno)
 {
 	struct pci_bus_region region;
 	u32 new, check, mask;
 	int reg;
+	struct resource *res = dev->resource + resno;
 
 	/*
 	 * Ignore resources for unimplemented BARs and unused resource slots
@@ -162,7 +163,7 @@ int pci_assign_resource(struct pci_dev *dev, int resno)
 	} else {
 		res->flags &= ~IORESOURCE_STARTALIGN;
 		if (resno < PCI_BRIDGE_RESOURCES)
-			pci_update_resource(dev, res, resno);
+			pci_update_resource(dev, resno);
 	}
 
 	return ret;
@@ -197,7 +198,7 @@ int pci_assign_resource_fixed(struct pci_dev *dev, int resno)
 		dev_err(&dev->dev, "BAR %d: can't allocate %s resource %pR\n",
 			resno, res->flags & IORESOURCE_IO ? "I/O" : "mem", res);
 	} else if (resno < PCI_BRIDGE_RESOURCES) {
-		pci_update_resource(dev, res, resno);
+		pci_update_resource(dev, resno);
 	}
 
 	return ret;
include/linux/pci.h +1 -1
index 085187b..43e1fc1 100644
@@ -626,7 +626,7 @@ int pcix_get_mmrbc(struct pci_dev *dev);
 int pcie_set_readrq(struct pci_dev *dev, int rq);
 int pci_reset_function(struct pci_dev *dev);
 int pci_execute_reset_function(struct pci_dev *dev);
-void pci_update_resource(struct pci_dev *dev, struct resource *res, int resno);
+void pci_update_resource(struct pci_dev *dev, int resno);
 int __must_check pci_assign_resource(struct pci_dev *dev, int i);
 int pci_select_bars(struct pci_dev *dev, unsigned long flags);
 
Signature · 1 line
1.5.6.4
#216 lore raw

Re: git repository for SR-IOV development?

In reply to H L
Quoted 14 lines · H L
Quoted 10 lines · Greg KH
Quoted 2 lines · H L

Has anyone initiated or given consideration to the

creation of a git

Quoted 1 line

repository (say, on kernel.org) for SR-IOV

development?

Why? It's only a few patches, right? Why would it need a whole new git
tree?

Quoted 8 lines

So as to minimize the time and effort patching a kernel, especially if the tree (and/or hash level) against which the patches were created fails to be specified on a mailing-list. Plus, there appears to be questions raised on how, precisely, the implementation should ultimately be modeled and especially given that, who knows at this point what number of patches will ultimately be submitted? I know I've built the "7-patch" one (painfully, by the way), and I'm aware there's another 15-patch set out there which I've not yet examined.

It's a mere 7 or 15 patches, you don't need a whole git tree for something small like that.

Especially as there only seems to be one developer doing real work...

thanks,

greg k-h

#217 lore raw

Re: git repository for SR-IOV development?

Hello Lance,

Thanks for your interest in SR-IOV. As Greg said we can't have a git tree for the change, but you are welcome to ask any question here and I also will keep you informed if there is any update on the SR-IOV patches.

Thanks,
Yu

Quoted 34 lines · Greg KH
Quoted 12 lines · H L
Quoted 9 lines · Greg KH
Quoted 1 line · H L

Has anyone initiated or given consideration to the

creation of a git

Quoted 1 line

repository (say, on kernel.org) for SR-IOV

development?

Why? It's only a few patches, right? Why would it need a whole new git
tree?

Quoted 8 lines

So as to minimize the time and effort patching a kernel, especially if the tree (and/or hash level) against which the patches were created fails to be specified on a mailing-list. Plus, there appears to be questions raised on how, precisely, the implementation should ultimately be modeled and especially given that, who knows at this point what number of patches will ultimately be submitted? I know I've built the "7-patch" one (painfully, by the way), and I'm aware there's another 15-patch set out there which I've not yet examined.

It's a mere 7 or 15 patches, you don't need a whole git tree for something small like that.

Especially as there only seems to be one developer doing real work...

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#218 lore raw

Re: git repository for SR-IOV development?

In reply to Greg KH

Hello Lance,

Thanks for your interest in SR-IOV. As Greg said we can't have a git tree for the change, but you are welcome to ask any question here and I also will keep you informed if there is any update on the SR-IOV patches.

Thanks,
Yu

Quoted 34 lines · Greg KH
Quoted 12 lines · H L
Quoted 9 lines · Greg KH
Quoted 1 line · H L

Has anyone initiated or given consideration to the

creation of a git

Quoted 1 line

repository (say, on kernel.org) for SR-IOV

development?

Why? It's only a few patches, right? Why would it need a whole new git
tree?

Quoted 8 lines

So as to minimize the time and effort patching a kernel, especially if the tree (and/or hash level) against which the patches were created fails to be specified on a mailing-list. Plus, there appears to be questions raised on how, precisely, the implementation should ultimately be modeled and especially given that, who knows at this point what number of patches will ultimately be submitted? I know I've built the "7-patch" one (painfully, by the way), and I'm aware there's another 15-patch set out there which I've not yet examined.

It's a mere 7 or 15 patches, you don't need a whole git tree for something small like that.

Especially as there only seems to be one developer doing real work...

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#219 lore raw

Re: git repository for SR-IOV development?

In reply to H L
Quoted 14 lines · H L
Quoted 10 lines · Greg KH
Quoted 2 lines · H L

Has anyone initiated or given consideration to the

creation of a git

Quoted 1 line

repository (say, on kernel.org) for SR-IOV

development?

Why? It's only a few patches, right? Why would it need a whole new git
tree?

Quoted 8 lines

So as to minimize the time and effort patching a kernel, especially if the tree (and/or hash level) against which the patches were created fails to be specified on a mailing-list. Plus, there appears to be questions raised on how, precisely, the implementation should ultimately be modeled and especially given that, who knows at this point what number of patches will ultimately be submitted? I know I've built the "7-patch" one (painfully, by the way), and I'm aware there's another 15-patch set out there which I've not yet examined.

It's a mere 7 or 15 patches, you don't need a whole git tree for something small like that.

Especially as there only seems to be one developer doing real work...

thanks,

greg k-h

#220 lore raw

Re: git repository for SR-IOV development?

In reply to Greg KH
Quoted 10 lines · Greg KH
Quoted 2 lines · H L

Has anyone initiated or given consideration to the

creation of a git

Quoted 1 line

repository (say, on kernel.org) for SR-IOV

development?

Why? It's only a few patches, right? Why would it need a whole new git
tree?

So as to minimize the time and effort patching a kernel, especially if the tree (and/or hash level) against which the patches were created fails to be specified on a mailing-list. Plus, there appears to be questions raised on how, precisely, the implementation should ultimately be modeled and especially given that, who knows at this point what number of patches will ultimately be submitted? I know I've built the "7-patch" one (painfully, by the way), and I'm aware there's another 15-patch set out there which I've not yet examined.

#221 lore raw

Re: git repository for SR-IOV development?

In reply to H L
Quoted 3 lines · H L

Has anyone initiated or given consideration to the creation of a git repository (say, on kernel.org) for SR-IOV development?

Why? It's only a few patches, right? Why would it need a whole new git tree?

thanks,

greg k-h

#222 lore raw

git repository for SR-IOV development?

In reply to Greg KH

Has anyone initiated or given consideration to the creation of a git repository (say, on kernel.org) for SR-IOV development?

--
Lance Hartmann

#223 lore raw
In reply to Greg KH

Greetings (from a new lurker to the list),

To your question Greg, "yes" and "sort of" ;-). I have started taking a look at these patches with a strong interest in understanding how they work. I've built a kernel with them and tried out a few things with real SR-IOV hardware.

--
Lance Hartmann

Quoted 14 lines · Greg KH

Is there any actual users of this API around yet? How was it tested as
there is no hardware to test on? Which drivers are going to have to be
rewritten to take advantage of this new interface?

thanks,

greg k-h
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/virtualization

author
#224 lore raw
In reply to Greg KH
Quoted 9 lines · Greg KH
Quoted 7 lines · Yu Zhao

Greetings,

Following patches are intended to support SR-IOV capability in the Linux kernel. With these patches, people can turn a PCI device with the capability into multiple ones from software perspective, which will benefit KVM and achieve other purposes such as QoS, security, and etc.

Quoted 3 lines

Is there any actual users of this API around yet? How was it tested as there is no hardware to test on? Which drivers are going to have to be rewritten to take advantage of this new interface?

Yes, the API is used by Intel, HP, NextIO and some other anonymous companies as they rise questions and send me feedback. I haven't seen their works but I guess some of drivers using SR-IOV API are going to be released soon.

My test was done with Intel 82576 Gigabit Ethernet Controller. The product brief is at
http://download.intel.com/design/network/ProdBrf/320025.pdf and the spec is available at
http://download.intel.com/design/network/datashts/82576_Datasheet_v2p1.pdf

Regards,
Yu

author
#225 lore raw
In reply to Greg KH
Quoted 9 lines · Greg KH
Quoted 7 lines · Yu Zhao

Greetings,

Following patches are intended to support SR-IOV capability in the Linux kernel. With these patches, people can turn a PCI device with the capability into multiple ones from software perspective, which will benefit KVM and achieve other purposes such as QoS, security, and etc.

Quoted 3 lines

Is there any actual users of this API around yet? How was it tested as there is no hardware to test on? Which drivers are going to have to be rewritten to take advantage of this new interface?

Yes, the API is used by Intel, HP, NextIO and some other anonymous companies as they rise questions and send me feedback. I haven't seen their works but I guess some of drivers using SR-IOV API are going to be released soon.

My test was done with Intel 82576 Gigabit Ethernet Controller. The product brief is at http://download.intel.com/design/network/ProdBrf/320025.pdf and the spec is available at http://download.intel.com/design/network/datashts/82576_Datasheet_v2p1.pdf

Regards,
Yu

#226 lore raw
Quoted 9 lines · Yu
Quoted 8 lines · Greg KH
Quoted 7 lines · Yu Zhao

Greetings,

Following patches are intended to support SR-IOV capability in the Linux kernel. With these patches, people can turn a PCI device with the capability into multiple ones from software perspective, which will benefit KVM and achieve other purposes such as QoS, security, and etc.

Quoted 8 lines
Quoted 3 lines

Is there any actual users of this API around yet? How was it tested as there is no hardware to test on? Which drivers are going to have to be rewritten to take advantage of this new interface?

Yes, the API is used by Intel, HP, NextIO and some other anonymous companies as they rise questions and send me feedback. I haven't seen their works but I guess some of drivers using SR-IOV API are going to be released soon.

Well, we can't merge infrastructure without seeing the users of that infrastructure, right?

Quoted 4 lines

My test was done with Intel 82576 Gigabit Ethernet Controller. The product brief is at http://download.intel.com/design/network/ProdBrf/320025.pdf and the spec is available at
http://download.intel.com/design/network/datashts/82576_Datasheet_v2p1.pdf

Cool, do you have that driver we can see?

How does it interact and handle the kvm and xen issues that have been posted?

thanks,

greg k-h

#227 lore raw
In reply to Zhao, Yu
Quoted 9 lines · Yu
Quoted 8 lines · Greg KH
Quoted 7 lines · Yu Zhao

Greetings,

Following patches are intended to support SR-IOV capability in the Linux kernel. With these patches, people can turn a PCI device with the capability into multiple ones from software perspective, which will benefit KVM and achieve other purposes such as QoS, security, and etc.

Quoted 8 lines
Quoted 3 lines

Is there any actual users of this API around yet? How was it tested as there is no hardware to test on? Which drivers are going to have to be rewritten to take advantage of this new interface?

Yes, the API is used by Intel, HP, NextIO and some other anonymous companies as they rise questions and send me feedback. I haven't seen their works but I guess some of drivers using SR-IOV API are going to be released soon.

Well, we can't merge infrastructure without seeing the users of that infrastructure, right?

Quoted 4 lines

My test was done with Intel 82576 Gigabit Ethernet Controller. The product brief is at http://download.intel.com/design/network/ProdBrf/320025.pdf and the spec is available at
http://download.intel.com/design/network/datashts/82576_Datasheet_v2p1.pdf

Cool, do you have that driver we can see?

How does it interact and handle the kvm and xen issues that have been posted?

thanks,

greg k-h

#228 lore raw
In reply to Yu Zhao
Quoted 7 lines · Yu Zhao

Greetings,

Following patches are intended to support SR-IOV capability in the Linux kernel. With these patches, people can turn a PCI device with the capability into multiple ones from software perspective, which will benefit KVM and achieve other purposes such as QoS, security, and etc.

Is there any actual users of this API around yet? How was it tested as there is no hardware to test on? Which drivers are going to have to be rewritten to take advantage of this new interface?

thanks,

greg k-h