mpt2sas and mpt3sas merge (again)
Back when the mpt3sas driver was first posted I suggested that it should be merged into mpt2sas, but my proposal didn't get much traction.
Illumos has now produced a shared driver and shown that the difference are basically limited to a different S/G list format [1], and a quick experiment on the Linux drivers confirms this mostly - the additional differences are various smaller workarounds for specific hardware revisions.
I think we'd all be served much better with a merged driver.
Quoted 9 lines · Christoph Hellwig
Back when the mpt3sas driver was first posted I suggested that it should be merged into mpt2sas, but my proposal didn't get much traction.
Illumos has now produced a shared driver and shown that the difference are basically limited to a different S/G list format [1], and a quick experiment on the Linux drivers confirms this mostly - the additional differences are various smaller workarounds for specific hardware revisions.
Quoted 3 lines
I think we'd all be served much better with a merged driver.
Please. Pretty please.
I've started a merge myself, but then never found time to finalize it.
Carrying three basically identical drivers just creates a maintenance overhead for everyone involved.
The original idea of splitting the driver and just maintaining the latest has never really worked out; all fixes to the latest driver turned out to be applicable to the others, too. So it just increased the workload for the maintainer for no real gain. I would _strongly_ vote for it.
Cheers,
Hannes
Signature · 8 lines
Dr. Hannes Reinecke zSeries & Storage [email protected] +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
James Bottomley
maintainerQuoted 10 lines · Hannes Reinecke
Quoted 9 lines · Christoph Hellwig
Back when the mpt3sas driver was first posted I suggested that it should be merged into mpt2sas, but my proposal didn't get much traction.
Illumos has now produced a shared driver and shown that the difference are basically limited to a different S/G list format [1], and a quick experiment on the Linux drivers confirms this mostly - the additional differences are various smaller workarounds for specific hardware revisions.
Quoted 5 lines
Quoted 3 lines
I think we'd all be served much better with a merged driver.
Please. Pretty please.
I support the concept, since I think everyone told LSI at the time that splitting the drivers would become a maintenance nightmare.
Quoted 10 lines
I've started a merge myself, but then never found time to finalize it.
Carrying three basically identical drivers just creates a maintenance overhead for everyone involved.
The original idea of splitting the driver and just maintaining the latest has never really worked out; all fixes to the latest driver turned out to be applicable to the others, too.
So it just increased the workload for the maintainer for no real gain. I would _strongly_ vote for it.
This isn't really a democracy; it's about who maintains the drivers and right now it's LSI (or whatever their new name is).
One of the big reasons we don't have a lot of leverage with them is that they always seem to slide updates around upstream via the distros (often, it has to be admitted the DKM route), so if Red Hat, SUSE, Oracle and Canonical can agree not to accept LSI updates until the driver is done this way, we'd have a lot more leverage.
James
Quoted 31 lines · James Bottomley
Quoted 10 lines · Hannes Reinecke
Quoted 9 lines · Christoph Hellwig
Back when the mpt3sas driver was first posted I suggested that it should be merged into mpt2sas, but my proposal didn't get much traction.
Illumos has now produced a shared driver and shown that the difference are basically limited to a different S/G list format [1], and a quick experiment on the Linux drivers confirms this mostly - the additional differences are various smaller workarounds for specific hardware revisions.
Quoted 5 lines
Quoted 3 lines
I think we'd all be served much better with a merged driver.
Please. Pretty please.
I support the concept, since I think everyone told LSI at the time that splitting the drivers would become a maintenance nightmare.
Quoted 10 lines
I've started a merge myself, but then never found time to finalize it.
Carrying three basically identical drivers just creates a maintenance overhead for everyone involved.
The original idea of splitting the driver and just maintaining the latest has never really worked out; all fixes to the latest driver turned out to be applicable to the others, too.
So it just increased the workload for the maintainer for no real gain. I would _strongly_ vote for it.
Quoted 9 lines
This isn't really a democracy; it's about who maintains the drivers and right now it's LSI (or whatever their new name is).
One of the big reasons we don't have a lot of leverage with them is that they always seem to slide updates around upstream via the distros (often, it has to be admitted the DKM route), so if Red Hat, SUSE, Oracle and Canonical can agree not to accept LSI updates until the driver is done this way, we'd have a lot more leverage.
Hmm. We (as SUSE) have been striving to have a 'upstream first' policy. IE for any new release the drivers have to be upstream before we consider including it in our release. This is most certainly true for the upcoming SLE-12 release, and also has been enforced for the current SLES11 SP3 release.
This is official company policy, and has been communicated to all our partners.
We do accept driver updates (ie patches which are not upstream ATM), but only on the understanding that the vendor will have to push the patches upstream eventually. If they don't the patches will be kicked out of the next release.
(Which is what happened to the mptsas v4 release; it never made it upstream and so got dropped from SLE-12).
However, this cuts both ways; we cannot go and tell our partners to change the driver if upstream hasn't done it first.
So the push has to come from us (as the linux kernel developers); after all, we should make the decision what goes in and what doesn't. If a driver is in a bad state (and it's actually us which defines the 'bad state') we should be discussing on how we would like to improve things. If the maintainer proves unwilling to implement our suggestions we can always go ahead and implement a separate driver.
Look what happened to hpsa; this was the pretty much the showcase on how it should be done:
Tomo went ahead and re-implemented the cciss driver, and eventually HP adopted it as their main driver. I agree that was pretty much the optimal case, though:-)
Cheers,
Hannes
Signature · 8 lines
Dr. Hannes Reinecke zSeries & Storage [email protected] +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
James Bottomley
maintainerQuoted 56 lines · Hannes Reinecke
Quoted 31 lines · James Bottomley
Quoted 10 lines · Hannes Reinecke
Quoted 9 lines · Christoph Hellwig
Back when the mpt3sas driver was first posted I suggested that it should be merged into mpt2sas, but my proposal didn't get much traction.
Illumos has now produced a shared driver and shown that the difference are basically limited to a different S/G list format [1], and a quick experiment on the Linux drivers confirms this mostly - the additional differences are various smaller workarounds for specific hardware revisions.
Quoted 5 lines
Quoted 3 lines
I think we'd all be served much better with a merged driver.
Please. Pretty please.
I support the concept, since I think everyone told LSI at the time that splitting the drivers would become a maintenance nightmare.
Quoted 10 lines
I've started a merge myself, but then never found time to finalize it.
Carrying three basically identical drivers just creates a maintenance overhead for everyone involved.
The original idea of splitting the driver and just maintaining the latest has never really worked out; all fixes to the latest driver turned out to be applicable to the others, too.
So it just increased the workload for the maintainer for no real gain. I would _strongly_ vote for it.Quoted 9 lines
This isn't really a democracy; it's about who maintains the drivers and right now it's LSI (or whatever their new name is).
One of the big reasons we don't have a lot of leverage with them is that they always seem to slide updates around upstream via the distros (often, it has to be admitted the DKM route), so if Red Hat, SUSE, Oracle and Canonical can agree not to accept LSI updates until the driver is done this way, we'd have a lot more leverage.
Hmm. We (as SUSE) have been striving to have a 'upstream first' policy. IE for any new release the drivers have to be upstream before we consider including it in our release.
This is most certainly true for the upcoming SLE-12 release, and also has been enforced for the current SLES11 SP3 release.This is official company policy, and has been communicated to all our partners.
We do accept driver updates (ie patches which are not upstream ATM), but only on the understanding that the vendor will have to push the patches upstream eventually.
If they don't the patches will be kicked out of the next release. (Which is what happened to the mptsas v4 release; it never made it upstream and so got dropped from SLE-12).
Quoted 2 lines
However, this cuts both ways; we cannot go and tell our partners to change the driver if upstream hasn't done it first.
I'm not saying we need to go into why this happened. Just that I'd like community agreement amongst the distros before trying to force the issue. I accept that the distros respond to their TAMs as well as the community, but if there's going to be TAM push back, I'd at least like to hear about it so I can have a word with the relevant people.
Quoted 7 lines
So the push has to come from us (as the linux kernel developers); after all, we should make the decision what goes in and what doesn't. If a driver is in a bad state (and it's actually us which defines the 'bad state') we should be discussing on how we would like to improve things.
If the maintainer proves unwilling to implement our suggestions we can always go ahead and implement a separate driver.
Then we need a maintainer of that driver ... remember this is a fat firmware driver with a proprietary interface. It's hard to maintain and update without docs ... unless you happen to have an NDA copy?
Quoted 5 lines
Look what happened to hpsa; this was the pretty much the showcase on how it should be done:
Tomo went ahead and re-implemented the cciss driver, and eventually HP adopted it as their main driver.
I agree that was pretty much the optimal case, though:-)
The best is to get LSI to agree, yes ... hence the need for unanimity.
James
Quoted 2 lines · James Bottomley
Quoted 1 line · Hannes Reinecke
On 07/14/2014 04:17 PM, James Bottomley wrote:
[ .. ]
Quoted 42 lines
Quoted 25 lines
Quoted 10 lines
This isn't really a democracy; it's about who maintains the drivers and right now it's LSI (or whatever their new name is).
One of the big reasons we don't have a lot of leverage with them is that they always seem to slide updates around upstream via the distros (often, it has to be admitted the DKM route), so if Red Hat, SUSE, Oracle and Canonical can agree not to accept LSI updates until the driver is done this way, we'd have a lot more leverage.
Hmm. We (as SUSE) have been striving to have a 'upstream first' policy. IE for any new release the drivers have to be upstream before we consider including it in our release.
This is most certainly true for the upcoming SLE-12 release, and also has been enforced for the current SLES11 SP3 release.This is official company policy, and has been communicated to all our partners.
We do accept driver updates (ie patches which are not upstream ATM), but only on the understanding that the vendor will have to push the patches upstream eventually.
If they don't the patches will be kicked out of the next release. (Which is what happened to the mptsas v4 release; it never made it upstream and so got dropped from SLE-12).Quoted 2 lines
However, this cuts both ways; we cannot go and tell our partners to change the driver if upstream hasn't done it first.
I'm not saying we need to go into why this happened. Just that I'd like community agreement amongst the distros before trying to force the issue. I accept that the distros respond to their TAMs as well as the community, but if there's going to be TAM push back, I'd at least like to hear about it so I can have a word with the relevant people.
Quoted 7 lines
So the push has to come from us (as the linux kernel developers); after all, we should make the decision what goes in and what doesn't. If a driver is in a bad state (and it's actually us which defines the 'bad state') we should be discussing on how we would like to improve things.
If the maintainer proves unwilling to implement our suggestions we can always go ahead and implement a separate driver.
Quoted 4 lines
Then we need a maintainer of that driver ... remember this is a fat firmware driver with a proprietary interface. It's hard to maintain and update without docs ... unless you happen to have an NDA copy?
Hmm. _if_ the driver is similar to the original one (which was the idea) it should be reasonably trivial to port the latest changes from the original driver to the merged one.
Quoted 8 lines
Quoted 5 lines
Look what happened to hpsa; this was the pretty much the showcase on how it should be done:
Tomo went ahead and re-implemented the cciss driver, and eventually HP adopted it as their main driver.
I agree that was pretty much the optimal case, though:-)The best is to get LSI to agree, yes ... hence the need for unanimity.
Agreed. Let's see what LSI has to say here.
Cheers,
Hannes
Signature · 8 lines
Dr. Hannes Reinecke zSeries & Storage [email protected] +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 4 lines · Hannes Reinecke
Quoted 2 lines · James Bottomley
Quoted 1 line · Hannes Reinecke
On 07/14/2014 04:17 PM, James Bottomley wrote:
[ .. ]
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
This isn't really a democracy; it's about who maintains the drivers and right now it's LSI (or whatever their new name is).
One of the big reasons we don't have a lot of leverage with them is that they always seem to slide updates around upstream via the distros (often, it has to be admitted the DKM route), so if Red Hat, SUSE, Oracle and Canonical can agree not to accept LSI updates until the driver is done this way, we'd have a lot more leverage.
We have a strong 'upstream first' policy in our releases for new drivers and for new patches, so in theory upstream could use the force, I still prefer a softer way that is discussing it with Avago(LSI) and getting their ack.
Quoted 31 lines
Quoted 31 lines
Quoted 16 lines
Quoted 1 line
Hmm. We (as SUSE) have been striving to have a 'upstream first' policy. IE for any new release the drivers have to be upstream before we consider including it in our release.
This is most certainly true for the upcoming SLE-12 release, and also has been enforced for the current SLES11 SP3 release.This is official company policy, and has been communicated to all our partners.
We do accept driver updates (ie patches which are not upstream ATM), but only on the understanding that the vendor will have to push the patches upstream eventually.
If they don't the patches will be kicked out of the next release. (Which is what happened to the mptsas v4 release; it never made it upstream and so got dropped from SLE-12).Quoted 2 lines
However, this cuts both ways; we cannot go and tell our partners to change the driver if upstream hasn't done it first.
I'm not saying we need to go into why this happened. Just that I'd like community agreement amongst the distros before trying to force the issue. I accept that the distros respond to their TAMs as well as the community, but if there's going to be TAM push back, I'd at least like to hear about it so I can have a word with the relevant people.
Quoted 7 lines
So the push has to come from us (as the linux kernel developers); after all, we should make the decision what goes in and what doesn't. If a driver is in a bad state (and it's actually us which defines the 'bad state') we should be discussing on how we would like to improve things.
If the maintainer proves unwilling to implement our suggestions we can always go ahead and implement a separate driver.
Quoted 7 lines
Quoted 4 lines
Then we need a maintainer of that driver ... remember this is a fat firmware driver with a proprietary interface. It's hard to maintain and update without docs ... unless you happen to have an NDA copy?
Hmm. _if_ the driver is similar to the original one (which was the idea) it should be reasonably trivial to port the latest changes from the original driver to the merged one.
I think you expect more or less that after that new driver is created it will be maintained again by Avago(LSI), so let us hear what they think first.
Quoted 1 line
From my point of view the not optimal thing already has happened, both driver exist
and both are included in our major releases.
The change is probably better for the long-term maintainability of the driver, but from the distro perspective any switch to a new driver adds some new work, because we don't want to remove any particular old driver which has been included to a major release.
That said, I agree that we (Red Hat) will manage the change, since it is the better approach for the long-term maintenance of the driver in the upstream".
Cheers,
Tomas
Quoted 13 lines
Quoted 8 lines
Quoted 6 lines
Look what happened to hpsa; this was the pretty much the showcase on how it should be done:
Tomo went ahead and re-implemented the cciss driver, and eventually HP adopted it as their main driver.
I agree that was pretty much the optimal case, though:-)The best is to get LSI to agree, yes ... hence the need for unanimity.
Agreed. Let's see what LSI has to say here.
Cheers,
Hannes
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"James" == James Bottomley <[email protected]> writes:
James> I support the concept, since I think everyone told LSI at the James> time that splitting the drivers would become a maintenance James> nightmare.
And it is. I'd like to see these drivers merged as well.
James> One of the big reasons we don't have a lot of leverage with them James> is that they always seem to slide updates around upstream via the James> distros (often, it has to be admitted the DKM route), so if Red James> Hat, SUSE, Oracle and Canonical can agree not to accept LSI James> updates until the driver is done this way, we'd have a lot more James> leverage.
Unless it's a trivial bug fix Oracle won't take anything that's not accepted upstream. We are working with Avago/LSI to streamline their driver process to be a better fit for both upstream development and distro kernels.
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering
A message that is not in the archive (sent off-list or not yet received): CAK=zhgoD9vVH0zCORTA2Mhu8Tf4m4VAih_hpig4PDdh-vgJmQg@mail.gmail.com
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Sreekanth" == Sreekanth Reddy <[email protected]> writes:
Hey Sreekanth,
Sreekanth> If we have single driver approach, making any changes in Sreekanth> driver require lots of regression and Q/A cycle so that Sreekanth> existing customer who are based on older controller does not Sreekanth> have any impact due some fixes/new features.
Sreekanth> With all these HBA specific features, it is unmanageable to Sreekanth> have a common driver for SAS2 and SAS3 HBAs.
But just as a counterexample to that: qla2xxx and lpfc are both capable of driving a wide range of chip generations and firmware interface versions with a single driver.
The problem I have with the mpt2sas/mpt3sas split is that there are way more commonalities than there are differences. It is trivial to handle multiple scatterlist format and features in a single driver. You guys already do that in megaraid_sas.
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering
Hi Martin,
megaraid driver was a single driver for both SAS2 and SAS3 controller because from day1 it was planned and developed that way. But it also certain cons
-Sometimes, there may be different settings needs to be done for IO for different type of controllers. In IO path, adding PNP id based checks may degrade the performance
-Making any fix for one controller family may affect other controllers code path as well, so code review and unit testing of fix has to done for all supported controllers.
e.g. in megaraid_sas driver, to maintain MFI controllers original design untouched, we have to deal with many synchronization issues in MPT controllers design. If there is some workaround for older controllers in driver, it may be carried forward sometimes, even if that issue can be fixed for new controllers.
-In case of megaraid_sas, all application and management commands are MFI frame based, so for legacy applications to work with MPT controllers(MPT frames), driver cannot post same MFI frame to firmware and has to internally allocate MPT frame corresponding to MFI frame, maintain the mapping of MFI to MPT frame and the send to firmware. Driver has to play as intermediate between apps and firmware, so driver has to do conversion.
To overcome these kind of issues, LSI has gone for separate drivers for mpt2sas and mpt3sas.
In case of mpt2sas, mpt3sas drivers, the only con is maintenance of separate drivers. Our SAS2 HBA is almost entering deep maintenance mode. We will not be posting any more feature support patches for mpt2sas. Whereas mpt3sas is at its initial baseline with a healthy road map ahead.
Regards,
Sreekanth
Quoted 23 lines · Martin K. Petersen
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Sreekanth" == Sreekanth Reddy <[email protected]> writes:
Hey Sreekanth,
Sreekanth> If we have single driver approach, making any changes in Sreekanth> driver require lots of regression and Q/A cycle so that Sreekanth> existing customer who are based on older controller does not Sreekanth> have any impact due some fixes/new features.
Sreekanth> With all these HBA specific features, it is unmanageable to Sreekanth> have a common driver for SAS2 and SAS3 HBAs.
But just as a counterexample to that: qla2xxx and lpfc are both capable of driving a wide range of chip generations and firmware interface versions with a single driver.
The problem I have with the mpt2sas/mpt3sas split is that there are way more commonalities than there are differences. It is trivial to handle multiple scatterlist format and features in a single driver. You guys already do that in
megaraid_sas.--
Martin K. Petersen Oracle Linux Engineering