OFED 1.5.2: libibumad.so version bump
Hi all,
In order to support RoCEE, a while ago I've added a new field to umad, thus introduced an ABI change.
There already was a discussion on the linux-rdma list, but due to the proximity of the upcoming OFED 1.5.2 release these concerns were raised again.
So my question is, other that *general* concerns about changing ABI, is anybody aware of the *actual* problem that will be caused by this? Any customer/3rd party solution that would be affected by this?
-- Yevgeny
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 5 lines · On Tue, 31 Aug 2010 02:27:29 -0700 Yevgeny Kliteynik…
Hi all,
In order to support RoCEE, a while ago I've added a new field to umad, thus introduced an ABI change.
Quoted 8 lines
There already was a discussion on the linux-rdma list, but due to the proximity of the upcoming OFED 1.5.2 release these concerns were raised again.
So my question is, other that *general* concerns about changing ABI, is anybody aware of the *actual* problem that will be caused by this? Any customer/3rd party solution that would be affected by this?
Because our MVAPICH depends on umad, libibumad.so.1 to be exact.[*] These ABI changes (to v2 and v3) would have forced our users to recompile their codes. We are maintaining the old ABI here until our next major release of CHAOS[#] to prevent this.
I think the thing to remember is that many people are using Open Fabrics software, but are _not_ using "OFED". What is tested with OFED is not the only thing which might be using these libraries. Our version of MVAPICH is a good example.
I am certainly not the expert in this area and I know that many people have tried to make this point in the past, but I will say it here again. Each of these Open Fabrics packages _must_ be maintained to stand on their own. Roland did this a long time ago with ibverbs.
I think now is a good time to start discussing breaking up the "management" git tree so that these libraries can live on their own. I will write a separate email regarding this.
Ira
[*] We are looking into removing the dependency.
[#] Shameless plug: http://code.google.com/p/chaos-release/wiki/CHAOS_Description
Quoted 7 lines
-- Yevgeny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://*vger.kernel.org/majordomo-info.html
Signature · 9 lines
Ira Weiny Math Programmer/Computer Scientist Lawrence Livermore National Lab 925-423-8008 [email protected] -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi Ira,
Quoted 27 lines · Ira Weiny
Quoted 5 lines · On Tue, 31 Aug 2010 02:27:29 -0700 Yevgeny Kliteynik…
Hi all,
In order to support RoCEE, a while ago I've added a new field to umad, thus introduced an ABI change.
Quoted 8 lines
There already was a discussion on the linux-rdma list, but due to the proximity of the upcoming OFED 1.5.2 release these concerns were raised again.
So my question is, other that *general* concerns about changing ABI, is anybody aware of the *actual* problem that will be caused by this? Any customer/3rd party solution that would be affected by this?
Because our MVAPICH depends on umad, libibumad.so.1 to be exact.[*] These ABI changes (to v2 and v3) would have forced our users to recompile their codes. We are maintaining the old ABI here until our next major release of CHAOS[#] to prevent this.
I think the thing to remember is that many people are using Open Fabrics software, but are _not_ using "OFED". What is tested with OFED is not the only thing which might be using these libraries. Our version of MVAPICH is a good example.
Quoted 7 lines
I am certainly not the expert in this area and I know that many people have tried to make this point in the past, but I will say it here again. Each of these Open Fabrics packages _must_ be maintained to stand on their own. Roland did this a long time ago with ibverbs.
I think now is a good time to start discussing breaking up the "management" git tree so that these libraries can live on their own.
How does breaking up the "management" git tree help with this issue ? To me, that's the admin part and is separate from the ABI issue raised. The ABI compatibility is not achieved by administrative means (separate repos, etc.) but rather than review and discipline to achieve this as a unmutable goal.
-- Hal
Quoted 16 lines
I will write a separate email regarding this.
Ira
[*] We are looking into removing the dependency.
[#] Shameless plug: http://code.google.com/p/chaos-release/wiki/CHAOS_DescriptionQuoted 7 lines
-- Yevgeny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://*vger.kernel.org/majordomo-info.html
Quoted 11 lines
--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): AANLkTim1_D5+GMWX80X0afu8A8_kizbQUq1KUhxxSLpz-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 35 lines · On Thu, 2 Sep 2010 13:35:44 -0700 Hal Rosenstock…
Hi Ira,
Quoted 32 lines · Ira Weiny
Quoted 5 lines · On Tue, 31 Aug 2010 02:27:29 -0700 Yevgeny Kliteynik…
Hi all,
In order to support RoCEE, a while ago I've added a new field to umad, thus introduced an ABI change.
Quoted 8 lines
There already was a discussion on the linux-rdma list, but due to the proximity of the upcoming OFED 1.5.2 release these concerns were raised again.
So my question is, other that *general* concerns about changing ABI, is anybody aware of the *actual* problem that will be caused by this? Any customer/3rd party solution that would be affected by this?
Because our MVAPICH depends on umad, libibumad.so.1 to be exact.[*] These ABI changes (to v2 and v3) would have forced our users to recompile their codes. We are maintaining the old ABI here until our next major release of CHAOS[#] to prevent this.
I think the thing to remember is that many people are using Open Fabrics software, but are _not_ using "OFED". What is tested with OFED is not the only thing which might be using these libraries. Our version of MVAPICH is a good example.
I am certainly not the expert in this area and I know that many people have tried to make this point in the past, but I will say it here again. Each of these Open Fabrics packages _must_ be maintained to stand on their own. Roland did this a long time ago with ibverbs.
Quoted 4 lines
Quoted 2 lines
I think now is a good time to start discussing breaking up the "management" git tree so that these libraries can live on their own.
How does breaking up the "management" git tree help with this issue ?
Creating and tracking of branches to maintain ABI would be easier with separate git trees. Furthermore, separate trees will help "force" the use of consistent ABI's and interfaces.
For example, if I currently want OpenSM version 3.3.6 I get a management tree with version libibumad 1.3.5. But this last ABI change to umad was only required for the latest infiniband-diags (ibstat utility). Why do I get all this "cruft" when pulling the latest OpenSM?
Quoted 2 lines
To me, that's the admin part and is separate from the ABI issue raised.
Yes it is separate. That is why I created another thread to discuss those issues.
Quoted 4 lines
The ABI compatibility is not achieved by administrative means (separate repos, etc.) but rather than review and discipline to achieve this as a unmutable goal.
I agree that ABI compatibility will require more discipline. That is what made me think of the separate git trees. I feel it will be _easier_ to maintain this discipline when the trees are separate.
Ira
Quoted 31 lines
-- Hal
Quoted 27 lines
I will write a separate email regarding this.
Ira
[*] We are looking into removing the dependency.
[#] Shameless plug: http://*code.google.com/p/chaos-release/wiki/CHAOS_DescriptionQuoted 7 lines
-- Yevgeny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://**vger.kernel.org/majordomo-info.html--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://*vger.kernel.org/majordomo-info.html
Signature · 5 lines
Ira Weiny Math Programmer/Computer Scientist Lawrence Livermore National Lab 925-423-8008 [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 7 lines · Ira Weiny
Quoted 3 lines
The ABI compatibility is not achieved by administrative means (separate repos, etc.) but rather than review and discipline to achieve this as a unmutable goal.
I agree that ABI compatibility will require more discipline. That is what made me think of the separate git trees. I feel it will be _easier_ to maintain this discipline when the trees are separate.
I would almost think the reverse.. It feels like there are *too many* GIT trees and too few maintainers. I do argee that including the core MAD libraries with opensm is kinda silly.
Consider bundling all the core libraries into a single git tree and considering a group of people to watch for these issues for all the libraries, together.
It feels like ongoing development of many of these libs has stopped, plus they are so tiny, it might be a better use of everyones time this way?
When I say combine together, I really mean that, shared configure script, etc, etc. Not like we see in the management.git.
As it is, building the core OFA userspace from source is a huge PITA. Just finding the proper git trees is often hard..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 43 lines · Ira Weiny
Quoted 35 lines · On Thu, 2 Sep 2010 13:35:44 -0700 Hal Rosenstock…
Hi Ira,
Quoted 32 lines · Ira Weiny
Quoted 5 lines · On Tue, 31 Aug 2010 02:27:29 -0700 Yevgeny Kliteynik…
Hi all,
In order to support RoCEE, a while ago I've added a new field to umad, thus introduced an ABI change.
Quoted 8 lines
There already was a discussion on the linux-rdma list, but due to the proximity of the upcoming OFED 1.5.2 release these concerns were raised again.
So my question is, other that *general* concerns about changing ABI, is anybody aware of the *actual* problem that will be caused by this? Any customer/3rd party solution that would be affected by this?
Because our MVAPICH depends on umad, libibumad.so.1 to be exact.[*] These ABI changes (to v2 and v3) would have forced our users to recompile their codes. We are maintaining the old ABI here until our next major release of CHAOS[#] to prevent this.
I think the thing to remember is that many people are using Open Fabrics software, but are _not_ using "OFED". What is tested with OFED is not the only thing which might be using these libraries. Our version of MVAPICH is a good example.
I am certainly not the expert in this area and I know that many people have tried to make this point in the past, but I will say it here again. Each of these Open Fabrics packages _must_ be maintained to stand on their own. Roland did this a long time ago with ibverbs.
Quoted 4 lines
Quoted 2 lines
I think now is a good time to start discussing breaking up the "management" git tree so that these libraries can live on their own.
How does breaking up the "management" git tree help with this issue ?
Quoted 8 lines
Creating and tracking of branches to maintain ABI would be easier with separate git trees. Furthermore, separate trees will help "force" the use of consistent ABI's and interfaces.
For example, if I currently want OpenSM version 3.3.6 I get a management tree with version libibumad 1.3.5. But this last ABI change to umad was only required for the latest infiniband-diags (ibstat utility). Why do I get all this "cruft" when pulling the latest OpenSM?
That's a totally different issue than which packages a particular OFED release picks up.
Quoted 8 lines
Quoted 3 lines
To me, that's the admin part and is separate from the ABI issue raised.
Yes it is separate. That is why I created another thread to discuss those issues.
Quoted 1 line
Quoted 7 lines
Quoted 3 lines
The ABI compatibility is not achieved by administrative means (separate repos, etc.) but rather than review and discipline to achieve this as a unmutable goal.
I agree that ABI compatibility will require more discipline. That is what made me think of the separate git trees. I feel it will be _easier_ to maintain this discipline when the trees are separate.
Call me a skeptic but I think the same thing would've occurred with separate git trees.
I have no real preference one way or the other. In fact, this was discussed in the early days which are long forgotten. The libraries are small and umad is a lot more stable than mad. It would just mean a lot of busy work for everyone with internal trees.
-- Hal
Quoted 36 lines
Ira
Quoted 31 lines
-- Hal
Quoted 27 lines
I will write a separate email regarding this.
Ira
[*] We are looking into removing the dependency.
[#] Shameless plug: http://*code.google.com/p/chaos-release/wiki/CHAOS_DescriptionQuoted 7 lines
-- Yevgeny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://**vger.kernel.org/majordomo-info.html--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://*vger.kernel.org/majordomo-info.html
Quoted 7 lines
--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html