Storage management (API & Library)
I would like to give a brief background on the storage management API and library that I have been working on** and discuss the design and implementation challenges that I have encountered thus far.
Most notable is the same topic Shyam Iyer brought up about proprietary plug-in support (http://marc.info/?l=linux-scsi&m=132454179114038&w=2).
This very feature has influenced libStorageMgmt heavily and has raised a number of discussions on design and licensing.
Discussion topics
* Scope (What a storage library should/should not do)
* Role (library position in management stack)
* Features (specific: must have to would be nice)
* Usage scenarios which highlight needed features
* Licensing (most appropriate open source license)
* Proprietary plug-in support
** http://sourceforge.net/apps/trac/libstoragemgmt
Regards,
Tony Asleson
Quoted 9 lines · Tony Asleson
I would like to give a brief background on the storage management API and library that I have been working on** and discuss the design and implementation challenges that I have encountered thus far.
Most notable is the same topic Shyam Iyer brought up about proprietary plug-in support (http://marc.info/?l=linux-scsi&m=132454179114038&w=2).
This very feature has influenced libStorageMgmt heavily and has raised a number of discussions on design and licensing.
Quoted 7 lines
Discussion topics
* Scope (What a storage library should/should not do)
* Role (library position in management stack)
* Features (specific: must have to would be nice)
* Usage scenarios which highlight needed features
* Licensing (most appropriate open source license)
* Proprietary plug-in support
Interesting, how does this relate, or not relate, to udisks in device-kit?
--
Dan
Quoted 1 line · Dan Williams
Interesting, how does this relate, or not relate, to udisks in device-kit?
The initial focus for libStorageMgmt is on managing external block level storage arrays (LUN creation, deletion, re-sizing, replication and mapping/un-mapping storage to initiators). Once the logical unit is available for the system, other libraries/tools will take over to get the storage ready for use (Partitioning, FS creation etc.).
One could envision udisks as being a user of libStorageMgmt. To dynamically add/remove storage devices to a system and to leverage the operations that can be completed on the array hardware vs. local system (e.g. hardware array snapshot vs. lvm snapshot).
Regards,
Tony
James Bottomley
maintainerQuoted 18 lines · Tony Asleson
I would like to give a brief background on the storage management API and library that I have been working on** and discuss the design and implementation challenges that I have encountered thus far.
Most notable is the same topic Shyam Iyer brought up about proprietary plug-in support (http://marc.info/?l=linux-scsi&m=132454179114038&w=2).
This very feature has influenced libStorageMgmt heavily and has raised a number of discussions on design and licensing.Discussion topics
* Scope (What a storage library should/should not do)
* Role (library position in management stack)
* Features (specific: must have to would be nice)
* Usage scenarios which highlight needed features
* Licensing (most appropriate open source license)
* Proprietary plug-in support
Quoted 1 line
Its a bit thin at 20 commits over 4 months.
It's also a bit insular ... the first thing you usually ask in open source is what can I steal^Wborrow from other projects and how do I recruit others to do the work for me. Once you can actually answer that, the list you have above mostly takes care of itself.
The first question is probably: is there anything we can liberate from the Oracle storage API fisasco to help with this. The second might be what would it take to get vendors interested in doing the array plugin glue.
James
Quoted 3 lines · James Bottomley
It's also a bit insular ... the first thing you usually ask in open source is what can I steal^Wborrow from other projects and how do I recruit others to do the work for me.
The project site is a little sparse on details about what we have looked at and considered. Ric Wheeler covered some of this during his presentation in Prague, but I will place more information on the project web site.
Quoted 2 lines
The first question is probably: is there anything we can liberate from the Oracle storage API fisasco to help with this.
The Oracle storage connect library was evaluated and subsequently rejected for the following reasons:
1. The Oracle storage connect library is dual licensed, GPL and a proprietary Oracle license which allows proprietary use. This allows hardware vendors the ability to write proprietary plug-ins. The design of the library has the library user and plug-in executing in the same address space. Based on the information presented on plug-ins
(http://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins) this design goes against the requirements of GPL license compliance. Proprietary plug-ins could be in compliance if they are implemented to execute in a separate process.
2. At the time the Oracle storage connect library was being evaluated we were unable to obtain a plug-in from a few different vendors. Without plug-ins the value of any library becomes greatly diminished.
Quoted 2 lines
The second might be what would it take to get vendors interested in doing the array plugin glue.
By providing:
* Permissive license (LGPL)
* Easy to use out of process plug-in support to allow proprietary plug-ins (IPC is abstracted)
* Language agnostic plug-in support (initial support is C and python)
We are hoping we can get hardware vendors interested in providing their own plug-ins. If anyone has additional ideas, we would certainly like to discuss them.
Regards,
Tony
James Bottomley
maintainerQuoted 9 lines · Tony Asleson
Quoted 3 lines · James Bottomley
It's also a bit insular ... the first thing you usually ask in open source is what can I steal^Wborrow from other projects and how do I recruit others to do the work for me.
The project site is a little sparse on details about what we have looked at and considered. Ric Wheeler covered some of this during his presentation in Prague, but I will place more information on the project web site.
That would be good for those who didn't go to that presentation ...
Quoted 6 lines
Quoted 2 lines
The first question is probably: is there anything we can liberate from the Oracle storage API fisasco to help with this.
The Oracle storage connect library was evaluated and subsequently rejected for the following reasons:
Quoted 9 lines
1. The Oracle storage connect library is dual licensed, GPL and a proprietary Oracle license which allows proprietary use. This allows hardware vendors the ability to write proprietary plug-ins. The design of the library has the library user and plug-in executing in the same address space. Based on the information presented on plug-ins
(http://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins) this design goes against the requirements of GPL license compliance. Proprietary plug-ins could be in compliance if they are implemented to execute in a separate process.
This is actually wrong thinking. Almost every Open Source project has an incompatible licence. The way we work around this (once we've determined there's something worth stealing) is to ask the project owner for a compatible grant (in this case, some pieces under LGPL). That's how we share driver code between Linux and BSD for instance.
If Oracle is intent on pursuing an open core business model for this and says "no" to the request, then you can say they're incompatible ... did anyone ask them?
Quoted 3 lines
2. At the time the Oracle storage connect library was being evaluated we were unable to obtain a plug-in from a few different vendors. Without plug-ins the value of any library becomes greatly diminished.
Why? You have no plug ins for your project currently ... that doesn't really diminish the value of it as an emerging project.
It sounds like you evaluated the Oracle project from an all or nothing standpoint rather than a what can we learn/steal standpoint.
One of the big things we could co-opt from oracle looks to be vendor buy in (they seem to be investing more in marketing than actual engineering). Vendors hate being confused or making choices, so one easy way to bring them on board might be API compatibility with the Oracle project.
Quoted 9 lines
Quoted 2 lines
The second might be what would it take to get vendors interested in doing the array plugin glue.
By providing:
* Permissive license (LGPL)
* Easy to use out of process plug-in support to allow proprietary plug-ins (IPC is abstracted)
* Language agnostic plug-in support (initial support is C and python)
Quoted 3 lines
We are hoping we can get hardware vendors interested in providing their own plug-ins. If anyone has additional ideas, we would certainly like to discuss them.
So no actual vendors have provided any input yet?
James
Quoted 31 lines · James Bottomley
Quoted 8 lines · Tony Asleson
Quoted 3 lines · James Bottomley
It's also a bit insular ... the first thing you usually ask in open source is what can I steal^Wborrow from other projects and how do I recruit others to do the work for me.
The project site is a little sparse on details about what we have looked at and considered. Ric Wheeler covered some of this during his presentation in Prague, but I will place more information on the project web site.
That would be good for those who didn't go to that presentation ...
Quoted 5 lines
Quoted 2 lines
The first question is probably: is there anything we can liberate from the Oracle storage API fisasco to help with this.
The Oracle storage connect library was evaluated and subsequently rejected for the following reasons:
Quoted 9 lines
1. The Oracle storage connect library is dual licensed, GPL and a proprietary Oracle license which allows proprietary use. This allows hardware vendors the ability to write proprietary plug-ins. The design of the library has the library user and plug-in executing in the same address space. Based on the information presented on plug-ins
(http://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins) this design goes against the requirements of GPL license compliance. Proprietary plug-ins could be in compliance if they are implemented to execute in a separate process.This is actually wrong thinking. Almost every Open Source project has an incompatible licence. The way we work around this (once we've determined there's something worth stealing) is to ask the project owner for a compatible grant (in this case, some pieces under LGPL). That's how we share driver code between Linux and BSD for instance.
Quoted 3 lines
If Oracle is intent on pursuing an open core business model for this and says "no" to the request, then you can say they're incompatible ... did anyone ask them?
I think that ignores the history behind the Oracle storage management project. We did look at the Oracle work, it certainly has been a long time coming and was not done in a community fashion as far as I can tell.
Putting the burden back on you, can you point to a single post about their project on a community list or name an individual (still employed by Oracle) that we can discuss with :) ?
We waited years for them to get to the dual license decision in the first place.
Having looked at their code, I think it is easier to write it - in a community way - the way we would have it done than to spend another year waiting on their legal team to change the license.
Clearly, using GPL for a library (not LGPL) is meant to push people into the commercial license....
Ric
Quoted 30 lines
Quoted 3 lines
2. At the time the Oracle storage connect library was being evaluated we were unable to obtain a plug-in from a few different vendors. Without plug-ins the value of any library becomes greatly diminished.
Why? You have no plug ins for your project currently ... that doesn't really diminish the value of it as an emerging project.
It sounds like you evaluated the Oracle project from an all or nothing standpoint rather than a what can we learn/steal standpoint.
One of the big things we could co-opt from oracle looks to be vendor buy in (they seem to be investing more in marketing than actual engineering). Vendors hate being confused or making choices, so one easy way to bring them on board might be API compatibility with the Oracle project.
Quoted 11 lines
Quoted 2 lines
The second might be what would it take to get vendors interested in doing the array plugin glue.
By providing:
* Permissive license (LGPL)
* Easy to use out of process plug-in support to allow proprietary plug-ins (IPC is abstracted)
* Language agnostic plug-in support (initial support is C and python)We are hoping we can get hardware vendors interested in providing their own plug-ins. If anyone has additional ideas, we would certainly like to discuss them.
So no actual vendors have provided any input yet?
James
James Bottomley
maintainerQuoted 32 lines · Ric Wheeler
Quoted 31 lines · James Bottomley
Quoted 8 lines · Tony Asleson
Quoted 3 lines · James Bottomley
It's also a bit insular ... the first thing you usually ask in open source is what can I steal^Wborrow from other projects and how do I recruit others to do the work for me.
The project site is a little sparse on details about what we have looked at and considered. Ric Wheeler covered some of this during his presentation in Prague, but I will place more information on the project web site.
That would be good for those who didn't go to that presentation ...
Quoted 5 lines
Quoted 2 lines
The first question is probably: is there anything we can liberate from the Oracle storage API fisasco to help with this.
The Oracle storage connect library was evaluated and subsequently rejected for the following reasons:
Quoted 9 lines
1. The Oracle storage connect library is dual licensed, GPL and a proprietary Oracle license which allows proprietary use. This allows hardware vendors the ability to write proprietary plug-ins. The design of the library has the library user and plug-in executing in the same address space. Based on the information presented on plug-ins
(http://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins) this design goes against the requirements of GPL license compliance. Proprietary plug-ins could be in compliance if they are implemented to execute in a separate process.This is actually wrong thinking. Almost every Open Source project has an incompatible licence. The way we work around this (once we've determined there's something worth stealing) is to ask the project owner for a compatible grant (in this case, some pieces under LGPL). That's how we share driver code between Linux and BSD for instance.
Quoted 7 lines
Quoted 3 lines
If Oracle is intent on pursuing an open core business model for this and says "no" to the request, then you can say they're incompatible ... did anyone ask them?
I think that ignores the history behind the Oracle storage management project. We did look at the Oracle work, it certainly has been a long time coming and was not done in a community fashion as far as I can tell.
I'm not denying it's been a royal screw up ... particularly with the "sure we're releasing it any day now" message we got for two years. But that is partly due to the fact that the open source guy left the project a while back.
Quoted 3 lines
Putting the burden back on you, can you point to a single post about their project on a community list or name an individual (still employed by Oracle) that we can discuss with :) ?
Um, that question is rhetorical, right? Or are you implying Martin Petersen is no longer at oracle (which will come as news to him, I think)?
Quoted 5 lines
We waited years for them to get to the dual license decision in the first place.
Having looked at their code, I think it is easier to write it - in a community way - the way we would have it done than to spend another year waiting on their legal team to change the license.
However annoyed we might be with Oracle, reinventing an existing project that already has committed vendor plugin contributors and thus forcing vendors to choose which project to support isn't going to be hugely helpful. Assuming the worst about Oracle, what's the barriers to reusing the plugins (the API is public and doesn't have a licensing issue)?
Quoted 2 lines
Clearly, using GPL for a library (not LGPL) is meant to push people into the commercial license....
You're assuming a malign motive where there could be simple open source ignorance. I'll dig around and see if I can get a read on what they're actually up to.
James
Quoted 1 line · Tony Asleson
On 01/21/2012 08:57 AM, James Bottomley wrote:
[ .. ]
Quoted 9 lines
Quoted 2 lines
The second might be what would it take to get vendors interested in doing the array plugin glue.
By providing:
* Permissive license (LGPL)
* Easy to use out of process plug-in support to allow proprietary plug-ins (IPC is abstracted)
* Language agnostic plug-in support (initial support is C and python)
Quoted 4 lines
We are hoping we can get hardware vendors interested in providing their own plug-ins. If anyone has additional ideas, we would certainly like to discuss them.
The various hardware vendors already have the means of managing their arrays, in most cases also providing a CLI.
I would be hoping we could draft the API support such that we can use the existing tools, without the need of special plug-ins.
Cheers,
Hannes
Quoted 4 lines · Hannes Reinecke
The various hardware vendors already have the means of managing their arrays, in most cases also providing a CLI.
I would be hoping we could draft the API support such that we can use the existing tools, without the need of special plug-ins.
Wrapping tools from libraries is more of a last resort than a proper solution. But if we can't get better APIs that's probably what we will have to resort to if vendors don't cooperate.
There's another project which is creating a very ad-hoc storage management abstraction, the nova-volume code in openstack. Right now it supports iSCSI with a few vendor-specific extensions for management, RBD and sheepdog, but it looks like the numbers of vendors that want to be supported is growing.
See https://github.com/openstack/nova/blob/master/nova/volume/driver.py
and other files in that directory for the little bits that exist at the moment.
Quoted 9 lines · Christoph Hellwig
There's another project which is creating a very ad-hoc storage management abstraction, the nova-volume code in openstack. Right now it supports iSCSI with a few vendor-specific extensions for management, RBD and sheepdog, but it looks like the numbers of vendors that want to be supported is growing.
See https://github.com/openstack/nova/blob/master/nova/volume/driver.py
and other files in that directory for the little bits that exist at the moment.
<two minutes prior>
Quoted 3 lines · Christoph Hellwig
Wrapping tools from libraries is more of a last resort than a proper solution. But if we can't get better APIs that's probably what we will have to resort to if vendors don't cooperate.
Nova is wrapping cmdline tools, and all the vendors interested so far are foss/cloudy.
This definitely shows others see a need for an API, but if we really want big storage vendor buy-in, then a better (C?/sockets?) API and working with non-foss code seems key.
Also, Nova code executes on the target machine, whereas libstoragemgmt is running on the mgmt node and the plugins are responsible for communication with the target node. (just wanted to point out another difference.)
-- Andy
Quoted 2 lines · Andy Grover
Nova is wrapping cmdline tools, and all the vendors interested so far are foss/cloudy.
I'm defintively not advocating that code, quite contrary. Just pointing out that it exists.
Quoted 8 lines
This definitely shows others see a need for an API, but if we really want big storage vendor buy-in, then a better (C?/sockets?) API and working with non-foss code seems key.
Also, Nova code executes on the target machine, whereas libstoragemgmt is running on the mgmt node and the plugins are responsible for communication with the target node. (just wanted to point out another difference.)
As said, I'm not fond of the nova code, and I'm defintively not fond of placing a storage management abstraction into highlevel "cloud" management code. But this just shows where the lack of a low-level abstraction gets us.
Quoted 11 lines · On Thu, 26 Jan 2012 12:53:16 -0800 Andy Grover <[email protected]> wrote:
Quoted 9 lines · Christoph Hellwig
There's another project which is creating a very ad-hoc storage management abstraction, the nova-volume code in openstack. Right now it supports iSCSI with a few vendor-specific extensions for management, RBD and sheepdog, but it looks like the numbers of vendors that want to be supported is growing.
See https://github.com/openstack/nova/blob/master/nova/volume/driver.py
and other files in that directory for the little bits that exist at the moment.
Quoted 7 lines
<two minutes prior>
Quoted 3 lines · Christoph Hellwig
Wrapping tools from libraries is more of a last resort than a proper solution. But if we can't get better APIs that's probably what we will have to resort to if vendors don't cooperate.
Nova is wrapping cmdline tools
I think that some drivers are better a bit. For example, looks like SolidFire uses the REST API to talk to their storage:
, and all the vendors interested so far
Quoted 1 line
are foss/cloudy.
True (HP lefthand is supported though).
Quoted 3 lines
This definitely shows others see a need for an API, but if we really want big storage vendor buy-in, then a better (C?/sockets?) API and working with non-foss code seems key.
Agreed.
Quoted 4 lines · FUJITA Tomonori
, and all the vendors interested so far
Quoted 1 line
are foss/cloudy.
True (HP lefthand is supported though).
NetApp is adding one, too:
https://blueprints.launchpad.net/nova/+spec/netapp-volume-driver
Seems like cloud currently is the better buzzword than Linux, even if this is all for Linux deployments, at least at the hypervisor level.