Documentation on device-mapper and friends
I'm looking for information on device-mapper, the kernel space utility for Logical Volume Management (LVM2). It seems that the relevant code resides under drivers/md and a lot of other information is under Documentation/device-mapper/ That's fine, but is there any other document that gives more finer details about the device-mapper. I've also recently subsribed to the device-mapper mailing list.
-Amit
Quoted 12 lines · Kumar Amit Mehta
I'm looking for information on device-mapper, the kernel space utility for
Logical Volume Management (LVM2). It seems that the relevant code resides underdrivers/mdand a lot of other information is underDocumentation/device-mapper/
That's fine, but is there any other document that gives more finer details about
the device-mapper. I've also recently subsribed to the device-mapper mailing
list.
Quoted 1 line
-Amit
A nice diagram of the overall storage subsystem is at http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram.html
Dm is just a single block in it, but it can help to see where it fits in overall.
Btw: that diagram doesn't show the legacy ata driver that creates /dev/hdx style devices. Has that been dropped while I wasn't paying attention? I haven't used it in years, but I thought it was still used on embedded systems.
Greg
Signature · 1 line
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Quoted 6 lines · Greg Freemyer
A nice diagram of the overall storage subsystem is at http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram.html
Dm is just a single block in it, but it can help to see where it fits in overall.
Btw: that diagram doesn't show the legacy ata driver that creates
/dev/hdxstyle devices. Has that been dropped while I wasn't paying attention? I haven't used it in years, but I thought it was still used on embedded systems.
Thank you for sharing the link, but I'm looking for more detailed information on I/O stack in Linux, dm-mapper and multipath in particular.
Quoted 11 lines · amit mehta
Quoted 6 lines · Greg Freemyer
A nice diagram of the overall storage subsystem is at http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram.html
Dm is just a single block in it, but it can help to see where it fits in overall.
Btw: that diagram doesn't show the legacy ata driver that creates
/dev/hdxstyle devices. Has that been dropped while I wasn't paying attention? I haven't used it in years, but I thought it was still used on embedded systems.Thank you for sharing the link, but I'm looking for more detailed information on I/O stack in Linux, dm-mapper and multipath in particular.
I actually assumed others would step in. I haven't seen any dm docs outside the kernel tree, but I haven't done and dm work.
I guess you know dm-cache just went into the kernel:
http://lwn.net/Articles/540996/
The primary use case is to use SSD (or similar fast storage) as a cache for a rotating disk.
I assume you found this very old (and simple) article, but I doubt it is useful 8 years later: http://lwn.net/Articles/124703/
For more useful links, I often go to wikipedia and follow the links:
You should find:
multi-path faq: http://christophe.varoqui.free.fr/faq.html
multi-path policies: http://christophe.varoqui.free.fr/policies.html
multi-path diagrams: http://christophe.varoqui.free.fr/graphics/
multi-path reference: http://christophe.varoqui.free.fr/refbook.html
Greg
Hi
Quoted 11 lines · amit mehta
Quoted 6 lines · Greg Freemyer
A nice diagram of the overall storage subsystem is at http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram.html
Dm is just a single block in it, but it can help to see where it fits in overall.
Btw: that diagram doesn't show the legacy ata driver that creates
/dev/hdxstyle devices. Has that been dropped while I wasn't paying attention? I haven't used it in years, but I thought it was still used on embedded systems.Thank you for sharing the link, but I'm looking for more detailed information on I/O stack in Linux, dm-mapper and multipath in particular.
Some docs about multipath can be found here
http://www.sourceware.org/lvm2/wiki/MultipathUsageGuide
http://christophe.varoqui.free.fr/refbook.html
The userspace part for tools is here
http://sourceware.org/lvm2/
Hi Amit,
I had compiled some notes on my blog.
Here are some links on writing your own device mapper target.
http://techgmm.blogspot.in/p/writing-your-own-device-mapper-target.html
Concept of device mapper target.
http://techgmm.blogspot.in/p/device-mapper-layer-explored-every.html
Thanks,
Gaurav.
Quoted 26 lines · Anatol Pomozov
Hi
Quoted 1 line · amit mehta
On Sun, Apr 28, 2013 at 5:24 PM, Greg Freemyer <[email protected]>
Quoted 1 line · wrote:
Quoted 1 line
A nice diagram of the overall storage subsystem is at
http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram.html
Quoted 2 lines
Quoted 2 lines
Dm is just a single block in it, but it can help to see where it fits
in overall.
Quoted 2 lines
Quoted 2 lines
Btw: that diagram doesn't show the legacy ata driver that creates
/dev/hdxstyle devices. Has that been dropped while I wasn't paying attention? I haven't used it in years, but I thought it was still used on embedded systems.Quoted 5 lines
Quoted 1 line
Thank you for sharing the link, but I'm looking for more detailed information on I/O stack in Linux, dm-mapper and multipath in particular.
Some docs about multipath can be found here
http://www.sourceware.org/lvm2/wiki/MultipathUsageGuide
http://christophe.varoqui.free.fr/refbook.html
Quoted 8 lines
The userspace part for tools is here
http://sourceware.org/lvm2/_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130430/ff8fcb5f/attachment.html
Hi Gaurav,
I went through your blog and it is really informative. But after reading
that i realized that i have a question:
If I want to write a block device driver which is going to sit on lvm
(and do some functionality on top of it) then should i go for the block device driver api
or write it as a device mapper target. What are the
advantages/disadvantages of both the approaches.
Regards,
Neha
On Tue, Apr 30, 2013 at 4:24 AM, Gaurav Mahajan <[email protected]
Quoted 53 lines
wrote:
Hi Amit,
I had compiled some notes on my blog.
Here are some links on writing your own device mapper target.
http://techgmm.blogspot.in/p/writing-your-own-device-mapper-target.htmlConcept of device mapper target.
http://techgmm.blogspot.in/p/device-mapper-layer-explored-every.htmlThanks,
Gaurav.Quoted 26 lines · Anatol Pomozov
Hi
Quoted 1 line · amit mehta
On Sun, Apr 28, 2013 at 5:24 PM, Greg Freemyer <[email protected]>
Quoted 1 line · wrote:
Quoted 1 line
A nice diagram of the overall storage subsystem is at
http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram.html
Quoted 2 lines
Quoted 2 lines
Dm is just a single block in it, but it can help to see where it fits
in overall.
Quoted 2 lines
Quoted 2 lines
Btw: that diagram doesn't show the legacy ata driver that creates
/dev/hdxstyle devices. Has that been dropped while I wasn't paying attention? I haven't used it in years, but I thought it was still used on embedded systems.Quoted 5 lines
Quoted 1 line
Thank you for sharing the link, but I'm looking for more detailed information on I/O stack in Linux, dm-mapper and multipath in particular.
Some docs about multipath can be found here
http://www.sourceware.org/lvm2/wiki/MultipathUsageGuide
http://christophe.varoqui.free.fr/refbook.htmlQuoted 8 lines
The userspace part for tools is here
http://sourceware.org/lvm2/_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Quoted 6 lines
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130501/ca68038c/attachment.html
Hi Neha,
LVM uses device mapper. Advantages of using device mapper is that you can stack different dm-targets on each other.
I am really not aware of block device drivers.
May be Greg can help us understand the actual pros and cons.
Thanks,
Gaurav
Quoted 9 lines · neha naik
Hi Gaurav,
I went through your blog and it is really informative. But after reading
that i realized that i have a question:
If I want to write a block device driver which is going to sit on lvm
(and do some functionality on top of it) then should i go for the block device driver api
or write it as a device mapper target. What are the
advantages/disadvantages of both the approaches.
Quoted 67 lines
Regards,
NehaQuoted 59 lines · Gaurav Mahajan
Hi Amit,
I had compiled some notes on my blog.
Here are some links on writing your own device mapper target.
http://techgmm.blogspot.in/p/writing-your-own-device-mapper-target.htmlConcept of device mapper target.
http://techgmm.blogspot.in/p/device-mapper-layer-explored-every.htmlThanks,
Gaurav.On Tue, Apr 30, 2013 at 5:05 AM, Anatol Pomozov <[email protected]
Quoted 29 lines
wrote:
Hi
Quoted 1 line · amit mehta
On Sun, Apr 28, 2013 at 5:24 PM, Greg Freemyer <
Quoted 1 line · greg.freemyer at gmail.com> wrote:
Quoted 1 line
A nice diagram of the overall storage subsystem is at
http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram.html
Quoted 2 lines
Quoted 2 lines
Dm is just a single block in it, but it can help to see where it fits
in overall.
Quoted 2 lines
Quoted 2 lines
Btw: that diagram doesn't show the legacy ata driver that creates
/dev/hdxstyle devices. Has that been dropped while I wasn't paying attention? I haven't used it in years, but I thought it was still used on embedded systems.Quoted 5 lines
Quoted 1 line
Thank you for sharing the link, but I'm looking for more detailed information on I/O stack in Linux, dm-mapper and multipath in particular.
Some docs about multipath can be found here
http://www.sourceware.org/lvm2/wiki/MultipathUsageGuide
http://christophe.varoqui.free.fr/refbook.htmlQuoted 8 lines
The userspace part for tools is here
http://sourceware.org/lvm2/_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130507/101dbe24/attachment-0001.html
The block layers can be layered both ways. DM is the newer infrastructure and was created in the early days of 2.6
If what I was writing could fit into a dm-target, that is what I would do.
There are significant projects like drbd and mdraid that are not dm-targets, but I think their is a long term goal to incorporate mdraid's functionality at a minimum into dm. I doubt drbd is ever moved to dm. It is just too big of a project and in use in lots of production server environments.
Greg
Quoted 90 lines · Gaurav Mahajan
Hi Neha,
LVM uses device mapper. Advantages of using device mapper is that you can stack different dm-targets on each other.
I am really not aware of block device drivers.May be Greg can help us understand the actual pros and cons.
Thanks,
GauravQuoted 76 lines · neha naik
Hi Gaurav,
I went through your blog and it is really informative. But after reading
that i realized that i have a question:
If I want to write a block device driver which is going to sit on lvm
(and do some functionality on top of it) then should i go for the block device driver api
or write it as a device mapper target. What are the
advantages/disadvantages of both the approaches.Regards,
NehaQuoted 59 lines · Gaurav Mahajan
Hi Amit,
I had compiled some notes on my blog.
Here are some links on writing your own device mapper target.
http://techgmm.blogspot.in/p/writing-your-own-device-mapper-target.htmlConcept of device mapper target.
http://techgmm.blogspot.in/p/device-mapper-layer-explored-every.htmlThanks,
Gaurav.Quoted 28 lines · Anatol Pomozov
Hi
Quoted 14 lines · amit mehta
Quoted 11 lines · Greg Freemyer
A nice diagram of the overall storage subsystem is at
http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram.htmlDm is just a single block in it, but it can help to see where it fits in overall.
Btw: that diagram doesn't show the legacy ata driver that creates
/dev/hdxstyle devices. Has that been dropped while I wasn't paying attention? I haven't used it in years, but I thought it was still used on embedded systems.Quoted 3 lines
Thank you for sharing the link, but I'm looking for more detailed information on I/O stack in Linux, dm-mapper and multipath in particular.
Some docs about multipath can be found here
http://www.sourceware.org/lvm2/wiki/MultipathUsageGuide
http://christophe.varoqui.free.fr/refbook.htmlQuoted 7 lines
The userspace part for tools is here
http://sourceware.org/lvm2/_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Hi Greg,
Thanks for the information. I have another question :).
Is there some less flexibility if we use device mapper target? For example in block device driver you can use the api such that it won't
use the OS io scheduler, so the io comes directly to the block device
driver through the 'make_request' call. With the device mapper i don't
think that happens(looking at the api calls). Does this mean that stuff like io scheduling. barrier control etc is done
by the device mapper itself and we can focus only on 'mapping' the io.
Regards,
Neha
Quoted 73 lines · Greg Freemyer
The block layers can be layered both ways. DM is the newer infrastructure and was created in the early days of 2.6
If what I was writing could fit into a dm-target, that is what I would do.
There are significant projects like drbd and mdraid that are not dm-targets, but I think their is a long term goal to incorporate mdraid's functionality at a minimum into dm. I doubt drbd is ever moved to dm. It is just too big of a project and in use in lots of production server environments.
Greg
Quoted 6 lines · Gaurav Mahajan
Hi Neha,
LVM uses device mapper. Advantages of using device mapper is that you can stack different dm-targets on each other.
I am really not aware of block device drivers.Quoted 10 lines
May be Greg can help us understand the actual pros and cons.
Thanks,
GauravQuoted 3 lines · neha naik
Hi Gaurav,
I went through your blog and it is really informative. But afterreading
Quoted 11 lines
Quoted 11 lines
that i realized that i have a question:
If I want to write a block device driver which is going to sit on lvm
(and do some functionality on top of it) then should i go for the block device driver api
or write it as a device mapper target. What are the
advantages/disadvantages of both the approaches.Regards,
NehaQuoted 8 lines
Quoted 8 lines
Quoted 6 lines · Gaurav Mahajan
Hi Amit,
I had compiled some notes on my blog.
Here are some links on writing your own device mapper target.http://techgmm.blogspot.in/p/writing-your-own-device-mapper-target.html
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Concept of device mapper target.
http://techgmm.blogspot.in/p/device-mapper-layer-explored-every.htmlThanks,
Gaurav.Quoted 3 lines · Anatol Pomozov
Hi
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines · amit mehta
Quoted 4 lines · Greg Freemyer
A nice diagram of the overall storage subsystem is at
http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram.htmlDm is just a single block in it, but it can help to see where it
Quoted 38 lines
fits
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
in overall.
Btw: that diagram doesn't show the legacy ata driver that creates
/dev/hdxstyle devices. Has that been dropped while I wasn'tpaying
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
attention? I haven't used it in years, but I thought it was still
used on
Quoted 29 lines
Quoted 28 lines
Quoted 27 lines
Quoted 12 lines
Quoted 6 lines
Quoted 2 lines
embedded systems.
Thank you for sharing the link, but I'm looking for more detailed information on I/O stack in Linux, dm-mapper and multipath in particular.
Some docs about multipath can be found here
http://www.sourceware.org/lvm2/wiki/MultipathUsageGuide
http://christophe.varoqui.free.fr/refbook.htmlQuoted 7 lines
The userspace part for tools is here
http://sourceware.org/lvm2/_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130508/a1bb35d8/attachment.html
You should take this to the device mapper list, but I'll try here.
For lurkers, this drawing may be helpful:
http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram/linux-io-stack-diagram_v1.0.pdf
Quoted 7 lines · neha naik
Hi Greg,
Thanks for the information. I have another question :).
Is there some less flexibility if we use device mapper target? For example in block device driver you can use the api such that it won't
use the OS io scheduler, so the io comes directly to the block device driver
through the 'make_request' call. With the device mapper i don't think that
happens(looking at the api calls).
I believe that is correct and one of the reasons DRBD is not part of DM. DRBD has various modes including those where it guarantees the I/Os on both the main target and on the replicated target hit in the exact same order. I don't believe that DM can be used to totally control disk I/O. It is meant to be stackable, so I think you lose some exact control.
Quoted 3 lines
Does this mean that stuff like io
scheduling. barrier control etc is done
by the device mapper itself and we can focus only on 'mapping' the io.
As shown in the diagram linked above, DM sits above the i/o schedulers so you should not have to worry about it. If you want to play with schedulers, I think that should be done outside of DM.
Thus I believe you can ignore barriers/scheduling UNLESS you create a target that needs special barrier/scheduling control. The obvious example of something needing that is raid5. If you get a barrier that forces data out to a single disk in a raid, you MUST ensure that the raid checksum is calculated and written out prior to calling the barrier complete. That is going to take special handling no matter what you do.
It's been a couple years since I dug into raid 5/6 as relates to barriers, but it used to be that code simply didn't do the right thing in mdraid and DM did not support raid 5/6, so yes the coders could ignore it, but they created broken logic when they did.
Greg