MTD driver for MMC cards
Subsystems (2)
RFC 1/1 MTD driver for MMC cards
This is an experiment on how an SD/MMC card could be used in the MTD layer. I don't currently have a system set up to test this, so this driver is completely _untested_ and therefore you should consider it _broken_.
You can get similar functionality by using the mmc_block driver together with block2mtd, so you may wonder what the point of another driver is. IMHO, there are two separate advantages from using a special driver:
* better use of low-level interfaces: the MTD driver can detect the erase block size of the card and erase sectors in advance instead of blocking in the write path. The MTD file systems also expect the underlying interface to be synchronous, so there is little point in using extra kernel threads to operate on the card in the background.
* It becomes possible to use MMC cards with jffs2 even with CONFIG_BLOCK disabled, which can save a significant amount of kernel memory on small machines that have an MMC slot but no other block device.
I still want to be sure that I'm on the right track with this driver and did not make a conceptual mistake.
---
drivers/mmc/mmc.c +4 -0
@@ -616,6 +616,8 @@ static void mmc_decode_csd(struct mmc_ca csd->r2w_factor =UNSTUFF_BITS (resp,26 ,3 ); csd->write_blkbits =UNSTUFF_BITS (resp,22 ,4 ); csd->write_partial =UNSTUFF_BITS (resp,21 ,1 ); + csd->erase_blksize = (UNSTUFF_BITS (resp,37 ,5 ) +1 ) * + (UNSTUFF_BITS (resp,42 ,5 ) +1 ); }else {/* * We only understand CSD structure v1.1 and v1.2.
@@ -651,6 +653,8 @@ static void mmc_decode_csd(struct mmc_ca csd->r2w_factor =UNSTUFF_BITS (resp,26 ,3 ); csd->write_blkbits =UNSTUFF_BITS (resp,22 ,4 ); csd->write_partial =UNSTUFF_BITS (resp,21 ,1 ); + csd->erase_blksize = (UNSTUFF_BITS (resp,37 ,5 ) +1 ) * + (UNSTUFF_BITS (resp,42 ,5 ) +1 ); } }
include/linux/mmc/card.h +1 -0
@@ -32,6 +32,7 @@ struct mmc_csd {unsigned int max_dtr;unsigned int read_blkbits;unsigned int write_blkbits; +unsigned int erase_blksize;unsigned int capacity; unsigned int read_partial:1, read_misalign:1,
drivers/mmc/mmc_mtd.c new file +274 -0
274 lines not on this page · raw message
drivers/mmc/Kconfig +11 -0
@@ -29,6 +29,17 @@ config MMC_BLOCK mount the filesystem. Almost everyone wishing MMC support should say Y or M here. +config MMC_MTD + tristate "MMC MTD device driver" + depends on MMC && MTD && (MMC_BLOCK = n || (MMC_BLOCK != y && MMC_BLOCK)) + help + Enable this to as an alternative to the MMC block driver. + It makes it possible to use MMC memory cards with the + MTD layer and file systems based on it, e.g. jffs2. + Since both drivers operate on the same devices, you can't + build them both into the kernel and you need to say M here + if you want to build both. + config MMC_ARMMMCI tristate "ARM AMBA Multimedia Card Interface support" depends on ARM_AMBA && MMC
drivers/mmc/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(CONFIG_MMC) += mmc_core.o # Media drivers # obj-$(CONFIG_MMC_BLOCK) += mmc_block.o +obj-$(CONFIG_MMC_MTD) += mmc_mtd.o # # Host drivers
Quoted 8 lines · Arnd Bergmann
This is an experiment on how an SD/MMC card could be used in the MTD layer. I don't currently have a system set up to test this, so this driver is completely _untested_ and therefore you should consider it _broken_.
You can get similar functionality by using the
mmc_blockdriver together with block2mtd, so you may wonder what the point of another driver is. IMHO, there are two separate advantages from using a special driver:
Quoted 6 lines
* better use of low-level interfaces: the MTD driver can detect the erase block size of the card and erase sectors in advance instead of blocking in the write path. The MTD file systems also expect the underlying interface to be synchronous, so there is little point in using extra kernel threads to operate on the card in the background.
I'm a complete MTD noob, but what uses does the MTD layer have besides JFFS2. If it's none, than this advantage isn't that big of a deal.
Quoted 10 lines
* It becomes possible to use MMC cards with jffs2 even with
CONFIG_BLOCKdisabled, which can save a significant amount of kernel memory on small machines that have an MMC slot but no other block device.From what I've heard, JFFS2 is close to unusuable on the sizes of modern SD/MMC cards. So I'd like to see some more use cases before I'm ready to let this in.
I still want to be sure that I'm on the right track with this driver and did not make a conceptual mistake.
I can comment it from a MMC perspective, but the MTD stuff I will have to assume is correct.
drivers/mmc/mmc.c static void mmc_decode_csd(struct mmc_ca · line 623
Quoted 9 lines
drivers/mmc/mmc.c +2
-0@@ -616,6 +616,8 @@ static void mmc_decode_csd(struct mmc_ca csd->r2w_factor =UNSTUFF_BITS (resp,26 ,3 ); csd->write_blkbits =UNSTUFF_BITS (resp,22 ,4 ); csd->write_partial =UNSTUFF_BITS (resp,21 ,1 ); + csd->erase_blksize = (UNSTUFF_BITS (resp,37 ,5 ) +1 ) * + (UNSTUFF_BITS (resp,42 ,5 ) +1 ); }else {/* * We only understand CSD structure v1.1 and v1.2.
NAK. SD uses another format for erase blocks. See the simplified physical spec.
drivers/mmc/mmc_mtd.c line 94
Quoted 13 lines
drivers/mmc/mmc_mtd.c new file +13
-0+/* +* transfer a block to/from the card. The block needs to be aligned +* to mtd->writesize. If we want to implement an mtd_writev method, +* this needs to use stream operations with an appropriate stop +* command as well. +*/ +static int mmc_mtd_transfer_low (struct mmc_card *card,loff_t off,size_t len, +size_t *retlen,u_char *buf,int write) +{ +struct scatterlist sg; +struct mmc_data data = { + .blksz =1 << card->csd.read_blkbits, + .blocks = len >> card->csd.read_blkbits,
First of all, you cannot assume that read_blkbits is a valid block size when doing writes.
Secondly, the cards default in a block size of 512 bytes, so you need to tell the card your desired block size during probe.
drivers/mmc/mmc_mtd.c line 103
Quoted 9 lines
drivers/mmc/mmc_mtd.c new file +9
-0+ .flags = write ? MMC_DATA_WRITE : MMC_DATA_READ, + .sg = &sg, + .sg_len = 1, + }; +struct mmc_command cmd = { + .arg = off, + .data = &data, + .flags =MMC_RSP_R1 |MMC_CMD_ADTC , + .opcode = write ?MMC_WRITE_BLOCK :MMC_READ_SINGLE_BLOCK ,
You set .blocks above, so I have to assume it can be more than 1. So you need to change the opcodes accordingly.
drivers/mmc/mmc_mtd.c line 108
Quoted 5 lines
drivers/mmc/mmc_mtd.c new file +5
-0+ }; +struct mmc_request mrq = { + .cmd = &cmd, + .data = &data, + };
And it also means you need a stop command.
Review
Quoted 2 lines
(quoted diff) +2
-0+ + /* copied from the block driver, don't understand why this is needed */
Now this gives me a bad feeling. Have you read any spec about the MMC protocol or are you just winging it?
It is needed because the card goes into programming state after a write, where it is very unresponsive to other commands.
drivers/mmc/mmc_mtd.c line 157
Quoted 8 lines
drivers/mmc/mmc_mtd.c new file +8
-0+ + ret =mmc_card_claim_host (card); +if (ret) { +dev_warn (&card->dev,"%s: mmc_card_claim_host returned %d\n" , + __FUNCTION__, ret); + ret = -EIO ; + goto error; + }
mmc_card_claim_host() is currently very stupid in that it requires you to call mmc_card_release_host() on error. I intend to fix that some time in the future.
Quoted 4 lines
drivers/mmc/mmc_mtd.c new file +4
-0+/* +* Initialize an mmc card. We create a new MTD device for each +* MMC card we find. The operations are rather straightforward, +* so we don't even need our own data structure to contain the
drivers/mmc/mmc_mtd.c line 190
Quoted 10 lines
drivers/mmc/mmc_mtd.c new file +10
-0+* mtd_info. +*/ +static int mmc_mtd_probe (struct mmc_card *card) +{ +struct mtd_info *mtd; +int ret; + +if (!(card->csd.cmdclass &CCC_ERASE )) +return -ENODEV ; +
You should probably check for CCC_BLOCK_READ here.
And your driver needs to check if the card support writes (both by mmc_card_readonly() and CCC_BLOCK_WRITE).
Rgds
Signature · 5 lines
-- Pierre Ossman Linux kernel, MMC maintainer http://www.kernel.org PulseAudio, core developer http://pulseaudio.org rdesktop, core developer http://www.rdesktop.org
Quoted 6 lines · Pierre Ossman
Arnd Bergmann wrote:
I'm a complete MTD noob, but what uses does the MTD layer have besides JFFS2. If it's none, than this advantage isn't that big of a deal.
AFAIK MTD is for device where erase is need to managed in "software" :
http://www.linux-mtd.infradead.org/faq/general.html
For SD card, doesn't a microcontroller on the card hide this, and make the sd card acts like a normal block device (no need to erase a block before writing it)?
Quoted 5 lines · Pierre Ossman
Arnd Bergmann wrote:
I'm a complete MTD noob, but what uses does the MTD layer have besides JFFS2. If it's none, than this advantage isn't that big of a deal.
Quoted 8 lines
Quoted 4 lines
* It becomes possible to use MMC cards with jffs2 even with
CONFIG_BLOCKdisabled, which can save a significant amount of kernel memory on small machines that have an MMC slot but no other block device.From what I've heard, JFFS2 is close to unusuable on the sizes of modern SD/MMC cards. So I'd like to see some more use cases before I'm ready to let this in.
There are multiple efforts in progress to get a jffs2 replacement. NAND flash in embedded devices has the same size as it has on MMC card potentially, so we will need one soon. David Woodhouse has pushed the limit that jffs2 can reasonably used to 512MB, which is the size used in the OLPC XO laptop. If there are ways to get beyond that (which I find unlikely), there will be a hard limit 2GB or 4GB because of limitations in the fs layout.
One promising effort for a replacement is Jörn's logfs
(http://wiki.laptop.org/go/Logfs), which should scale well to many gigabytes. A driver based on MMC would be a nice development tool for that, since it enables regular PCs as a debugging machine instead of having to load test kernels onto an actual embedded machine.
Another thing I have been thinking about was an MTD version of fat16/fat32. There are a number of optimizations that you can do for flash media, including:
- limiting the number of writes to the FAT
- erasing blocks when they are freed in the FS
- always writing full erase blocks if the erase block size matches the cluster size
- optimize for wear leveling instead of avoiding fragmentation
I read that the SD cards have some restrictions of how the fat fs needs to be laid out on them, presumably to make sure clusters are aligned with erase blocks. Do you have any specific information on what SD actually requires?
Quoted 5 lines
Quoted 3 lines
I still want to be sure that I'm on the right track with this driver and did not make a conceptual mistake.
I can comment it from a MMC perspective, but the MTD stuff I will have to assume is correct.
ok, that's fine. I've talked to a few MTD people about it already and I understand much more about MTD than I do about MMC ;-)
Quoted 11 lines
drivers/mmc/mmc.c static void mmc_decode_csd(struct mmc_ca · line 623
Quoted 9 lines
drivers/mmc/mmc.c +2
-0@@ -616,6 +616,8 @@ static void mmc_decode_csd(struct mmc_ca csd->r2w_factor =UNSTUFF_BITS (resp,26 ,3 ); csd->write_blkbits =UNSTUFF_BITS (resp,22 ,4 ); csd->write_partial =UNSTUFF_BITS (resp,21 ,1 ); + csd->erase_blksize = (UNSTUFF_BITS (resp,37 ,5 ) +1 ) * + (UNSTUFF_BITS (resp,42 ,5 ) +1 ); }else {/* * We only understand CSD structure v1.1 and v1.2. NAK. SD uses another format for erase blocks. See the simplified physical spec.
ok, I'll have a look. I keep having trouble identifying the right specifications (physical spec sounded like it was only about wiring and electric properties, so I did not look at that). Maybe it would be good if you could put pointers to the relevant documents into your Wiki?
Quoted 8 lines
Quoted 8 lines
drivers/mmc/mmc_mtd.c new file +8
-0+/* +* transfer a block to/from the card. The block needs to be aligned +* to mtd->writesize. If we want to implement an mtd_writev method, +* this needs to use stream operations with an appropriate stop +* command as well. +*/ +static int mmc_mtd_transfer_low (struct mmc_card *card,loff_t off,size_t len, +size_t *retlen,u_char *buf,int write)
Quoted 8 lines
drivers/mmc/mmc_mtd.c line 94
Quoted 5 lines
drivers/mmc/mmc_mtd.c new file +5
-0+{ +struct scatterlist sg; +struct mmc_data data = { + .blksz =1 << card->csd.read_blkbits, + .blocks = len >> card->csd.read_blkbits,First of all, you cannot assume that
read_blkbitsis a valid block size when doing writes.
Right, I see. I introduced that bug when I merged parts of the read and write paths.
Is it fair to assume that write_blkbits is always bigger than read_blkbits, so that one can be used in both cases?
Quoted 2 lines
Secondly, the cards default in a block size of 512 bytes, so you need to tell the card your desired block size during probe.
Ok.
Quoted 13 lines
drivers/mmc/mmc_mtd.c line 103
Quoted 9 lines
drivers/mmc/mmc_mtd.c new file +9
-0+ .flags = write ? MMC_DATA_WRITE : MMC_DATA_READ, + .sg = &sg, + .sg_len = 1, + }; +struct mmc_command cmd = { + .arg = off, + .data = &data, + .flags =MMC_RSP_R1 |MMC_CMD_ADTC , + .opcode = write ?MMC_WRITE_BLOCK :MMC_READ_SINGLE_BLOCK ,You set .blocks above, so I have to assume it can be more than 1. So you need to change the opcodes accordingly.
Quoted 7 lines
drivers/mmc/mmc_mtd.c line 108
Quoted 5 lines
drivers/mmc/mmc_mtd.c new file +5
-0+ }; +struct mmc_request mrq = { + .cmd = &cmd, + .data = &data, + };And it also means you need a stop command.
I tried to do multiple block access at first, but then took it out again. If it turns out valuable to have these, I'll implement it properly later. Does it make a difference performance-wise to do larger accesses?
Quoted 5 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ + /* copied from the block driver, don't understand why this is needed */Now this gives me a bad feeling. Have you read any spec about the MMC protocol or are you just winging it?
Understanding the details would be one of the next steps. This was just a quick hack for now, to see how complex the driver gets, and to see if it's worth doing.
Quoted 2 lines
It is needed because the card goes into programming state after a write, where it is very unresponsive to other commands.
ok, thanks for the explanation.
Quoted 4 lines
Quoted 4 lines
drivers/mmc/mmc_mtd.c new file +4
-0+ + ret =mmc_card_claim_host (card); +if (ret) { + dev_warn(&card->dev, "%s: mmc_card_claim_host returned %d\n",
Quoted 8 lines
drivers/mmc/mmc_mtd.c line 157
Quoted 4 lines
drivers/mmc/mmc_mtd.c new file +4
-0+ __FUNCTION__, ret); + ret = -EIO ; + goto error; + }
mmc_card_claim_host()is currently very stupid in that it requires you to callmmc_card_release_host()on error. I intend to fix that some time in the future.
ok.
Quoted 17 lines
Quoted 4 lines
drivers/mmc/mmc_mtd.c new file +4
-0+/* +* Initialize an mmc card. We create a new MTD device for each +* MMC card we find. The operations are rather straightforward, +* so we don't even need our own data structure to contain the drivers/mmc/mmc_mtd.c line 190
Quoted 10 lines
drivers/mmc/mmc_mtd.c new file +10
-0+* mtd_info. +*/ +static int mmc_mtd_probe (struct mmc_card *card) +{ +struct mtd_info *mtd; +int ret; + +if (!(card->csd.cmdclass &CCC_ERASE )) +return -ENODEV ; +You should probably check for
CCC_BLOCK_READhere.
Quoted 2 lines
And your driver needs to check if the card support writes (both by
mmc_card_readonly()andCCC_BLOCK_WRITE).
ok.
Thanks a lot for your detailed response!
Arnd <><
Quoted 7 lines · Arnd Bergmann
There are multiple efforts in progress to get a jffs2 replacement. NAND flash in embedded devices has the same size as it has on MMC card potentially, so we will need one soon. David Woodhouse has pushed the limit that jffs2 can reasonably used to 512MB, which is the size used in the OLPC XO laptop. If there are ways to get beyond that (which I find unlikely), there will be a hard limit 2GB or 4GB because of limitations in the fs layout.
The main weakness of JFFS2 (at this kind of size) is that there _is_ no fs layout -- so there isn't a hard 2GiB or 4GiB limit in the format, because we never encode offsets anywhere but in memory.
We'll push JFFS2 further than the current 512MiB by enlarging the data nodes -- so each node covers something like 16KiB of data instead of only 4KiB, and then there'll be about 1/3 as many of them, which will cut the memory usage and reduce the amount we need to read in the "summary" blocks. But logfs is the way forward, I agree.
Signature · 1 line
dwmw2
Quoted 8 lines · Arnd Bergmann
One promising effort for a replacement is Jörn's logfs
(http://wiki.laptop.org/go/Logfs), which should scale well to many gigabytes. A driver based on MMC would be a nice development tool for that, since it enables regular PCs as a debugging machine instead of having to load test kernels onto an actual embedded machine.
A bit of a niche area, but as long as this driver doesn't look like high maintenance then it could be enough.
Quoted 11 lines
Another thing I have been thinking about was an MTD version of fat16/fat32. There are a number of optimizations that you can do for flash media, including:
- limiting the number of writes to the FAT
- erasing blocks when they are freed in the FS
- always writing full erase blocks if the erase block size matches the cluster size
- optimize for wear leveling instead of avoiding fragmentation
These sound like they would be nicer in the block layer, to cover other devices where you know there is flash at the bottom.
Quoted 6 lines
I read that the SD cards have some restrictions of how the fat fs needs to be laid out on them, presumably to make sure clusters are aligned with erase blocks. Do you have any specific information on what SD actually requires?
No, as we don't give a rats ass about them. I don't know why they stuck a FAT requirement into the spec. Perhaps Microsoft wanted a chance at the extortio^Wlicense money for any patent issues.
Quoted 4 lines
ok, I'll have a look. I keep having trouble identifying the right specifications (physical spec sounded like it was only about wiring and electric properties, so I did not look at that). Maybe it would
That had me fooled for quite a while as well.
Quoted 2 lines
be good if you could put pointers to the relevant documents into your Wiki?
Probably. I haven't really put that much time into the wiki lately. It turned out to be a one man show, so I'm doubting its usefulness.
Quoted 9 lines
Quoted 2 lines
First of all, you cannot assume that
read_blkbitsis a valid block size when doing writes.Right, I see. I introduced that bug when I merged parts of the read and write paths.
Is it fair to assume that
write_blkbitsis always bigger thanread_blkbits, so that one can be used in both cases?
There is some relation, yes, but I don't remember the details right now. More important is that the card can only be set to one block size at any given time (both read and write). So unless you want terrible latency by switching block size back and forth I'd suggest selecting one size and sticking with it.
As the newer cards only support a block size of 512 bytes, the most future proof would be to use that.
Quoted 5 lines
I tried to do multiple block access at first, but then took it out again. If it turns out valuable to have these, I'll implement it properly later. Does it make a difference performance-wise to do larger accesses?
Yes. On my rather slow ISA device, the speedup was over 100% for writes.
Rgds
Signature · 5 lines
-- Pierre Ossman Linux kernel, MMC maintainer http://www.kernel.org PulseAudio, core developer http://pulseaudio.org rdesktop, core developer http://www.rdesktop.org
This is a new version of the driver I posted back in January. I now have hardware to test it and fixed a number of bugs, most of which are the ones that Pierre told me about in the first place.
It now seems to work fine with the mtdblock driver, which of course it entirely pointless.
I've tried using it with jffs2 once, but get an immediate oops, which still needs some investigation.
I'm also not sure what to do about SDHC media, but probably the easiest solutions is to disallow them with this driver -- the mtd layer doesn't deal with media larger than 4GB anyway at this point.
There is also still some need for performance testing. Jörn brought up the point that if a specific card can't have multiple open erase block simulateously, it's rather pointless for logfs. It might still be useful to use jffs2 on those cards, because IFAIK that only writes to one erase block at any time.
---
drivers/mmc/mmc.c +5 -0
@@ -621,6 +621,7 @@ static void mmc_decode_csd(struct mmc_ca csd->r2w_factor =UNSTUFF_BITS (resp,26 ,3 ); csd->write_blkbits =UNSTUFF_BITS (resp,22 ,4 ); csd->write_partial =UNSTUFF_BITS (resp,21 ,1 ); + csd->erase_blksize =UNSTUFF_BITS (resp,39 ,7 );break ;case 1 :/*
@@ -649,6 +650,8 @@ static void mmc_decode_csd(struct mmc_ca csd->r2w_factor =4 ;/* Unused */ csd->write_blkbits =9 ; csd->write_partial =0 ; +#warning need to read au_size for sdhc + csd->erase_blksize =0 ;// 8192 << au_size; break ;default : printk("%s: unrecognised CSD structure version %d\n",
@@ -691,6 +694,8 @@ static void mmc_decode_csd(struct mmc_ca csd->r2w_factor =UNSTUFF_BITS (resp,26 ,3 ); csd->write_blkbits =UNSTUFF_BITS (resp,22 ,4 ); csd->write_partial =UNSTUFF_BITS (resp,21 ,1 ); + csd->erase_blksize = (UNSTUFF_BITS (resp,37 ,5 ) +1 ) * + (UNSTUFF_BITS (resp,42 ,5 ) +1 ); } }
include/linux/mmc/card.h +1 -0
@@ -32,6 +32,7 @@ struct mmc_csd {unsigned int max_dtr;unsigned int read_blkbits;unsigned int write_blkbits; +unsigned int erase_blksize;unsigned int capacity; unsigned int read_partial:1, read_misalign:1,
drivers/mmc/mmc_mtd.c new file +366 -0
366 lines not on this page · raw message
drivers/mmc/Kconfig +11 -0
@@ -29,6 +29,17 @@ config MMC_BLOCK mount the filesystem. Almost everyone wishing MMC support should say Y or M here. +config MMC_MTD + tristate "MMC MTD device driver" + depends on MMC && MTD && (MMC_BLOCK = n || (MMC_BLOCK != y && MMC_BLOCK)) + help + Enable this to as an alternative to the MMC block driver. + It makes it possible to use MMC memory cards with the + MTD layer and file systems based on it, e.g. jffs2. + Since both drivers operate on the same devices, you can't + build them both into the kernel and you need to say M here + if you want to build both. + config MMC_ARMMMCI tristate "ARM AMBA Multimedia Card Interface support" depends on ARM_AMBA && MMC
drivers/mmc/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(CONFIG_MMC) += mmc_core.o # Media drivers # obj-$(CONFIG_MMC_BLOCK) += mmc_block.o +obj-$(CONFIG_MMC_MTD) += mmc_mtd.o # # Host drivers
Quoted 7 lines · Arnd Bergmann
There is also still some need for performance testing. Jörn brought up the point that if a specific card can't have multiple open erase block simulateously, it's rather pointless for logfs. It might still be useful to use jffs2 on those cards, because IFAIK that only writes to one erase block at any time.
This appears to be a problem for practically all consumer-available flash media. They spend a lot of effort trying to hide any flash properties from their users. And while this is a decent strategy for FAT, ext3, ntfs and similar, it is actually very inefficient for a flash filesystem.
After talking to several manufacturers, most seemed to be fairly open-minded towards supporting an alternate interface with raw flash access. So much for the good news. Bad news is that such an elternate interface still needs to be defined.
Jörn
Signature · 3 lines
Courage is not the absence of fear, but rather the judgement that something else is more important than fear. -- Ambrose Redmoon