erofs-utils: support --blobdev with block map
PATCH 1/1 erofs-utils: support --blobdev with block map
Add support for using --blobdev with the FORCE_INODE_BLOCK_MAP chunk format. Previously this combination returned -EINVAL with a TODO noting it could be implemented via deviceslot mapped_blkaddr.
This implements it by utilizing the deviceslot unified addressing (uniaddr) to map external blob device blocks into a contiguous address space, bypassing the need for 8-byte chunk indexes.
For block map format, chunk addresses are encoded as:
blkaddr = devs[0].uniaddr + erofs_blknr(data_offset)
At mount time, the kernel's erofs_map_dev() resolves these unified addresses back to the correct external device transparently.
Changes:
- mkfs/main.c: remove the error blocking blobdev + block map
- lib/super.c: initialize uniaddr with primarydevice_blocks
- lib/blobchunk.c: encode unified addresses in 4-byte block maps
- lib/data.c: fix erofs_map_dev() to set m_deviceid when resolving unified addresses, so userspace tools (fsck, dump) work correctly
--- lib/blobchunk.c | 19 +++++++++++++++++-- lib/data.c | 1 + lib/super.c | 5 +++++ mkfs/main.c | 9 ++++++--- 4 files changed, 29 insertions(+), 5 deletions(-)
lib/blobchunk.c +17 -2
@@ -95,7 +95,14 @@ static struct erofs_blobchunk *erofs_blob_getchunk(struct erofs_sb_info *sbi, chunk->device_id =1 ; else chunk->device_id = 0; - chunk->blkaddr =erofs_blknr (sbi, blkpos); + +/* For block map with blobdev, use unified addressing */ +if (sbi->extra_devices && cfg.c_force_chunkformat ==FORCE_INODE_BLOCK_MAP ) { + chunk->blkaddr = sbi->devs[0 ].uniaddr +erofs_blknr (sbi, blkpos); + chunk->device_id =0 ;/* unified address space */ + }else { + chunk->blkaddr =erofs_blknr (sbi, blkpos); + }erofs_dbg ("Writing chunk (%llu bytes) to %llu" , chunksize |0ULL , chunk->blkaddr |0ULL );
@@ -324,7 +331,7 @@ int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd, chunksize =1ULL << chunkbits; count =DIV_ROUND_UP (inode->i_size, chunksize); -if (sbi->extra_devices) +if (sbi->extra_devices && cfg.c_force_chunkformat !=FORCE_INODE_BLOCK_MAP ) inode->u.chunkformat |=EROFS_CHUNK_FORMAT_INDEXES ;if (inode->u.chunkformat &EROFS_CHUNK_FORMAT_INDEXES ) unit =sizeof (struct erofs_inode_chunk_index );
@@ -498,6 +505,14 @@ int tarerofs_write_chunkes(struct erofs_inode *inode, erofs_off_t data_offset) unit =sizeof (struct erofs_inode_chunk_index );DBG_BUGON (erofs_blkoff (sbi, data_offset)); blkaddr =erofs_blknr (sbi, data_offset); + }else if (cfg.c_blobdev_path && + cfg.c_force_chunkformat ==FORCE_INODE_BLOCK_MAP ) { +/* Block map with blobdev: use unified addressing via deviceslot */ + device_id =0 ;/* unified address space, no device_id needed */ + unit =EROFS_BLOCK_MAP_ENTRY_SIZE ; +DBG_BUGON (erofs_blkoff (sbi, data_offset)); +/* Map to unified address space using device's uniaddr base */ + blkaddr = sbi->devs[0 ].uniaddr +erofs_blknr (sbi, data_offset); }else { device_id =0 ; unit =EROFS_BLOCK_MAP_ENTRY_SIZE ;
lib/data.c +1 -0
@@ -184,6 +184,7 @@ int erofs_map_dev(struct erofs_sb_info *sbi, struct erofs_map_dev *map)if (map->m_pa >= startoff && map->m_pa < startoff + length) { map->m_pa -= startoff; + map->m_deviceid = id +1 ;break ; } }
lib/super.c +5 -0
@@ -393,6 +393,11 @@ int erofs_mkfs_init_devices(struct erofs_sb_info *sbi, unsigned int devices) sbi->bh_devt = bh; sbi->devt_slotoff =erofs_btell (bh, false) /EROFS_DEVT_SLOT_SIZE ; sbi->extra_devices = devices; + +/* Initialize uniaddr for block map with blobdev support */ +if (devices >0 ) + sbi->devs[0 ].uniaddr = sbi->primarydevice_blocks; +erofs_sb_set_device_table (sbi);return 0 ; }
mkfs/main.c +6 -3
@@ -1565,11 +1565,14 @@ static int mkfs_parse_options_cfg(struct erofs_importer_params *params,return -EINVAL ; } -/* TODO: can be implemented with (deviceslot) mapped_blkaddr */ +/* Blobdev with block map uses deviceslot unified addressing */ if (cfg.c_blobdev_path && cfg.c_force_chunkformat ==FORCE_INODE_BLOCK_MAP ) { -erofs_err ("--blobdev cannot work with block map currently" ); -return -EINVAL ; +/* Block map format can work with blobdev using unified addressing */ +if (!g_sbi.extra_devices) { +erofs_err ("--blobdev requires device table setup" ); +return -EINVAL ; + } }if (optind >= argc) {
Signature · 1 line
2.51.0.windows.1
Hi Ajay,
Thanks for looking into this, but this patch is redundant. I already addressed this exact TODO item last month.
My initial patch was sent on March 7th. After discussing the implementation with the maintainers, specifically addressing Yifan Zhao's feedback regarding the 32-bit address limits for block map entries and the necessary fix to reset m_deviceid in erofs_map_dev(), I submitted a finalized v2 on March 9th.
Additionally, following Gao Xiang's guidance, I have already written and submitted the corresponding test case to theexperimental-tests branch as of March 19th.
I strongly suggest checking the recent mailing list archives or the pending patch queue before picking up a TODO item to avoid stepping on ongoing work and duplicating effort.
Regards,
Nithurshen
Quoted 11 lines · Nithurshen
Hi Ajay,
Thanks for looking into this, but this patch is redundant. I already addressed this exact TODO item last month.
My initial patch was sent on March 7th. After discussing the implementation with the maintainers, specifically addressing Yifan Zhao's feedback regarding the 32-bit address limits for block map entries and the necessary fix to reset
m_deviceidinerofs_map_dev(), I submitted a finalized v2 on March 9th.
Quoted 7 lines
Additionally, following Gao Xiang's guidance, I have already written and submitted the corresponding test case to theexperimental-tests branch as of March 19th.
I strongly suggest checking the recent mailing list archives or the pending patch queue before picking up a TODO item to avoid stepping on ongoing work and duplicating effort.
I really think you guys don't focus on the random stuff: this TODO really has low priority.
Thanks,
Gao Xiang
Quoted 3 lines
Regards,
Nithurshen
Okay sir, I got it.
Thanks,
Ajay Rajera
Quoted 33 lines · Gao Xiang
Quoted 11 lines · Nithurshen
Hi Ajay,
Thanks for looking into this, but this patch is redundant. I already addressed this exact TODO item last month.
My initial patch was sent on March 7th. After discussing the implementation with the maintainers, specifically addressing Yifan Zhao's feedback regarding the 32-bit address limits for block map entries and the necessary fix to reset
m_deviceidinerofs_map_dev(), I submitted a finalized v2 on March 9th.Quoted 7 lines
Additionally, following Gao Xiang's guidance, I have already written and submitted the corresponding test case to theexperimental-tests branch as of March 19th.
I strongly suggest checking the recent mailing list archives or the pending patch queue before picking up a TODO item to avoid stepping on ongoing work and duplicating effort.
I really think you guys don't focus on the random stuff: this TODO really has low priority.
Thanks,
Gao XiangQuoted 3 lines
Regards,
Nithurshen
I apologize, I didn't know about it.
Thanks,
Ajay
Quoted 22 lines · Nithurshen
Hi Ajay,
Thanks for looking into this, but this patch is redundant. I already addressed this exact TODO item last month.
My initial patch was sent on March 7th. After discussing the implementation with the maintainers, specifically addressing Yifan Zhao's feedback regarding the 32-bit address limits for block map entries and the necessary fix to reset
m_deviceidinerofs_map_dev(), I submitted a finalized v2 on March 9th.Additionally, following Gao Xiang's guidance, I have already written and submitted the corresponding test case to theexperimental-tests branch as of March 19th.
I strongly suggest checking the recent mailing list archives or the pending patch queue before picking up a TODO item to avoid stepping on ongoing work and duplicating effort.
Regards,
Nithurshen