exfatprogs: optimise mkfs zeroing out and fix stdio output
Subsystems (2)
cover exfatprogs: optimise mkfs zeroing out and fix stdio output
Following is the performance impact profiling. I don't have a spare NVME SSD to test with so I used AWS EC2 :(
Your contributions welcome.
I'm sure pgsteal_* in /proc/vmstat can be used as yet another metric on a busy host, but I didn't bother to go that far to collect that data.
UAS:
- Host: Raspberry Pi 4 Model B Rev 1.5
- Device: Samsung BAR+ 64GB (via USB 3.0)
- Target partition size: 1998585856 bytes
Pre-patch:
System time (seconds): 4.54
Percent of CPU this job got: 7%
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:05.14 Maximum resident set size (kbytes): 3200 Voluntary context switches: 10297 Involuntary context switches: 978
File system inputs: 2168
File system outputs: 3903664
Post-patch:
System time (seconds): 0.29
Percent of CPU this job got: 0%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:56.49 Maximum resident set size (kbytes): 3200 Voluntary context switches: 5792 Involuntary context switches: 1322
File system inputs: 2224
File system outputs: 176
NVME:
- Host: AWS EC2 c7gd.medium
- Device: "locally attached" SSD volume
- Size: 59000000000 bytes
Pre-patch:
System time (seconds): 22.64
Percent of CPU this job got: 0%
Elapsed (wall clock) time (h:mm:ss or m:ss): 45:47.70 Maximum resident set size (kbytes): 2792 Voluntary context switches: 227200 Involuntary context switches: 7352
File system inputs: 1360
File system outputs: 115235032
Post-patch:
System time (seconds): 1.14
Percent of CPU this job got: 0%
Elapsed (wall clock) time (h:mm:ss or m:ss): 26:08.06 Maximum resident set size (kbytes): 2788 Voluntary context switches: 56409 Involuntary context switches: 60
File system inputs: 1727
File system outputs: 656
David Timber (8):
exfat2img, treewide: do not clobber stdout
mkfs, fsck: refactor getopt_long() return values
treewide: fix factual errors and reduce column length of usage message
libexfat: refactor exfat_bad_char()
libexfat: add exfat_secerase_blocks() and exfat_zeroout_blocks()
libexfat: add exfat_trace()
mkfs: use BLKZEROOUT for disk zeroing out
mkfs: issue POSIX_FADV_NOREUSE before zeroing out
dump/dump.c | 14 +-- exfat2img/exfat2img.c | 2 +- fsck/fsck.c | 14 ++- include/libexfat.h | 33 +++++- label/label.c | 6 +- lib/libexfat.c | 114 +++++++++++++------ mkfs/mkfs.c | 254 +++++++++++++++++++++++++++++------------- mkfs/mkfs.h | 1 + tune/tune.c | 18 +-- 9 files changed, 316 insertions(+), 140 deletions(-)
Signature · 1 line
2.55.0
PATCH 1/8 exfat2img, treewide: do not clobber stdout
Print the version info to stderr instead of stdout in show_version() to not interfere with the utils that use stdout to output data.
exfat2img uses the stdout fd(1) to output the metadata image file, but the stdout FILE pointer is used directly throughout the program via [p]write(). This is UB: if the stdout is flushed before the program starts using the fd, the strings buffered in printf() may end up at the start of the metadata file. Although, with glibc, this doesn't seem to be the case:
$ sudo exfat2img -o - /dev/... | hexdump -C | tail * 05a404c0 00 00 00 00 04 f7 62 1d 00 65 78 66 61 74 70 72 |......b..exfatpr| 05a404d0 6f 67 73 20 76 65 72 73 69 6f 6e 20 3a 20 31 2e |ogs version : 1.| 05a404e0 34 2e 32 20 28 32 30 32 36 2d 30 36 2d 31 35 29 |4.2 (2026-06-15)| 05a404f0 0a 31 37 35 33 39 20 66 69 6c 65 73 20 66 6f 75 |.17539 files fou| 05a40500 6e 64 2c 20 32 34 39 20 64 69 72 65 63 74 6f 72 |nd, 249 director| 05a40510 69 65 73 20 64 75 6d 70 65 64 2c 20 39 32 34 31 |ies dumped, 9241| 05a40520 36 20 6b 62 79 74 65 73 20 77 72 69 74 74 65 6e |6 kbytes written| 05a40530 0a |.| 05a40531
Before exiting exfat2img, print the stats data to stderr so that the output data is not clobbered.
--- exfat2img/exfat2img.c | 2 +- lib/libexfat.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
exfat2img/exfat2img.c +1 -1
@@ -995,7 +995,7 @@ int main(int argc, char * const argv[]) } } -printf ("%ld files found, %ld directories dumped, %llu kbytes written\n" , +fprintf (stderr,"%ld files found, %ld directories dumped, %llu kbytes written\n" , exfat_stat.file_count, exfat_stat.dir_count, (unsigned long long )DIV_ROUND_UP (exfat_stat.written_bytes,1024 ));
lib/libexfat.c +1 -1
@@ -288,7 +288,7 @@ void boot_calc_checksum(const unsigned char *sector, size_t size,void show_version (void ) { -printf ("exfatprogs version : %s (%s)\n" , +fprintf (stderr,"exfatprogs version : %s (%s)\n" ,EXFAT_PROGS_VERSION ,EXFAT_PROGS_RELEASE_DATE ); }
Signature · 1 line
2.55.0
PATCH 2/8 mkfs, fsck: refactor getopt_long() return values
In reply to David TimberReplace longopt ids with enum for the return values of getopt_long() for cleaner code. No functional change.
--- fsck/fsck.c | 14 ++++++++------ mkfs/mkfs.c | 20 +++++++++++--------- 2 files changed, 19 insertions(+), 15 deletions(-)
fsck/fsck.c +8 -6
@@ -48,8 +48,10 @@ struct exfat_fsck exfat_fsck;struct exfat_stat exfat_stat;struct path_resolve_ctx path_resolve_ctx; -#define OPTID_PUT_MBR (128 + 0) -#define OPTID_CLEAR_MBR (128 + 1) +enum longopt_num { +LON_PUT_MBR =CHAR_MAX +1 , +LON_CLEAR_MBR , +};static struct option opts[] = { {"repair" , no_argument,NULL ,'r' },
@@ -63,8 +65,8 @@ static struct option opts[] = { {"?", no_argument, NULL, '?' }, {"ignore-bad-fs", no_argument, NULL, 'b' }, {"progress", no_argument, NULL, 'P' }, - {"put-mbr", no_argument, NULL, OPTID_PUT_MBR }, - {"clear-mbr", no_argument, NULL, OPTID_CLEAR_MBR }, + {"put-mbr", no_argument, NULL, LON_PUT_MBR }, + {"clear-mbr", no_argument, NULL, LON_CLEAR_MBR }, {NULL, 0, NULL, 0} };
@@ -2026,12 +2028,12 @@ int main(int argc, char * const argv[])case 'P' : ui.options |=FSCK_OPTS_PROGRESS_BAR ;break ; -case OPTID_PUT_MBR : +case LON_PUT_MBR :if (exfat_fsck.mbr)usage (argv[0 ]); exfat_fsck.mbr =1 ;break ; -case OPTID_CLEAR_MBR : +case LON_CLEAR_MBR :if (exfat_fsck.mbr)usage (argv[0 ]); exfat_fsck.mbr =-1 ;
mkfs/mkfs.c +11 -9
@@ -870,9 +870,11 @@ static void usage(void)exit (EXIT_FAILURE ); } -#define PACK_BITMAP (CHAR_MAX + 1) -#define UPCASE_FILE (CHAR_MAX + 2) -#define BOOTCODE_MSG (CHAR_MAX + 3) +enum longopt_num { +LON_PACK_BITMAP =CHAR_MAX +1 , +LON_UPCASE_FILE , +LON_BOOTCODE_MSG , +};static const struct option opts[] = { {"volume-label" , required_argument,NULL ,'L' },
@@ -880,9 +882,9 @@ static const struct option opts[] = { {"sector-size", required_argument, NULL, 's' }, {"cluster-size", required_argument, NULL, 'c' }, {"boundary-align", required_argument, NULL, 'b' }, - {"upcase", required_argument, NULL, UPCASE_FILE }, - {"bootcode-msg", required_argument, NULL, BOOTCODE_MSG }, - {"pack-bitmap", no_argument, NULL, PACK_BITMAP }, + {"upcase", required_argument, NULL, LON_UPCASE_FILE }, + {"bootcode-msg", required_argument, NULL, LON_BOOTCODE_MSG }, + {"pack-bitmap", no_argument, NULL, LON_PACK_BITMAP }, {"full-format", no_argument, NULL, 'f' }, {"force", no_argument, NULL, 'F' }, {"check-written", no_argument, NULL, 'C' },
@@ -1512,13 +1514,13 @@ int main(int argc, char *argv[]) } ui.boundary_align = ret;break ; -case UPCASE_FILE : +case LON_UPCASE_FILE : ui.upcase.file = optarg;break ; -case PACK_BITMAP : +case LON_PACK_BITMAP : ui.pack_bitmap = true;break ; -case BOOTCODE_MSG : +case LON_BOOTCODE_MSG : ui.bootcode_msg = optarg;/* A long message will just get truncated silently. */ break ;
Signature · 1 line
2.55.0
PATCH 3/8 treewide: fix factual errors and reduce column length of usage message
In reply to David TimberThe usage string took too many columns. This is not ideal even with modern terminal emulators. Reduce it to 80.
Fix the factual errors in the usage message of mkfs.
- "Overwrite an existing non-exFAT filesystem" -> "Overwrite existing filesystem": mkfs fails if run without the -F option if exFAT is detected(the whole point of filesystem detection is to prevent unintentional data loss)
- "Verify written filesystem metadata by read-back" -> "Verify written data by read-back": written zeros are also verified
--- dump/dump.c | 14 ++++++-------- label/label.c | 6 +++--- mkfs/mkfs.c | 35 ++++++++++++++++++----------------- tune/tune.c | 18 +++++++++--------- 4 files changed, 36 insertions(+), 37 deletions(-)
dump/dump.c +6 -8
@@ -36,14 +36,12 @@static void usage (void ) {fprintf (stderr,"Usage: dump.exfat\n" ); -fprintf (stderr,"\t-d | --dentry-set=path Show directory entry set\n" ); -fprintf (stderr,"\t-c | --cluster-chain Show cluster chain\n" ); -fprintf (stderr, -"\t-s | --scan-dir=dir-path Scan and show directory entry sets\n" ); -fprintf (stderr, -"\t-r | --recursive Scan and show directory entry sets recursively\n" ); -fprintf (stderr,"\t-V | --version Show version\n" ); -fprintf (stderr,"\t-h | --help Show help\n" ); +fprintf (stderr,"\t-d | --dentry-set=path Show directory entry set\n" ); +fprintf (stderr,"\t-c | --cluster-chain Show cluster chain\n" ); +fprintf (stderr,"\t-s | --scan-dir=dir-path Scan and show directory entry sets\n" ); +fprintf (stderr,"\t-r | --recursive Scan and show directory entry sets recursively\n" ); +fprintf (stderr,"\t-V | --version Show version\n" ); +fprintf (stderr,"\t-h | --help Show help\n" );exit (EXIT_FAILURE ); }
label/label.c +3 -3
@@ -18,9 +18,9 @@static void usage (void ) {fprintf (stderr,"Usage: exfatlabel\n" ); -fprintf (stderr,"\t-i | --volume-serial Switch to volume serial mode\n" ); -fprintf (stderr,"\t-V | --version Show version\n" ); -fprintf (stderr,"\t-h | --help Show help\n" ); +fprintf (stderr,"\t-i | --volume-serial Switch to volume serial mode\n" ); +fprintf (stderr,"\t-V | --version Show version\n" ); +fprintf (stderr,"\t-h | --help Show help\n" );exit (EXIT_FAILURE ); }
mkfs/mkfs.c +18 -17
@@ -848,23 +848,24 @@ static int exfat_create_root_dir(struct exfat_blk_dev *bd,static void usage (void ) {fputs ("Usage: mkfs.exfat\n" -"\t-L | --volume-label=label Set volume label\n" -"\t-U | --volume-guid=guid Set volume GUID\n" -"\t-s | --sector-size=size(or suffixed by 'K') Specify sector size\n" -"\t-c | --cluster-size=size(or suffixed by 'K' or 'M') Specify cluster size\n" -"\t-b | --boundary-align=size(or suffixed by 'K' or 'M') Specify boundary alignment\n" -"\t --pack-bitmap Move bitmap into FAT segment\n" -"\t --upcase=file Specify up-case table binary file\n" -"\t --bootcode-msg=message Specify custom message in MBR bootstrap code\n" -"\t-P | --partition-table=auto|none|mbr|gpt Specify partition table\n" -"\t-f | --full-format Full format\n" -"\t-F | --force Overwrite an existing non-exFAT filesystem\n" -"\t-C | --check-written Verify written filesystem metadata by read-back\n" -"\t-K | --no-discard Do not discard blocks\n" -"\t-V | --version Show version\n" -"\t-q | --quiet Print only errors\n" -"\t-v | --verbose Print debug\n" -"\t-h | --help Show help\n" , +"\t-L | --volume-label=label Set volume label\n" +"\t-U | --volume-guid=guid Set volume GUID\n" +"\t-s | --sector-size=size[K] Specify sector size\n" +"\t-c | --cluster-size=size[K|M] Specify cluster size\n" +"\t-b | --boundary-align=size[K|M] Specify boundary alignment\n" +"\t --pack-bitmap Move bitmap into FAT segment\n" +"\t --upcase=file Specify up-case table binary file\n" +"\t --bootcode-msg=message Specify custom message in MBR\n" +"\t bootstrap code\n" +"\t-P | --partition-table=auto|none|mbr|gpt Specify partition table\n" +"\t-f | --full-format Full format\n" +"\t-F | --force Overwrite existing filesystem\n" +"\t-C | --check-written Verify written data by read-back\n" +"\t-K | --no-discard Do not discard blocks\n" +"\t-V | --version Show version\n" +"\t-q | --quiet Print only errors\n" +"\t-v | --verbose Print debug\n" +"\t-h | --help Show help\n" , stderr);exit (EXIT_FAILURE );
tune/tune.c +9 -9
@@ -18,15 +18,15 @@static void usage (void ) {fprintf (stderr,"Usage: tune.exfat\n" ); -fprintf (stderr,"\t-l | --print-label Print volume label\n" ); -fprintf (stderr,"\t-L | --set-label=label Set volume label\n" ); -fprintf (stderr,"\t-u | --print-guid Print volume GUID\n" ); -fprintf (stderr,"\t-U | --set-guid=guid Set volume GUID\n" ); -fprintf (stderr,"\t-i | --print-serial Print volume serial\n" ); -fprintf (stderr,"\t-I | --set-serial=value Set volume serial\n" ); -fprintf (stderr,"\t-V | --version Show version\n" ); -fprintf (stderr,"\t-v | --verbose Print debug\n" ); -fprintf (stderr,"\t-h | --help Show help\n" ); +fprintf (stderr,"\t-l | --print-label Print volume label\n" ); +fprintf (stderr,"\t-L | --set-label=label Set volume label\n" ); +fprintf (stderr,"\t-u | --print-guid Print volume GUID\n" ); +fprintf (stderr,"\t-U | --set-guid=guid Set volume GUID\n" ); +fprintf (stderr,"\t-i | --print-serial Print volume serial\n" ); +fprintf (stderr,"\t-I | --set-serial=value Set volume serial\n" ); +fprintf (stderr,"\t-V | --version Show version\n" ); +fprintf (stderr,"\t-v | --verbose Print debug\n" ); +fprintf (stderr,"\t-h | --help Show help\n" );exit (EXIT_FAILURE ); }
Signature · 1 line
2.55.0
PATCH 4/8 libexfat: refactor exfat_bad_char()
In reply to David TimberRemove check_bad_utf16_char() and elect exfat_bad_char() as the standard function for checking bad characters in exfat file names. No functional change, no significant change in machine code.
--- include/libexfat.h | 20 +++++++++++++++++++- lib/libexfat.c | 18 +----------------- 2 files changed, 20 insertions(+), 18 deletions(-)
include/libexfat.h +19 -1
@@ -232,7 +232,25 @@ unsigned int exfat_count_used_clusters(const void *bitmap, const size_t size,void show_version (void ); -wchar_t exfat_bad_char (wchar_t w); +static inline bool exfat_bad_char (unsigned short w) +{ +switch (w) { +case 0x0022 :/* QUOTATION MARK: (") */ +case 0x002A :/* ASTERISK: (*) */ +case 0x002F :/* FORWARD SLASH: (/) */ +case 0x003A :/* COLON: (:) */ +case 0x003C :/* LESS-THAN SIGN: (<) */ +case 0x003E :/* GREATER-THAN SIGN: (>) */ +case 0x003F :/* QUESTION MARK: (?) */ +case 0x005C :/* BACKSLASH: (\) */ +case 0x007C :/* PIPE: (|) */ +return true; + } + +/* Control characters */ +return w <0x0020 ; +} +void boot_calc_checksum (const unsigned char *sector,size_t size,bool is_boot_sec,__le32 *checksum);void exfat_init_user_input (struct exfat_user_input *ui);
lib/libexfat.c +1 -17
@@ -257,15 +257,6 @@ unsigned int exfat_count_used_clusters(const void *bitmap, const size_t size,return ret; } - -wchar_t exfat_bad_char (wchar_t w) -{ -return (w <0x0020 ) - || (w =='*' ) || (w =='?' ) || (w =='<' ) || (w =='>' ) - || (w =='|' ) || (w =='"' ) || (w ==':' ) || (w =='/' ) - || (w =='\\' ); -} -void boot_calc_checksum (const unsigned char *sector,size_t size,bool is_boot_sec,__le32 *checksum) {
@@ -2013,19 +2004,12 @@ int exfat_cmp_swver(const unsigned short *in_a, const unsigned short *in_b)return 0 ; } -static inline int check_bad_utf16_char (unsigned short w) -{ -return (w <0x0020 ) || (w =='*' ) || (w =='?' ) || (w =='<' ) || - (w =='>' ) || (w =='|' ) || (w =='"' ) || (w ==':' ) || - (w =='/' ) || (w =='\\' ); -} -int exfat_check_name (__le16 *utf16_name,int len) {int i;for (i =0 ; i < len; i++) { -if (check_bad_utf16_char (le16_to_cpu (utf16_name[i]))) +if (exfat_bad_char (le16_to_cpu (utf16_name[i])))break ; }
Signature · 1 line
2.55.0
PATCH 5/8 libexfat: add exfat_secerase_blocks() and exfat_zeroout_blocks()
In reply to David TimberAdd the wrapper functions for BLKSECDISCARD and BLKZEROOUT. In IO stats, BLKSECDISCARD and BLKDISCARD are accounted using the same variables because, in practice, only either one of them will be used in a single program run. As a way of determining which one is used, the bit EXFAT_IOSTAT_SECURE is set in io_stat.err to save memory.
Do not require BLKDISCARD at compile time to support libc from kernel version 2.6.x era.
No functional change in the utils as of yet.
--- include/libexfat.h | 9 +++++ lib/libexfat.c | 94 ++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 87 insertions(+), 16 deletions(-)
include/libexfat.h +9 -0
@@ -299,7 +299,16 @@ bool exfat_write_full(int fd, const void *buf, size_t size, off_t offset);int exfat_write_zero (int fd,off_t size,off_t offset);int exfat_write_zero2 (int fd,off_t size,off_t offset,size_t bs);int exfat_fsync (int fd); +/* BLKDISCARD wrapper */ int exfat_discard_blocks (int fd,uint64_t start,uint64_t len); +/* +* BLKSECDISCARD wrapper (for sdcards that support it) +* +* Link: https://lwn.net/Articles/395147/ +*/ +int exfat_secerase_blocks (int fd,uint64_t start,uint64_t len); +/* BLKZEROOUT wrapper (for devices that support it) */ +int exfat_zeroout_blocks (int fd,uint64_t start,uint64_t len);size_t exfat_utf16_len (const __le16 *str,size_t max_size);ssize_t exfat_utf16_enc (const char *in_str,__u16 *out_str,size_t out_size);
lib/libexfat.c +78 -16
152 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 6/8 libexfat: add exfat_trace()
In reply to David TimberAdd the macro function exfat_trace(), a higher print level than exfat_debug() for debug trace.
--- include/libexfat.h | 4 ++++ 1 file changed, 4 insertions(+)
include/libexfat.h +4 -0
@@ -498,6 +498,7 @@ extern unsigned int print_level;#define EXFAT_ERROR (1)#define EXFAT_INFO (2)#define EXFAT_DEBUG (3) +#define EXFAT_TRACE0 (4)#define exfat_msg (level, dir, fmt, ...) \ do { \
@@ -513,6 +514,9 @@ extern unsigned int print_level;#define exfat_debug (fmt, ...) exfat_msg(EXFAT_DEBUG, stdout, \ "[%s:%4d] " fmt, __func__, \ __LINE__, ##__VA_ARGS__) +#define exfat_trace (N , fmt, ...) exfat_msg(EXFAT_TRACE0 + (N), \ + stdout, "[%s:%4d] " fmt, \ + __func__, __LINE__, ##__VA_ARGS__)#define exfat_print_guid (f, msg, guid) \ f("%s: %02x%02x%02x%02x-%02x%02x-%02x%02x-%02x%02x-%02x%02x%02x%02x%02x%02x\n",\
Signature · 1 line
2.55.0
PATCH 7/8 mkfs: use BLKZEROOUT for disk zeroing out
In reply to David TimberTo prevent meaningless buffer copy from userspace to kernel for zeroing out the entire disk, use BLKZEROOUT ioctl.
If the underlying transport protocol has efficient method of conveying zeros such as SCSI WRITE_SAME and NVME Write Zeroes, the underlying kernel driver should be able to use the command to zero out the regions of the storage device. If not, the kernel should fallback to issuing write command with buffer filled with zeroes which is still a win since no significant amount of data is copied between the user and kernel space.
Abstract the existing stepping algorithm as the new function iterate_over_disk() for BLKDISCARD and BLKZEROOUT. Select the write size using a crude linear algorithm based on the size of the target device. The possible values are generally much larger than the previous fixed value(cluster size) for greater efficiency.
If run with the option -q, be quiet. If the option -v is used more than once, interpret that as raising the print level to and above EXFAT_TRACE0. In order to trace the progress of BLKDISCARD and BLKZEROOUT of the entire disk, run mkfs.exfat -vv ...
--- mkfs/mkfs.c | 165 ++++++++++++++++++++++++++++++++++++---------------- mkfs/mkfs.h | 1 + 2 files changed, 115 insertions(+), 51 deletions(-)
mkfs/mkfs.c +114 -51
@@ -1135,29 +1135,124 @@ static int exfat_build_mkfs_info(struct exfat_blk_dev *bd, struct exfat_user_inpreturn 0 ; } +/* +* The block discarding/zeroing happens in smaller batches so it can be +* interrupted prematurely +*/ +static int iterate_over_disk (int fd,uint64_t len,const uint64_t step, +int (*foreach_step)(int fd,uint64_t start,uint64_t len), +const char *human_name,const char *api_name) +{ +uint64_t offset =0 ; +uint64_t tmp_step; +int err; + +/* The same check seen in the kernel */ +assert (step && (step &511 ) ==0 ); + +while (offset < len) { + tmp_step = len - offset; +if (step < tmp_step) + tmp_step = step; + + err =foreach_step (fd, offset, tmp_step); +/* +* We intentionally ignore errors from the ioctl. It is not +* necessary for the mkfs functionality but just an +* optimization. However we should stop on error. +*/ +if (err ==0 ) { +if (offset ==0 ) { +if (human_name !=NULL ) +exfat_info ("%s blocks: " , human_name); +exfat_debug ("%s: " , api_name); + } +exfat_trace (0 ,"%" PRIu64"-%" PRIu64"\n" , offset, offset + tmp_step); +fflush (stdout); + }else { +exfat_debug ("%s: %s\n" , api_name,strerror (err)); +if (offset >0 && human_name !=NULL ) +exfat_info ("\n" ); +return err; + } + + offset += tmp_step; + } +if (offset >0 && human_name !=NULL ) +exfat_info ("done\n" ); + +return 0 ; +} +static int exfat_zero_out_disk (struct exfat_blk_dev *bd, -struct exfat_user_input *ui) +struct exfat_user_input *ui,const bool quiet) {int ret =0 ;bool mapped = false;const void *zm =NULL ; -const size_t iosize = ui->cluster_size; +size_t iosize;unsigned long long target_zerolen; -assert (iosize >0 ); +assert (ui->cluster_size >0 );if (ui->quick) target_zerolen =MIN (bd->size,EXFAT_HEAD_ZERO_OUT );else target_zerolen = bd->size; -exfat_info ("Zeroing out %llu bytes: " , target_zerolen); -fflush (stdout); +if (!quiet) { +exfat_info ("Zeroing out %llu bytes: " , target_zerolen); +fflush (stdout); + } +if (bd->isblk) { +/* +* When mmap() fails, the memory will be calloc()'d so we can't be so generous +* here. +* +* The write speed of SSDs goes up to few TB's. 32MB may seem so small, but +* that's still 32K syscalls/NVME commands per second, which is not that much on +* modern systems. +* +* Can't go over 64MB because modern consumer HDDs are still stuck at ~100MB/s. +*/ +if (bd->size >=1024ULL *GB ) +/* >= 1TB: 32MB */ + iosize =32 *MB ; +else if (bd->size >=256ULL *GB ) +/* >= 256GB: 16MB */ + iosize =16 *MB ; +else if (bd->size >=64ULL *GB ) +/* >= 64GB: 8MB */ + iosize =8 *MB ; +else +/* 1MB("legacy" devices) */ + iosize =1 *MB ; + +/* +* Notes on cache coherency issues with BLKZEROOUT ioctl +* +* Historically, it seems that the Linux kernel suffered from cache coherency +* issues with BLKZEROOUT. However, it shouldn't affect mkfs.exfat because mkfs +* doesn't read the existing data on the target device after performing foreign +* filesystem detection. Except for the case of -C option, mkfs only writes to +* the device. For -C option, mkfs accesses the device via O_DIRECT so it's not +* affected by the cache coherency issue. +* +* Link: https://lwn.net/Articles/702632/ +*/ + ret =iterate_over_disk (bd->dev_fd, target_zerolen, iosize, + exfat_zeroout_blocks,NULL ,"BLKZEROOUT" ); +if (ret ==0 ) + goto done; + } + +/* File and ENOTTY fallback: this is buffered I/O */ + iosize = ui->cluster_size; ret =exfat_write_zero2 (bd->dev_fd, target_zerolen,0 , iosize);if (ret) goto out; - +done:if (ui->verify) {off_t ofs =0 , rem = target_zerolen;
@@ -1185,7 +1280,8 @@ out:if (ret)exfat_err ("write failed(errno : %d)\n" , errno);else { -exfat_info ("done\n" ); +if (!quiet) +exfat_info ("done\n" );exfat_debug ("zero out written size : %llu\n" , target_zerolen); }
@@ -1270,53 +1366,17 @@ alloc_err: }static void exfat_discard_dev (struct exfat_blk_dev *bd, -struct exfat_user_input *ui) +struct exfat_user_input *ui,const bool quiet) { -uint64_t offset =0 ; -uint64_t tmp_step; -int err; -/* Discard the device 2G at a time */ -const uint64_t step =2ULL <<30 ; -const uint64_t count = bd->num_sectors * bd->sector_size; -if (!ui->discard || !bd->isblk) {exfat_debug ("no-discard requested or the device is a file\n" );return ; } -/* -* The block discarding happens in smaller batches so it can be -* interrupted prematurely -*/ -while (offset < count) { - tmp_step = count - offset; -if (step < tmp_step) - tmp_step = step; - - err =exfat_discard_blocks (bd->dev_fd, offset, tmp_step); -/* -* We intentionally ignore errors from the discard ioctl. It is -* not necessary for the mkfs functionality but just an -* optimization. However we should stop on error. -*/ -if (err ==0 ) { -if (offset ==0 ) { -exfat_info ("Discarding blocks: " ); -exfat_debug ("BLKDISCARD: " ); - } -exfat_debug ("%" PRIu64"-%" PRIu64" " , offset, offset + tmp_step); -fflush (stdout); - }else { -exfat_debug ("BLKDISCARD: %s\n" ,strerror (err)); -if (offset >0 ) -exfat_info ("\n" ); -return ; - } - - offset += tmp_step; - } -if (offset >0 ) -exfat_info ("done\n" ); +/* Discard the device 2G at a time */ +iterate_over_disk (bd->dev_fd, bd->size,2ULL <<30 , + exfat_discard_blocks, quiet ?NULL :"Discarding" , +"BLKDISCARD" ); }static int make_exfat (struct exfat_blk_dev *bd,struct exfat_user_input *ui)
@@ -1560,7 +1620,10 @@ int main(int argc, char *argv[]) quiet = true;break ;case 'v' : - print_level =EXFAT_DEBUG ; +if (print_level <EXFAT_DEBUG ) + print_level =EXFAT_DEBUG ; +else + print_level++;break ;case '?' :case 'h' :
@@ -1652,13 +1715,13 @@ int main(int argc, char *argv[])if (ret) goto out; -exfat_discard_dev (&bd, &ui); +exfat_discard_dev (&bd, &ui, quiet);/* * Zeroing out still needs to be conducted as per JESD84-B51 6.6.9: * "content of an explicitly erased memory range shall be ‘0’ or ‘1’ * depending on different memory technology," */ - ret =exfat_zero_out_disk (&bd, &ui); + ret =exfat_zero_out_disk (&bd, &ui, quiet);if (ret) goto out;
mkfs/mkfs.h +1 -0
@@ -10,6 +10,7 @@#define MIN_NUM_SECTOR (2048)#define EXFAT_MAX_CLUSTER_SIZE (32*1024*1024) +/* MUST be multiple of 512 */ #define EXFAT_HEAD_ZERO_OUT (0x10000)#define EXFAT_GPT_ENTRY_SIZE (128UL)
Signature · 1 line
2.55.0
PATCH 8/8 mkfs: issue POSIX_FADV_NOREUSE before zeroing out
In reply to David TimberWhen zeroing out the disk in "full format mode", if mkfs falls back to using the traditional method of exfat_write_zero2(), the huge amount of page caches spawned as a result of the write could thrash the system.
To alleviate the cache pollution problem, issue POSIX_FADV_NOREUSE on the target device so that FMODE_NOREUSE is set for the new pages.
--- mkfs/mkfs.c | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-)
mkfs/mkfs.c +33 -1
@@ -1192,13 +1192,27 @@ static int exfat_zero_out_disk(struct exfat_blk_dev *bd,const void *zm =NULL ;size_t iosize;unsigned long long target_zerolen; +#ifdef POSIX_FADV_NOREUSE +bool issued_fadvise = false; +#endif assert (ui->cluster_size >0 );if (ui->quick) target_zerolen =MIN (bd->size,EXFAT_HEAD_ZERO_OUT ); -else +else { target_zerolen = bd->size; +#ifdef POSIX_FADV_NOREUSE +/* +* Unfortunately, Linux kernel hadn't supported this fadvise() +* op properly until quite recently. See posix_fadvise(2). +* +* - relevant code in kernel: vma_has_recency() +* - currently, the offset and length are ignored by the kernel +*/ + issued_fadvise =posix_fadvise (bd->dev_fd,0 ,0 ,POSIX_FADV_NOREUSE ) ==0 ; +#endif + }if (!quiet) {exfat_info ("Zeroing out %llu bytes: " , target_zerolen);
@@ -1285,6 +1299,24 @@ out:exfat_debug ("zero out written size : %llu\n" , target_zerolen); } +#ifdef POSIX_FADV_NOREUSE +/* +* Restore the default fmode so that the volume structures(header, +* bitmap, upcase table ...) are cached as usual. The fs implementation +* would appreciate it when the volume is mounted immediately +* afterwards. +* +* The #ifdef checks against POSIX_FADV_NOREUSE, but POSIX_FADV_NORMAL +* is used here. This is intentional - it wouldn't make sense for the +* platform to support POSIX_FADV_NOREUSE but not POSIX_FADV_NORMAL. +* ie, this shouldn't happen: +* +* defined(POSIX_FADV_NOREUSE) && !defined(POSIX_FADV_NORMAL) +*/ +if (issued_fadvise) +posix_fadvise (bd->dev_fd,0 ,0 ,POSIX_FADV_NORMAL ); +#endif +return ret; }
Signature · 1 line
2.55.0
Namjae Jeon
maintainerQuoted 56 lines · David Timber
Following is the performance impact profiling. I don't have a spare NVME SSD to test with so I used AWS EC2 :(
Your contributions welcome.
I'm sure pgsteal_* in
/proc/vmstatcan be used as yet another metric on a busy host, but I didn't bother to go that far to collect that data.UAS:
- Host: Raspberry Pi 4 Model B Rev 1.5
- Device: Samsung BAR+ 64GB (via USB 3.0)
- Target partition size: 1998585856 bytesPre-patch:
System time (seconds): 4.54
Percent of CPU this job got: 7%
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:05.14 Maximum resident set size (kbytes): 3200 Voluntary context switches: 10297 Involuntary context switches: 978
File system inputs: 2168
File system outputs: 3903664Post-patch:
System time (seconds): 0.29
Percent of CPU this job got: 0%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:56.49 Maximum resident set size (kbytes): 3200 Voluntary context switches: 5792 Involuntary context switches: 1322
File system inputs: 2224
File system outputs: 176NVME:
- Host: AWS EC2 c7gd.medium
- Device: "locally attached" SSD volume
- Size: 59000000000 bytesPre-patch:
System time (seconds): 22.64
Percent of CPU this job got: 0%
Elapsed (wall clock) time (h:mm:ss or m:ss): 45:47.70 Maximum resident set size (kbytes): 2792 Voluntary context switches: 227200 Involuntary context switches: 7352
File system inputs: 1360
File system outputs: 115235032Post-patch:
Quoted 8 lines
System time (seconds): 1.14
Percent of CPU this job got: 0%
Elapsed (wall clock) time (h:mm:ss or m:ss): 26:08.06 Maximum resident set size (kbytes): 2788 Voluntary context switches: 56409 Involuntary context switches: 60
File system inputs: 1727
File system outputs: 656
Looks great to me:)
Applied them to #exfat-next now.
Thanks!