bcachefs!
Subsystems (2)
Short announcement, because I'm in the process of moving - but I wanted to get this out there because the code is up and I think it's reasonably stable right now.
Bcachefs is a posix filesystem that I've been working towards for - well, quite awhile now: it's intended as a competitor/replacement for ext4/xfs/btrfs.
Current features
- multiple devices
- replication
- tiering
- data checksumming and compression (zlib only; also the code doesn't work with tiering yet)
- most of the normal posix fs features (no fallocate or quotas yet)
Planned features:
- snapshots!
- erasure coding
- more
There will be a longer announcement on LKML/linux-fs in the near future (after I'm finished moving) - but I'd like to get it a bit more testing from a wider audience first, if possible.
You need the bcache-dev branch, and the new bcache tools - be warned, this code is _not_ compatible with the upstream bcache on disk format:
$ git clone -b bcache-dev http://evilpiepirate.org/git/linux-bcache.git $ git clone -b dev http://evilpiepirate.org/git/bcache-tools.git
Then do the usual compiling...
# bcacheadm format -C /dev/sda1 # mount /dev/sda1 /mnt
The usual caveats apply - it might eat your data, the on disk format has _not_ been stabilized yet, etc. But it's been reasonably stable for me, and passes all but 2-3 of the supported xfstests.
Try it out and let me know how it goes!
Also, programmers please check out the bcache guide - feedback is appreciated:
http://bcache.evilpiepirate.org/BcacheGuide/
Thanks!
A message that is not in the archive (sent off-list or not yet received): CACaajQtwx45r8GcRmchrQwDts1GH-V8g0x1FwGfDvnfm02bq+Q@mail.gmail.com
Quoted 1 line · Vasiliy Tolstov
Does it support discards?
Yes
Quoted 2 lines
Does this announce means that bcache block device
no longer maintained by developers?
There's no plural, it's just me :)
I've been overly stressed and burned out from a startup gone horribly wrong - I'm going to try to start doing some maintainence of upstream bcache again once I've moved.
I apologize for my absence on the list these past months, I've had (and still have) entirely too much to juggle - hint hint, if anyone wants to jump in and help out.
Quoted 1 line
What about performance tests with compared to plain btrfs/ext4?
I haven't done much comparison benchmarking yet, I'll post them once I have. If anyone else wants to do some benchmarking I'd love to see the results.
(From the testing I have done - untarring the kernel (on a fast device, with the tar file already uncompressed and in kernel so we're purely cpu bound) - bcachefs is equal to ext4 to within the margin of error. On dbench when we're purely cpu bound, bcachefs is roughly 30% off from ext4. I think the majority of that is bcachefs's dirent code being somewhat cpu heavy).
The long term goal is to be at least as fast on ext4/xfs for any given workload, and on typical workloads we ought to be faster - and in particular bcachefs should have better and more predictable latency, due to the way journalling works.
Quoted 1 line
Format command have one device, how provide tiering?
--tier specifies the tier of the devices that come after it, where the smaller index is the faster tier.
Only tiers 0 and 1 are supported for now, that will be increased whenever someone gets around to it.
If /dev/sda is your fast device and /dev/sdb is your slow device, run
# bcacheadm format -C /dev/sda --tier 1 /dev/sdb
bcacheadm format --help gives you the full list of options.
Quoted 64 lines
14 июля 2015 г. 3:58 пользователь "Kent Overstreet" < [email protected]> написал:
Quoted 61 lines
Short announcement, because I'm in the process of moving - but I wanted to get
this out there because the code is up and I think it's reasonably stable right
now.Bcachefs is a posix filesystem that I've been working towards for - well, quite
awhile now: it's intended as a competitor/replacement for ext4/xfs/btrfs.Current features
- multiple devices
- replication
- tiering
- data checksumming and compression (zlib only; also the code doesn't
work with
tiering yet)
- most of the normal posix fs features (no fallocate or quotas yet)Planned features:
- snapshots!
- erasure coding
- moreThere will be a longer announcement on LKML/linux-fs in the near future (after
I'm finished moving) - but I'd like to get it a bit more testing from a wider
audience first, if possible.You need the bcache-dev branch, and the new bcache tools - be warned, this code
is _not_ compatible with the upstream bcache on disk format:$ git clone -b bcache-dev http://evilpiepirate.org/git/linux-bcache.git $ git clone -b dev http://evilpiepirate.org/git/bcache-tools.gitThen do the usual compiling...
# bcacheadm format -C /dev/sda1 # mount /dev/sda1 /mntThe usual caveats apply - it might eat your data, the on disk format has _not_
been stabilized yet, etc. But it's been reasonably stable for me, and passes all
but 2-3 of the supported xfstests.Try it out and let me know how it goes!
Also, programmers please check out the bcache guide - feedback is appreciated:
http://bcache.evilpiepirate.org/BcacheGuide/
Thanks!
--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 8 lines · Kent Overstreet
Quoted 3 lines · Vasiliy Tolstov
Does it support discards?
Format command have one device, how provide tiering?
--tier specifies the tier of the devices that come after it, where the smaller index is the faster tier.
Quoted 8 lines
Only tiers 0 and 1 are supported for now, that will be increased whenever someone gets around to it.
If
/dev/sdais your fast device and/dev/sdbis your slow device, run# bcacheadm format -C /dev/sda --tier 1 /dev/sdbbcacheadm format --help gives you the full list of options.
Ok, I am really confused right now. So, the format utility still allows -B (backing device). Is it just an artifact left over after block caching? Because I could not make it work. Any thread encountering the formatted backing device just hangs inside the kernel space until reboot. Is there something wrong with my setup or backing devices are just an illusion, a figment of kernel's imagination? And, if not, how do they relate to tiers? Could you give a simple example of a usual setup - one small and fast SSD and huge but slow RAID-6? Should I now format both partitions as -C and assign tier 1 to RAID-6?
Quoted 69 lines
Quoted 65 lines
14 июля 2015 г. 3:58 пользователь "Kent Overstreet" < [email protected]> написал:
Quoted 61 lines
Short announcement, because I'm in the process of moving - but I wanted to get
this out there because the code is up and I think it's reasonably stable right
now.Bcachefs is a posix filesystem that I've been working towards for - well, quite
awhile now: it's intended as a competitor/replacement for ext4/xfs/btrfs.Current features
- multiple devices
- replication
- tiering
- data checksumming and compression (zlib only; also the code doesn't
work with
tiering yet)
- most of the normal posix fs features (no fallocate or quotas yet)Planned features:
- snapshots!
- erasure coding
- moreThere will be a longer announcement on LKML/linux-fs in the near future (after
I'm finished moving) - but I'd like to get it a bit more testing from a wider
audience first, if possible.You need the bcache-dev branch, and the new bcache tools - be warned, this code
is _not_ compatible with the upstream bcache on disk format:$ git clone -b bcache-dev http://evilpiepirate.org/git/linux-bcache.git $ git clone -b dev http://evilpiepirate.org/git/bcache-tools.gitThen do the usual compiling...
# bcacheadm format -C /dev/sda1 # mount /dev/sda1 /mntThe usual caveats apply - it might eat your data, the on disk format has _not_
been stabilized yet, etc. But it's been reasonably stable for me, and passes all
but 2-3 of the supported xfstests.Try it out and let me know how it goes!
Also, programmers please check out the bcache guide - feedback is appreciated:
http://bcache.evilpiepirate.org/BcacheGuide/
Thanks!
--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 1 line
Denis
A message that is not in the archive (sent off-list or not yet received): CAC7rs0uWSt85F443PRw1zvybccg+EfebaSyH9EhUwHjhTGryRA@mail.gmail.com
A message that is not in the archive (sent off-list or not yet received): CAC7rs0upqkuH1CPd-OAmrpQ=8PmaDpzHYY1MaBDpAL6TS_iKyw@mail.gmail.com
I don't think I found anything in the design description or anywhere else explaining how tiering works and what data, when and why ends up on the next tier. And how to control this. The old bcache has a pretty advanced set of knobs allowing you to fine-tune this behavior (read-ahead limit, sequential cutoff, congestion thresholds, etc.) If I overlooked, please point me to the right direction.
Quoted 134 lines · Kent Overstreet
And yes, format ask drives as -C and use --tier for the slower drives.
-B means an externally managed device - bcache doesn't remap anything to it, it exposes a passthrough block device just so it can snoop on accesses to it and cache them. The functionality still exists in the bcache codebase, but has no meaning in bcachefs land
Quoted 126 lines · On Jul 19, 2015 6:41 PM, [email protected] wrote:
Probably broken because I haven't tested them recently. Once I've unpacked my computers (in a few weeks) I'll debug it
Quoted 121 lines · Denis Bychkov
Quoted 13 lines · Kent Overstreet
Quoted 3 lines · Vasiliy Tolstov
Does it support discards?
Format command have one device, how provide tiering?
--tier specifies the tier of the devices that come after it, where the smaller
index is the faster tier.Only tiers 0 and 1 are supported for now, that will be increased whenever
someone gets around to it.Quoted 5 lines
If
/dev/sdais your fast device and/dev/sdbis your slow device, run# bcacheadm format -C /dev/sda --tier 1 /dev/sdbbcacheadm format --help gives you the full list of options.
Ok, I am really confused right now. So, the format utility still allows -B (backing device). Is it just an artifact left over after block caching? Because I could not make it work. Any thread encountering the formatted backing device just hangs inside the kernel space until reboot. Is there something wrong with my setup or backing devices are just an illusion, a figment of kernel's imagination? And, if not, how do they relate to tiers? Could you give a simple example of a usual setup - one small and fast SSD and huge but slow RAID-6? Should I now format both partitions as -C and assign tier 1 to RAID-6?
Quoted 77 lines
Quoted 77 lines
14 июля 2015 г. 3:58 пользователь "Kent Overstreet" < [email protected]> написал:
Quoted 73 lines
Short announcement, because I'm in the process of moving - but I wanted to
get
this out there because the code is up and I think it's reasonably stable
right
now.Bcachefs is a posix filesystem that I've been working towards for - well,
quite
awhile now: it's intended as a competitor/replacement for ext4/xfs/btrfs.Current features
- multiple devices
- replication
- tiering
- data checksumming and compression (zlib only; also the code
doesn't
work with
tiering yet)
- most of the normal posix fs features (no fallocate or quotas yet)Planned features:
- snapshots!
- erasure coding
- moreThere will be a longer announcement on LKML/linux-fs in the near future
(after
I'm finished moving) - but I'd like to get it a bit more testing from a
wider
audience first, if possible.You need the bcache-dev branch, and the new bcache tools - be warned, this
code
is _not_ compatible with the upstream bcache on disk format:$ git clone -b bcache-dev http://evilpiepirate.org/git/linux-bcache.git $ git clone -b dev http://evilpiepirate.org/git/bcache-tools.gitThen do the usual compiling...
# bcacheadm format -C /dev/sda1 # mount /dev/sda1 /mntThe usual caveats apply - it might eat your data, the on disk format has
_not_
been stabilized yet, etc. But it's been reasonably stable for me, and
passes all
but 2-3 of the supported xfstests.Try it out and let me know how it goes!
Also, programmers please check out the bcache guide - feedback is appreciated:
http://bcache.evilpiepirate.org/BcacheGuide/
Thanks!
--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.htmlQuoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html--
Denis
Signature · 1 line
Denis
Quoted 6 lines · Denis Bychkov
I don't think I found anything in the design description or anywhere else explaining how tiering works and what data, when and why ends up on the next tier. And how to control this. The old bcache has a pretty advanced set of knobs allowing you to fine-tune this behavior (read-ahead limit, sequential cutoff, congestion thresholds, etc.) If I overlooked, please point me to the right direction.
All those additional knobs don't exist yet in bcachefs/tiering land - I want to rethink all of that, and also wait until there's actual users/use cases that need that stuff so we have some idea of what we're trying to accomplish.
The way it works right now is:
- Foreground writes always go to tier 0
If tier 0 is full, they wait - there's code to slowly throttle foreground writes if tier 0 is getting close to full and give tiering/copygc a chance to catch up, so they hopefully don't get stuck waiting nearly forever when tier 0 gets completely full
- Tiering scans the extents btree looking for data that is present on tier 0 but not tier 1, and then writes an additional copy of that data on tier 1
- Extra replicas are considered cached, so the copy on tier 0 will no longer be considered dirty and can be reclaimed
- On the read side, if we read from tier 1 the cache_promote() path tries to write another copy to tier 0
No fancy knobs yet. In the future (a ways off), if we want to readd fancy knobs/behaviour we should try and rethink this stuff in the context of a filesystem - like we could potentially have persistent inode flags for "this file should always live on the slow tier", and also if we want to send particular IOs to the slow tier possibly try and do that from the code that interacts with the pagecache, where we've got more information about how much data we're going to be reading/writing.
Quoted 27 lines · Kent Overstreet
Short announcement, because I'm in the process of moving - but I wanted to get this out there because the code is up and I think it's reasonably stable right now.
Bcachefs is a posix filesystem that I've been working towards for - well, quite awhile now: it's intended as a competitor/replacement for ext4/xfs/btrfs.
Current features
- multiple devices
- replication
- tiering
- data checksumming and compression (zlib only; also the code doesn't work with tiering yet)
- most of the normal posix fs features (no fallocate or quotas yet)Planned features:
- snapshots!
- erasure coding
- moreThere will be a longer announcement on LKML/linux-fs in the near future (after I'm finished moving) - but I'd like to get it a bit more testing from a wider audience first, if possible.
You need the bcache-dev branch, and the new bcache tools - be warned, this code is _not_ compatible with the upstream bcache on disk format:
Quoted 7 lines
$ git clone -b bcache-dev http://evilpiepirate.org/git/linux-bcache.git $ git clone -b dev http://evilpiepirate.org/git/bcache-tools.gitThen do the usual compiling...
# bcacheadm format -C /dev/sda1 # mount /dev/sda1 /mnt
How to mount it?
root@afa03:~# bcacheadm format -C /dev/sdt UUID: 87b0f1e2-e0dc-4453-b0c0-6afca64d402c Set UUID: b5f3bc6a-2aab-4fe1-a8db-e35cc763388c version: 6 nbuckets: 143051 block_size: 1 bucket_size: 4096 nr_in_set: 1 nr_this_dev: 0 first_bucket: 3
root@afa03:~# mount /dev/sdt /mnt/ mount: mount(2) failed: No such file or directory
root@afa03:~# mount -t bcache /dev/sdt /mnt/ mount: mount(2) failed: No such file or directory
A message that is not in the archive (sent off-list or not yet received): CAC7rs0sbg2ci6=niQ0X11AONZbr2AOYhRbxfDH_w4N4A7dyPLw@mail.gmail.com
Quoted 1 line · Kent Overstreet
Can you strace it?
Strange. Now error message changed.
root@afa03:~# mount /dev/sdt /mnt/ mount: mount(2) failed: No such file or directory
strace:
execve("/bin/mount", ["mount", "/dev/sdt", "/mnt/"], [/* 17 vars */]) = 0
brk(0) = 0x17c0000access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe72a554000access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3fstat(3, {st_mode=S_IFREG|0644, st_size=80188, ...}) = 0mmap(NULL, 80188, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fe72a540000
close(3) = 0access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)open("/lib/x86_64-linux-gnu/libmount.so.1", O_RDONLY|O_CLOEXEC) = 3read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\216\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0644, st_size=274504, ...}) = 0mmap(NULL, 2373920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe72a0ef000 mprotect(0x7fe72a130000, 2097152, PROT_NONE) = 0
mmap(0x7fe72a330000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x41000) = 0x7fe72a330000 mmap(0x7fe72a332000, 2336, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe72a332000
close(3) = 0access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\v\2\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1869392, ...}) = 0mmap(NULL, 3972864, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe729d25000 mprotect(0x7fe729ee5000, 2097152, PROT_NONE) = 0
mmap(0x7fe72a0e5000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c0000) = 0x7fe72a0e5000 mmap(0x7fe72a0eb000, 16128, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe72a0eb000
close(3) = 0access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)open("/lib/x86_64-linux-gnu/libblkid.so.1", O_RDONLY|O_CLOEXEC) = 3read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\200\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0644, st_size=258376, ...}) = 0mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe72a53f000 mmap(NULL, 2357800, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe729ae5000 mprotect(0x7fe729b20000, 2097152, PROT_NONE) = 0mmap(0x7fe729d20000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3b000) = 0x7fe729d20000 mmap(0x7fe729d24000, 2600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe729d24000
close(3) = 0access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300[\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0644, st_size=138400, ...}) = 0mmap(NULL, 2242448, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe7298c1000 mprotect(0x7fe7298e2000, 2093056, PROT_NONE) = 0
mmap(0x7fe729ae1000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x20000) = 0x7fe729ae1000 mmap(0x7fe729ae3000, 6032, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe729ae3000
close(3) = 0access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)open("/lib/x86_64-linux-gnu/libuuid.so.1", O_RDONLY|O_CLOEXEC) = 3read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\25\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0644, st_size=19000, ...}) = 0mmap(NULL, 2113920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe7296bc000 mprotect(0x7fe7296c0000, 2093056, PROT_NONE) = 0mmap(0x7fe7298bf000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7fe7298bf000
close(3) = 0access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)open("/lib/x86_64-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\26\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0644, st_size=444344, ...}) = 0mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe72a53e000 mmap(NULL, 2539880, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe72944f000 mprotect(0x7fe7294bb000, 2093056, PROT_NONE) = 0mmap(0x7fe7296ba000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6b000) = 0x7fe7296ba000
close(3) = 0access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\16\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0644, st_size=14592, ...}) = 0mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe72924b000 mprotect(0x7fe72924e000, 2093056, PROT_NONE) = 0mmap(0x7fe72944d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fe72944d000
close(3) = 0access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340`\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=142080, ...}) = 0mmap(NULL, 2217232, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe72902d000 mprotect(0x7fe729045000, 2097152, PROT_NONE) = 0mmap(0x7fe729245000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x18000) = 0x7fe729245000 mmap(0x7fe729247000, 13584, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe729247000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe72a53d000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe72a53c000 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe72a53a000
arch_prctl(ARCH_SET_FS, 0x7fe72a53a840) = 0mprotect(0x7fe72a0e5000, 16384, PROT_READ) = 0mprotect(0x7fe729245000, 4096, PROT_READ) = 0mprotect(0x7fe72944d000, 4096, PROT_READ) = 0mprotect(0x7fe7296ba000, 4096, PROT_READ) = 0mprotect(0x7fe7298bf000, 4096, PROT_READ) = 0mprotect(0x7fe729ae1000, 4096, PROT_READ) = 0mprotect(0x7fe729d20000, 12288, PROT_READ) = 0mprotect(0x7fe72a330000, 4096, PROT_READ) = 0mprotect(0x608000, 4096, PROT_READ) = 0mprotect(0x7fe72a556000, 4096, PROT_READ) = 0
munmap(0x7fe72a540000, 80188) = 0 set_tid_address(0x7fe72a53ab10) = 3012 set_robust_list(0x7fe72a53ab20, 24) = 0
rt_sigaction(SIGRTMIN, {0x7fe729032bb0, [], SA_RESTORER|SA_SIGINFO, 0x7fe72903dd10}, NULL, 8) = 0 rt_sigaction(SIGRT_1, {0x7fe729032c40, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fe72903dd10}, NULL, 8) = 0 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0 statfs("/sys/fs/selinux", 0x7ffe81f62fe0) = -1 ENOENT (No such file or directory)
statfs("/selinux", 0x7ffe81f62fe0) = -1 ENOENT (No such file or directory)
brk(0) = 0x17c0000
brk(0x17e1000) = 0x17e1000
open("/proc/filesystems", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe72a553000
read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tr"..., 1024) = 351
read(3, "", 1024) = 0
close(3) = 0
munmap(0x7fe72a553000, 4096) = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=1852464, ...}) = 0
mmap(NULL, 1852464, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fe72a375000
close(3) = 0
getuid() = 0
geteuid() = 0lstat("/dev", {st_mode=S_IFDIR|0755, st_size=5260, ...}) = 0 lstat("/dev/sdt", {st_mode=S_IFBLK|0660, st_rdev=makedev(65, 48), ...}) = 0 stat("/dev/sdt", {st_mode=S_IFBLK|0660, st_rdev=makedev(65, 48), ...}) = 0 lstat("/mnt", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
access("/dev/sdt", F_OK) = 0
open("/dev/sdt", O_RDONLY|O_CLOEXEC) = 3
fadvise64(3, 0, 0, POSIX_FADV_RANDOM) = 0fstat(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(65, 48), ...}) = 0uname({sys="Linux", node="msl-lab-afa03", ...}) = 0ioctl(3, BLKGETSIZE64, 300000000000) = 0open("/sys/dev/block/65:48", O_RDONLY|O_CLOEXEC) = 4openat(4, "dm/uuid", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
close(4) = 0open("/sys/dev/block/65:48", O_RDONLY|O_CLOEXEC) = 4newfstatat(4, "partition", 0x7ffe81f61a30, 0) = -1 ENOENT (No such file or directory) openat(4, "dm/uuid", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
close(4) = 0ioctl(3, CDROM_GET_CAPABILITY or SNDRV_SEQ_IOCTL_UNSUBSCRIBE_PORT, 0) = -1 EINVAL (Invalid argument)lseek(3, 299999887360, SEEK_SET) = 299999887360read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 64) = 64lseek(3, 299999989760, SEEK_SET) = 299999989760read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 256) = 256lseek(3, 0, SEEK_SET) = 0read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 256) = 256lseek(3, 4096, SEEK_SET) = 4096read(3, "\341\362\332\304P'\235\256\10\0\0\0\0\0\0\0\6\0\0\0\0\0\0\0\306\205s\366N\32E\312"..., 256) = 256lseek(3, 299999999488, SEEK_SET) = 299999999488read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512lseek(3, 299999868416, SEEK_SET) = 299999868416read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512lseek(3, 299999998976, SEEK_SET) = 299999998976read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 48) = 48lseek(3, 299999967744, SEEK_SET) = 299999967744read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999869440, SEEK_SET) = 299999869440read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999868928, SEEK_SET) = 299999868928read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999991808, SEEK_SET) = 299999991808read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999795712, SEEK_SET) = 299999795712read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999697408, SEEK_SET) = 299999697408read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999654400, SEEK_SET) = 299999654400read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999623680, SEEK_SET) = 299999623680read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999533568, SEEK_SET) = 299999533568read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999501312, SEEK_SET) = 299999501312read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999492608, SEEK_SET) = 299999492608read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299999513088, SEEK_SET) = 299999513088read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24lseek(3, 299998419456, SEEK_SET) = 299998419456read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 24) = 24
lseek(3, 299999994368, SEEK_SET) = 299999994368 read(3, "\0\0\0\0", 4) = 4 lseek(3, 4096, SEEK_SET) = 4096
read(3, "\341\362\332\304P'\235\256\10\0\0\0\0\0\0\0\6\0\0\0\0\0\0\0\306\205s\366N\32E\312"..., 1024) = 1024lseek(3, 4096, SEEK_SET) = 4096read(3, "\341\362\332\304P'\235\256\10\0\0\0\0\0\0\0\6\0\0\0\0\0\0\0\306\205s\366N\32E\312"..., 2256) = 2256lseek(3, 299999995904, SEEK_SET) = 299999995904read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096lseek(3, 0, SEEK_SET) = 0read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024lseek(3, 1024, SEEK_SET) = 1024read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024lseek(3, 1048576, SEEK_SET) = 1048576read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024lseek(3, 3072, SEEK_SET) = 3072read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024lseek(3, 7168, SEEK_SET) = 7168read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024lseek(3, 15360, SEEK_SET) = 15360read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024lseek(3, 31744, SEEK_SET) = 31744read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024lseek(3, 64512, SEEK_SET) = 64512read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024lseek(3, 0, SEEK_SET) = 0mmap(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe728fec000 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 262144) = 262144lseek(3, 393216, SEEK_SET) = 393216read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 397312, SEEK_SET) = 397312read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 401408, SEEK_SET) = 401408read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 405504, SEEK_SET) = 405504read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 409600, SEEK_SET) = 409600read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 413696, SEEK_SET) = 413696read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 417792, SEEK_SET) = 417792read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 421888, SEEK_SET) = 421888read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 425984, SEEK_SET) = 425984read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 430080, SEEK_SET) = 430080read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 434176, SEEK_SET) = 434176read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 438272, SEEK_SET) = 438272read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 442368, SEEK_SET) = 442368read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 446464, SEEK_SET) = 446464read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 450560, SEEK_SET) = 450560read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 454656, SEEK_SET) = 454656read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 458752, SEEK_SET) = 458752read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 462848, SEEK_SET) = 462848read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 466944, SEEK_SET) = 466944read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 471040, SEEK_SET) = 471040read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 475136, SEEK_SET) = 475136read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 479232, SEEK_SET) = 479232read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 483328, SEEK_SET) = 483328read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 487424, SEEK_SET) = 487424read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 491520, SEEK_SET) = 491520read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 495616, SEEK_SET) = 495616read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 499712, SEEK_SET) = 499712read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 503808, SEEK_SET) = 503808read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 507904, SEEK_SET) = 507904read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 512000, SEEK_SET) = 512000read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 516096, SEEK_SET) = 516096read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 520192, SEEK_SET) = 520192read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 41) = 41lseek(3, 262144, SEEK_SET) = 262144read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1377) = 1377lseek(3, 2097152, SEEK_SET) = 2097152read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024
ioctl(3, BLKSSZGET, 512) = 0
close(3) = 0
munmap(0x7fe728fec000, 266240) = 0
stat("/sbin/mount.bcache", 0x7ffe81f61d60) = -1 ENOENT (No such file or directory)
stat("/sbin/fs.d/mount.bcache", 0x7ffe81f61d60) = -1 ENOENT (No such file or directory)
stat("/sbin/fs/mount.bcache", 0x7ffe81f61d60) = -1 ENOENT (No such file or directory)
getuid() = 0
geteuid() = 0
getgid() = 0
getegid() = 0
prctl(PR_GET_DUMPABLE) = 1
getuid() = 0
geteuid() = 0
getgid() = 0
getegid() = 0
prctl(PR_GET_DUMPABLE) = 1stat("/run", {st_mode=S_IFDIR|0755, st_size=760, ...}) = 0lstat("/etc/mtab", {st_mode=S_IFLNK|0777, st_size=12, ...}) = 0 lstat("/run/mount/utab", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 open("/run/mount/utab", O_RDWR|O_CREAT|O_CLOEXEC, 0644) = 3
close(3) = 0mount("/dev/sdt", "/mnt", "bcache", MS_MGC_VAL, NULL) = -1 ENOENT (No such file or directory) lstat("/mnt", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0stat("/mnt", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0stat("/dev/sdt", {st_mode=S_IFBLK|0660, st_rdev=makedev(65, 48), ...}) = 0 open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3fstat(3, {st_mode=S_IFREG|0644, st_size=2570, ...}) = 0mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe72a553000 read(3, "# Locale name alias data base.\n#"..., 4096) = 2570
read(3, "", 4096) = 0 close(3) = 0 munmap(0x7fe72a553000, 4096) = 0
open("/usr/share/locale/en_US/LC_MESSAGES/util-linux.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en/LC_MESSAGES/util-linux.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale-langpack/en_US/LC_MESSAGES/util-linux.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale-langpack/en/LC_MESSAGES/util-linux.mo", O_RDONLY) = -1 ENOENT (No such file or directory)write(2, "mount: ", 7mount: ) = 7write(2, "mount(2) failed", 15mount(2) failed) = 15write(2, ": ", 2: ) = 2open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale-langpack/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) write(2, "No such file or directory\n", 26No such file or directory
) = 26
close(1) = 0 close(2) = 0 exit_group(32) = ?
+++ exited with 32 +++
Quoted 4 lines · Ming Lin
Quoted 1 line · Kent Overstreet
Can you strace it?
Strange. Now error message changed.
I mean sometimes it showed:
mount: /dev/sdt already mounted or /mnt/ busy
Quoted 3 lines
root@afa03:~# mount /dev/sdt /mnt/ mount: mount(2) failed: No such file or directory
Quoted 9 lines · Ming Lin
Quoted 4 lines · Ming Lin
Quoted 1 line · Kent Overstreet
Can you strace it?
Strange. Now error message changed.
I mean sometimes it showed:
mount:
/dev/sdtalready mounted or/mnt/busy
I have no idea what's going on, it works for me - is there anything unusual about your setup? what kind of block device is /dev/sdt? is there any chance there's another process that has it open? maybe try rebooting?
Quoted 7 lines · Kent Overstreet
Quoted 6 lines · Ming Lin
Quoted 4 lines · Ming Lin
Quoted 1 line · Kent Overstreet
Can you strace it?
Strange. Now error message changed.
Quoted 7 lines
Quoted 3 lines
I mean sometimes it showed:
mount:
/dev/sdtalready mounted or/mnt/busyI have no idea what's going on, it works for me - is there anything unusual about your setup? what kind of block device is
/dev/sdt? is there any chance there's another process that has it open? maybe try rebooting?
It's a regular HDD. I tried rebooting several times.
Now I try in qemu-kvm. Only the first time it can be mounted.
On host: qemu-img create hdd1.img 20G
On guest: it's /dev/vda
root@block:~# bcacheadm format -C /dev/vda UUID: 4730ed95-4c57-42db-856c-dbce36085625 Set UUID: e69ef0e0-0344-40d7-a6b1-c23d14745a32 version: 6 nbuckets: 40960 block_size: 1 bucket_size: 1024 nr_in_set: 1 nr_this_dev: 0 first_bucket: 3
root@block:~# mount -t bcache /dev/vda /mnt/
root@block:~# mount |grep bcache /dev/vda on /mnt type bcache (rw,relatime)
root@block:~# reboot
root@block:~# dmesg |grep -i bcache [ 2.548754] bcache: bch_journal_replay() journal replay done, 1 keys in 1 entries, seq 3 [ 2.636217] bcache: register_cache() registered cache device vda
root@block:~# mount -t bcache /dev/vda /mnt/ mount: No such file or directory
Now dmesg shows:
bcache: bch_open_as_blockdevs() register_cache_set err device already registered
Quoted 47 lines · Ming Lin
Quoted 7 lines · Kent Overstreet
Quoted 6 lines · Ming Lin
Quoted 4 lines · Ming Lin
Quoted 1 line · Kent Overstreet
Can you strace it?
Strange. Now error message changed.
Quoted 7 lines
Quoted 3 lines
I mean sometimes it showed:
mount:
/dev/sdtalready mounted or/mnt/busyI have no idea what's going on, it works for me - is there anything unusual about your setup? what kind of block device is
/dev/sdt? is there any chance there's another process that has it open? maybe try rebooting?It's a regular HDD. I tried rebooting several times.
Now I try in qemu-kvm. Only the first time it can be mounted.
On host: qemu-img create hdd1.img 20G
On guest: it's/dev/vdaroot@block:~# bcacheadm format -C /dev/vda UUID: 4730ed95-4c57-42db-856c-dbce36085625 Set UUID: e69ef0e0-0344-40d7-a6b1-c23d14745a32 version: 6 nbuckets: 40960 block_size: 1 bucket_size: 1024 nr_in_set: 1 nr_this_dev: 0 first_bucket: 3root@block:~# mount -t bcache /dev/vda /mnt/root@block:~# mount |grep bcache /dev/vda on /mnt type bcache (rw,relatime)root@block:~# rebootroot@block:~# dmesg |grep -i bcache [ 2.548754] bcache: bch_journal_replay() journal replay done, 1 keys in 1 entries, seq 3 [ 2.636217] bcache: register_cache() registered cache device vda
Quoted 6 lines
root@block:~# mount -t bcache /dev/vda /mnt/ mount: No such file or directoryNow dmesg shows:
bcache:
bch_open_as_blockdevs()register_cache_seterr device already registered
Ohhhh.
The cache set is getting registered by the udev hooks. We should be able to mount it anyways - same as you can mount any other fs in multiple locations.
I won't be able to fix this for at least a couple days, but for now - just shut it down it via sysfs (echo 1 > /sys/fs/bcache/<uuid>/stop), then mount it.
Quoted 55 lines · Kent Overstreet
Quoted 47 lines · Ming Lin
Quoted 7 lines · Kent Overstreet
Quoted 6 lines · Ming Lin
Quoted 4 lines · Ming Lin
Quoted 1 line · Kent Overstreet
Can you strace it?
Strange. Now error message changed.
Quoted 7 lines
Quoted 3 lines
I mean sometimes it showed:
mount:
/dev/sdtalready mounted or/mnt/busyI have no idea what's going on, it works for me - is there anything unusual about your setup? what kind of block device is
/dev/sdt? is there any chance there's another process that has it open? maybe try rebooting?It's a regular HDD. I tried rebooting several times.
Now I try in qemu-kvm. Only the first time it can be mounted.
On host: qemu-img create hdd1.img 20G
On guest: it's/dev/vdaroot@block:~# bcacheadm format -C /dev/vda UUID: 4730ed95-4c57-42db-856c-dbce36085625 Set UUID: e69ef0e0-0344-40d7-a6b1-c23d14745a32 version: 6 nbuckets: 40960 block_size: 1 bucket_size: 1024 nr_in_set: 1 nr_this_dev: 0 first_bucket: 3root@block:~# mount -t bcache /dev/vda /mnt/root@block:~# mount |grep bcache /dev/vda on /mnt type bcache (rw,relatime)root@block:~# rebootroot@block:~# dmesg |grep -i bcache [ 2.548754] bcache: bch_journal_replay() journal replay done, 1 keys in 1 entries, seq 3 [ 2.636217] bcache: register_cache() registered cache device vdaQuoted 6 lines
root@block:~# mount -t bcache /dev/vda /mnt/ mount: No such file or directoryNow dmesg shows:
bcache:
bch_open_as_blockdevs()register_cache_seterr device already registered
Quoted 7 lines
Ohhhh.
The cache set is getting registered by the udev hooks. We should be able to mount it anyways - same as you can mount any other fs in multiple locations.
I won't be able to fix this for at least a couple days, but for now - just shut it down it via sysfs (echo 1 > /sys/fs/bcache/<uuid>/stop), then mount it.
It works!
Any hint how to fix it? On udev or bcache-tool or kernel? I'd like to fix it.
root@block:~# echo 1 > /sys/fs/bcache/e69ef0e0-0344-40d7-a6b1-c23d14745a32/stop root@block:~# mount -t bcache /dev/vda /mnt/ /dev/vda on /mnt type bcache (rw,relatime) root@block:~# mount |grep bcache /dev/vda on /mnt type bcache (rw,relatime)
Quoted 62 lines · Ming Lin
Quoted 60 lines · Kent Overstreet
Quoted 47 lines · Ming Lin
Quoted 7 lines · Kent Overstreet
Quoted 6 lines · Ming Lin
Quoted 4 lines · Ming Lin
Quoted 1 line · Kent Overstreet
Can you strace it?
Strange. Now error message changed.
Quoted 7 lines
Quoted 3 lines
I mean sometimes it showed:
mount:
/dev/sdtalready mounted or/mnt/busyI have no idea what's going on, it works for me - is there anything unusual about your setup? what kind of block device is
/dev/sdt? is there any chance there's another process that has it open? maybe try rebooting?It's a regular HDD. I tried rebooting several times.
Now I try in qemu-kvm. Only the first time it can be mounted.
On host: qemu-img create hdd1.img 20G
On guest: it's/dev/vdaroot@block:~# bcacheadm format -C /dev/vda UUID: 4730ed95-4c57-42db-856c-dbce36085625 Set UUID: e69ef0e0-0344-40d7-a6b1-c23d14745a32 version: 6 nbuckets: 40960 block_size: 1 bucket_size: 1024 nr_in_set: 1 nr_this_dev: 0 first_bucket: 3root@block:~# mount -t bcache /dev/vda /mnt/root@block:~# mount |grep bcache /dev/vda on /mnt type bcache (rw,relatime)root@block:~# rebootroot@block:~# dmesg |grep -i bcache [ 2.548754] bcache: bch_journal_replay() journal replay done, 1 keys in 1 entries, seq 3 [ 2.636217] bcache: register_cache() registered cache device vdaQuoted 6 lines
root@block:~# mount -t bcache /dev/vda /mnt/ mount: No such file or directoryNow dmesg shows:
bcache:
bch_open_as_blockdevs()register_cache_seterr device already registeredOhhhh.
The cache set is getting registered by the udev hooks. We should be able to mount it anyways - same as you can mount any other fs in multiple locations.
Quoted 6 lines
Quoted 2 lines
I won't be able to fix this for at least a couple days, but for now - just shut it down it via sysfs (echo 1 > /sys/fs/bcache/<uuid>/stop), then mount it.
It works!
Any hint how to fix it? On udev or bcache-tool or kernel? I'd like to fix it.
The relevant code is in drivers/md/bcache/fs.c, bch_mount() -> bch_open_as_blockdevs().
Part of the problem is that bcachefs isn't able to use much of the normal generic mount path for block devices, partly because a fs can span multiple block devices (same as btrfs).
I'm not sure the right way to fix it - it's going to take some thought, but we want to do something like "is it already open? just take a ref on the existing cache set".
Quoted 73 lines · Kent Overstreet
Quoted 62 lines · Ming Lin
Quoted 60 lines · Kent Overstreet
Quoted 47 lines · Ming Lin
Quoted 7 lines · Kent Overstreet
Quoted 6 lines · Ming Lin
Quoted 4 lines · Ming Lin
Quoted 1 line · Kent Overstreet
Can you strace it?
Strange. Now error message changed.
Quoted 7 lines
Quoted 3 lines
I mean sometimes it showed:
mount:
/dev/sdtalready mounted or/mnt/busyI have no idea what's going on, it works for me - is there anything unusual about your setup? what kind of block device is
/dev/sdt? is there any chance there's another process that has it open? maybe try rebooting?It's a regular HDD. I tried rebooting several times.
Now I try in qemu-kvm. Only the first time it can be mounted.
On host: qemu-img create hdd1.img 20G
On guest: it's/dev/vdaroot@block:~# bcacheadm format -C /dev/vda UUID: 4730ed95-4c57-42db-856c-dbce36085625 Set UUID: e69ef0e0-0344-40d7-a6b1-c23d14745a32 version: 6 nbuckets: 40960 block_size: 1 bucket_size: 1024 nr_in_set: 1 nr_this_dev: 0 first_bucket: 3root@block:~# mount -t bcache /dev/vda /mnt/root@block:~# mount |grep bcache /dev/vda on /mnt type bcache (rw,relatime)root@block:~# rebootroot@block:~# dmesg |grep -i bcache [ 2.548754] bcache: bch_journal_replay() journal replay done, 1 keys in 1 entries, seq 3 [ 2.636217] bcache: register_cache() registered cache device vdaQuoted 6 lines
root@block:~# mount -t bcache /dev/vda /mnt/ mount: No such file or directoryNow dmesg shows:
bcache:
bch_open_as_blockdevs()register_cache_seterr device already registeredOhhhh.
The cache set is getting registered by the udev hooks. We should be able to mount it anyways - same as you can mount any other fs in multiple locations.
Quoted 6 lines
Quoted 2 lines
I won't be able to fix this for at least a couple days, but for now - just shut it down it via sysfs (echo 1 > /sys/fs/bcache/<uuid>/stop), then mount it.
It works!
Any hint how to fix it? On udev or bcache-tool or kernel? I'd like to fix it.The relevant code is in
drivers/md/bcache/fs.c,bch_mount()->bch_open_as_blockdevs().
Quoted 7 lines
Part of the problem is that bcachefs isn't able to use much of the normal generic mount path for block devices, partly because a fs can span multiple block devices (same as btrfs).
I'm not sure the right way to fix it - it's going to take some thought, but we want to do something like "is it already open? just take a ref on the existing cache set".
I'll look into it.
And, echo 1 > /sys/fs/bcache/<uuid>/stop, got below. I'll also try to fix it.
[ 25.826280] ======================================================
[ 25.828038] [ INFO: possible circular locking dependency detected ]
[ 25.828587] 4.1.0-00943-g3683e624 #7 Not tainted
[ 25.828587] -------------------------------------------------------
[ 25.828587] kworker/2:1/660 is trying to acquire lock:
[ 25.828587] (s_active#31){++++.+}, at: [<ffffffff811bc5ee>] kernfs_remove+0x24/0x33
[ 25.828587]
[ 25.828587] but task is already holding lock:
[ 25.828587] (&bch_register_lock){+.+.+.}, at: [<ffffffff815c40e1>] cache_set_flush+0x46/0xa6
[ 25.828587]
[ 25.828587] which lock already depends on the new lock.
[ 25.828587]
[ 25.828587]
[ 25.828587] the existing dependency chain (in reverse order) is:
[ 25.828587]
-> #1 (&bch_register_lock){+.+.+.}:
[ 25.828587] [<ffffffff8108e179>] __lock_acquire+0x73f/0xb0f
[ 25.828587] [<ffffffff8108ecfa>] lock_acquire+0x149/0x25c
[ 25.828587] [<ffffffff816ff284>] mutex_lock_nested+0x6e/0x38f
[ 25.828587] [<ffffffff815c9d01>] bch_cache_set_store+0x2f/0x9e
[ 25.828587] [<ffffffff811bd1ca>] kernfs_fop_write+0x100/0x14a
[ 25.828587] [<ffffffff81154aa5>] __vfs_write+0x26/0xbe
[ 25.828587] [<ffffffff8115511b>] vfs_write+0xbe/0x166
[ 25.828587] [<ffffffff811558b7>] SyS_write+0x51/0x92
[ 25.828587] [<ffffffff81703817>] system_call_fastpath+0x12/0x6f
[ 25.828587]
-> #0 (s_active#31){++++.+}:
[ 25.828587] [<ffffffff8108af24>] validate_chain.isra.31+0x942/0xfc3
[ 25.828587] [<ffffffff8108e179>] __lock_acquire+0x73f/0xb0f
[ 25.828587] [<ffffffff8108ecfa>] lock_acquire+0x149/0x25c
[ 25.828587] [<ffffffff811bba0d>] __kernfs_remove+0x1d1/0x2fd
[ 25.828587] [<ffffffff811bc5ee>] kernfs_remove+0x24/0x33
[ 25.828587] [<ffffffff81402c76>] kobject_del+0x18/0x42
[ 25.828587] [<ffffffff815c40fc>] cache_set_flush+0x61/0xa6
[ 25.828587] [<ffffffff8105ca00>] process_one_work+0x2cc/0x6c4
[ 25.828587] [<ffffffff8105dd21>] worker_thread+0x27a/0x374
[ 25.828587] [<ffffffff81062798>] kthread+0xfb/0x103
[ 25.828587] [<ffffffff81703c02>] ret_from_fork+0x42/0x70
[ 25.828587]
[ 25.828587] other info that might help us debug this:
[ 25.828587]
[ 25.828587] Possible unsafe locking scenario:
[ 25.828587]
[ 25.828587] CPU0 CPU1
[ 25.828587] ---- ----
[ 25.828587] lock(&bch_register_lock);
[ 25.828587] lock(s_active#31);
[ 25.828587] lock(&bch_register_lock);
[ 25.828587] lock(s_active#31);
[ 25.828587]
[ 25.828587] *** DEADLOCK ***
[ 25.828587]
[ 25.828587] 3 locks held by kworker/2:1/660:
[ 25.828587] #0: ("events"){.+.+.+}, at: [<ffffffff8105c8d2>] process_one_work+0x19e/0x6c4
[ 25.828587] #1: ((&cl->work)#3){+.+.+.}, at: [<ffffffff8105c8d2>] process_one_work+0x19e/0x6c4
[ 25.828587] #2: (&bch_register_lock){+.+.+.}, at: [<ffffffff815c40e1>] cache_set_flush+0x46/0xa6
[ 25.828587]
[ 25.828587] stack backtrace:
[ 25.828587] CPU: 2 PID: 660 Comm: kworker/2:1 Not tainted 4.1.0-00943-g3683e624 #7
[ 25.828587] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20150306_163512-brownie 04/01/2014
[ 25.828587] Workqueue: events cache_set_flush
[ 25.828587] ffffffff827d7bd0 ffff880235937a78 ffffffff816fba3b 0000000000000002
[ 25.828587] ffffffff827f10c0 ffff880235937ac8 ffffffff81089bb8 ffff880235937b00
[ 25.828587] ffff880235a46a90 ffff880235937ac8 ffff880235a46a90 ffff880235a47390
[ 25.828587] Call Trace:
[ 25.828587] [<ffffffff816fba3b>] dump_stack+0x4f/0x7b
[ 25.828587] [<ffffffff81089bb8>] print_circular_bug+0x2b1/0x2c2
[ 25.828587] [<ffffffff8108af24>] validate_chain.isra.31+0x942/0xfc3
[ 25.828587] [<ffffffff8108e179>] __lock_acquire+0x73f/0xb0f
[ 25.828587] [<ffffffff8108ecfa>] lock_acquire+0x149/0x25c
[ 25.828587] [<ffffffff811bc5ee>] ? kernfs_remove+0x24/0x33
[ 25.828587] [<ffffffff811bba0d>] __kernfs_remove+0x1d1/0x2fd
[ 25.828587] [<ffffffff811bc5ee>] ? kernfs_remove+0x24/0x33
[ 25.828587] [<ffffffff811bc5ee>] kernfs_remove+0x24/0x33
[ 25.828587] [<ffffffff81402c76>] kobject_del+0x18/0x42
[ 25.828587] [<ffffffff815c40fc>] cache_set_flush+0x61/0xa6
[ 25.828587] [<ffffffff8105ca00>] process_one_work+0x2cc/0x6c4
[ 25.828587] [<ffffffff8105dd21>] worker_thread+0x27a/0x374
[ 25.828587] [<ffffffff8105daa7>] ? rescuer_thread+0x2a6/0x2a6
[ 25.828587] [<ffffffff81062798>] kthread+0xfb/0x103
[ 25.828587] [<ffffffff8108be8a>] ? trace_hardirqs_on_caller+0x1bb/0x1da
[ 25.828587] [<ffffffff8106269d>] ? kthread_create_on_node+0x1c0/0x1c0
[ 25.828587] [<ffffffff81703c02>] ret_from_fork+0x42/0x70
[ 25.828587] [<ffffffff8106269d>] ? kthread_create_on_node+0x1c0/0x1c0
[ 25.952174] bcache: cache_set_free() Cache set 80166ca9-ed99-4eb2-aca3-1f518531ca72 unregisteredQuoted 78 lines · Ming Lin
Quoted 77 lines · Kent Overstreet
Quoted 62 lines · Ming Lin
Quoted 60 lines · Kent Overstreet
Quoted 47 lines · Ming Lin
Quoted 7 lines · Kent Overstreet
Quoted 6 lines · Ming Lin
Quoted 4 lines · Ming Lin
Quoted 1 line · Kent Overstreet
Can you strace it?
Strange. Now error message changed.
Quoted 7 lines
Quoted 3 lines
I mean sometimes it showed:
mount:
/dev/sdtalready mounted or/mnt/busyI have no idea what's going on, it works for me - is there anything unusual about your setup? what kind of block device is
/dev/sdt? is there any chance there's another process that has it open? maybe try rebooting?It's a regular HDD. I tried rebooting several times.
Now I try in qemu-kvm. Only the first time it can be mounted.
On host: qemu-img create hdd1.img 20G
On guest: it's/dev/vdaroot@block:~# bcacheadm format -C /dev/vda UUID: 4730ed95-4c57-42db-856c-dbce36085625 Set UUID: e69ef0e0-0344-40d7-a6b1-c23d14745a32 version: 6 nbuckets: 40960 block_size: 1 bucket_size: 1024 nr_in_set: 1 nr_this_dev: 0 first_bucket: 3root@block:~# mount -t bcache /dev/vda /mnt/root@block:~# mount |grep bcache /dev/vda on /mnt type bcache (rw,relatime)root@block:~# rebootroot@block:~# dmesg |grep -i bcache [ 2.548754] bcache: bch_journal_replay() journal replay done, 1 keys in 1 entries, seq 3 [ 2.636217] bcache: register_cache() registered cache device vdaQuoted 6 lines
root@block:~# mount -t bcache /dev/vda /mnt/ mount: No such file or directoryNow dmesg shows:
bcache:
bch_open_as_blockdevs()register_cache_seterr device already registeredOhhhh.
The cache set is getting registered by the udev hooks. We should be able to mount it anyways - same as you can mount any other fs in multiple locations.
Quoted 6 lines
Quoted 2 lines
I won't be able to fix this for at least a couple days, but for now - just shut it down it via sysfs (echo 1 > /sys/fs/bcache/<uuid>/stop), then mount it.
It works!
Any hint how to fix it? On udev or bcache-tool or kernel? I'd like to fix it.The relevant code is in
drivers/md/bcache/fs.c,bch_mount()->bch_open_as_blockdevs().Part of the problem is that bcachefs isn't able to use much of the normal generic mount path for block devices, partly because a fs can span multiple block devices (same as btrfs).
Quoted 5 lines
Quoted 3 lines
I'm not sure the right way to fix it - it's going to take some thought, but we want to do something like "is it already open? just take a ref on the existing cache set".
I'll look into it.
Thanks
Quoted 6 lines
And, echo 1 > /sys/fs/bcache/<uuid>/stop, got below. I'll also try to fix it.
[ 25.826280] ====================================================== [ 25.828038] [ INFO: possible circular locking dependency detected ] [ 25.828587] 4.1.0-00943-g3683e624 #7 Not tainted
I think the correct fix is to change cache_set_flush() to not hold register_lock while it's calling into the sysfs code. Want to do that, and add a comment so it doesn't get screwed up again? Also try and make sure we don't actually need register_lock for what you take out from under it.
BTW - probably the most valuable thing you could help out with is the documentation, in particular the guide:
http://bcache.evilpiepirate.org/BcacheGuide/
Can you read through (at least some of) that, and tell me what's useful and what needs clarifying? And tell me what you'd like to see added to the guide next - I'll try and work on documentation over the next two weeks, since I probably won't be able to do much real coding with my test machines offline.
Quoted 8 lines · Kent Overstreet
BTW - probably the most valuable thing you could help out with is the documentation, in particular the guide:
http://bcache.evilpiepirate.org/BcacheGuide/Can you read through (at least some of) that, and tell me what's useful and what needs clarifying? And tell me what you'd like to see added to the guide next - I'll try and work on documentation over the next two weeks, since I probably won't be able to do much real coding with my test machines offline.
Yes, I'll read through that.
Quoted 11 lines · Ming Lin
Quoted 8 lines · Kent Overstreet
BTW - probably the most valuable thing you could help out with is the documentation, in particular the guide:
http://bcache.evilpiepirate.org/BcacheGuide/Can you read through (at least some of) that, and tell me what's useful and what needs clarifying? And tell me what you'd like to see added to the guide next - I'll try and work on documentation over the next two weeks, since I probably won't be able to do much real coding with my test machines offline.
Yes, I'll read through that.
Would you add some example to explain how the extents/inodes/dirents are stored in the btree on disk?
I'm reading the debug code in drivers/md/bcache/debug.c. It seems helpful to learn about the internal btree structure.
Thanks.
Quoted 11 lines · Ming Lin
Quoted 10 lines · Ming Lin
Quoted 8 lines · Kent Overstreet
BTW - probably the most valuable thing you could help out with is the documentation, in particular the guide:
http://bcache.evilpiepirate.org/BcacheGuide/Can you read through (at least some of) that, and tell me what's useful and what needs clarifying? And tell me what you'd like to see added to the guide next - I'll try and work on documentation over the next two weeks, since I probably won't be able to do much real coding with my test machines offline.
Quoted 7 lines
Quoted 1 line
Yes, I'll read through that.
Would you add some example to explain how the extents/inodes/dirents are stored in the btree on disk?
I'm reading the debug code in
drivers/md/bcache/debug.c. It seems helpful to learn about the internal btree structure.
For whom interested:
"cat" these files to dump the btree.
root@bee:/sys/kernel/debug/bcache/07170fb5-b9bd-4f44-a7e3-c657a367a960# ls dirents dirents-formats extents extents-formats inodes inodes-formats xattrs xattrs-formats
Quoted 11 lines · Ming Lin
Quoted 10 lines · Ming Lin
Quoted 8 lines · Kent Overstreet
BTW - probably the most valuable thing you could help out with is the documentation, in particular the guide:
http://bcache.evilpiepirate.org/BcacheGuide/Can you read through (at least some of) that, and tell me what's useful and what needs clarifying? And tell me what you'd like to see added to the guide next - I'll try and work on documentation over the next two weeks, since I probably won't be able to do much real coding with my test machines offline.
Quoted 4 lines
Quoted 1 line
Yes, I'll read through that.
Would you add some example to explain how the extents/inodes/dirents are stored in the btree on disk?
Can you be more specific? Like how inodes/dirents map to keys in the btree, or how it all ends up on disk?
The inodes/dirents code is pretty short, I'd look at inode.c and dirent.c
Quoted 2 lines
I'm reading the debug code in
drivers/md/bcache/debug.c. It seems helpful to learn about the internal btree structure.
Are you interested in more the format of the btree node itself, on disk? Like struct btree_node, struct btree_node_entry, struct bset, and the packing?
I could try and elaborate on that in the guide, give me some specific questions to cover if you've got any
Quoted 12 lines · Kent Overstreet
Quoted 11 lines · Ming Lin
Quoted 10 lines · Ming Lin
Quoted 8 lines · Kent Overstreet
BTW - probably the most valuable thing you could help out with is the documentation, in particular the guide:
http://bcache.evilpiepirate.org/BcacheGuide/Can you read through (at least some of) that, and tell me what's useful and what needs clarifying? And tell me what you'd like to see added to the guide next - I'll try and work on documentation over the next two weeks, since I probably won't be able to do much real coding with my test machines offline.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Yes, I'll read through that.
Would you add some example to explain how the extents/inodes/dirents are stored in the btree on disk?
Can you be more specific? Like how inodes/dirents map to keys in the btree, or how it all ends up on disk?
How it all ends up on disk.
Quoted 11 lines
The inodes/dirents code is pretty short, I'd look at
inode.canddirent.cQuoted 2 lines
I'm reading the debug code in
drivers/md/bcache/debug.c. It seems helpful to learn about the internal btree structure.Are you interested in more the format of the btree node itself, on disk? Like struct
btree_node, structbtree_node_entry, struct bset, and the packing?I could try and elaborate on that in the guide, give me some specific questions to cover if you've got any
Only textbook knowledge of b+tree.
Yes, I'm interested in the format of btree on disk. Some graph like this could help.
https://btrfs.wiki.kernel.org/index.php/Btrfs_design
And I want to learn how the btree node insert/delete/update happens on disk. These maybe too detail. I'm going to write a small tool to dump the file system. Then I could understand better the on disk btree format.
Thanks.
Quoted 5 lines · Ming Lin
And I want to learn how the btree node insert/delete/update happens on disk. These maybe too detail. I'm going to write a small tool to dump the file system. Then I could understand better the on disk btree format.
Here is my simple tool to dump parts of the on-disk format.
http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=deb258e2
It's not in good shape, but simple enough to learn the on-disk format.
Quoted 9 lines · Ming Lin
Quoted 5 lines · Ming Lin
And I want to learn how the btree node insert/delete/update happens on disk. These maybe too detail. I'm going to write a small tool to dump the file system. Then I could understand better the on disk btree format.
Here is my simple tool to dump parts of the on-disk format.
http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=deb258e2
Actually: http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=3121eec
Quoted 2 lines
It's not in good shape, but simple enough to learn the on-disk format.
Quoted 8 lines · Ming Lin
Quoted 7 lines · Ming Lin
Quoted 5 lines · Ming Lin
And I want to learn how the btree node insert/delete/update happens on disk. These maybe too detail. I'm going to write a small tool to dump the file system. Then I could understand better the on disk btree format.
Quoted 7 lines
Quoted 2 lines
Here is my simple tool to dump parts of the on-disk format.
http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=deb258e2Actually: http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=3121eec
Quoted 2 lines
It's not in good shape, but simple enough to learn the on-disk format.
Hi Kent,
I'm trying to understand how the root inode is stored in the inode btree.
dd if=/dev/zero of=fs.img bs=10M count=1
bcacheadm format -C fs.img
mount -t bcache -o loop fs.img /mnt
umount /mnt
hexdump -C fs.img > fs.hex
From my simple tool, I know that the inode btree starts from offset 0xec000
000ec000 43 ef f3 df ff ff ff ff 86 c1 47 1e 99 25 51 35 |C.........G..%Q5| 000ec010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ec020 00 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff |................| 000ec030 ff ff ff ff ff ff ff ff 01 05 00 00 00 00 00 00 |................| 000ec040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ec070 88 b5 38 e2 45 36 eb f6 00 00 00 00 00 00 00 00 |..8.E6..........| 000ec080 01 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 |................| 000ec090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ed000 31 66 fd 31 ff ff ff ff 88 b5 38 e2 45 36 eb f6 |1f.1......8.E6..| 000ed010 02 00 00 00 00 00 00 00 01 00 00 00 03 00 0b 00 |................| 000ed020 0b 01 80 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ed030 00 00 00 00 00 00 00 00 00 10 00 00 00 00 00 00 |................| 000ed040 ed 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |.A..............| 000ed050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ed070 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ed080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
btree_node (0xec000)
bset (0xed008) ---> bset->u64s = 0x0b = 11
bkey_packed (0xed020)
bkey (0xed020)
bch_inode (0xed040 to 0xed077) ---> root inodeIs the decode above correct?
I found the root inode manually. But how is it actually found by code?
Could you help to explain what it is from 0xec070 to 0xed007? Are they also bsets?
Thanks,
Ming
Quoted 22 lines · Ming Lin
Quoted 8 lines · Ming Lin
Quoted 7 lines · Ming Lin
Quoted 5 lines · Ming Lin
And I want to learn how the btree node insert/delete/update happens on disk. These maybe too detail. I'm going to write a small tool to dump the file system. Then I could understand better the on disk btree format.
Quoted 7 lines
Quoted 2 lines
Here is my simple tool to dump parts of the on-disk format.
http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=deb258e2Actually: http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=3121eec
Quoted 2 lines
It's not in good shape, but simple enough to learn the on-disk format.
Hi Kent,
I'm trying to understand how the root inode is stored in the inode btree.
Quoted 8 lines
dd if=/dev/zero of=fs.img bs=10M count=1
bcacheadm format -C fs.img
mount -t bcache -o loop fs.img /mnt
umount /mnt
hexdump -C fs.img > fs.hexFrom my simple tool, I know that the inode btree starts from offset
0xec000
The root node of the inode btree? Are you handling trees with multiple nodes yet?
Quoted 22 lines
000ec000 43 ef f3 df ff ff ff ff 86 c1 47 1e 99 25 51 35 |C.........G..%Q5| 000ec010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ec020 00 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff |................| 000ec030 ff ff ff ff ff ff ff ff 01 05 00 00 00 00 00 00 |................| 000ec040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ec070 88 b5 38 e2 45 36 eb f6 00 00 00 00 00 00 00 00 |..8.E6..........| 000ec080 01 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 |................| 000ec090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ed000 31 66 fd 31 ff ff ff ff 88 b5 38 e2 45 36 eb f6 |1f.1......8.E6..| 000ed010 02 00 00 00 00 00 00 00 01 00 00 00 03 00 0b 00 |................| 000ed020 0b 01 80 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ed030 00 00 00 00 00 00 00 00 00 10 00 00 00 00 00 00 |................| 000ed040 ed 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |.A..............| 000ed050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ed070 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ed080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|*
Quoted 7 lines
btree_node(0xec000)bset (0xed008) ---> bset->u64s = 0x0b = 11 bkey_packed (0xed020) bkey (0xed020) bch_inode (0xed040 to 0xed077) ---> root inodeIs the decode above correct?
I think so. The code that deals with reading in a btree node disk and interpreting the contents is mainly in bch_btree_node_read_done(), btree_io.c - it looks like you found that?
Quoted 1 line
I found the root inode manually. But how is it actually found by code?
The root inode is the inode with inode number BCACHE_ROOT_INO (4096) -
http://evilpiepirate.org/git/linux-bcache.git/tree/drivers/md/bcache/fs.c?h=bcache-dev&id=5cf7fb11d124839eea2191fd7e8eddecb296d67d#n2285
So to do it correctly, you'll need the bkey packing code in order to unpack the key (if it was packed) so that you can get the actual inode number of the key.
You'll also need to do something like the mergesort algorithm (or something equivalent; you don't need to do the actual mergesort if you're just doing a linear search for one key). That is - if there's multiple bsets, they will likely contain duplicates and keys in newer bsets overwrite keys in older bsets.
Quoted 2 lines
Could you help to explain what it is from
0xec070to0xed007? Are they also bsets?
Without knowing your block size and spending a fair amount of time staring at the hexdump, I don't know what starts there - but quite possibly yes; bsets that aren't at the start of the btree node are embeddedd in a struct btree_node_entry, not a struct btree_node.
To tell if it's a valid bset, you compare bset->seq against the seq in the first bset - it's a random number generated for each new btree node; if they match then the bset there goes with that btree node.
Quoted 29 lines · Kent Overstreet
Quoted 28 lines · Ming Lin
Quoted 8 lines · Ming Lin
Quoted 7 lines · Ming Lin
Quoted 5 lines · Ming Lin
And I want to learn how the btree node insert/delete/update happens on disk. These maybe too detail. I'm going to write a small tool to dump the file system. Then I could understand better the on disk btree format.
Quoted 7 lines
Quoted 2 lines
Here is my simple tool to dump parts of the on-disk format.
http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=deb258e2Actually: http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=3121eec
Quoted 2 lines
It's not in good shape, but simple enough to learn the on-disk format.
Hi Kent,
I'm trying to understand how the root inode is stored in the inode btree.
dd if=/dev/zero of=fs.img bs=10M count=1
bcacheadm format -C fs.img
mount -t bcache -o loop fs.img /mnt
umount /mnt
hexdump -C fs.img > fs.hex
Quoted 5 lines
Quoted 2 lines
From my simple tool, I know that the inode btree starts from offset
0xec000The root node of the inode btree? Are you handling trees with multiple nodes yet?
Yes and no.
Quoted 29 lines
Quoted 29 lines
000ec000 43 ef f3 df ff ff ff ff 86 c1 47 1e 99 25 51 35 |C.........G..%Q5| 000ec010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ec020 00 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff |................| 000ec030 ff ff ff ff ff ff ff ff 01 05 00 00 00 00 00 00 |................| 000ec040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ec070 88 b5 38 e2 45 36 eb f6 00 00 00 00 00 00 00 00 |..8.E6..........| 000ec080 01 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 |................| 000ec090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ed000 31 66 fd 31 ff ff ff ff 88 b5 38 e2 45 36 eb f6 |1f.1......8.E6..| 000ed010 02 00 00 00 00 00 00 00 01 00 00 00 03 00 0b 00 |................| 000ed020 0b 01 80 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ed030 00 00 00 00 00 00 00 00 00 10 00 00 00 00 00 00 |................| 000ed040 ed 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |.A..............| 000ed050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ed070 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ed080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|*
btree_node(0xec000)bset (0xed008) ---> bset->u64s = 0x0b = 11 bkey_packed (0xed020) bkey (0xed020) bch_inode (0xed040 to 0xed077) ---> root inode
Quoted 5 lines
Quoted 1 line
Is the decode above correct?
I think so. The code that deals with reading in a btree node disk and interpreting the contents is mainly in
bch_btree_node_read_done(),btree_io.c- it looks like you found that?
I haven't dig into the code yet.
Firstly to understand the on-disk structure by hexdump.
Quoted 6 lines
Quoted 2 lines
I found the root inode manually. But how is it actually found by code?
The root inode is the inode with inode number
BCACHE_ROOT_INO(4096) -
http://evilpiepirate.org/git/linux-bcache.git/tree/drivers/md/bcache/fs.c?h=bcache-dev&id=5cf7fb11d124839eea2191fd7e8eddecb296d67d#n2285
Quoted 7 lines
So to do it correctly, you'll need the bkey packing code in order to unpack the key (if it was packed) so that you can get the actual inode number of the key.
You'll also need to do something like the mergesort algorithm (or something equivalent; you don't need to do the actual mergesort if you're just doing a linear search for one key). That is - if there's multiple bsets, they will likely contain duplicates and keys in newer bsets overwrite keys in older bsets.
Don't understand this part for now. I'll learn it.
Quoted 4 lines
Quoted 3 lines
Could you help to explain what it is from
0xec070to0xed007? Are they also bsets?
Quoted 8 lines
Without knowing your block size and spending a fair amount of time staring at the hexdump, I don't know what starts there - but quite possibly yes; bsets that aren't at the start of the btree node are embeddedd in a struct
btree_node_entry, not a structbtree_node.To tell if it's a valid bset, you compare
bset->seqagainst the seq in the first bset - it's a random number generated for each new btree node; if they match then the bset there goes with that btree node.
The block size is 4K.
OK, now I can interpret the hexdump.
000ec000 43 ef f3 df ff ff ff ff 86 c1 47 1e 99 25 51 35 |C.........G..%Q5| 000ec010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ec020 00 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff |................| 000ec030 ff ff ff ff ff ff ff ff 01 05 00 00 00 00 00 00 |................| 000ec040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ec070 88 b5 38 e2 45 36 eb f6 00 00 00 00 00 00 00 00 |..8.E6..........| 000ec080 01 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 |................| 000ec090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ed000 31 66 fd 31 ff ff ff ff 88 b5 38 e2 45 36 eb f6 |1f.1......8.E6..| 000ed010 02 00 00 00 00 00 00 00 01 00 00 00 03 00 0b 00 |................| 000ed020 0b 01 80 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ed030 00 00 00 00 00 00 00 00 00 10 00 00 00 00 00 00 |................| 000ed040 ed 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |.A..............| 000ed050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000ed070 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000ed080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*000ee000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
There are 2 bsets: bset->seq "88 b5 38 e2 45 36 eb f6"
btree_node (0xec000)
bset_1 (0xec070) ---> bset->u64s = 0 (a empty bset?)
btree_node_entry (0xed000)
bset_2 (0xed008) ---> bset->u64s = 0x0b = 11
bkey_packed (0xed020)
bkey (0xed020)
bch_inode (0xed040 to 0xed077) ---> root inodeWhy is there a empty bset at the start of the btree node?
Quoted 11 lines · Ming Lin
Quoted 5 lines · Ming Lin
And I want to learn how the btree node insert/delete/update happens on disk. These maybe too detail. I'm going to write a small tool to dump the file system. Then I could understand better the on disk btree format.
Here is my simple tool to dump parts of the on-disk format.
http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=deb258e2It's not in good shape, but simple enough to learn the on-disk format.
Hey! Sorry for taking so long to respond, just got my computer set up back in Alaska.
If you want to keep going with your tool, this might be a starting point for a debugfs tool - which bcache definitely needs at some point.
Quoted 11 lines · Kent Overstreet
Quoted 10 lines · Ming Lin
Quoted 5 lines · Ming Lin
And I want to learn how the btree node insert/delete/update happens on disk. These maybe too detail. I'm going to write a small tool to dump the file system. Then I could understand better the on disk btree format.
Here is my simple tool to dump parts of the on-disk format.
http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=deb258e2
Quoted 7 lines
Quoted 1 line
It's not in good shape, but simple enough to learn the on-disk format.
Hey! Sorry for taking so long to respond, just got my computer set up back in Alaska.
If you want to keep going with your tool, this might be a starting point for a debugfs tool - which bcache definitely needs at some point.
Yes, that's my goal.
I'll improve it once I get more familiar with bcachefs on-disk format.
Quoted 15 lines · Ming Lin
Quoted 13 lines · Kent Overstreet
Quoted 11 lines · Ming Lin
Quoted 5 lines · Ming Lin
And I want to learn how the btree node insert/delete/update happens on disk. These maybe too detail. I'm going to write a small tool to dump the file system. Then I could understand better the on disk btree format.
Here is my simple tool to dump parts of the on-disk format.
http://www.minggr.net/cgit/cgit.cgi/bcache-tools/commit/?id=deb258e2It's not in good shape, but simple enough to learn the on-disk format.
Quoted 8 lines
Quoted 5 lines
Hey! Sorry for taking so long to respond, just got my computer set up back in Alaska.
If you want to keep going with your tool, this might be a starting point for a debugfs tool - which bcache definitely needs at some point.
Yes, that's my goal.
I'll improve it once I get more familiar with bcachefs on-disk format.
I imagine the sanest thing to do will be to reuse some of the kernel side code - at the very least, the bkey packing code. That code is already pretty self contained, and it's very algorithmic - no point in redoing it, and no real reason to do it differently.
If it makes things easier, we could probably shuffle code around a bit so that perhaps bkey.c contains only code that can be easily compiled in userspace.
I'm not sure if there's any other significant code that you'd want to use in userspace - possibly the mergesort code (i.e. bch_extent_sort_fix_overlapping()), but that code is going to be harder to lift out and compile in userspace without changes.
Journal replay is going to be another major issue... the problem is, the btree isn't up to date until you do journal replay, and the way bcache does journal replay is with the same index update path that it uses at runtime - which modifies the btree, i.e. it can't do journal replay without modifying what's on disk.
We don't want the userspace debugfs tool to be modifying the disk image, so the method bcache uses is right out.
The method I had in mind was that when you read the journal, you keep that list of index updates to do around, in memory - then, when you read or are looking at any given btree node, you iterate over all the keys in the journal replay list and apply only the ones that apply to the current node. If the insertions don't fit into the current node (i.e. if we would have to split the node if we were doing a normal index update) - just grow the node in memory, since we're just going to be tossing it out when we're done instead of writing out our changes.
Hi,
I actually tried to compile that code recently (maybe 5 days ago) and it did not change since then. There was a bunch of trivial errors, that I was able to fix - files moved around without #include updated, types missing. But at some point I ran into something definitely
non-trivial: there is a variable in io.c, which is supposed to have a node id in it, but the init code commented out and there is no obvious way to fix it. So I gave up. I definitely tried the branch you are talking about - bcache-dev.
Here is the I code I mentioned:
static void bch_read_retry(struct bbio *bbio)
{
struct bio *bio = &bbio->bio;
struct bio *parent;
u64 inode;
trace_bcache_read_retry(bio);
/*
* This used to be a leaf bio from bch_read_fn(), but
* since we don't know what happened to the btree in
* the meantime, we have to re-submit it via the
* top-level bch_read() entry point. Before doing that,
* we have to reset the bio, preserving the biovec. *
* The inode, offset and size come from the bbio's key,
* which was set by bch_read_fn().
*/
//inode = bbio->key.k.p.inode;
parent = bio->bi_private;
bch_bbio_reset(bbio);bio_chain(bio, parent);
bch_read(bbio->ca->set, bio, inode); bio_endio(parent, 0); /* for bio_chain() in bch_read_fn() */ bio_endio(bio, 0);
}
Quoted 50 lines · Kent Overstreet
Short announcement, because I'm in the process of moving - but I wanted to get this out there because the code is up and I think it's reasonably stable right now.
Bcachefs is a posix filesystem that I've been working towards for - well, quite awhile now: it's intended as a competitor/replacement for ext4/xfs/btrfs.
Current features
- multiple devices
- replication
- tiering
- data checksumming and compression (zlib only; also the code doesn't work with tiering yet)
- most of the normal posix fs features (no fallocate or quotas yet)Planned features:
- snapshots!
- erasure coding
- moreThere will be a longer announcement on LKML/linux-fs in the near future (after I'm finished moving) - but I'd like to get it a bit more testing from a wider audience first, if possible.
You need the bcache-dev branch, and the new bcache tools - be warned, this code is _not_ compatible with the upstream bcache on disk format:
$ git clone -b bcache-dev http://evilpiepirate.org/git/linux-bcache.git $ git clone -b dev http://evilpiepirate.org/git/bcache-tools.gitThen do the usual compiling...
# bcacheadm format -C /dev/sda1 # mount /dev/sda1 /mntThe usual caveats apply - it might eat your data, the on disk format has _not_ been stabilized yet, etc. But it's been reasonably stable for me, and passes all but 2-3 of the supported xfstests.
Try it out and let me know how it goes!
Also, programmers please check out the bcache guide - feedback is appreciated:
http://bcache.evilpiepirate.org/BcacheGuide/
Thanks!
--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 1 line
Denis
Quoted 10 lines · Denis Bychkov
Hi,
I actually tried to compile that code recently (maybe 5 days ago) and it did not change since then. There was a bunch of trivial errors, that I was able to fix - files moved around without #include updated, types missing. But at some point I ran into something definitely
non-trivial: there is a variable inio.c, which is supposed to have a node id in it, but the init code commented out and there is no obvious way to fix it. So I gave up. I definitely tried the branch you are talking about - bcache-dev.
It didn't build? Weird - can you post some of the compiler errors you were seeing, as well as your archictecture/gcc version? I don't know of any build errors right now...
Quoted 24 lines
Here is the I code I mentioned:
static void
bch_read_retry(struct bbio *bbio)
{
struct bio *bio =&bbio->bio;
struct bio *parent;
u64 inode;
trace_bcache_read_retry(bio);/*
* This used to be a leaf bio frombch_read_fn(), but
* since we don't know what happened to the btree in
* the meantime, we have to re-submit it via the
* top-levelbch_read()entry point. Before doing that,
* we have to reset the bio, preserving the biovec. *
* The inode, offset and size come from the bbio's key,
* which was set bybch_read_fn().
*/
//inode =bbio->key.k.p.inode;
parent =bio->bi_private;
Quoted 7 lines
bch_bbio_reset(bbio);bio_chain(bio, parent);
bch_read(bbio->ca->set, bio, inode);bio_endio(parent, 0); /* forbio_chain()inbch_read_fn()*/bio_endio(bio, 0);
}
The read retry path is currently non functional, since I added checksumming/compression support - the read path needs a fair bit more work. But that shouldn't cause a build error - and the race the retry path is for is damn near impossible to trigger without fault injection (I don't think I've ever seen it happen without fault injection).
Hi,
I managed to build it tonight. Although, it took some tuning. The problem is with bcache.h header file (the one that resides in include/trace/events/). It includes "alloc_types.h", but there is no alloc_types.h in that directory. I think, you use customized Makefile with extra directories added to -I argument. I fixed it by moving alloc_types and clock_types headers to include/trace/events/ from the bcache module directory. I attached the diff if anyone is interested. To anyone who wants to build this bcache version: you'll need to either rebase the code on the kernel version you are working with or extract a diff from bcache-dev branch against 4.1 vanilla kernel and then apply it to your kernel, then the patch I attached. Or probably Kent will fix it soon enough, so you won't need silly workarounds.
Quoted 53 lines · Kent Overstreet
Quoted 10 lines · Denis Bychkov
Hi,
I actually tried to compile that code recently (maybe 5 days ago) and it did not change since then. There was a bunch of trivial errors, that I was able to fix - files moved around without #include updated, types missing. But at some point I ran into something definitely
non-trivial: there is a variable inio.c, which is supposed to have a node id in it, but the init code commented out and there is no obvious way to fix it. So I gave up. I definitely tried the branch you are talking about - bcache-dev.It didn't build? Weird - can you post some of the compiler errors you were seeing, as well as your archictecture/gcc version? I don't know of any build errors right now...
Quoted 24 lines
Here is the I code I mentioned:
static void
bch_read_retry(struct bbio *bbio)
{
struct bio *bio =&bbio->bio;
struct bio *parent;
u64 inode;
trace_bcache_read_retry(bio);/*
* This used to be a leaf bio frombch_read_fn(), but
* since we don't know what happened to the btree in
* the meantime, we have to re-submit it via the
* top-levelbch_read()entry point. Before doing that,
* we have to reset the bio, preserving the biovec. *
* The inode, offset and size come from the bbio's key,
* which was set bybch_read_fn().
*/
//inode =bbio->key.k.p.inode;
parent =bio->bi_private;Quoted 7 lines
bch_bbio_reset(bbio);bio_chain(bio, parent);
bch_read(bbio->ca->set, bio, inode);bio_endio(parent, 0); /* forbio_chain()inbch_read_fn()*/bio_endio(bio, 0);
}The read retry path is currently non functional, since I added checksumming/compression support - the read path needs a fair bit more work. But that shouldn't cause a build error - and the race the retry path is for is damn near impossible to trigger without fault injection (I don't think I've ever seen it happen without fault injection).
Signature · 1 line
Denis
drivers/md/bcache/alloc_types.h deleted +0 -97
@@ -1,97 +0,0 @@ -#ifndef _BCACHE_ALLOC_TYPES_H -#define _BCACHE_ALLOC_TYPES_H - -#include <linux/mutex.h> - -#include "clock_types.h" - -/* -* There's two of these clocks, one for reads and one for writes: -* -* All fields protected by bucket_lock -*/ -struct prio_clock { -/* -* "now" in (read/write) IO time - incremented whenever we do X amount -* of reads or writes. -* -* Goes with the bucket read/write prios: when we read or write to a -* bucket we reset the bucket's prio to the current hand; thus hand - -* prio = time since bucket was last read/written. -* -* The units are some amount (bytes/sectors) of data read/written, and -* the units can change on the fly if we need to rescale to fit -* everything in a u16 - your only guarantee is that the units are -* consistent. -*/ -u16 hand; -u16 min_prio; - -int rw; - -struct io_timer rescale; -}; - -/* There is one reserve for each type of btree, one for prios and gens -* and one for moving GC */ -enum alloc_reserve { -RESERVE_PRIO , -RESERVE_BTREE , -RESERVE_METADATA_LAST =RESERVE_BTREE , -RESERVE_MOVINGGC , - -RESERVE_NONE , -RESERVE_NR , -}; - -static inline bool allocation_is_metadata (enum alloc_reserve id) -{ -return id <=RESERVE_METADATA_LAST ; -} - -/* Enough for 16 cache devices, 2 tiers and some left over for pipelining */ -#define OPEN_BUCKETS_COUNT 256 - -#define WRITE_POINT_COUNT 16 - -struct open_bucket { -struct list_head list; -struct mutex lock; -atomic_t pin; -unsigned sectors_free; -unsigned nr_ptrs; -struct bch_extent_ptr ptrs[BKEY_EXTENT_PTRS_MAX ]; -}; - -struct write_point { -struct open_bucket *b; - -/* -* Throttle writes to this write point if tier 0 is full? -*/ -bool throttle; - -/* -* If 0, use the desired replica count for the cache set. -* Otherwise, this is the number of replicas desired (generally 1). -*/ -unsigned nr_replicas; - -/* -* Bucket reserve to allocate from. -*/ -enum alloc_reserve reserve; - -/* -* If not NULL, cache group for tiering, promotion and moving GC - -* always allocates a single replica -*/ -struct cache_group *group; - -/* -* Otherwise do a normal replicated bucket allocation that could come -* from any device in tier 0 (foreground write) -*/ -}; - -#endif /* _BCACHE_ALLOC_TYPES_H */
drivers/md/bcache/clock_types.h deleted +0 -32
@@ -1,32 +0,0 @@ -#ifndef _BCACHE_CLOCK_TYPES_H -#define _BCACHE_CLOCK_TYPES_H - -#define NR_IO_TIMERS 8 - -/* -* Clocks/timers in units of sectors of IO: -* -* Note - they use percpu batching, so they're only approximate. -*/ - -struct io_timer ; -typedef void (*io_timer_fn )(struct io_timer *); - -struct io_timer { -io_timer_fn fn; -unsigned long expire; -}; - -/* Amount to buffer up on a percpu counter */ -#define IO_CLOCK_PCPU_SECTORS 128 - -struct io_clock { -atomic_long_t now; -u16 __percpu *pcpu_buf; - -spinlock_t timer_lock; - DECLARE_HEAP(struct io_timer *, timers); -}; - -#endif /* _BCACHE_CLOCK_TYPES_H */ -
drivers/md/bcache/io.c +1 -0
@@ -1649,6 +1649,7 @@* which was set by bch_read_fn(). */ //inode = bbio->key.k.p.inode; + inode =0 ; parent = bio->bi_private;bch_bbio_reset (bbio);
include/trace/events/alloc_types.h new file +97 -0
97 lines not on this page · raw message
include/trace/events/clock_types.h new file +32 -0
@@ -0,0 +1,32 @@ +#ifndef _BCACHE_CLOCK_TYPES_H +#define _BCACHE_CLOCK_TYPES_H + +#define NR_IO_TIMERS 8 + +/* +* Clocks/timers in units of sectors of IO: +* +* Note - they use percpu batching, so they're only approximate. +*/ + +struct io_timer ; +typedef void (*io_timer_fn )(struct io_timer *); + +struct io_timer { +io_timer_fn fn; +unsigned long expire; +}; + +/* Amount to buffer up on a percpu counter */ +#define IO_CLOCK_PCPU_SECTORS 128 + +struct io_clock { +atomic_long_t now; +u16 __percpu *pcpu_buf; + +spinlock_t timer_lock; + DECLARE_HEAP(struct io_timer *, timers); +}; + +#endif /* _BCACHE_CLOCK_TYPES_H */ +
drivers/md/bcache/alloc.h +1 -1
@@ -1,7 +1,7 @@#ifndef _BCACHE_ALLOC_H#define _BCACHE_ALLOC_H -#include "alloc_types.h" +#include <trace/events/alloc_types.h> struct bkey ;struct bucket ;
drivers/md/bcache/bcache.h +2 -2
@@ -207,10 +207,10 @@#define bch_meta_write_fault (name) \ dynamic_fault("bcache:meta:write:" name) -#include "alloc_types.h" +#include <trace/events/alloc_types.h> #include "blockdev_types.h" #include "buckets_types.h" -#include "clock_types.h" +#include <trace/events/clock_types.h> #include "io_types.h" #include "journal_types.h" #include "keylist_types.h"
Quoted 16 lines · Kent Overstreet
Short announcement, because I'm in the process of moving - but I wanted to get this out there because the code is up and I think it's reasonably stable right now.
Bcachefs is a posix filesystem that I've been working towards for - well, quite awhile now: it's intended as a competitor/replacement for ext4/xfs/btrfs.
Current features
- multiple devices
- replication
- tiering
- data checksumming and compression (zlib only; also the code doesn't
work with
tiering yet)
- most of the normal posix fs features (no fallocate or quotas yet)
Quoted 8 lines
Planned features:
- snapshots!
- erasure coding
- moreThere will be a longer announcement on LKML/linux-fs in the near future (after I'm finished moving) - but I'd like to get it a bit more testing from a wider audience first, if possible.
Hi Kent,
one quick question about the roadmap at this point: As far as I understand bcachefs basically integrates bcache features directly in the filesystem. So does this deprecate bcache itself in your opinion? Bcache is obviously still useful for other FS, but I just want to know how things will get maintained in the future.
I wanted to suggest / possibly start implementing bcache support for the debian installer - obviously that only makes sense if I can expect it to be in the mainline kernel for the foreseeable future :-).
Thanks,
~David
Quoted 22 lines · David Mohr
Quoted 21 lines · Kent Overstreet
Short announcement, because I'm in the process of moving - but I wanted to get this out there because the code is up and I think it's reasonably stable right now.
Bcachefs is a posix filesystem that I've been working towards for - well, quite awhile now: it's intended as a competitor/replacement for ext4/xfs/btrfs.
Current features
- multiple devices
- replication
- tiering
- data checksumming and compression (zlib only; also the code
doesn't work with
tiering yet)
- most of the normal posix fs features (no fallocate or quotas yet)Planned features:
- snapshots!
- erasure coding
- more
Quoted 8 lines
Quoted 3 lines
There will be a longer announcement on LKML/linux-fs in the near future (after I'm finished moving) - but I'd like to get it a bit more testing from a wider audience first, if possible.
Hi Kent,
one quick question about the roadmap at this point: As far as I understand bcachefs basically integrates bcache features directly in the filesystem. So does this deprecate bcache itself in your opinion? Bcache is obviously still useful for other FS, but I just want to know how things will get maintained in the future.
It would be rather disappointing if this were the case. bcache is quite useful for backing block devices that have no local filesystem, such as devices exported via iSCSI, devices used directly by VMs, etc...
Quoted 8 lines
I wanted to suggest / possibly start implementing bcache support for the debian installer - obviously that only makes sense if I can expect it to be in the mainline kernel for the foreseeable future :-).
Thanks,
~DavidSignature · 3 lines
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 43 lines · Jason Warr
Quoted 32 lines · David Mohr
Quoted 25 lines · Kent Overstreet
Short announcement, because I'm in the process of moving - but I wanted to get
this out there because the code is up and I think it's reasonably stable right
now.Bcachefs is a posix filesystem that I've been working towards for - well, quite
awhile now: it's intended as a competitor/replacement for ext4/xfs/btrfs.Current features
- multiple devices
- replication
- tiering
- data checksumming and compression (zlib only; also the code doesn't work with
tiering yet)
- most of the normal posix fs features (no fallocate or quotas yet)Planned features:
- snapshots!
- erasure coding
- moreQuoted 5 lines
There will be a longer announcement on LKML/linux-fs in the near future (after
I'm finished moving) - but I'd like to get it a bit more testing from a wider
audience first, if possible.Quoted 8 lines
Hi Kent,
one quick question about the roadmap at this point: As far as I understand bcachefs basically integrates bcache features directly in the filesystem. So does this deprecate bcache itself in your opinion? Bcache is obviously still useful for other FS, but I just want to know how things will get maintained in the future.
Quoted 3 lines
It would be rather disappointing if this were the case. bcache is quite useful for backing block devices that have no local filesystem, such as devices exported via iSCSI, devices used directly by VMs, etc...
- bcachefs/bcache2 getting merged is a _long_ way off, and when it does it's going to be more of an ext2/ext3 thing - the existing upstream bcache version will stay there for the foreseeable future.
- bcachefs/bcache2 still has all the same functionality bcache has for caching other block devices, and exporting thin provisioned block devices - that functionality won't be going away any time soon, if ever - so you'll be able to migrate to the new bcache code and on disk format without changing anything about how you use it.
The "backing device/cached dev" path _might_ eventually get deprecated in favor of having bcache manage all the block devices directly and export thin provisioned block devices - this is the existing "flash_vol_create" functionality.
Reason being the thin provisioned/fully bcache managed block devices path is quite a bit simpler and diverges less from the functionality bcachefs uses - and also cache coherency is fundamentally easier with bcache managing all the storage so performance should be better too.
However, if the backing device functionality ever gets removed it's a _long_ ways off, and I'll be asking for user feedback and making sure the thin provisioned/bcache managed block devices functionality works for everyone first.
Quoted 65 lines · Kent Overstreet
Quoted 41 lines · Jason Warr
Quoted 31 lines · David Mohr
Quoted 25 lines · Kent Overstreet
Short announcement, because I'm in the process of moving - but I wanted to get
this out there because the code is up and I think it's reasonably stable right
now.Bcachefs is a posix filesystem that I've been working towards for - well, quite
awhile now: it's intended as a competitor/replacement for ext4/xfs/btrfs.Current features
- multiple devices
- replication
- tiering
- data checksumming and compression (zlib only; also the code doesn't work with
tiering yet)
- most of the normal posix fs features (no fallocate or quotas yet)Planned features:
- snapshots!
- erasure coding
- moreQuoted 5 lines
There will be a longer announcement on LKML/linux-fs in the near future (after
I'm finished moving) - but I'd like to get it a bit more testing from a wider
audience first, if possible.Quoted 8 lines
Hi Kent,
one quick question about the roadmap at this point: As far as I understand bcachefs basically integrates bcache features directly in the filesystem. So does this deprecate bcache itself in your opinion? Bcache is obviously still useful for other FS, but I just want to know how things will get maintained in the future.
Quoted 3 lines
It would be rather disappointing if this were the case. bcache is quite useful for backing block devices that have no local filesystem, such as devices exported via iSCSI, devices used directly by VMs, etc...
- bcachefs/bcache2 getting merged is a _long_ way off, and when it does it's going to be more of an ext2/ext3 thing - the existing upstream bcache version will stay there for the foreseeable future.
- bcachefs/bcache2 still has all the same functionality bcache has for caching other block devices, and exporting thin provisioned block devices - that functionality won't be going away any time soon, if ever - so you'll be able to migrate to the new bcache code and on disk format without changing anything about how you use it.
The "backing device/cached dev" path _might_ eventually get deprecated in favor of having bcache manage all the block devices directly and export thin provisioned block devices - this is the existing "
flash_vol_create" functionality.Reason being the thin provisioned/fully bcache managed block devices path is quite a bit simpler and diverges less from the functionality bcachefs uses - and also cache coherency is fundamentally easier with bcache managing all the storage so performance should be better too.
Quoted 4 lines
However, if the backing device functionality ever gets removed it's a _long_ ways off, and I'll be asking for user feedback and making sure the thin provisioned/bcache managed block devices functionality works for everyone first.
As long as we get the same basic functionality in addition to the filesystem layer, it sounds like you plan on that, I'm happy.
Thank you for addressing this.
Quoted 2 lines · David Mohr
one quick question about the roadmap at this point: As far as I understand bcachefs basically integrates bcache features directly in the filesystem. So does this deprecate bcache itself in your opinion? Bcache is obviously still useful for other FS, but I just want to know how things will get maintained in the future.
If they remove bcache from the kernel a lot of peaple are going to have serious troubles, as it's not easy to remove. But to quote the developer:
"no don't worry. it's not going to be deleted from upstream"
Quoted 1 line
I wanted to suggest / possibly start implementing bcache support for the debian installer - obviously that only makes sense if I can expect it to be in the mainline kernel for the foreseeable future :-).
I can also make a quote on this question:
"the btree code is also hugely improved over what's in mainline, i'd like to get the improvements backported but i think it's just way way too much work" "bcache will be deprecated when a stable bcachefs is upstream (but it's going to be awhile before the on disk format is stable again)"
More info on what bcachefs actually is:
You initialize some fast storage as a caching-device. These store a btree-journal-change (or whatever is actually used internally) key-value storage system on disk. Next you can use this btree-caching-device to put a file system on top OR use it to store the cache data for a caching-block-device.
(Not sure if you can combine a caching device and a backing device into the same FS, but you will probably be able to.)
--
Killian De Volder