bcachefs with cache device and backing device

DISCUSSION Martin McClure 3 participants 10 messages May 13, 2016 – May 20, 2016 bcache (block layer cache)

Share & tools

Martin McClure

author

2016-05-13 01:56 UTC #0 lore raw

I'm trying out bcachefs for the first time. I can follow the instructions (format a cache device, mount it type bcache) and it works.

What I'd like to do, however, is to have an SSD cache device, and a rotating-disk backing device, as a bcachefs filesystem. Is this something that bcachefs can do at present? If so, what's the basic procedure?

Thanks,

-Martin

Kent Overstreet

2016-05-13 04:36 UTC #1 lore raw

Quoted 7 lines · Martin McClure

I'm trying out bcachefs for the first time. I can follow the instructions (format a cache device, mount it type bcache) and it works.
What I'd like to do, however, is to have an SSD cache device, and a rotating-disk backing device, as a bcachefs filesystem. Is this something that bcachefs can do at present? If so, what's the basic procedure?

Yeah - tiering replaces cache/backing devices

IIRC,

bcache format --tier 0 -C <SSD> --tier 1 -C <spinning rust>

(the -C is going to go away at some point)

Martin McClure

author

2016-05-13 04:37 UTC #2 lore raw

Quoted 5 lines · Kent Overstreet

Quoted 4 lines
What I'd like to do, however, is to have an SSD cache device, and a rotating-disk backing device, as a bcachefs filesystem. Is this something that bcachefs can do at present? If so, what's the basic procedure?

Quoted 7 lines

Yeah - tiering replaces cache/backing devices
IIRC,
bcache format --tier 0 -C <SSD> --tier 1 -C <spinning rust>
(the -C is going to go away at some point)

Thanks! I'll try that out tomorrow.

-Martin

Martin McClure

author

2016-05-18 02:46 UTC #3 lore raw

In reply to Kent Overstreet

Quoted 9 lines · Kent Overstreet

Yeah - tiering replaces cache/backing devices
IIRC,
bcache format --tier 0 -C <SSD> --tier 1 -C <spinning rust>
(the -C is going to go away at some point)

Had a chance to play with this some more, but still not getting it to work...

Formatting seems to work, and once I do this:

echo /dev/sdd1 > /sys/fs/bcache/register
echo /dev/nvme0n1 > /sys/fs/bcache/register
echo 1 > /sys/fs/bcache/<set-uuid>/blockdev_volume_create

a /dev/bcache0 has been created. However, if I try to mount it:

mount -t bcache /dev/bcache0 /mnt

it says:

mount: No such file or directory

with a return code of 32, which is documented as "mount failure".

At this point I reach the limit of my current understanding, but would like to understand more.

Thanks,

-Martin

Kent Overstreet

2016-05-18 04:01 UTC #4 lore raw

Quoted 24 lines · Martin McClure

Quoted 9 lines · Kent Overstreet
Yeah - tiering replaces cache/backing devices
IIRC,
bcache format --tier 0 -C <SSD> --tier 1 -C <spinning rust>
(the -C is going to go away at some point)
Had a chance to play with this some more, but still not getting it to work...
Formatting seems to work, and once I do this:
echo /dev/sdd1 > /sys/fs/bcache/register
echo /dev/nvme0n1 > /sys/fs/bcache/register
echo 1 > /sys/fs/bcache/<set-uuid>/blockdev_volume_create
a /dev/bcache0 has been created. However, if I try to mount it:
mount -t bcache /dev/bcache0 /mnt

Quoted 8 lines

it says:
mount: No such file or directory
with a return code of 32, which is documented as "mount failure".
At this point I reach the limit of my current understanding, but would like to understand more.

The intended mount path for multi device filesystems is currently broken... Chris got it working (to my surprise) by - I belive - registering all the devices via /sys/fs/bcache/register, and then mounting just one of the block devices - Chris, is that correct?

FYI though: tiering mode is _not_ as well tested currently, and Chris managed to hit a bug when he completely filled up tier 0 and it then wouldn't mount, because the btree updates journal replay was doing wouldn't fit on tier 0 - and I haven't merged the fix I sent him because it was kind of a hack.

But I just got back from vacation so I should be able to start working through these issues...

Christopher James Halse Rogers

2016-05-18 05:12 UTC #5 lore raw

Quoted 32 lines · Kent Overstreet

Quoted 31 lines · Martin McClure
Quoted 9 lines · Kent Overstreet
Yeah - tiering replaces cache/backing devices
IIRC,
bcache format --tier 0 -C <SSD> --tier 1 -C <spinning rust>
(the -C is going to go away at some point)
Had a chance to play with this some more, but still not getting it
to
work...
Formatting seems to work, and once I do this:
echo /dev/sdd1 > /sys/fs/bcache/register
echo /dev/nvme0n1 > /sys/fs/bcache/register echo 1 > /sys/fs/bcache/<set-uuid>/blockdev_volume_create
a /dev/bcache0 has been created. However, if I try to mount it:
mount -t bcache /dev/bcache0 /mnt
it says:
mount: No such file or directory
with a return code of 32, which is documented as "mount failure".

Quoted 8 lines

Quoted 3 lines
At this point I reach the limit of my current understanding, but
would
like to understand more.
The intended mount path for multi device filesystems is currently broken... Chris got it working (to my surprise) by - I belive - registering all the devices via /sys/fs/bcache/register, and then mounting just one of the block devices - Chris, is that correct?

That is indeed correct. Once the volume has all its components registered, it can be mounted by any of the block devices.

Kent Overstreet

2016-05-18 05:41 UTC #6 lore raw

Quoted 46 lines · Christopher James Halse Rogers

Quoted 36 lines · Kent Overstreet
Quoted 27 lines · Martin McClure
Quoted 9 lines · Kent Overstreet
Yeah - tiering replaces cache/backing devices
IIRC,
bcache format --tier 0 -C <SSD> --tier 1 -C <spinning rust>
(the -C is going to go away at some point)
Had a chance to play with this some more, but still not getting it
to
work...
Formatting seems to work, and once I do this:
echo /dev/sdd1 > /sys/fs/bcache/register
echo /dev/nvme0n1 > /sys/fs/bcache/register echo 1 > /sys/fs/bcache/<set-uuid>/blockdev_volume_create
a /dev/bcache0 has been created. However, if I try to mount it:
mount -t bcache /dev/bcache0 /mnt
it says:
Quoted 7 lines
mount: No such file or directory
with a return code of 32, which is documented as "mount failure".
At this point I reach the limit of my current understanding, but
would
like to understand more.
Quoted 7 lines
The intended mount path for multi device filesystems is currently broken...
Chris got it working (to my surprise) by - I belive - registering all the
devices via /sys/fs/bcache/register, and then mounting just one of the block
devices - Chris, is that correct?

Quoted 2 lines

That is indeed correct. Once the volume has all its components registered, it can be mounted by any of the block devices.

Actually, I just tested and mounting multile devices directly _does_ work - you just pass a colon separated list of devices to mount:

mount -t bcache /dev/sdb:/dev/sdc /mnt

I could have sworn this was broken, but worked just now...

Martin McClure

author

2016-05-20 00:41 UTC #7 lore raw

Quoted 6 lines · Kent Overstreet

Actually, I just tested and mounting multile devices directly _does_ work - you just pass a colon separated list of devices to mount:
mount -t bcache /dev/sdb:/dev/sdc /mnt
I could have sworn this was broken, but worked just now...

This works for me. Thanks for the feedback!

For any others wanting to walk this path in the near future, here's the entire sequence that is working for me (as of kernel and tools pulled on 2016-05-13):

  bcache format --tier 0 --bucket=512k --cache_mode=writeback \
      -C /dev/nvme0n1 \
      --tier 1 -C /dev/sdd1

echo /dev/sdd1 > /sys/fs/bcache/register echo /dev/nvme0n1 > /sys/fs/bcache/register mount -t bcache /dev/nvme0n1:/dev/sdd1 /mnt/whatever

Regards,

-Martin

Christopher James Halse Rogers

2016-05-20 00:44 UTC #8 lore raw

Quoted 15 lines · Martin McClure

Quoted 7 lines · Kent Overstreet
Actually, I just tested and mounting multile devices directly
_does_ work - you
just pass a colon separated list of devices to mount:
mount -t bcache /dev/sdb:/dev/sdc /mnt
I could have sworn this was broken, but worked just now...
This works for me. Thanks for the feedback!
For any others wanting to walk this path in the near future, here's the entire sequence that is working for me (as of kernel and tools pulled on 2016-05-13):

Quoted 6 lines

  bcache format --tier 0 --bucket=512k --cache_mode=writeback \
      -C /dev/nvme0n1 \
      --tier 1 -C /dev/sdd1
echo /dev/sdd1 > /sys/fs/bcache/register
echo /dev/nvme0n1 > /sys/fs/bcache/register

You should be able to remove these steps, I think?

Quoted 2 lines

mount -t bcache /dev/nvme0n1:/dev/sdd1 /mnt/whatever

If you've pre-registered, you should be able to just mount -t bcache /dev/nvme0n1 /mnt/whatever or, entirely equivalently:
mount -t bcache /dev/sdd1 /mnt/whatever

Martin McClure

author

2016-05-20 01:18 UTC #9 lore raw

Quoted 9 lines · Christopher James Halse Rogers

Quoted 7 lines
  bcache format --tier 0 --bucket=512k --cache_mode=writeback \
      -C /dev/nvme0n1 \
      --tier 1 -C /dev/sdd1
echo /dev/sdd1 > /sys/fs/bcache/register
echo /dev/nvme0n1 > /sys/fs/bcache/register
You should be able to remove these steps, I think?

Yes, I don't seem to need those.

Quoted 9 lines

Quoted 3 lines
mount -t bcache /dev/nvme0n1:/dev/sdd1 /mnt/whatever
If you've pre-registered, you should be able to just mount -t bcache /dev/nvme0n1 /mnt/whatever or, entirely equivalently:
mount -t bcache /dev/sdd1 /mnt/whatever

Yes. Looks like I need to *either* explicitly pre-register the devices *or* specify them both in the mount command.

Thanks for the clarification.

The option that I'm going with for now is:

  bcache format --tier 0 --bucket=512k --cache_mode=writeback \
      -C /dev/nvme0n1 \
      --tier 1 -C /dev/sdd1

mount -t bcache /dev/nvme0n1:/dev/sdd1 /mnt/whatever

since that shows me both devices in the output of "mount", reminding me and anyone else trying to figure out what's going on with this machine that both devices are involved in this filesystem.

Regards,

-Martin