Skip to contentSkip to conversation

Can't resgister cache

DISCUSSION Franco 3 participants 5 messages Feb 15, 2012 – Feb 16, 2012 2 messages not in the archive bcache (block layer cache)

lore
Share & tools
lore

Franco

author
#0 lore raw

Hi

I was in the process of testing linux-bcache but was seeing some strange behaviour so I thought I would remount my cached filesystem. I then noticed that the performance results started looking like the cache just wasn't there, so I unregistered it:

bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered
bcache: Caching disabled for sdc

but now when I try to register it again, I get bcache: error opening /dev/sdb: device busy.

I haven't tried rebooting yet but probably will do after I've sent this message.

These were the strange results I was getting that prompted me to unmount and remount the filesystems.

Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD for bcache.

Filesystem            Size  Used Avail Use% Mounted on
/dev/bcache0          1.9T   26G  1.8T   2% /data33
/dev/sdd              1.9T   23G  1.8T   2% /data34

Wrote an 8GB file to each filesystem, then tried reading it back. Machine has 8GB RAM.

Speed for random reads from identical SSD = 181.766523 MB/Sec, random read test done in the following order with measured speed in MB/Sec. The random IO is actually just reading with a 4GB stride, wrapping around at the end of the file.

/data33 49
/data34 114
/data33 114
/data33 and /data44 simultaneously 78 and 59 respectively /data33 76
/data33 68
/data34 114
/data34 96
/data34 69
/data34 74
/data33 118
/data33 40
/data34 91

After remounting both filesystems:

/data34 148
/data33 120
/data34 120
/data33 82

After unregistering the cache

/data33 65
/data34 83
/data33 99
/data34 113

Cheers,
Franco

#1 lore raw

Not sure what's going on with not being able to register again - not a lot of info to go off of. It was working last I checked in the bcache-3.2-dev branch, it might be buggy in the older bcache branch.

From your numbers it looks like the 8 gb of data wasn't in the cache - bcache bypasses sequential IO by default. You can flip it off by echoing 0 to sequential_cutoff.

Quoted 71 lines · Franco

Hi

I was in the process of testing linux-bcache but was seeing some strange behaviour so I thought I would remount my cached filesystem. I then noticed that the performance results started looking like the cache just wasn't there, so I unregistered it:

bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered
bcache: Caching disabled for sdc

but now when I try to register it again, I get bcache: error opening /dev/sdb: device busy.

I haven't tried rebooting yet but probably will do after I've sent this message.

These were the strange results I was getting that prompted me to unmount and remount the filesystems.

Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD for bcache.

Filesystem            Size  Used Avail Use% Mounted on /dev/bcache0          1.9T   26G  1.8T   2% /data33 /dev/sdd              1.9T   23G  1.8T   2% /data34

Wrote an 8GB file to each filesystem, then tried reading it back. Machine has 8GB RAM.

Speed for random reads from identical SSD = 181.766523 MB/Sec, random read test done in the following order with measured speed in MB/Sec. The random IO is actually just reading with a 4GB stride, wrapping around at the end of the file.

/data33 49
/data34 114
/data33 114
/data33 and /data44 simultaneously 78 and 59 respectively /data33 76
/data33 68
/data34 114
/data34 96
/data34 69
/data34 74
/data33 118
/data33 40
/data34 91

After remounting both filesystems:

/data34 148
/data33 120
/data34 120
/data33 82

After unregistering the cache

/data33 65
/data34 83
/data33 99
/data34 113

Cheers,
Franco

--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected] More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): CAH+dOx+g=eKDgTCnVdZYKrtn7tZjAnWeMdYP0KuZwD=rktsuJw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

author
#3 lore raw

I did a reboot and got it registered again.

bio: create slab <bio-1> at 1
bcache: invalidating existing data
bcache: registered cache device sdb
bcache: Caching sdc, inserted new UUID 87ba7904-7690-4985-b6a8-daeba41712a3 EXT4-fs (bcache0): mounted filesystem with ordered data mode. Opts: (null)

First read went at 119 MB/Sec, second at 37MB/Sec

Here's what's in stats_hour

/sys/block/sdc/bcache/stats_hour/cache_bypass_hits ::::::::::::::
44
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_bypass_misses ::::::::::::::
81172
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_hit_ratio ::::::::::::::
0
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_hits ::::::::::::::
24
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_miss_collisions ::::::::::::::
14
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_misses ::::::::::::::
407422
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_readaheads ::::::::::::::
0

With echo 0 >/sys/block/sdc/bcache/sequential_cutoff

33 MB/Sec

Quoted 80 lines · Kent Overstreet

Not sure what's going on with not being able to register again - not a lot of info to go off of. It was working last I checked in the bcache-3.2-dev branch, it might be buggy in the older bcache branch.

Quoted 1 line

From your numbers it looks like the 8 gb of data wasn't in the cache -

bcache bypasses sequential IO by default. You can flip it off by echoing 0 to sequential_cutoff.

Quoted 71 lines · Franco

Hi

I was in the process of testing linux-bcache but was seeing some strange behaviour so I thought I would remount my cached filesystem. I then noticed that the performance results started looking like the cache just wasn't there, so I unregistered it:

bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered
bcache: Caching disabled for sdc

but now when I try to register it again, I get bcache: error opening /dev/sdb: device busy.

I haven't tried rebooting yet but probably will do after I've sent this message.

These were the strange results I was getting that prompted me to unmount and remount the filesystems.

Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD for bcache.

Filesystem            Size  Used Avail Use% Mounted on
/dev/bcache0          1.9T   26G  1.8T   2% /data33
/dev/sdd              1.9T   23G  1.8T   2% /data34

Wrote an 8GB file to each filesystem, then tried reading it back. Machine has 8GB RAM.

Speed for random reads from identical SSD = 181.766523 MB/Sec, random read test done in the following order with measured speed in MB/Sec. The random IO is actually just reading with a 4GB stride, wrapping around at the end of the file.

/data33 49
/data34 114
/data33 114
/data33 and /data44 simultaneously 78 and 59 respectively /data33 76
/data33 68
/data34 114
/data34 96
/data34 69
/data34 74
/data33 118
/data33 40
/data34 91

After remounting both filesystems:

/data34 148
/data33 120
/data34 120
/data33 82

After unregistering the cache

/data33 65
/data34 83
/data33 99
/data34 113

Cheers,
Franco

--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw

That sounds like your SSD is slow and bottlenecking you... What kind of SSD is it?

Could be some strange plugging thing, maybe...

Quoted 134 lines · Franco

I did a reboot and got it registered again.

bio: create slab <bio-1> at 1
bcache: invalidating existing data
bcache: registered cache device sdb
bcache: Caching sdc, inserted new UUID 87ba7904-7690-4985-b6a8-daeba41712a3 EXT4-fs (bcache0): mounted filesystem with ordered data mode. Opts: (null)

First read went at 119 MB/Sec, second at 37MB/Sec

Here's what's in stats_hour

/sys/block/sdc/bcache/stats_hour/cache_bypass_hits ::::::::::::::
44
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_bypass_misses ::::::::::::::
81172
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_hit_ratio ::::::::::::::
0
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_hits ::::::::::::::
24
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_miss_collisions ::::::::::::::
14
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_misses ::::::::::::::
407422
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_readaheads ::::::::::::::
0

With echo 0 >/sys/block/sdc/bcache/sequential_cutoff

33 MB/Sec

Quoted 4 lines · Kent Overstreet

Not sure what's going on with not being able to register again - not a lot of info to go off of. It was working last I checked in the bcache-3.2-dev branch, it might be buggy in the older bcache branch.

Quoted 76 lines
Quoted 1 line

From your numbers it looks like the 8 gb of data wasn't in the cache -

bcache bypasses sequential IO by default. You can flip it off by echoing 0 to sequential_cutoff.

Quoted 71 lines · Franco

Hi

I was in the process of testing linux-bcache but was seeing some strange behaviour so I thought I would remount my cached filesystem. I then noticed that the performance results started looking like the cache just wasn't there, so I unregistered it:

bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered
bcache: Caching disabled for sdc

but now when I try to register it again, I get bcache: error opening /dev/sdb: device busy.

I haven't tried rebooting yet but probably will do after I've sent this message.

These were the strange results I was getting that prompted me to unmount and remount the filesystems.

Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD for bcache.

Filesystem            Size  Used Avail Use% Mounted on /dev/bcache0          1.9T   26G  1.8T   2% /data33 /dev/sdd              1.9T   23G  1.8T   2% /data34

Wrote an 8GB file to each filesystem, then tried reading it back. Machine has 8GB RAM.

Speed for random reads from identical SSD = 181.766523 MB/Sec, random read test done in the following order with measured speed in MB/Sec. The random IO is actually just reading with a 4GB stride, wrapping around at the end of the file.

/data33 49
/data34 114
/data33 114
/data33 and /data44 simultaneously 78 and 59 respectively /data33 76
/data33 68
/data34 114
/data34 96
/data34 69
/data34 74
/data33 118
/data33 40
/data34 91

After remounting both filesystems:

/data34 148
/data33 120
/data34 120
/data33 82

After unregistering the cache

/data33 65
/data34 83
/data33 99
/data34 113

Cheers,
Franco

--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected] More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected] More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): CAC7rs0sadP553tU0foAgDMzWh9euk4LyUYFL0_13rdeD+bBdWA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#6 lore raw

Also, IO is still getting bypassed. If sequential_cutoff is 0, then the only reason that'd happen is if the latency to the SSD is getting too high...

Quoted 140 lines · Kent Overstreet

That sounds like your SSD is slow and bottlenecking you... What kind of SSD is it?

Could be some strange plugging thing, maybe...

Quoted 134 lines · Franco

I did a reboot and got it registered again.

bio: create slab <bio-1> at 1
bcache: invalidating existing data
bcache: registered cache device sdb
bcache: Caching sdc, inserted new UUID 87ba7904-7690-4985-b6a8-daeba41712a3 EXT4-fs (bcache0): mounted filesystem with ordered data mode. Opts: (null)

First read went at 119 MB/Sec, second at 37MB/Sec

Here's what's in stats_hour

/sys/block/sdc/bcache/stats_hour/cache_bypass_hits ::::::::::::::
44
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_bypass_misses ::::::::::::::
81172
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_hit_ratio ::::::::::::::
0
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_hits ::::::::::::::
24
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_miss_collisions ::::::::::::::
14
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_misses ::::::::::::::
407422
::::::::::::::
/sys/block/sdc/bcache/stats_hour/cache_readaheads ::::::::::::::
0

With echo 0 >/sys/block/sdc/bcache/sequential_cutoff

33 MB/Sec

Quoted 4 lines · Kent Overstreet

Not sure what's going on with not being able to register again - not a lot of info to go off of. It was working last I checked in the bcache-3.2-dev branch, it might be buggy in the older bcache branch.

Quoted 76 lines
Quoted 1 line

From your numbers it looks like the 8 gb of data wasn't in the cache -

bcache bypasses sequential IO by default. You can flip it off by echoing 0 to sequential_cutoff.

Quoted 71 lines · Franco

Hi

I was in the process of testing linux-bcache but was seeing some strange behaviour so I thought I would remount my cached filesystem. I then noticed that the performance results started looking like the cache just wasn't there, so I unregistered it:

bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered
bcache: Caching disabled for sdc

but now when I try to register it again, I get bcache: error opening /dev/sdb: device busy.

I haven't tried rebooting yet but probably will do after I've sent this message.

These were the strange results I was getting that prompted me to unmount and remount the filesystems.

Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD for bcache.

Filesystem            Size  Used Avail Use% Mounted on /dev/bcache0          1.9T   26G  1.8T   2% /data33 /dev/sdd              1.9T   23G  1.8T   2% /data34

Wrote an 8GB file to each filesystem, then tried reading it back. Machine has 8GB RAM.

Speed for random reads from identical SSD = 181.766523 MB/Sec, random read test done in the following order with measured speed in MB/Sec. The random IO is actually just reading with a 4GB stride, wrapping around at the end of the file.

/data33 49
/data34 114
/data33 114
/data33 and /data44 simultaneously 78 and 59 respectively /data33 76
/data33 68
/data34 114
/data34 96
/data34 69
/data34 74
/data33 118
/data33 40
/data34 91

After remounting both filesystems:

/data34 148
/data33 120
/data34 120
/data33 82

After unregistering the cache

/data33 65
/data34 83
/data33 99
/data34 113

Cheers,
Franco

--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected] More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in the body of a message to [email protected] More majordomo info at  http://vger.kernel.org/majordomo-info.html