Skip to contentSkip to conversation

How to test in-bound dedupe

DISCUSSION Jitendra 2 participants 2 messages Mar 28, 2017 – Mar 29, 2017 Btrfs file system

lore
Share & tools
lore

Jitendra

author
#0 lore raw

Hi Qu/All,

I am looking into in-memory in-bound dedup. I have cloned your git tree from following urls.

Linux Tree: https://github.com/adam900710/linux.git wang_dedupe_latest
btrfs-progs: https://[email protected]:adam900710/btrfs-progs.git dedupe_20170316

Then run the follwoing test https://btrfs.wiki.kernel.org/index.php/User_notes_on_dedupe.

Is there any stuff that needs to be tested for it? if something, then please point out me.

My goal is to understand in-memory dedup and try to participate in on-disk de-dup.

---
Jitendra

#1 lore raw
Quoted 6 lines · Jitendra

Hi Qu/All,

I am looking into in-memory in-bound dedup. I have cloned your git tree from
following urls.

Quoted 6 lines

Linux Tree: https://github.com/adam900710/linux.git wang_dedupe_latest
btrfs-progs: https://[email protected]:adam900710/btrfs-progs.git
dedupe_20170316

Then run the follwoing test
https://btrfs.wiki.kernel.org/index.php/User_notes_on_dedupe.

Thanks for having interest in in-band dedupe.

Quoted 4 lines

Is there any stuff that needs to be tested for it? if something, then please
point out me.

Please note that, CONFIG_BTRFS_DEBUG must be set, or inband dedupe ioctl will still be hidden.

Basically, you could try all xfstests with inband dedupe enabled.
While it's not officially supported yet, you could modify _scratch_mount and _test_mount to always enable dedupe just after mount.

Quoted 3 lines

My goal is to understand in-memory dedup and try to participate in on-disk de-dup.

On-disk dedupe backend is not provided yet, although planned and not hard to implement.

Our plan is to add on-disk dedupe backend after in-memory dedupe is merged and tested.

And about understanding current dedupe algorithm, it should not be that hard to understand, while the really hard part is from kernel itself.

Tons of facilities is involved in fs, from memory management to VFS to page cache to delayed allocation.

Thanks,
Qu

Quoted 5 lines

---
Jitendra