Skip to contentSkip to conversation

deduplication, at LVM level or FS level?

linux-lvm Anthony Clark 2 participants 2 messages Nov 16, 2009 – Nov 17, 2009 1 message not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive

Hi All,

I'm dealing with ever-larger HSM and backups, as I'm sure many of us are. I was wondering where the "best" place to perform data deduplication was? (compression, ha!)

Should dedup happen at the filesystem level or the LVM level? I'm somewhat motivated to try to shoehorn lessfs' code into something that doesn't require FUSE, but I'm still at the early stages of examining this.

Anyone doing anything similar?

Thanks

Anthony

#2 lore raw
Quoted 1 line · Anthony Clark

Should dedup happen at the filesystem level or the LVM level?

Along with the recent announcement of ZFS dedup, Jeff Bonwick listed a few pros/cons of different dedup approaches:
http://blogs.sun.com/bonwick/entry/zfs_dedup
("What to dedup: Files, blocks, or bytes?")

HTH,
C.

Signature · 3 lines
BOFH excuse #441:

Hash table has woodworm