Disabling barriers on NVC-backed HDD
Hi list,
I'm not sure this is the correct list to post my question; if so, feel free to ignore this message.
On XFS FAQs I (rightfully) read that barrier should be *always* enabled (and they are, by default), unless a write-loss protected writeback cache is used at the RAID card level. So far, so good.
Newer HDDs has themselves have a non-volatile cache (NVC) of some sort, basically:
- DRAM + eMCL: in a powerloss event, the DRAM cache is immediately dumped on the eMLC flash;
- DRAM + NOR flash: NOR mirrors a (small) portion of the DRAM cache, used for write acceleration purpose;
- dedicated "dump" areas on the disks: they effectively mirror a portion of the DRAM cache.
My question is: do you think it is safe to disable barriers, both for XFS and in general terms, on these disks? Or they should be considered as the same "dumb" unprotected DRAM caches found on classical HDD?
From a side, these *are* powerloss-protected caches. Problem is that all these power-protection schemes are considered "secret sauce / trade secret" by HDD vendors and, for this reason, there are very little (if any) informations on their inner working.
Regards.
Signature · 5 lines
Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: [email protected] - [email protected] GPG public key ID: FF5F32A8
Quoted 17 lines · Gionatan Danti
Hi list,
I'm not sure this is the correct list to post my question; if so, feel free to ignore this message.On XFS FAQs I (rightfully) read that barrier should be *always* enabled (and they are, by default), unless a write-loss protected writeback cache is used at the RAID card level. So far, so good.
Newer HDDs has themselves have a non-volatile cache (NVC) of some sort, basically:
- DRAM + eMCL: in a powerloss event, the DRAM cache is immediately dumped on the eMLC flash;
- DRAM + NOR flash: NOR mirrors a (small) portion of the DRAM cache, used for write acceleration purpose;
- dedicated "dump" areas on the disks: they effectively mirror a portion of the DRAM cache.
Quoted 8 lines
My question is: do you think it is safe to disable barriers, both for XFS and in general terms, on these disks? Or they should be considered as the same "dumb" unprotected DRAM caches found on classical HDD?
From a side, these *are* powerloss-protected caches. Problem is that all these power-protection schemes are considered "secret sauce / trade secret" by HDD vendors and, for this reason, there are very little (if any) informations on their inner working.
I would evaluate these drives to find out if you really /can/ yank the power without losing anything.
That said, if the manufacturers aren't willing to tell you how that feature works, I'd just as soon pretend the feature didn't exist and continue sending flushes to the drive.
FWIW if the drive really /does/ have a non-volatile WC then a flush should have nearly zero overhead. (Or so you'd think...)
--D
Quoted 12 lines
Regards.
Signature · 9 lines
Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: [email protected] - [email protected] GPG public key ID: FF5F32A8 -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Il 15-11-2017 18:31 Darrick J. Wong ha scritto:
Quoted 3 lines
I would evaluate these drives to find out if you really /can/ yank the power without losing anything.
If I can get some hard drives for testing, I'll surely do that. However, any in-house testing inevitably has limited scope and it can miss some important failure modes, so I am interested in other users' first-hand experience.
Quoted 3 lines
That said, if the manufacturers aren't willing to tell you how that feature works, I'd just as soon pretend the feature didn't exist and continue sending flushes to the drive.
From a point, I agree: I do not like black boxes. On the other hand, hardware RAID controllers *are* black boxes, and we all use them...
Quoted 2 lines
FWIW if the drive really /does/ have a non-volatile WC then a flush should have nearly zero overhead. (Or so you'd think...)
From my understanding (and I can be wrong), these drives honour cache flushes, with the associated performance drop.
Ok, too much conjectures from my side, I think. I should *really* get my hand on some of these drives...
Regards.
Signature · 5 lines
Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: [email protected] - [email protected] GPG public key ID: FF5F32A8
Quoted 9 lines · Darrick J. Wong
I would evaluate these drives to find out if you really /can/ yank the power without losing anything.
That said, if the manufacturers aren't willing to tell you how that feature works, I'd just as soon pretend the feature didn't exist and continue sending flushes to the drive.
FWIW if the drive really /does/ have a non-volatile WC then a flush should have nearly zero overhead. (Or so you'd think...)
And in that case it will report WCE=0 and Linux won't even flush. As is the case for typical enterprise disks.
Il 15-11-2017 20:47 Christoph Hellwig ha scritto:
Quoted 2 lines
And in that case it will report WCE=0 and Linux won't even flush. As is the case for typical enterprise disks.
Good point. Based on what I read here[1], page n.4, it *seems* that Seagate AWC (advanced write cache) enabled drives report WCD (write cache disabled) to the host OS.
I think this is the key parameter to watch: if the HDD vendor is confident enough to report WCE=0/WCD, than it should be safe running without barriers (after all, in this case Linux *will* disable barrier by default).
Thank you Christoph for pointing that.
Regards.
Signature · 5 lines
Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: [email protected] - [email protected] GPG public key ID: FF5F32A8
Il 15-11-2017 21:17 Gionatan Danti ha scritto:
Quoted 8 lines
Il 15-11-2017 20:47 Christoph Hellwig ha scritto:
Quoted 2 lines
And in that case it will report WCE=0 and Linux won't even flush. As is the case for typical enterprise disks.
Good point. Based on what I read here[1], page n.4, it *seems* that Seagate AWC (advanced write cache) enabled drives report WCD (write cache disabled) to the host OS.
Quoted 7 lines
I think this is the key parameter to watch: if the HDD vendor is confident enough to report WCE=0/WCD, than it should be safe running without barriers (after all, in this case Linux *will* disable barrier by default).
Thank you Christoph for pointing that.
Regards.
Sorry, I forgot the link: https://www.seagate.com/files/www-content/product-content/enterprise-performance-savvio-fam/enterprise-performance-15k-hdd/_cross-product/_shared/doc/enchanced-cache-advantage-tp691.1-1610us.pdf
Signature · 5 lines
Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: [email protected] - [email protected] GPG public key ID: FF5F32A8
Il 15-11-2017 21:18 Gionatan Danti ha scritto:
Quoted 2 lines
Sorry, I forgot the link:
https://www.seagate.com/files/www-content/product-content/enterprise-performance-savvio-fam/enterprise-performance-15k-hdd/_cross-product/_shared/doc/enchanced-cache-advantage-tp691.1-1610us.pdf
Ok, it seems that for Seagate drives the real document to read is that one[1]. Quoting:
"When WCE=0, Advanced provides NVC-protected write caching over the portion of the DRAM used to coalesce writes. Write data only goes into NVC when there is an unexpected power loss to the drive. The NVC has 90-day data retention. When WCE=1, a Advanced Caching drive will operate on writes like a standard drive–writes in cache are not protected by NVC and may be lost with power loss."
In short:
- WCE=1 -> entire cache enabled -> powerloss = dataloss;
- WCE=0 -> small write cache segment enabled -> powerloss = nodataloss.
Unfortunately, I can not find anything on that regard about HGST's MediaCache...
Regards.
Signature · 5 lines
Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: [email protected] - [email protected] GPG public key ID: FF5F32A8