Skip to contentSkip to conversation

xfs: assure zeroed memory buffers for certain kmem allocations

PATCH Superseded Bill O'Donnell 5 participants 17 messages Sep 16, 2019 – Sep 24, 2019 XFS filesystem

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfs: assure zeroed memory buffers for certain kmem allocations

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Introduce a xfs_buf_flag, _XBF_KMZ, to indicate a request for a zeroed buffer, and use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

---
 fs/xfs/xfs_buf.c | 8 ++++++--
 fs/xfs/xfs_buf.h | 4 +++-
 2 files changed, 9 insertions(+), 3 deletions(-)
fs/xfs/xfs_buf.c +6 -2
index 120ef99d09e8..916a3f782950 100644
@@ -345,16 +345,19 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
 
 	/*
 	 * for buffers that are contained within a single page, just allocate
 	 * the memory from the heap - there's no need for the complexity of
 	 * page arrays to keep allocation down to order 0.
 	 */
+	if (flags & _XBF_KMZ)
+		kmflag_mask |= KM_ZERO;
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
@@ -391,7 +394,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);
 		if (unlikely(page == NULL)) {
 			if (flags & XBF_READ_AHEAD) {
 				bp->b_page_count = i;
@@ -683,6 +686,7 @@ xfs_buf_get_map(
 	struct xfs_buf		*new_bp;
 	int			error = 0;
 
+	flags |= _XBF_KMZ;
 	error = xfs_buf_find(target, map, nmaps, flags, NULL, &bp);
 
 	switch (error) {
fs/xfs/xfs_buf.h +3 -1
index f6ce17d8d848..416ff588240a 100644
@@ -38,6 +38,7 @@
 #define _XBF_PAGES	 (1 << 20)/* backed by refcounted pages */
 #define _XBF_KMEM	 (1 << 21)/* backed by heap memory */
 #define _XBF_DELWRI_Q	 (1 << 22)/* buffer on a delwri queue */
+#define _XBF_KMZ	 (1 << 23)/* zeroed buffer required */
 
 typedef unsigned int xfs_buf_flags_t;
 
@@ -54,7 +55,8 @@ typedef unsigned int xfs_buf_flags_t;
 	{ XBF_UNMAPPED,		"UNMAPPED" },	/* ditto */\
 	{ _XBF_PAGES,		"PAGES" }, \
 	{ _XBF_KMEM,		"KMEM" }, \
-	{ _XBF_DELWRI_Q,	"DELWRI_Q" }
+	{ _XBF_DELWRI_Q,	"DELWRI_Q" }, \
+	{ _XBF_KMZ,             "KMEM_Z" }
 
 
 /*
Signature · 1 line
2.21.0
#1 lore raw
Quoted 5 lines · Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Quoted 8 lines

Introduce a xfs_buf_flag, _XBF_KMZ, to indicate a request for a zeroed buffer, and use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

I think this can probably be further optimized by not obtaining zeroed memory when we're about to fill the buffer from disk as the very next step.

(in this case, xfs_buf_read_map calls xfs_buf_get_map and then immediately reads the buffer from disk with _xfs_buf_read) xfs_buf_read_map adds XBF_READ to the flags during this process.

So I wonder if this can be simplified/optimized by just checking for XBF_READ in xfs_buf_allocate_memory's flags, and if it's not set, then request zeroed memory, because that indicates a buffer we'll be filling in from memory and subsequently writing to disk.

-Eric

Quoted 70 lines
---
 fs/xfs/xfs_buf.c | 8 ++++++--
 fs/xfs/xfs_buf.h | 4 +++-
 2 files changed, 9 insertions(+), 3 deletions(-)
fs/xfs/xfs_buf.c +6 -2
index 120ef99d09e8..916a3f782950 100644
@@ -345,16 +345,19 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
 
 	/*
 	 * for buffers that are contained within a single page, just allocate
 	 * the memory from the heap - there's no need for the complexity of
 	 * page arrays to keep allocation down to order 0.
 	 */
+	if (flags & _XBF_KMZ)
+		kmflag_mask |= KM_ZERO;
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
@@ -391,7 +394,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);
 		if (unlikely(page == NULL)) {
 			if (flags & XBF_READ_AHEAD) {
 				bp->b_page_count = i;
@@ -683,6 +686,7 @@ xfs_buf_get_map(
 	struct xfs_buf		*new_bp;
 	int			error = 0;
 
+	flags |= _XBF_KMZ;
 	error = xfs_buf_find(target, map, nmaps, flags, NULL, &bp);
 
 	switch (error) {
fs/xfs/xfs_buf.h +3 -1
index f6ce17d8d848..416ff588240a 100644
@@ -38,6 +38,7 @@
 #define _XBF_PAGES	 (1 << 20)/* backed by refcounted pages */
 #define _XBF_KMEM	 (1 << 21)/* backed by heap memory */
 #define _XBF_DELWRI_Q	 (1 << 22)/* buffer on a delwri queue */
+#define _XBF_KMZ	 (1 << 23)/* zeroed buffer required */
 
 typedef unsigned int xfs_buf_flags_t;
 
@@ -54,7 +55,8 @@ typedef unsigned int xfs_buf_flags_t;
 	{ XBF_UNMAPPED,		"UNMAPPED" },	/* ditto */\
 	{ _XBF_PAGES,		"PAGES" }, \
 	{ _XBF_KMEM,		"KMEM" }, \
-	{ _XBF_DELWRI_Q,	"DELWRI_Q" }
+	{ _XBF_DELWRI_Q,	"DELWRI_Q" }, \
+	{ _XBF_KMZ,             "KMEM_Z" }
 
 
 /*
#2 lore raw
Quoted 19 lines · Eric Sandeen
Quoted 5 lines · Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Quoted 8 lines

Introduce a xfs_buf_flag, _XBF_KMZ, to indicate a request for a zeroed buffer, and use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

I think this can probably be further optimized by not obtaining zeroed memory when we're about to fill the buffer from disk as the very next step.

Quoted 8 lines

(in this case, xfs_buf_read_map calls xfs_buf_get_map and then immediately reads the buffer from disk with _xfs_buf_read) xfs_buf_read_map adds XBF_READ to the flags during this process.

So I wonder if this can be simplified/optimized by just checking for XBF_READ in xfs_buf_allocate_memory's flags, and if it's not set, then request zeroed memory, because that indicates a buffer we'll be filling in from memory and subsequently writing to disk.

I was wondering that ("Why can't we allocate a zeroed buffer only for the get_buf case so that we don't have to do that for the read_buf case?") too. Once you do that then you can then remove all the explicit memset calls too.

Quoted 17 lines

-Eric

Quoted 15 lines
---
 fs/xfs/xfs_buf.c | 8 ++++++--
 fs/xfs/xfs_buf.h | 4 +++-
 2 files changed, 9 insertions(+), 3 deletions(-)
fs/xfs/xfs_buf.c +1 -0
index 120ef99d09e8..916a3f782950 100644
@@ -345,16 +345,19 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
 
Quoted 12 lines

fs/xfs/xfs_buf.c xfs_buf_allocate_memory( · line 360

Quoted 12 lines
fs/xfs/xfs_buf.c +3 -1
 	/*
 	 * for buffers that are contained within a single page, just allocate
 	 * the memory from the heap - there's no need for the complexity of
 	 * page arrays to keep allocation down to order 0.
 	 */
+	if (flags & _XBF_KMZ)
+		kmflag_mask |= KM_ZERO;
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS | kmflag_mask);

Does this overflow 80 columns?

--D

Quoted 43 lines
Quoted 43 lines
fs/xfs/xfs_buf.c +2 -1
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
@@ -391,7 +394,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);
 		if (unlikely(page == NULL)) {
 			if (flags & XBF_READ_AHEAD) {
 				bp->b_page_count = i;
@@ -683,6 +686,7 @@ xfs_buf_get_map(
 	struct xfs_buf		*new_bp;
 	int			error = 0;
 
+	flags |= _XBF_KMZ;
 	error = xfs_buf_find(target, map, nmaps, flags, NULL, &bp);
 
 	switch (error) {
fs/xfs/xfs_buf.h +3 -1
index f6ce17d8d848..416ff588240a 100644
@@ -38,6 +38,7 @@
 #define _XBF_PAGES	 (1 << 20)/* backed by refcounted pages */
 #define _XBF_KMEM	 (1 << 21)/* backed by heap memory */
 #define _XBF_DELWRI_Q	 (1 << 22)/* buffer on a delwri queue */
+#define _XBF_KMZ	 (1 << 23)/* zeroed buffer required */
 
 typedef unsigned int xfs_buf_flags_t;
 
@@ -54,7 +55,8 @@ typedef unsigned int xfs_buf_flags_t;
 	{ XBF_UNMAPPED,		"UNMAPPED" },	/* ditto */\
 	{ _XBF_PAGES,		"PAGES" }, \
 	{ _XBF_KMEM,		"KMEM" }, \
-	{ _XBF_DELWRI_Q,	"DELWRI_Q" }
+	{ _XBF_DELWRI_Q,	"DELWRI_Q" }, \
+	{ _XBF_KMZ,             "KMEM_Z" }
 
 
 /*
#3 lore raw
In reply to Eric Sandeen
Quoted 10 lines · Eric Sandeen
Quoted 9 lines · Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Introduce a xfs_buf_flag, _XBF_KMZ, to indicate a request for a zeroed buffer, and use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

Quoted 8 lines
Quoted 4 lines

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

I think this can probably be further optimized by not obtaining zeroed memory when we're about to fill the buffer from disk as the very next step.

Yep. I missed that redundancy.

Quoted 9 lines

(in this case, xfs_buf_read_map calls xfs_buf_get_map and then immediately reads the buffer from disk with _xfs_buf_read) xfs_buf_read_map adds XBF_READ to the flags during this process.

So I wonder if this can be simplified/optimized by just checking for XBF_READ in xfs_buf_allocate_memory's flags, and if it's not set, then request zeroed memory, because that indicates a buffer we'll be filling in from memory and subsequently writing to disk.

nod.

Quoted 73 lines

-Eric

Quoted 70 lines
---
 fs/xfs/xfs_buf.c | 8 ++++++--
 fs/xfs/xfs_buf.h | 4 +++-
 2 files changed, 9 insertions(+), 3 deletions(-)
fs/xfs/xfs_buf.c +6 -2
index 120ef99d09e8..916a3f782950 100644
@@ -345,16 +345,19 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
 
 	/*
 	 * for buffers that are contained within a single page, just allocate
 	 * the memory from the heap - there's no need for the complexity of
 	 * page arrays to keep allocation down to order 0.
 	 */
+	if (flags & _XBF_KMZ)
+		kmflag_mask |= KM_ZERO;
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
@@ -391,7 +394,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);
 		if (unlikely(page == NULL)) {
 			if (flags & XBF_READ_AHEAD) {
 				bp->b_page_count = i;
@@ -683,6 +686,7 @@ xfs_buf_get_map(
 	struct xfs_buf		*new_bp;
 	int			error = 0;
 
+	flags |= _XBF_KMZ;
 	error = xfs_buf_find(target, map, nmaps, flags, NULL, &bp);
 
 	switch (error) {
fs/xfs/xfs_buf.h +3 -1
index f6ce17d8d848..416ff588240a 100644
@@ -38,6 +38,7 @@
 #define _XBF_PAGES	 (1 << 20)/* backed by refcounted pages */
 #define _XBF_KMEM	 (1 << 21)/* backed by heap memory */
 #define _XBF_DELWRI_Q	 (1 << 22)/* buffer on a delwri queue */
+#define _XBF_KMZ	 (1 << 23)/* zeroed buffer required */
 
 typedef unsigned int xfs_buf_flags_t;
 
@@ -54,7 +55,8 @@ typedef unsigned int xfs_buf_flags_t;
 	{ XBF_UNMAPPED,		"UNMAPPED" },	/* ditto */\
 	{ _XBF_PAGES,		"PAGES" }, \
 	{ _XBF_KMEM,		"KMEM" }, \
-	{ _XBF_DELWRI_Q,	"DELWRI_Q" }
+	{ _XBF_DELWRI_Q,	"DELWRI_Q" }, \
+	{ _XBF_KMZ,             "KMEM_Z" }
 
 
 /*
#4 lore raw
In reply to Bill O'Donnell
Quoted 53 lines · Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Introduce a xfs_buf_flag, _XBF_KMZ, to indicate a request for a zeroed buffer, and use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

---
 fs/xfs/xfs_buf.c | 8 ++++++--
 fs/xfs/xfs_buf.h | 4 +++-
 2 files changed, 9 insertions(+), 3 deletions(-)
fs/xfs/xfs_buf.c +5 -2
index 120ef99d09e8..916a3f782950 100644
@@ -345,16 +345,19 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
 
 	/*
 	 * for buffers that are contained within a single page, just allocate
 	 * the memory from the heap - there's no need for the complexity of
 	 * page arrays to keep allocation down to order 0.
 	 */
+	if (flags & _XBF_KMZ)
+		kmflag_mask |= KM_ZERO;
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
@@ -391,7 +394,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);
 		if (unlikely(page == NULL)) {
 			if (flags & XBF_READ_AHEAD) {
 				bp->b_page_count = i;

fs/xfs/xfs_buf.c xfs_buf_get_map( · line 690

Quoted 5 lines
fs/xfs/xfs_buf.c +1 -0
@@ -683,6 +686,7 @@ xfs_buf_get_map(
 	struct xfs_buf		*new_bp;
 	int			error = 0;
 
+	flags |= _XBF_KMZ;
 	error = xfs_buf_find(target, map, nmaps, flags, NULL, &bp);

IIRC, this flag was supposed to go into xfs_trans_get_buf_map() and direct callers of xfs_buf_get*() that weren't in the read path. That avoids the need for zeroing pages that we are going to DMA actual data into before it gets to users...

fs/xfs/xfs_buf.h line 41

Quoted 10 lines
fs/xfs/xfs_buf.c +0 -0
 	switch (error) {
fs/xfs/xfs_buf.h +1 -0
index f6ce17d8d848..416ff588240a 100644
@@ -38,6 +38,7 @@
 #define _XBF_PAGES	 (1 << 20)/* backed by refcounted pages */
 #define _XBF_KMEM	 (1 << 21)/* backed by heap memory */
 #define _XBF_DELWRI_Q	 (1 << 22)/* buffer on a delwri queue */
+#define _XBF_KMZ	 (1 << 23)/* zeroed buffer required */

"KMZ" isn't very descriptive, and it shouldn't have a "_" prefix as it's not internal to the buffer cache - it's a caller controlled flag like XBF_TRYLOCK.

I'd suggest something like XBF_INIT_PAGES or XBF_ZERO to make it clear we are asking for ithe buffer to be explicitly initialised to zero.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#5 lore raw

PATCH v2 1/1 xfs: assure zeroed memory buffers for certain kmem allocations

In reply to Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Introduce a xfs_buf_flag, XBF_ZERO, to indicate a request for a zeroed buffer, and use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

---
v2: zeroed buffer not required for XBF_READ case. Correct placement
and rename the XBF_ZERO flag.

 fs/xfs/xfs_buf.c       | 9 +++++++--
 fs/xfs/xfs_buf.h       | 2 ++
 fs/xfs/xfs_trans_buf.c | 2 ++
 3 files changed, 11 insertions(+), 2 deletions(-)
fs/xfs/xfs_buf.c +7 -2
index 120ef99d09e8..0d96efff451e 100644
@@ -345,6 +345,10 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
+
+	if ((flags & XBF_ZERO) && !(flags & XBF_READ))
+		kmflag_mask |= KM_ZERO;
 
 	/*
 	 * for buffers that are contained within a single page, just allocate
@@ -354,7 +358,8 @@ xfs_buf_allocate_memory(
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask,
+					   KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
@@ -391,7 +396,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);
 		if (unlikely(page == NULL)) {
 			if (flags & XBF_READ_AHEAD) {
 				bp->b_page_count = i;
fs/xfs/xfs_buf.h +2 -0
index f6ce17d8d848..dccdd653c9dc 100644
@@ -33,6 +33,7 @@
 /* flags used only as arguments to access routines */
 #define XBF_TRYLOCK	 (1 << 16)/* lock requested, but do not wait */
 #define XBF_UNMAPPED	 (1 << 17)/* do not map the buffer */
+#define XBF_ZERO	 (1 << 18)/* zeroed buffer required */
 
 /* flags used only internally */
 #define _XBF_PAGES	 (1 << 20)/* backed by refcounted pages */
@@ -52,6 +53,7 @@ typedef unsigned int xfs_buf_flags_t;
 	{ XBF_WRITE_FAIL,	"WRITE_FAIL" }, \
 	{ XBF_TRYLOCK,		"TRYLOCK" },	/* should never be set */\
 	{ XBF_UNMAPPED,		"UNMAPPED" },	/* ditto */\
+	{ XBF_ZERO,		"KMEM_ZERO" }, \
 	{ _XBF_PAGES,		"PAGES" }, \
 	{ _XBF_KMEM,		"KMEM" }, \
 	{ _XBF_DELWRI_Q,	"DELWRI_Q" }
fs/xfs/xfs_trans_buf.c +2 -0
index b5b3a78ef31c..087d413c1490 100644
@@ -123,6 +123,8 @@ xfs_trans_get_buf_map(
 	xfs_buf_t		*bp;
 	struct xfs_buf_log_item	*bip;
 
+	flags |= XBF_ZERO;
+
 	if (!tp)
 		return xfs_buf_get_map(target, map, nmaps, flags);
 
Signature · 1 line
2.21.0
#6 lore raw
Quoted 27 lines · Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Introduce a xfs_buf_flag, XBF_ZERO, to indicate a request for a zeroed buffer, and use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

---
v2: zeroed buffer not required for XBF_READ case. Correct placement
and rename the XBF_ZERO flag.

 fs/xfs/xfs_buf.c       | 9 +++++++--
 fs/xfs/xfs_buf.h       | 2 ++
 fs/xfs/xfs_trans_buf.c | 2 ++
 3 files changed, 11 insertions(+), 2 deletions(-)
fs/xfs/xfs_buf.c +0 -0
index 120ef99d09e8..0d96efff451e 100644

fs/xfs/xfs_buf.c xfs_buf_allocate_memory( · line 350

Quoted 6 lines
fs/xfs/xfs_buf.c +3 -0
@@ -345,6 +345,10 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
+
+	if ((flags & XBF_ZERO) && !(flags & XBF_READ))

The sole caller of xfs_buf_allocate_memory is xfs_buf_get_map. If _get_map is called from the *read_buf* functions, they pass in XBF_READ to ensure buffer contents are read. The other _get_map callers seem to be initializing metadata blocks and do not set XBF_READ.

So I wonder, do you even need XBF_ZERO? Or could this be reduced to:

	if (!(flags & XBF_READ))
		km_flag_mask |= KM_ZERO;

?

--D

Quoted 59 lines
fs/xfs/xfs_buf.c +4 -2
+		kmflag_mask |= KM_ZERO;
 
 	/*
 	 * for buffers that are contained within a single page, just allocate
@@ -354,7 +358,8 @@ xfs_buf_allocate_memory(
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask,
+					   KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
@@ -391,7 +396,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);
 		if (unlikely(page == NULL)) {
 			if (flags & XBF_READ_AHEAD) {
 				bp->b_page_count = i;
fs/xfs/xfs_buf.h +2 -0
index f6ce17d8d848..dccdd653c9dc 100644
@@ -33,6 +33,7 @@
 /* flags used only as arguments to access routines */
 #define XBF_TRYLOCK	 (1 << 16)/* lock requested, but do not wait */
 #define XBF_UNMAPPED	 (1 << 17)/* do not map the buffer */
+#define XBF_ZERO	 (1 << 18)/* zeroed buffer required */
 
 /* flags used only internally */
 #define _XBF_PAGES	 (1 << 20)/* backed by refcounted pages */
@@ -52,6 +53,7 @@ typedef unsigned int xfs_buf_flags_t;
 	{ XBF_WRITE_FAIL,	"WRITE_FAIL" }, \
 	{ XBF_TRYLOCK,		"TRYLOCK" },	/* should never be set */\
 	{ XBF_UNMAPPED,		"UNMAPPED" },	/* ditto */\
+	{ XBF_ZERO,		"KMEM_ZERO" }, \
 	{ _XBF_PAGES,		"PAGES" }, \
 	{ _XBF_KMEM,		"KMEM" }, \
 	{ _XBF_DELWRI_Q,	"DELWRI_Q" }
fs/xfs/xfs_trans_buf.c +2 -0
index b5b3a78ef31c..087d413c1490 100644
@@ -123,6 +123,8 @@ xfs_trans_get_buf_map(
 	xfs_buf_t		*bp;
 	struct xfs_buf_log_item	*bip;
 
+	flags |= XBF_ZERO;
+
 	if (!tp)
 		return xfs_buf_get_map(target, map, nmaps, flags);
 
Signature · 1 line
2.21.0
#7 lore raw
Quoted 40 lines · Darrick J. Wong
Quoted 27 lines · Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Introduce a xfs_buf_flag, XBF_ZERO, to indicate a request for a zeroed buffer, and use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

---
v2: zeroed buffer not required for XBF_READ case. Correct placement
and rename the XBF_ZERO flag.

 fs/xfs/xfs_buf.c       | 9 +++++++--
 fs/xfs/xfs_buf.h       | 2 ++
 fs/xfs/xfs_trans_buf.c | 2 ++
 3 files changed, 11 insertions(+), 2 deletions(-)
fs/xfs/xfs_buf.c +0 -0
index 120ef99d09e8..0d96efff451e 100644

fs/xfs/xfs_buf.c xfs_buf_allocate_memory( · line 350

Quoted 6 lines
fs/xfs/xfs_buf.c +3 -0
@@ -345,6 +345,10 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
+
+	if ((flags & XBF_ZERO) && !(flags & XBF_READ))

The sole caller of xfs_buf_allocate_memory is xfs_buf_get_map. If _get_map is called from the *read_buf* functions, they pass in XBF_READ to ensure buffer contents are read. The other _get_map callers seem to be initializing metadata blocks and do not set XBF_READ.

Quoted 8 lines

So I wonder, do you even need XBF_ZERO? Or could this be reduced to:

	if (!(flags & XBF_READ))
		km_flag_mask |= KM_ZERO;

?

--D

Yeah, I see no cases of XBF_ZERO true AND XBF_READ true. Anytime XBF_ZERO is true, XBF_READ is false. So, yes, it could be reduced by eliminating XBF_ZERO.

Thanks,
Bill

Quoted 60 lines
Quoted 60 lines
fs/xfs/xfs_buf.c +4 -2
+		kmflag_mask |= KM_ZERO;
 
 	/*
 	 * for buffers that are contained within a single page, just allocate
@@ -354,7 +358,8 @@ xfs_buf_allocate_memory(
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask,
+					   KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
@@ -391,7 +396,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);
 		if (unlikely(page == NULL)) {
 			if (flags & XBF_READ_AHEAD) {
 				bp->b_page_count = i;
fs/xfs/xfs_buf.h +2 -0
index f6ce17d8d848..dccdd653c9dc 100644
@@ -33,6 +33,7 @@
 /* flags used only as arguments to access routines */
 #define XBF_TRYLOCK	 (1 << 16)/* lock requested, but do not wait */
 #define XBF_UNMAPPED	 (1 << 17)/* do not map the buffer */
+#define XBF_ZERO	 (1 << 18)/* zeroed buffer required */
 
 /* flags used only internally */
 #define _XBF_PAGES	 (1 << 20)/* backed by refcounted pages */
@@ -52,6 +53,7 @@ typedef unsigned int xfs_buf_flags_t;
 	{ XBF_WRITE_FAIL,	"WRITE_FAIL" }, \
 	{ XBF_TRYLOCK,		"TRYLOCK" },	/* should never be set */\
 	{ XBF_UNMAPPED,		"UNMAPPED" },	/* ditto */\
+	{ XBF_ZERO,		"KMEM_ZERO" }, \
 	{ _XBF_PAGES,		"PAGES" }, \
 	{ _XBF_KMEM,		"KMEM" }, \
 	{ _XBF_DELWRI_Q,	"DELWRI_Q" }
fs/xfs/xfs_trans_buf.c +2 -0
index b5b3a78ef31c..087d413c1490 100644
@@ -123,6 +123,8 @@ xfs_trans_get_buf_map(
 	xfs_buf_t		*bp;
 	struct xfs_buf_log_item	*bip;
 
+	flags |= XBF_ZERO;
+
 	if (!tp)
 		return xfs_buf_get_map(target, map, nmaps, flags);
 
Signature · 1 line
2.21.0
#8 lore raw

PATCH v3 1/1 xfs: assure zeroed memory buffers for certain kmem allocations

In reply to Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

---
v3: remove XBF_ZERO flag, and instead use XBF_READ flag only.
v2: zeroed buffer not required for XBF_READ case. Correct placement
and rename the XBF_ZERO flag.

fs/xfs/xfs_buf.c | 9 +++++++--

 1 file changed, 7 insertions(+), 2 deletions(-)
fs/xfs/xfs_buf.c +7 -2
index 120ef99d09e8..6fbe63f34a68 100644
@@ -345,6 +345,10 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
+
+	if (!(flags & XBF_READ))
+		kmflag_mask |= KM_ZERO;
 
 	/*
 	 * for buffers that are contained within a single page, just allocate
@@ -354,7 +358,8 @@ xfs_buf_allocate_memory(
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask,
+					   KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
@@ -391,7 +396,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);
 		if (unlikely(page == NULL)) {
 			if (flags & XBF_READ_AHEAD) {
 				bp->b_page_count = i;
Signature · 1 line
2.21.0
#9 lore raw

fs/xfs/xfs_buf.c xfs_buf_allocate_memory( · line 399

Quoted 11 lines · Bill O'Donnell
fs/xfs/xfs_buf.c +5 -1
+	uint			kmflag_mask = 0;
+
+	if (!(flags & XBF_READ))
+		kmflag_mask |= KM_ZERO;
@@ -391,7 +396,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);

alloc_page takes GFP_ flags, not KM_. In fact sparse should have warned about this.

#10 lore raw
Quoted 6 lines · Christoph Hellwig
Quoted 4 lines · Bill O'Donnell
fs/xfs/xfs_buf.c +4 -0
+	uint			kmflag_mask = 0;
+
+	if (!(flags & XBF_READ))
+		kmflag_mask |= KM_ZERO;
Quoted 9 lines

fs/xfs/xfs_buf.c xfs_buf_allocate_memory( · line 399

Quoted 6 lines
fs/xfs/xfs_buf.c +1 -1
@@ -391,7 +396,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);

alloc_page takes GFP_ flags, not KM_. In fact sparse should have warned about this.

I wondered if the KM flag needed conversion to GFP, but saw no warning. Thanks-
Bill

#11 lore raw
Quoted 11 lines · Bill O'Donnell
Quoted 9 lines

fs/xfs/xfs_buf.c xfs_buf_allocate_memory( · line 399

Quoted 6 lines
fs/xfs/xfs_buf.c +1 -1
@@ -391,7 +396,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);

alloc_page takes GFP_ flags, not KM_. In fact sparse should have warned about this.

I wondered if the KM flag needed conversion to GFP, but saw no warning.

I'd be tempted to just do a manual memset after either kind of allocation.

#12 lore raw
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Bill O'Donnell
Quoted 9 lines

fs/xfs/xfs_buf.c xfs_buf_allocate_memory( · line 399

Quoted 6 lines
fs/xfs/xfs_buf.c +1 -1
@@ -391,7 +396,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);

alloc_page takes GFP_ flags, not KM_. In fact sparse should have warned about this.

Quoted 4 lines
Quoted 1 line

I wondered if the KM flag needed conversion to GFP, but saw no warning.

I'd be tempted to just do a manual memset after either kind of allocation.

At some point I think Dave had suggested that at least when allocating pages, using the flag would be more efficient?

-Eric

#13 lore raw
Quoted 12 lines · Eric Sandeen
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Bill O'Donnell
Quoted 9 lines

fs/xfs/xfs_buf.c xfs_buf_allocate_memory( · line 399

Quoted 6 lines
fs/xfs/xfs_buf.c +1 -1
@@ -391,7 +396,7 @@ xfs_buf_allocate_memory(
 		struct page	*page;
 		uint		retries = 0;
 retry:
-		page = alloc_page(gfp_mask);
+		page = alloc_page(gfp_mask | kmflag_mask);

alloc_page takes GFP_ flags, not KM_. In fact sparse should have warned about this.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

I wondered if the KM flag needed conversion to GFP, but saw no warning.

I'd be tempted to just do a manual memset after either kind of allocation.

At some point I think Dave had suggested that at least when allocating pages, using the flag would be more efficient?

With some configurations pages come from the free lists pre-zeroed, and so don't need zeroing to initialise them (e.g. when memory poisoning is turned on, or pages are being zeroed on free). Hence if you use __GFP_ZERO the it will only zero if the page obtained from the freelist isn't already zero. The __GFP_ZERO call will also use the most efficient method of zeroing the page for the platform via clear_page() rather than memset()....

/me shrugs and doesn't really care either way....

-Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#14 lore raw

PATCH v4 1/1 xfs: assure zeroed memory buffers for certain kmem allocations

In reply to Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

---
v4: use __GFP_ZERO as part of gfp_mask (instead of KM_ZERO)
v3: remove XBF_ZERO flag, and instead use XBF_READ flag only.
v2: zeroed buffer not required for XBF_READ case. Correct placement
and rename the XBF_ZERO flag.

 fs/xfs/xfs_buf.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
fs/xfs/xfs_buf.c +11 -1
index 120ef99d09e8..5d0a68de5fa6 100644
@@ -345,6 +345,15 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
+
+	/*
+	 * assure zeroed buffer for non-read cases.
+	 */
+	if (!(flags & XBF_READ)) {
+		kmflag_mask |= KM_ZERO;
+		gfp_mask |= __GFP_ZERO;
+	}
 
 	/*
 	 * for buffers that are contained within a single page, just allocate
@@ -354,7 +363,8 @@ xfs_buf_allocate_memory(
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask,
+					   KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
Signature · 1 line
2.21.0
Reviewed-by on v4 1/1
#15 lore raw
Quoted 28 lines · Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

---
v4: use __GFP_ZERO as part of gfp_mask (instead of KM_ZERO)
v3: remove XBF_ZERO flag, and instead use XBF_READ flag only.
v2: zeroed buffer not required for XBF_READ case. Correct placement
and rename the XBF_ZERO flag.

 fs/xfs/xfs_buf.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
fs/xfs/xfs_buf.c +0 -0
index 120ef99d09e8..5d0a68de5fa6 100644
@@ -345,6 +345,15 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;

fs/xfs/xfs_buf.c xfs_buf_allocate_memory( · line 356

Quoted 10 lines
fs/xfs/xfs_buf.c +9 -0
 	int			error;
+	uint			kmflag_mask = 0;
+
+	/*
+	 * assure zeroed buffer for non-read cases.
+	 */
+	if (!(flags & XBF_READ)) {
+		kmflag_mask |= KM_ZERO;
+		gfp_mask |= __GFP_ZERO;
+	}

Jeez it feels grody to have to set two different flags variables just to get __GFP_ZERO consistently but I'll run it through xfstests overnight.

--D

Quoted 16 lines
fs/xfs/xfs_buf.c +2 -1
 	/*
 	 * for buffers that are contained within a single page, just allocate
@@ -354,7 +363,8 @@ xfs_buf_allocate_memory(
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask,
+					   KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
Signature · 1 line
2.21.0
#16 lore raw
In reply to Bill O'Donnell
Quoted 11 lines · Bill O'Donnell

Guarantee zeroed memory buffers for cases where potential memory leak to disk can occur. In these cases, kmem_alloc is used and doesn't zero the buffer, opening the possibility of information leakage to disk.

Use existing infrastucture (xfs_buf_allocate_memory) to obtain the already zeroed buffer from kernel memory.

This solution avoids the performance issue that would occur if a wholesale change to replace kmem_alloc with kmem_zalloc was done.

Quoted 8 lines

---
v4: use __GFP_ZERO as part of gfp_mask (instead of KM_ZERO)
v3: remove XBF_ZERO flag, and instead use XBF_READ flag only.
v2: zeroed buffer not required for XBF_READ case. Correct placement
and rename the XBF_ZERO flag.

 fs/xfs/xfs_buf.c | 12 +++++++++++-

/me wakes up and wonders, what about the other kmem_alloc_io callers in xfs? I think we need to slip a KM_ZERO into the allocation call when we allocate the log buffers, right? What about log recovery?

--D

Quoted 35 lines
 1 file changed, 11 insertions(+), 1 deletion(-)
fs/xfs/xfs_buf.c +11 -1
index 120ef99d09e8..5d0a68de5fa6 100644
@@ -345,6 +345,15 @@ xfs_buf_allocate_memory(
 	unsigned short		page_count, i;
 	xfs_off_t		start, end;
 	int			error;
+	uint			kmflag_mask = 0;
+
+	/*
+	 * assure zeroed buffer for non-read cases.
+	 */
+	if (!(flags & XBF_READ)) {
+		kmflag_mask |= KM_ZERO;
+		gfp_mask |= __GFP_ZERO;
+	}
 
 	/*
 	 * for buffers that are contained within a single page, just allocate
@@ -354,7 +363,8 @@ xfs_buf_allocate_memory(
 	size = BBTOB(bp->b_length);
 	if (size < PAGE_SIZE) {
 		int align_mask = xfs_buftarg_dma_alignment(bp->b_target);
-		bp->b_addr = kmem_alloc_io(size, align_mask, KM_NOFS);
+		bp->b_addr = kmem_alloc_io(size, align_mask,
+					   KM_NOFS | kmflag_mask);
 		if (!bp->b_addr) {
 			/* low memory - use alloc_page loop instead */
 			goto use_alloc_page;
Signature · 1 line
2.21.0