blktap: Sync with XCP, dropping zero-copy.
Hi all.
This is the better half of what XCP developments and testing brought for blktap.
It's fairly a big change in how I/O buffers are managed. Prior to this series, we had zero-copy I/O down to userspace. Unfortunately, blktap2 always had to jump through a couple of extra loops to do so. Present state of that is that we dropped that, so all tapdev I/O is bounced to/from a bunch of normal pages. Essentially replacing the old VMA management with a couple insert/zap VM calls.
One issue was that the kernel can't cope with recursive I/O. Submitting an iovec on a tapdev, passing it to userspace and then reissuing the same vector via AIO apparently doesn't fit well with the lock protocol applied to those pages. This is the main reason why blktap had to deal a lot with grant refs. About as much as blkback already does before passing requests on. What happens there is that it's aliasing those granted pages under a different PFN, thereby in a separate page struct. Not pretty, but it worked, so it's not the reason why we chose to drop that at some point.
The more prevalent problem was network storage, especially anything involving TCP. That includes VHD on both NFS and iSCSI. The problem with those is that retransmits (by the transport) and I/O op completion (on the application layer) are never synchronized. With sufficiently bad timing and bit of jitter on the network, it's perfectly common for the kernel to complete an AIO request with a late ack on the input queue just when retransmission timer is about to fire underneath. The completion will unmap the granted frame, crashing any uncanceled retransmission on an empty page frame. There are different ways to deal with that. Page destructors might be one way, but as far as I heard they are not particularly popular upstream. Issuing the block I/O on dom0 memory is straightforward and avoids the hassle. One could go argue that retransmits after DIO completion are still a potential privacy problem (I did), but it's not Xen's problem after all.
If zero-copy becomes more attractive again, the plan would be to rather use grantdev in userspace, such as a filter driver for tapdisk instead. Until then, there's presumably a notable difference in L2 cache footprint. Then again, there's also a whole number of cycles not spent in redundant hypercalls now, to deal with the pseudophysical map.
There are also benefits or non-issues.
- This blktap is rather xen-independent. Certainly depends on the common ring macros, but lacking grant stuff it compiles on bare metal Linux with no CONFIG_XEN. Not consummated here, because that's going to move the source tree out of drivers/xen. But I'd like to post a new branch proposing to do so.
- Blktaps size in dom0 didn't really change. Frames (now pages) were always pooled. We used to balloon memory to claim space for redundant grant mappings. Now we reserve, by default, the same volume in normal memory.
- The previous code would runs all I/O on a single pool. Typically two rings worth of requests. Sufficient for a whole lot of systems, especially with single storage backends, but not so nice when I/O on a number of otherwise independent filers or volumes collides.
Pools are refcounted kobjects in sysfs. Toolstacks using the new code can thereby choose to elimitate bottlenecks by grouping taps on different buffer pools. Pools can also be resized, to accomodate greater queue depths. [Note that blkback still has the same issue, so guests won't take advantage of that before that's resolved as well.]
- XCP started to make some use of stacking tapdevs. Think pointing the image chain of a bunch of "leaf" taps to a shared parent node. That works fairly well, but definitely takes independent resource pools to avoid deadlock by parent starvation then.
Please pull upstream/xen/dom0/backend/blktap2 fromgit://xenbits.xensource.com/people/dstodden/linux.git
.. and/or upstream/xen/next for a merge.
I also pulled in the pending warning fix from Teck Choon Giam.
Cheers,
Daniel
PATCH 1/5 blktap: Manage segment buffers in mempools.
- Replaces the request free list with a (mempooled) slab.
- Replaces request buckets with a mempool. No buckets, because we're doing full s/g on page granularity anyway, so can gfp() independent pages everywhere. Allocations are 1-11 page-sized segments.
- Adds support for multiple page pools.
- Adds pools to sysfs. Linked as a 'pools' kset to blktap-control.
- Makes the per-tap pool selectable. Attribute 'pool' on the tap device.
- Make pools online-resizeable. Attributes free/size on the pool kobj.
--- drivers/xen/blktap/blktap.h | 35 ++- drivers/xen/blktap/control.c | 80 ++++++-- drivers/xen/blktap/device.c | 2 +- drivers/xen/blktap/request.c | 509 +++++++++++++++++++++++++----------------- drivers/xen/blktap/ring.c | 10 +- drivers/xen/blktap/sysfs.c | 36 +++ 6 files changed, 433 insertions(+), 239 deletions(-)
drivers/xen/blktap/blktap.h +24 -11
@@ -121,17 +121,19 @@ struct blktap_statistics { };struct blktap_request { +struct blktap *tap;struct request *rq;uint16_t usr_idx;uint8_t status;atomic_t pendcnt; -uint8_t nr_pages;unsigned short operation;struct timeval time;struct grant_handle_pair handles[BLKIF_MAX_SEGMENTS_PER_REQUEST ]; -struct list_head free_list; + +struct page *pages[BLKIF_MAX_SEGMENTS_PER_REQUEST ]; +int nr_pages; };struct blktap {
@@ -140,6 +142,7 @@ struct blktap {struct blktap_ring ring;struct blktap_device device; +struct blktap_page_pool *pool;int pending_cnt;struct blktap_request *pending_requests[MAX_PENDING_REQS ];
@@ -152,6 +155,13 @@ struct blktap {struct blktap_statistics stats; }; +struct blktap_page_pool { +struct mempool_s *bufs; +spinlock_t lock; +struct kobject kobj; +wait_queue_head_t wait; +}; +extern struct mutex blktap_lock;extern struct blktap **blktaps;extern int blktap_max_minor;
@@ -165,7 +175,6 @@ size_t blktap_ring_debug(struct blktap *, char *, size_t);int blktap_ring_create (struct blktap *);int blktap_ring_destroy (struct blktap *);void blktap_ring_kick_user (struct blktap *); -void blktap_ring_kick_all (void );int blktap_sysfs_init (void );void blktap_sysfs_exit (void );
@@ -181,19 +190,23 @@ void blktap_device_destroy_sync(struct blktap *);int blktap_device_run_queue (struct blktap *);void blktap_device_end_request (struct blktap *,struct blktap_request *,int ); -int blktap_request_pool_init (void ); -void blktap_request_pool_free (void ); -int blktap_request_pool_grow (void ); -int blktap_request_pool_shrink (void ); -struct blktap_request *blktap_request_allocate (struct blktap *); +int blktap_page_pool_init (struct kobject *); +void blktap_page_pool_exit (void ); +struct blktap_page_pool *blktap_page_pool_get (const char *); + +size_t blktap_request_debug (struct blktap *,char *,size_t ); +struct blktap_request *blktap_request_alloc (struct blktap *); +int blktap_request_get_pages (struct blktap *,struct blktap_request *,int );void blktap_request_free (struct blktap *,struct blktap_request *); -struct page *request_to_page (struct blktap_request *,int ); +void blktap_request_bounce (struct blktap *,struct blktap_request *,int ,int );static inline unsigned long request_to_kaddr (struct blktap_request *req,int seg) { -unsigned long pfn =page_to_pfn (request_to_page (req, seg)); -return (unsigned long )pfn_to_kaddr (pfn); +return (unsigned long )page_address (req->pages[seg]); } +#define request_to_page (_request, _seg) ((_request)->pages[_seg]) + +#endif
drivers/xen/blktap/control.c +65 -15
151 lines not on this page · raw message
drivers/xen/blktap/device.c +1 -1
@@ -605,7 +605,7 @@ blktap_device_run_queue(struct blktap *tap)break ; } - request =blktap_request_allocate (tap); + request =blktap_request_alloc (tap);if (!request) { tap->stats.st_oo_req++; goto wait;
drivers/xen/blktap/request.c +306 -203
603 lines not on this page · raw message
drivers/xen/blktap/ring.c +1 -9
@@ -17,8 +17,6 @@int blktap_ring_major;static struct cdev blktap_ring_cdev; -static DECLARE_WAIT_QUEUE_HEAD (blktap_poll_wait); -static inline struct blktap *vma_to_blktap (struct vm_area_struct *vma) {
@@ -409,7 +407,7 @@ static unsigned int blktap_ring_poll(struct file *filp, poll_table *wait)struct blktap_ring *ring = &tap->ring;int work =0 ; -poll_wait (filp, &blktap_poll_wait, wait); +poll_wait (filp, &tap->pool->wait, wait);poll_wait (filp, &ring->poll_wait, wait);down_read (¤t->mm->mmap_sem);
@@ -440,12 +438,6 @@ blktap_ring_kick_user(struct blktap *tap)wake_up (&tap->ring.poll_wait); } -void -blktap_ring_kick_all (void ) -{ -wake_up (&blktap_poll_wait); -} -int blktap_ring_destroy (struct blktap *tap) {
drivers/xen/blktap/sysfs.c +36 -0
@@ -104,6 +104,8 @@ blktap_sysfs_debug_device(struct device *dev, struct device_attribute *attr, cha s +=blktap_control_debug (tap, s, end - s); + s +=blktap_request_debug (tap, s, end - s); + s +=blktap_device_debug (tap, s, end - s); s +=blktap_ring_debug (tap, s, end - s);
@@ -129,6 +131,38 @@ blktap_sysfs_show_task(struct device *dev, struct device_attribute *attr, char * } static DEVICE_ATTR(task, S_IRUGO, blktap_sysfs_show_task, NULL); +static ssize_t +blktap_sysfs_show_pool (struct device *dev, +struct device_attribute *attr, +char *buf) +{ +struct blktap *tap =dev_get_drvdata (dev); +return sprintf (buf,"%s" ,kobject_name (&tap->pool->kobj)); +} + +static ssize_t +blktap_sysfs_store_pool (struct device *dev, +struct device_attribute *attr, +const char *buf,size_t size) +{ +struct blktap *tap =dev_get_drvdata (dev); +struct blktap_page_pool *pool, *tmp = tap->pool; + +if (tap->device.gd) +return -EBUSY ; + + pool =blktap_page_pool_get (buf); +if (IS_ERR (pool)) +return PTR_ERR (pool); + + tap->pool = pool; +kobject_put (&tmp->kobj); + +return size; +} +DEVICE_ATTR (pool,S_IRUSR |S_IWUSR , + blktap_sysfs_show_pool, blktap_sysfs_store_pool); +int blktap_sysfs_create (struct blktap *tap) {
@@ -151,6 +185,8 @@ blktap_sysfs_create(struct blktap *tap)if (!err) err =device_create_file (dev, &dev_attr_task);if (!err) + err =device_create_file (dev, &dev_attr_pool); +if (!err) ring->dev = dev;else device_unregister (dev);
Signature · 1 line
1.7.0.4
PATCH 2/5 blktap: Make VMAs non-foreign and bounce buffered.
In reply to Daniel StoddenDrop zero-copy I/O. Removes all grantmap mechanism from blktap, bouncing I/O from/to a pool of dom0 memory and request SGs. Essentially renders blktap without any residual dependencies on Xen whatsoever.
--- drivers/xen/blktap/blktap.h | 43 ++-- drivers/xen/blktap/control.c | 8 +- drivers/xen/blktap/device.c | 564 ++++++------------------------------------ drivers/xen/blktap/request.c | 20 ++- drivers/xen/blktap/ring.c | 319 +++++++++++++------------ 5 files changed, 285 insertions(+), 669 deletions(-)
drivers/xen/blktap/blktap.h +20 -23
@@ -7,7 +7,6 @@#include <linux/init.h> #include <linux/scatterlist.h> #include <xen/blkif.h> -#include <xen/grant_table.h> extern int blktap_debug_level;extern int blktap_ring_major;
@@ -27,7 +26,6 @@ extern int blktap_device_major;#define MAX_BLKTAP_DEVICE 1024 -#define BLKTAP_CONTROL 1#define BLKTAP_DEVICE 4#define BLKTAP_DEVICE_CLOSED 5#define BLKTAP_SHUTDOWN_REQUESTED 8
@@ -94,11 +92,13 @@ struct blktap_ring {struct task_struct *task;struct vm_area_struct *vma; -struct blkif_front_ring ring; -struct vm_foreign_map foreign_map; +struct blkif_front_ring ring;unsigned long ring_vstart;unsigned long user_vstart; +int n_pending; +struct blktap_request *pending[MAX_PENDING_REQS ]; +wait_queue_head_t poll_wait;dev_t devno;
@@ -123,19 +123,21 @@ struct blktap_statistics {struct blktap_request {struct blktap *tap;struct request *rq; -uint16_t usr_idx; - -uint8_t status; -atomic_t pendcnt; -unsigned short operation; +int usr_idx; +int operation;struct timeval time; -struct grant_handle_pair handles[BLKIF_MAX_SEGMENTS_PER_REQUEST ]; +struct scatterlist sg_table[BLKIF_MAX_SEGMENTS_PER_REQUEST ];struct page *pages[BLKIF_MAX_SEGMENTS_PER_REQUEST ];int nr_pages; }; +#define blktap_for_each_sg (_sg, _req, _i) \ + for (_sg = (_req)->sg_table, _i = 0; \ + _i < (_req)->nr_pages; \ + (_sg)++, (_i)++) +struct blktap {int minor;unsigned long dev_inuse;
@@ -144,10 +146,6 @@ struct blktap {struct blktap_device device;struct blktap_page_pool *pool; -int pending_cnt; -struct blktap_request *pending_requests[MAX_PENDING_REQS ]; -struct scatterlist sg[BLKIF_MAX_SEGMENTS_PER_REQUEST ]; -wait_queue_head_t remove_wait;struct work_struct remove_work;char name[BLKTAP2_MAX_MESSAGE_LEN ];
@@ -174,6 +172,13 @@ void blktap_ring_exit(void);size_t blktap_ring_debug (struct blktap *,char *,size_t );int blktap_ring_create (struct blktap *);int blktap_ring_destroy (struct blktap *); +struct blktap_request *blktap_ring_make_request (struct blktap *); +void blktap_ring_free_request (struct blktap *,struct blktap_request *); +void blktap_ring_submit_request (struct blktap *,struct blktap_request *); +int blktap_ring_map_request_segment (struct blktap *,struct blktap_request *,int ); +int blktap_ring_map_request (struct blktap *,struct blktap_request *); +void blktap_ring_unmap_request (struct blktap *,struct blktap_request *); +void blktap_ring_set_message (struct blktap *,int );void blktap_ring_kick_user (struct blktap *);int blktap_sysfs_init (void );
@@ -187,7 +192,7 @@ size_t blktap_device_debug(struct blktap *, char *, size_t);int blktap_device_create (struct blktap *,struct blktap_params *);int blktap_device_destroy (struct blktap *);void blktap_device_destroy_sync (struct blktap *); -int blktap_device_run_queue (struct blktap *); +void blktap_device_run_queue (struct blktap *);void blktap_device_end_request (struct blktap *,struct blktap_request *,int );int blktap_page_pool_init (struct kobject *);
@@ -200,13 +205,5 @@ int blktap_request_get_pages(struct blktap *, struct blktap_request *, int);void blktap_request_free (struct blktap *,struct blktap_request *);void blktap_request_bounce (struct blktap *,struct blktap_request *,int ,int ); -static inline unsigned long -request_to_kaddr (struct blktap_request *req,int seg) -{ -return (unsigned long )page_address (req->pages[seg]); -} - -#define request_to_page (_request, _seg) ((_request)->pages[_seg]) -#endif
drivers/xen/blktap/control.c +1 -7
@@ -18,13 +18,10 @@ blktap_control_get_minor(void)int minor;struct blktap *tap; - tap =kmalloc (sizeof (*tap),GFP_KERNEL ); + tap =kzalloc (sizeof (*tap),GFP_KERNEL );if (unlikely (!tap))return NULL ; -memset (tap,0 ,sizeof (*tap)); -sg_init_table (tap->sg,BLKIF_MAX_SEGMENTS_PER_REQUEST ); -mutex_lock (&blktap_lock);for (minor =0 ; minor < blktap_max_minor; minor++)
@@ -290,9 +287,6 @@ blktap_init(void) {int err; -if (!xen_pv_domain ()) -return -ENODEV ; - err =blktap_device_init ();if (err) goto fail;
drivers/xen/blktap/device.c +79 -485
632 lines not on this page · raw message
drivers/xen/blktap/request.c +19 -1
@@ -3,7 +3,6 @@#include <linux/mutex.h> #include <linux/sched.h> #include <linux/device.h> -#include <xen/balloon.h> #include "blktap.h"
@@ -129,6 +128,25 @@ blktap_request_free(struct blktap *tap,__page_pool_wake (tap->pool); } +void +blktap_request_bounce (struct blktap *tap, +struct blktap_request *request, +int seg,int write) +{ +struct scatterlist *sg = &request->sg_table[seg]; +void *s, *p; + +BUG_ON (seg >= request->nr_pages); + + s =sg_virt (sg); + p =page_address (request->pages[seg]) + sg->offset; + +if (write) +memcpy (p, s, sg->length); +else +memcpy (s, p, sg->length); +} +static void blktap_request_ctor (void *obj) {
drivers/xen/blktap/ring.c +166 -153
434 lines not on this page · raw message
Signature · 1 line
1.7.0.4
PATCH 3/5 blktap: Add queue access macros.
In reply to Daniel StoddenReduce request queue access in different contexts to a small set of macros. Also simplifies transitions between different kernel versions.
--- drivers/xen/blktap/device.c | 54 +++++++++++++++++++++++++++++++++++-------- 1 files changed, 44 insertions(+), 10 deletions(-)
drivers/xen/blktap/device.c +44 -10
@@ -103,6 +103,42 @@ static struct block_device_operations blktap_device_file_operations = { .getgeo = blktap_device_getgeo }; +/* NB. __blktap holding the queue lock; blktap where unlocked */ + +static inline struct request * +__blktap_next_queued_rq (struct request_queue *q) +{ +return elv_next_request (q); +} + +static inline void +__blktap_dequeue_rq (struct request *rq) +{ +blkdev_dequeue_request (rq); +} + +/* NB. err == 0 indicates success, failures < 0 */ + +static inline void +__blktap_end_queued_rq (struct request *rq,int err) +{ +__blk_end_request (rq, err,blk_rq_bytes (rq)); +} + +static inline void +__blktap_end_rq (struct request *rq,int err) +{ +__blk_end_request (rq, err,blk_rq_bytes (rq)); +} + +static inline void +blktap_end_rq (struct request *rq,int err) +{ +spin_lock_irq (rq->q->queue_lock); +__blktap_end_rq (rq, err); +spin_unlock_irq (rq->q->queue_lock); +} + void blktap_device_end_request(struct blktap *tap, struct blktap_request *request,
@@ -119,9 +155,7 @@ blktap_device_end_request(struct blktap *tap,"end_request: op=%d error=%d bytes=%d\n" , rq_data_dir(rq), error, blk_rq_bytes(rq)); -spin_lock_irq (&tapdev->lock); -end_request (request->rq, !error); -spin_unlock_irq (&tapdev->lock); +blktap_end_rq (rq, error); }int
@@ -201,17 +235,17 @@ blktap_device_run_queue(struct blktap *tap)queue_flag_clear (QUEUE_FLAG_STOPPED , q); do { - rq = elv_next_request(q); + rq = __blktap_next_queued_rq(q); if (!rq) break; if (!blk_fs_request(rq)) { - end_queued_request(rq, 0); + __blktap_end_queued_rq(rq, -EOPNOTSUPP); continue; } if (blk_empty_barrier(rq)) { - end_queued_request(rq, 1); + __blktap_end_queued_rq(rq, 0); continue; }
@@ -226,10 +260,10 @@ blktap_device_run_queue(struct blktap *tap)break ; } -blkdev_dequeue_request (req); +__blktap_dequeue_rq (rq);if (unlikely (err)) -end_request (rq,0 ); +__blktap_end_rq (rq, err); }while (1 );spin_unlock_irq (&tapdev->lock);
@@ -365,11 +399,11 @@ blktap_device_fail_queue(struct blktap *tap)queue_flag_clear (QUEUE_FLAG_STOPPED , q);do { -struct request *rq =elv_next_request (q); +struct request *rq =__blktap_next_queued_rq (q);if (!rq)break ; -end_request (rq, -EIO ); +__blktap_end_queued_rq (rq, -EIO ); }while (1 );spin_unlock_irq (&tapdev->lock);
Signature · 1 line
1.7.0.4
PATCH 4/5 blktap: Forward port to 2.6.32
In reply to Daniel StoddenSome dust off the topic branch.
--- drivers/xen/blktap/device.c | 22 +++++++++------------- drivers/xen/blktap/ring.c | 2 +- 2 files changed, 10 insertions(+), 14 deletions(-)
drivers/xen/blktap/device.c +9 -13
@@ -108,13 +108,13 @@ static struct block_device_operations blktap_device_file_operations = {static inline struct request *__blktap_next_queued_rq (struct request_queue *q) { -return elv_next_request (q); +return blk_peek_request (q); }static inline void __blktap_dequeue_rq (struct request *rq) { -blkdev_dequeue_request (rq); +blk_start_request (rq); }/* NB. err == 0 indicates success, failures < 0 */
@@ -122,6 +122,7 @@ __blktap_dequeue_rq(struct request *rq)static inline void __blktap_end_queued_rq (struct request *rq,int err) { +blk_start_request (rq);__blk_end_request (rq, err,blk_rq_bytes (rq)); }
@@ -151,7 +152,7 @@ blktap_device_end_request(struct blktap *tap,blktap_ring_free_request (tap, request); -dev_dbg (&tapdev->gd->dev, +dev_dbg (disk_to_dev (tapdev->gd),"end_request: op=%d error=%d bytes=%d\n" ,rq_data_dir (rq), error,blk_rq_bytes (rq));
@@ -180,7 +181,7 @@ blktap_device_make_request(struct blktap *tap, struct request *rq) write =rq_data_dir (rq) ==WRITE ; nsegs =blk_rq_map_sg (rq->q, rq, request->sg_table); -dev_dbg (&tapdev->gd->dev, +dev_dbg (disk_to_dev (tapdev->gd),"make_request: op=%c bytes=%d nsegs=%d\n" , write ?'w' :'r' ,blk_rq_bytes (rq), nsegs);
@@ -210,7 +211,7 @@ _out:return err; fail:if (printk_ratelimit ()) -dev_warn (&tapdev->gd->dev, +dev_warn (disk_to_dev (tapdev->gd),"make request: %d, failing\n" , err); goto _out; }
@@ -244,11 +245,6 @@ blktap_device_run_queue(struct blktap *tap)continue ; } -if (blk_empty_barrier (rq)) { -__blktap_end_queued_rq (rq,0 ); -continue ; - } -spin_unlock_irq (&tapdev->lock); err =blktap_device_make_request (tap, rq);
@@ -491,8 +487,8 @@ blktap_device_create(struct blktap *tap, struct blktap_params *params)set_bit (BLKTAP_DEVICE , &tap->dev_inuse); -dev_info (&gd->dev,"sector-size: %u capacity: %llu\n" , - rq->hardsect_size,get_capacity (gd)); +dev_info (disk_to_dev (gd),"sector-size: %u capacity: %llu\n" , +queue_logical_block_size (rq),get_capacity (gd));return 0 ;
@@ -520,7 +516,7 @@ blktap_device_debug(struct blktap *tap, char *buf, size_t size) s +=snprintf (s, end - s,"disk capacity:%llu sector size:%u\n" , -get_capacity (disk),queue_hardsect_size (q)); +get_capacity (disk),queue_logical_block_size (q)); s += snprintf(s, end - s, "queue flags:%#lx plugged:%d stopped:%d empty:%d\n",
drivers/xen/blktap/ring.c +1 -1
@@ -247,7 +247,7 @@ blktap_ring_submit_request(struct blktap *tap, breq =RING_GET_REQUEST (&ring->ring, ring->ring.req_prod_pvt); breq->id = request->usr_idx; - breq->sector_number = request->rq->sector; + breq->sector_number =blk_rq_pos (request->rq); breq->handle =0 ; breq->operation = request->operation; breq->nr_segments = request->nr_pages;
Signature · 1 line
1.7.0.4
PATCH 5/5 Fix compilation format warning in drivers/xen/blktap/device.c
In reply to Daniel StoddenFrom: Giam Teck Choon <[email protected]>
drivers/xen/blktap/device.c: In function ‘blktap_device_create’:drivers/xen/blktap/device.c:869: warning: format ‘%llu’ expects type ‘long long unsigned int’, but argument 5 has type ‘sector_t’ drivers/xen/blktap/device.c: In function ‘blktap_device_debug’:drivers/xen/blktap/device.c:898: warning: format ‘%llu’ expects type ‘long long unsigned int’, but argument 4 has type ‘sector_t’
--- drivers/xen/blktap/device.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-)
drivers/xen/blktap/device.c +4 -2
@@ -488,7 +488,8 @@ blktap_device_create(struct blktap *tap, struct blktap_params *params)set_bit (BLKTAP_DEVICE , &tap->dev_inuse);dev_info (disk_to_dev (gd),"sector-size: %u capacity: %llu\n" , -queue_logical_block_size (rq),get_capacity (gd)); +queue_logical_block_size (rq), + (unsigned long long )get_capacity (gd));return 0 ;
@@ -516,7 +517,8 @@ blktap_device_debug(struct blktap *tap, char *buf, size_t size) s +=snprintf (s, end - s,"disk capacity:%llu sector size:%u\n" , -get_capacity (disk),queue_logical_block_size (q)); + (unsigned long long )get_capacity (disk), +queue_logical_block_size (q)); s += snprintf(s, end - s, "queue flags:%#lx plugged:%d stopped:%d empty:%d\n",
Signature · 1 line
1.7.0.4
Quoted 6 lines · Daniel Stodden
It's fairly a big change in how I/O buffers are managed. Prior to this series, we had zero-copy I/O down to userspace. Unfortunately, blktap2 always had to jump through a couple of extra loops to do so. Present state of that is that we dropped that, so all tapdev I/O is bounced to/from a bunch of normal pages. Essentially replacing the old VMA management with a couple insert/zap VM calls.
Do you have any performance results comparing the two approaches?
Quoted 10 lines
One issue was that the kernel can't cope with recursive I/O. Submitting an iovec on a tapdev, passing it to userspace and then reissuing the same vector via AIO apparently doesn't fit well with the lock protocol applied to those pages. This is the main reason why blktap had to deal a lot with grant refs. About as much as blkback already does before passing requests on. What happens there is that it's aliasing those granted pages under a different PFN, thereby in a separate page struct. Not pretty, but it worked, so it's not the reason why we chose to drop that at some point.
Quoted 15 lines
The more prevalent problem was network storage, especially anything involving TCP. That includes VHD on both NFS and iSCSI. The problem with those is that retransmits (by the transport) and I/O op completion (on the application layer) are never synchronized. With sufficiently bad timing and bit of jitter on the network, it's perfectly common for the kernel to complete an AIO request with a late ack on the input queue just when retransmission timer is about to fire underneath. The completion will unmap the granted frame, crashing any uncanceled retransmission on an empty page frame. There are different ways to deal with that. Page destructors might be one way, but as far as I heard they are not particularly popular upstream. Issuing the block I/O on dom0 memory is straightforward and avoids the hassle. One could go argue that retransmits after DIO completion are still a potential privacy problem (I did), but it's not Xen's problem after all.
Surely this can be dealt with by replacing the mapped granted page with a local copy if the refcount is elevated? Then that can catch any stray residual references while we can still return the granted page to its owner. And obviously, not reuse that pfn for grants until the refcount is zero...
Quoted 6 lines
If zero-copy becomes more attractive again, the plan would be to rather use grantdev in userspace, such as a filter driver for tapdisk instead. Until then, there's presumably a notable difference in L2 cache footprint. Then again, there's also a whole number of cycles not spent in redundant hypercalls now, to deal with the pseudophysical map.
Frankly, I think the idea of putting blkback+tapdisk entirely in usermode is all upside with no (obvious) downsides. It:
1. avoids having to upstream anything
2. avoids having to upstream anything
3. avoids having to upstream anything
4. gets us back zero-copy (if that's important)
5. makes the IO path nice and straightforward
6. seems to address all the other problems you mentioned
The only caveat is the stray unmapping problem, but I think gntdev can be modified to deal with that pretty easily.
qemu has usermode blkback support already, and an actively improving block-IO infrastructure, so one approach might be to consider putting (parts of) tapdisk into qemu - and makes it pretty natural to reuse it with non-Xen guests via virtio-block, emulated devices, etc. But I'm not sold on that; having a standalone tapdisk w/ blkback makes sense to me as well.
On the other hand, I don't think we're going to be able to get away with putting netback in usermode, so we still need to deal with that - but I think an all-copying version will be fine to get started with at least.
Quoted 25 lines
There are also benefits or non-issues.
- This blktap is rather xen-independent. Certainly depends on the common ring macros, but lacking grant stuff it compiles on bare metal Linux with no
CONFIG_XEN. Not consummated here, because that's going to move the source tree out ofdrivers/xen. But I'd like to post a new branch proposing to do so.- Blktaps size in dom0 didn't really change. Frames (now pages) were always pooled. We used to balloon memory to claim space for redundant grant mappings. Now we reserve, by default, the same volume in normal memory.
- The previous code would runs all I/O on a single pool. Typically two rings worth of requests. Sufficient for a whole lot of systems, especially with single storage backends, but not so nice when I/O on a number of otherwise independent filers or volumes collides.
Pools are refcounted kobjects in sysfs. Toolstacks using the new code can thereby choose to elimitate bottlenecks by grouping taps on different buffer pools. Pools can also be resized, to accomodate greater queue depths. [Note that blkback still has the same issue, so guests won't take advantage of that before that's resolved as well.]
Quoted 7 lines
- XCP started to make some use of stacking tapdevs. Think pointing the image chain of a bunch of "leaf" taps to a shared parent node. That works fairly well, but definitely takes independent resource pools to avoid deadlock by parent starvation then.
Please pull upstream/xen/dom0/backend/blktap2 from
git://xenbits.xensource.com/people/dstodden/linux.git
OK, I've pulled it, but I haven't had a chance to test it yet.
Thanks,
J
Hey.
Quoted 9 lines · Jeremy Fitzhardinge
Quoted 6 lines · Daniel Stodden
It's fairly a big change in how I/O buffers are managed. Prior to this series, we had zero-copy I/O down to userspace. Unfortunately, blktap2 always had to jump through a couple of extra loops to do so. Present state of that is that we dropped that, so all tapdev I/O is bounced to/from a bunch of normal pages. Essentially replacing the old VMA management with a couple insert/zap VM calls.
Do you have any performance results comparing the two approaches?
No. One could probably go try large ramdisks or an AIO backend in tmpfs. All the storage I'm concerned with here terminates either on the NIC or a local spindle. That's hard to thwart in cache bandwidth.
Quoted 26 lines
Quoted 10 lines
One issue was that the kernel can't cope with recursive I/O. Submitting an iovec on a tapdev, passing it to userspace and then reissuing the same vector via AIO apparently doesn't fit well with the lock protocol applied to those pages. This is the main reason why blktap had to deal a lot with grant refs. About as much as blkback already does before passing requests on. What happens there is that it's aliasing those granted pages under a different PFN, thereby in a separate page struct. Not pretty, but it worked, so it's not the reason why we chose to drop that at some point.
Quoted 15 lines
The more prevalent problem was network storage, especially anything involving TCP. That includes VHD on both NFS and iSCSI. The problem with those is that retransmits (by the transport) and I/O op completion (on the application layer) are never synchronized. With sufficiently bad timing and bit of jitter on the network, it's perfectly common for the kernel to complete an AIO request with a late ack on the input queue just when retransmission timer is about to fire underneath. The completion will unmap the granted frame, crashing any uncanceled retransmission on an empty page frame. There are different ways to deal with that. Page destructors might be one way, but as far as I heard they are not particularly popular upstream. Issuing the block I/O on dom0 memory is straightforward and avoids the hassle. One could go argue that retransmits after DIO completion are still a potential privacy problem (I did), but it's not Xen's problem after all.
Quoted 2 lines
Surely this can be dealt with by replacing the mapped granted page with a local copy if the refcount is elevated?
Yeah. We briefly discussed this when the problem started to pop up (again).
I had a patch, for blktap1 in XS 5.5 iirc, which would fill mapping with a dummy page mapped in. You wouldn't need a copy, a R/O zero map easily does the job. On UP that'd be just a matter of disabling interrupts for a while.
I dropped it after it became clear that XS was moving to SMP, where one would end up with a full barrier to orchestrate the TLB flushes everywhere. Now, the skb runs prone to crash all run in softirq context, I wouldn't exactly expect a huge performance win from syncing on that kind of thing across all nodes, compared to local memcpy. Nor would I want storage stuff to touch locks shared with TCP, that's just not our business. Correct me if I'm mistaken.
I'd like to see maybe stuff like node affinity on NUMA getting a bit more work. I think the patch presently just fills the queue node in, but that didn't see much testing, and one would have to correlate that.
Quoted 12 lines
Then that can catch any stray
residual references while we can still return the granted page to its owner. And obviously, not reuse that pfn for grants until the refcount is zero...Quoted 6 lines
If zero-copy becomes more attractive again, the plan would be to rather use grantdev in userspace, such as a filter driver for tapdisk instead. Until then, there's presumably a notable difference in L2 cache footprint. Then again, there's also a whole number of cycles not spent in redundant hypercalls now, to deal with the pseudophysical map.
Quoted 8 lines
Frankly, I think the idea of putting blkback+tapdisk entirely in usermode is all upside with no (obvious) downsides. It:
1. avoids having to upstream anything
2. avoids having to upstream anything
3. avoids having to upstream anything4. gets us back zero-copy (if that's important)
(No, unfortunately. DIO on a granted frame under blktap would be as vulnerable as DIO on a granted frame under a userland blkback, userland won't buy us anthing as far as the zcopy side of things is concerned).
Quoted 2 lines
5. makes the IO path nice and straightforward
6. seems to address all the other problems you mentioned
I'm not at all against a userland blkback. Just wouldn't go as far as considering this a silver bullet.
The main thing I'm scared of is ending up hacking cheesy stuff into the user ABI to take advantage of things immediately available to FSes on the bio layer, but harder (or at least slower) to get made available to userland.
DISCARD support is one currently hot example, do you see that in AIO somewhere? Ok, probably a good thing for everybody anyway, so maybe patching that is useful work. But it's extra work right now and probably no more fun to maintain than blktap is.
The second issue I see is the XCP side of things. XenServer got a lot of benefit out of blktap2, and particularly because of the tapdevs. It promotes a fairly rigorous split between a blkback VBD, controlled by the agent, and tapdevs, controlled by XS's storage manager.
That doesn't prevent blkback to go into userspace, but it better won't share a process with some libblktap, which in turn would better not be controlled under the same xenstore path.
So for XCP it'd be AIO on tapdevs for the time being, and with that whatever the syscall interface lets you do.
Quoted 2 lines
The only caveat is the stray unmapping problem, but I think gntdev can be modified to deal with that pretty easily.
Not easier than anything else in kernel space, but when dealing only with the refcounts, that's as as good a place as anwhere else, yes.
Quoted 7 lines
qemu has usermode blkback support already, and an actively improving block-IO infrastructure, so one approach might be to consider putting (parts of) tapdisk into qemu - and makes it pretty natural to reuse it with non-Xen guests via virtio-block, emulated devices, etc. But I'm not sold on that; having a standalone tapdisk w/ blkback makes sense to me as well.
Quoted 8 lines
On the other hand, I don't think we're going to be able to get away with putting netback in usermode, so we still need to deal with that - but I think an all-copying version will be fine to get started with at least.
Quoted 2 lines
Please pull upstream/xen/dom0/backend/blktap2 from
git://xenbits.xensource.com/people/dstodden/linux.gitOK, I've pulled it, but I haven't had a chance to test it yet.
Thanks.
Daniel
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines · Daniel Stodden
Quoted 2 lines
Surely this can be dealt with by replacing the mapped granted page with a local copy if the refcount is elevated?
Yeah. We briefly discussed this when the problem started to pop up (again).
I had a patch, for blktap1 in XS 5.5 iirc, which would fill mapping with a dummy page mapped in. You wouldn't need a copy, a R/O zero map easily does the job.
Hm, I'd be a bit concerned that that might cause problems if used generically. If the page is being used RO, then replacing with a copy shouldn't be a problem, but getting a consistent snapshot of a mutable page is obviously going to be a problem.
Quoted 2 lines
On UP that'd be just a matter of disabling interrupts for
a while.
Disable for what purpose? You mean to do an exchange of the mapping, or something else?
Quoted 7 lines
I dropped it after it became clear that XS was moving to SMP, where one would end up with a full barrier to orchestrate the TLB flushes everywhere. Now, the skb runs prone to crash all run in softirq context, I wouldn't exactly expect a huge performance win from syncing on that kind of thing across all nodes, compared to local memcpy. Nor would I want storage stuff to touch locks shared with TCP, that's just not our business. Correct me if I'm mistaken.
I don't follow what your high-level concern is here. If we update the
pte to unmap the granted page, then its up to Xen to arrange for any TLB flushes to make sure the page is no longer accessible to the domain. We don't need to do anything explicit, and its independent of whether we're simply unmapping the page or replacing the mapping with another one (ie, any TLB flushing necessary is already going on).
I was also under the impression that this is a relatively rare event rather than something that's going to be necessary for every page, so the overhead should be minimal.
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
If zero-copy becomes more attractive again, the plan would be to rather use grantdev in userspace, such as a filter driver for tapdisk instead. Until then, there's presumably a notable difference in L2 cache footprint. Then again, there's also a whole number of cycles not spent in redundant hypercalls now, to deal with the pseudophysical map.
Frankly, I think the idea of putting blkback+tapdisk entirely in usermode is all upside with no (obvious) downsides. It:
Quoted 8 lines
Quoted 5 lines
1. avoids having to upstream anything
2. avoids having to upstream anything
3. avoids having to upstream anything4. gets us back zero-copy (if that's important)
(No, unfortunately. DIO on a granted frame under blktap would be as vulnerable as DIO on a granted frame under a userland blkback, userland won't buy us anthing as far as the zcopy side of things is concerned).
Why's that? Are you talking about the stray page reference vulnerability, or something else? You're right that it doesn't really help with stray references because its still kernel code which ends up doing the dereference rather than usermode.
Quoted 9 lines
Quoted 2 lines
5. makes the IO path nice and straightforward
6. seems to address all the other problems you mentionedI'm not at all against a userland blkback. Just wouldn't go as far as considering this a silver bullet.
The main thing I'm scared of is ending up hacking cheesy stuff into the user ABI to take advantage of things immediately available to FSes on the bio layer, but harder (or at least slower) to get made available to userland.
But that hasn't been a problem for tapdisk so far. Given that it is using DIO then any completed write has at least been submitted to a storage device, so then its just a question of how to make sure that any buffers are fully flushed, which would be fdatasync() I guess.
Besides, our requirements are hardly unique; if there's some clear need for a new API, then the course of action is to work with broader Linux community to work out what it should be and how it should be implemented. There's no need for "hacking cheesy stuff" - there's been enough of that already.
Quoted 4 lines
DISCARD support is one currently hot example, do you see that in AIO somewhere? Ok, probably a good thing for everybody anyway, so maybe patching that is useful work. But it's extra work right now and probably no more fun to maintain than blktap is.
fallocate() is being extended to allow hole-punching in files. I don't know what work is being done to do discard on random block devices, but that's clearly a generically useful thing to have.
Quoted 8 lines
The second issue I see is the XCP side of things. XenServer got a lot of benefit out of blktap2, and particularly because of the tapdevs. It promotes a fairly rigorous split between a blkback VBD, controlled by the agent, and tapdevs, controlled by XS's storage manager.
That doesn't prevent blkback to go into userspace, but it better won't share a process with some libblktap, which in turn would better not be controlled under the same xenstore path.
Could you elaborate on this? What was the benefit?
Quoted 4 lines
Quoted 2 lines
The only caveat is the stray unmapping problem, but I think gntdev can be modified to deal with that pretty easily.
Not easier than anything else in kernel space, but when dealing only with the refcounts, that's as as good a place as anwhere else, yes.
I think the refcount test is pretty straightforward - if the refcount is 1, then we're the sole owner of the page and we don't need to worry about any other users. If its > 1, then somebody else has it, and we need to make sure it no longer refers to a granted page (which is just a matter of doing a set_pte_atomic() to remap from present to present).
Then we'd have a set of frames whose lifetimes are being determined by some other subsystem. We can either maintain a list of them and poll waiting for them to become free, or just release them and let them be managed by the normal kernel lifetime rules (which requires that the memory attached to them be completely normal, of course).
J
Quoted 5 lines · Jeremy Fitzhardinge
Then we'd have a set of frames whose lifetimes are being determined by some other subsystem. We can either maintain a list of them and poll waiting for them to become free, or just release them and let them be managed by the normal kernel lifetime rules (which requires that the memory attached to them be completely normal, of course).
GNTTABOP_unmap_and_replace is probably the answer to this? This is what netback uses (via gnttab_copy_grant_page) when it wants to copy and skb which has remained in flight for too long.
Maybe this doesn't work since the memcpy and replace are not atomic and we don't have a lock to use, since we don't know which subsystem holds the reference. However it should work if we don't really care about the contents of the replacement too much, which I don't think we do in this case since the data could just as easily get clobbered by the userspace process which thinks the write has completed fully, in which case we just replace the grant mapping without bothering with the copy.
Ian.
Quoted 10 lines · Ian Campbell
Quoted 5 lines · Jeremy Fitzhardinge
Then we'd have a set of frames whose lifetimes are being determined by some other subsystem. We can either maintain a list of them and poll waiting for them to become free, or just release them and let them be managed by the normal kernel lifetime rules (which requires that the memory attached to them be completely normal, of course).
GNTTABOP_unmap_and_replaceis probably the answer to this? This is what netback uses (viagnttab_copy_grant_page) when it wants to copy and skb which has remained in flight for too long.
Quoted 7 lines
Maybe this doesn't work since the memcpy and replace are not atomic and we don't have a lock to use, since we don't know which subsystem holds the reference. However it should work if we don't really care about the contents of the replacement too much, which I don't think we do in this case since the data could just as easily get clobbered by the userspace process which thinks the write has completed fully, in which case we just replace the grant mapping without bothering with the copy.
Well, if atomicity is important (and it is a relatively rare event), I dare say Xen could de-schedule all the VCPUs a lot more efficiently than we could run stop_machine() from within the kernel...
J
Quoted 19 lines · Jeremy Fitzhardinge
Quoted 10 lines · Ian Campbell
Quoted 5 lines · Jeremy Fitzhardinge
Then we'd have a set of frames whose lifetimes are being determined by some other subsystem. We can either maintain a list of them and poll waiting for them to become free, or just release them and let them be managed by the normal kernel lifetime rules (which requires that the memory attached to them be completely normal, of course).
GNTTABOP_unmap_and_replaceis probably the answer to this? This is what netback uses (viagnttab_copy_grant_page) when it wants to copy and skb which has remained in flight for too long.Quoted 7 lines
Maybe this doesn't work since the memcpy and replace are not atomic and we don't have a lock to use, since we don't know which subsystem holds the reference. However it should work if we don't really care about the contents of the replacement too much, which I don't think we do in this case since the data could just as easily get clobbered by the userspace process which thinks the write has completed fully, in which case we just replace the grant mapping without bothering with the copy.
Quoted 3 lines
Well, if atomicity is important (and it is a relatively rare event), I dare say Xen could de-schedule all the VCPUs a lot more efficiently than we could run
stop_machine()from within the kernel...
Absolutely.
GNTTABOP_unmap_and_replace is itself atomic according to the comment in the header, it's not clear what it is atomic WRT, presumably just other gnttab or mmu ops. In any case it doesn't handle synchronising the content of the old and new pages so if we care about that we would need a new GNTAPOP_copy_unmap_and_replace or something like that.
Ian.
Quoted 24 lines · Ian Campbell
Quoted 19 lines · Jeremy Fitzhardinge
Quoted 10 lines · Ian Campbell
Quoted 5 lines · Jeremy Fitzhardinge
Then we'd have a set of frames whose lifetimes are being determined by some other subsystem. We can either maintain a list of them and poll waiting for them to become free, or just release them and let them be managed by the normal kernel lifetime rules (which requires that the memory attached to them be completely normal, of course).
GNTTABOP_unmap_and_replaceis probably the answer to this? This is what netback uses (viagnttab_copy_grant_page) when it wants to copy and skb which has remained in flight for too long.Quoted 7 lines
Maybe this doesn't work since the memcpy and replace are not atomic and we don't have a lock to use, since we don't know which subsystem holds the reference. However it should work if we don't really care about the contents of the replacement too much, which I don't think we do in this case since the data could just as easily get clobbered by the userspace process which thinks the write has completed fully, in which case we just replace the grant mapping without bothering with the copy.
Quoted 3 lines
Well, if atomicity is important (and it is a relatively rare event), I dare say Xen could de-schedule all the VCPUs a lot more efficiently than we could run
stop_machine()from within the kernel...
Quoted 7 lines
Absolutely.
GNTTABOP_unmap_and_replaceis itself atomic according to the comment in the header, it's not clear what it is atomic WRT, presumably just other gnttab or mmu ops. In any case it doesn't handle synchronising the content of the old and new pages so if we care about that we would need a newGNTAPOP_copy_unmap_and_replaceor something like that.
Ah, nice. First time I hear we have a dedicated call for that. So far I was assuming dom0 would have to unmap/remap and doing so atomically across cores would be left as an exercise to dom0.
One would need a copy for protocols where it isn't clear that the dup gets dropped on the receiving side. So I guess the main hazard would be writes on sth like NFS over UDP.
In that case, I think blktap has a pretty safe spot to copy data between I/O completion from userland and replacing the page, without extra atomicity.
Daniel
Quoted 6 lines · Jeremy Fitzhardinge
Quoted 5 lines · Daniel Stodden
Quoted 2 lines
Surely this can be dealt with by replacing the mapped granted page with a local copy if the refcount is elevated?
Yeah. We briefly discussed this when the problem started to pop up (again).
Quoted 6 lines
Quoted 3 lines
I had a patch, for blktap1 in XS 5.5 iirc, which would fill mapping with a dummy page mapped in. You wouldn't need a copy, a R/O zero map easily does the job.
Hm, I'd be a bit concerned that that might cause problems if used generically.
Yeah. It wasn't a problem because all the network backends are on TCP, where one can be rather sure that the dups are going to be properly dropped.
Does this hold everywhere ..? -- As mentioned below, the problem is rather in AIO/DIO than being Xen-specific, so you can see the same behavior on bare metal kernels too. A userspace app seeing an AIO complete and then reusing that buffer elsewhere will occassionally resend garbage over the network.
How often that would happen in practice probably depends on the popularity of DIO on NFS for normal apps. Not too many, so yeah, very generically one would maybe want to memcpy().
Quoted 3 lines
If the page is being used RO, then replacing with a copy
shouldn't be a problem, but getting a consistent snapshot of a mutable page is obviously going to be a problem.
I think it's safe to assume the problem is limited to writes, so the buffers aren't really mutable.
Quoted 5 lines
Quoted 2 lines
On UP that'd be just a matter of disabling interrupts for
a while.Disable for what purpose? You mean to do an exchange of the mapping, or something else?
[I was referring to the prospect of doing a gref unmap/map sequence in dom0. Since we do have a swap operation in Xen, just asking flipping the PTE entry and flushing across all vcpus sounds sufficient to me, indeed.]
Quoted 33 lines
Quoted 7 lines
I dropped it after it became clear that XS was moving to SMP, where one would end up with a full barrier to orchestrate the TLB flushes everywhere. Now, the skb runs prone to crash all run in softirq context, I wouldn't exactly expect a huge performance win from syncing on that kind of thing across all nodes, compared to local memcpy. Nor would I want storage stuff to touch locks shared with TCP, that's just not our business. Correct me if I'm mistaken.
I don't follow what your high-level concern is here. If we update the
pte to unmap the granted page, then its up to Xen to arrange for any TLB flushes to make sure the page is no longer accessible to the domain. We don't need to do anything explicit, and its independent of whether we're simply unmapping the page or replacing the mapping with another one (ie, any TLB flushing necessary is already going on).I was also under the impression that this is a relatively rare event rather than something that's going to be necessary for every page, so the overhead should be minimal.
Quoted 14 lines
Quoted 14 lines
Quoted 6 lines
If zero-copy becomes more attractive again, the plan would be to rather use grantdev in userspace, such as a filter driver for tapdisk instead. Until then, there's presumably a notable difference in L2 cache footprint. Then again, there's also a whole number of cycles not spent in redundant hypercalls now, to deal with the pseudophysical map.
Frankly, I think the idea of putting blkback+tapdisk entirely in usermode is all upside with no (obvious) downsides. It:
1. avoids having to upstream anything
2. avoids having to upstream anything
3. avoids having to upstream anything4. gets us back zero-copy (if that's important)
Quoted 8 lines
Quoted 3 lines
(No, unfortunately. DIO on a granted frame under blktap would be as vulnerable as DIO on a granted frame under a userland blkback, userland won't buy us anthing as far as the zcopy side of things is concerned).
Why's that? Are you talking about the stray page reference vulnerability, or something else? You're right that it doesn't really help with stray references because its still kernel code which ends up doing the dereference rather than usermode.
[Still was talking about the stray reference issues].
Quoted 21 lines
Quoted 9 lines
Quoted 2 lines
5. makes the IO path nice and straightforward
6. seems to address all the other problems you mentionedI'm not at all against a userland blkback. Just wouldn't go as far as considering this a silver bullet.
The main thing I'm scared of is ending up hacking cheesy stuff into the user ABI to take advantage of things immediately available to FSes on the bio layer, but harder (or at least slower) to get made available to userland.
But that hasn't been a problem for tapdisk so far. Given that it is using DIO then any completed write has at least been submitted to a storage device, so then its just a question of how to make sure that any buffers are fully flushed, which would be
fdatasync()I guess.Besides, our requirements are hardly unique; if there's some clear need for a new API, then the course of action is to work with broader Linux community to work out what it should be and how it should be implemented. There's no need for "hacking cheesy stuff" - there's been enough of that already.
Quoted 8 lines
Quoted 4 lines
DISCARD support is one currently hot example, do you see that in AIO somewhere? Ok, probably a good thing for everybody anyway, so maybe patching that is useful work. But it's extra work right now and probably no more fun to maintain than blktap is.
fallocate()is being extended to allow hole-punching in files. I don't know what work is being done to do discard on random block devices, but that's clearly a generically useful thing to have.
Hole punching for userland being in the works is actually great news. Just saw the patchset on ext4-devel flying by, great.
Quoted 11 lines
Quoted 8 lines
The second issue I see is the XCP side of things. XenServer got a lot of benefit out of blktap2, and particularly because of the tapdevs. It promotes a fairly rigorous split between a blkback VBD, controlled by the agent, and tapdevs, controlled by XS's storage manager.
That doesn't prevent blkback to go into userspace, but it better won't share a process with some libblktap, which in turn would better not be controlled under the same xenstore path.
Could you elaborate on this? What was the benefit?
It's been mainly a matter of who controls what. Blktap1 was basically a VBD, controlled by the agent. Blktap2 is a VDI represented as a block device. Leaving management of that to XCP's storage manager, which just hands that device node over to Xapi simplified many things. Before, the agent had to understand a lot about the type of storage, then talk to the right backend accordingly. Worse, in order to have storage management control a couple datapath features, you'd basically have to talk to Xapi, which would talk though xenstore to blktap, which was a bit tedious. :)
Merging the VDI side of things with VBDs again just didn't sound so attrative at first sight.
But I've thinking a little longer about the issues in the meantime. I think it's actually not so hard to do that without collateral damage.
Let's say we create an extension to tapdisk which speaks blkback's datapath in userland. We'd basically put one of those tapdisks on every storage node, independent of the image type, such as a bare LUN or a VHD. We add a couple additional IPC calls to make it directly connect/disconnect to/from (ring-ref,event-channel) pairs.
Means it doesn't even need to talk xenstore, the control plane could all be left to some single daemon, which knows how to instruct the right tapdev (via libblktapctl) by looking at the physical-device node. I guess getting the control stuff out of the kernel is always a good idea.
There are some important parts which would go missing. Such as ratelimiting gntdev accesses -- 200 thundering tapdisks each trying to gntmap 352 pages simultaneously isn't so good, so there still needs to be some bridge arbitrating them. I'd rather keep that in kernel space, okay to cram stuff like that into gntdev? It'd be much more straightforward than IPC.
Also, I was absolutely certain I once saw VM_FOREIGN support in gntdev.. Can't find it now, what happened? Without, there's presently still no zero-copy.
Once the issues were solved, it'd be kinda nice. Simplifies stuff like memshr for blktap, which depends on getting hold of original grefs.
We'd presumably still need the tapdev nodes, for qemu, etc. But those can stay non-xen aware then.
Quoted 10 lines
Quoted 4 lines
Quoted 2 lines
The only caveat is the stray unmapping problem, but I think gntdev can be modified to deal with that pretty easily.
Not easier than anything else in kernel space, but when dealing only with the refcounts, that's as as good a place as anwhere else, yes.
I think the refcount test is pretty straightforward - if the refcount is 1, then we're the sole owner of the page and we don't need to worry about any other users. If its > 1, then somebody else has it, and we need to make sure it no longer refers to a granted page (which is just a matter of doing a
set_pte_atomic()to remap from present to present).
[set_pte_atomic over grant ptes doesn't work, or does it?]
Quoted 5 lines
Then we'd have a set of frames whose lifetimes are being determined by some other subsystem. We can either maintain a list of them and poll waiting for them to become free, or just release them and let them be managed by the normal kernel lifetime rules (which requires that the memory attached to them be completely normal, of course).
The latter sounds like a good alternative to polling. So an unmap_and_replace, and giving up ownership thereafter. Next run of the dispatcher thread can can just refill the foreign pfn range via alloc_empty_pages(), to rebalance.
Daniel
Quoted 22 lines · Daniel Stodden
Let's say we create an extension to tapdisk which speaks blkback's datapath in userland. We'd basically put one of those tapdisks on every storage node, independent of the image type, such as a bare LUN or a VHD. We add a couple additional IPC calls to make it directly connect/disconnect to/from (ring-ref,event-channel) pairs.
Means it doesn't even need to talk xenstore, the control plane could all be left to some single daemon, which knows how to instruct the right tapdev (via libblktapctl) by looking at the physical-device node. I guess getting the control stuff out of the kernel is always a good idea.
There are some important parts which would go missing. Such as ratelimiting gntdev accesses -- 200 thundering tapdisks each trying to gntmap 352 pages simultaneously isn't so good, so there still needs to be some bridge arbitrating them. I'd rather keep that in kernel space, okay to cram stuff like that into gntdev? It'd be much more straightforward than IPC.
Also, I was absolutely certain I once saw
VM_FOREIGNsupport in gntdev.. Can't find it now, what happened? Without, there's presently still no zero-copy.
Quoted 6 lines
Once the issues were solved, it'd be kinda nice. Simplifies stuff like memshr for blktap, which depends on getting hold of original grefs.
We'd presumably still need the tapdev nodes, for qemu, etc. But those can stay non-xen aware then.
Considering that there is a blkback implementation in qemu already, why don't use it? I don't certainly feel the need of yet another blkback implementation.
A lot of people are working on qemu nowadays and this would let us exploit some of that work and contribute to it ourselves.
We would only need to write a vhd block driver in qemu (even though a "vdi" driver is already present, I assume is not actually compatible?) and everything else is already there.
We could reuse their qcow and qcow2 drivers that honestly are better maintained than ours (we receive a bug report per week about qcow/qcow2 not working properly).
Finally qemu needs to be able to do I/O anyway because of the IDE emulation, so it has to be in the picture in a way or another. One day not far from now when we make virtio work on Xen, even the fast PV data path might go through qemu, so we might as well optimize it. After talking to the xapi guys to better understand their requirements, I am pretty sure that the new upstream qemu with QMP support would be able to satisfy them without issues.
Of all the possible solutions, this is certainly the one that requires less lines of code and would allow us to reuse more resource that otherwise would just remain untapped.
I backported the upstream xen_disk implementation to qemu-xen and run a test on the upstream 2.6.37rc1 kernel as dom0: VMs boot fine and performances seem to be interesting. For the moment I am thinking about enabling the qemu blkback implementation as a fallback in case blktap2 is not present in the system (ie: 2.6.37 kernels).
Quoted 3 lines
I backported the upstream
xen_diskimplementation to qemu-xen and run a test on the upstream 2.6.37rc1 kernel as dom0: VMs boot fine and performances seem to be interesting. For the moment I am thinking
Can you quantify "interesting"? As in sucks-but-I-can-live-with-it or wow-it-is-as-fast-as-kernel-blkback? Or should ask this question later when you had a chance to run proper benchmarks?
Quoted 6 lines · Konrad Rzeszutek Wilk
Quoted 3 lines
I backported the upstream
xen_diskimplementation to qemu-xen and run a test on the upstream 2.6.37rc1 kernel as dom0: VMs boot fine and performances seem to be interesting. For the moment I am thinkingCan you quantify "interesting"? As in sucks-but-I-can-live-with-it or wow-it-is-as-fast-as-kernel-blkback?
wow-it-is-as-fast-as-kernel-blkback
Quoted 2 lines
Or should ask this question later
when you had a chance to run proper benchmarks?
that would be better :)
Quoted 52 lines · Stefano Stabellini
Quoted 22 lines · Daniel Stodden
Let's say we create an extension to tapdisk which speaks blkback's datapath in userland. We'd basically put one of those tapdisks on every storage node, independent of the image type, such as a bare LUN or a VHD. We add a couple additional IPC calls to make it directly connect/disconnect to/from (ring-ref,event-channel) pairs.
Means it doesn't even need to talk xenstore, the control plane could all be left to some single daemon, which knows how to instruct the right tapdev (via libblktapctl) by looking at the physical-device node. I guess getting the control stuff out of the kernel is always a good idea.
There are some important parts which would go missing. Such as ratelimiting gntdev accesses -- 200 thundering tapdisks each trying to gntmap 352 pages simultaneously isn't so good, so there still needs to be some bridge arbitrating them. I'd rather keep that in kernel space, okay to cram stuff like that into gntdev? It'd be much more straightforward than IPC.
Also, I was absolutely certain I once saw
VM_FOREIGNsupport in gntdev.. Can't find it now, what happened? Without, there's presently still no zero-copy.Quoted 6 lines
Once the issues were solved, it'd be kinda nice. Simplifies stuff like memshr for blktap, which depends on getting hold of original grefs.
We'd presumably still need the tapdev nodes, for qemu, etc. But those can stay non-xen aware then.
Considering that there is a blkback implementation in qemu already, why don't use it? I don't certainly feel the need of yet another blkback implementation.
A lot of people are working on qemu nowadays and this would let us exploit some of that work and contribute to it ourselves.
We would only need to write a vhd block driver in qemu (even though a "vdi" driver is already present, I assume is not actually compatible?) and everything else is already there.
We could reuse their qcow and qcow2 drivers that honestly are better maintained than ours (we receive a bug report per week about qcow/qcow2 not working properly).
Finally qemu needs to be able to do I/O anyway because of the IDE emulation, so it has to be in the picture in a way or another. One day not far from now when we make virtio work on Xen, even the fast PV data path might go through qemu, so we might as well optimize it. After talking to the xapi guys to better understand their requirements, I am pretty sure that the new upstream qemu with QMP support would be able to satisfy them without issues.
Of all the possible solutions, this is certainly the one that requires less lines of code and would allow us to reuse more resource that otherwise would just remain untapped.
Quoted 5 lines
I backported the upstream
xen_diskimplementation to qemu-xen and run a test on the upstream 2.6.37rc1 kernel as dom0: VMs boot fine and performances seem to be interesting. For the moment I am thinking about enabling the qemu blkback implementation as a fallback in case blktap2 is not present in the system (ie: 2.6.37 kernels).
I'm not against reducing code and effort. But in order to switch to a different base we would need a drop-in match for VHD and at least a good match for all the control machinery on which xen-sm presently depends. There's also a lot of investment in filter drivers etc.
Then there is SM control, stuff like pause/unpause to get guests off the storage nodes for snapshot/coalesce, more recently calls for statistics and monitoring, tweaking some physical I/O details, etc. Used to be a bitch, nowadays it's somewhat simpler, but that's all stuff we completely depend on.
Moving blkback out of kernel space, into tapdisk, is predictable in size and complexity. Replacing tapdisks altogether would be quite a different story.
The remainder below isn't fully qualified, just random bits coming to my mind, assuming you're not talking about sharing code/libs and frameworks, but actual processes.
1st, what's the rationale with fully PV'd guests on xen? (That argument might not count if just taking qemu as the container process and stripping emulation for those.)
Related, there's the question of memory footprint. Kernel blkback is extremely lightweight. Moving the datapath into userland can create headaches, especially on 32bit dom0s with lot of guests and disks on backends which used to be bare LUNs under blkback. That's a problem tapdisk has to face too, just wondering about the size of the issue in qemu.
Related, Xapi depends a lot on dom0 plugs, where the datapath can be somewhat hairy when it comes to blocking I/O and resource allocation.
Then there is sharing. Storage activation normally doesn't operate in a specific VM context. It presently doesn't even relate to a particular VBD, much less a VM. For qemu alone, putting storage virtualization into the same address space is an obvious choice. For Xen, enforcing that sounds like a step backward.
Quoted 1 line
From the shared-framework perspective, and the amount of code involved:
The ring path alone is too small to consider, and the more difficult parts on top of that like state machines for write ordering and syncing etc are hard to share because the depend on the queue implementation and image driver interface.
Control might be a different story. As far as frontend/backend IPC via xenstore goes, right now I still feel like those backends could be managed by a single daemon, similar to what blktapctrl did (let's just make it stateless/restartable this time). I guess qemu processes run their xenstore trees already fine, but internally?
Daniel
Quoted 5 lines · Daniel Stodden
I'm not against reducing code and effort. But in order to switch to a different base we would need a drop-in match for VHD and at least a good match for all the control machinery on which xen-sm presently depends. There's also a lot of investment in filter drivers etc.
I am hoping we don't actually have to rewrite all that code but that we would be able to refactor it to fit the qemu driver APIs.
Quoted 6 lines
Then there is SM control, stuff like pause/unpause to get guests off the storage nodes for snapshot/coalesce, more recently calls for statistics and monitoring, tweaking some physical I/O details, etc. Used to be a bitch, nowadays it's somewhat simpler, but that's all stuff we completely depend on.
Upstream qemu has an RPC interface called QMP, that can be used to issue commands and retrieve informations. For example snapshots are already supported by this interface.
We need to support QMP one way or another, even only for the pure qemu emulation use case, so we might as well exploit it.
Quoted 4 lines
Moving blkback out of kernel space, into tapdisk, is predictable in size and complexity. Replacing tapdisks altogether would be quite a different story.
This is not a one day project, we have time for this.
I was thinking about starting to make use of blkback qemu (with the current qemu-xen) as a fallback when blkback2 is not present, mainly to allow developers to work with upstream 2.6.37.
Meanwhile in the next few months upstream qemu should apply our patches and therefore we could start using upstream qemu for development with xen-unstable. Upstream qemu will offer much better aio support and QMP. At that point we could start adding a VHD driver to upstream qemu and slowly everything else we need. By the time 2.6.38/39 is out we could have a proper backend with VHD support.
What do you think?
Quoted 8 lines
The remainder below isn't fully qualified, just random bits coming to my mind, assuming you're not talking about sharing code/libs and frameworks, but actual processes.
1st, what's the rationale with fully PV'd guests on xen? (That argument might not count if just taking qemu as the container process and stripping emulation for those.)
PV guest needs qemu already for the console and the framebuffer backends, so it just fits in the current picture without modifications.
Quoted 6 lines
Related, there's the question of memory footprint. Kernel blkback is extremely lightweight. Moving the datapath into userland can create headaches, especially on 32bit dom0s with lot of guests and disks on backends which used to be bare LUNs under blkback. That's a problem tapdisk has to face too, just wondering about the size of the issue in qemu.
The PV qemu (the qemu run for PV guests) is very different from the HVM
qemu: it does very little, only runs the backends. I expect its memory footprint to be really small.
Quoted 4 lines
Related, Xapi depends a lot on dom0 plugs, where the datapath can be somewhat hairy when it comes to blocking I/O and resource allocation.
I am not sure what do you mean here.
Quoted 5 lines
Then there is sharing. Storage activation normally doesn't operate in a specific VM context. It presently doesn't even relate to a particular VBD, much less a VM. For qemu alone, putting storage virtualization into the same address space is an obvious choice. For Xen, enforcing that sounds like a step backward.
Having the backend in qemu and running it in a VM context are two different things. Qemu is very flexible in this regard, with one line change (or maybe just different command line options) you can have qemu doing hardware emulation, PV backends, both, or only some PV backends. You could have:
- 1 qemu doing hardware emulation and PV backend;
- 1 qemu doing hardware emulation and another one doing PV backends;
- 1 qemu doing hardware emulation and 1 qemu doing some PV backends and 1 qemu doing the other PV backends;
and so on.
The only thing that cannot be easily split at the moment is the hardware emulation, but the backends are completely modular.
Quoted 7 lines
Quoted 2 lines
From the shared-framework perspective, and the amount of code involved:
The ring path alone is too small to consider, and the more difficult parts on top of that like state machines for write ordering and syncing etc are hard to share because the depend on the queue implementation and image driver interface.
Yeah, if you are thinking about refactoring blktap2 in libraries and use them in both the stand alone tapdisk case and the qemu case, it is probably not worth it.
In any case in qemu they do not depend on the image driver interface because it is generic.
Quoted 5 lines
Control might be a different story. As far as frontend/backend IPC via xenstore goes, right now I still feel like those backends could be managed by a single daemon, similar to what blktapctrl did (let's just make it stateless/restartable this time). I guess qemu processes run their xenstore trees already fine, but internally?
Yes, they do. There is a generic xen_backend interface that adds support for Xen frontend/backend pairs. Using xen_backend each qemu instance listens to its own xenstore backend path.
On 17 Nov 2010, at 12:35, Stefano Stabellini wrote:
On Wed, 17 Nov 2010, Daniel Stodden wrote:
Related, Xapi depends a lot on dom0 plugs, where the datapath can be somewhat hairy when it comes to blocking I/O and resource allocation.
I am not sure what do you mean here.
We would very much like to move away from dom0 block attaches. Currently they're required for (probably non exhaustive list):
- pygrub
- VM import/export
- vdi import/export
- suspend/resume
- Demo Linux VM installation (XS only)
Dave is already working on moving the export code to a domU. The VDI stuff is similar, and suspend resume could be done via plugging into e.g. qemu stub-domains, and pygrub could be eventually replaced by PVgrub.
So it's not inconceivable that XCP might no longer require dom0 block attaches in a similar timescale to that being discussed.
Jon
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Hi,
Re: XCP's use of blktap2:
Quoted 18 lines · Jeremy Fitzhardinge
Quoted 2 lines · Daniel Stodden
Quoted 1 line
The second issue I see is the XCP side of things. XenServer got a
lot of
Quoted 2 lines
Quoted 2 lines
benefit out of blktap2, and particularly because of the tapdevs. It promotes a fairly rigorous split between a blkback VBD, controlled
by
Quoted 3 lines
Quoted 3 lines
the agent, and tapdevs, controlled by XS's storage manager.
That doesn't prevent blkback to go into userspace, but it better
won't
Quoted 1 line
Quoted 1 line
share a process with some libblktap, which in turn would better not
be
Quoted 4 lines
Quoted 1 line
controlled under the same xenstore path.
Could you elaborate on this? What was the benefit?
Quoted 9 lines
It's been mainly a matter of who controls what. Blktap1 was basically a VBD, controlled by the agent. Blktap2 is a VDI represented as a block device. Leaving management of that to XCP's storage manager, which just hands that device node over to Xapi simplified many things. Before, the agent had to understand a lot about the type of storage, then talk to the right backend accordingly. Worse, in order to have storage management control a couple datapath features, you'd basically have to talk to Xapi, which would talk though xenstore to blktap, which was a bit tedious. :)
As Daniel says, XCP currently separates domain management (setting up, rebooting VMs) from storage management (attaching disks, snapshot, coalesce). In the current design the storage layer handles the storage control-path (instigating snapshots, clones, coalesce, dedup in future) through a storage API ("SMAPI") and provides a uniform interface to qemu, blkback for the data-path (currently in the form of a dom0 block device). In a VM start, xapi will first ask the storage control-path to make a disk available, and then pass this information to blkback/qemu.
One of the trickiest things XCP handles is vhd "coalesce": merging a vhd file into its "parent". This comes up because vhds are arranged in a tree structure where the leaves are separate independent VM disks and the nodes represent shared common blocks, the result of (eg) cloning a single VM lots of times. When guest disks are deleted and the vhd leaves are removed, it sometimes becomes possible to save space by merging nodes together. The tricky bit is doing this while I/O is still being performed in parallel against logically separate (but related by parentage/history) disks on different hosts. It's necessary for the thing doing the coalescing to know where all the I/O is going on (eg to be able to find the host and pid where the related tapdisks (or qemus) live) and it's necessary for it to be able to signal to these processes when they need to re-read the vhd tree metadata.
In the bad old blktap1 days, the storage control-path didn't know enough about the data-path to reliably signal the active tapdisks: IIRC the tapdisks were spawned by blktapctrl as a side-effect of the domain manager writing to xenstore. In the much better blktap2 days :) the storage control-path sets up (registers?) the data-path (currently via tap-ctl and a dom0 block device) and so it knows who to talk to in order to co-ordinate a coalesce.
So I think the critical thing is to be able to have the storage control-path able to do something to "register" a data-path, enabling it to find later and signal any processes using that data-path. There are a bunch of different possibilities the storage control-path could use instead of using tap-ctl to create a block device, including:
1. directly spawn a tapdisk2 userspace process. Some identifier (pid, unix domain socket) could be passed to qemu allowing it to perform I/O. The block backend could be either in the tapdisk2 directly or in qemu?
2. return a (path to vhd file, callback unix domain socket). This could be passed to qemu (or something else) and qemu could use the callback socket to register its intention to use the data-path (and hence that it needs to be signaled if something changes)
I'm sure there are lots of possibilities :-)
Cheers,
Dave
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Quoted 32 lines · Dave Scott
Hi,
Re: XCP's use of blktap2:
Quoted 18 lines · Jeremy Fitzhardinge
Quoted 2 lines · Daniel Stodden
Quoted 1 line
The second issue I see is the XCP side of things. XenServer got a
lot of
Quoted 2 lines
Quoted 2 lines
benefit out of blktap2, and particularly because of the tapdevs. It promotes a fairly rigorous split between a blkback VBD, controlled
by
Quoted 3 lines
Quoted 3 lines
the agent, and tapdevs, controlled by XS's storage manager.
That doesn't prevent blkback to go into userspace, but it better
won't
Quoted 1 line
Quoted 1 line
share a process with some libblktap, which in turn would better not
be
Quoted 4 lines
Quoted 1 line
controlled under the same xenstore path.
Could you elaborate on this? What was the benefit?
Quoted 9 lines
It's been mainly a matter of who controls what. Blktap1 was basically a VBD, controlled by the agent. Blktap2 is a VDI represented as a block device. Leaving management of that to XCP's storage manager, which just hands that device node over to Xapi simplified many things. Before, the agent had to understand a lot about the type of storage, then talk to the right backend accordingly. Worse, in order to have storage management control a couple datapath features, you'd basically have to talk to Xapi, which would talk though xenstore to blktap, which was a bit tedious. :)
Quoted 3 lines
As Daniel says, XCP currently separates domain management (setting up, rebooting VMs) from storage management (attaching disks, snapshot, coalesce). In the current design the storage layer handles the storage control-path (instigating snapshots, clones, coalesce, dedup in future) through a storage API ("SMAPI") and provides a uniform interface to qemu, blkback for the data-path (currently in the form of a dom0 block device). In a VM start, xapi will first ask the storage control-path to make a disk available, and then pass this information to blkback/qemu.
One of the trickiest things XCP handles is vhd "coalesce": merging a vhd file into its "parent". This comes up because vhds are arranged in a tree structure where the leaves are separate independent VM disks and the nodes represent shared common blocks, the result of (eg) cloning a single VM lots of times. When guest disks are deleted and the vhd leaves are removed, it sometimes becomes possible to save space by merging nodes together. The tricky bit is doing this while I/O is still being performed in parallel against logically separate (but related by parentage/history) disks on different hosts. It's necessary for the thing doing the coalescing to know where all the I/O is going on (eg to be able to find the host and pid where the related tapdisks (or qemus) live) and it's necessary for
it to be able to signal to these processes when they need to re-read the vhd tree metadata.
Quoted 5 lines
In the bad old blktap1 days, the storage control-path didn't know enough about the data-path to reliably signal the active tapdisks: IIRC the tapdisks were spawned by blktapctrl as a side-effect of the domain manager writing to xenstore. In the much better blktap2 days :) the storage control-path sets up (registers?) the data-path (currently via tap-ctl and a dom0 block device) and so it knows who to talk to in order to co-ordinate a coalesce.
So I think the critical thing is to be able to have the storage control-path able to do something to "register" a data-path, enabling it to find later and signal any processes using that data-path. There are a bunch of different possibilities the storage control-path could use instead of using tap-ctl to create a block device, including:
Qemu could be spawned directly (even before the VM) and QMP could be use to communicate with it.
The qemu pid and/or the socket to issue QMP commands could be used as identifiers.
Quoted 2 lines
I'm sure there are lots of possibilities :-)
Indeed.
Quoted 16 lines · Daniel Stodden
Quoted 11 lines · Jeremy Fitzhardinge
Quoted 8 lines · Daniel Stodden
Quoted 2 lines
Surely this can be dealt with by replacing the mapped granted page with a local copy if the refcount is elevated?
Yeah. We briefly discussed this when the problem started to pop up (again).
I had a patch, for blktap1 in XS 5.5 iirc, which would fill mapping with a dummy page mapped in. You wouldn't need a copy, a R/O zero map easily does the job.
Hm, I'd be a bit concerned that that might cause problems if used generically.
Yeah. It wasn't a problem because all the network backends are on TCP, where one can be rather sure that the dups are going to be properly dropped.
Quoted 5 lines
Does this hold everywhere ..? -- As mentioned below, the problem is rather in AIO/DIO than being Xen-specific, so you can see the same behavior on bare metal kernels too. A userspace app seeing an AIO complete and then reusing that buffer elsewhere will occassionally resend garbage over the network.
Yeah, that sounds like a generic security problem. I presume the protocol will just discard the excess retransmit data, but it might mean a usermode program ends up transmitting secrets it never intended to...
Quoted 6 lines
There are some important parts which would go missing. Such as ratelimiting gntdev accesses -- 200 thundering tapdisks each trying to gntmap 352 pages simultaneously isn't so good, so there still needs to be some bridge arbitrating them. I'd rather keep that in kernel space, okay to cram stuff like that into gntdev? It'd be much more straightforward than IPC.
What's the problem? If you do nothing then it will appear to the kernel as a bunch of processes doing memory allocations, and they'll get blocked/rate-limited accordingly if memory is getting short. There's plenty of existing mechanisms to control that sort of thing (cgroups, etc) without adding anything new to the kernel. Or are you talking about something other than simple memory pressure?
And there's plenty of existing IPC mechanisms if you want them to explicitly coordinate with each other, but I'd tend to thing that's premature unless you have something specific in mind.
Quoted 3 lines
Also, I was absolutely certain I once saw
VM_FOREIGNsupport in gntdev.. Can't find it now, what happened? Without, there's presently still no zero-copy.
gntdev doesn't need VM_FOREIGN any more - it uses the (relatively new-ish) mmu notifier infrastructure which is intended to allow a device to sync an external MMU with usermode mappings. We're not using it in precisely that way, but it allows us to wrangle grant mappings before the generic code tries to do normal pte ops on them.
Quoted 8 lines
Once the issues were solved, it'd be kinda nice. Simplifies stuff like memshr for blktap, which depends on getting hold of original grefs.
We'd presumably still need the tapdev nodes, for qemu, etc. But those can stay non-xen aware then.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The only caveat is the stray unmapping problem, but I think gntdev can be modified to deal with that pretty easily.
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
Not easier than anything else in kernel space, but when dealing only with the refcounts, that's as as good a place as anwhere else, yes.
I think the refcount test is pretty straightforward - if the refcount is 1, then we're the sole owner of the page and we don't need to worry about any other users. If its > 1, then somebody else has it, and we need to make sure it no longer refers to a granted page (which is just a matter of doing a
set_pte_atomic()to remap from present to present).[
set_pte_atomicover grant ptes doesn't work, or does it?]
No, I forgot about grant ptes magic properties. But there is the hypercall.
Quoted 9 lines
Quoted 5 lines
Then we'd have a set of frames whose lifetimes are being determined by some other subsystem. We can either maintain a list of them and poll waiting for them to become free, or just release them and let them be managed by the normal kernel lifetime rules (which requires that the memory attached to them be completely normal, of course).
The latter sounds like a good alternative to polling. So an
unmap_and_replace, and giving up ownership thereafter. Next run of the dispatcher thread can can just refill the foreign pfn range viaalloc_empty_pages(), to rebalance.
Do we actually need a "foreign page range"? Won't any pfn do? If we start with a specific range of foreign pfns and then start freeing those pfns back to the kernel, we won't have one for long...
J
Quoted 34 lines · Jeremy Fitzhardinge
Quoted 16 lines · Daniel Stodden
Quoted 11 lines · Jeremy Fitzhardinge
Quoted 8 lines · Daniel Stodden
Quoted 2 lines
Surely this can be dealt with by replacing the mapped granted page with a local copy if the refcount is elevated?
Yeah. We briefly discussed this when the problem started to pop up (again).
I had a patch, for blktap1 in XS 5.5 iirc, which would fill mapping with a dummy page mapped in. You wouldn't need a copy, a R/O zero map easily does the job.
Hm, I'd be a bit concerned that that might cause problems if used generically.
Yeah. It wasn't a problem because all the network backends are on TCP, where one can be rather sure that the dups are going to be properly dropped.
Quoted 5 lines
Does this hold everywhere ..? -- As mentioned below, the problem is rather in AIO/DIO than being Xen-specific, so you can see the same behavior on bare metal kernels too. A userspace app seeing an AIO complete and then reusing that buffer elsewhere will occassionally resend garbage over the network.
Yeah, that sounds like a generic security problem. I presume the protocol will just discard the excess retransmit data, but it might mean a usermode program ends up transmitting secrets it never intended to...
Quoted 6 lines
There are some important parts which would go missing. Such as ratelimiting gntdev accesses -- 200 thundering tapdisks each trying to gntmap 352 pages simultaneously isn't so good, so there still needs to be some bridge arbitrating them. I'd rather keep that in kernel space, okay to cram stuff like that into gntdev? It'd be much more straightforward than IPC.
Quoted 3 lines
What's the problem? If you do nothing then it will appear to the kernel as a bunch of processes doing memory allocations, and they'll get blocked/rate-limited accordingly if memory is getting short.
The problem is that just letting the page allocator work through allocations isn't going to scale anywhere.
The worst case memory requested under load is <number-of-disks> * (32 * 11 pages). As a (conservative) rule of thumb, N will be 200 or rather better.
The number of I/O actually in-flight at any point, in contrast, is derived from the queue/sg sizes of the physical device. For a simple disk, that's about a ring or two.
Quoted 13 lines
There's
plenty of existing mechanisms to control that sort of thing (cgroups, etc) without adding anything new to the kernel. Or are you talking about something other than simple memory pressure?And there's plenty of existing IPC mechanisms if you want them to explicitly coordinate with each other, but I'd tend to thing that's premature unless you have something specific in mind.
Quoted 3 lines
Also, I was absolutely certain I once saw
VM_FOREIGNsupport in gntdev.. Can't find it now, what happened? Without, there's presently still no zero-copy.
Quoted 5 lines
gntdev doesn't need
VM_FOREIGNany more - it uses the (relatively new-ish) mmu notifier infrastructure which is intended to allow a device to sync an external MMU with usermode mappings. We're not using it in precisely that way, but it allows us to wrangle grant mappings before the generic code tries to do normal pte ops on them.
The mmu notifiers were for safe teardown only. They are not sufficient for DIO, which wants gup() to work. If you want zcopy on gntdev, we'll need to back those VMAs with page structs. Or bounce again (gulp, just mentioning it). As with the blktap2 patches, note there is no difference in the dom0 memory bill, it takes page frames.
This is pretty much exactly the pooling stuff in current drivers/blktap. The interface could look as follows ([] designates users).
* [toolstack]
Calling some ctls to create/destroy ctls pools of frames. (Blktap currently does this in sysfs.)
* [toolstack]
Optionally resize them, according to the physical queue depth [estimate] of the underlying storage.
* [tapdisk]
A backend instance, when starting up, opens a gntdev, then uses a ctl to bind its gntdev handle to a frame pool.
* [tapdisk]
The .mmap call now will allocate frames to back the VMA. This operation can fail/block under congestion. Neither is desirable, so we need a .poll.
* [tapdisk]
To integrate grant mappings with a single-threaded event loop, use .poll. The handle fires as soon as a request can be mapped.
Under congestion, the .poll code will queue waiting disks and wake them round-robin, once VMAs are released.
(A [tapdisk] doesn't mean to dismiss a potential [qemu].)
Still confident we want that? (Seriously asking). A lot of the code to do so has been written for blktap, it wouldn't be hard to bend into a gntdev extension.
Quoted 10 lines
Quoted 10 lines
Once the issues were solved, it'd be kinda nice. Simplifies stuff like memshr for blktap, which depends on getting hold of original grefs.
We'd presumably still need the tapdev nodes, for qemu, etc. But those can stay non-xen aware then.
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
The only caveat is the stray unmapping problem, but I think gntdev can be modified to deal with that pretty easily.
Not easier than anything else in kernel space, but when dealing only with the refcounts, that's as as good a place as anwhere else, yes.
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
I think the refcount test is pretty straightforward - if the refcount is 1, then we're the sole owner of the page and we don't need to worry about any other users. If its > 1, then somebody else has it, and we need to make sure it no longer refers to a granted page (which is just a matter of doing a
set_pte_atomic()to remap from present to present).[
set_pte_atomicover grant ptes doesn't work, or does it?]No, I forgot about grant ptes magic properties. But there is the hypercall.
Yup.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Then we'd have a set of frames whose lifetimes are being determined by some other subsystem. We can either maintain a list of them and poll waiting for them to become free, or just release them and let them be managed by the normal kernel lifetime rules (which requires that the memory attached to them be completely normal, of course).
Quoted 8 lines
Quoted 4 lines
The latter sounds like a good alternative to polling. So an
unmap_and_replace, and giving up ownership thereafter. Next run of the dispatcher thread can can just refill the foreign pfn range viaalloc_empty_pages(), to rebalance.Do we actually need a "foreign page range"? Won't any pfn do? If we start with a specific range of foreign pfns and then start freeing those pfns back to the kernel, we won't have one for long...
I guess we've been meaning the same thing here, unless I'm misunderstanding you. Any pfn does, and the balloon pagevec allocations default to order 0 entries indeed. Sorry, you're right, that's not a 'range'. With a pending re-xmit, the backend can find a couple (or all) of the request frames have count>1. It can flip and abandon those as normal memory. But it will need those lost memory slots back, straight away or next time it's running out of frames. As order-0 allocations.
Foreign memory is deliberately short. Blkback still defaults to 2 rings worth of address space, iirc, globally. That's what that mempool sysfs stuff in the later blktap2 patches aimed at -- making the size configurable where queue length matters, and isolate throughput between physical backends, where the toolstack wants to care.
Daniel
Quoted 13 lines · Daniel Stodden
Quoted 2 lines
Quoted 2 lines
Also, I was absolutely certain I once saw
VM_FOREIGNsupport ingntdev..
Quoted 1 line
Quoted 1 line
Can't find it now, what happened? Without, there's presently still
no
Quoted 4 lines
Quoted 1 line
zero-copy.
gntdev doesn't need
VM_FOREIGNany more - it uses the (relatively new-ish) mmu notifier infrastructure which is intended to allow adevice
Quoted 1 line
to sync an external MMU with usermode mappings. We're not using it
in
Quoted 1 line
precisely that way, but it allows us to wrangle grant mappings
Quoted 8 lines
before
Quoted 1 line
the generic code tries to do normal pte ops on them.
The mmu notifiers were for safe teardown only. They are not sufficient for DIO, which wants
gup()to work. If you want zcopy on gntdev, we'll need to back those VMAs with page structs. Or bounce again (gulp, just mentioning it). As with the blktap2 patches, note there is no difference in the dom0 memory bill, it takes page frames.
I though the VM_FOREIGN stuff which blktap[12] hooks into gup() was there in order to avoid some deadlock arising from having a single struct page taking part in nested block I/O. i.e. one I/O created by blkback and the second from the tapdisk process deadlock against each other.
I though that in order to work around this blktap creates a second range of struct pages (the struct vm_foreign_map array in vma->vm_private_data) which aliases the pages under I/O and then it hooks gup() to do the switcheroo when necessary.
If the blkback interface is running in userspace (whether in a tapdisk or qemu process) then there is no nesting of I/O and the only I/O is from process context and therefore this particular issue is no longer a problem because we can use a properly struct page backed page without needing a magic VM_FOREIGN shadow of it?
Have I misunderstood something about the reason for VM_FOREIGN?
Ian.
Quoted 36 lines · Ian Campbell
Quoted 13 lines · Daniel Stodden
Quoted 2 lines
Quoted 2 lines
Also, I was absolutely certain I once saw
VM_FOREIGNsupport ingntdev..
Quoted 1 line
Quoted 1 line
Can't find it now, what happened? Without, there's presently still
no
Quoted 4 lines
Quoted 1 line
zero-copy.
gntdev doesn't need
VM_FOREIGNany more - it uses the (relatively new-ish) mmu notifier infrastructure which is intended to allow adevice
Quoted 1 line
to sync an external MMU with usermode mappings. We're not using it
in
Quoted 1 line
precisely that way, but it allows us to wrangle grant mappings
Quoted 8 lines
before
Quoted 1 line
the generic code tries to do normal pte ops on them.
The mmu notifiers were for safe teardown only. They are not sufficient for DIO, which wants
gup()to work. If you want zcopy on gntdev, we'll need to back those VMAs with page structs. Or bounce again (gulp, just mentioning it). As with the blktap2 patches, note there is no difference in the dom0 memory bill, it takes page frames.I though the
VM_FOREIGNstuff which blktap[12] hooks intogup()was there in order to avoid some deadlock arising from having a single struct page taking part in nested block I/O. i.e. one I/O created by blkback and the second from the tapdisk process deadlock against each other.I though that in order to work around this blktap creates a second range of struct pages (the struct
vm_foreign_maparray invma->vm_private_data) which aliases the pages under I/O and then it hooksgup()to do the switcheroo when necessary.
Quoted 7 lines
If the blkback interface is running in userspace (whether in a tapdisk or qemu process) then there is no nesting of I/O and the only I/O is from process context and therefore this particular issue is no longer a problem because we can use a properly struct page backed page without needing a magic
VM_FOREIGNshadow of it?Have I misunderstood something about the reason for
VM_FOREIGN?
VM_FOREIGN is a special case of grant mapping frames into a user's VMA.
This stuff is primarily there to make gup() grab a pagevec from the VMA struct, instead of relying on follow_page in order to map user-virtual to pseudophysical, which is what gup normally does.
In brief, it's hacking gup() to keep DIO working.
In other words: Userland can't I/O on some VM_PFNMAP or VM_IO or similar. If you ask for DMA, the kernel quite urgently wants this to look like normal memory.
Your description of the aliasing is correct, but that's yet another thing. It implies redoing the grantmap and p2m entries privately.
To make this clearer: If the aliasing weren't necessary, blktap2 would just have had to grab blkback's prepared page struct from the request SG vector, and .mmap that to userland, but still with VM_FOREIGN and some pagevec pointer in the VMA.
Instead, there's blkback-pagemap, specifically to map that SG page entry *back* to the original gref from a table, and *redo* the entire gntmap + p2m thing another time, privately.
Twisted, agreed. :)
Cheers,
Daniel
Quoted 60 lines · Daniel Stodden
Quoted 36 lines · Ian Campbell
Quoted 13 lines · Daniel Stodden
Quoted 2 lines
Quoted 2 lines
Also, I was absolutely certain I once saw
VM_FOREIGNsupport ingntdev..
Quoted 1 line
Quoted 1 line
Can't find it now, what happened? Without, there's presently still
no
Quoted 4 lines
Quoted 1 line
zero-copy.
gntdev doesn't need
VM_FOREIGNany more - it uses the (relatively new-ish) mmu notifier infrastructure which is intended to allow adevice
Quoted 1 line
to sync an external MMU with usermode mappings. We're not using it
in
Quoted 1 line
precisely that way, but it allows us to wrangle grant mappings
Quoted 8 lines
before
Quoted 1 line
the generic code tries to do normal pte ops on them.
The mmu notifiers were for safe teardown only. They are not sufficient for DIO, which wants
gup()to work. If you want zcopy on gntdev, we'll need to back those VMAs with page structs. Or bounce again (gulp, just mentioning it). As with the blktap2 patches, note there is no difference in the dom0 memory bill, it takes page frames.I though the
VM_FOREIGNstuff which blktap[12] hooks intogup()was there in order to avoid some deadlock arising from having a single struct page taking part in nested block I/O. i.e. one I/O created by blkback and the second from the tapdisk process deadlock against each other.I though that in order to work around this blktap creates a second range of struct pages (the struct
vm_foreign_maparray invma->vm_private_data) which aliases the pages under I/O and then it hooksgup()to do the switcheroo when necessary.Quoted 7 lines
If the blkback interface is running in userspace (whether in a tapdisk or qemu process) then there is no nesting of I/O and the only I/O is from process context and therefore this particular issue is no longer a problem because we can use a properly struct page backed page without needing a magic
VM_FOREIGNshadow of it?Have I misunderstood something about the reason for
VM_FOREIGN?
VM_FOREIGNis a special case of grant mapping frames into a user's VMA.This stuff is primarily there to make
gup()grab a pagevec from the VMA struct, instead of relying onfollow_pagein order to map user-virtual to pseudophysical, which is what gup normally does.In brief, it's hacking
gup()to keep DIO working.In other words: Userland can't I/O on some
VM_PFNMAPorVM_IOor similar. If you ask for DMA, the kernel quite urgently wants this to look like normal memory.Your description of the aliasing is correct, but that's yet another thing. It implies redoing the grantmap and p2m entries privately.
Quoted 8 lines
To make this clearer: If the aliasing weren't necessary, blktap2 would just have had to grab blkback's prepared page struct from the request SG vector, and .mmap that to userland, but still with
VM_FOREIGNand some pagevec pointer in the VMA.Instead, there's blkback-pagemap, specifically to map that SG page entry *back* to the original gref from a table, and *redo* the entire gntmap + p2m thing another time, privately.
In the userland blkback case do you need to redo the mapping? Isn't the original mapping (including pte update) of the granted mfn into the struct page associated with the user address sufficient?
Is the reason gup() doesn't work is that it tries to go from a pte_entry to a struct page (via a pfn, e.g. in vm_normal_page) which involves a m2p translation of the pte value, and this is not valid for a foreign page (since the m2p entry is for the remote domain)?
Or is the issue that we try and dynamically fault in those addresses and that doesn't work because you need a special set_pte variant (aka the gnttab map hypercall) to map a granted page?
Maybe both...
Quoted 8 lines
Twisted, agreed. :)
Cheers,
Daniel
Quoted 66 lines · Ian Campbell
Quoted 65 lines · Daniel Stodden
Quoted 36 lines · Ian Campbell
Quoted 13 lines · Daniel Stodden
Quoted 2 lines
Quoted 2 lines
Also, I was absolutely certain I once saw
VM_FOREIGNsupport ingntdev..
Quoted 1 line
Quoted 1 line
Can't find it now, what happened? Without, there's presently still
no
Quoted 4 lines
Quoted 1 line
zero-copy.
gntdev doesn't need
VM_FOREIGNany more - it uses the (relatively new-ish) mmu notifier infrastructure which is intended to allow adevice
Quoted 1 line
to sync an external MMU with usermode mappings. We're not using it
in
Quoted 1 line
precisely that way, but it allows us to wrangle grant mappings
Quoted 8 lines
before
Quoted 1 line
the generic code tries to do normal pte ops on them.
The mmu notifiers were for safe teardown only. They are not sufficient for DIO, which wants
gup()to work. If you want zcopy on gntdev, we'll need to back those VMAs with page structs. Or bounce again (gulp, just mentioning it). As with the blktap2 patches, note there is no difference in the dom0 memory bill, it takes page frames.I though the
VM_FOREIGNstuff which blktap[12] hooks intogup()was there in order to avoid some deadlock arising from having a single struct page taking part in nested block I/O. i.e. one I/O created by blkback and the second from the tapdisk process deadlock against each other.I though that in order to work around this blktap creates a second range of struct pages (the struct
vm_foreign_maparray invma->vm_private_data) which aliases the pages under I/O and then it hooksgup()to do the switcheroo when necessary.Quoted 7 lines
If the blkback interface is running in userspace (whether in a tapdisk or qemu process) then there is no nesting of I/O and the only I/O is from process context and therefore this particular issue is no longer a problem because we can use a properly struct page backed page without needing a magic
VM_FOREIGNshadow of it?Have I misunderstood something about the reason for
VM_FOREIGN?
VM_FOREIGNis a special case of grant mapping frames into a user's VMA.This stuff is primarily there to make
gup()grab a pagevec from the VMA struct, instead of relying onfollow_pagein order to map user-virtual to pseudophysical, which is what gup normally does.In brief, it's hacking
gup()to keep DIO working.In other words: Userland can't I/O on some
VM_PFNMAPorVM_IOor similar. If you ask for DMA, the kernel quite urgently wants this to look like normal memory.Your description of the aliasing is correct, but that's yet another thing. It implies redoing the grantmap and p2m entries privately.
To make this clearer: If the aliasing weren't necessary, blktap2 would just have had to grab blkback's prepared page struct from the request SG vector, and .mmap that to userland, but still with
VM_FOREIGNand some pagevec pointer in the VMA.
Quoted 7 lines
Quoted 3 lines
Instead, there's blkback-pagemap, specifically to map that SG page entry *back* to the original gref from a table, and *redo* the entire gntmap + p2m thing another time, privately.
In the userland blkback case do you need to redo the mapping? Isn't the original mapping (including pte update) of the granted mfn into the struct page associated with the user address sufficient?
It's completely sufficient. Running the sring in tapdisk straight away implies mapping once and for all. The present aliasing happens because of the two arrows in the following request submission chain: blkback -> tapdev -> disk. Each a designating I/O submissing to a blk request queue. It's just for the stacking.
Quoted 4 lines
Is the reason
gup()doesn't work is that it tries to go from apte_entryto a struct page (via a pfn, e.g. invm_normal_page) which involves a m2p translation of the pte value, and this is not valid for a foreign page (since the m2p entry is for the remote domain)?
Exactly. So either it's VM_FOREIGN, or we maybe go for a somewhat cleaner solution by teaching mfn_to_pfn new tricks, such as going through a private mfn->gntpfn lookup on the m2p failure path. See the related thread with Jeremy. I guess you'd have additional thoughts on that.
Quoted 3 lines
Or is the issue that we try and dynamically fault in those addresses and that doesn't work because you need a special
set_ptevariant (aka the gnttab map hypercall) to map a granted page?
Not a real issue. Blktap presently vm_insert()s, but demand paging makes no difference: gup() completely relies on follow_page. Means that a pte miss falls through to handle_mm_fault, then retries.
Thx,
Daniel
Quoted 13 lines · Daniel Stodden
Quoted 11 lines · Ian Campbell
Quoted 6 lines · Daniel Stodden
Quoted 1 line · Ian Campbell
On Tue, 2010-11-16 at 21:28 +0000, Daniel Stodden wrote:
Instead, there's blkback-pagemap, specifically to map that SG page entry *back* to the original gref from a table, and *redo* the entire gntmap + p2m thing another time, privately.
In the userland blkback case do you need to redo the mapping? Isn't the original mapping (including pte update) of the granted mfn into the struct page associated with the user address sufficient?
Quoted 5 lines
It's completely sufficient. Running the sring in tapdisk straight away implies mapping once and for all. The present aliasing happens because of the two arrows in the following request submission chain: blkback -> tapdev -> disk. Each a designating I/O submissing to a blk request queue. It's just for the stacking.
Great, just wanted to check I'd understood correctly.
Quoted 10 lines
Quoted 4 lines
Is the reason
gup()doesn't work is that it tries to go from apte_entryto a struct page (via a pfn, e.g. invm_normal_page) which involves a m2p translation of the pte value, and this is not valid for a foreign page (since the m2p entry is for the remote domain)?Exactly. So either it's
VM_FOREIGN, or we maybe go for a somewhat cleaner solution by teachingmfn_to_pfnnew tricks, such as going through a privatemfn->gntpfnlookup on the m2p failure path. See the related thread with Jeremy. I guess you'd have additional thoughts on that.
I saw the thread but don't have any particular thoughts, it seems like an eminently plausible idea...
Ian.
Quoted 9 lines · Daniel Stodden
Quoted 3 lines
What's the problem? If you do nothing then it will appear to the kernel as a bunch of processes doing memory allocations, and they'll get blocked/rate-limited accordingly if memory is getting short.
The problem is that just letting the page allocator work through allocations isn't going to scale anywhere.
The worst case memory requested under load is <number-of-disks> * (32 * 11 pages). As a (conservative) rule of thumb, N will be 200 or rather better.
Under what circumstances would you end up needing to allocate that many pages?
Quoted 3 lines
The number of I/O actually in-flight at any point, in contrast, is derived from the queue/sg sizes of the physical device. For a simple disk, that's about a ring or two.
Wouldn't that be the worst case?
Quoted 12 lines
Quoted 12 lines
There's
plenty of existing mechanisms to control that sort of thing (cgroups, etc) without adding anything new to the kernel. Or are you talking about something other than simple memory pressure?And there's plenty of existing IPC mechanisms if you want them to explicitly coordinate with each other, but I'd tend to thing that's premature unless you have something specific in mind.
Quoted 3 lines
Also, I was absolutely certain I once saw
VM_FOREIGNsupport in gntdev.. Can't find it now, what happened? Without, there's presently still no zero-copy.
Quoted 8 lines
Quoted 5 lines
gntdev doesn't need
VM_FOREIGNany more - it uses the (relatively new-ish) mmu notifier infrastructure which is intended to allow a device to sync an external MMU with usermode mappings. We're not using it in precisely that way, but it allows us to wrangle grant mappings before the generic code tries to do normal pte ops on them.The mmu notifiers were for safe teardown only. They are not sufficient for DIO, which wants
gup()to work. If you want zcopy on gntdev, we'll need to back those VMAs with page structs.
The pages will have struct page, because they're normal kernel pages which happen to be backed by mapped granted pages. Are you talking about the #ifdef CONFIG_XEN code in the middle of __get_user_pages()? Isn't that just there to cope with the nested-IO-on-the-same-page problem that the current blktap architecture provokes? If there's only a single IO on each page - the one initiated by usermode - then it shouldn't be necessary, right?
Quoted 3 lines
Or bounce again (gulp, just
mentioning it). As with the blktap2 patches, note there is no difference in the dom0 memory bill, it takes page frames.
(And perhaps actual pages to substitute for the granted pages.)
Quoted 7 lines
I guess we've been meaning the same thing here, unless I'm misunderstanding you. Any pfn does, and the balloon pagevec allocations default to order 0 entries indeed. Sorry, you're right, that's not a 'range'. With a pending re-xmit, the backend can find a couple (or all) of the request frames have count>1. It can flip and abandon those as normal memory. But it will need those lost memory slots back, straight away or next time it's running out of frames. As order-0 allocations.
Right. GFP_KERNEL order 0 allocations are pretty reliable; they only fail if the system is under extreme memory pressure. And it has the nice property that if those allocations block or fail it rate limits IO ingress from domains rather than being crushed by memory pressure at the backend (ie, the problem with trying to allocate memory in the writeout path).
Also the cgroup mechanism looks like an extremely powerful way to control the allocations for a process or group of processes to stop them from dominating the whole machine.
J
Quoted 7 lines · Jeremy Fitzhardinge
Quoted 6 lines · Daniel Stodden
Quoted 3 lines
What's the problem? If you do nothing then it will appear to the kernel as a bunch of processes doing memory allocations, and they'll get blocked/rate-limited accordingly if memory is getting short.
The problem is that just letting the page allocator work through allocations isn't going to scale anywhere.
Quoted 6 lines
Quoted 3 lines
The worst case memory requested under load is <number-of-disks> * (32 * 11 pages). As a (conservative) rule of thumb, N will be 200 or rather better.
Under what circumstances would you end up needing to allocate that many pages?
I don't. Independently running tapdisks would do, on behalf of guests queuing I/O.
That's why one wouldn't just let them run and allocate their own memory. The memory space set aside for I/O should be a shared resource.
Quoted 5 lines
Quoted 3 lines
The number of I/O actually in-flight at any point, in contrast, is derived from the queue/sg sizes of the physical device. For a simple disk, that's about a ring or two.
Wouldn't that be the worst case?
Yes. It's quite small. A 2 or 3 megs per physical backend are usually sufficient.
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
There's
plenty of existing mechanisms to control that sort of thing (cgroups, etc) without adding anything new to the kernel. Or are you talking about something other than simple memory pressure?And there's plenty of existing IPC mechanisms if you want them to explicitly coordinate with each other, but I'd tend to thing that's premature unless you have something specific in mind.
Quoted 3 lines
Also, I was absolutely certain I once saw
VM_FOREIGNsupport in gntdev.. Can't find it now, what happened? Without, there's presently still no zero-copy.gntdev doesn't need
VM_FOREIGNany more - it uses the (relatively new-ish) mmu notifier infrastructure which is intended to allow a device to sync an external MMU with usermode mappings. We're not using it in precisely that way, but it allows us to wrangle grant mappings before the generic code tries to do normal pte ops on them.
Quoted 6 lines
Quoted 3 lines
The mmu notifiers were for safe teardown only. They are not sufficient for DIO, which wants
gup()to work. If you want zcopy on gntdev, we'll need to back those VMAs with page structs.The pages will have struct page, because they're normal kernel pages which happen to be backed by mapped granted pages.
And, like all granted frames, not owning them implies they are not resolvable via mfn_to_pfn, thereby failing in follow_page, thereby gup() without the VM_FOREIGN hack.
Correct me if I'm mistaken. I used to be quicker looking up stuff on arch-xen kernels, but I think fundamental constants of the Xen universe didn't change since last time.
Quoted 6 lines
Are you talking
about the #ifdefCONFIG_XENcode in the middle of__get_user_pages()? Isn't that just there to cope with the nested-IO-on-the-same-page problem that the current blktap architecture provokes? If there's only a single IO on each page - the one initiated by usermode - then it shouldn't be necessary, right?
No. Jake brought the aliasing in specifically to get blktap2 working with zero-copy.
VM_FOREIGN is much older. Only blktap2 does the recursive thing, because it's a blkdev above some physical dev. Blktap1 went from the guest ring straight down to userland. As would be the case with a gntdev-based blkback.
Quoted 5 lines
Quoted 3 lines
Or bounce again (gulp, just
mentioning it). As with the blktap2 patches, note there is no difference in the dom0 memory bill, it takes page frames.(And perhaps actual pages to substitute for the granted pages.)
Well yes, that's right. Still fine as long there's some relatively small constant boundary on the worst case. O(n) for large systems would go in the hundreds of megs. Given that the *reasonable* amount of memory used simultaneously is pretty small in any case, even going through the memory allocator can be skipped.
[
Part of the reason why blktap *never* frees those pages, apart from being slightly greedy, are deadlock hazards when writing those nodes in dom0 through the pagecache, as dom0 might. You need memory pools on the datapath to guarantee progress under pressure. That got pretty ugly after 2.6.27, btw.
]
In any case, let's skip trying what happens if a thundering herd of several hundred userspace disks tries gfp()ing their grant slots out of dom0 without without arbitration.
Quoted 15 lines
Quoted 7 lines
I guess we've been meaning the same thing here, unless I'm misunderstanding you. Any pfn does, and the balloon pagevec allocations default to order 0 entries indeed. Sorry, you're right, that's not a 'range'. With a pending re-xmit, the backend can find a couple (or all) of the request frames have count>1. It can flip and abandon those as normal memory. But it will need those lost memory slots back, straight away or next time it's running out of frames. As order-0 allocations.
Right.
GFP_KERNELorder 0 allocations are pretty reliable; they only fail if the system is under extreme memory pressure. And it has the nice property that if those allocations block or fail it rate limits IO ingress from domains rather than being crushed by memory pressure at the backend (ie, the problem with trying to allocate memory in the writeout path).
Quoted 3 lines
Also the cgroup mechanism looks like an extremely powerful way to control the allocations for a process or group of processes to stop them from dominating the whole machine.
Ah. In case it can be put to work to bind processes allocating pagecache entries for dirtying to some boundary, I'd be really interested. I think I came across it once but didn't take the time to read the docs thoroughly. Can it?
Daniel
Quoted 3 lines · Daniel Stodden
And, like all granted frames, not owning them implies they are not resolvable via
mfn_to_pfn, thereby failing infollow_page, therebygup()without theVM_FOREIGNhack.
Hm, I see. Well, I wonder if using _PAGE_SPECIAL would help (it is put on usermode ptes which don't have a backing struct page). After all, there's no fundamental reason why it would need a pfn; the mfn in the pte is what's actually needed to ultimately generate a DMA descriptor.
Quoted 3 lines
Correct me if I'm mistaken. I used to be quicker looking up stuff on arch-xen kernels, but I think fundamental constants of the Xen universe didn't change since last time.
No, but Linux has.
Quoted 7 lines
[
Part of the reason why blktap *never* frees those pages, apart from being slightly greedy, are deadlock hazards when writing those nodes in dom0 through the pagecache, as dom0 might. You need memory pools on the datapath to guarantee progress under pressure. That got pretty ugly after 2.6.27, btw.
]
That's what mempools are intended to solve.
Quoted 3 lines
In any case, let's skip trying what happens if a thundering herd of several hundred userspace disks tries gfp()ing their grant slots out of dom0 without without arbitration.
I'm not against arbitration, but I don't think that's something that should be implemented as part of a Xen driver.
Quoted 14 lines
Quoted 14 lines
Quoted 7 lines
I guess we've been meaning the same thing here, unless I'm misunderstanding you. Any pfn does, and the balloon pagevec allocations default to order 0 entries indeed. Sorry, you're right, that's not a 'range'. With a pending re-xmit, the backend can find a couple (or all) of the request frames have count>1. It can flip and abandon those as normal memory. But it will need those lost memory slots back, straight away or next time it's running out of frames. As order-0 allocations.
Right.
GFP_KERNELorder 0 allocations are pretty reliable; they only fail if the system is under extreme memory pressure. And it has the nice property that if those allocations block or fail it rate limits IO ingress from domains rather than being crushed by memory pressure at the backend (ie, the problem with trying to allocate memory in the writeout path).
Quoted 7 lines
Quoted 3 lines
Also the cgroup mechanism looks like an extremely powerful way to control the allocations for a process or group of processes to stop them from dominating the whole machine.
Ah. In case it can be put to work to bind processes allocating pagecache entries for dirtying to some boundary, I'd be really interested. I think I came across it once but didn't take the time to read the docs thoroughly. Can it?
I'm not sure about dirtyness - it seems like something that should be within its remit, even if it doesn't currently have it.
The cgroup mechanism is extremely powerful, now that I look at it. You can do everything from setting block IO priorities and QoS parameters to CPU limits.
J
Quoted 9 lines · Jeremy Fitzhardinge
Quoted 3 lines · Daniel Stodden
And, like all granted frames, not owning them implies they are not resolvable via
mfn_to_pfn, thereby failing infollow_page, therebygup()without theVM_FOREIGNhack.Hm, I see. Well, I wonder if using
_PAGE_SPECIALwould help (it is put on usermode ptes which don't have a backing struct page). After all, there's no fundamental reason why it would need a pfn; the mfn in the pte is what's actually needed to ultimately generate a DMA descriptor.
The kernel needs the page structs at least for locking and refcounting.
There's also a some trickier stuff in there. Like redirtying disk-backed user memory after read completion, in case it's been laundered. (So that an AIO on unpinned user memory doesn't subsequently get flashed back when cycling through swap, if I understood that thing correctly.)
Doesn't apply for blktap (it's all reserved pages). All I mean is: I wouldn't exactly see some innocent little dio hack or so shape up in there.
Kernel allowing to DMA into a bare pfnmap -- From the platform POV, I'd agree. E.g. there's a concept of devices DMA-ing into arbitrary I/O memory space, not host memory, on some bus architectures. PCI would come to my mind (the old shared medium stuff, unsure about those newfangled P-t-P topologies). But not in Linux, so I presently don't see anybody upstream bothering to make block-I/O request addressing more forgiving than it is.
PAGE_SPECIAL -- to the kernel, that means the opposite: page structs which aren't backed by 'real' memory, so gup(), for example, is told to fail (how nasty). In contrast, VM_FOREIGN is non-memory backed by page structs.
Quoted 5 lines
Quoted 3 lines
Correct me if I'm mistaken. I used to be quicker looking up stuff on arch-xen kernels, but I think fundamental constants of the Xen universe didn't change since last time.
No, but Linux has.
Not in that respect.
There's certainly a way to get VM_FOREIGN out of the mainline code. It would involve an unlikely() branch in .pte_val(=xen_pte_val) to fall back into a private local m2p hash lookup. Assuming that kind of thing gets nowhere inlined. Not nice, but still more upstreamable than VM_FOREIGN.
Quoted 9 lines
Quoted 7 lines
[
Part of the reason why blktap *never* frees those pages, apart from being slightly greedy, are deadlock hazards when writing those nodes in dom0 through the pagecache, as dom0 might. You need memory pools on the datapath to guarantee progress under pressure. That got pretty ugly after 2.6.27, btw.
]That's what mempools are intended to solve.
That's why the blktap frame pool is now a mempool, indeed.
Quoted 6 lines
Quoted 3 lines
In any case, let's skip trying what happens if a thundering herd of several hundred userspace disks tries gfp()ing their grant slots out of dom0 without without arbitration.
I'm not against arbitration, but I don't think that's something that should be implemented as part of a Xen driver.
Uhm, maybe I'm misunderstanding you, isn't the whole thing a Xen driver? What do you suggest?
Quoted 22 lines
Quoted 14 lines
Quoted 14 lines
Quoted 7 lines
I guess we've been meaning the same thing here, unless I'm misunderstanding you. Any pfn does, and the balloon pagevec allocations default to order 0 entries indeed. Sorry, you're right, that's not a 'range'. With a pending re-xmit, the backend can find a couple (or all) of the request frames have count>1. It can flip and abandon those as normal memory. But it will need those lost memory slots back, straight away or next time it's running out of frames. As order-0 allocations.
Right.
GFP_KERNELorder 0 allocations are pretty reliable; they only fail if the system is under extreme memory pressure. And it has the nice property that if those allocations block or fail it rate limits IO ingress from domains rather than being crushed by memory pressure at the backend (ie, the problem with trying to allocate memory in the writeout path).Quoted 7 lines
Quoted 3 lines
Also the cgroup mechanism looks like an extremely powerful way to control the allocations for a process or group of processes to stop them from dominating the whole machine.
Ah. In case it can be put to work to bind processes allocating pagecache entries for dirtying to some boundary, I'd be really interested. I think I came across it once but didn't take the time to read the docs thoroughly. Can it?
Quoted 6 lines
I'm not sure about dirtyness - it seems like something that should be within its remit, even if it doesn't currently have it.
The cgroup mechanism is extremely powerful, now that I look at it. You can do everything from setting block IO priorities and QoS parameters to CPU limits.
Thanks. I'll keep it under my pillow then.
Daniel
Quoted 10 lines · Daniel Stodden
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 3 lines · Daniel Stodden
And, like all granted frames, not owning them implies they are not resolvable via
mfn_to_pfn, thereby failing infollow_page, therebygup()without theVM_FOREIGNhack.Hm, I see. Well, I wonder if using
_PAGE_SPECIALwould help (it is put on usermode ptes which don't have a backing struct page). After all, there's no fundamental reason why it would need a pfn; the mfn in the pte is what's actually needed to ultimately generate a DMA descriptor.The kernel needs the page structs at least for locking and refcounting.
Yeah.
Quoted 17 lines
There's also a some trickier stuff in there. Like redirtying disk-backed user memory after read completion, in case it's been laundered. (So that an AIO on unpinned user memory doesn't subsequently get flashed back when cycling through swap, if I understood that thing correctly.)
Doesn't apply for blktap (it's all reserved pages). All I mean is: I wouldn't exactly see some innocent little dio hack or so shape up in there.
Kernel allowing to DMA into a bare pfnmap -- From the platform POV, I'd agree. E.g. there's a concept of devices DMA-ing into arbitrary I/O memory space, not host memory, on some bus architectures. PCI would come to my mind (the old shared medium stuff, unsure about those newfangled P-t-P topologies). But not in Linux, so I presently don't see anybody upstream bothering to make block-I/O request addressing more forgiving than it is.
Quoted 3 lines
PAGE_SPECIAL-- to the kernel, that means the opposite: page structs which aren't backed by 'real' memory, sogup(), for example, is told to fail (how nasty).
It's pfns with no corresponding struct page - it's the pte level equivalent of VM_PFNMAP at the VMA level. But you're right that we can't do without struct pages.
So we're back to needing a way of mapping from a random mfn to a pfn so we can find the corresponding struct page. I would be tempted to put a layer over m2p to allow local m2p mappings to override the global m2p table.
Quoted 2 lines
In contrast,
VM_FOREIGNis non-memory backed by page
structs.
Yep.
J
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Daniel Stodden
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 7 lines · Daniel Stodden
Quoted 3 lines
So we're back to needing a way of mapping from a random mfn to a pfn so we can find the corresponding struct page. I would be tempted to put a layer over m2p to allow local m2p mappings to override the global m2p table.
I think a local m2p lookup on a slow path is a superior option, iff you do think it's doable. Without e.g. risking to bloat some inline stuff, I mean.
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
Where do you think in the call stack down into pvops code would the lookup go? Asking because I'd expect the kernel to potentially learn more tricks with that.
I don't think m2p is all that performance critical. p2m is used way more.
p2m is already a radix tree;
Yes, but pfns are dense plug holes, aren't they?
Yes.
Quoted 35 lines
Quoted 5 lines
I think m2p could be done somewhat
similarly, where undefined entries fall through to the global m2p. The main problem is probably making sure the memory allocation for new m2p entries can be done in a blocking context, so we don't have to rely onGFP_ATOMIC.Whatever the index is going to be, all the backends I'm aware of run their rings on a kthread. Sounds to me like
GFP_WAITfollowed by an rwlock is perfectly sufficient. Only the reversal commonly ends up in interrupt context.Quoted 25 lines
That particular m2p lookup would be in
xen_pte_val(), but I think that's the callsite for pretty much all m2p lookups.Quoted 11 lines
A
mfn->grefmapping would obsolete blkback-pagemap. Well, iff the kernel-blktap zerocopy stuff wants to be resurrected.It would also be a cheap way to implement current blktap to do
virt->greflookups for tapdisks. Some tapdisk filters want this, present major example is the memshr patch, and it's sort of nicer than a ring message hack.Wouldn't storing the handle allow unmapping grant ptes on the normal user PT teardown path? I think we always was this .zap_pte vma-operation in blktap, iirc. MMU notifier replacement? Maybe not a good one.
I think mmu notifiers are fine; this is exactly what they're for after all. Very few address spaces have granted pages mapped into them, so keeping the normal pte operations as fast as possible and using more expensive notifiers for the afflicted mms seems like the right tradeoff.
Hm. Before Gerd highlighted mmu notifiers as the right way to deal with granted pages in gntdev, I had the idea of allocating a shadow page for each pte page and storing grant refs there, where the shadow is hung of the pte page's struct page, where
set_ptecould use it to do the special thing if needed.
Quoted 8 lines
Quoted 4 lines
I wonder if we could do something similar here, where we store the pfn for a given pte in the shadow? But how would it get used? There's no "read pte" pvop, and
pte_valgets the pte value, not its address, so there'd be no way to find the shadow. So I guess that doesn't work.I'm not sure about the radix variant. All the backends do order-0 allocations, as discussed above. Depending on the driver pair behavior, the mfn ranges can get arbitrarily sparse. The real M2P makes completely different assumptions in density and size, or not?
Well, there's two use-cases for the local m2p idea. One is for granted pages, which are going to be all over the place, but the other is for hardware mfns, which are more likely to be densely clustered.
A radix tree for grant mfns is likely to be pretty inefficient - but the worst case is one radix page per mfn, which isn't too bad, since we're not expecting that many granted mfns to be around. But perhaps a hash or rbtree would be a better fit. Or we could insist on making the mfns contiguous.
Quoted 2 lines
Well, maybe one could shadow (cow, actually) just that? Saves the index. Likely a dumb idea. :)
I guess Xen won't let us map over the m2p, but maybe we could alias it. But that's going to waste lots of address space in a 32b dom0.
Quoted 3 lines
What numbers of grant refs do we run? I remember times when the tables were bumped up massively, for stuff like pvfb, a long time ago. I guess rest remained rather conservative.
We only really need to worry about mfns which are actually going to be mapped into userspace and guped. We could even propose a (new?) mmu notifier to prepare for gup so that it can be deferred as late as possible.
Quoted 7 lines
The shadow idea sounds pretty cool, mainly because the vaddr spaces are often contiguous. At least for userland pages.
Thing which would bother me is that settling on a single shadow page already limits map entries to sizeof(pte), so ideas like additionally mapping to grefs/handles/flag already go out of the window. Or grow bigger leaf, but on average that's also more space overhead.
At least we can always fit a pointer into sizeof(pte_t), so there's some scope for having more storage if needed. But I don't see how it can help for gup...
J
Quoted 9 lines · Daniel Stodden
Quoted 6 lines
Quoted 3 lines
In any case, let's skip trying what happens if a thundering herd of several hundred userspace disks tries gfp()ing their grant slots out of dom0 without without arbitration.
I'm not against arbitration, but I don't think that's something that should be implemented as part of a Xen driver.
Uhm, maybe I'm misunderstanding you, isn't the whole thing a Xen driver? What do you suggest?
Just misread you, sorry. You mean arbitration via IPC.
Somewhat, but ugly. Just counting pages with cmpxchg in some shm segment isnt't a big deal, agreed. But userspace fixing the counter after detecting some process crash would already start to complicate that. Next, someone has to do the ballooning, and you need gntmap to understand the VMA type anyway. From there on, going the rest of the way and let the kernel driver round-robin the frame pool altogether is much smaller and cleaner.
Daniel