Remove zone reset all emulation
Subsystems (8)
Damien Le Moal
maintainerv2 cover Remove zone reset all emulation
Jens, Mike,
Here is a set of patches based on block/for-next to remove the emulation for zone reset all from the block layer and move it to device mapper. This is done because device mapper is the only zoned device driver that does not natively support REQ_OP_ZONE_RESET_ALL. With this change, the emulation that may be required depending on a mapped device zone mapping is moved to device mapper and the reset all feature (BLK_FEAT_ZONE_RESETALL) can be deleted, as well as all the code handling this feature in blk-zoned.c. The DM-based handling of REQ_OP_ZONE_RESET_ALL can also be much faster than the block layer emulation as that operation can be forwarded as is to targets mapping all sequential write required zones.
Patch 1 modifies null_blk to add the zone_full parameter to initialize the sequential zones of a zoned null_blk device to be full. This is convenient when testing read workloads as well as zone management operations as that avoids having to first write to device to make the zones full.
Patch 2 is a simple prep patch for patch 3. Patch 3 implements the emulation for zone reset all in device mapper core code.
Patch 4 removes the block layer based emulation and modifies all drivers setting the BLK_FEAT_ZONE_RESETALL queue limit feature to not set this feature (and the feature is removed). This enables the use of the device mapper emulation.
Patch 5 is a cleanup of blk-zoned.c made possible with patch 4.
Changes from v1:
- Fixed typo in the commit message and long comment line of patch 3. Also removed the stubbed definition of dm_zone_get_reset_bitmap() in dm.h as it is not necessary.
- Modified patch 4 to have submit_bio_noacct() treat REQ_OP_ZONE_RESET_ALL the same way as other zone operations.
Damien Le Moal (5):
null_blk: Introduce the zone_full parameter
dm: Refactor is_abnormal_io()
dm: handle REQ_OP_ZONE_RESET_ALL
block: Remove REQ_OP_ZONE_RESET_ALL emulation
block: Remove blk_alloc_zone_bitmap()
block/blk-core.c | 5 +- block/blk-zoned.c | 88 +---------------- drivers/block/null_blk/main.c | 9 +- drivers/block/null_blk/null_blk.h | 1 + drivers/block/null_blk/zoned.c | 12 ++- drivers/block/ublk_drv.c | 2 +- drivers/block/virtio_blk.c | 2 +- drivers/md/dm-zone.c | 50 +++++++++- drivers/md/dm.c | 159 +++++++++++++++++++++++++++--- drivers/md/dm.h | 3 + drivers/nvme/host/zns.c | 2 +- drivers/scsi/sd_zbc.c | 2 +- include/linux/blkdev.h | 5 - include/linux/device-mapper.h | 7 ++ 14 files changed, 229 insertions(+), 118 deletions(-)
Signature · 1 line
2.45.2
Damien Le Moal
maintainerPATCH v2 1/5 null_blk: Introduce the zone_full parameter
Allow creating a zoned null_blk device with the initial state of its sequential write required zones to be FULL. This is convenient to avoid having to first write these zones to perform read performance evaluation or test zone management operations such as zone reset (and zone reset all).
--- drivers/block/null_blk/main.c | 9 ++++++++- drivers/block/null_blk/null_blk.h | 1 + drivers/block/null_blk/zoned.c | 10 ++++++++-- 3 files changed, 17 insertions(+), 3 deletions(-)
drivers/block/null_blk/main.c +8 -1
@@ -262,6 +262,10 @@ module_param_named(zone_append_max_sectors, g_zone_append_max_sectors, int, 0444MODULE_PARM_DESC (zone_append_max_sectors,"Maximum size of a zone append command (in 512B sectors). Specify 0 for zone append emulation" ); +static bool g_zone_full; +module_param_named (zone_full, g_zone_full, bool,S_IRUGO ); +MODULE_PARM_DESC (zone_full,"Initialize the sequential write required zones of a zoned device to be full. Default: false" ); +static struct nullb_device *null_alloc_dev (void );static void null_free_dev (struct nullb_device *dev);static void null_del_dev (struct nullb *nullb);
@@ -458,6 +462,7 @@ NULLB_DEVICE_ATTR(zone_nr_conv, uint, NULL);NULLB_DEVICE_ATTR (zone_max_open, uint,NULL );NULLB_DEVICE_ATTR (zone_max_active, uint,NULL );NULLB_DEVICE_ATTR (zone_append_max_sectors, uint,NULL ); +NULLB_DEVICE_ATTR (zone_full, bool,NULL );NULLB_DEVICE_ATTR (virt_boundary, bool,NULL );NULLB_DEVICE_ATTR (no_sched, bool,NULL );NULLB_DEVICE_ATTR (shared_tags, bool,NULL );
@@ -610,6 +615,7 @@ static struct configfs_attribute *nullb_device_attrs[] = { &nullb_device_attr_zone_append_max_sectors, &nullb_device_attr_zone_readonly, &nullb_device_attr_zone_offline, + &nullb_device_attr_zone_full, &nullb_device_attr_virt_boundary, &nullb_device_attr_no_sched, &nullb_device_attr_shared_tags,
@@ -700,7 +706,7 @@ static ssize_t memb_group_features_show(struct config_item *item, char *page)"shared_tags,size,submit_queues,use_per_node_hctx," "virt_boundary,zoned,zone_capacity,zone_max_active," "zone_max_open,zone_nr_conv,zone_offline,zone_readonly," - "zone_size,zone_append_max_sectors\n"); + "zone_size,zone_append_max_sectors,zone_full\n"); }CONFIGFS_ATTR_RO (memb_group_, features);
@@ -781,6 +787,7 @@ static struct nullb_device *null_alloc_dev(void) dev->zone_max_open = g_zone_max_open; dev->zone_max_active = g_zone_max_active; dev->zone_append_max_sectors = g_zone_append_max_sectors; + dev->zone_full = g_zone_full; dev->virt_boundary = g_virt_boundary; dev->no_sched = g_no_sched; dev->shared_tags = g_shared_tags;
drivers/block/null_blk/null_blk.h +1 -0
@@ -101,6 +101,7 @@ struct nullb_device {bool memory_backed;/* if data is stored in memory */ bool discard;/* if support discard */ bool zoned;/* if device is zoned */ +bool zone_full;/* Initialize zones to be full */ bool virt_boundary;/* virtual boundary on/off for the device */ bool no_sched;/* no IO scheduler for the device */ bool shared_tags;/* share tag set between devices for blk-mq */
drivers/block/null_blk/zoned.c +8 -2
@@ -145,7 +145,7 @@ int null_init_zoned_dev(struct nullb_device *dev, zone = &dev->zones[i];null_init_zone_lock (dev, zone); - zone->start = zone->wp = sector; + zone->start = sector;if (zone->start + dev->zone_size_sects > dev_capacity_sects) zone->len = dev_capacity_sects - zone->start;else
@@ -153,7 +153,13 @@ int null_init_zoned_dev(struct nullb_device *dev, zone->capacity =min_t (sector_t, zone->len, zone_capacity_sects); zone->type =BLK_ZONE_TYPE_SEQWRITE_REQ ; - zone->cond =BLK_ZONE_COND_EMPTY ; +if (dev->zone_full) { + zone->cond =BLK_ZONE_COND_FULL ; + zone->wp = zone->start + zone->capacity; + }else { + zone->cond =BLK_ZONE_COND_EMPTY ; + zone->wp = zone->start; + } sector += dev->zone_size_sects; }
Signature · 1 line
2.45.2
Damien Le Moal
maintainerPATCH v2 2/5 dm: Refactor is_abnormal_io()
In reply to Damien Le MoalUse a single switch-case to simplify is_abnormal_io() and make this function more readable and easier to modify.
--- drivers/md/dm.c | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-)
drivers/md/dm.c +11 -13
@@ -1598,20 +1598,18 @@ static void __send_abnormal_io(struct clone_info *ci, struct dm_target *ti,static bool is_abnormal_io (struct bio *bio) { -enum req_op op =bio_op (bio); - -if (op !=REQ_OP_READ && op !=REQ_OP_WRITE && op !=REQ_OP_FLUSH ) { -switch (op) { -case REQ_OP_DISCARD : -case REQ_OP_SECURE_ERASE : -case REQ_OP_WRITE_ZEROES : -return true; -default : -break ; - } +switch (bio_op (bio)) { +case REQ_OP_READ : +case REQ_OP_WRITE : +case REQ_OP_FLUSH : +return false; +case REQ_OP_DISCARD : +case REQ_OP_SECURE_ERASE : +case REQ_OP_WRITE_ZEROES : +return true; +default : +return false; } - -return false; } static blk_status_t __process_abnormal_io(struct clone_info *ci,
Signature · 1 line
2.45.2
Damien Le Moal
maintainerPATCH v2 3/5 dm: handle REQ_OP_ZONE_RESET_ALL
In reply to Damien Le MoalThis commit implements processing of the REQ_OP_ZONE_RESET_ALL operation for zoned mapped devices. Given that this operation always has a BIO sector of 0 and a 0 size, processing through the regular BIO __split_and_process_bio() function does not work because this function would always select the first target. Instead, handling of this operation is implemented using the function __send_zone_reset_all().
Similarly to the __send_empty_flush() function, the new __send_zone_reset_all() function manually goes through all targets of a mapped device table doing the following:
1) If the target can natively support REQ_OP_ZONE_RESET_ALL, __send_duplicate_bios() is used to forward the reset all operation to the target. This case is handled with the __send_zone_reset_all_native() function.
2) For other targets, the function __send_zone_reset_all_emulated() is executed to emulate the execution of REQ_OP_ZONE_RESET_ALL using regular REQ_OP_ZONE_RESET operations.
Targets that can natively support REQ_OP_ZONE_RESET_ALL are identified using the new target field zone_reset_all_supported. This boolean is set to true in for targets that have reliable zone limits, that is, targets that map all sequential write required zones of their zoned device(s). Setting this field is handled in dm_set_zones_restrictions() and device_get_zone_resource_limits().
For targets with unreliable zone limits, REQ_OP_ZONE_RESET_ALL must be emulated (case 2 above). This is implemented with __send_zone_reset_all_emulated() and is similar to the block layer function blkdev_zone_reset_all_emulated(): first a report zones is done for the zones of the target to identify zones that need reset, that is, any sequential write required zone that is not already empty. This is done using a bitmap and the function dm_zone_get_reset_bitmap() which sets to 1 the bit corresponding to a zone that needs reset. Next, this zone bitmap is inspected and a clone BIO modified to use the REQ_OP_ZONE_RESET operation issued for any zone with its bit set in the zone bitmap.
This implementation is more efficient than what the block layer does with blkdev_zone_reset_all_emulated(), which is always used for DM zoned devices currently: as we can natively use REQ_OP_ZONE_RESET_ALL on targets mapping all sequential write required zones, resetting all zones of a zoned mapped device can be much faster compared to always emulating this operation using regular per-zone reset. In the worst case, this implementation is as-efficient as the block layer emulation. This reduction in the time it takes to reset all zones of a zoned mapped device depends directly on the mapped device targets mapping (reliable zone limits or not).
--- drivers/md/dm-zone.c | 50 ++++++++++++- drivers/md/dm.c | 135 +++++++++++++++++++++++++++++++++- drivers/md/dm.h | 3 + include/linux/device-mapper.h | 7 ++ 4 files changed, 190 insertions(+), 5 deletions(-)
drivers/md/dm-zone.c +48 -2
@@ -292,10 +292,12 @@ static int device_get_zone_resource_limits(struct dm_target *ti,/* * If the target does not map all sequential zones, the limits -* will not be reliable. +* will not be reliable and we cannot use REQ_OP_ZONE_RESET_ALL. */ -if (zc.target_nr_seq_zones < zc.total_nr_seq_zones) +if (zc.target_nr_seq_zones < zc.total_nr_seq_zones) { zlim->reliable_limits = false; + ti->zone_reset_all_supported = false; + }/* * If the target maps less sequential zones than the limit values, then
@@ -353,6 +355,14 @@ int dm_set_zones_restrictions(struct dm_table *t, struct request_queue *q,for (unsigned int i =0 ; i < t->num_targets; i++) {struct dm_target *ti =dm_table_get_target (t, i); +/* +* Assume that the target can accept REQ_OP_ZONE_RESET_ALL. +* device_get_zone_resource_limits() may adjust this if one of +* the device used by the target does not have all its +* sequential write required zones mapped. +*/ + ti->zone_reset_all_supported = true; +if (!ti->type->iterate_devices || ti->type->iterate_devices (ti, device_get_zone_resource_limits, &zlim)) {
@@ -420,3 +430,39 @@ void dm_zone_endio(struct dm_io *io, struct bio *clone)return ; } + +static int dm_zone_need_reset_cb (struct blk_zone *zone,unsigned int idx, +void *data) +{ +/* +* For an all-zones reset, ignore conventional, empty, read-only +* and offline zones. +*/ +switch (zone->cond) { +case BLK_ZONE_COND_NOT_WP : +case BLK_ZONE_COND_EMPTY : +case BLK_ZONE_COND_READONLY : +case BLK_ZONE_COND_OFFLINE : +return 0 ; +default : +set_bit (idx, (unsigned long *)data); +return 0 ; + } +} + +int dm_zone_get_reset_bitmap (struct mapped_device *md,struct dm_table *t, +sector_t sector,unsigned int nr_zones, +unsigned long *need_reset) +{ +int ret; + + ret =dm_blk_do_report_zones (md, t, sector, nr_zones, + dm_zone_need_reset_cb, need_reset); +if (ret != nr_zones) { +DMERR ("Get %s zone reset bitmap failed\n" , + md->disk->disk_name); +return -EIO ; + } + +return 0 ; +}
drivers/md/dm.c +132 -3
165 lines not on this page · raw message
drivers/md/dm.h +3 -0
@@ -110,6 +110,9 @@ int dm_blk_report_zones(struct gendisk *disk, sector_t sector, unsigned int nr_zones, report_zones_cb cb, void *data);bool dm_is_zone_write (struct mapped_device *md,struct bio *bio);int dm_zone_map_bio (struct dm_target_io *io); +int dm_zone_get_reset_bitmap (struct mapped_device *md,struct dm_table *t, +sector_t sector,unsigned int nr_zones, +unsigned long *need_reset);#else #define dm_blk_report_zones NULLstatic inline bool dm_is_zone_write (struct mapped_device *md,struct bio *bio)
include/linux/device-mapper.h +7 -0
@@ -357,6 +357,13 @@ struct dm_target {*/ bool discards_supported:1; +/* +* Automatically set by dm-core if this target supports +* REQ_OP_ZONE_RESET_ALL. Otherwise, this operation will be emulated +* using REQ_OP_ZONE_RESET. Target drivers must not set this manually. +*/ + bool zone_reset_all_supported:1; +/* * Set if this target requires that discards be split on * 'max_discard_sectors' boundaries.
Signature · 1 line
2.45.2
Damien Le Moal
maintainerPATCH v2 4/5 block: Remove REQ_OP_ZONE_RESET_ALL emulation
In reply to Damien Le MoalNow that device mapper can handle resetting all zones of a mapped zoned device using REQ_OP_ZONE_RESET_ALL, all zoned block device drivers support this operation. With this, the request queue feature BLK_FEAT_ZONE_RESETALL is not necessary and the emulation code in blk-zone.c can be removed.
--- block/blk-core.c | 5 +-- block/blk-zoned.c | 76 ++-------------------------------- drivers/block/null_blk/zoned.c | 2 +- drivers/block/ublk_drv.c | 2 +- drivers/block/virtio_blk.c | 2 +- drivers/nvme/host/zns.c | 2 +- drivers/scsi/sd_zbc.c | 2 +- include/linux/blkdev.h | 5 --- 8 files changed, 9 insertions(+), 87 deletions(-)
block/blk-core.c +1 -4
@@ -830,11 +830,8 @@ void submit_bio_noacct(struct bio *bio)case REQ_OP_ZONE_OPEN :case REQ_OP_ZONE_CLOSE :case REQ_OP_ZONE_FINISH : -if (!bdev_is_zoned (bio->bi_bdev)) - goto not_supported; -break ;case REQ_OP_ZONE_RESET_ALL : -if (!bdev_is_zoned (bio->bi_bdev) || !blk_queue_zone_resetall (q)) +if (!bdev_is_zoned (bio->bi_bdev)) goto not_supported;break ;case REQ_OP_DRV_IN :
block/blk-zoned.c +3 -73
96 lines not on this page · raw message
drivers/block/null_blk/zoned.c +1 -1
@@ -164,7 +164,7 @@ int null_init_zoned_dev(struct nullb_device *dev, sector += dev->zone_size_sects; } - lim->features |=BLK_FEAT_ZONED |BLK_FEAT_ZONE_RESETALL ; + lim->features |=BLK_FEAT_ZONED ; lim->chunk_sectors = dev->zone_size_sects; lim->max_zone_append_sectors = dev->zone_append_max_sectors; lim->max_open_zones = dev->zone_max_open;
drivers/block/ublk_drv.c +1 -1
@@ -2194,7 +2194,7 @@ static int ublk_ctrl_start_dev(struct ublk_device *ub, struct io_uring_cmd *cmd)if (!IS_ENABLED (CONFIG_BLK_DEV_ZONED ))return -EOPNOTSUPP ; - lim.features |=BLK_FEAT_ZONED |BLK_FEAT_ZONE_RESETALL ; + lim.features |=BLK_FEAT_ZONED ; lim.max_active_zones = p->max_active_zones; lim.max_open_zones = p->max_open_zones; lim.max_zone_append_sectors = p->max_zone_append_sectors;
drivers/block/virtio_blk.c +1 -1
@@ -728,7 +728,7 @@ static int virtblk_read_zoned_limits(struct virtio_blk *vblk,dev_dbg (&vdev->dev,"probing host-managed zoned device\n" ); - lim->features |=BLK_FEAT_ZONED |BLK_FEAT_ZONE_RESETALL ; + lim->features |=BLK_FEAT_ZONED ; virtio_cread(vdev, struct virtio_blk_config, zoned.max_open_zones, &v);
drivers/nvme/host/zns.c +1 -1
@@ -108,7 +108,7 @@ int nvme_query_zone_info(struct nvme_ns *ns, unsigned lbaf,void nvme_update_zone_info (struct nvme_ns *ns,struct queue_limits *lim,struct nvme_zone_info *zi) { - lim->features |=BLK_FEAT_ZONED |BLK_FEAT_ZONE_RESETALL ; + lim->features |=BLK_FEAT_ZONED ; lim->max_open_zones = zi->max_open_zones; lim->max_active_zones = zi->max_active_zones; lim->max_zone_append_sectors = ns->ctrl->max_zone_append;
drivers/scsi/sd_zbc.c +1 -1
@@ -599,7 +599,7 @@ int sd_zbc_read_zones(struct scsi_disk *sdkp, struct queue_limits *lim,if (sdkp->device->type !=TYPE_ZBC )return 0 ; - lim->features |=BLK_FEAT_ZONED |BLK_FEAT_ZONE_RESETALL ; + lim->features |=BLK_FEAT_ZONED ;/* * Per ZBC and ZAC specifications, writes in sequential write required
include/linux/blkdev.h +0 -5
@@ -318,9 +318,6 @@ typedef unsigned int __bitwise blk_features_t;/* is a zoned device */ #define BLK_FEAT_ZONED ((__force blk_features_t)(1u << 10)) -/* supports Zone Reset All */ -#define BLK_FEAT_ZONE_RESETALL ((__force blk_features_t)(1u << 11)) -/* supports PCI(e) p2p requests */ #define BLK_FEAT_PCI_P2PDMA ((__force blk_features_t)(1u << 12))
@@ -618,8 +615,6 @@ void blk_queue_flag_clear(unsigned int flag, struct request_queue *q); test_bit(QUEUE_FLAG_NOXMERGES, &(q)->queue_flags)#define blk_queue_nonrot (q) (!((q)->limits.features & BLK_FEAT_ROTATIONAL))#define blk_queue_io_stat (q) ((q)->limits.features & BLK_FEAT_IO_STAT) -#define blk_queue_zone_resetall (q) \ - ((q)->limits.features & BLK_FEAT_ZONE_RESETALL)#define blk_queue_dax (q) ((q)->limits.features & BLK_FEAT_DAX)#define blk_queue_pci_p2pdma (q) ((q)->limits.features & BLK_FEAT_PCI_P2PDMA)#ifdef CONFIG_BLK_RQ_ALLOC_TIME
Signature · 1 line
2.45.2
Damien Le Moal
maintainerPATCH v2 5/5 block: Remove blk_alloc_zone_bitmap()
In reply to Damien Le MoalRemove the helper function blk_alloc_zone_bitmap() and replace its single call site with a call to bitmap_alloc(). To be consistent with this change, use bitmap_free() to free a disk convnetional zone bitmap.
--- block/blk-zoned.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
block/blk-zoned.c +2 -10
@@ -150,13 +150,6 @@ int blkdev_report_zones(struct block_device *bdev, sector_t sector, }EXPORT_SYMBOL_GPL (blkdev_report_zones); -static inline unsigned long *blk_alloc_zone_bitmap (int node, -unsigned int nr_zones) -{ -return kcalloc_node (BITS_TO_LONGS (nr_zones),sizeof (unsigned long ), -GFP_NOIO , node); -} -static int blkdev_zone_reset_all (struct block_device *bdev) {struct bio bio;
@@ -1485,7 +1478,7 @@ void disk_free_zone_resources(struct gendisk *disk)mempool_destroy (disk->zone_wplugs_pool); disk->zone_wplugs_pool =NULL ; -kfree (disk->conv_zones_bitmap); +bitmap_free (disk->conv_zones_bitmap); disk->conv_zones_bitmap =NULL ; disk->zone_capacity =0 ; disk->last_zone_capacity =0 ;
@@ -1607,7 +1600,6 @@ static int blk_revalidate_conv_zone(struct blk_zone *zone, unsigned int idx, struct blk_revalidate_zone_args *args) {struct gendisk *disk = args->disk; -struct request_queue *q = disk->queue;if (zone->capacity != zone->len) { pr_warn("%s: Invalid conventional zone capacity\n",
@@ -1623,7 +1615,7 @@ static int blk_revalidate_conv_zone(struct blk_zone *zone, unsigned int idx,if (!args->conv_zones_bitmap) { args->conv_zones_bitmap = -blk_alloc_zone_bitmap (q->node, args->nr_zones); +bitmap_zalloc (args->nr_zones,GFP_NOIO );if (!args->conv_zones_bitmap)return -ENOMEM ; }
Signature · 1 line
2.45.2
Looks good,
Damien,
Quoted 11 lines
Here is a set of patches based on
block/for-nextto remove the emulation for zone reset all from the block layer and move it to device mapper. This is done because device mapper is the only zoned device driver that does not natively supportREQ_OP_ZONE_RESET_ALL. With this change, the emulation that may be required depending on a mapped device zone mapping is moved to device mapper and the reset all feature (BLK_FEAT_ZONE_RESETALL) can be deleted, as well as all the code handling this feature inblk-zoned.c. The DM-based handling ofREQ_OP_ZONE_RESET_ALLcan also be much faster than the block layer emulation as that operation can be forwarded as is to targets mapping all sequential write required zones.
LGTM.
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering
Quoted 14 lines · Damien Le Moal
Jens, Mike,
Here is a set of patches based on
block/for-nextto remove the emulation for zone reset all from the block layer and move it to device mapper. This is done because device mapper is the only zoned device driver that does not natively supportREQ_OP_ZONE_RESET_ALL. With this change, the emulation that may be required depending on a mapped device zone mapping is moved to device mapper and the reset all feature (BLK_FEAT_ZONE_RESETALL) can be deleted, as well as all the code handling this feature inblk-zoned.c. The DM-based handling ofREQ_OP_ZONE_RESET_ALLcan also be much faster than the block layer emulation as that operation can be forwarded as is to targets mapping all sequential write required zones.
Quoted 1 line
[...]
Applied, thanks!
[1/5] null_blk: Introduce the zone_full parameter
commit: f4d5dc33c823ef1d7ccbbd2d1e40b871fad0012b
[2/5] dm: Refactor is_abnormal_io()
commit: ae7e965b36e3132238d16b4ccd223f65162397b5
[3/5] dm: handle REQ_OP_ZONE_RESET_ALL
commit: 81e7706345f06e1e97a092f59697b7e20a0ee868
[4/5] block: Remove REQ_OP_ZONE_RESET_ALL emulation
commit: f2a7bea23710fceb99dac6da4ef82c3cc8932f7f
[5/5] block: Remove blk_alloc_zone_bitmap()
commit: 2f20872ed43185780a5f30581472599342c86d4aBest regards,
Signature · 1 line
Jens Axboe