iOS and Apple Silicon host support
Subsystems (3)
v6 cover iOS and Apple Silicon host support
Based-on: [email protected] ([PATCH v4 00/43] Mirror map JIT memory for TCG)
These set of changes brings QEMU TCG to iOS devices and future Apple Silicon devices. They were originally developed last year and have been working in the UTM app. Recently, we ported the changes to master, re-wrote a lot of the build script changes for meson, and broke up the patches into more distinct units.
A summary of the changes:
* CONFIG_IOS defined when building for iOS and iOS specific changes (as well as unsupported code) are gated behind it.
* A new dependency, libucontext is added since iOS does not have native ucontext and broken support for sigaltstack. libucontext is available as a new option for coroutine backend.
Since v6:
* Dropped the Apple Silicon JIT support patch (superseded by another patchset)
* Changed libucontext to be a Meson subproject
* Cache availablity check for preadv/pwritev on macOS 11 and iOS 14
Since v5:
* Fixed some more instances of QAPI define of CONFIG_HOST_BLOCK_DEVICE
* Fixed libucontext build on newer version of GCC
Since v4:
* Updated QAPI schema for CONFIG_HOST_BLOCK_DEVICE
* Updated maintainers file for iOS host support
* Moved system() changes to osdep.h
* Fixed typo in libucontext meson.build change
Since v3:
* Moved mirror JIT support to a different patch set.
* Removed dependency on pthread_jit_write_protect_np because it was redundent and also crashes if called on a non-jailbroken iOS device.
* Removed --enable-cross-compile option
* Fixed checkpatch errors
* Fixed iOS build on master due to new test recently added which calls system()
Since v2:
* Changed getting mirror pointer from a macro to inline functions
* Split constification of TCG code pointers to separate patch
* Removed slirp updates (will send future patch once slirp changes are in)
* Removed shared library patch (will send future patch)
-j
Joelle van Dyne (7):
configure: option to disable host block devices
configure: cross-compiling with empty cross_prefix
qemu: add support for iOS host
coroutine: add libucontext as external library
configure: cross compile should use x86_64 cpu_family
slirp: update build flags for iOS resolv fix
block: check availablity for preadv/pwritev on mac
docs/devel/index.rst | 1 + docs/devel/ios.rst | 28 ++++++++++++++ configure | 81 +++++++++++++++++++++++++++++++++++---- meson.build | 16 +++++++- qapi/block-core.json | 10 +++-- include/qemu/osdep.h | 11 ++++++ block.c | 2 +- block/file-posix.c | 74 ++++++++++++++++++++++++++++------- net/slirp.c | 16 ++++---- qga/commands-posix.c | 6 +++ util/coroutine-ucontext.c | 9 +++++ .gitmodules | 3 ++ MAINTAINERS | 7 ++++ meson_options.txt | 2 + subprojects/libucontext | 1 + tests/qtest/meson.build | 7 ++-- 16 files changed, 236 insertions(+), 38 deletions(-) create mode 100644 docs/devel/ios.rst create mode 160000 subprojects/libucontext
Signature · 1 line
2.28.0
PATCH v6 1/7 configure: option to disable host block devices
Some hosts (iOS) have a sandboxed filesystem and do not provide low-level APIs for interfacing with host block devices.
--- configure | 4 ++++ meson.build | 1 + qapi/block-core.json | 10 +++++++--- block/file-posix.c | 10 +++++++++- 4 files changed, 21 insertions(+), 4 deletions(-)
configure +4 -0
@@ -451,6 +451,7 @@ skip_meson=no gettext="" fuse="auto" fuse_lseek="auto" +host_block_device_support="yes" bogus_os="no" malloc_trim="auto"
@@ -5937,6 +5938,9 @@ if test "$default_devices" = "yes" ; then else echo "CONFIG_MINIKCONF_MODE=--allnoconfig" >> $config_host_mak fi +if test "$host_block_device_support" = "yes" ; then + echo "CONFIG_HOST_BLOCK_DEVICE=y" >> $config_host_mak +fi if test "$debug_tcg" = "yes" ; then echo "CONFIG_DEBUG_TCG=y" >> $config_host_mak fi
meson.build +1 -0
@@ -2219,6 +2219,7 @@ summary_info += {'vvfat support': config_host.has_key('CONFIG_VVFAT')} summary_info += {'qed support': config_host.has_key('CONFIG_QED')} summary_info += {'parallels support': config_host.has_key('CONFIG_PARALLELS')} summary_info += {'sheepdog support': config_host.has_key('CONFIG_SHEEPDOG')} +summary_info += {'host block dev support': config_host.has_key('CONFIG_HOST_BLOCK_DEVICE')} summary_info += {'capstone': capstone_opt == 'disabled' ? false : capstone_opt} summary_info += {'libpmem support': config_host.has_key('CONFIG_LIBPMEM')} summary_info += {'libdaxctl support': config_host.has_key('CONFIG_LIBDAXCTL')}
qapi/block-core.json +7 -3
@@ -959,7 +959,8 @@ 'discriminator': 'driver', 'data': { 'file': 'BlockStatsSpecificFile', - 'host_device': 'BlockStatsSpecificFile', + 'host_device': { 'type': 'BlockStatsSpecificFile', + 'if': 'defined(CONFIG_HOST_BLOCK_DEVICE)' }, 'nvme': 'BlockStatsSpecificNvme' } } ##
@@ -2827,7 +2828,9 @@ { 'enum': 'BlockdevDriver', 'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs', 'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', - 'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi', + 'gluster', 'host_cdrom', + {'name': 'host_device', 'if': 'defined(CONFIG_HOST_BLOCK_DEVICE)' }, + 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd', { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
@@ -3995,7 +3998,8 @@ 'ftps': 'BlockdevOptionsCurlFtps', 'gluster': 'BlockdevOptionsGluster', 'host_cdrom': 'BlockdevOptionsFile', - 'host_device':'BlockdevOptionsFile', + 'host_device': { 'type': 'BlockdevOptionsFile', + 'if': 'defined(CONFIG_HOST_BLOCK_DEVICE)' }, 'http': 'BlockdevOptionsCurlHttp', 'https': 'BlockdevOptionsCurlHttps', 'iscsi': 'BlockdevOptionsIscsi',
block/file-posix.c +9 -1
@@ -41,7 +41,7 @@#include "scsi/pr-manager.h" #include "scsi/constants.h" -#if defined(__APPLE__) && (__MACH__) +#if defined(CONFIG_HOST_BLOCK_DEVICE ) && defined(__APPLE__) && (__MACH__)#include <paths.h> #include <sys/param.h> #include <IOKit/IOKitLib.h>
@@ -3014,6 +3014,7 @@ static BlockStatsSpecific *raw_get_specific_stats(BlockDriverState *bs)return stats; } +#if defined(CONFIG_HOST_BLOCK_DEVICE )static BlockStatsSpecific *hdev_get_specific_stats (BlockDriverState *bs) {BlockStatsSpecific *stats =g_new (BlockStatsSpecific,1 );
@@ -3023,6 +3024,7 @@ static BlockStatsSpecific *hdev_get_specific_stats(BlockDriverState *bs)return stats; } +#endif /* CONFIG_HOST_BLOCK_DEVICE */ static QemuOptsList raw_create_opts = { .name ="raw-create-opts" ,
@@ -3247,6 +3249,8 @@ BlockDriver bdrv_file = {/***********************************************/ /* host device */ +#if defined(CONFIG_HOST_BLOCK_DEVICE ) +#if defined(__APPLE__) && defined(__MACH__)static kern_return_t GetBSDPath (io_iterator_t mediaIterator,char *bsdPath,CFIndex maxPathSize,int flags);
@@ -3872,6 +3876,8 @@ static BlockDriver bdrv_host_cdrom = { };#endif /* __FreeBSD__ */ +#endif /* CONFIG_HOST_BLOCK_DEVICE */ +static void bdrv_file_init (void ) {/*
@@ -3879,6 +3885,7 @@ static void bdrv_file_init(void)* registered last will get probed first. */ bdrv_register (&bdrv_file); +#if defined(CONFIG_HOST_BLOCK_DEVICE )bdrv_register (&bdrv_host_device);#ifdef __linux__bdrv_register (&bdrv_host_cdrom);
@@ -3886,6 +3893,7 @@ static void bdrv_file_init(void)#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__)bdrv_register (&bdrv_host_cdrom);#endif +#endif /* CONFIG_HOST_BLOCK_DEVICE */ }block_init (bdrv_file_init);
Signature · 1 line
2.28.0
PATCH v6 2/7 configure: cross-compiling with empty cross_prefix
In reply to Joelle van DyneThe iOS toolchain does not use the host prefix naming convention. So we need to enable cross-compile options while allowing the PREFIX to be blank.
--- configure | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
configure +4 -2
@@ -234,6 +234,7 @@ cpu="" iasl="iasl" interp_prefix="/usr/gnemul/qemu-%M" static="no" +cross_compile="no" cross_prefix="" audio_drv_list="" block_drv_rw_whitelist=""
@@ -461,6 +462,7 @@ for opt do optarg=$(expr "x$opt" : 'x[^=]*=\(.*\)') case "$opt" in --cross-prefix=*) cross_prefix="$optarg" + cross_compile="yes" ;; --cc=*) CC="$optarg" ;;
@@ -1674,7 +1676,7 @@ $(echo Deprecated targets: $deprecated_targets_list | \ --target-list-exclude=LIST exclude a set of targets from the default target-list Advanced options (experts only): - --cross-prefix=PREFIX use PREFIX for compile tools [$cross_prefix] + --cross-prefix=PREFIX use PREFIX for compile tools, PREFIX can be blank [$cross_prefix] --cc=CC use C compiler CC [$cc] --iasl=IASL use ACPI compiler IASL [$iasl] --host-cc=CC use C compiler CC [$host_cc] for code run at
@@ -6963,7 +6965,7 @@ if has $sdl2_config; then fi echo "strip = [$(meson_quote $strip)]" >> $cross echo "windres = [$(meson_quote $windres)]" >> $cross -if test -n "$cross_prefix"; then +if test "$cross_compile" = "yes"; then cross_arg="--cross-file config-meson.cross" echo "[host_machine]" >> $cross if test "$mingw32" = "yes" ; then
Signature · 1 line
2.28.0
PATCH v6 3/7 qemu: add support for iOS host
In reply to Joelle van DyneThis introduces support for building for iOS hosts. When the correct Xcode toolchain is used, iOS host will be detected automatically.
* block: disable features not supported by iOS sandbox
* slirp: disable SMB features for iOS
* osdep: disable system() calls for iOS
--- docs/devel/index.rst | 1 + docs/devel/ios.rst | 28 +++++++++++++++++++++++++++ configure | 43 ++++++++++++++++++++++++++++++++++++++++- meson.build | 2 +- include/qemu/osdep.h | 11 +++++++++++ block.c | 2 +- block/file-posix.c | 31 +++++++++++++++++------------ net/slirp.c | 16 +++++++-------- qga/commands-posix.c | 6 ++++++ MAINTAINERS | 7 +++++++ tests/qtest/meson.build | 7 +++---- 11 files changed, 127 insertions(+), 27 deletions(-) create mode 100644 docs/devel/ios.rst
docs/devel/index.rst +1 -0
@@ -35,3 +35,4 @@ Contents: clocks qom block-coroutine-wrapper + ios
docs/devel/ios.rst new file +28 -0
@@ -0,0 +1,28 @@ +=========== +iOS Support +=========== + +To run qemu on the iOS platform, some modifications were required. Most of the +modifications are conditioned on the ``CONFIG_IOS`` and configuration variable. + +Build support +------------- + +For the code to compile, certain changes in the block driver and the slirp +driver had to be made. There is no ``system()`` call, so it has been replaced +with an assertion error. There should be no code path that call system() from +iOS. + +``ucontext`` support is broken on iOS. The implementation from ``libucontext`` +is used instead. + +JIT support +----------- + +On iOS, allocating RWX pages require special entitlements not usually granted to +apps. However, it is possible to use `bulletproof JIT`_ with a development +certificate. This means that we need to allocate one chunk of memory with RX +permissions and then mirror map the same memory with RW permissions. We generate +code to the mirror mapping and execute the original mapping. + +.. _bulletproof JIT: https://www.blackhat.com/docs/us-16/materials/us-16-Krstic.pdf
configure +42 -1
79 lines not on this page · raw message
meson.build +1 -1
@@ -181,7 +181,7 @@ if targetos == 'windows' include_directories: include_directories('.')) elif targetos == 'darwin' coref = dependency('appleframeworks', modules: 'CoreFoundation') - iokit = dependency('appleframeworks', modules: 'IOKit') + iokit = dependency('appleframeworks', modules: 'IOKit', required: 'CONFIG_IOS' not in config_host) cocoa = dependency('appleframeworks', modules: 'Cocoa', required: get_option('cocoa')) elif targetos == 'sunos' socket = [cc.find_library('socket'),
include/qemu/osdep.h +11 -0
@@ -686,4 +686,15 @@ char *qemu_get_host_name(Error **errp);*/ size_t qemu_get_host_physmem (void ); +/** +* iOS does not support system() so we replace it with an assertion failure. +*/ +#ifdef CONFIG_IOS +#define system ios_does_not_support_system +static inline int ios_does_not_support_system (const char *command) +{ +assert (0 ); +} +#endif /* CONFIG_IOS */ +#endif
block.c +1 -1
@@ -54,7 +54,7 @@#ifdef CONFIG_BSD #include <sys/ioctl.h> #include <sys/queue.h> -#ifndef __DragonFly__ +#if !defined(__DragonFly__) && !defined(CONFIG_IOS )#include <sys/disk.h> #endif #endif
block/file-posix.c +19 -12
@@ -181,7 +181,17 @@ typedef struct BDRVRawReopenState {bool check_cache_dropped; } BDRVRawReopenState; -static int fd_open (BlockDriverState *bs); +static int fd_open (BlockDriverState *bs) +{ +BDRVRawState *s = bs->opaque; + +/* this is just to ensure s->fd is sane (its called by io ops) */ +if (s->fd >=0 ) { +return 0 ; + } +return -EIO ; +} +static int64_t raw_getlength (BlockDriverState *bs); typedef struct RawPosixAIOData {
@@ -252,6 +262,12 @@ static int raw_normalize_devicepath(const char **filename, Error **errp) }#endif +#if defined(CONFIG_IOS ) +static int probe_logical_blocksize (int fd,unsigned int *sector_size_p) +{ +return -ENOTSUP ;/* not supported on iOS */ +} +#else /* CONFIG_IOS */ /* * Get logical block size via ioctl. On success store it in @sector_size_p. */
@@ -284,6 +300,7 @@ static int probe_logical_blocksize(int fd, unsigned int *sector_size_p) return success ? 0 : -errno; } +#endif /* !CONFIG_IOS */ /** * Get physical block size of @fd.
@@ -2307,7 +2324,7 @@ again: } if (size == 0)#endif -#if defined(__APPLE__) && defined(__MACH__) +#if !defined(CONFIG_IOS ) && defined(__APPLE__) && defined(__MACH__) {uint64_t sectors =0 ;uint32_t sector_size =0 ;
@@ -3543,16 +3560,6 @@ hdev_co_ioctl(BlockDriverState *bs, unsigned long int req, void *buf) }#endif /* linux */ -static int fd_open (BlockDriverState *bs) -{ -BDRVRawState *s = bs->opaque; - -/* this is just to ensure s->fd is sane (its called by io ops) */ -if (s->fd >=0 ) -return 0 ; -return -EIO ; -} - static coroutine_fn inthdev_co_pdiscard (BlockDriverState *bs,int64_t offset,int bytes) {
net/slirp.c +8 -8
@@ -27,7 +27,7 @@#include "net/slirp.h" -#ifndef _WIN32 +#if !defined(_WIN32) && !defined(CONFIG_IOS )#include <pwd.h> #include <sys/wait.h> #endif
@@ -90,7 +90,7 @@ typedef struct SlirpState {Slirp *slirp;Notifier poll_notifier;Notifier exit_notifier; -#ifndef _WIN32 +#if !defined(_WIN32) && !defined(CONFIG_IOS )gchar *smb_dir;#endif GSList *fwd;
@@ -103,7 +103,7 @@ static QTAILQ_HEAD(, SlirpState) slirp_stacks =static int slirp_hostfwd (SlirpState *s,const char *redir_str,Error **errp);static int slirp_guestfwd (SlirpState *s,const char *config_str,Error **errp); -#ifndef _WIN32 +#if !defined(_WIN32) && !defined(CONFIG_IOS )static int slirp_smb (SlirpState *s,const char *exported_dir,struct in_addr vserver_addr,Error **errp);static void slirp_smb_cleanup (SlirpState *s);
@@ -368,7 +368,7 @@ static int net_slirp_init(NetClientState *peer, const char *model,struct in6_addr ip6_prefix;struct in6_addr ip6_host;struct in6_addr ip6_dns; -#ifndef _WIN32 +#if !defined(_WIN32) && !defined(CONFIG_IOS )struct in_addr smbsrv = { .s_addr =0 };#endif NetClientState *nc;
@@ -478,7 +478,7 @@ static int net_slirp_init(NetClientState *peer, const char *model,return -1 ; } -#ifndef _WIN32 +#if !defined(_WIN32) && !defined(CONFIG_IOS )if (vsmbserver && !inet_aton (vsmbserver, &smbsrv)) {error_setg (errp,"Failed to parse SMB address" );return -1 ;
@@ -593,7 +593,7 @@ static int net_slirp_init(NetClientState *peer, const char *model, } } } -#ifndef _WIN32 +#if !defined(_WIN32) && !defined(CONFIG_IOS )if (smb_export) {if (slirp_smb (s, smb_export, smbsrv, errp) <0 ) { goto error;
@@ -785,7 +785,7 @@ void hmp_hostfwd_add(Monitor *mon, const QDict *qdict) } -#ifndef _WIN32 +#if !defined(_WIN32) && !defined(CONFIG_IOS )/* automatic user mode samba server configuration */ static void slirp_smb_cleanup (SlirpState *s)
@@ -900,7 +900,7 @@ static int slirp_smb(SlirpState* s, const char *exported_dir, return 0; } -#endif /* !defined(_WIN32) */ +#endif /* !defined(_WIN32) && !defined(CONFIG_IOS) */ static int guestfwd_can_read (void *opaque) {
qga/commands-posix.c +6 -0
@@ -34,6 +34,12 @@#ifndef CONFIG_HAS_ENVIRON #ifdef __APPLE__ +#include "TargetConditionals.h" +#if !TARGET_OS_IPHONE && !TARGET_IPHONE_SIMULATOR +#define APPLE_USE_CRT_EXTERNS +#endif +#endif +#ifdef APPLE_USE_CRT_EXTERNS #include <crt_externs.h> #define environ (*_NSGetEnviron())#else
MAINTAINERS +7 -0
@@ -538,6 +538,13 @@ F: include/*/*win32* X: qga/*win32* F: qemu.nsi +IOS +M: Joelle van Dyne <[email protected]> +S: Maintained +K: ^Subject:.*(?i)iOS +F: docs/devel/ios.rst +F: include/tcg/tcg-apple-jit.h + Alpha Machines -------------- M: Richard Henderson <[email protected]>
tests/qtest/meson.build +3 -4
@@ -46,12 +46,11 @@ qtests_i386 = \ (config_all_devices.has_key('CONFIG_TPM_TIS_ISA') ? ['tpm-tis-test'] : []) + \ (config_all_devices.has_key('CONFIG_TPM_TIS_ISA') ? ['tpm-tis-swtpm-test'] : []) + \ (config_all_devices.has_key('CONFIG_RTL8139_PCI') ? ['rtl8139-test'] : []) + \ + (not config_host.has_key('CONFIG_IOS') ? ['bios-tables-test', 'hd-geo-test'] : []) + \ qtests_pci + \ ['fdc-test', 'ide-test', - 'hd-geo-test', 'boot-order-test', - 'bios-tables-test', 'rtc-test', 'i440fx-test', 'fuzz-test',
@@ -148,9 +147,9 @@ qtests_arm = \ 'boot-serial-test', 'hexloader-test'] -# TODO: once aarch64 TCG is fixed on ARM 32 bit host, make bios-tables-test unconditional +# TODO: once aarch64 TCG is fixed on ARM 32 bit host, make bios-tables-test unconditional (except on iOS) qtests_aarch64 = \ - (cpu != 'arm' ? ['bios-tables-test'] : []) + \ + (cpu != 'arm' and not config_host.has_key('CONFIG_IOS') ? ['bios-tables-test'] : []) + \ (config_all_devices.has_key('CONFIG_TPM_TIS_SYSBUS') ? ['tpm-tis-device-test'] : []) + \ (config_all_devices.has_key('CONFIG_TPM_TIS_SYSBUS') ? ['tpm-tis-device-swtpm-test'] : []) + \ ['arm-cpu-features',
Signature · 1 line
2.28.0
Re: [PATCH v6 3/7] qemu: add support for iOS host
Hi Joelle,
Quoted 46 lines · Joelle van Dyne
This introduces support for building for iOS hosts. When the correct Xcode toolchain is used, iOS host will be detected automatically.
* block: disable features not supported by iOS sandbox
* slirp: disable SMB features for iOS
* osdep: disablesystem()calls for iOSSigned-off-by Joelle van Dyne <[email protected]>--- docs/devel/index.rst | 1 + docs/devel/ios.rst | 28 +++++++++++++++++++++++++++ configure | 43 ++++++++++++++++++++++++++++++++++++++++- meson.build | 2 +- include/qemu/osdep.h | 11 +++++++++++ block.c | 2 +- block/file-posix.c | 31 +++++++++++++++++------------ net/slirp.c | 16 +++++++-------- qga/commands-posix.c | 6 ++++++ MAINTAINERS | 7 +++++++ tests/qtest/meson.build | 7 +++---- 11 files changed, 127 insertions(+), 27 deletions(-) create mode 100644 docs/devel/ios.rstdocs/devel/index.rst +1
-0@@ -35,3 +35,4 @@ Contents: clocks qom block-coroutine-wrapper + iosdocs/devel/ios.rst new file +7
-0@@ -0,0 +1,28 @@ +=========== +iOS Support +=========== + +To run qemu on the iOS platform, some modifications were required. Most of the +modifications are conditioned on the ``CONFIG_IOS`` and configuration variable. +
docs/devel/ios.rst line 17
Quoted 10 lines
docs/devel/ios.rst new file +10
-0+Build support +------------- + +For the code to compile, certain changes in the block driver and the slirp +driver had to be made. There is no ``system()`` call, so it has been replaced +with an assertion error. There should be no code path that call system() from +iOS. + +``ucontext`` support is broken on iOS. The implementation from ``libucontext`` +is used instead.
Do you have a CI testing plan for these builds?
Is it possible to add a Gitlab-CI job? If not, on Cirrus-CI?
Thanks,
Phil.
Re: [PATCH v6 3/7] qemu: add support for iOS host
I think this was discussed before but the main issue right now is that there's no packaging system (like homebrew) for iOS cross building on Mac. Stefan suggested caching built libraries in an external location to use with the CI. I think this can be done but would require some thought to it. I am open to other ideas as well (like pretending to build for iOS while actually building for macOS).
-j
Quoted 70 lines · Philippe Mathieu-Daudé
Hi Joelle,
Quoted 46 lines · Joelle van Dyne
This introduces support for building for iOS hosts. When the correct Xcode toolchain is used, iOS host will be detected automatically.
* block: disable features not supported by iOS sandbox
* slirp: disable SMB features for iOS
* osdep: disablesystem()calls for iOSSigned-off-by Joelle van Dyne <[email protected]>--- docs/devel/index.rst | 1 + docs/devel/ios.rst | 28 +++++++++++++++++++++++++++ configure | 43 ++++++++++++++++++++++++++++++++++++++++- meson.build | 2 +- include/qemu/osdep.h | 11 +++++++++++ block.c | 2 +- block/file-posix.c | 31 +++++++++++++++++------------ net/slirp.c | 16 +++++++-------- qga/commands-posix.c | 6 ++++++ MAINTAINERS | 7 +++++++ tests/qtest/meson.build | 7 +++---- 11 files changed, 127 insertions(+), 27 deletions(-) create mode 100644 docs/devel/ios.rstdocs/devel/index.rst +1
-0@@ -35,3 +35,4 @@ Contents: clocks qom block-coroutine-wrapper + iosdocs/devel/ios.rst new file +7
-0@@ -0,0 +1,28 @@ +=========== +iOS Support +=========== + +To run qemu on the iOS platform, some modifications were required. Most of the +modifications are conditioned on the ``CONFIG_IOS`` and configuration variable. +docs/devel/ios.rst line 17
Quoted 10 lines
docs/devel/ios.rst new file +10
-0+Build support +------------- + +For the code to compile, certain changes in the block driver and the slirp +driver had to be made. There is no ``system()`` call, so it has been replaced +with an assertion error. There should be no code path that call system() from +iOS. + +``ucontext`` support is broken on iOS. The implementation from ``libucontext`` +is used instead.Do you have a CI testing plan for these builds?
Is it possible to add a Gitlab-CI job? If not, on Cirrus-CI?
Thanks,
Phil.
Re: [PATCH v6 3/7] qemu: add support for iOS host
In reply to Joelle van DyneQuoted 39 lines · Joelle van Dyne
This introduces support for building for iOS hosts. When the correct Xcode toolchain is used, iOS host will be detected automatically.
* block: disable features not supported by iOS sandbox
* slirp: disable SMB features for iOS
* osdep: disablesystem()calls for iOSSigned-off-by Joelle van Dyne <[email protected]>--- docs/devel/index.rst | 1 + docs/devel/ios.rst | 28 +++++++++++++++++++++++++++ configure | 43 ++++++++++++++++++++++++++++++++++++++++- meson.build | 2 +- include/qemu/osdep.h | 11 +++++++++++ block.c | 2 +- block/file-posix.c | 31 +++++++++++++++++------------ net/slirp.c | 16 +++++++-------- qga/commands-posix.c | 6 ++++++ MAINTAINERS | 7 +++++++ tests/qtest/meson.build | 7 +++---- 11 files changed, 127 insertions(+), 27 deletions(-) create mode 100644 docs/devel/ios.rstdocs/devel/index.rst +1
-0@@ -35,3 +35,4 @@ Contents: clocks qom block-coroutine-wrapper + iosdocs/devel/ios.rst new file +0
-0
docs/devel/ios.rst line 5
Quoted 5 lines
docs/devel/ios.rst new file +5
-0@@ -0,0 +1,28 @@ +=========== +iOS Support +=========== + +To run qemu on the iOS platform, some modifications were required. Most of the
QEMU is upper-cased.
docs/devel/ios.rst line 13
Quoted 8 lines
docs/devel/ios.rst new file +8
-0+modifications are conditioned on the ``CONFIG_IOS`` and configuration variable. + +Build support +------------- + +For the code to compile, certain changes in the block driver and the slirp +driver had to be made. There is no ``system()`` call, so it has been replaced +with an assertion error. There should be no code path that call system() from
"calls"
docs/devel/ios.rst line 22
Quoted 9 lines
docs/devel/ios.rst new file +9
-0+iOS. + +``ucontext`` support is broken on iOS. The implementation from ``libucontext`` +is used instead. + +JIT support +----------- + +On iOS, allocating RWX pages require special entitlements not usually granted to
"requires"
Quoted 33 lines
docs/devel/ios.rst new file +6
-0+apps. However, it is possible to use `bulletproof JIT`_ with a development +certificate. This means that we need to allocate one chunk of memory with RX +permissions and then mirror map the same memory with RW permissions. We generate +code to the mirror mapping and execute the original mapping. + +.. _bulletproof JIT: https://www.blackhat.com/docs/us-16/materials/us-16-Krstic.pdfconfigure +13
-0@@ -560,6 +560,19 @@ EOF compile_object } +check_ios() { + cat > $TMPC <<EOF +#ifdef __APPLE__ +#import "TargetConditionals.h" +#if !TARGET_OS_IPHONE +#error TARGET_OS_IPHONE not true +#endif +#endif +int main(void) { return 0; } +EOF + compile_object +} + check_include() { cat > $TMPC <<EOF #include <$1>@@ -602,7 +615,11 @@ elif check_define __DragonFly__ ; then elif check_define __NetBSD__; then targetos='NetBSD'
configure elif check_define __DragonFly__ ; then · line 625
Quoted 10 lines
configure +5
-1elif check_define __APPLE__; then - targetos='Darwin' + if check_ios ; then + targetos='iOS' + else + targetos='Darwin' + fi else # This is a fatal error, but don't report it yet, because we # might be going to just print the --help text, or it might
So here targetos=iOS and targetos=Darwin are separate things...
configure if test "$cross_compile" = "yes"; then · line 7017
Quoted 7 lines
configure +3
-0@@ -6974,6 +7012,9 @@ if test "$cross_compile" = "yes"; then if test "$linux" = "yes" ; then echo "system = 'linux'" >> $cross fi + if test "$darwin" = "yes" ; then + echo "system = 'darwin'" >> $cross + fi
...so why is this needed if we're not "darwin", but "iOS"...
Quoted 8 lines
configure +0
-0case "$ARCH" in i386|x86_64) echo "cpu_family = 'x86'" >> $crossmeson.build +0
-0
meson.build if targetos == 'windows' · line 187
Quoted 8 lines
meson.build +1
-1@@ -181,7 +181,7 @@ if targetos == 'windows' include_directories: include_directories('.')) elif targetos == 'darwin' coref = dependency('appleframeworks', modules: 'CoreFoundation') - iokit = dependency('appleframeworks', modules: 'IOKit') + iokit = dependency('appleframeworks', modules: 'IOKit', required: 'CONFIG_IOS' not in config_host) cocoa = dependency('appleframeworks', modules: 'Cocoa', required: get_option('cocoa')) elif targetos == 'sunos' socket = [cc.find_library('socket'),
...and here ios seems to be a subtype of darwin, not a different kind of targetos. That's a bit confusing. Maybe this is Meson's fault ?
Quoted 25 lines
include/qemu/osdep.h +11
-0@@ -686,4 +686,15 @@ char *qemu_get_host_name(Error **errp);*/ size_t qemu_get_host_physmem (void ); +/** +* iOS does not support system() so we replace it with an assertion failure. +*/ +#ifdef CONFIG_IOS +#define system ios_does_not_support_system +static inline int ios_does_not_support_system (const char *command) +{ +assert (0 ); +} +#endif /* CONFIG_IOS */ +#endif block.c +0
-0
block.c line 59
Quoted 7 lines
block.c +1
-1@@ -54,7 +54,7 @@#ifdef CONFIG_BSD #include <sys/ioctl.h> #include <sys/queue.h> -#ifndef __DragonFly__ +#if !defined(__DragonFly__) && !defined(CONFIG_IOS )#include <sys/disk.h> #endif
Instead of extending this ifdef of OSes, can we add a meson.build test for "does sys/disk.h exist" and then use #ifdef HAVE_SYS_DISK_H ? (HAVE_PTY_H is probably a good example to crib from.)
This is part of a general preference we have for not testing "are we building for host OS Foo" if we can test the specific property we care about instead.
Quoted 85 lines
block.c +0
-0#endif block/file-posix.c +19
-12@@ -181,7 +181,17 @@ typedef struct BDRVRawReopenState {bool check_cache_dropped; } BDRVRawReopenState; -static int fd_open (BlockDriverState *bs); +static int fd_open (BlockDriverState *bs) +{ +BDRVRawState *s = bs->opaque; + +/* this is just to ensure s->fd is sane (its called by io ops) */ +if (s->fd >=0 ) { +return 0 ; + } +return -EIO ; +} +static int64_t raw_getlength (BlockDriverState *bs); typedef struct RawPosixAIOData {@@ -252,6 +262,12 @@ static int raw_normalize_devicepath(const char **filename, Error **errp) }#endif +#if defined(CONFIG_IOS ) +static int probe_logical_blocksize (int fd,unsigned int *sector_size_p) +{ +return -ENOTSUP ;/* not supported on iOS */ +} +#else /* CONFIG_IOS */ /* * Get logical block size via ioctl. On success store it in @sector_size_p. */ @@ -284,6 +300,7 @@ static int probe_logical_blocksize(int fd, unsigned int *sector_size_p) return success ? 0 : -errno; } +#endif /* !CONFIG_IOS */ /** * Get physical block size of @fd. @@ -2307,7 +2324,7 @@ again: } if (size == 0)#endif -#if defined(__APPLE__) && defined(__MACH__) +#if !defined(CONFIG_IOS ) && defined(__APPLE__) && defined(__MACH__) {uint64_t sectors =0 ;uint32_t sector_size =0 ;@@ -3543,16 +3560,6 @@ hdev_co_ioctl(BlockDriverState *bs, unsigned long int req, void *buf) }#endif /* linux */ -static int fd_open (BlockDriverState *bs) -{ -BDRVRawState *s = bs->opaque; - -/* this is just to ensure s->fd is sane (its called by io ops) */ -if (s->fd >=0 ) -return 0 ; -return -EIO ; -} - static coroutine_fn inthdev_co_pdiscard (BlockDriverState *bs,int64_t offset,int bytes) {net/slirp.c +1
-1@@ -27,7 +27,7 @@#include "net/slirp.h" -#ifndef _WIN32 +#if !defined(_WIN32) && !defined(CONFIG_IOS )#include <pwd.h> #include <sys/wait.h> #endif
net/slirp.c typedef struct SlirpState { · line 96
Quoted 8 lines
net/slirp.c +1
-1@@ -90,7 +90,7 @@ typedef struct SlirpState {Slirp *slirp;Notifier poll_notifier;Notifier exit_notifier; -#ifndef _WIN32 +#if !defined(_WIN32) && !defined(CONFIG_IOS )gchar *smb_dir;#endif GSList *fwd;
This sort of ifdeffery should probably also be changed to test what it really cares about rather than just adding another host OS type to the list.
qga/commands-posix.c
Quoted 4 lines
qga/commands-posix.c +0
-0
Does it even make sense to build the guest agent for iOS ? It's not like you can run iOS inside QEMU...
Quoted 19 lines
tests/qtest/meson.build +1
-2@@ -46,12 +46,11 @@ qtests_i386 = \ (config_all_devices.has_key('CONFIG_TPM_TIS_ISA') ? ['tpm-tis-test'] : []) + \ (config_all_devices.has_key('CONFIG_TPM_TIS_ISA') ? ['tpm-tis-swtpm-test'] : []) + \ (config_all_devices.has_key('CONFIG_RTL8139_PCI') ? ['rtl8139-test'] : []) + \ + (not config_host.has_key('CONFIG_IOS') ? ['bios-tables-test', 'hd-geo-test'] : []) + \ qtests_pci + \ ['fdc-test', 'ide-test', - 'hd-geo-test', 'boot-order-test', - 'bios-tables-test', 'rtc-test', 'i440fx-test', 'fuzz-test',
tests/qtest/meson.build qtests_arm = \ · line 150
Quoted 5 lines
tests/qtest/meson.build +1
-1@@ -148,9 +147,9 @@ qtests_arm = \ 'boot-serial-test', 'hexloader-test'] -# TODO: once aarch64 TCG is fixed on ARM 32 bit host, make bios-tables-test unconditional +# TODO: once aarch64 TCG is fixed on ARM 32 bit host, make bios-tables-test unconditional (except on iOS)
So, why is bios-tables-test not something to run if we're building for iOS? If there's a good reason for the exclusion the comment should say what it is.
Quoted 8 lines
tests/qtest/meson.build +1
-2qtests_aarch64 = \ - (cpu != 'arm' ? ['bios-tables-test'] : []) + \ + (cpu != 'arm' and not config_host.has_key('CONFIG_IOS') ? ['bios-tables-test'] : []) + \ (config_all_devices.has_key('CONFIG_TPM_TIS_SYSBUS') ? ['tpm-tis-device-test'] : []) + \ (config_all_devices.has_key('CONFIG_TPM_TIS_SYSBUS') ? ['tpm-tis-device-swtpm-test'] : []) + \ ['arm-cpu-features', --2.28.0
thanks
-- PMM
Re: [PATCH v6 3/7] qemu: add support for iOS host
Quoted 118 lines · Peter Maydell
Quoted 39 lines · Joelle van Dyne
This introduces support for building for iOS hosts. When the correct Xcode toolchain is used, iOS host will be detected automatically.
* block: disable features not supported by iOS sandbox
* slirp: disable SMB features for iOS
* osdep: disablesystem()calls for iOSSigned-off-by Joelle van Dyne <[email protected]>--- docs/devel/index.rst | 1 + docs/devel/ios.rst | 28 +++++++++++++++++++++++++++ configure | 43 ++++++++++++++++++++++++++++++++++++++++- meson.build | 2 +- include/qemu/osdep.h | 11 +++++++++++ block.c | 2 +- block/file-posix.c | 31 +++++++++++++++++------------ net/slirp.c | 16 +++++++-------- qga/commands-posix.c | 6 ++++++ MAINTAINERS | 7 +++++++ tests/qtest/meson.build | 7 +++---- 11 files changed, 127 insertions(+), 27 deletions(-) create mode 100644 docs/devel/ios.rstdocs/devel/index.rst +1
-0@@ -35,3 +35,4 @@ Contents: clocks qom block-coroutine-wrapper + iosdocs/devel/ios.rst new file +0
-0docs/devel/ios.rst line 5
Quoted 5 lines
docs/devel/ios.rst new file +5
-0@@ -0,0 +1,28 @@ +=========== +iOS Support +=========== + +To run qemu on the iOS platform, some modifications were required. Most of theQEMU is upper-cased.
docs/devel/ios.rst line 13
Quoted 8 lines
docs/devel/ios.rst new file +8
-0+modifications are conditioned on the ``CONFIG_IOS`` and configuration variable. + +Build support +------------- + +For the code to compile, certain changes in the block driver and the slirp +driver had to be made. There is no ``system()`` call, so it has been replaced +with an assertion error. There should be no code path that call system() from"calls"
docs/devel/ios.rst line 22
Quoted 9 lines
docs/devel/ios.rst new file +9
-0+iOS. + +``ucontext`` support is broken on iOS. The implementation from ``libucontext`` +is used instead. + +JIT support +----------- + +On iOS, allocating RWX pages require special entitlements not usually granted to"requires"
Quoted 33 lines
docs/devel/ios.rst new file +6
-0+apps. However, it is possible to use `bulletproof JIT`_ with a development +certificate. This means that we need to allocate one chunk of memory with RX +permissions and then mirror map the same memory with RW permissions. We generate +code to the mirror mapping and execute the original mapping. + +.. _bulletproof JIT: https://www.blackhat.com/docs/us-16/materials/us-16-Krstic.pdfconfigure +13
-0@@ -560,6 +560,19 @@ EOF compile_object } +check_ios() { + cat > $TMPC <<EOF +#ifdef __APPLE__ +#import "TargetConditionals.h" +#if !TARGET_OS_IPHONE +#error TARGET_OS_IPHONE not true +#endif +#endif +int main(void) { return 0; } +EOF + compile_object +} + check_include() { cat > $TMPC <<EOF #include <$1>@@ -602,7 +615,11 @@ elif check_define __DragonFly__ ; then elif check_define __NetBSD__; then targetos='NetBSD'configure elif check_define __DragonFly__ ; then · line 625
Quoted 10 lines
configure +5
-1elif check_define __APPLE__; then - targetos='Darwin' + if check_ios ; then + targetos='iOS' + else + targetos='Darwin' + fi else # This is a fatal error, but don't report it yet, because we # might be going to just print the --help text, or it mightSo here targetos=iOS and targetos=Darwin are separate things...
Quoted 9 lines
configure if test "$cross_compile" = "yes"; then · line 7017
Quoted 7 lines
configure +3
-0@@ -6974,6 +7012,9 @@ if test "$cross_compile" = "yes"; then if test "$linux" = "yes" ; then echo "system = 'linux'" >> $cross fi + if test "$darwin" = "yes" ; then + echo "system = 'darwin'" >> $cross + fi...so why is this needed if we're not "darwin", but "iOS"...
iOS and macOS being treated the same works in 99% of the cases which is why this patch is relatively small. For the 1% of time the two systems behave differently, I added CONFIG_IOS. It's a bit of a hack, but the alternative is to include <TargetConditionals.h> and check for "TARGET_OS_IPHONE" (which is how it's usually done).
Quoted 12 lines
Quoted 12 lines
configure +0
-0case "$ARCH" in i386|x86_64) echo "cpu_family = 'x86'" >> $crossmeson.build +0
-0@@ -181,7 +181,7 @@ if targetos == 'windows' include_directories: include_directories('.')) elif targetos == 'darwin' coref = dependency('appleframeworks', modules: 'CoreFoundation')
Quoted 8 lines
meson.build if targetos == 'windows' · line 187
Quoted 5 lines
meson.build +1
-1- iokit = dependency('appleframeworks', modules: 'IOKit') + iokit = dependency('appleframeworks', modules: 'IOKit', required: 'CONFIG_IOS' not in config_host) cocoa = dependency('appleframeworks', modules: 'Cocoa', required: get_option('cocoa')) elif targetos == 'sunos' socket = [cc.find_library('socket'),...and here ios seems to be a subtype of darwin, not a different kind of targetos. That's a bit confusing. Maybe this is Meson's fault ?
Meson recognizes 'darwin' only for both macOS and iOS.
Quoted 29 lines
Quoted 29 lines
include/qemu/osdep.h +11
-0@@ -686,4 +686,15 @@ char *qemu_get_host_name(Error **errp);*/ size_t qemu_get_host_physmem (void ); +/** +* iOS does not support system() so we replace it with an assertion failure. +*/ +#ifdef CONFIG_IOS +#define system ios_does_not_support_system +static inline int ios_does_not_support_system (const char *command) +{ +assert (0 ); +} +#endif /* CONFIG_IOS */ +#endif block.c +0
-0@@ -54,7 +54,7 @@#ifdef CONFIG_BSD #include <sys/ioctl.h> #include <sys/queue.h>
Quoted 8 lines
block.c line 59
Quoted 4 lines
block.c +1
-1-#ifndef __DragonFly__ +#if !defined(__DragonFly__) && !defined(CONFIG_IOS )#include <sys/disk.h> #endif Instead of extending this ifdef of OSes, can we add a meson.build test for "does
sys/disk.hexist" and then use #ifdefHAVE_SYS_DISK_H? (HAVE_PTY_His probably a good example to crib from.)
Can be done.
Quoted 94 lines
This is part of a general preference we have for not testing "are we building for host OS Foo" if we can test the specific property we care about instead.
Quoted 89 lines
block.c +0
-0#endif block/file-posix.c +19
-12@@ -181,7 +181,17 @@ typedef struct BDRVRawReopenState {bool check_cache_dropped; } BDRVRawReopenState; -static int fd_open (BlockDriverState *bs); +static int fd_open (BlockDriverState *bs) +{ +BDRVRawState *s = bs->opaque; + +/* this is just to ensure s->fd is sane (its called by io ops) */ +if (s->fd >=0 ) { +return 0 ; + } +return -EIO ; +} +static int64_t raw_getlength (BlockDriverState *bs); typedef struct RawPosixAIOData {@@ -252,6 +262,12 @@ static int raw_normalize_devicepath(const char **filename, Error **errp) }#endif +#if defined(CONFIG_IOS ) +static int probe_logical_blocksize (int fd,unsigned int *sector_size_p) +{ +return -ENOTSUP ;/* not supported on iOS */ +} +#else /* CONFIG_IOS */ /* * Get logical block size via ioctl. On success store it in @sector_size_p. */ @@ -284,6 +300,7 @@ static int probe_logical_blocksize(int fd, unsigned int *sector_size_p) return success ? 0 : -errno; } +#endif /* !CONFIG_IOS */ /** * Get physical block size of @fd. @@ -2307,7 +2324,7 @@ again: } if (size == 0)#endif -#if defined(__APPLE__) && defined(__MACH__) +#if !defined(CONFIG_IOS ) && defined(__APPLE__) && defined(__MACH__) {uint64_t sectors =0 ;uint32_t sector_size =0 ;@@ -3543,16 +3560,6 @@ hdev_co_ioctl(BlockDriverState *bs, unsigned long int req, void *buf) }#endif /* linux */ -static int fd_open (BlockDriverState *bs) -{ -BDRVRawState *s = bs->opaque; - -/* this is just to ensure s->fd is sane (its called by io ops) */ -if (s->fd >=0 ) -return 0 ; -return -EIO ; -} - static coroutine_fn inthdev_co_pdiscard (BlockDriverState *bs,int64_t offset,int bytes) {net/slirp.c +1
-2@@ -27,7 +27,7 @@#include "net/slirp.h" -#ifndef _WIN32 +#if !defined(_WIN32) && !defined(CONFIG_IOS )#include <pwd.h> #include <sys/wait.h> #endif @@ -90,7 +90,7 @@ typedef struct SlirpState {Slirp *slirp;Notifier poll_notifier;Notifier exit_notifier; -#ifndef _WIN32
Quoted 8 lines
net/slirp.c typedef struct SlirpState { · line 96
Quoted 4 lines
net/slirp.c +1
-0+#if !defined(_WIN32) && !defined(CONFIG_IOS )gchar *smb_dir;#endif GSList *fwd;This sort of ifdeffery should probably also be changed to test what it really cares about rather than just adding another host OS type to the list.
I can refactor to test for SMB support and replace the host OS checks.
Quoted 8 lines
qga/commands-posix.c
Quoted 5 lines
qga/commands-posix.c +0
-0Does it even make sense to build the guest agent for iOS ? It's not like you can run iOS inside QEMU...
Not currently no, but do we preclude the option?
Quoted 23 lines
Quoted 23 lines
tests/qtest/meson.build +1
-2@@ -46,12 +46,11 @@ qtests_i386 = \ (config_all_devices.has_key('CONFIG_TPM_TIS_ISA') ? ['tpm-tis-test'] : []) + \ (config_all_devices.has_key('CONFIG_TPM_TIS_ISA') ? ['tpm-tis-swtpm-test'] : []) + \ (config_all_devices.has_key('CONFIG_RTL8139_PCI') ? ['rtl8139-test'] : []) + \ + (not config_host.has_key('CONFIG_IOS') ? ['bios-tables-test', 'hd-geo-test'] : []) + \ qtests_pci + \ ['fdc-test', 'ide-test', - 'hd-geo-test', 'boot-order-test', - 'bios-tables-test', 'rtc-test', 'i440fx-test', 'fuzz-test',@@ -148,9 +147,9 @@ qtests_arm = \ 'boot-serial-test', 'hexloader-test']
Quoted 5 lines
tests/qtest/meson.build qtests_arm = \ · line 150
Quoted 2 lines
tests/qtest/meson.build +1
-1-# TODO: once aarch64 TCG is fixed on ARM 32 bit host, make bios-tables-test unconditional +# TODO: once aarch64 TCG is fixed on ARM 32 bit host, make bios-tables-test unconditional (except on iOS)So, why is bios-tables-test not something to run if we're building for iOS? If there's a good reason for the exclusion the comment should say what it is.
Actually this was left over from a previous version of the patch where I excluded all tests that use system(). Now calling system() will just fail the test so there's no reason to exclude these anymore.
-j
Quoted 12 lines
Quoted 9 lines
tests/qtest/meson.build +1
-2qtests_aarch64 = \ - (cpu != 'arm' ? ['bios-tables-test'] : []) + \ + (cpu != 'arm' and not config_host.has_key('CONFIG_IOS') ? ['bios-tables-test'] : []) + \ (config_all_devices.has_key('CONFIG_TPM_TIS_SYSBUS') ? ['tpm-tis-device-test'] : []) + \ (config_all_devices.has_key('CONFIG_TPM_TIS_SYSBUS') ? ['tpm-tis-device-swtpm-test'] : []) + \ ['arm-cpu-features', --2.28.0
thanks
-- PMM
Re: [PATCH v6 3/7] qemu: add support for iOS host
In reply to Joelle van DyneQuoted 10 lines · Joelle van Dyne
This introduces support for building for iOS hosts. When the correct Xcode toolchain is used, iOS host will be detected automatically.
* block: disable features not supported by iOS sandbox
* slirp: disable SMB features for iOS
* osdep: disablesystem()calls for iOS+``ucontext`` support is broken on iOS. The implementation from ``libucontext`` +is used instead.
Just a note since it came up in another thread today, but looking at libucontext its aarch64 backend doesn't handle the floating point registers. I think if the *context routines don't save/restore the callee-saves fp regs (v8-v15, FPCR) then it's liable to result in tricky-to-track down bugs where some kept-in-a-callee-saves-fp-register data from a function further up the callstack gets corrupted, depending on what the compiler happens to do. It would be good to work with the libucontext maintainers to add that functionality.
thanks
-- PMM
Re: [PATCH v6 3/7] qemu: add support for iOS host
(Cc'ing Ariadne, libucontext maintainer)
Quoted 12 lines · Peter Maydell
Quoted 10 lines · Joelle van Dyne
This introduces support for building for iOS hosts. When the correct Xcode toolchain is used, iOS host will be detected automatically.
* block: disable features not supported by iOS sandbox
* slirp: disable SMB features for iOS
* osdep: disablesystem()calls for iOS+``ucontext`` support is broken on iOS. The implementation from ``libucontext`` +is used instead.
Quoted 9 lines
Just a note since it came up in another thread today, but looking at libucontext its aarch64 backend doesn't handle the floating point registers. I think if the *context routines don't save/restore the callee-saves fp regs (v8-v15, FPCR) then it's liable to result in tricky-to-track down bugs where some kept-in-a-callee-saves-fp-register data from a function further up the callstack gets corrupted, depending on what the compiler happens to do. It would be good to work with the libucontext maintainers to add that functionality.
Per https://github.com/kaniini/libucontext/blob/master/README.md#caveats
this is a design choice:
Only basic GPR registers are saved and restored when context swapping. The glibc implementation uses hardware capability detection to save/restore other register groups, such as the FPU registers or vector processing (AltiVec/AVX/NEON) registers. Adding this capability detection would significantly increase the complexity of the project and thus is not implemented. Support for compiling in code to save/restore FPU registers or vector registers may be added in a later release as a build-time setting -- for now, we assume a soft-float ABI with no optional processor features.
Minor update in 2022:
https://github.com/kaniini/libucontext/commit/5244775fb93ab9
This is a work in progress, as newer compilers will spill even non-floating-point state through floating point registers when allowed to do so.
Regards,
Phil.
PATCH v6 4/7 coroutine: add libucontext as external library
In reply to Joelle van DyneiOS does not support ucontext natively for aarch64 and the sigaltstack is also unsupported (even worse, it fails silently, see:
https://openradar.appspot.com/13002712 )
As a workaround we include a library implementation of ucontext and add it as a build option.
--- configure | 23 ++++++++++++++++++++--- meson.build | 11 ++++++++++- util/coroutine-ucontext.c | 9 +++++++++ .gitmodules | 3 +++ meson_options.txt | 2 ++ subprojects/libucontext | 1 + 6 files changed, 45 insertions(+), 4 deletions(-) create mode 160000 subprojects/libucontext
configure +20 -3
@@ -1767,7 +1767,7 @@ Advanced options (experts only): --oss-lib path to OSS library --cpu=CPU Build for host CPU [$cpu] --with-coroutine=BACKEND coroutine backend. Supported options: - ucontext, sigaltstack, windows + ucontext, libucontext, sigaltstack, windows --enable-gcov enable test coverage analysis with gcov --disable-blobs disable installing provided firmware blobs --with-vss-sdk=SDK-path enable Windows VSS support in QEMU Guest Agent
@@ -4913,6 +4913,8 @@ if test "$coroutine" = ""; then coroutine=win32 elif test "$ucontext_works" = "yes"; then coroutine=ucontext + elif test "$ios" = "yes"; then + coroutine=libucontext else coroutine=sigaltstack fi
@@ -4936,12 +4938,27 @@ else error_exit "only the 'windows' coroutine backend is valid for Windows" fi ;; + libucontext) + ;; *) error_exit "unknown coroutine backend $coroutine" ;; esac fi +case $coroutine in +libucontext) + git_submodules="${git_submodules} subprojects/libucontext" + mkdir -p libucontext + coroutine_impl=ucontext + libucontext="enabled" + ;; +*) + coroutine_impl=$coroutine + libucontext="disabled" + ;; +esac + if test "$coroutine_pool" = ""; then coroutine_pool=yes fi
@@ -6487,7 +6504,7 @@ if test "$rbd" = "yes" ; then echo "RBD_LIBS=$rbd_libs" >> $config_host_mak fi -echo "CONFIG_COROUTINE_BACKEND=$coroutine" >> $config_host_mak +echo "CONFIG_COROUTINE_BACKEND=$coroutine_impl" >> $config_host_mak if test "$coroutine_pool" = "yes" ; then echo "CONFIG_COROUTINE_POOL=1" >> $config_host_mak else
@@ -7069,7 +7086,7 @@ NINJA=$ninja $meson setup \ -Dcocoa=$cocoa -Dmpath=$mpath -Dsdl=$sdl -Dsdl_image=$sdl_image \ -Dvnc=$vnc -Dvnc_sasl=$vnc_sasl -Dvnc_jpeg=$vnc_jpeg -Dvnc_png=$vnc_png \ -Dgettext=$gettext -Dxkbcommon=$xkbcommon -Du2f=$u2f -Dvirtiofsd=$virtiofsd \ - -Dcapstone=$capstone -Dslirp=$slirp -Dfdt=$fdt \ + -Dcapstone=$capstone -Dslirp=$slirp -Dfdt=$fdt -Ducontext=$libucontext \ -Diconv=$iconv -Dcurses=$curses -Dlibudev=$libudev\ -Ddocs=$docs -Dsphinx_build=$sphinx_build -Dinstall_blobs=$blobs \ -Dvhost_user_blk_server=$vhost_user_blk_server \
meson.build +10 -1
@@ -1308,9 +1308,17 @@ if not fdt.found() and fdt_required.length() > 0 error('fdt not available but required by targets ' + ', '.join(fdt_required)) endif +ucontext = dependency('libucontext', required : false) +if not ucontext.found() and get_option('ucontext').enabled() + libucontext_proj = subproject('libucontext', + default_options: ['freestanding=true']) + ucontext = libucontext_proj.get_variable('libucontext_dep') +endif + config_host_data.set('CONFIG_CAPSTONE', capstone.found()) config_host_data.set('CONFIG_FDT', fdt.found()) config_host_data.set('CONFIG_SLIRP', slirp.found()) +config_host_data.set('CONFIG_LIBUCONTEXT', ucontext.found()) ##################### # Generated sources #
@@ -1528,7 +1536,7 @@ util_ss.add_all(trace_ss) util_ss = util_ss.apply(config_all, strict: false) libqemuutil = static_library('qemuutil', sources: util_ss.sources() + stub_ss.sources() + genh, - dependencies: [util_ss.dependencies(), m, glib, socket, malloc]) + dependencies: [util_ss.dependencies(), m, glib, socket, malloc, ucontext]) qemuutil = declare_dependency(link_with: libqemuutil, sources: genh + version_res)
@@ -2188,6 +2196,7 @@ if targetos == 'windows' summary_info += {'QGA MSI support': config_host.has_key('CONFIG_QGA_MSI')} endif summary_info += {'seccomp support': config_host.has_key('CONFIG_SECCOMP')} +summary_info += {'libucontext support': ucontext.found()} summary_info += {'coroutine backend': config_host['CONFIG_COROUTINE_BACKEND']} summary_info += {'coroutine pool': config_host['CONFIG_COROUTINE_POOL'] == '1'} summary_info += {'debug stack usage': config_host.has_key('CONFIG_DEBUG_STACK_USAGE')}
util/coroutine-ucontext.c +9 -0
@@ -23,7 +23,16 @@#undef _FORTIFY_SOURCE#endif #include "qemu/osdep.h" +#if defined(CONFIG_LIBUCONTEXT ) +#include <libucontext/libucontext.h> +#define ucontext_t libucontext_ucontext_t +#define getcontext libucontext_getcontext +#define setcontext libucontext_setcontext +#define swapcontext libucontext_swapcontext +#define makecontext libucontext_makecontext +#else #include <ucontext.h> +#endif #include "qemu/coroutine_int.h" #ifdef CONFIG_VALGRIND_H
.gitmodules +3 -0
@@ -64,3 +64,6 @@ [submodule "roms/vbootrom"] path = roms/vbootrom url = https://git.qemu.org/git/vbootrom.git +[submodule "libucontext"] + path = subprojects/libucontext + url = https://github.com/utmapp/libucontext.git
meson_options.txt +2 -0
@@ -70,6 +70,8 @@ option('fuse', type: 'feature', value: 'auto', description: 'FUSE block device export') option('fuse_lseek', type : 'feature', value : 'auto', description: 'SEEK_HOLE/SEEK_DATA support for FUSE exports') +option('ucontext', type : 'feature', value : 'disabled', + description: 'libucontext support') option('capstone', type: 'combo', value: 'auto', choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
subprojects/libucontext new file +1 -0
@@ -0,0 +1 @@ +Subproject commit fc3b5e724e21c2c01e2c93b344ab0e34b5a6649b
Signature · 1 line
2.28.0
Re: [PATCH v6 4/7] coroutine: add libucontext as external library
Quoted 23 lines · Joelle van Dyne
iOS does not support ucontext natively for aarch64 and the sigaltstack is also unsupported (even worse, it fails silently, see:
https://openradar.appspot.com/13002712 )As a workaround we include a library implementation of ucontext and add it as a build option.
Signed-off-by Joelle van Dyne <[email protected]>--- configure | 23 ++++++++++++++++++++--- meson.build | 11 ++++++++++- util/coroutine-ucontext.c | 9 +++++++++ .gitmodules | 3 +++ meson_options.txt | 2 ++ subprojects/libucontext | 1 + 6 files changed, 45 insertions(+), 4 deletions(-) create mode 160000 subprojects/libucontext.gitmodules +0
-0
.gitmodules line 69
Quoted 6 lines
.gitmodules +3
-0@@ -64,3 +64,6 @@ [submodule "roms/vbootrom"] path = roms/vbootrom url = https://git.qemu.org/git/vbootrom.git +[submodule "libucontext"] + path = subprojects/libucontext + url = https://github.com/utmapp/libucontext.git
Using libucontext looks like a good idea to me, but I noticed that this is a pointing to a fork of the main libucontext project at
https://github.com/kaniini/libucontext
The main project appears 100's of commits ahead of the utmapp fork
What is in the utmapp fork that isn't present in the primary libucontext repo ? I think if we're going to use libucontext we will want to point to the primary project, and this means anything custom in the fork will need to get submitted upstream. Maybe you've already started doing that making this a non-issue ?
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v6 4/7] coroutine: add libucontext as external library
Yes since the submission of this patch, we have been working with the original authors to merge all the changes upstream. The next patch should point to the original repo.
-j
Quoted 56 lines · Daniel P. Berrangé
Quoted 5 lines · Joelle van Dyne
iOS does not support ucontext natively for aarch64 and the sigaltstack is also unsupported (even worse, it fails silently, see:
https://openradar.appspot.com/13002712 )As a workaround we include a library implementation of ucontext and add
it
Quoted 18 lines
as a build option.
Signed-off-by Joelle van Dyne <[email protected]>--- configure | 23 ++++++++++++++++++++--- meson.build | 11 ++++++++++- util/coroutine-ucontext.c | 9 +++++++++ .gitmodules | 3 +++ meson_options.txt | 2 ++ subprojects/libucontext | 1 + 6 files changed, 45 insertions(+), 4 deletions(-) create mode 160000 subprojects/libucontext.gitmodules +0
-0.gitmodules line 69
Quoted 6 lines
.gitmodules +3
-0@@ -64,3 +64,6 @@ [submodule "roms/vbootrom"] path = roms/vbootrom url = https://git.qemu.org/git/vbootrom.git +[submodule "libucontext"] + path = subprojects/libucontext + url = https://github.com/utmapp/libucontext.gitUsing libucontext looks like a good idea to me, but I noticed that this is a pointing to a fork of the main libucontext project at
https://github.com/kaniini/libucontext
The main project appears 100's of commits ahead of the utmapp fork
What is in the utmapp fork that isn't present in the primary libucontext repo ? I think if we're going to use libucontext we will want to point to the primary project, and this means anything custom in the fork will need to get submitted upstream. Maybe you've already started doing that making this a non-issue ?
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/
dberrange :|
|: https://libvirt.org -o-
https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/
dberrange :|
PATCH v6 5/7 configure: cross compile should use x86_64 cpu_family
In reply to Joelle van Dyne--- configure | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
configure +4 -1
@@ -7033,9 +7033,12 @@ if test "$cross_compile" = "yes"; then echo "system = 'darwin'" >> $cross fi case "$ARCH" in - i386|x86_64) + i386) echo "cpu_family = 'x86'" >> $cross ;; + x86_64) + echo "cpu_family = 'x86_64'" >> $cross + ;; ppc64le) echo "cpu_family = 'ppc64'" >> $cross ;;
Signature · 1 line
2.28.0
Re: [PATCH v6 5/7] configure: cross compile should use x86_64 cpu_family
Quoted 14 lines · Joelle van Dyne
Signed-off-by Joelle van Dyne <[email protected]>--- configure | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)configure +0
-0@@ -7033,9 +7033,12 @@ if test "$cross_compile" = "yes"; then echo "system = 'darwin'" >> $cross fi case "$ARCH" in
configure if test "$cross_compile" = "yes"; then · line 7044
Quoted 10 lines
configure +4
-1- i386|x86_64) + i386) echo "cpu_family = 'x86'" >> $cross ;; + x86_64) + echo "cpu_family = 'x86_64'" >> $cross + ;; ppc64le) echo "cpu_family = 'ppc64'" >> $cross ;;
PATCH v6 6/7 slirp: update build flags for iOS resolv fix
In reply to Joelle van DyneA future libslirp update will use libresolv on Darwin systems, so we add the flags in QEMU build now.
--- meson.build | 2 ++ 1 file changed, 2 insertions(+)
meson.build +2 -0
@@ -1205,6 +1205,8 @@ if have_system slirp_deps = [] if targetos == 'windows' slirp_deps = cc.find_library('iphlpapi') + elif targetos == 'darwin' + slirp_deps = cc.find_library('resolv') endif slirp_conf = configuration_data() slirp_conf.set('SLIRP_MAJOR_VERSION', meson.project_version().split('.')[0])
Signature · 1 line
2.28.0
Re: [PATCH v6 6/7] slirp: update build flags for iOS resolv fix
Hi Joelle,
Quoted 14 lines · Joelle van Dyne
A future libslirp update will use libresolv on Darwin systems, so we add the flags in QEMU build now.
Reviewed-by Stefan Hajnoczi <[email protected]>Signed-off-by Joelle van Dyne <[email protected]>--- meson.build | 2 ++ 1 file changed, 2 insertions(+)meson.build +0
-0
meson.build if have_system · line 1209
Quoted 5 lines
meson.build +2
-0@@ -1205,6 +1205,8 @@ if have_system slirp_deps = [] if targetos == 'windows' slirp_deps = cc.find_library('iphlpapi') + elif targetos == 'darwin' + slirp_deps = cc.find_library('resolv')
This is done in libSLiRP commit 9f82a47 ("Add DNS resolving for iOS").
Maybe this commit need to be backported in the stable branch, or QEMU should use a newer commit. Cc'ing Samuel and Marc-André who maintain the project (and the SLiRP list).
Quoted 4 lines
meson.build +0
-0endif slirp_conf = configuration_data() slirp_conf.set('SLIRP_MAJOR_VERSION', meson.project_version().split('.')[0])
Re: [Slirp] [PATCH v6 6/7] slirp: update build flags for iOS resolv fix
Hi
Quoted 26 lines · Philippe Mathieu-Daudé
Hi Joelle,
Quoted 1 line · Joelle van Dyne
A future libslirp update will use libresolv on Darwin systems, so we add
the
Quoted 13 lines
flags in QEMU build now.
Reviewed-by Stefan Hajnoczi <[email protected]>Signed-off-by Joelle van Dyne <[email protected]>--- meson.build | 2 ++ 1 file changed, 2 insertions(+)meson.build +0
-0meson.build if have_system · line 1209
Quoted 5 lines
meson.build +2
-0@@ -1205,6 +1205,8 @@ if have_system slirp_deps = [] if targetos == 'windows' slirp_deps = cc.find_library('iphlpapi') + elif targetos == 'darwin' + slirp_deps = cc.find_library('resolv')This is done in libSLiRP commit 9f82a47 ("Add DNS resolving for iOS").
Quoted 4 lines
Maybe this commit need to be backported in the stable branch, or QEMU should use a newer commit. Cc'ing Samuel and Marc-André who maintain the project (and the SLiRP list).
Yes, I am going to send an update to git upstream.
thanks
Signature · 1 line
Marc-André Lureau
PATCH v6 7/7 block: check availablity for preadv/pwritev on mac
In reply to Joelle van DynemacOS 11/iOS 14 added preadv/pwritev APIs. Due to weak linking, configure will succeed with CONFIG_PREADV even when targeting a lower OS version. We therefore need to check at run time if we can actually use these APIs.
--- block/file-posix.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
block/file-posix.c +33 -0
@@ -1390,17 +1390,50 @@ static int handle_aiocb_flush(void *opaque)#ifdef CONFIG_PREADV static bool preadv_present = true; +static bool preadv_checked;static ssize_t qemu_preadv (int fd,const struct iovec *iov,int nr_iov,off_t offset) { +#ifdef CONFIG_DARWIN /* preadv introduced in macOS 11 */ +if (unlikely (!preadv_checked)) { + if (__builtin_available(macOS 11, iOS 14, watchOS 7, tvOS 14, *)) { + preadv_checked = true; + }else { + preadv_present = false; +return -ENOSYS ; + } + } +/* Now we suppress the availability warning since we use the cached check */ +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wunguarded-availability-new" +return preadv (fd, iov, nr_iov, offset); +#pragma clang diagnostic pop +#else /* CONFIG_DARWIN */ return preadv (fd, iov, nr_iov, offset); +#endif }static ssize_t qemu_pwritev (int fd,const struct iovec *iov,int nr_iov,off_t offset) { +#ifdef CONFIG_DARWIN /* preadv introduced in macOS 11 */ +if (unlikely (!preadv_checked)) { + if (__builtin_available(macOS 11, iOS 14, watchOS 7, tvOS 14, *)) { + preadv_checked = true; + }else { + preadv_present = false; +return -ENOSYS ; + } + } +/* Now we suppress the availability warning since we use the cached check */ +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wunguarded-availability-new" +return pwritev (fd, iov, nr_iov, offset); +#pragma clang diagnostic pop +#else /* CONFIG_DARWIN */ return pwritev (fd, iov, nr_iov, offset); +#endif }#else
Signature · 1 line
2.28.0
Quoted 17 lines · Joelle van Dyne
Based-on: [email protected] ([PATCH v4 00/43] Mirror map JIT memory for TCG)
These set of changes brings QEMU TCG to iOS devices and future Apple Silicon devices. They were originally developed last year and have been working in the UTM app. Recently, we ported the changes to master, re-wrote a lot of the build script changes for meson, and broke up the patches into more distinct units.
A summary of the changes:
*
CONFIG_IOSdefined when building for iOS and iOS specific changes (as well as unsupported code) are gated behind it.
* A new dependency, libucontext is added since iOS does not have native ucontext and broken support for sigaltstack. libucontext is available as a new option for coroutine backend.
Quoted 5 lines
Since v6:
* Dropped the Apple Silicon JIT support patch (superseded by another patchset)
* Changed libucontext to be a Meson subproject
* Cache availablity check for preadv/pwritev on macOS 11 and iOS 14
So is this series now purely iOS support, or are there still some pieces of it that are needed for Mac systems with Apple silicon ?
thanks
-- PMM
After decoupling the MAP_JIT and APRR patches, this should only be for iOS support. I didn't change the title because I didn't know if patchew would be smart enough to group it with the older patches.
-j
Quoted 30 lines · Peter Maydell
Quoted 17 lines · Joelle van Dyne
Based-on: [email protected] ([PATCH v4 00/43] Mirror map JIT memory for TCG)
These set of changes brings QEMU TCG to iOS devices and future Apple Silicon devices. They were originally developed last year and have been working in the UTM app. Recently, we ported the changes to master, re-wrote a lot of the build script changes for meson, and broke up the patches into more distinct units.
A summary of the changes:
*
CONFIG_IOSdefined when building for iOS and iOS specific changes (as well as unsupported code) are gated behind it.
* A new dependency, libucontext is added since iOS does not have native ucontext and broken support for sigaltstack. libucontext is available as a new option for coroutine backend.Quoted 5 lines
Since v6:
* Dropped the Apple Silicon JIT support patch (superseded by another patchset)
* Changed libucontext to be a Meson subproject
* Cache availablity check for preadv/pwritev on macOS 11 and iOS 14So is this series now purely iOS support, or are there still some pieces of it that are needed for Mac systems with Apple silicon ?
thanks
-- PMM