Skip to contentSkip to conversation

fs: remove implicit compiler calls to memset/memcpy

PATCH In review Pete Batard 3 participants 8 messages Apr 10, 2016 – Apr 18, 2016

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fs: remove implicit compiler calls to memset/memcpy

Hi,

I am using the GRUB codebase to build generic read-only EFI file system drivers [1] and during that process, I found that some compilers (e.g. MSVC, but most likely others) may insert implicit calls to memset/memcpy when initializing or copying structure content, which defeats the use of grub_memset/grub_memcpy as well as the avoidance of standard libraries.

This patch ensures that grub_memset/grub_memcpy are used where required in the file system sources (as well as some corollary files).

The patch also removes unnecessary trailing whitespaces for the affected files.

A couple of additional notes:

* The patch also removes page breaks on some files as this is something my git client (TortoiseGit) does automatically on its own and that doesn't seem to be configurable. I do hope that this isn't a deal breaker, as it would be a major PITA for me to rework patches to keep the breaks in, and I kind of fail to see why we'd want to keep them in the sources.

* Yes, the 'const char nl[2] = "\n";' in misc.h does generate an implicit memcpy(), at least with MSVC for ARM. For what is worth, every non whitespace change that was applied in this patch stems from looking at the generated assembly, and isolating the memset/memcpy references.

* In zfs.c, you'll see that some function calls had to be modified to get their parameters by reference, as passing them by value was also generating implicit memcpy calls.

Regards,

/Pete

[1] https://github.com/pbatard/efifs

From f107607cbe2052effe09aadd1e441d416c73f95f Mon Sep 17 00:00:00 2001
From: Pete Batard <[email protected]>
Date: Sun, 10 Apr 2016 14:45:05 +0200
Subject: [PATCH] fs: remove implicit compiler calls to memset/memcpy

Some compilers may insert calls to memset/memcpy when initializing or copying structure content. This patch ensures that grub_memset and grub_memcpy are used instead.

Also fix trailing whitespaces for affected files.

---
 grub-core/fs/affs.c    |  18 ++--
 grub-core/fs/bfs.c     |  33 +++---
 grub-core/fs/btrfs.c   |   8 +-
 grub-core/fs/hfs.c     |  42 ++++----
 grub-core/fs/hfsplus.c |  16 +--
 grub-core/fs/iso9660.c |  30 +++---
 grub-core/fs/udf.c     |   4 +-
 grub-core/fs/zfs/zfs.c | 266 ++++++++++++++++++++++++-------------------------
 grub-core/io/gzio.c    |   8 +-
 include/grub/misc.h    |   2 +-
 10 files changed, 214 insertions(+), 213 deletions(-)
grub-core/fs/affs.c +9 -9
index f673897..26b74a7 100644
@@ -99,7 +99,7 @@ enum
 #define AFFS_MAX_LOG_BLOCK_SIZE 4
 #define AFFS_MAX_SUPERBLOCK 1
 
-
+
 
 struct grub_fshelp_node
 {
@@ -127,7 +127,7 @@ struct grub_affs_data
 
 static grub_dl_t my_mod;
 
-
+
 static grub_disk_addr_t
 grub_affs_read_block (grub_fshelp_node_t node, grub_disk_addr_t fileblock)
 {
@@ -345,8 +345,8 @@ grub_affs_create_node (grub_fshelp_node_t dir,
   if (len > sizeof (fil->name))
     len = sizeof (fil->name);
   *grub_latin1_to_utf8 (name_u8, fil->name, len) = '\0';
-  
-  (*node)->di = *fil;
+
+  grub_memcpy (&(*node)->di, fil, sizeof (*fil));
   for (nest = 0; nest < 8; nest++)
     {
       switch ((*node)->di.type)
@@ -408,8 +408,8 @@ grub_affs_iterate_dir (grub_fshelp_node_t dir,
   node = grub_zalloc (sizeof (*node));
   if (!node)
     return 1;
-    
-  *node = *dir;
+
+  grub_memcpy (node, dir, sizeof (*dir));
   if (hook (".", GRUB_FSHELP_DIR, node, hook_data))
     return 1;
   if (dir->parent)
@@ -417,7 +417,7 @@ grub_affs_iterate_dir (grub_fshelp_node_t dir,
       node = grub_zalloc (sizeof (*node));
       if (!node)
 	return 1;
-      *node = *dir->parent;
+      grub_memcpy (node, dir->parent, sizeof (*dir->parent));
       if (hook ("..", GRUB_FSHELP_DIR, node, hook_data))
 	return 1;
     }
@@ -491,7 +491,7 @@ grub_affs_open (struct grub_file *file, const char *name)
     goto fail;
 
   file->size = grub_be_to_cpu32 (fdiro->di.size);
-  data->diropen = *fdiro;
+  grub_memcpy (&data->diropen, fdiro, sizeof (*fdiro));
   grub_free (fdiro);
 
   file->data = data;
@@ -681,7 +681,7 @@ grub_affs_mtime (grub_device_t device, grub_int32_t *t)
   return GRUB_ERR_NONE;
 }
 
-
+
 static struct grub_fs grub_affs_fs =
   {
     .name = "affs",
grub-core/fs/bfs.c +17 -16
index d2b490b..71ca674 100644
@@ -530,13 +530,13 @@ iterate_in_b_tree (grub_disk_t disk,
       err = read_b_node (disk, sb, ino,
 			 node_off,
 			 &node,
-			 &key_data, 
+			 &key_data,
 			 &keylen_idx,
 			 &key_values);
 
       if (err)
 	return 0;
-      
+
       for (i = 0; i < grub_bfs_to_cpu_treehead (node->count_keys); i++)
 	{
 	  char c;
@@ -682,7 +682,7 @@ find_in_b_tree (grub_disk_t disk,
 		level--;
 		grub_free (node);
 		continue;
-	      }	      
+	      }
 	  }
 	else if (level != 0
 		 && i + 1 < grub_bfs_to_cpu_treehead (node->count_keys))
@@ -802,11 +802,11 @@ find_file (const char *path, grub_disk_t disk,
 	   enum grub_fshelp_filetype exptype)
 {
   grub_err_t err;
-  struct grub_fshelp_node root = {
-    .disk = disk,
-    .sb = sb,
-  };
-  struct grub_fshelp_node *found;
+  struct grub_fshelp_node root, *found;
+
+  grub_memset (&root, 0, sizeof (root));
+  root.disk = disk;
+  root.sb = sb;
 
   err = read_extent (disk, sb, &sb->root_dir, 0, 0, &root.ino,
 		     sizeof (root.ino));
@@ -827,7 +827,7 @@ mount (grub_disk_t disk, struct grub_bfs_superblock *sb)
   grub_err_t err;
   err = grub_disk_read (disk, SUPERBLOCK, 0, sizeof (*sb), sb);
   if (err == GRUB_ERR_OUT_OF_RANGE)
-    return grub_error (GRUB_ERR_BAD_FS, 
+    return grub_error (GRUB_ERR_BAD_FS,
 #ifdef MODE_AFS
 		       "not an AFS filesystem"
 #else
@@ -843,7 +843,7 @@ mount (grub_disk_t disk, struct grub_bfs_superblock *sb)
       || (grub_bfs_to_cpu32 (sb->bsize)
 	  != (1U << grub_bfs_to_cpu32 (sb->log2_bsize)))
       || grub_bfs_to_cpu32 (sb->log2_bsize) < GRUB_DISK_SECTOR_BITS)
-    return grub_error (GRUB_ERR_BAD_FS, 
+    return grub_error (GRUB_ERR_BAD_FS,
 #ifdef MODE_AFS
 		       "not an AFS filesystem"
 #else
@@ -887,13 +887,14 @@ static grub_err_t
 grub_bfs_dir (grub_device_t device, const char *path,
 	      grub_fs_dir_hook_t hook, void *hook_data)
 {
-  struct grub_bfs_dir_ctx ctx = {
-    .device = device,
-    .hook = hook,
-    .hook_data = hook_data
-  };
+  struct grub_bfs_dir_ctx ctx;
   grub_err_t err;
 
+  grub_memset (&ctx, 0, sizeof (ctx));
+  ctx.device = device;
+  ctx.hook = hook;
+  ctx.hook_data = hook_data;
+
   err = mount (device->disk, &ctx.sb);
   if (err)
     return err;
@@ -930,7 +931,7 @@ grub_bfs_open (struct grub_file *file, const char *name)
     data = grub_zalloc (sizeof (struct grub_bfs_data));
     if (!data)
       return grub_errno;
-    data->sb = sb;
+    grub_memcpy (&data->sb, &sb, sizeof (sb));
     grub_memcpy (&data->ino, &ino, sizeof (data->ino));
     file->data = data;
     file->size = grub_bfs_to_cpu64 (ino.size);
grub-core/fs/btrfs.c +4 -4
index 9cffa91..8d99818 100644
@@ -355,7 +355,7 @@ next (struct grub_btrfs_data *data,
   *outsize = grub_le_to_cpu32 (leaf.size);
   *outaddr = desc->data[desc->depth - 1].addr + sizeof (struct btrfs_header)
     + grub_le_to_cpu32 (leaf.offset);
-  *key_out = leaf.key;
+  grub_memcpy (key_out, &leaf.key, sizeof (leaf.key));
   return 1;
 }
 
@@ -436,7 +436,7 @@ lower_bound (struct grub_btrfs_data *data,
 		}
 	      if (key_cmp (&node.key, key_in) > 0)
 		break;
-	      node_last = node;
+	      grub_memcpy (&node_last, &node, sizeof (node));
 	      have_last = 1;
 	    }
 	  if (have_last)
@@ -490,7 +490,7 @@ lower_bound (struct grub_btrfs_data *data,
 	      break;
 
 	    have_last = 1;
-	    leaf_last = leaf;
+	    grub_memcpy (&leaf_last, &leaf, sizeof (leaf));
 	  }
 
 	if (have_last)
@@ -1486,7 +1486,7 @@ find_path (struct grub_btrfs_data *data,
 	      grub_free (origpath);
 	      return err;
 	    }
-	  *key = cdirel->key;
+	  grub_memcpy (key, &cdirel->key, sizeof (cdirel->key));
 	  if (*type == GRUB_BTRFS_DIR_ITEM_TYPE_DIRECTORY)
 	    key->type = GRUB_BTRFS_ITEM_TYPE_DIR_ITEM;
 	  break;
grub-core/fs/hfs.c +21 -21

117 lines not on this page · raw message

grub-core/fs/hfsplus.c +8 -8
index 21159e8..a8663f0 100644
@@ -19,7 +19,7 @@
 
 /* HFS+ is documented at http://developer.apple.com/technotes/tn/tn1150.html */
 
-#define grub_fshelp_node grub_hfsplus_file 
+#define grub_fshelp_node grub_hfsplus_file
 #include <grub/err.h>
 #include <grub/file.h>
 #include <grub/mm.h>
@@ -105,7 +105,7 @@ enum grub_hfsplus_filetype
 
 static grub_dl_t my_mod;
 
-
+
 
 grub_err_t (*grub_hfsplus_open_compressed) (struct grub_fshelp_node *node);
 grub_ssize_t (*grub_hfsplus_read_compressed) (struct grub_hfsplus_file *node,
@@ -145,7 +145,7 @@ grub_hfsplus_read_block (grub_fshelp_node_t node, grub_disk_addr_t fileblock)
 {
   struct grub_hfsplus_btnode *nnode = 0;
   grub_disk_addr_t blksleft = fileblock;
-  struct grub_hfsplus_extent *extents = node->compressed 
+  struct grub_hfsplus_extent *extents = node->compressed
     ? &node->resource_extents[0] : &node->extents[0];
 
   while (1)
@@ -461,7 +461,7 @@ grub_hfsplus_cmp_extkey (struct grub_hfsplus_key *keya,
 
   if (extkey_a->type < extkey_b->type)
     return -1;
-  
+
   akey = grub_be_to_cpu32 (extkey_a->start);
   if (akey > extkey_b->start)
     return 1;
@@ -548,7 +548,7 @@ grub_hfsplus_btree_search (struct grub_hfsplus_btree *btree,
 			   struct grub_hfsplus_key_internal *key,
 			   int (*compare_keys) (struct grub_hfsplus_key *keya,
 						struct grub_hfsplus_key_internal *keyb),
-			   struct grub_hfsplus_btnode **matchnode, 
+			   struct grub_hfsplus_btnode **matchnode,
 			   grub_off_t *keyoffset)
 {
   grub_uint64_t currnode;
@@ -800,7 +800,7 @@ grub_hfsplus_iterate_dir (grub_fshelp_node_t dir,
     fsnode = grub_malloc (sizeof (*fsnode));
     if (!fsnode)
       return 1;
-    *fsnode = *dir;
+    grub_memcpy (fsnode, dir, sizeof (*dir));
     if (hook (".", GRUB_FSHELP_DIR, fsnode, hook_data))
       return 1;
   }
@@ -853,7 +853,7 @@ grub_hfsplus_open (struct grub_file *file, const char *name)
     }
 
   file->size = fdiro->size;
-  data->opened_file = *fdiro;
+  grub_memcpy (&data->opened_file, fdiro, sizeof (*fdiro));
   grub_free (fdiro);
 
   file->data = data;
@@ -1074,7 +1074,7 @@ grub_hfsplus_uuid (grub_device_t device, char **uuid)
 }
 
 
-
+
 static struct grub_fs grub_hfsplus_fs =
   {
     .name = "hfsplus",
grub-core/fs/iso9660.c +15 -15
index c9c8374..30253df 100644
@@ -174,13 +174,13 @@ enum
   };
 
 static grub_dl_t my_mod;
-
+
 
 static grub_err_t
 iso9660_to_unixtime (const struct grub_iso9660_date *i, grub_int32_t *nix)
 {
   struct grub_datetime datetime;
-  
+
   if (! i->year[0] && ! i->year[1]
       && ! i->year[2] && ! i->year[3]
       && ! i->month[0] && ! i->month[1]
@@ -197,7 +197,7 @@ iso9660_to_unixtime (const struct grub_iso9660_date *i, grub_int32_t *nix)
   datetime.hour = (i->hour[0] - '0') * 10 + (i->hour[1] - '0');
   datetime.minute = (i->minute[0] - '0') * 10 + (i->minute[1] - '0');
   datetime.second = (i->second[0] - '0') * 10 + (i->second[1] - '0');
-  
+
   if (!grub_datetime2unixtime (&datetime, nix))
     return grub_error (GRUB_ERR_BAD_NUMBER, "incorrect date");
   *nix -= i->offset * 60 * 15;
@@ -215,7 +215,7 @@ iso9660_to_unixtime2 (const struct grub_iso9660_date2 *i, grub_int32_t *nix)
   datetime.hour = i->hour;
   datetime.minute = i->minute;
   datetime.second = i->second;
-  
+
   if (!grub_datetime2unixtime (&datetime, nix))
     return 0;
   *nix -= i->offset * 60 * 15;
@@ -408,7 +408,7 @@ set_rockridge (struct grub_iso9660_data *data)
       rootnode.alloc_dirents = ARRAY_SIZE (rootnode.dirents);
       rootnode.have_dirents = 1;
       rootnode.have_symlink = 0;
-      rootnode.dirents[0] = data->voldesc.rootdir;
+      grub_memcpy (&rootnode.dirents[0], &data->voldesc.rootdir, sizeof (data->voldesc.rootdir));
 
       /* The 2nd data byte stored how many bytes are skipped every time
 	 to get to the SUA (System Usage Area).  */
@@ -498,7 +498,7 @@ grub_iso9660_mount (grub_disk_t disk)
 static char *
 grub_iso9660_read_symlink (grub_fshelp_node_t node)
 {
-  return node->have_symlink 
+  return node->have_symlink
     ? grub_strdup (node->symlink
 		   + (node->have_dirents) * sizeof (node->dirents[0])
 		   - sizeof (node->dirents)) : grub_strdup ("");
@@ -537,7 +537,7 @@ add_part (struct iterate_dir_ctx *ctx,
     return;
 
   grub_memcpy (ctx->symlink + size, part, len2);
-  ctx->symlink[size + len2] = 0;  
+  ctx->symlink[size + len2] = 0;
 }
 
 static grub_err_t
@@ -762,7 +762,7 @@ grub_iso9660_iterate_dir (grub_fshelp_node_t dir,
             ctx.filename_alloc = 1;
           }
 
-	node->dirents[0] = dirent;
+	grub_memcpy (&node->dirents[0], &dirent, sizeof (dirent));
 	while (dirent.flags & FLAG_MORE_EXTENTS)
 	  {
 	    offset += dirent.len;
@@ -777,7 +777,7 @@ grub_iso9660_iterate_dir (grub_fshelp_node_t dir,
 	      {
 		struct grub_fshelp_node *new_node;
 		node->alloc_dirents *= 2;
-		new_node = grub_realloc (node, 
+		new_node = grub_realloc (node,
 					 sizeof (struct grub_fshelp_node)
 					 + ((node->alloc_dirents
 					     - ARRAY_SIZE (node->dirents))
@@ -791,7 +791,7 @@ grub_iso9660_iterate_dir (grub_fshelp_node_t dir,
 		  }
 		node = new_node;
 	      }
-	    node->dirents[node->have_dirents++] = dirent;
+	    grub_memcpy (&node->dirents[node->have_dirents++], &dirent, sizeof (dirent));
 	  }
 	if (ctx.symlink)
 	  {
@@ -837,7 +837,7 @@ grub_iso9660_iterate_dir (grub_fshelp_node_t dir,
 }
 
 
-
+
 /* Context for grub_iso9660_dir.  */
 struct grub_iso9660_dir_ctx
 {
@@ -881,7 +881,7 @@ grub_iso9660_dir (grub_device_t device, const char *path,
   rootnode.alloc_dirents = 0;
   rootnode.have_dirents = 1;
   rootnode.have_symlink = 0;
-  rootnode.dirents[0] = data->voldesc.rootdir;
+  grub_memcpy (&rootnode.dirents[0], &data->voldesc.rootdir, sizeof (data->voldesc.rootdir));
 
   /* Use the fshelp function to traverse the path.  */
   if (grub_fshelp_find_file (path, &rootnode,
@@ -924,7 +924,7 @@ grub_iso9660_open (struct grub_file *file, const char *name)
   rootnode.alloc_dirents = 0;
   rootnode.have_dirents = 1;
   rootnode.have_symlink = 0;
-  rootnode.dirents[0] = data->voldesc.rootdir;
+  grub_memcpy (&rootnode.dirents[0], &data->voldesc.rootdir, sizeof (data->voldesc.rootdir));
 
   /* Use the fshelp function to traverse the path.  */
   if (grub_fshelp_find_file (name, &rootnode,
@@ -1069,7 +1069,7 @@ grub_iso9660_uuid (grub_device_t device, char **uuid)
 }
 
 /* Get writing time of filesystem. */
-static grub_err_t 
+static grub_err_t
 grub_iso9660_mtime (grub_device_t device, grub_int32_t *timebuf)
 {
   struct grub_iso9660_data *data;
@@ -1094,7 +1094,7 @@ grub_iso9660_mtime (grub_device_t device, grub_int32_t *timebuf)
 }
 
 
-
+
 
 static struct grub_fs grub_iso9660_fs =
   {
grub-core/fs/udf.c +2 -2
index 839bff8..4441ad4 100644
@@ -539,7 +539,7 @@ grub_udf_read_block (grub_fshelp_node_t node, grub_disk_addr_t fileblock)
 		    (buf + sizeof (struct grub_udf_aed));
 	      continue;
 	    }
-	      
+
 	  if (filebytes < adlen)
 	    {
 	      grub_uint32_t ad_block_num = ad->block.block_num;
@@ -793,7 +793,7 @@ grub_udf_mount (grub_disk_t disk)
       goto fail;
     }
 
-  data->root_icb = root_fs.root_icb;
+  grub_memcpy (&data->root_icb, &root_fs.root_icb, sizeof (root_fs.root_icb));
 
   return data;
 
grub-core/fs/zfs/zfs.c +133 -133

863 lines not on this page · raw message

grub-core/io/gzio.c +4 -4
index 0f2ea6b..232408e 100644
@@ -562,7 +562,7 @@ huft_build (unsigned *b,	/* code lengths in bits (all assumed <= BMAX) */
 		  r.e = (uch) (16 + j);		/* bits in this table */
 		  r.v.t = q;	/* pointer to this table */
 		  j = i >> (w - l);	/* (get around Turbo C bug) */
-		  u[h - 1][j] = r;	/* connect to last table */
+		  grub_memcpy (&u[h - 1][j], &r, sizeof (r));	/* connect to last table */
 		}
 	    }
 
@@ -585,7 +585,7 @@ huft_build (unsigned *b,	/* code lengths in bits (all assumed <= BMAX) */
 	  /* fill code-like entries with r */
 	  f = 1 << (k - w);
 	  for (j = i >> w; j < z; j += f)
-	    q[j] = r;
+	    grub_memcpy (&q[j], &r, sizeof (r));
 
 	  /* backwards increment the k-bit code i */
 	  for (j = 1 << (k - 1); i & j; j >>= 1)
@@ -1178,7 +1178,7 @@ static int
 test_zlib_header (grub_gzio_t gzio)
 {
   grub_uint8_t cmf, flg;
-  
+
   cmf = get_byte (gzio);
   flg = get_byte (gzio);
 
@@ -1345,7 +1345,7 @@ grub_deflate_decompress (char *inbuf, grub_size_t insize, grub_off_t off,
   return ret;
 }
 
-
+
 
 static struct grub_fs grub_gzio_fs =
   {
include/grub/misc.h +1 -1
index 2a9f87c..fdce592 100644
@@ -314,7 +314,7 @@ extern void (*EXPORT_VAR (grub_xputs)) (const char *str);
 static inline int
 grub_puts (const char *s)
 {
-  const char nl[2] = "\n";
+  const char *nl = "\n";
   grub_xputs (s);
   grub_xputs (nl);
 
Signature · 1 line
1.9.5.msysgit.1
#1 lore raw
Quoted 8 lines · Pete Batard

Hi,

I am using the GRUB codebase to build generic read-only EFI file system drivers [1] and during that process, I found that some compilers (e.g. MSVC, but most likely others) may insert implicit calls to memset/memcpy when initializing or copying structure content, which defeats the use of grub_memset/grub_memcpy as well as the avoidance of standard libraries.

We alias memcpy/memset to their grub equivalents because of this. Why is the same solution not suitable for you? Where compiler inserts those references is unpredictable.

Quoted 35 lines

This patch ensures that grub_memset/grub_memcpy are used where required in the file system sources (as well as some corollary files).

The patch also removes unnecessary trailing whitespaces for the affected files.

A couple of additional notes:

* The patch also removes page breaks on some files as this is something my git client (TortoiseGit) does automatically on its own and that doesn't seem to be configurable. I do hope that this isn't a deal breaker, as it would be a major PITA for me to rework patches to keep the breaks in, and I kind of fail to see why we'd want to keep them in the sources.

* Yes, the 'const char nl[2] = "\n";' in misc.h does generate an implicit memcpy(), at least with MSVC for ARM. For what is worth, every non whitespace change that was applied in this patch stems from looking at the generated assembly, and isolating the memset/memcpy references.

* In zfs.c, you'll see that some function calls had to be modified to get their parameters by reference, as passing them by value was also generating implicit memcpy calls.

Regards,

/Pete

[1] https://github.com/pbatard/efifs
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel

#2 lore raw
Quoted 1 line · Vladimir 'phcoder' Serbinenko

Why is the same solution not suitable for you?

That's the first thing I attempted, but the MSVC compiler is uncooperative.

If you try to redefine memset/memcpy, you get the following compiler error:

error C2169: '_memcpy' : Intrinsic function, cannot be defined

And of course, this means that you then have to link with the standard Windows MSVC libraries, which creates issues when building UEFI drivers...

Apparently, there used to be a way to disable intrinsic functions with older versions of Visual Studio (using a pragma), but this capability was removed in the latest versions. Either that, or Microsoft does not consider memcpy and friends as belonging to the general "intrinsic" category that the compiler can disable.

This means that, right now, the only option available is to patch the GRUB source.

Regards,

/Pete

#3 lore raw

You can use asm to get around msvc limitations. Sth like

.global memcpy
memcpy:
jmp grub_memcpy

Where implicit memcpy is inserted is pretty much unpredictable and we're not going to maintain memcpy-free environment because of this

Quoted 32 lines · Pete Batard
Quoted 1 line · Vladimir 'phcoder' Serbinenko

Why is the same solution not suitable for you?

That's the first thing I attempted, but the MSVC compiler is uncooperative.

If you try to redefine memset/memcpy, you get the following compiler error:

error C2169: '_memcpy' : Intrinsic function, cannot be defined

And of course, this means that you then have to link with the standard Windows MSVC libraries, which creates issues when building UEFI drivers...

Apparently, there used to be a way to disable intrinsic functions with older versions of Visual Studio (using a pragma), but this capability was removed in the latest versions. Either that, or Microsoft does not consider memcpy and friends as belonging to the general "intrinsic" category that the compiler can disable.

This means that, right now, the only option available is to patch the GRUB source.

Regards,

/Pete

_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel

#4 lore raw
Quoted 5 lines · Vladimir 'phcoder' Serbinenko

You can use asm to get around msvc limitations. Sth like

.global memcpy
memcpy:
jmp grub_memcpy

Yes I'm well aware I could try to create my own library (or equivalent) that redefines memcpy/memset, using some workaround to avoid the MSVC compiler error (if needed, I could create a compatible aliasing lib with gcc). But that's something I'd prefer to avoid because it's never that simple in the MSVC world, and I'm also compiling for multiple archs (x86 and ARM, possibly more in the future), which makes assembly workarounds an annoyance. Besides, I already have to patch the GRUB source anyway, with some MSVC specific quirks (some of which I doubt you guys would like [1]), so adding some more to my current patch series to remove the implicits is not exactly a big deal. But of course, I sure wouldn't mind minimizing the amount of code I need to patch to use GRUB in my scope, which is the whole point of this submission.

Quoted 1 line

Where implicit memcpy is inserted is pretty much unpredictable

You're missing the purpose of my request. I'm not asking the GRUB project to do anything about predicting where memset/memcpy are going inserted, or even attempt to be preemptive about that in the future. I'm simply asking it to address the ones that were isolated with 100% accuracy (compiler flag to generate assembly with source + grep on said assembly) from using the current GRUB source in the specific context that I have. There's no "predictive" component in what I am requesting here, not even an implied one.

So, in case that is the crux of the issue, should this patch be integrated, I am not asking any GRUB contributor to try to keep conscious of or try to identify potential implicits as they go about modifying fs or any other parts of the code. It is in fact of little consequence if someone comes in and breaks the applied implicits removal the day after it is applied, as I am using GRUB as a git submodule, therefore it'll remain tied to the specific git rev where it isn't broken. And if I try to update the submodule, and identify breakage, I will of course submit a new patch to this list as needed.

Quoted 2 lines

and we're
not going to maintain memcpy-free environment because of this

Again, this is not what I am asking. This is a simple "you scratch my back, I'll scratch yours" request, that, in a way, is pretty much akin to asking the GRUB project to add extra parenthesis in _very specific_ parts of source, to make it more palatable when these specific parts are used with compilers that are not officially supported by the project, and as a one-off thing (i.e. without asking for anyone to be tasked to maintaining that new parenthesis layout going forward).

If you want to say: "Well, we don't official support MSVC, so we're not going to pick a patch that is essentially aimed at improving MSVC support, especially if only applies to a limited set of sources", that's fine with me. But please don't try to imply that the patch has a much larger scope than it actually does, as your justification for rejection, or that it is going to require any extra work/maintainance from GRUB contributors once applied.

Regards,

/Pete

[1] https://github.com/pbatard/efifs/blob/master/0001-GRUB-fixes-for-MSVC-compilation-part-1.patch#L366-L410

#5 lore raw
Quoted 9 lines · Pete Batard
Quoted 6 lines · Vladimir 'phcoder' Serbinenko

You can use asm to get around msvc limitations. Sth like

.global memcpy
memcpy:
jmp grub_memcpy

Quoted 6 lines

Yes I'm well aware I could try to create my own library (or equivalent) that redefines memcpy/memset, using some workaround to avoid the MSVC compiler error (if needed, I could create a compatible aliasing lib with gcc). But that's something I'd prefer to avoid because it's never that simple in the MSVC world, and I'm also compiling for multiple archs (x86 and ARM, possibly more in the future), which makes assembly workarounds an annoyance. Besides,

grub so far was relatively good at localizing platform and compiler specific code. It sounds like something that fits well.

Quoted 10 lines

I already have to patch the GRUB source anyway, with some MSVC specific quirks (some of which I doubt you guys would like [1]), so adding some more to my current patch series to remove the implicits is not exactly a big deal. But of course, I sure wouldn't mind minimizing the amount of code I need to patch to use GRUB in my scope, which is the whole point of this submission.

Quoted 1 line

Where implicit memcpy is inserted is pretty much unpredictable

Quoted 9 lines

You're missing the purpose of my request. I'm not asking the GRUB project to do anything about predicting where memset/memcpy are going inserted, or even attempt to be preemptive about that in the future. I'm simply asking it to address the ones that were isolated with 100% accuracy (compiler flag to generate assembly with source + grep on said assembly) from using the current GRUB source in the specific context that I have.
There's no "predictive" component in what I am requesting here, not even an implied one.

Compilers quite likely avoid calling external functions in this case; so it prevents optimizing code on other platforms/compilers.

Quoted 22 lines

So, in case that is the crux of the issue, should this patch be integrated, I am not asking any GRUB contributor to try to keep conscious of or try to identify potential implicits as they go about modifying fs or any other parts of the code. It is in fact of little consequence if someone comes in and breaks the applied implicits removal the day after it is applied, as I am using GRUB as a git submodule, therefore it'll remain tied to the specific git rev where it isn't broken. And if I try to update the submodule, and identify breakage, I will of course submit a new patch to this list as needed.

Quoted 2 lines

and we're
not going to maintain memcpy-free environment because of this

Again, this is not what I am asking. This is a simple "you scratch my back, I'll scratch yours" request, that, in a way, is pretty much akin to asking the GRUB project to add extra parenthesis in _very specific_ parts of source, to make it more palatable when these specific parts are used with compilers that are not officially supported by the project, and as a one-off thing (i.e. without asking for anyone to be tasked to maintaining that new parenthesis layout going forward).

Quoted 4 lines

If you want to say: "Well, we don't official support MSVC, so we're not going to pick a patch that is essentially aimed at improving MSVC support, especially if only applies to a limited set of sources", that's fine with me.

Personally I would welcome any patch aimed at improving MSVC support as long as it is not added at expense of another platforms. We do support native grub on Windows so it is just logical to support native Windows build environment. It is just that no current grub maintainers is using MSVC so if you volunteer to step in it would be great.

Quoted 16 lines

But please don't try to imply that the patch has a much larger scope than it actually does, as your justification for rejection, or that it is going to require any extra work/maintainance from GRUB contributors once applied.

Regards,

/Pete

[1]
https://github.com/pbatard/efifs/blob/master/0001-GRUB-fixes-for-MSVC-compilation-part-1.patch#L366-L410

_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel

#6 lore raw
Quoted 4 lines · Andrei Borzenkov
Quoted 2 lines · Pete Batard
Quoted 1 line · Vladimir 'phcoder' Serbinenko
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

You can use asm to get around msvc limitations. Sth like

.global memcpy
memcpy:
jmp grub_memcpy

Yes I'm well aware I could try to create my own library (or equivalent)

that

Quoted 1 line
Quoted 1 line

redefines memcpy/memset, using some workaround to avoid the MSVC

compiler

Quoted 1 line
Quoted 1 line

error (if needed, I could create a compatible aliasing lib with gcc).

But

Quoted 1 line
Quoted 1 line

that's something I'd prefer to avoid because it's never that simple in

the

Quoted 1 line
Quoted 1 line

MSVC world, and I'm also compiling for multiple archs (x86 and ARM,

possibly

Quoted 1 line
Quoted 1 line

more in the future), which makes assembly workarounds an annoyance.

Besides,

Quoted 6 lines

grub so far was relatively good at localizing platform and compiler specific code. It sounds like something that fits well.

Quoted 2 lines

I already have to patch the GRUB source anyway, with some MSVC specific quirks (some of which I doubt you guys would like [1]), so adding some

more

Quoted 2 lines
Quoted 2 lines

to my current patch series to remove the implicits is not exactly a big deal. But of course, I sure wouldn't mind minimizing the amount of code

I

Quoted 7 lines
Quoted 7 lines

need to patch to use GRUB in my scope, which is the whole point of this submission.

Quoted 1 line

Where implicit memcpy is inserted is pretty much unpredictable

You're missing the purpose of my request. I'm not asking the GRUB

project to

Quoted 1 line
Quoted 1 line

do anything about predicting where memset/memcpy are going inserted, or

even

Quoted 1 line
Quoted 1 line

attempt to be preemptive about that in the future. I'm simply asking it

to

Quoted 4 lines
Quoted 4 lines

address the ones that were isolated with 100% accuracy (compiler flag to generate assembly with source + grep on said assembly) from using the current GRUB source in the specific context that I have. There's no "predictive" component in what I am requesting here, not

even an

Quoted 7 lines
Quoted 2 lines

implied one.

Compilers quite likely avoid calling external functions in this case; so it prevents optimizing code on other platforms/compilers.

Quoted 1 line

So, in case that is the crux of the issue, should this patch be

integrated,

Quoted 1 line
Quoted 1 line

I am not asking any GRUB contributor to try to keep conscious of or try

to

Quoted 2 lines
Quoted 2 lines

identify potential implicits as they go about modifying fs or any other parts of the code. It is in fact of little consequence if someone comes

in

Quoted 1 line
Quoted 1 line

and breaks the applied implicits removal the day after it is applied,

as I

Quoted 10 lines
Quoted 10 lines

am using GRUB as a git submodule, therefore it'll remain tied to the specific git rev where it isn't broken. And if I try to update the submodule, and identify breakage, I will of course submit a new patch to this list as needed.

Quoted 2 lines

and we're
not going to maintain memcpy-free environment because of this

Again, this is not what I am asking. This is a simple "you scratch my

back,

Quoted 1 line
Quoted 1 line

I'll scratch yours" request, that, in a way, is pretty much akin to

asking

Quoted 2 lines
Quoted 2 lines

the GRUB project to add extra parenthesis in _very specific_ parts of source, to make it more palatable when these specific parts are used

with

Quoted 1 line
Quoted 1 line

compilers that are not officially supported by the project, and as a

one-off

Quoted 1 line
Quoted 1 line

thing (i.e. without asking for anyone to be tasked to maintaining that

new

Quoted 4 lines
Quoted 4 lines

parenthesis layout going forward).

If you want to say: "Well, we don't official support MSVC, so we're not going to pick a patch that is essentially aimed at improving MSVC

support,

Quoted 1 line
Quoted 1 line

especially if only applies to a limited set of sources", that's fine

with

Quoted 8 lines
Quoted 1 line

me.

Personally I would welcome any patch aimed at improving MSVC support as long as it is not added at expense of another platforms. We do support native grub on Windows so it is just logical to support native Windows build environment. It is just that no current grub maintainers is using MSVC so if you volunteer to step in it would be great.

Supporting MSVC would also allow compiling to EBC, even though benefits of this are questionable.
This patch is not about full support but only about one specific folder and doesn't give us full support.
I think we might be setting wrong expectations here. In any case it's not 2.02 material, so I put it lower in my priority list

Quoted 1 line
Quoted 1 line

But please don't try to imply that the patch has a much larger scope

than it

Quoted 1 line
Quoted 1 line

actually does, as your justification for rejection, or that it is going

to

Quoted 8 lines
Quoted 8 lines

require any extra work/maintainance from GRUB contributors once applied.

Regards,

/Pete

[1]

https://github.com/pbatard/efifs/blob/master/0001-GRUB-fixes-for-MSVC-compilation-part-1.patch#L366-L410

Quoted 11 lines
Quoted 6 lines

_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel

_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel

#7 lore raw
Quoted 2 lines · Vladimir 'phcoder' Serbinenko

In any case it's
not 2.02 material, so I put it lower in my priority list

Thanks for considering it further - much appreciated.

I should point out that there's obviously nothing blocking for me, so I'm fine with any delay.

Quoted 2 lines · Andrei Borzenkov

It is just that no current grub maintainers
is using MSVC so if you volunteer to step in it would be great.

I wouldn't mind, and that is something I have done before with libcdio -- sadly that effort was stopped short of adding full Visual Studio support (i.e. with with project files), but at least now the libcdio codebase is fully compatible with MS compilers. My only issue is that I'm very busy with various projects, so time constraints are a major factor.

Still, I have been benefiting greatly from GRUB and I do have additional MSVC compatibility patches, especially the ones that have to do struct packing, that I wouldn't mind seeing integrated in the source. So if it can help, I'll see if I can take a stab at providing MSVC support for the whole project. Overall, short of very specific quirks, I haven't found much in the way of compiling the GRUB code with MSVC, so maybe the effort required won't be that bad... I'll keep you posted.

Regards,

/Pete