i386-pc: build btrfs zstd support into separate module
PATCH 1/1 i386-pc: build btrfs zstd support into separate module
The zstd support in btrfs brings significant size increment to the on-disk image that it can no longer fit into btrfs bootloader area and short mbr gap.
In order to support grub update on outstanding i386-pc setup with these size constraints remain in place, here we build the zstd suppprt of btrfs into a separate module, named btrfs_zstd, to alleviate the size change. Please note this only makes it's way to i386-pc, other architecture is not affected.
Therefore if the system has enough space of embedding area for grub then zstd support for btrfs will be enabled automatically in the process of running grub-install through inserting btrfs_zstd module to the on-disk image, otherwise a warning will be logged on screen to indicate user that zstd support for btrfs is disabled due to the size limit.
--- Makefile.util.def | 1 + grub-core/Makefile.core.def | 11 ++++ grub-core/fs/btrfs.c | 114 +++++--------------------------- grub-core/fs/btrfs_zstd.c | 36 +++++++++++ grub-core/lib/zstd.c | 126 ++++++++++++++++++++++++++++++++++++ include/grub/btrfs.h | 6 ++ include/grub/lib/zstd.h | 26 ++++++++ include/grub/util/install.h | 2 + util/grub-install.c | 20 ++++++ util/setup.c | 43 ++++++++++++ 10 files changed, 288 insertions(+), 97 deletions(-) create mode 100644 grub-core/fs/btrfs_zstd.c create mode 100644 grub-core/lib/zstd.c create mode 100644 include/grub/lib/zstd.h
Makefile.util.def +1 -0
@@ -176,6 +176,7 @@ library = { common = grub-core/lib/zstd/xxhash.c; common = grub-core/lib/zstd/zstd_common.c; common = grub-core/lib/zstd/zstd_decompress.c; + common = grub-core/lib/zstd.c; }; program = {
grub-core/Makefile.core.def +11 -0
@@ -1370,10 +1370,21 @@ module = { name = btrfs; common = fs/btrfs.c; common = lib/crc.c; + nopc = lib/zstd.c; cflags = '$(CFLAGS_POSIX) -Wno-undef'; + i386_pc_cppflags = '-I$(srcdir)/lib/posix_wrap -I$(srcdir)/lib/minilzo -DMINILZO_HAVE_CONFIG_H'; cppflags = '-I$(srcdir)/lib/posix_wrap -I$(srcdir)/lib/minilzo -I$(srcdir)/lib/zstd -DMINILZO_HAVE_CONFIG_H'; }; +module = { + name = btrfs_zstd; + common = fs/btrfs_zstd.c; + common = lib/zstd.c; + cflags = '$(CFLAGS_POSIX) -Wno-undef'; + cppflags = '-I$(srcdir)/lib/posix_wrap -I$(srcdir)/lib/zstd'; + enable = i386_pc; +}; + module = { name = archelp; common = fs/archelp.c;
grub-core/fs/btrfs.c +17 -97
144 lines not on this page · raw message
grub-core/fs/btrfs_zstd.c new file +36 -0
@@ -0,0 +1,36 @@ +/* +* GRUB -- GRand Unified Bootloader +* Copyright (C) 2008 Free Software Foundation, Inc. +* +* GRUB is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation, either version 3 of the License, or +* (at your option) any later version. +* +* GRUB is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with GRUB. If not, see <http://www.gnu.org/licenses/>. +*/ + +#include <grub/types.h> +#include <grub/dl.h> +/* For NULL. */ +#include <grub/mm.h> +#include <grub/btrfs.h> +#include <grub/lib/zstd.h> + +GRUB_MOD_LICENSE ("GPLv3+" ); + +GRUB_MOD_INIT (btrfs_zstd) +{ + grub_btrfs_zstd_decompress_func = grub_zstd_decompress; +} + +GRUB_MOD_FINI (btrfs_zstd) +{ + grub_btrfs_zstd_decompress_func =NULL ; +}
grub-core/lib/zstd.c new file +126 -0
126 lines not on this page · raw message
include/grub/btrfs.h +6 -0
@@ -69,4 +69,10 @@ struct grub_btrfs_inode_ref char name[0]; }; +#ifdef GRUB_MACHINE_PCBIOS +extern grub_ssize_t (*EXPORT_VAR (grub_btrfs_zstd_decompress_func )) (char *ibuf, +grub_size_t isize,grub_off_t off, +char *obuf,grub_size_t osize); +#endif +#endif
include/grub/lib/zstd.h new file +26 -0
@@ -0,0 +1,26 @@ +/* +* GRUB -- GRand Unified Bootloader +* Copyright (C) 2008 Free Software Foundation, Inc. +* +* GRUB is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation, either version 3 of the License, or +* (at your option) any later version. +* +* GRUB is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with GRUB. If not, see <http://www.gnu.org/licenses/>. +*/ + +#ifndef GRUB_ZSTD_HEADER +#define GRUB_ZSTD_HEADER 1 + +grub_ssize_t +grub_zstd_decompress (char *ibuf,grub_size_t isize,grub_off_t off, +char *obuf,grub_size_t osize); + +#endif /* ! GRUB_ZSTD_HEADER */
include/grub/util/install.h +2 -0
@@ -208,6 +208,8 @@ grub_util_sparc_setup (const char *dir, const char *dest, int force, int fs_probe, int allow_floppy, int add_rs_codes, int warn_short_mbr_gap); +int +grub_util_try_partmap_embed (const char *dest,unsigned int *nsec);char *grub_install_get_image_targets_string (void );
util/grub-install.c +20 -0
@@ -1482,6 +1482,26 @@ main (int argc, char *argv[]) } } +if (install_drive + && platform ==GRUB_INSTALL_PLATFORM_I386_PC + &&grub_strcmp (grub_fs->name,"btrfs" ) ==0 ) + { +char *mod; + mod =grub_util_path_concat (2 , grub_install_source_directory,"btrfs_zstd.mod" ); +if (grub_util_is_regular (mod)) + { +unsigned int nsec =GRUB_MIN_RECOMMENDED_MBR_GAP ; +int ret =grub_util_try_partmap_embed (install_drive, &nsec); +if (ret ==0 ) +grub_install_push_module ("btrfs_zstd" ); +else if (ret ==1 ) +grub_util_warn ("%s" ,_ ("btrfs zstd compression is disabled, please change install device to disk" )); +else +grub_util_warn ("%s" ,_ ("btrfs zstd compression is disabled due to not enough space to embed image" )); + } +grub_free (mod); + } +if (!have_abstractions) { if ((disk_module && grub_strcmp (disk_module, "biosdisk") != 0)
util/setup.c +43 -0
49 lines not on this page · raw message
Signature · 1 line
2.31.1
Quoted 9 lines · Michael Chang via Grub-devel
The zstd support in btrfs brings significant size increment to the on-disk image that it can no longer fit into btrfs bootloader area and short mbr gap.
In order to support grub update on outstanding i386-pc setup with these size constraints remain in place, here we build the zstd suppprt of btrfs into a separate module, named
btrfs_zstd, to alleviate the size change. Please note this only makes it's way to i386-pc, other architecture is not affected.
I am OK with extracting zstd code from btrfs code. However, I want that be done for all architectures and platforms. No exceptions.
[...]
Quoted 26 lines
grub-core/fs/btrfs_zstd.c new file +20
-0@@ -0,0 +1,36 @@ +/* +* GRUB -- GRand Unified Bootloader +* Copyright (C) 2008 Free Software Foundation, Inc. +* +* GRUB is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation, either version 3 of the License, or +* (at your option) any later version. +* +* GRUB is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with GRUB. If not, see <http://www.gnu.org/licenses/>. +*/ + +#include <grub/types.h> +#include <grub/dl.h>
grub-core/fs/btrfs_zstd.c line 30
Quoted 10 lines
grub-core/fs/btrfs_zstd.c new file +10
-0+/* For NULL. */ +#include <grub/mm.h> +#include <grub/btrfs.h> +#include <grub/lib/zstd.h> + +GRUB_MOD_LICENSE ("GPLv3+" ); + +GRUB_MOD_INIT (btrfs_zstd) +{ + grub_btrfs_zstd_decompress_func = grub_zstd_decompress;
Please create a function in btrfs module which will do this for you.
grub-core/fs/btrfs_zstd.c line 35
Quoted 5 lines
grub-core/fs/btrfs_zstd.c new file +5
-0+} + +GRUB_MOD_FINI (btrfs_zstd) +{ + grub_btrfs_zstd_decompress_func =NULL ;
Ditto.
Last but not least, please CC original author of zstd support next time.
Daniel
Quoted 5 lines · Daniel Kiper
Quoted 4 lines · Michael Chang via Grub-devel
The zstd support in btrfs brings significant size increment to the on-disk image that it can no longer fit into btrfs bootloader area and short mbr gap.
Quoted 8 lines
Quoted 5 lines
In order to support grub update on outstanding i386-pc setup with these size constraints remain in place, here we build the zstd suppprt of btrfs into a separate module, named
btrfs_zstd, to alleviate the size change. Please note this only makes it's way to i386-pc, other architecture is not affected.I am OK with extracting zstd code from btrfs code. However, I want that be done for all architectures and platforms. No exceptions.
May I ask for more background about this decision ?
Given that btrfs compression is per file property and can be recompressed on the fly, there is no way to detect it beforehand thus the safest bet is to assume that it is always needed. In other words if the platform has no known limitation on the size of image, the zstd code should be indivisible to btrfs.mo.
Quoted 33 lines
[...]
Quoted 30 lines
grub-core/fs/btrfs_zstd.c new file +24
-0@@ -0,0 +1,36 @@ +/* +* GRUB -- GRand Unified Bootloader +* Copyright (C) 2008 Free Software Foundation, Inc. +* +* GRUB is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation, either version 3 of the License, or +* (at your option) any later version. +* +* GRUB is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with GRUB. If not, see <http://www.gnu.org/licenses/>. +*/ + +#include <grub/types.h> +#include <grub/dl.h> +/* For NULL. */ +#include <grub/mm.h> +#include <grub/btrfs.h> +#include <grub/lib/zstd.h>
Quoted 8 lines
grub-core/fs/btrfs_zstd.c line 30
Quoted 6 lines
grub-core/fs/btrfs_zstd.c new file +6
-0+ +GRUB_MOD_LICENSE ("GPLv3+" ); + +GRUB_MOD_INIT (btrfs_zstd) +{ + grub_btrfs_zstd_decompress_func = grub_zstd_decompress;Please create a function in btrfs module which will do this for you.
Sure. np.
Quoted 8 lines
grub-core/fs/btrfs_zstd.c line 35
Quoted 6 lines
grub-core/fs/btrfs_zstd.c new file +5
-0+} + +GRUB_MOD_FINI (btrfs_zstd) +{ + grub_btrfs_zstd_decompress_func =NULL ;Ditto.
OK.
Quoted 2 lines
Last but not least, please CC original author of zstd support next time.
OK. I will CC Nick Terrell next time.
Thank you very much for the review.
Regards,
Michael
Quoted 7 lines
Daniel
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Quoted 12 lines · Michael Chang via Grub-devel
Quoted 11 lines · Daniel Kiper
Quoted 9 lines · Michael Chang via Grub-devel
The zstd support in btrfs brings significant size increment to the on-disk image that it can no longer fit into btrfs bootloader area and short mbr gap.
In order to support grub update on outstanding i386-pc setup with these size constraints remain in place, here we build the zstd suppprt of btrfs into a separate module, named
btrfs_zstd, to alleviate the size change. Please note this only makes it's way to i386-pc, other architecture is not affected.
Quoted 4 lines
Quoted 2 lines
I am OK with extracting zstd code from btrfs code. However, I want that be done for all architectures and platforms. No exceptions.
May I ask for more background about this decision ?
Yes, I want the same code and behavior for all architectures and platforms if possible.
Quoted 5 lines
Given that btrfs compression is per file property and can be recompressed on the fly, there is no way to detect it beforehand thus the safest bet is to assume that it is always needed. In other words if the platform has no known limitation on the size of image, the zstd code should be indivisible to btrfs.mo.
Wait, you said this in the commit message too:
Therefore if the system has enough space of embedding area for grub then zstd support for btrfs will be enabled automatically in the process of running grub-install through inserting btrfs_zstd module to the on-disk image, otherwise a warning will be logged on screen to indicate user that zstd support for btrfs is disabled due to the size limit.
So, I thought this may work in the same way in all cases. If this is not true you have to fix this. Otherwise I cannot take this patch.
TBH, I should reject this patch immediately because this is "small MBR gap stuff". And as I said a few months ago, I want to stop to pay attention to "small MBR gap stuff" in upstream. Sorry for being blunt but it is full can of worms, i.e., each patch like that one adds more cruft which we have to take care because a few folks in the wild could not spend a few hours to repartition their systems. I think distros should start putting more effort in educating their users WRT to small MBR gap and problems related to it instead of pushing again and again upstream patches which make the GRUB code more complicated.
Daniel
Quoted 29 lines · Daniel Kiper
Quoted 12 lines · Michael Chang via Grub-devel
Quoted 11 lines · Daniel Kiper
Quoted 9 lines · Michael Chang via Grub-devel
The zstd support in btrfs brings significant size increment to the on-disk image that it can no longer fit into btrfs bootloader area and short mbr gap.
In order to support grub update on outstanding i386-pc setup with these size constraints remain in place, here we build the zstd suppprt of btrfs into a separate module, named
btrfs_zstd, to alleviate the size change. Please note this only makes it's way to i386-pc, other architecture is not affected.Quoted 4 lines
Quoted 2 lines
I am OK with extracting zstd code from btrfs code. However, I want that be done for all architectures and platforms. No exceptions.
May I ask for more background about this decision ?
Yes, I want the same code and behavior for all architectures and platforms if possible.
Quoted 5 lines
Given that btrfs compression is per file property and can be recompressed on the fly, there is no way to detect it beforehand thus the safest bet is to assume that it is always needed. In other words if the platform has no known limitation on the size of image, the zstd code should be indivisible to btrfs.mo.
Wait, you said this in the commit message too:
Quoted 8 lines
Therefore if the system has enough space of embedding area for grub then zstd support for btrfs will be enabled automatically in the process of running grub-install through inserting
btrfs_zstdmodule to the on-disk image, otherwise a warning will be logged on screen to indicate user that zstd support for btrfs is disabled due to the size limit.So, I thought this may work in the same way in all cases. If this is not true you have to fix this. Otherwise I cannot take this patch.
I literally don't like to propagate this (somewhat awkward) fix to other architectures that do not have sensible requirement to it, though it looks possible to do so.
If this doesn't meet the expection then I'm sorry.
Quoted 10 lines
TBH, I should reject this patch immediately because this is "small MBR gap stuff". And as I said a few months ago, I want to stop to pay attention to "small MBR gap stuff" in upstream. Sorry for being blunt but it is full can of worms, i.e., each patch like that one adds more cruft which we have to take care because a few folks in the wild could not spend a few hours to repartition their systems. I think distros should start putting more effort in educating their users WRT to small MBR gap and problems related to it instead of pushing again and again upstream patches which make the GRUB code more complicated.
Well, I think the short mbr gap issue is all well received been discussed several times. The reason I'm poking this beehive again is just that I can't resist to fix problem from our users who opted to use "btrfs partition" which differs to "short mbr gap" with a lot more user base and sensible usecases.
FWIW we want to address this problem, because mbr gap is adjustable via re-partitioning but btrfs bootloader area is not.
Thanks,
Michael
Quoted 7 lines
Daniel
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Quoted 55 lines · Michael Chang via Grub-devel
Quoted 29 lines · Daniel Kiper
Quoted 12 lines · Michael Chang via Grub-devel
Quoted 11 lines · Daniel Kiper
Quoted 9 lines · Michael Chang via Grub-devel
The zstd support in btrfs brings significant size increment to the on-disk image that it can no longer fit into btrfs bootloader area and short mbr gap.
In order to support grub update on outstanding i386-pc setup with these size constraints remain in place, here we build the zstd suppprt of btrfs into a separate module, named
btrfs_zstd, to alleviate the size change. Please note this only makes it's way to i386-pc, other architecture is not affected.Quoted 4 lines
Quoted 2 lines
I am OK with extracting zstd code from btrfs code. However, I want that be done for all architectures and platforms. No exceptions.
May I ask for more background about this decision ?
Yes, I want the same code and behavior for all architectures and platforms if possible.
Quoted 5 lines
Given that btrfs compression is per file property and can be recompressed on the fly, there is no way to detect it beforehand thus the safest bet is to assume that it is always needed. In other words if the platform has no known limitation on the size of image, the zstd code should be indivisible to btrfs.mo.
Wait, you said this in the commit message too:
Quoted 8 lines
Therefore if the system has enough space of embedding area for grub then zstd support for btrfs will be enabled automatically in the process of running grub-install through inserting
btrfs_zstdmodule to the on-disk image, otherwise a warning will be logged on screen to indicate user that zstd support for btrfs is disabled due to the size limit.So, I thought this may work in the same way in all cases. If this is not true you have to fix this. Otherwise I cannot take this patch.
I literally don't like to propagate this (somewhat awkward) fix to other architectures that do not have sensible requirement to it, though it looks possible to do so.
If this doesn't meet the expection then I'm sorry.
Quoted 9 lines
TBH, I should reject this patch immediately because this is "small MBR gap stuff". And as I said a few months ago, I want to stop to pay attention to "small MBR gap stuff" in upstream. Sorry for being blunt but it is full can of worms, i.e., each patch like that one adds more cruft which we have to take care because a few folks in the wild could not spend a few hours to repartition their systems. I think distros should start putting more effort in educating their users WRT to small MBR gap and problems related to it instead of pushing again and again upstream patches which make the GRUB code more complicated.
Quoted 2 lines
Well, I think the short mbr gap issue is all well received been discussed several times. The reason I'm poking this beehive again is
Cool! Thanks!
Quoted 6 lines
just that I can't resist to fix problem from our users who opted to use "btrfs partition" which differs to "short mbr gap" with a lot more user base and sensible usecases.
FWIW we want to address this problem, because mbr gap is adjustable via re-partitioning but btrfs bootloader area is not.
Huh! The "btrfs partition" sounds like not resizable MBR gap! I am not happy with it just at the beginning. Anyway, could you explain your use case in more details including example commands and why core.img seems landing in the "btrfs partition". I am especially curious about the latter because I think the "btrfs partition" and things like that was designed for something else, e.g., storing grubenv data. And why this solution is i386 specific?
Daniel
Quoted 5 lines · Daniel Kiper
Quoted 4 lines · Michael Chang via Grub-devel
Quoted 3 lines · Daniel Kiper
Quoted 2 lines · Michael Chang via Grub-devel
Quoted 1 line · Daniel Kiper
On Tue, Aug 31, 2021 at 03:12:28PM +0800, Michael Chang via Grub-devel wrote:
[snip]
Quoted 7 lines
Quoted 6 lines
just that I can't resist to fix problem from our users who opted to use "btrfs partition" which differs to "short mbr gap" with a lot more user base and sensible usecases.
FWIW we want to address this problem, because mbr gap is adjustable via re-partitioning but btrfs bootloader area is not.
Quoted 7 lines
Huh! The "btrfs partition" sounds like not resizable MBR gap! I am not happy with it just at the beginning. Anyway, could you explain your use case in more details including example commands and why core.img seems landing in the "btrfs partition". I am especially curious about the latter because I think the "btrfs partition" and things like that was designed for something else, e.g., storing grubenv data. And why this solution is i386 specific?
Installing to btrfs partition is not something exotic to grub, it is actually a supported usecase by grub for a long time. Also zfs has similar design, see:
c7ba4f698 Support BtrFS embedding.
ba102053c Support ZFS embedding.
To make it happen, you just have to point the btrfs device to grub-install and it will setup and embed image there. That worked quite nicely until the zstd support came into play.
On current git head, the command fails with core.img is too large.
# ./grub-install -d ./grub-core /dev/vda2 Installing for i386-pc platform. ./grub-install: warning: your core.img is unusually large. It won't fit in the embedding area. ./grub-install: error: filesystem `btrfs' doesn't support blocklists.
With this patch, the embedding still works but warns the user zstd is disabled, also instructing them to use MBR if they need the zstd feature to work.
# ./grub-install -d ./grub-core /dev/vda2 Installing for i386-pc platform. ./grub-install: warning: btrfs zstd compression is disabled, please change install device to disk. Installation finished. No error reported.
It is very i386-pc centric, as it is often used by legacy mbr boot code with which active partition is chainloaded. Some people still thinks this is the right thing to do and feels more comfortable this way to manage their multiple distrubution booting, compared with os-prober.
Thanks,
Michael
Quoted 7 lines
Daniel
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Quoted 51 lines · Michael Chang via Grub-devel
Quoted 5 lines · Daniel Kiper
Quoted 4 lines · Michael Chang via Grub-devel
Quoted 3 lines · Daniel Kiper
Quoted 2 lines · Michael Chang via Grub-devel
Quoted 1 line · Daniel Kiper
On Tue, Aug 31, 2021 at 03:12:28PM +0800, Michael Chang via Grub-devel wrote:
[snip]
Quoted 7 lines
Quoted 6 lines
just that I can't resist to fix problem from our users who opted to use "btrfs partition" which differs to "short mbr gap" with a lot more user base and sensible usecases.
FWIW we want to address this problem, because mbr gap is adjustable via re-partitioning but btrfs bootloader area is not.
Quoted 7 lines
Huh! The "btrfs partition" sounds like not resizable MBR gap! I am not happy with it just at the beginning. Anyway, could you explain your use case in more details including example commands and why core.img seems landing in the "btrfs partition". I am especially curious about the latter because I think the "btrfs partition" and things like that was designed for something else, e.g., storing grubenv data. And why this solution is i386 specific?
Installing to btrfs partition is not something exotic to grub, it is actually a supported usecase by grub for a long time. Also zfs has similar design, see:
c7ba4f698Support BtrFS embedding.
ba102053cSupport ZFS embedding.To make it happen, you just have to point the btrfs device to grub-install and it will setup and embed image there. That worked quite nicely until the zstd support came into play.
On current git head, the command fails with core.img is too large.
# ./grub-install -d ./grub-core /dev/vda2 Installing for i386-pc platform. ./grub-install: warning: your core.img is unusually large. It won't fit in the embedding area. ./grub-install: error: filesystem `btrfs' doesn't support blocklists.With this patch, the embedding still works but warns the user zstd is disabled, also instructing them to use MBR if they need the zstd feature to work.
# ./grub-install -d ./grub-core /dev/vda2 Installing for i386-pc platform. ./grub-install: warning: btrfs zstd compression is disabled, please change install device to disk. Installation finished. No error reported.
Quoted 4 lines
It is very i386-pc centric, as it is often used by legacy mbr boot code with which active partition is chainloaded. Some people still thinks this is the right thing to do and feels more comfortable this way to manage their multiple distrubution booting, compared with os-prober.
I do not like this solution because it is similar to all the patches trying to solve small MBR problem. Though, after chatting with Btrfs folks in Oracle, it seems to me there is better solution for this issue. I think we should make the GRUB installer smarter. If it cannot put the core image in first 64 KiB before primary superblock it should install the core image behind the primary superblock, i.e. starting from 64 KiB + 4 KiB but below 1 MiB. Please take a look at [1] which nicely explains the Btrfs bootloader support. Does it make sense for you?
Daniel
[1] https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs(5)#BOOTLOADER_SUPPORT
Quoted 57 lines · Daniel Kiper
Quoted 51 lines · Michael Chang via Grub-devel
Quoted 5 lines · Daniel Kiper
Quoted 4 lines · Michael Chang via Grub-devel
Quoted 3 lines · Daniel Kiper
Quoted 2 lines · Michael Chang via Grub-devel
Quoted 1 line · Daniel Kiper
On Tue, Aug 31, 2021 at 03:12:28PM +0800, Michael Chang via Grub-devel wrote:
[snip]
Quoted 7 lines
Quoted 6 lines
just that I can't resist to fix problem from our users who opted to use "btrfs partition" which differs to "short mbr gap" with a lot more user base and sensible usecases.
FWIW we want to address this problem, because mbr gap is adjustable via re-partitioning but btrfs bootloader area is not.
Quoted 7 lines
Huh! The "btrfs partition" sounds like not resizable MBR gap! I am not happy with it just at the beginning. Anyway, could you explain your use case in more details including example commands and why core.img seems landing in the "btrfs partition". I am especially curious about the latter because I think the "btrfs partition" and things like that was designed for something else, e.g., storing grubenv data. And why this solution is i386 specific?
Installing to btrfs partition is not something exotic to grub, it is actually a supported usecase by grub for a long time. Also zfs has similar design, see:
c7ba4f698Support BtrFS embedding.
ba102053cSupport ZFS embedding.To make it happen, you just have to point the btrfs device to grub-install and it will setup and embed image there. That worked quite nicely until the zstd support came into play.
On current git head, the command fails with core.img is too large.
# ./grub-install -d ./grub-core /dev/vda2 Installing for i386-pc platform. ./grub-install: warning: your core.img is unusually large. It won't fit in the embedding area. ./grub-install: error: filesystem `btrfs' doesn't support blocklists.With this patch, the embedding still works but warns the user zstd is disabled, also instructing them to use MBR if they need the zstd feature to work.
# ./grub-install -d ./grub-core /dev/vda2 Installing for i386-pc platform. ./grub-install: warning: btrfs zstd compression is disabled, please change install device to disk. Installation finished. No error reported.Quoted 4 lines
It is very i386-pc centric, as it is often used by legacy mbr boot code with which active partition is chainloaded. Some people still thinks this is the right thing to do and feels more comfortable this way to manage their multiple distrubution booting, compared with os-prober.
Quoted 8 lines
I do not like this solution because it is similar to all the patches trying to solve small MBR problem. Though, after chatting with Btrfs folks in Oracle, it seems to me there is better solution for this issue. I think we should make the GRUB installer smarter. If it cannot put the core image in first 64 KiB before primary superblock it should install the core image behind the primary superblock, i.e. starting from 64 KiB + 4 KiB but below 1 MiB. Please take a look at [1] which nicely explains the Btrfs bootloader support. Does it make sense for you?
Yes. The idea is brilliant and would be the sanest solution out there. I'll try to come up with a patch and discuss here.
Quoted 4 lines
Daniel
[1] https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs(5)#BOOTLOADER_SUPPORT
Thanks for pointing this out. I am looking for it for a long time as I used to come across this paragraph but not being able to make it again.
Thanks,
Michael
Quoted 5 lines
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel