Error with --no-huge when compiled with -fsanitize=address from gcc 15

Hi folks,

I recently updated to Fedora 42 that comes with GCC 15.

When building with -fsanitize=address (libasan.so.8), it seems that the --no-huge mode (along with --no-shconf) fails at initialization for some obscure reason.

"couldn't allocate memory due to IOVA exceeding limits of current DMA mask."

I have looked around everywhere and I don't see how to get around this issue.

It is working fine with GCC 14. Maybe some changes were made to libasan related to the address ranges exposed to the mocked allocation functions.

Is this something that needs to be fixed in DPDK?

Here is a reproducer:

$ meson setup build -Dtests=false -Denable_docs=false -Ddeveloper_mode=disabled \
	-Denable_drivers=net/tap -Denable_libs=gso -Denable_apps=test-pmd \
	-Db_sanitize=address
The Meson build system
Version: 1.7.0
Source dir: /home/rjarry/upstream/dpdk
Build dir: /home/rjarry/upstream/dpdk/build
Build type: native build
Project name: DPDK
Project version: 25.03.0
C compiler for the host machine: ccache cc (gcc 15.1.1 "cc (GCC) 15.1.1 20250425 (Red Hat 15.1.1-1)")
C linker for the host machine: cc ld.bfd 2.44-3
Host machine cpu family: x86_64
Host machine cpu: x86_64
...
Library asan found: YES
...
$ ninja -C build
ninja: Entering directory `build'
[259/259] Linking target app/dpdk-testpmd

$ ldd build/app/dpdk-testpmd | grep asan
	libasan.so.8 => /lib64/libasan.so.8 (0x00007fac50e00000)

$ rpm -qa *asan*
libasan-15.1.1-1.fc42.x86_64

$ build/app/dpdk-testpmd -l 0 --no-shconf --no-huge -m 2048 --log-level=lib.eal:debug
...
EAL: Buses did not request a specific IOVA mode.
EAL: Physical addresses are unavailable, selecting IOVA as VA mode.
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: Module /sys/module/vfio not found! error 2 (No such file or directory)
EAL: VFIO modules not loaded, skipping VFIO support...
EAL: Ask a virtual area of 0x2e000 bytes
EAL: WARNING! Base virtual address hint (0x100005000 != 0x7f9bc4cc1000) not respected!
EAL:    This may cause issues with mapping memory into secondary processes
EAL: Virtual area found at 0x7f9bc4cc1000 (size = 0x2e000)
EAL: Setting up physically contiguous memory...
EAL: Setting maximum number of open files to 524288
EAL: Ask a virtual area of 0x1811000 bytes
EAL: WARNING! Base virtual address hint (0x10000b000 != 0x7b9bbeae5000) not respected!
EAL:    This may cause issues with mapping memory into secondary processes
EAL: Virtual area found at 0x7b9bbeae5000 (size = 0x1811000)
EAL: Memseg list allocated at socket 0, page size 0x4kB
EAL: Using memfd for anonymous memory
EAL: Ask a virtual area of 0x80000000 bytes
EAL: WARNING! Base virtual address hint (0x100011000 != 0x7b9b3eae5000) not respected!
EAL:    This may cause issues with mapping memory into secondary processes
EAL: Virtual area found at 0x7b9b3eae5000 (size = 0x80000000)
EAL: VA reserved for memseg list at 0x7b9b3eae5000, size 80000000
EAL: memseg iova 7b9b3eae5000, len 1000, out of range
EAL: 	using dma mask fffffc0000000000
EAL: eal_legacy_hugepage_init(): couldn't allocate memory due to IOVA exceeding limits of current DMA mask.
EAL: Cannot init memory
EAL: Error - exiting with code: 1
Cannot init EAL: Cannot allocate memory

Does that ring a bell to anyone?

Robin

+Anatoly and Ferruh in direct address.

Robin Jarry, May 15, 2025 at 12:32:

Hi Anatoly, Ferruh, all,

Allow me to bump this. GCC 15 has been out for quite a while now and it seems that libasan cannot be used anymore with the DPDK memory allocator as it is.

I have tracked the commit that introduced this warning up to 2018:

https://git.dpdk.org/dpdk/commit/?id=165c89b84538f

I would be much grateful if you could help. I don't know the internals of libasan, but I assume it shuffles mapped memory around and it conflicts with the DPDK allocator inner workings.

I have found this libasan commit which "could" be related:

https://github.com/llvm/llvm-project/commit/7ede1c497302

In fact, part of it was reverted a while after on macos because of suspected breakage:

https://github.com/llvm/llvm-project/commit/4e332bba2f3a

Robin

Hi Robin,

I have been trying to reproduce this but I'm unable to. I run some old CentOS so I built GCC15 manually and tried reproducing with that. The version I'm using is 15.2, could you please retest with latest GCC15 and see if it was fixed?

(if not, I'll try to get a more recent distro and reproduce again)

The DMA mask checking you're referring to is there for cases where e.g. in a VM the emulated IOMMU will not have full address width (39 bits was common at the time), and thus we couldn't use the full VA space for IOVA addressing, and had to resort to using real physical addresses (because kernel ensure those are to be within support range of IOMMU). Were you running this test in a VM, by any chance?

Thanks,
Anatoly

Hey Anatoly,

Burakov, Anatoly, Sep 04, 2025 at 16:35:

I am running on Fedora 42 that has GCC 15.2.1 and the problem persists.

No I am running on a physical machine. I tested again without --no-huge and 2M/1G huge pages, and the issue persists as well.

I tried both as root, and as a regular user.

Robin

It should work if you use --iova-mode=pa (even for VM case), this will require root though. However, I don't quite get how you get the error that you do.

Specifically, it's weird that your DMA width is 42 bits (indicated by "using dma mask fffffc0000000000" log), running on a physical machine - mine sits at 57 bits, so despite having VA's within the same range as yours, this works on my machine because my DMA mask is much wider. Maybe there's a kernel command-line of some sort that limits your DMA width?

At first glance I don't think the issue has anything to do with DPDK, at least it doesn't look like it does to me. I do not see how DMA mask width would be affected by libasan or GCC15, to me it sounds more like a kernel configuration/HW setup problem.

Thanks,
Anatoly

Hi Anatoly,

Burakov, Anatoly, Sep 05, 2025 at 11:17:

NB: The problem *only* appears when I compile with -fsanitize=address. Hence my suspicion around GCC/libasan. There must be something done by ASAN that changes the address space.

Digging further with your advice (and some help from AI...), I found that the DMA width (42 bits) as reported by EAL debug logs actually come from my laptop having a new "security feature" enabled in the BIOS:

Kernel DMA Protection

When this "feature" is enabled, IOMMU is forced-on during early boot:

DMAR: Intel-IOMMU force enabled due to platform opt in

Also, my IOMMU exposes a limited DMA width (I don't know why). My CPU model is:

	Intel(R) Core(TM) Ultra 7 165U

I have disabled DMA protection in the BIOS, and the problem is now gone.

Interestingly, whenever I explicitly enable IOMMU in the kernel boot arguments (iommu=pt intel_iommu=on), the issue manifests itself again. This is therefore a weird interaction between ASAN, the IOMMU on my laptop and DPDK memory initialization.

Anyway, sorry for the noise. Maybe, my experience may be useful to others :)

Thanks for your assistance!

Robin

Hi Robin,

ASAN wouldn't interfere with address space itself but it will cause the addresses to go higher (the default starting address is such that it should fit within even smaller DMA masks so normally this isn't an issue) because of shadowing, so while ASAN itself doesn't affect the DMA mask, it will affect the addresses you get.

This is interesting. I've never dealt with running DPDK on a laptop, maybe the limited DMA width is endemic to laptops or something.

No problem!

Thanks,
Anatoly