Skip to contentSkip to conversation

New: [BUG] aacraid: DMA mapping leak in aac_send_raw_srb() causes eventual -ENOMEM failure

Bug 220504 [email protected] 1 participant 4 messages Aug 27, 2025 SCSI

lore
Share & tools
lore
#0 lore raw

https://bugzilla.kernel.org/show_bug.cgi?id=220504

Bug ID: 220504
Summary: [BUG] aacraid: DMA mapping leak in aac_send_raw_srb()
causes eventual -ENOMEM failure
Product: SCSI Drivers
Version: 2.5
Hardware: All
OS: Linux
Status: NEW
Severity: high
Priority: P3
Component: AACRAID
Assignee: [email protected]
Reporter: [email protected]
Regression: No

Created attachment 308556
--> https://bugzilla.kernel.org/attachment.cgi?id=308556&action=edit
Please refer to the details in the "Attachments" section of the main text

# Summary

On systems using the aacraid driver, repeated execution of FSACTL_SEND_RAW_SRB ioctls, such as from smartctl -d aacraid,..., triggers a DMA mapping leak in the aac_send_raw_srb() function. Over time, this exhausts DMA resources, leading to allocation failures (-ENOMEM, reported as -12) and eventual system instability or crash.

## Description

The aac_send_raw_srb() function in drivers/scsi/aacraid/commctrl.c is responsible for handling raw SRB commands from userspace. When processing a command with a scatter/gather (SG) list, the function iterates through the list, allocates kernel memory for each entry (kmalloc), and maps it for DMA using dma_map_single.

However, the corresponding dma_unmap_single call is missing from all code paths, including both the success and error-handling (cleanup) paths. Consequently, every SG entry processed through this ioctl leaks its DMA mapping.

This continuous resource leak eventually prevents new DMA mappings from being created, causing I/O operations to fail with -ENOMEM. This is observed in kernel logs as aac_read:

```
aac_read: aac_fib_send failed with status: -12
```

Eventually, the system becomes unstable or crashes.

## Affected Versions

Observed on AlmaLinux 9 kernels (5.14.x) through kernel-ml 6.14.9 from ELRepo. Based on code inspection, the issue likely persists in the latest upstream aacraid driver.

**Note:** AlmaLinux 9.x and kernel-ml packages use the upstream aacraid code without distribution-specific modifications.

## Steps to Reproduce

1. Use a system with an Adaptec RAID controller (e.g., ASR81605ZQ or ASR71605).
2. Run the following accelerated test repeatedly for 1–2 days:

   ```
   while true; do
       smartctl -d aacraid,N1,N2,N3 /dev/sdX
   done
   ```

(Replace N1,N2,N3 and device names as appropriate.)

3. Monitor with the attached bpftrace script (aac_leak_monitor_pre1.bt) to observe allocation/free imbalance during aac_send_raw_srb() execution.

## Observed Behavior

- During smartctl stress, alloc - free difference grows monotonically.
- Eventually, kernel logs show:

  ```
  aac_read: aac_fib_send failed with status: -12
  ```

followed by system hang or reboot.

- Other tools (e.g., arcconf / MaxView) may cause temporary diffs, but they return to zero.

## Sample Logs

```
kernel: aacraid: Host adapter abort request.
kernel: aacraid: Host bus reset request. SCSI hang?
kernel: aacraid 0000:12:00.0: outstanding cmd: midlevel-0
kernel: aacraid 0000:12:00.0: outstanding cmd: firmware-1
kernel: aacraid 0000:12:00.0: Issuing IOP reset
kernel: aacraid 0000:12:00.0: IOP reset succeeded
kernel: aac_read: aac_fib_send failed with status: -12.
(repeated hundreds of times before crash)
```

## bpftrace Output (excerpt)

```
Time Alloc: 1024 Free: 1010 Diff: 14
Bytes: 65536 Live: 14
...
Time Alloc: 2048 Free: 2010 Diff: 38
Bytes: 131072 Live: 38
```

The Diff value increases steadily during smartctl stress and never returns to zero, confirming a leak in the aac_send_raw_srb() scope.

## Workaround

Enable expose_physicals in aacraid so that smartctl does not require -d. In this mode, the leak does not occur.

## Additional Notes

Attached is a bpftrace script used to confirm the leak.
Also attached is an experimental patch (AI-assisted, not for direct use) that attempts to fix the leak by ensuring all DMA mappings are unmapped in cleanup paths. This patch is provided for reference only to highlight the suspected problematic areas in aac_send_raw_srb().
The final resolution should be determined by maintainers.
This patch was generated with the assistance of AI tools and adapted by a non-expert user. Applying AI-generated patches without a deep understanding of the subsystem and established development practices is not recommended. The final resolution should be determined by the maintainers.

_Some logs also contained the message "Host bus reset request. SCSI hang?", which initially led me to suspect a relation to the issue reported on linux-scsi (see: https://marc.info/?l=linux-scsi&m=168781894020549&w=2). I also tested with the recent patch associated with that report, but it appears unrelated to this problem. For the background and progression of my investigation until reaching this conclusion, please refer to the following
report: "0001514: kmod-aacraid Issue Resurfaces with ASR71605 on AlmaLinux 9.5"
(https://elrepo.org/bugs/view.php?id=1514)._

## Attachments

- aacraid-smartctl-test - script for accelerated test
- aac_leak_monitor_pre1.bt – bpftrace script for leak detection
- aacraid-fix-aac_send_raw_srb-memleak.patch – experimental patch (reference only)

## Test Environments

### **Test Environments A**

```
System:
  Host: serverA Kernel: 6.14.9-1.el9.elrepo.x86_64 arch: x86_64 bits: 64
  Console: pty pts/0 Distro: AlmaLinux 9.6 (Sage Margay)
Machine:
  Type: Unknown Mobo: ASRockRack model: X470D4U serial: M80-D8000100878 UEFI:
American Megatrends
    LLC. v: L4.29A date: 03/11/2024
Memory:
  System RAM: total: 32 GiB available: 30.21 GiB used: 18.28 GiB (60.5%)
  Array-1: capacity: 128 GiB slots: 4 modules: 2 EC: Multi-bit ECC
  Device-1: Channel-A DIMM 0 type: no module installed
  Device-2: Channel-A DIMM 1 type: DDR4 size: 16 GiB speed: 3200 MT/s
  Device-3: Channel-B DIMM 0 type: no module installed
  Device-4: Channel-B DIMM 1 type: DDR4 size: 16 GiB speed: 3200 MT/s
CPU:
  Info: 6-core model: AMD Ryzen 5 PRO 4650G with Radeon Graphics
```
```
# arcconf getconfig 1 AD | grep 'Model'
   Controller Model                           : Adaptec ASR81605ZQ
```
```
# arcconf getversion 1
Controllers found: 1
Controller #1
==============
Firmware                               : 7.18-0 (33556)
Staged Firmware                        : 7.18-0 (33556)
BIOS                                   : 7.18-0 (33556)
Driver                                 : 1.2-1 (50983)
Boot Flash                             : 7.18-0 (33556)
CPLD (Load version/ Flash version)     : 12/ 12
SEEPROM (Load version/ Flash version)  : 1/ 1
FCT Custom Init String Version         : 0x0
```

---

### **Test Environments B**

```
System:
  Host: serverB Kernel: 6.14.9-1.el9.elrepo.x86_64 arch: x86_64 bits: 64
  Console: pty pts/4 Distro: AlmaLinux 9.6 (Sage Margay)
Machine:
  Type: Unknown Mobo: ASRockRack model: X470D4U serial: M80-D6013900227
    UEFI-[Legacy]: American Megatrends v: P3.50 date: 11/02/2020
Memory:
  System RAM: total: 48 GiB available: 46.73 GiB used: 16.34 GiB (35.0%)
  Array-1: capacity: 128 GiB slots: 4 modules: 4 EC: Multi-bit ECC
  Device-1: Channel-A DIMM 0 type: DDR4 size: 16 GiB speed: spec: 3200 MT/s
actual: 2933 MT/s
  Device-2: Channel-A DIMM 1 type: DDR4 size: 8 GiB speed: spec: 3200 MT/s
actual: 2933 MT/s
  Device-3: Channel-B DIMM 0 type: DDR4 size: 16 GiB speed: spec: 3200 MT/s
actual: 2933 MT/s
  Device-4: Channel-B DIMM 1 type: DDR4 size: 8 GiB speed: spec: 3200 MT/s
actual: 2933 MT/s
CPU:
  Info: 6-core model: AMD Ryzen 5 2600
```
```
# arcconf getconfig 1 AD | grep 'Model'
   Controller Model                           : Adaptec ASR71605
```
```
# arcconf getversion 1
Controllers found: 1
Controller #1
==============
Firmware                               : 7.5-0 (32118)
Staged Firmware                        : 7.5-0 (32118)
BIOS                                   : 7.5-0 (32118)
Driver                                 : 1.2-1 (50983)
Boot Flash                             : 7.5-0 (32118)
CPLD (Load version/ Flash version)     : 8/ 10
SEEPROM (Load version/ Flash version)  : 1/ 1
```

---

### **Test Environments C**

```
System:
  Host: serverC Kernel: 6.14.9-1.el9.elrepo.x86_64 arch: x86_64 bits: 64
  Console: pty pts/0 Distro: AlmaLinux 9.6 (Sage Margay)
Machine:
  Type: Desktop Mobo: ASUSTeK model: ROG STRIX B350-F GAMING v: Rev X.0x
serial: 171012711504156
    UEFI: American Megatrends v: 6232 date: 09/29/2024
Memory:
  System RAM: total: 16 GiB available: 15.28 GiB used: 1.28 GiB (8.3%)
  Array-1: capacity: 128 GiB slots: 4 modules: 2 EC: None
  Device-1: DIMM_A1 type: no module installed
  Device-2: DIMM_A2 type: DDR4 size: 8 GiB speed: 2666 MT/s
  Device-3: DIMM_B1 type: no module installed
  Device-4: DIMM_B2 type: DDR4 size: 8 GiB speed: 2666 MT/s
CPU:
  Info: 6-core model: AMD Ryzen 5 1600
```
```
# arcconf getconfig 1 AD | grep 'Model'
   Controller Model                           : Adaptec ASR71605
```
```
# arcconf getversion 1
Controllers found: 1
Controller #1
==============
Firmware                               : 7.5-0 (32118)
Staged Firmware                        : 7.5-0 (32118)
BIOS                                   : 7.5-0 (32118)
Driver                                 : 1.2-1 (50983)
Boot Flash                             : 7.5-0 (32118)
CPLD (Load version/ Flash version)     : 7/ 10
SEEPROM (Load version/ Flash version)  : 0/ 1
```
Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.
#1 lore raw

[Bug 220504] [BUG] aacraid: DMA mapping leak in aac_send_raw_srb() causes eventual -ENOMEM failure

https://bugzilla.kernel.org/show_bug.cgi?id=220504

--- Comment #1 from UMEMURA Shobu ([email protected]) --- Created attachment 308557
--> https://bugzilla.kernel.org/attachment.cgi?id=308557&action=edit
script for accelerated test

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.
#2 lore raw

[Bug 220504] [BUG] aacraid: DMA mapping leak in aac_send_raw_srb() causes eventual -ENOMEM failure

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=220504

--- Comment #2 from UMEMURA Shobu ([email protected]) --- Created attachment 308558
--> https://bugzilla.kernel.org/attachment.cgi?id=308558&action=edit
bpftrace script for leak detection

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.
#3 lore raw

[Bug 220504] [BUG] aacraid: DMA mapping leak in aac_send_raw_srb() causes eventual -ENOMEM failure

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=220504

--- Comment #3 from UMEMURA Shobu ([email protected]) --- Created attachment 308559
--> https://bugzilla.kernel.org/attachment.cgi?id=308559&action=edit
experimental patch (reference only)

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.