Skip to contentSkip to conversation

SWIOTLB on 32-bit PAE.

DISCUSSION Christian Melki 5 participants 6 messages Oct 2, 2015 – Oct 5, 2015 3 messages not in the archive PCI

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

I discovered a strange error on my machine. 32-bit PAE 4.2.0 without IOMMU code (yeah, I know).
When writing to an ext4 filesystem on a USB disk my kernel would hang and not return control to userspace. It would spew kernel messages to syslog as fast as it could without giving userspace any time, ie "hang" as far as a user sees it. The device itself is ok and the filesystem is clean.
I get messages like this, millions of them.

Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:

overflow 25dcac000+1024 of device mask ffffffff

I disovered that SWIOTLB was not on. It is only provided for 64-bit
machines where it is default. On 32-bit SWIOTLB seems disabled unless you are running with some IOMMU/AGPGART code.

But what about 32-bit PAE? Would that not qualify for SWIOTLB without IOMMU?

I am a novice in this area. Maybe this is a Kconfig "bug" for us that like to hang ourselves with CONFIG_* rope? :) I'd like to think that SWIOTLB should be default for 32-bit PAE too?

Best regards,
Christian

#2 lore raw

[+cc Konrad, Joerg, iommu list]

Quoted 34 lines · Christian Melki

I discovered a strange error on my machine. 32-bit PAE 4.2.0 without IOMMU code (yeah, I know).
When writing to an ext4 filesystem on a USB disk my kernel would hang and not return control to userspace. It would spew kernel messages to syslog as fast as it could without giving userspace any time, ie "hang" as far as a user sees it. The device itself is ok and the filesystem is clean.
I get messages like this, millions of them.

Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:

overflow 25dcac000+1024 of device mask ffffffff

I disovered that SWIOTLB was not on. It is only provided for 64-bit machines where it is default. On 32-bit SWIOTLB seems disabled unless you are running with some IOMMU/AGPGART code.

But what about 32-bit PAE? Would that not qualify for SWIOTLB without IOMMU?

I am a novice in this area. Maybe this is a Kconfig "bug" for us that like to hang ourselves with CONFIG_* rope? :) I'd like to think that SWIOTLB should be default for 32-bit PAE too?

Best regards,
Christian

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
Quoted 24 lines · Bjorn Helgaas

[+cc Konrad, Joerg, iommu list]

Quoted 21 lines · Christian Melki

I discovered a strange error on my machine. 32-bit PAE 4.2.0 without IOMMU code (yeah, I know).
When writing to an ext4 filesystem on a USB disk my kernel would hang and not return control to userspace. It would spew kernel messages to syslog as fast as it could without giving userspace any time, ie "hang" as far as a user sees it. The device itself is ok and the filesystem is clean.
I get messages like this, millions of them.

Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:

overflow 25dcac000+1024 of device mask ffffffff

I disovered that SWIOTLB was not on. It is only provided for 64-bit machines where it is default. On 32-bit SWIOTLB seems disabled unless you are running with some IOMMU/AGPGART code.

But what about 32-bit PAE? Would that not qualify for SWIOTLB without IOMMU?

It most certainly would!

Quoted 4 lines
Quoted 4 lines

I am a novice in this area. Maybe this is a Kconfig "bug" for us that like to hang ourselves with CONFIG_* rope? :) I'd like to think that SWIOTLB should be default for 32-bit PAE too?

Aye. I presume that you had done a small change already for this? Would you be willing to post it on lkml and CC me ?

Thank you.

Quoted 9 lines
Quoted 9 lines

Best regards,
Christian

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
Quoted 2 lines · Konrad Rzeszutek Wilk

Aye. I presume that you had done a small change already for this? Would you be willing to post it on lkml and CC me ?

Yes, I also think it is needed. SWIOTLB already correctly handles buffers in HighMem, so enabling it for 32-bit PAE should be easy. Please also CC me on the patch.

Joerg

A message that is not in the archive (sent off-list or not yet received): 20151003180007.GA4732-bi+AKbBUZKY6gyzm1THtWbp2dZbC/[email protected]

#6 lore raw

Hi.

I think I rang the bell a bit early. The corruption is in grub legacy (I assumed that would be in Linux too, but it wasn't). The on disk format of ext4 apparently stores byte ordering somehow? I thought the on disk format of ext4 was a specific endian (little). Since only that file is unreadable in grub, but readable in Linux.. I am guessing endian issues somehow.. which still is strange.

Regards,
Christian

Quoted 45 lines · Konrad Rzeszutek Wilk
Quoted 24 lines · Bjorn Helgaas

[+cc Konrad, Joerg, iommu list]

Quoted 21 lines · Christian Melki

I discovered a strange error on my machine. 32-bit PAE 4.2.0 without IOMMU code (yeah, I know).
When writing to an ext4 filesystem on a USB disk my kernel would hang and not return control to userspace. It would spew kernel messages to syslog as fast as it could without giving userspace any time, ie "hang" as far as a user sees it. The device itself is ok and the filesystem is clean.
I get messages like this, millions of them.

Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:

overflow 25dcac000+1024 of device mask ffffffff

I disovered that SWIOTLB was not on. It is only provided for 64-bit machines where it is default. On 32-bit SWIOTLB seems disabled unless you are running with some IOMMU/AGPGART code.

But what about 32-bit PAE? Would that not qualify for SWIOTLB without IOMMU?

It most certainly would!

Quoted 4 lines
Quoted 4 lines

I am a novice in this area. Maybe this is a Kconfig "bug" for us that like to hang ourselves with CONFIG_* rope? :) I'd like to think that SWIOTLB should be default for 32-bit PAE too?

Aye. I presume that you had done a small change already for this? Would you be willing to post it on lkml and CC me ?

Thank you.

Quoted 9 lines
Quoted 9 lines

Best regards,
Christian

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw

Never mind my ranting.
The problem was in the extent disk format of ext4. I had forgot that ext4 uses extents for all new files by default. Old grub won't read extents.

Oh, well... nothing like wasting hard working peoples time with stupid things. Anyway. SWIOTLB for 32-bit PAE should be a good thing though. :)

Regards,
Christian

Quoted 59 lines · Christian Melki

Hi.

I think I rang the bell a bit early. The corruption is in grub legacy (I assumed that would be in Linux too, but it wasn't). The on disk format of ext4 apparently stores byte ordering somehow? I thought the on disk format of ext4 was a specific endian (little). Since only that file is unreadable in grub, but readable in Linux.. I am guessing endian issues somehow.. which still is strange.

Regards,
Christian

Quoted 46 lines · Konrad Rzeszutek Wilk
Quoted 25 lines · Bjorn Helgaas

[+cc Konrad, Joerg, iommu list]

Quoted 22 lines · Christian Melki

I discovered a strange error on my machine. 32-bit PAE 4.2.0 without IOMMU code (yeah, I know).
When writing to an ext4 filesystem on a USB disk my kernel would hang and not return control to userspace. It would spew kernel messages to syslog as fast as it could without giving userspace any time, ie "hang" as far as a user sees it. The device itself is ok and the filesystem is clean.
I get messages like this, millions of them.

Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:

overflow 25dcac000+1024 of device mask ffffffff

I disovered that SWIOTLB was not on. It is only provided for 64-bit machines where it is default. On 32-bit SWIOTLB seems disabled unless you are running with some IOMMU/AGPGART code.

But what about 32-bit PAE? Would that not qualify for SWIOTLB without IOMMU?

It most certainly would!

Quoted 4 lines
Quoted 4 lines

I am a novice in this area. Maybe this is a Kconfig "bug" for us that like to hang ourselves with CONFIG_* rope? :) I'd like to think that SWIOTLB should be default for 32-bit PAE too?

Aye. I presume that you had done a small change already for this? Would you be willing to post it on lkml and CC me ?

Thank you.

Quoted 9 lines
Quoted 9 lines

Best regards,
Christian

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html