Skip to contentSkip to conversation

QEMU fw_cfg DMA interface

PATCH Superseded Marc Marí 7 participants 29 messages Aug 31, 2015 – Sep 10, 2015 Device tree bindings

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Marc Marí

author
#0 lore raw

Implementation of the FW CFG DMA interface.

When running a Linux guest on top of QEMU, using the -kernel options, this is the timing improvement for x86:

QEMU commit 090d0bf and SeaBIOS commit 2fc20dc
QEMU startup time: .078
BIOS startup time: .060
Kernel setup time: .578
Total time: .716

QEMU with this patch series and SeaBIOS with this patch series QEMU startup time: .080
BIOS startup time: .039
Kernel setup time: .002
Total time: .121

QEMU startup time is the time between the start and the first kvm_entry. BIOS startup time is the time between the first kvm_entry and the start of function do_boot, in SeaBIOS.
Kernel setup time is the time between the start of the function do_boot in SeaBIOS and the jump to the Linux kernel.

As you can see, both the BIOS (because of ACPI tables and other configurations) and the Linux kernel boot (because of the copy to memory) are greatly improved with this new interface.

Also, this new interface is an addon to the old interface. Both interfaces are compatible and interchangeable.

Changes from v1:
- Take into account order of fields in the FWCfgDmaAccess structure
- Check and change endianness of FWCfgDmaAccess fields
- Change order of fields in the FWCfgDmaAccess structure
- Add FW_CFG_DMA_CTL_SKIP feature for control field
- Split FW_CFG_SIZE in QEMU
- Make FW_CFG_ID a bitmap of features
- Add 64 bit address support for the transfer. Trigger when writing the low address, and address is 0 by default and at the end of each transfer.
- Align ports and addresses.
- Preserve old fw_cfg_comb_valid behaviour in QEMU
- Update documentation to reflect all these changes
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#1 lore raw

v2 cover fw_cfg DMA interface

Implement host-side of the FW CFG DMA interface both for x86 and ARM.

Based on Gerd Hoffman's initial implementation.

Gabriel L. Somlo (1):
fw_cfg: document fw_cfg_modify_iXX() update functions

Marc Marí (4):
fw_cfg DMA interface documentation
Implement fw_cfg DMA interface
Enable fw_cfg DMA interface for ARM
Enable fw_cfg DMA interface for x86

 docs/specs/fw_cfg.txt     |  79 +++++++++++++-
 hw/arm/virt.c             |  13 ++-
 hw/i386/pc.c              |  11 +-
 hw/nvram/fw_cfg.c         | 261 +++++++++++++++++++++++++++++++++++++++++++---
 include/hw/nvram/fw_cfg.h |  15 ++-
 5 files changed, 351 insertions(+), 28 deletions(-)
Signature · 1 line
2.4.3
author
#2 lore raw

PATCH v2 1/5 fw_cfg: document fw_cfg_modify_iXX() update functions

From: "Gabriel L. Somlo" <[email protected]>

Document the behavior of fw_cfg_modify_iXX() for leak-less updating of integer-type blobs.

Currently only fw_cfg_modify_i16() is coded, but 32- and 64-bit versions may be added later if necessary..

---
 docs/specs/fw_cfg.txt | 11 +++++++++++
 1 file changed, 11 insertions(+)
docs/specs/fw_cfg.txt +11 -0
index 74351dd..5bc7b96 100644
@@ -159,6 +159,17 @@ will convert a 16-, 32-, or 64-bit integer to little-endian, then add
 a dynamically allocated copy of the appropriately sized item to fw_cfg
 under the given selector key value.
 
+== fw_cfg_modify_iXX() ==
+
+Modify the value of an XX-bit item (where XX may be 16, 32, or 64).
+Similarly to the corresponding fw_cfg_add_iXX() function set, convert
+a 16-, 32-, or 64-bit integer to little endian, create a dynamically
+allocated copy of the required size, and replace the existing item at
+the given selector key value with the newly allocated one. The previous
+item, assumed to have been allocated during an earlier call to
+fw_cfg_add_iXX() or fw_cfg_modify_iXX() (of the same width XX), is freed
+before the function returns.
+
 == fw_cfg_add_file() ==
 
 Given a filename (i.e., fw_cfg item name), starting pointer, and size,
Signature · 1 line
2.4.3
#3 lore raw

Re: [Qemu-devel] [PATCH v2 1/5] fw_cfg: document fw_cfg_modify_iXX() update functions

Quoted 27 lines · Marc Marí

From: "Gabriel L. Somlo" <[email protected]>

Document the behavior of fw_cfg_modify_iXX() for leak-less updating of integer-type blobs.

Currently only fw_cfg_modify_i16() is coded, but 32- and 64-bit versions may be added later if necessary..

---
 docs/specs/fw_cfg.txt | 11 +++++++++++
 1 file changed, 11 insertions(+)
docs/specs/fw_cfg.txt +4 -0
index 74351dd..5bc7b96 100644
@@ -159,6 +159,17 @@ will convert a 16-, 32-, or 64-bit integer to little-endian, then add
 a dynamically allocated copy of the appropriately sized item to fw_cfg
 under the given selector key value.
 
+== fw_cfg_modify_iXX() ==
+
+Modify the value of an XX-bit item (where XX may be 16, 32, or 64).
+Similarly to the corresponding fw_cfg_add_iXX() function set, convert

docs/specs/fw_cfg.txt will convert a 16-, 32-, or 64-bit integer to little-endian, then add · line 175

Quoted 10 lines
docs/specs/fw_cfg.txt +7 -0
+a 16-, 32-, or 64-bit integer to little endian, create a dynamically
+allocated copy of the required size, and replace the existing item at
+the given selector key value with the newly allocated one. The previous
+item, assumed to have been allocated during an earlier call to
+fw_cfg_add_iXX() or fw_cfg_modify_iXX() (of the same width XX), is freed
+before the function returns.
+
 == fw_cfg_add_file() ==
 
 Given a filename (i.e., fw_cfg item name), starting pointer, and size,

This doesn't cover fw_cfg_modify_file(); is that intentional?

As an aside, shouldn't this function-level documentation be done via doc-comments in the header file where the prototypes are declared? (You don't need to move the docs around in this series, but it might be nice to do at some point.)

thanks
-- PMM

#4 lore raw

Re: [Qemu-devel] [PATCH v2 1/5] fw_cfg: document fw_cfg_modify_iXX() update functions

Quoted 37 lines · Peter Maydell
Quoted 36 lines · Marc Marí

From: "Gabriel L. Somlo" <[email protected]>

Document the behavior of fw_cfg_modify_iXX() for leak-less updating of integer-type blobs.

Currently only fw_cfg_modify_i16() is coded, but 32- and 64-bit versions may be added later if necessary..

---
 docs/specs/fw_cfg.txt | 11 +++++++++++
 1 file changed, 11 insertions(+)
docs/specs/fw_cfg.txt +11 -0
index 74351dd..5bc7b96 100644
@@ -159,6 +159,17 @@ will convert a 16-, 32-, or 64-bit integer to little-endian, then add
 a dynamically allocated copy of the appropriately sized item to fw_cfg
 under the given selector key value.
 
+== fw_cfg_modify_iXX() ==
+
+Modify the value of an XX-bit item (where XX may be 16, 32, or 64).
+Similarly to the corresponding fw_cfg_add_iXX() function set, convert
+a 16-, 32-, or 64-bit integer to little endian, create a dynamically
+allocated copy of the required size, and replace the existing item at
+the given selector key value with the newly allocated one. The previous
+item, assumed to have been allocated during an earlier call to
+fw_cfg_add_iXX() or fw_cfg_modify_iXX() (of the same width XX), is freed
+before the function returns.
+
 == fw_cfg_add_file() ==
 
Quoted 3 lines

docs/specs/fw_cfg.txt will convert a 16-, 32-, or 64-bit integer to little-endian, then add · line 175

Quoted 1 line
docs/specs/fw_cfg.txt +0 -0
 Given a filename (i.e., fw_cfg item name), starting pointer, and size,

This doesn't cover fw_cfg_modify_file(); is that intentional?

There should already be a paragraph in there (further down from where this is supposed to go in) describing fw_cfg_modify_file(), which isn't affected by this.

Quoted 5 lines

As an aside, shouldn't this function-level documentation be done via doc-comments in the header file where the prototypes are declared? (You don't need to move the docs around in this series, but it might be nice to do at some point.)

You mean, leave docs/specs/fw_cfg.txt to deal with the guest-side port/mmio api only, and have the host-side functions simply documented as comments in include/hw/nvram/fw_cfg.h ?

That should be relatively painless, if that's the agreed-upon convention...

Thanks,
--Gabriel

#5 lore raw

Re: [Qemu-devel] [PATCH v2 1/5] fw_cfg: document fw_cfg_modify_iXX() update functions

Quoted 6 lines · Gabriel L. Somlo
Quoted 1 line · Peter Maydell

This doesn't cover fw_cfg_modify_file(); is that intentional?

There should already be a paragraph in there (further down from where this is supposed to go in) describing fw_cfg_modify_file(), which isn't affected by this.

Sorry, so there is.

Quoted 11 lines
Quoted 4 lines

As an aside, shouldn't this function-level documentation be done via doc-comments in the header file where the prototypes are declared? (You don't need to move the docs around in this series, but it might be nice to do at some point.)

You mean, leave docs/specs/fw_cfg.txt to deal with the guest-side port/mmio api only, and have the host-side functions simply documented as comments in include/hw/nvram/fw_cfg.h ?

That should be relatively painless, if that's the agreed-upon convention...

Yes. I think at the point this file was written we probably hadn't started using doc-comments for our APIs. (My usual place to crib the formatting for doc-comments is the extract/deposit APIs in bitops.h.)

thanks
-- PMM

#6 lore raw

Re: [Qemu-devel] [PATCH v2 1/5] fw_cfg: document fw_cfg_modify_iXX() update functions

Quoted 11 lines · Peter Maydell
Quoted 10 lines · Gabriel L. Somlo
Quoted 4 lines · Peter Maydell

As an aside, shouldn't this function-level documentation be done via doc-comments in the header file where the prototypes are declared? (You don't need to move the docs around in this series, but it might be nice to do at some point.)

You mean, leave docs/specs/fw_cfg.txt to deal with the guest-side port/mmio api only, and have the host-side functions simply documented as comments in include/hw/nvram/fw_cfg.h ?

Quoted 7 lines
Quoted 2 lines

That should be relatively painless, if that's the agreed-upon convention...

Yes. I think at the point this file was written we probably hadn't started using doc-comments for our APIs. (My usual place to crib the formatting for doc-comments is the extract/deposit APIs in bitops.h.)

OK, I guess I'll wait until after the dust settles on fw_cfg/DMA before further mucking with the doc or header files, but again, this shouldn't be too painless...

Also, since I'll be tinkering with fw_cfg again, and you mentioned using DT on arm and ACPI on x86 to auto-detect the presence (and location) of fw_cfg from the guest-side in a related thread:

Right now, I don't think fw_cfg is listed in ACPI, and I would like it to be. So what should I do, simply figure out how to add a new node somewhere in the SSDT and submit a patch for that ? Could it be that simple, or am I missing something ? :)

Thanks,
--Gabriel

#7 lore raw

Re: [Qemu-devel] [PATCH v2 1/5] fw_cfg: document fw_cfg_modify_iXX() update functions

Quoted 3 lines · Gabriel L. Somlo

Also, since I'll be tinkering with fw_cfg again, and you mentioned using DT on arm and ACPI on x86 to auto-detect the presence (and location) of fw_cfg from the guest-side in a related thread:

I meant DT or ACPI on ARM, actually. I don't know what the x86 approach is for fw_cfg but I think it's just "known address". (For ARM we don't I think currently expose fw_cfg in the ACPI tables, but we probably ought to; only needed for the edge case where you actually wanted to read fw_cfg in the guest kernel rather than just the guest firmware, though.)

thanks
-- PMM

#8 lore raw

Re: [Qemu-devel] [PATCH v2 1/5] fw_cfg: document fw_cfg_modify_iXX() update functions

Quoted 11 lines · Peter Maydell
Quoted 3 lines · Gabriel L. Somlo

Also, since I'll be tinkering with fw_cfg again, and you mentioned using DT on arm and ACPI on x86 to auto-detect the presence (and location) of fw_cfg from the guest-side in a related thread:

I meant DT or ACPI on ARM, actually. I don't know what the x86 approach is for fw_cfg but I think it's just "known address". (For ARM we don't I think currently expose fw_cfg in the ACPI tables, but we probably ought to; only needed for the edge case where you actually wanted to read fw_cfg in the guest kernel rather than just the guest firmware, though.)

Right, that's what I'm trying to do -- expose fw_cfg in /sys/firmware/... on the guest -- which involves figuring out how to determine if it is present (DT on ARM, blindly poking port 0x510 on x86 for now, hence my interest in having it listed in ACPI).

Now, if we did expose fw_cfg in ACPI on *both* ARM and x86, I wonder I could get away with an ACPI-only detection scheme across both architectures... But ACPI is all we have on x86, so on the QEMU side of things it looks like I'll have to add it regardless...

Thanks,
--Gabriel

#9 lore raw

Re: [Qemu-devel] [PATCH v2 1/5] fw_cfg: document fw_cfg_modify_iXX() update functions

Quoted 8 lines · Gabriel L. Somlo

Right, that's what I'm trying to do -- expose fw_cfg in /sys/firmware/... on the guest -- which involves figuring out how to determine if it is present (DT on ARM, blindly poking port 0x510 on x86 for now, hence my interest in having it listed in ACPI).

Now, if we did expose fw_cfg in ACPI on *both* ARM and x86, I wonder I could get away with an ACPI-only detection scheme across both architectures...

Nope, because there's no requirement for ACPI on ARM -- you could be booting the kernel via device tree.

thanks
-- PMM

#10 lore raw

Re: [Qemu-devel] [PATCH v2 1/5] fw_cfg: document fw_cfg_modify_iXX() update functions

In reply to Peter Maydell
Quoted 7 lines · Peter Maydell
Quoted 3 lines · Gabriel L. Somlo

Also, since I'll be tinkering with fw_cfg again, and you mentioned using DT on arm and ACPI on x86 to auto-detect the presence (and location) of fw_cfg from the guest-side in a related thread:

I meant DT or ACPI on ARM, actually. I don't know what the x86 approach is for fw_cfg but I think it's just "known address".

Yes, "known address". And given that the firmware actually loads the acpi tables via fw_cfg that is very unlikely to change. Adding it to the acpi tables might be useful nevertheless so the guest os knows the ioports are in use.

I think on arm the acpi situation is the same, but I think firmware detecting the location via DT should work without chicken&egg problems.

cheers,
Gerd

#11 lore raw

Re: [Qemu-devel] [PATCH v2 1/5] fw_cfg: document fw_cfg_modify_iXX() update functions

Quoted 9 lines · Gerd Hoffmann
Quoted 7 lines · Peter Maydell
Quoted 3 lines · Gabriel L. Somlo

Also, since I'll be tinkering with fw_cfg again, and you mentioned using DT on arm and ACPI on x86 to auto-detect the presence (and location) of fw_cfg from the guest-side in a related thread:

I meant DT or ACPI on ARM, actually. I don't know what the x86 approach is for fw_cfg but I think it's just "known address".

Quoted 7 lines

Yes, "known address". And given that the firmware actually loads the acpi tables via fw_cfg that is very unlikely to change. Adding it to the acpi tables might be useful nevertheless so the guest os knows the ioports are in use.

I think on arm the acpi situation is the same, but I think firmware detecting the location via DT should work without chicken&egg problems.

First of all I apologize for lagging severely behind this thread; in practice I can't read emails longer than 20 lines or so. This one qualifies, so I'll respond:

Yes, on qemu-system-(arm|aarch64) -M virt, detecting the fw_cfg registers via DT should continue to work without any dependencies.

Thanks
Laszlo

author
#12 lore raw

PATCH v2 2/5 fw_cfg DMA interface documentation

In reply to Marc Marí

Add fw_cfg DMA interface specification in the documentation.

Based on Gerd Hoffman's initial implementation.

---
 docs/specs/fw_cfg.txt | 68 ++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 64 insertions(+), 4 deletions(-)
docs/specs/fw_cfg.txt +64 -4
index 5bc7b96..06302f6 100644
@@ -76,6 +76,13 @@ increasing address order, similar to memcpy().
 
 Selector Register IOport: 0x510
 Data Register IOport:     0x511
+DMA Address IOport:       0x514
+
+=== ARM Register Locations ===
+
+Selector Register address: 0x09020000
+Data Register address:     0x09020008
+DMA Address address:       0x0902000c
 
 == Firmware Configuration Items ==
 
@@ -86,11 +93,12 @@ by selecting the "signature" item using key 0x0000 (FW_CFG_SIGNATURE),
 and reading four bytes from the data register. If the fw_cfg device is
 present, the four bytes read will contain the characters "QEMU".
 
-=== Revision (Key 0x0001, FW_CFG_ID) ===
+=== Revision / feature bitmap (Key 0x0001, FW_CFG_ID) ===
 
-A 32-bit little-endian unsigned int, this item is used as an interface
-revision number, and is currently set to 1 by QEMU when fw_cfg is
-initialized.
+A 32-bit little-endian unsigned int, this item is used to check for enabled
+features.
+ - Bit 0: traditional interface. Always set.
+ - Bit 1: DMA interface.
 
 === File Directory (Key 0x0019, FW_CFG_FILE_DIR) ===
 
@@ -132,6 +140,58 @@ Selector Reg.    Range Usage
 In practice, the number of allowed firmware configuration items is given
 by the value of FW_CFG_MAX_ENTRY (see fw_cfg.h).
 
+= Guest-side DMA Interface =
+
+If bit 1 of the feature bitmap is set, the DMA interface is present. This does
+not replace the existing fw_cfg interface, it is an add-on. This interface
+can be used through the 64-bit wide address register.
+
+The address register, as the selector register, is in little-endian format
+when using IOports, and in big-endian format when using MMIO. The value for
+the register is 0 at startup and after an operation. A write to the lower
+half triggers an operation. This means, that operations with 32-bit addresses
+can be triggered with just one write, whereas operations with 64-bit addresses
+can be triggered with one 64-bit write or two 32-bit writes, starting with the
+higher part.
+
+In this register, a physical RAM address to a FWCfgDmaAccess structure should
+be written. This is the format of the FWCfgDmaAccess structure:
+
+typedef struct FWCfgDmaAccess {
+    uint32_t control;
+    uint32_t length;
+    uint64_t address;
+} FWCfgDmaAccess;
+
+The fields of the structure are in big endian mode, and the field at the lowest
+address is the "control" field.
+
+The "control" field has the following bits:
+ - Bit 0: Error
+ - Bit 1: Read
+ - Bit 2: Skip
+
+When an operation is triggered, if the "control" field has bit 1 set, a read
+operation will be performed. "length" bytes for the current selector and
+offset will be copied into the address specified by the "address" field.
+
+If the control field has only bit 2 set, a skip operation will be perfomed.
+The offset for the current selector will be advanced "length" bytes.
+
+To check result, read the "control" field:
+   error bit set        ->  something went wrong.
+   all bits cleared     ->  transfer finished successfully.
+   otherwise            ->  transfer still in progress (doesn't happen
+                            today due to implementation not being async,
+                            but may in the future).
+
+Target address goes up and transfer length goes down as the transfer happens,
+so after a successful transfer the length field is zero and the address field
+points right after the memory block written.
+
+If a partial transfer happened before an error occured the address and
+length registers indicate how much data has been transfered successfully.
+
 = Host-side API =
 
 The following functions are available to the QEMU programmer for adding
Signature · 1 line
2.4.3
#13 lore raw

Re: [Qemu-devel] [PATCH v2 2/5] fw_cfg DMA interface documentation

Quoted 3 lines · Marc Marí

Add fw_cfg DMA interface specification in the documentation.

Based on Gerd Hoffman's initial implementation.

Thanks for working on this.

[...]

Quoted 41 lines

+= Guest-side DMA Interface =

docs/specs/fw_cfg.txt +40 -0
+
+If bit 1 of the feature bitmap is set, the DMA interface is present. This does
+not replace the existing fw_cfg interface, it is an add-on. This interface
+can be used through the 64-bit wide address register.
+
+The address register, as the selector register, is in little-endian format
+when using IOports, and in big-endian format when using MMIO. The value for
+the register is 0 at startup and after an operation. A write to the lower
+half triggers an operation. This means, that operations with 32-bit addresses
+can be triggered with just one write, whereas operations with 64-bit addresses
+can be triggered with one 64-bit write or two 32-bit writes, starting with the
+higher part.
+
+In this register, a physical RAM address to a FWCfgDmaAccess structure should
+be written. This is the format of the FWCfgDmaAccess structure:
+
+typedef struct FWCfgDmaAccess {
+    uint32_t control;
+    uint32_t length;
+    uint64_t address;
+} FWCfgDmaAccess;
+
+The fields of the structure are in big endian mode, and the field at the lowest
+address is the "control" field.
+
+The "control" field has the following bits:
+ - Bit 0: Error
+ - Bit 1: Read
+ - Bit 2: Skip
+
+When an operation is triggered, if the "control" field has bit 1 set, a read
+operation will be performed. "length" bytes for the current selector and
+offset will be copied into the address specified by the "address" field.
+
+If the control field has only bit 2 set, a skip operation will be perfomed.
+The offset for the current selector will be advanced "length" bytes.
+
+To check result, read the "control" field:
+   error bit set        ->  something went wrong.
+   all bits cleared     ->  transfer finished successfully.

docs/specs/fw_cfg.txt Selector Reg. Range Usage · line 193

Quoted 10 lines
docs/specs/fw_cfg.txt +10 -0
+   otherwise            ->  transfer still in progress (doesn't happen
+                            today due to implementation not being async,
+                            but may in the future).
+
+Target address goes up and transfer length goes down as the transfer happens,
+so after a successful transfer the length field is zero and the address field
+points right after the memory block written.
+
+If a partial transfer happened before an error occured the address and
+length registers indicate how much data has been transfered successfully.

If the interface will support asynchronous transfers, then it is necessary to document which field QEMU will update last. That way, the guest will know which particular field to wait on. It should probably also be documented that that particular field must have native alignment (eg, alignment of 4 for a 32bit field).

The reason this is important, is because of a subtle race condition. For example, if QEMU updates "length" and then "control", but the guest spins on "length" then it is possible for the guest to exit its spin loop and use the memory containing "control" for something else before QEMU finishes updating "control". This would result in corruption of that memory when QEMU later overwrites it.

My suggestion would be for QEMU to only ever update "control" and not modify any other parts of the FWCfgDmaAccess struct. This makes the interface and documentation simpler and it sidesteps this issue.

-Kevin

#14 lore raw

Re: [Qemu-devel] [PATCH v2 2/5] fw_cfg DMA interface documentation

In reply to Marc Marí
Quoted 14 lines · Marc Marí

Add fw_cfg DMA interface specification in the documentation.

Based on Gerd Hoffman's initial implementation.

---
 docs/specs/fw_cfg.txt | 68 ++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 64 insertions(+), 4 deletions(-)
docs/specs/fw_cfg.txt +0 -0
index 5bc7b96..06302f6 100644

docs/specs/fw_cfg.txt increasing address order, similar to memcpy(). · line 85

Quoted 10 lines
docs/specs/fw_cfg.txt +7 -0
@@ -76,6 +76,13 @@ increasing address order, similar to memcpy().
 
 Selector Register IOport: 0x510
 Data Register IOport:     0x511
+DMA Address IOport:       0x514
+
+=== ARM Register Locations ===
+
+Selector Register address: 0x09020000
+Data Register address:     0x09020008
+DMA Address address:       0x0902000c

These addresses shouldn't be documented -- the correct API is that the guest needs to find the base address of the fw_cfg device via device tree or ACPI table. You can document the layout of the registers within the device, obviously (ie +0, +4, +8).

Quoted 23 lines
docs/specs/fw_cfg.txt +5 -4
 == Firmware Configuration Items ==
 
@@ -86,11 +93,12 @@ by selecting the "signature" item using key 0x0000 (FW_CFG_SIGNATURE),
 and reading four bytes from the data register. If the fw_cfg device is
 present, the four bytes read will contain the characters "QEMU".
 
-=== Revision (Key 0x0001, FW_CFG_ID) ===
+=== Revision / feature bitmap (Key 0x0001, FW_CFG_ID) ===
 
-A 32-bit little-endian unsigned int, this item is used as an interface
-revision number, and is currently set to 1 by QEMU when fw_cfg is
-initialized.
+A 32-bit little-endian unsigned int, this item is used to check for enabled
+features.
+ - Bit 0: traditional interface. Always set.
+ - Bit 1: DMA interface.
 
 === File Directory (Key 0x0019, FW_CFG_FILE_DIR) ===
 
@@ -132,6 +140,58 @@ Selector Reg.    Range Usage
 In practice, the number of allowed firmware configuration items is given
 by the value of FW_CFG_MAX_ENTRY (see fw_cfg.h).
 

docs/specs/fw_cfg.txt Selector Reg. Range Usage · line 152

Quoted 10 lines
docs/specs/fw_cfg.txt +10 -0
+= Guest-side DMA Interface =
+
+If bit 1 of the feature bitmap is set, the DMA interface is present. This does
+not replace the existing fw_cfg interface, it is an add-on. This interface
+can be used through the 64-bit wide address register.
+
+The address register, as the selector register, is in little-endian format
+when using IOports, and in big-endian format when using MMIO. The value for
+the register is 0 at startup and after an operation. A write to the lower
+half triggers an operation. This means, that operations with 32-bit addresses

Delete this comma.

docs/specs/fw_cfg.txt Selector Reg. Range Usage · line 157

Quoted 5 lines
docs/specs/fw_cfg.txt +5 -0
+can be triggered with just one write, whereas operations with 64-bit addresses
+can be triggered with one 64-bit write or two 32-bit writes, starting with the
+higher part.
+
+In this register, a physical RAM address to a FWCfgDmaAccess structure should

"the physical address of a FWCfgDmaAccess structure in RAM"

Quoted 12 lines
docs/specs/fw_cfg.txt +12 -0
+be written. This is the format of the FWCfgDmaAccess structure:
+
+typedef struct FWCfgDmaAccess {
+    uint32_t control;
+    uint32_t length;
+    uint64_t address;
+} FWCfgDmaAccess;
+
+The fields of the structure are in big endian mode, and the field at the lowest
+address is the "control" field.
+
+The "control" field has the following bits:

docs/specs/fw_cfg.txt Selector Reg. Range Usage · line 179

Quoted 10 lines
docs/specs/fw_cfg.txt +10 -0
+ - Bit 0: Error
+ - Bit 1: Read
+ - Bit 2: Skip
+
+When an operation is triggered, if the "control" field has bit 1 set, a read
+operation will be performed. "length" bytes for the current selector and
+offset will be copied into the address specified by the "address" field.
+
+If the control field has only bit 2 set, a skip operation will be perfomed.
+The offset for the current selector will be advanced "length" bytes.

The implication here is that the operation completes before the guest write to the address register returns,

docs/specs/fw_cfg.txt Selector Reg. Range Usage · line 186

Quoted 6 lines
docs/specs/fw_cfg.txt +6 -0
+To check result, read the "control" field:
+   error bit set        ->  something went wrong.
+   all bits cleared     ->  transfer finished successfully.
+   otherwise            ->  transfer still in progress (doesn't happen
+                            today due to implementation not being async,
+                            but may in the future).

Is there much point in having an async transfer interface which requires the guest to busy-wait polling the control field?

docs/specs/fw_cfg.txt Selector Reg. Range Usage · line 192

Quoted 6 lines
docs/specs/fw_cfg.txt +6 -0
+
+Target address goes up and transfer length goes down as the transfer happens,
+so after a successful transfer the length field is zero and the address field
+points right after the memory block written.
+
+If a partial transfer happened before an error occured the address and

"occurred".

docs/specs/fw_cfg.txt Selector Reg. Range Usage · line 193

Quoted 1 line
docs/specs/fw_cfg.txt +1 -0
+length registers indicate how much data has been transfered successfully.

"transferred".

docs/specs/fw_cfg.txt Selector Reg. Range Usage · line 197

Quoted 4 lines
docs/specs/fw_cfg.txt +1 -0
+
 = Host-side API =
 
 The following functions are available to the QEMU programmer for adding

thanks
-- PMM

#15 lore raw

Re: [Qemu-devel] [PATCH v2 2/5] fw_cfg DMA interface documentation

Quoted 11 lines · Peter Maydell

docs/specs/fw_cfg.txt increasing address order, similar to memcpy(). · line 85

Quoted 5 lines · Marc Marí

+=== ARM Register Locations ===

docs/specs/fw_cfg.txt +4 -0
+
+Selector Register address: 0x09020000
+Data Register address:     0x09020008
+DMA Address address:       0x0902000c

These addresses shouldn't be documented -- the correct API is that the guest needs to find the base address of the fw_cfg device via device tree or ACPI table. You can document the layout of the registers within the device, obviously (ie +0, +4, +8).

...and this is not what your patchset implements anyway. This should be:

offset 0: data register (8 bytes wide)
offset 8: selector register (4 bytes wide)
offset 12: reserved
offset 16: DMA address register (8 bytes wide)

thanks
-- PMM

author
#16 lore raw

PATCH v2 3/5 Implement fw_cfg DMA interface

In reply to Marc Marí

Based on the specifications on docs/specs/fw_cfg.txt

This interface is an addon. The old interface can still be used as usual.

Based on Gerd Hoffman's initial implementation.

---
 hw/arm/virt.c             |   2 +-
 hw/nvram/fw_cfg.c         | 261 +++++++++++++++++++++++++++++++++++++++++++---
 include/hw/nvram/fw_cfg.h |  15 ++-
 3 files changed, 260 insertions(+), 18 deletions(-)
hw/arm/virt.c +1 -1
index d5a8417..b88c104 100644
@@ -620,7 +620,7 @@ static void create_fw_cfg(const VirtBoardInfo *vbi)
     hwaddr size = vbi->memmap[VIRT_FW_CFG].size;
     char *nodename;
 
-    fw_cfg_init_mem_wide(base + 8, base, 8);
+    fw_cfg_init_mem_wide(base + 8, base, 8, 0, NULL);
 
     nodename = g_strdup_printf("/fw-cfg@%" PRIx64, base);
     qemu_fdt_add_subnode(vbi->fdt, nodename);
hw/nvram/fw_cfg.c +246 -15

384 lines not on this page · raw message

include/hw/nvram/fw_cfg.h +13 -2
index e60d3ca..d0cbc88 100644
@@ -61,6 +61,15 @@ typedef struct FWCfgFiles {
     FWCfgFile f[];
 } FWCfgFiles;
 
+/* Control as first field allows for different structures selected by this
+ * field, which might be useful in the future
+ */
+typedef struct FWCfgDmaAccess {
+    uint32_t control;
+    uint32_t length;
+    uint64_t address;
+} QEMU_PACKED FWCfgDmaAccess;
+
 typedef void (*FWCfgCallback)(void *opaque, uint8_t *data);
 typedef void (*FWCfgReadCallback)(void *opaque, uint32_t offset);
 
@@ -77,10 +86,12 @@ void fw_cfg_add_file_callback(FWCfgState *s, const char *filename,
                               void *data, size_t len);
 void *fw_cfg_modify_file(FWCfgState *s, const char *filename, void *data,
                          size_t len);
+FWCfgState *fw_cfg_init_io_dma(uint32_t iobase, AddressSpace *dma_as);
 FWCfgState *fw_cfg_init_io(uint32_t iobase);
 FWCfgState *fw_cfg_init_mem(hwaddr ctl_addr, hwaddr data_addr);
-FWCfgState *fw_cfg_init_mem_wide(hwaddr ctl_addr, hwaddr data_addr,
-                                 uint32_t data_width);
+FWCfgState *fw_cfg_init_mem_wide(hwaddr ctl_addr,
+                                 hwaddr data_addr, uint32_t data_width,
+                                 hwaddr dma_addr, AddressSpace *dma_as);
 
 FWCfgState *fw_cfg_find(void);
 
Signature · 1 line
2.4.3
#17 lore raw

Re: [Qemu-devel] [PATCH v2 3/5] Implement fw_cfg DMA interface

Quoted 103 lines · Marc Marí

Based on the specifications on docs/specs/fw_cfg.txt

This interface is an addon. The old interface can still be used as usual.

Based on Gerd Hoffman's initial implementation.

---
 hw/arm/virt.c             |   2 +-
 hw/nvram/fw_cfg.c         | 261 +++++++++++++++++++++++++++++++++++++++++++---
 include/hw/nvram/fw_cfg.h |  15 ++-
 3 files changed, 260 insertions(+), 18 deletions(-)
hw/arm/virt.c +1 -1
index d5a8417..b88c104 100644
@@ -620,7 +620,7 @@ static void create_fw_cfg(const VirtBoardInfo *vbi)
     hwaddr size = vbi->memmap[VIRT_FW_CFG].size;
     char *nodename;
 
-    fw_cfg_init_mem_wide(base + 8, base, 8);
+    fw_cfg_init_mem_wide(base + 8, base, 8, 0, NULL);
 
     nodename = g_strdup_printf("/fw-cfg@%" PRIx64, base);
     qemu_fdt_add_subnode(vbi->fdt, nodename);
hw/nvram/fw_cfg.c +32 -2
index 88481b7..7e5ba96 100644
@@ -23,6 +23,7 @@
  */
 #include "hw/hw.h"
 #include "sysemu/sysemu.h"
+#include "sysemu/dma.h"
 #include "hw/isa/isa.h"
 #include "hw/nvram/fw_cfg.h"
 #include "hw/sysbus.h"
@@ -30,7 +31,8 @@
 #include "qemu/error-report.h"
 #include "qemu/config-file.h"
 
-#define FW_CFG_SIZE 2
+#define FW_CFG_IO_SIZE 12 /* Address aligned to 4 bytes */
+#define FW_CFG_CTL_SIZE 2
 #define FW_CFG_NAME "fw_cfg"
 #define FW_CFG_PATH "/machine/" FW_CFG_NAME
 
@@ -42,6 +44,15 @@
 #define FW_CFG_IO(obj)  OBJECT_CHECK(FWCfgIoState,  (obj), TYPE_FW_CFG_IO)
 #define FW_CFG_MEM(obj) OBJECT_CHECK(FWCfgMemState, (obj), TYPE_FW_CFG_MEM)
 
+/* FW_CFG_VERSION bits */
+#define FW_CFG_VERSION      0x01
+#define FW_CFG_VERSION_DMA  0x02
+
+/* FW_CFG_DMA_CONTROL bits */
+#define FW_CFG_DMA_CTL_ERROR   0x01
+#define FW_CFG_DMA_CTL_READ    0x02
+#define FW_CFG_DMA_CTL_SKIP    0x04
+
 typedef struct FWCfgEntry {
     uint32_t len;
     uint8_t *data;
@@ -59,6 +70,10 @@ struct FWCfgState {
     uint16_t cur_entry;
     uint32_t cur_offset;
     Notifier machine_ready;
+
+    bool dma_enabled;
+    AddressSpace *dma_as;
+    dma_addr_t dma_addr;
 };
 
 struct FWCfgIoState {
@@ -75,7 +90,7 @@ struct FWCfgMemState {
     FWCfgState parent_obj;
     /*< public >*/
 
-    MemoryRegion ctl_iomem, data_iomem;
+    MemoryRegion ctl_iomem, data_iomem, dma_iomem;
     uint32_t data_width;
     MemoryRegionOps wide_data_ops;
 };
@@ -294,6 +309,142 @@ static void fw_cfg_data_mem_write(void *opaque, hwaddr addr,
     } while (i);
 }
 
+static void fw_cfg_dma_transfer(FWCfgState *s)
+{
+    dma_addr_t len;
+    uint8_t *ptr;
+    void *addr;
+    FWCfgDmaAccess dma;
+    int arch = !!(s->cur_entry & FW_CFG_ARCH_LOCAL);
+    FWCfgEntry *e = &s->entries[arch][s->cur_entry & FW_CFG_ENTRY_MASK];
+
+    len = sizeof(dma);
+    addr = dma_memory_map(s->dma_as, s->dma_addr, &len,
+                                DMA_DIRECTION_FROM_DEVICE);
+
+    s->dma_addr = 0;
+

hw/nvram/fw_cfg.c static void fw_cfg_data_mem_write(void *opaque, hwaddr addr, · line 336

Quoted 10 lines
hw/nvram/fw_cfg.c +10 -0
+    if (!addr || !len) {
+        return;
+    }
+
+    dma.address = be64_to_cpu(*(uint64_t *)(addr +
+                    offsetof(FWCfgDmaAccess, address)));
+    dma.length = be32_to_cpu(*(uint32_t *)(addr +
+                    offsetof(FWCfgDmaAccess, length)));
+    dma.control = be32_to_cpu(*(uint32_t *)(addr +
+                    offsetof(FWCfgDmaAccess, control)));

I am not that familiar with QEMU, but shouldn't that be DMA_DIRECTION_TO_DEVICE?

It looks like other drivers use dma_memory_read() which would simplify this code:

    dma_memory_read(s->dma_as, s->dma_addr, &dma, sizeof(dma));
    dma.address = be64_to_cpu(dma.address);
    dma.length = be32_to_cpu(dma.length);
    dma.control = be32_to_cpu(dma.control);
Quoted 63 lines
hw/nvram/fw_cfg.c +63 -0
+    if (dma.control & FW_CFG_DMA_CTL_ERROR) {
+        goto out;
+    }
+
+    if (!(dma.control & (FW_CFG_DMA_CTL_READ | FW_CFG_DMA_CTL_SKIP))) {
+        goto out;
+    }
+
+    while (dma.length > 0) {
+        if (s->cur_entry == FW_CFG_INVALID || !e->data ||
+                                s->cur_offset >= e->len) {
+            len = dma.length;
+
+            /* If the access is not a read access, it will be a skip access,
+             * tested before.
+             */
+            if (dma.control & FW_CFG_DMA_CTL_READ) {
+                ptr = dma_memory_map(s->dma_as, dma.address, &len,
+                                     DMA_DIRECTION_FROM_DEVICE);
+                if (!ptr || !len) {
+                    dma.control |= FW_CFG_DMA_CTL_ERROR;
+                    goto out;
+                }
+
+                memset(ptr, 0, len);
+
+                dma_memory_unmap(s->dma_as, ptr, len,
+                                 DMA_DIRECTION_FROM_DEVICE, len);
+            }
+
+        } else {
+            if (dma.length <= e->len) {
+                len = dma.length;
+            } else {
+                len = e->len;
+            }
+
+            if (e->read_callback) {
+                e->read_callback(e->callback_opaque, s->cur_offset);
+            }
+
+            /* If the access is not a read access, it will be a skip access,
+             * tested before.
+             */
+            if (dma.control & FW_CFG_DMA_CTL_READ) {
+                ptr = dma_memory_map(s->dma_as, dma.address, &len,
+                                     DMA_DIRECTION_FROM_DEVICE);
+                if (!ptr || !len) {
+                    dma.control |= FW_CFG_DMA_CTL_ERROR;
+                    goto out;
+                }
+
+                memcpy(ptr, &e->data[s->cur_offset], len);
+
+                dma_memory_unmap(s->dma_as, ptr, len,
+                                 DMA_DIRECTION_FROM_DEVICE, len);
+            }
+
+            s->cur_offset += len;
+
+        }
+
+        dma.address += len;

hw/nvram/fw_cfg.c static void fw_cfg_data_mem_write(void *opaque, hwaddr addr, · line 410

Quoted 10 lines
hw/nvram/fw_cfg.c +10 -0
+        dma.length  -= len;
+        dma.control = 0;
+
+        *(uint64_t *)(addr + offsetof(FWCfgDmaAccess, address)) =
+                                                cpu_to_be64(dma.address);
+        *(uint32_t *)(addr + offsetof(FWCfgDmaAccess, length)) =
+                                                cpu_to_be32(dma.length);
+        *(uint32_t *)(addr + offsetof(FWCfgDmaAccess, control)) =
+                                                cpu_to_be32(dma.control);
+    }

I don't think it makes sense for this update to be performed within the loop.

As I mentioned in another email, I think just updating control would be sufficient. Looks like include/sysemu/dma.h defines a stl_be_dma() macro for performing a single 32bit dma write.

Thanks,
-Kevin

#18 lore raw

Re: [Qemu-devel] [PATCH v2 3/5] Implement fw_cfg DMA interface

In reply to Marc Marí
Quoted 27 lines · Marc Marí

Based on the specifications on docs/specs/fw_cfg.txt

This interface is an addon. The old interface can still be used as usual.

Based on Gerd Hoffman's initial implementation.

hw/nvram/fw_cfg.c +15 -0
@@ -294,6 +309,142 @@ static void fw_cfg_data_mem_write(void *opaque, hwaddr addr,
     } while (i);
 }
 
+static void fw_cfg_dma_transfer(FWCfgState *s)
+{
+    dma_addr_t len;
+    uint8_t *ptr;
+    void *addr;
+    FWCfgDmaAccess dma;
+    int arch = !!(s->cur_entry & FW_CFG_ARCH_LOCAL);
+    FWCfgEntry *e = &s->entries[arch][s->cur_entry & FW_CFG_ENTRY_MASK];
+
+    len = sizeof(dma);
+    addr = dma_memory_map(s->dma_as, s->dma_addr, &len,
+                                DMA_DIRECTION_FROM_DEVICE);
+
+    s->dma_addr = 0;
+

hw/nvram/fw_cfg.c static void fw_cfg_data_mem_write(void *opaque, hwaddr addr, · line 336

Quoted 10 lines
hw/nvram/fw_cfg.c +10 -0
+    if (!addr || !len) {
+        return;
+    }
+
+    dma.address = be64_to_cpu(*(uint64_t *)(addr +
+                    offsetof(FWCfgDmaAccess, address)));
+    dma.length = be32_to_cpu(*(uint32_t *)(addr +
+                    offsetof(FWCfgDmaAccess, length)));
+    dma.control = be32_to_cpu(*(uint32_t *)(addr +
+                    offsetof(FWCfgDmaAccess, control)));

There are only three fields in this struct, so rather than using dma_memory_map/unmap and manual byteswapping, how about:

dma.control = ldl_be_dma(s->dma_as, s->dma_addr); dma.length = ldl_be_dma(s->dma_as, s->dma_addr + 4); dma.address = ldq_be_dma(s->dma_as, s->dma_addr + 8);

Kevin's suggestion to use dma_memory_read() would be OK as well, if you really want to check for the return value from the memory transaction. But you probably don't care because if the guest has pointed the dma address register into nowhere then it deserves whatever it gets as long as we don't fall over; and we need to not fall over whatever the values of control/length/address are. In any case, please don't use dma_memory_map/unmap.

There are st*_ versions for writing back updated values.

Quoted 19 lines
hw/nvram/fw_cfg.c +19 -0
+
+    if (dma.control & FW_CFG_DMA_CTL_ERROR) {
+        goto out;
+    }
+
+    if (!(dma.control & (FW_CFG_DMA_CTL_READ | FW_CFG_DMA_CTL_SKIP))) {
+        goto out;
+    }
+
+    while (dma.length > 0) {
+        if (s->cur_entry == FW_CFG_INVALID || !e->data ||
+                                s->cur_offset >= e->len) {
+            len = dma.length;
+
+            /* If the access is not a read access, it will be a skip access,
+             * tested before.
+             */
+            if (dma.control & FW_CFG_DMA_CTL_READ) {
+                ptr = dma_memory_map(s->dma_as, dma.address, &len,

hw/nvram/fw_cfg.c static void fw_cfg_data_mem_write(void *opaque, hwaddr addr, · line 365

Quoted 10 lines
hw/nvram/fw_cfg.c +10 -0
+                                     DMA_DIRECTION_FROM_DEVICE);
+                if (!ptr || !len) {
+                    dma.control |= FW_CFG_DMA_CTL_ERROR;
+                    goto out;
+                }
+
+                memset(ptr, 0, len);
+
+                dma_memory_unmap(s->dma_as, ptr, len,
+                                 DMA_DIRECTION_FROM_DEVICE, len);

dma_memory_write() would be better than map-zero-unmap I think, though you would need to have a handy buffer of zeroes to pass to that function, so maybe not.

Quoted 18 lines
hw/nvram/fw_cfg.c +18 -0
+            }
+
+        } else {
+            if (dma.length <= e->len) {
+                len = dma.length;
+            } else {
+                len = e->len;
+            }
+
+            if (e->read_callback) {
+                e->read_callback(e->callback_opaque, s->cur_offset);
+            }
+
+            /* If the access is not a read access, it will be a skip access,
+             * tested before.
+             */
+            if (dma.control & FW_CFG_DMA_CTL_READ) {
+                ptr = dma_memory_map(s->dma_as, dma.address, &len,

hw/nvram/fw_cfg.c static void fw_cfg_data_mem_write(void *opaque, hwaddr addr, · line 393

Quoted 10 lines
hw/nvram/fw_cfg.c +10 -0
+                                     DMA_DIRECTION_FROM_DEVICE);
+                if (!ptr || !len) {
+                    dma.control |= FW_CFG_DMA_CTL_ERROR;
+                    goto out;
+                }
+
+                memcpy(ptr, &e->data[s->cur_offset], len);
+
+                dma_memory_unmap(s->dma_as, ptr, len,
+                                 DMA_DIRECTION_FROM_DEVICE, len);

...and dma_memory_write() is definitely better than map-memcpy-unmap.

Quoted 26 lines
hw/nvram/fw_cfg.c +26 -0
+            }
+
+            s->cur_offset += len;
+
+        }
+
+        dma.address += len;
+        dma.length  -= len;
+        dma.control = 0;
+
+        *(uint64_t *)(addr + offsetof(FWCfgDmaAccess, address)) =
+                                                cpu_to_be64(dma.address);
+        *(uint32_t *)(addr + offsetof(FWCfgDmaAccess, length)) =
+                                                cpu_to_be32(dma.length);
+        *(uint32_t *)(addr + offsetof(FWCfgDmaAccess, control)) =
+                                                cpu_to_be32(dma.control);
+    }
+
+    trace_fw_cfg_read(s, 0);
+
+out:
+    dma_memory_unmap(s->dma_as, addr, sizeof(dma),
+                                DMA_DIRECTION_FROM_DEVICE, sizeof(dma));
+    return;
+
+}

hw/nvram/fw_cfg.c static void fw_cfg_data_mem_write(void *opaque, hwaddr addr, · line 429

Quoted 10 lines
hw/nvram/fw_cfg.c +10 -0
+
+static void fw_cfg_dma_mem_write(void *opaque, hwaddr addr,
+                                 uint64_t value, unsigned size)
+{
+    FWCfgState *s = opaque;
+
+    if (size == 4) {
+        if (addr == 0) {
+            /* FWCfgDmaAccess high address */
+            s->dma_addr = (0xFFFFFFFF & value) << 32;

The mask here is unnecessary, since you're shifting by 32 bits.

Quoted 48 lines
hw/nvram/fw_cfg.c +33 -3
+        } else if (addr == 4) {
+            /* FWCfgDmaAccess low address */
+            s->dma_addr |= value;
+            fw_cfg_dma_transfer(s);
+        }
+    } else if (size == 8 && addr == 0) {
+        s->dma_addr = value;
+        fw_cfg_dma_transfer(s);
+    }
+}
+
+static bool fw_cfg_dma_mem_valid(void *opaque, hwaddr addr,
+                                  unsigned size, bool is_write)
+{
+    return is_write && ((size == 4 && (addr == 0 || addr == 4)) ||
+                        (size == 8 && addr == 0));
+}
+
 static bool fw_cfg_data_mem_valid(void *opaque, hwaddr addr,
                                   unsigned size, bool is_write)
 {
@@ -321,20 +472,35 @@ static uint64_t fw_cfg_comb_read(void *opaque, hwaddr addr,
 static void fw_cfg_comb_write(void *opaque, hwaddr addr,
                               uint64_t value, unsigned size)
 {
-    switch (size) {
+    FWCfgState *s;
+
+    switch (addr) {
+    case 0:
+        fw_cfg_select(opaque, (uint16_t)value);
+        break;
     case 1:
         fw_cfg_write(opaque, (uint8_t)value);
         break;
-    case 2:
-        fw_cfg_select(opaque, (uint16_t)value);
+    case 4:
+        /* FWCfgDmaAccess low address */
+        s = opaque;
+        s->dma_addr |= value;
+        fw_cfg_dma_transfer(s);
         break;
+    case 8:
+        /* FWCfgDmaAccess high address */
+        s = opaque;
+        s->dma_addr = (0xFFFFFFFF & value) << 32;
     }

hw/nvram/fw_cfg.c static uint64_t fw_cfg_comb_read(void *opaque, hwaddr addr, · line 503

Quoted 10 lines
hw/nvram/fw_cfg.c +4 -1
 }
 
 static bool fw_cfg_comb_valid(void *opaque, hwaddr addr,
                                   unsigned size, bool is_write)
 {
-    return (size == 1) || (is_write && size == 2);
+    FWCfgState *s = opaque;
+
+    return (size == 1) || (is_write && size == 2) ||
+            (s->dma_enabled && is_write && addr >= 4);

This doesn't look right, since it will allow 1 and 2 byte writes to the dma_address port, which doesn't seem like a great idea.

Maybe it would be easier to make the dma port its own memory region (and then init and sysbus_add_io() it in fw_cfg_io_realize()) ? Then you wouldn't need to mess with the already confusing logic for handling combined config/data ports (which have to be handled together because of the weird "1 byte accesses are data and 2 byte accesses are config" semantics).

thanks
-- PMM

author
#19 lore raw

PATCH v2 4/5 Enable fw_cfg DMA interface for ARM

In reply to Marc Marí

Enable the fw_cfg DMA interface for the ARM virt machine.

Based on Gerd Hoffman's initial implementation.

---
 hw/arm/virt.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
hw/arm/virt.c +9 -4
index b88c104..54d5f54 100644
@@ -111,7 +111,7 @@ static const MemMapEntry a15memmap[] = {
     [VIRT_GIC_V2M] =            { 0x08020000, 0x00001000 },
     [VIRT_UART] =               { 0x09000000, 0x00001000 },
     [VIRT_RTC] =                { 0x09010000, 0x00001000 },
-    [VIRT_FW_CFG] =             { 0x09020000, 0x0000000a },
+    [VIRT_FW_CFG] =             { 0x09020000, 0x00000014 },
     [VIRT_MMIO] =               { 0x0a000000, 0x00000200 },
     /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */
     [VIRT_PLATFORM_BUS] =       { 0x0c000000, 0x02000000 },
@@ -614,13 +614,13 @@ static void create_flash(const VirtBoardInfo *vbi)
     g_free(nodename);
 }
 
-static void create_fw_cfg(const VirtBoardInfo *vbi)
+static void create_fw_cfg(AddressSpace *as, const VirtBoardInfo *vbi)
 {
     hwaddr base = vbi->memmap[VIRT_FW_CFG].base;
     hwaddr size = vbi->memmap[VIRT_FW_CFG].size;
     char *nodename;
 
-    fw_cfg_init_mem_wide(base + 8, base, 8, 0, NULL);
+    fw_cfg_init_mem_wide(base + 8, base, 8, base + 16, as);
 
     nodename = g_strdup_printf("/fw-cfg@%" PRIx64, base);
     qemu_fdt_add_subnode(vbi->fdt, nodename);
@@ -808,6 +808,7 @@ static void machvirt_init(MachineState *machine)
     VirtGuestInfoState *guest_info_state = g_malloc0(sizeof *guest_info_state);
     VirtGuestInfo *guest_info = &guest_info_state->info;
     char **cpustr;
+    AddressSpace *as = NULL;
 
     if (!cpu_model) {
         cpu_model = "cortex-a15";
@@ -845,6 +846,10 @@ static void machvirt_init(MachineState *machine)
         }
         cpuobj = object_new(object_class_get_name(oc));
 
+        if (!as) {
+            as = CPU(cpuobj)->as;
+        }
+
         /* Handle any CPU options specified by the user */
         cc->parse_features(CPU(cpuobj), cpuopts, &err);
         g_free(cpuopts);
@@ -897,7 +902,7 @@ static void machvirt_init(MachineState *machine)
      */
     create_virtio_devices(vbi, pic);
 
-    create_fw_cfg(vbi);
+    create_fw_cfg(as, vbi);
     rom_set_fw(fw_cfg_find());
 
     guest_info->smp_cpus = smp_cpus;
Signature · 1 line
2.4.3
#20 lore raw

Re: [Qemu-devel] [PATCH v2 4/5] Enable fw_cfg DMA interface for ARM

Quoted 23 lines · Marc Marí

Enable the fw_cfg DMA interface for the ARM virt machine.

Based on Gerd Hoffman's initial implementation.

---
 hw/arm/virt.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
hw/arm/virt.c +1 -1
index b88c104..54d5f54 100644
@@ -111,7 +111,7 @@ static const MemMapEntry a15memmap[] = {
     [VIRT_GIC_V2M] =            { 0x08020000, 0x00001000 },
     [VIRT_UART] =               { 0x09000000, 0x00001000 },
     [VIRT_RTC] =                { 0x09010000, 0x00001000 },
-    [VIRT_FW_CFG] =             { 0x09020000, 0x0000000a },
+    [VIRT_FW_CFG] =             { 0x09020000, 0x00000014 },
     [VIRT_MMIO] =               { 0x0a000000, 0x00000200 },
     /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */
     [VIRT_PLATFORM_BUS] =       { 0x0c000000, 0x02000000 },

hw/arm/virt.c static void create_flash(const VirtBoardInfo *vbi) · line 617

Quoted 5 lines
hw/arm/virt.c +1 -1
@@ -614,13 +614,13 @@ static void create_flash(const VirtBoardInfo *vbi)
     g_free(nodename);
 }
 
-static void create_fw_cfg(const VirtBoardInfo *vbi)
+static void create_fw_cfg(AddressSpace *as, const VirtBoardInfo *vbi)

Please keep vbi as the first argument; this matches the other functions in this file.

Calling the argument dma_as would probably be a little more informative.

Quoted 19 lines
hw/arm/virt.c +2 -1
 {
     hwaddr base = vbi->memmap[VIRT_FW_CFG].base;
     hwaddr size = vbi->memmap[VIRT_FW_CFG].size;
     char *nodename;
 
-    fw_cfg_init_mem_wide(base + 8, base, 8, 0, NULL);
+    fw_cfg_init_mem_wide(base + 8, base, 8, base + 16, as);
 
     nodename = g_strdup_printf("/fw-cfg@%" PRIx64, base);
     qemu_fdt_add_subnode(vbi->fdt, nodename);
@@ -808,6 +808,7 @@ static void machvirt_init(MachineState *machine)
     VirtGuestInfoState *guest_info_state = g_malloc0(sizeof *guest_info_state);
     VirtGuestInfo *guest_info = &guest_info_state->info;
     char **cpustr;
+    AddressSpace *as = NULL;
 
     if (!cpu_model) {
         cpu_model = "cortex-a15";

hw/arm/virt.c static void machvirt_init(MachineState *machine) · line 851

Quoted 6 lines
hw/arm/virt.c +3 -0
@@ -845,6 +846,10 @@ static void machvirt_init(MachineState *machine)
         }
         cpuobj = object_new(object_class_get_name(oc));
 
+        if (!as) {
+            as = CPU(cpuobj)->as;
+        }

This seems like a weird thing to set the fw_cfg DMA address space to. (Either the fw_cfg device shouldn't care which CPU it is being accessing by and shouldn't use any particular CPU's address space, or it needs to really care about the CPU that's doing any particular write to it and use that exact CPU's address space, selected at runtime. The former is the most likely and matches what actual DMA hardware devices will do.)

Why not just use address_space_memory ?

Quoted 14 lines
hw/arm/virt.c +2 -2
+
         /* Handle any CPU options specified by the user */
         cc->parse_features(CPU(cpuobj), cpuopts, &err);
         g_free(cpuopts);
@@ -897,7 +902,7 @@ static void machvirt_init(MachineState *machine)
      */
     create_virtio_devices(vbi, pic);
 
-    create_fw_cfg(vbi);
+    create_fw_cfg(as, vbi);
     rom_set_fw(fw_cfg_find());
 
     guest_info->smp_cpus = smp_cpus;
--
Quoted 1 line

2.4.3

thanks
-- PMM

author
#21 lore raw

PATCH v2 5/5 Enable fw_cfg DMA interface for x86

In reply to Marc Marí

Enable the fw_cfg DMA interface for all the x86 platforms.

Based on Gerd Hoffman's initial implementation.

---
 hw/i386/pc.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
hw/i386/pc.c +8 -3
index 9f2924e..c6dc84f 100644
@@ -753,14 +753,15 @@ static void pc_build_smbios(FWCfgState *fw_cfg)
     }
 }
 
-static FWCfgState *bochs_bios_init(void)
+static FWCfgState *bochs_bios_init(AddressSpace *as)
 {
     FWCfgState *fw_cfg;
     uint64_t *numa_fw_cfg;
     int i, j;
     unsigned int apic_id_limit = pc_apic_id_limit(max_cpus);
 
-    fw_cfg = fw_cfg_init_io(BIOS_CFG_IOPORT);
+    fw_cfg = fw_cfg_init_io_dma(BIOS_CFG_IOPORT, as);
+
     /* FW_CFG_MAX_CPUS is a bit confusing/problematic on x86:
      *
      * SeaBIOS needs FW_CFG_MAX_CPUS for CPU hotplug, but the CPU hotplug
@@ -1316,6 +1317,7 @@ FWCfgState *pc_memory_init(PCMachineState *pcms,
     MemoryRegion *ram_below_4g, *ram_above_4g;
     FWCfgState *fw_cfg;
     MachineState *machine = MACHINE(pcms);
+    AddressSpace *as;
 
     assert(machine->ram_size == pcms->below_4g_mem_size +
                                 pcms->above_4g_mem_size);
@@ -1407,7 +1409,10 @@ FWCfgState *pc_memory_init(PCMachineState *pcms,
                                         option_rom_mr,
                                         1);
 
-    fw_cfg = bochs_bios_init();
+    as = g_malloc(sizeof(*as));
+    address_space_init(as, ram_below_4g, "pc.as");
+    fw_cfg = bochs_bios_init(as);
+
     rom_set_fw(fw_cfg);
 
     if (guest_info->has_reserved_memory && pcms->hotplug_memory.base) {
Signature · 1 line
2.4.3
author
#22 lore raw

PATCH v2 1/1 QEMU fw_cfg DMA interface documentation

In reply to Marc Marí

Add fw_cfg DMA interface specfication in the fw_cfg documentation.

---
 Documentation/devicetree/bindings/arm/fw-cfg.txt | 51 +++++++++++++++++++++++-
 1 file changed, 50 insertions(+), 1 deletion(-)
Documentation/devicetree/bindings/arm/fw-cfg.txt +50 -1
index 953fb64..766ddbe 100644
@@ -45,6 +45,53 @@ blob to be read from the data register has size 4, and it is to be interpreted
 as a uint32_t value in little endian byte order. The current value
 (corresponding to the above outer protocol) is zero.
 
+If bit 1 of the feature bitmap is set, the DMA interface is present. This
+can be used through the 64-bit wide address register.
+
+The address register is in big-endian format. The value for the register is 0
+at startup and after an operation. A write to the lower half triggers an
+operation. This means, that operations with 32-bit addresses can be triggered
+with just one write, whereas operations with 64-bit addresses can be triggered
+with one 64-bit write or two 32-bit writes, starting with the higher part.
+
+In this register, a physical RAM address to a FWCfgDmaAccess structure should
+be written. This is the format of the FWCfgDmaAccess structure:
+
+typedef struct FWCfgDmaAccess {
+    uint32_t control;
+    uint32_t length;
+    uint64_t address;
+} FWCfgDmaAccess;
+
+The fields of the structure are in big endian mode, and the field at the lowest
+address is the "control" field.
+
+The "control" field has the following bits:
+ - Bit 0: Error
+ - Bit 1: Read
+ - Bit 2: Skip
+
+When an operation is triggered, if the "control" field has bit 1 set, a read
+operation will be performed. "length" bytes for the current selector and
+offset will be copied into the address specified by the "address" field.
+
+If the control field has only bit 2 set, a skip operation will be perfomed.
+The offset for the current selector will be advanced "length" bytes.
+
+To check result, read the "control" field:
+   error bit set        ->  something went wrong.
+   all bits cleared     ->  transfer finished successfully.
+   otherwise            ->  transfer still in progress (doesn't happen
+                            today due to implementation not being async,
+                            but may in the future).
+
+Target address goes up and transfer length goes down as the transfer happens,
+so after a successful transfer the length field is zero and the address field
+points right after the memory block written.
+
+If a partial transfer happened before an error occured the address and
+length registers indicate how much data has been transfered successfully.
+
 The guest kernel is not expected to use these registers (although it is
 certainly allowed to); the device tree bindings are documented here because
 this is where device tree bindings reside in general.
@@ -56,6 +103,8 @@ Required properties:
 - reg: the MMIO region used by the device.
   * Bytes 0x0 to 0x7 cover the data register.
   * Bytes 0x8 to 0x9 cover the selector register.
+  * With DMA interface enabled: Bytes 0xc to 0x13 cover the DMA address
+    register.
   * Further registers may be appended to the region in case of future interface
     revisions / feature bits.
 
@@ -66,7 +115,7 @@ Example:
 	#address-cells = <0x2>;
 
 	fw-cfg@9020000 {
+		reg = <0x0 0x9020000 0x0 0x14>;
 		compatible = "qemu,fw-cfg-mmio";
-		reg = <0x0 0x9020000 0x0 0xa>;
 	};
 };
Signature · 6 lines
2.4.3

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reviewed-by on v2 1/1
#23 lore raw

Re: [PATCH v2] QEMU fw_cfg DMA interface documentation

Quoted 22 lines · Marc Marí

Add fw_cfg DMA interface specfication in the fw_cfg documentation.

---
 Documentation/devicetree/bindings/arm/fw-cfg.txt | 51 +++++++++++++++++++++++-
 1 file changed, 50 insertions(+), 1 deletion(-)
Documentation/devicetree/bindings/arm/fw-cfg.txt +7 -0
index 953fb64..766ddbe 100644
@@ -45,6 +45,53 @@ blob to be read from the data register has size 4, and it is to be interpreted
 as a uint32_t value in little endian byte order. The current value
 (corresponding to the above outer protocol) is zero.
 
+If bit 1 of the feature bitmap is set, the DMA interface is present. This
+can be used through the 64-bit wide address register.
+
+The address register is in big-endian format. The value for the register is 0
+at startup and after an operation. A write to the lower half triggers an
+operation. This means, that operations with 32-bit addresses can be triggered
+with just one write, whereas operations with 64-bit addresses can be triggered

Documentation/devicetree/bindings/arm/fw-cfg.txt blob to be read from the data register has size 4, and it is to be interpreted · line 64

Quoted 10 lines
Documentation/devicetree/bindings/arm/fw-cfg.txt +10 -0
+with one 64-bit write or two 32-bit writes, starting with the higher part.
+
+In this register, a physical RAM address to a FWCfgDmaAccess structure should
+be written. This is the format of the FWCfgDmaAccess structure:
+
+typedef struct FWCfgDmaAccess {
+    uint32_t control;
+    uint32_t length;
+    uint64_t address;
+} FWCfgDmaAccess;

I think including the selector field would be nice to avoid extra register accesses, but I'm not that familiar with fw_cfg so maybe there's a reason not to include that field.

Documentation/devicetree/bindings/arm/fw-cfg.txt blob to be read from the data register has size 4, and it is to be interpreted · line 76

Quoted 11 lines
Documentation/devicetree/bindings/arm/fw-cfg.txt +11 -0
+The fields of the structure are in big endian mode, and the field at the lowest
+address is the "control" field.
+
+The "control" field has the following bits:
+ - Bit 0: Error
+ - Bit 1: Read
+ - Bit 2: Skip
+
+When an operation is triggered, if the "control" field has bit 1 set, a read
+operation will be performed. "length" bytes for the current selector and
+offset will be copied into the address specified by the "address" field.

Minor clarification:
s/address/physical RAM address/

author
#24 lore raw

Re: [PATCH v2] QEMU fw_cfg DMA interface documentation

Quoted 31 lines · On Wed, 2 Sep 2015 09:20:12 +0100 Stefan Hajnoczi…
Quoted 30 lines · Marc Marí

Add fw_cfg DMA interface specfication in the fw_cfg documentation.

---
 Documentation/devicetree/bindings/arm/fw-cfg.txt | 51

+++++++++++++++++++++++- 1 file changed, 50 insertions(+), 1 deletion(-)

(quoted diff) +0 -0

b/Documentation/devicetree/bindings/arm/fw-cfg.txt index 953fb64..766ddbe 100644 --- a/Documentation/devicetree/bindings/arm/fw-cfg.txt +++ b/Documentation/devicetree/bindings/arm/fw-cfg.txt @@ -45,6 +45,53 @@ blob to be read from the data register has size 4, and it is to be interpreted as a uint32_t value in little endian byte order. The current value (corresponding to the above outer protocol) is zero.

+If bit 1 of the feature bitmap is set, the DMA interface is present. This +can be used through the 64-bit wide address register.

(quoted diff) +2 -0
+
+The address register is in big-endian format. The value for the

register is 0 +at startup and after an operation. A write to the lower half triggers an +operation. This means, that operations with 32-bit addresses can be triggered +with just one write, whereas operations with 64-bit addresses can be triggered +with one 64-bit write or two 32-bit writes, starting with the higher part. + +In this register, a physical RAM address to a FWCfgDmaAccess structure should +be written. This is the format of the FWCfgDmaAccess structure: +
+typedef struct FWCfgDmaAccess {

Quoted 8 lines

Documentation/devicetree/bindings/arm/fw-cfg.txt blob to be read from the data register has size 4, and it is to be interpreted · line 64

Quoted 4 lines
Documentation/devicetree/bindings/arm/fw-cfg.txt +4 -0
+    uint32_t control;
+    uint32_t length;
+    uint64_t address;
+} FWCfgDmaAccess;

I think including the selector field would be nice to avoid extra register accesses, but I'm not that familiar with fw_cfg so maybe there's a reason not to include that field.

It's just simplicity. If you want to read a few times from the same field (like in ACPI tables, read the data size and then the data), you need a way to enable and disable the selector and manage the current offset for that entry. This is already provided with the "old" interface.

Moreover, if, for some reason, both systems are being used simultaneously, some way of interacting both selectors and offsets is needed.

I think the overhead of writing the selector apart is not that big, compared to the trouble of adding a new one.

Thanks
Marc

Quoted 16 lines
Quoted 9 lines

+The fields of the structure are in big endian mode, and the field at the lowest +address is the "control" field.

Documentation/devicetree/bindings/arm/fw-cfg.txt +7 -0
+
+The "control" field has the following bits:
+ - Bit 0: Error
+ - Bit 1: Read
+ - Bit 2: Skip
+
+When an operation is triggered, if the "control" field has bit 1
Quoted 3 lines

set, a read +operation will be performed. "length" bytes for the current selector and +offset will be copied into the address specified by the "address" field.

Minor clarification:
s/address/physical RAM address/

Quoted 1 line

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#25 lore raw

Re: [PATCH v2] QEMU fw_cfg DMA interface documentation

Hi,

Quoted 5 lines

It's just simplicity. If you want to read a few times from the same field (like in ACPI tables, read the data size and then the data), you need a way to enable and disable the selector and manage the current offset for that entry. This is already provided with the "old" interface.

Could be handled with a 'select' control bit. Only when set select entry and reset offset to zero.

Also: would it make sense to allow an *array* of FWCfgDmaAccess structs? With a 'more' bit in control we could indicate that there are more entries. I'm not sure firmware would actually use that though ...

cheers,
Gerd

#26 lore raw

Re: [PATCH v2] QEMU fw_cfg DMA interface documentation

Quoted 8 lines · Gerd Hoffmann

Hi,

Quoted 5 lines

It's just simplicity. If you want to read a few times from the same field (like in ACPI tables, read the data size and then the data), you need a way to enable and disable the selector and manage the current offset for that entry. This is already provided with the "old" interface.

Quoted 6 lines

Could be handled with a 'select' control bit. Only when set select entry and reset offset to zero.

Also: would it make sense to allow an *array* of FWCfgDmaAccess structs? With a 'more' bit in control we could indicate that there are more entries. I'm not sure firmware would actually use that though ...

Seems far-fetched.

Thanks
Laszlo

Quoted 4 lines

cheers,
Gerd

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#27 lore raw

Re: [PATCH v2] QEMU fw_cfg DMA interface documentation

In reply to Gerd Hoffmann
Quoted 8 lines · Gerd Hoffmann
Quoted 5 lines

It's just simplicity. If you want to read a few times from the same field (like in ACPI tables, read the data size and then the data), you need a way to enable and disable the selector and manage the current offset for that entry. This is already provided with the "old" interface.

Could be handled with a 'select' control bit. Only when set select entry and reset offset to zero.

I think two features would help "round off" the new fw_cfg DMA
proposal: add a select bit as you describe (that uses the 16 most significant bits of the "control" field for the "select entry" when the bit is set), and define a static signature (eg, "QEMU CFG") when reading the 64bit MMIO dma register.

Both are optional features that don't change the fundamental interface; I was thinking of sending them as two patches on top of Marc's next version of his patch series (if no one else gets to it first).

-Kevin

author
#28 lore raw

Re: [PATCH v2] QEMU fw_cfg DMA interface documentation

Quoted 16 lines · On Tue, 8 Sep 2015 12:46:08 -0400 "Kevin O'Connor" <[email protected]> wrote:
Quoted 8 lines · Gerd Hoffmann
Quoted 5 lines

It's just simplicity. If you want to read a few times from the same field (like in ACPI tables, read the data size and then the data), you need a way to enable and disable the selector and manage the current offset for that entry. This is already provided with the "old" interface.

Could be handled with a 'select' control bit. Only when set select entry and reset offset to zero.

I think two features would help "round off" the new fw_cfg DMA
proposal: add a select bit as you describe (that uses the 16 most significant bits of the "control" field for the "select entry" when the bit is set), and define a static signature (eg, "QEMU CFG") when reading the 64bit MMIO dma register.

Quoted 5 lines

Both are optional features that don't change the fundamental interface; I was thinking of sending them as two patches on top of Marc's next version of his patch series (if no one else gets to it first).

As there will be (at least) another version, I can add those simple changes in the next version.

Thanks
Marc