Skip to contentSkip to conversation

Add i2c arbitration support for new SoCs

PATCH In review Jan Dabros 7 participants 30 messages Sep 16, 2022 – Mar 20, 2023

Subsystems (4)
lore
Share & tools
lore

Jan Dabros

author
#0 lore raw

cover Add i2c arbitration support for new SoCs

This patchset comprises changes into i2c-designware-amdpsp.c module which aims to add support for new SoCs, while keep backward compatibility with Cezanne platforms.

Beside new algorithm introduced for the PSP-x86 communication, it also switches from MSR/MMIO access to SMN (System Management Network) since only the latter is working on both old new revisions of SoCs.

Jan Dabros (2):
i2c: designware: Switch from using MMIO access to SMN access
i2c: designware: Add support for new SoCs in AMDPSP driver

 arch/x86/include/asm/amd_nb.h               |   1 +
 arch/x86/kernel/amd_nb.c                    |   3 +-
 drivers/i2c/busses/i2c-designware-amdpsp.c  | 199 +++++++++++++-------
 drivers/i2c/busses/i2c-designware-core.h    |   1 -
 drivers/i2c/busses/i2c-designware-platdrv.c |   1 -
 5 files changed, 134 insertions(+), 71 deletions(-)
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author
#1 lore raw

PATCH 1/2 i2c: designware: Switch from using MMIO access to SMN access

Due to a change in silicon compared to Cezanne, in future revisions MSR access can't be used to get the base address of the PSP MMIO region that contains the PSP mailbox interface.

Modify driver to use SMN access also for Cezanne platforms (it is working there) in order to simplify codebase when adding support for new SoC versions.

Export amd_cache_northbridges() which was unexported by

e1907d3: "x86/amd_nb: Unexport amd_cache_northbridges()"

since function which registers i2c-designware-platdrv is a subsys_initcall that is executed before fs_initcall (when enumeration of NB descriptors occurs). Thus in order to use SMN accesses it's necessary to explicitly call amd_cache_northrbidges() from within this driver.

---
 arch/x86/include/asm/amd_nb.h               |   1 +
 arch/x86/kernel/amd_nb.c                    |   3 +-
 drivers/i2c/busses/i2c-designware-amdpsp.c  | 112 +++++++++++---------
 drivers/i2c/busses/i2c-designware-core.h    |   1 -
 drivers/i2c/busses/i2c-designware-platdrv.c |   1 -
 5 files changed, 66 insertions(+), 52 deletions(-)
arch/x86/include/asm/amd_nb.h +1 -0
index ed0eaf65c437..00d1a400b7a1 100644
@@ -16,6 +16,7 @@ extern const struct amd_nb_bus_dev_range amd_nb_bus_dev_ranges[];
 
 extern bool early_is_amd_nb(u32 value);
 extern struct resource *amd_get_mmconfig_range(struct resource *res);
+extern int amd_cache_northbridges(void);
 extern void amd_flush_garts(void);
 extern int amd_numa_init(void);
 extern int amd_get_subcaches(int);
arch/x86/kernel/amd_nb.c +2 -1
index 4266b64631a4..2077b6cfa8ad 100644
@@ -201,7 +201,7 @@ int amd_smn_write(u16 node, u32 address, u32 value)
 EXPORT_SYMBOL_GPL(amd_smn_write);
 
 
-static int amd_cache_northbridges(void)
+int amd_cache_northbridges(void)
 {
 	const struct pci_device_id *misc_ids = amd_nb_misc_ids;
 	const struct pci_device_id *link_ids = amd_nb_link_ids;
@@ -303,6 +303,7 @@ static int amd_cache_northbridges(void)
 
 	return 0;
 }
+EXPORT_SYMBOL_GPL(amd_cache_northbridges);
 
 /*
  * Ignores subdevice/subvendor but as far as I can figure out
drivers/i2c/busses/i2c-designware-amdpsp.c +63 -49

197 lines not on this page · raw message

drivers/i2c/busses/i2c-designware-core.h +0 -1
index 70b80e710990..80dad6c8f321 100644
@@ -382,7 +382,6 @@ int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev);
 
 #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP)
 int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev);
-void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev);
 #endif
 
 int i2c_dw_validate_speed(struct dw_i2c_dev *dev);
drivers/i2c/busses/i2c-designware-platdrv.c +0 -1
index ba043b547393..99f54fe583e1 100644
@@ -214,7 +214,6 @@ static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
 #ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
 	{
 		.probe = i2c_dw_amdpsp_probe_lock_support,
-		.remove = i2c_dw_amdpsp_remove_lock_support,
 	},
 #endif
 	{}
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
#2 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

+ the people who signed off on that original commit that is essentially unreverted.

Comments below.

Quoted 52 lines · Jan Dabros

Due to a change in silicon compared to Cezanne, in future revisions MSR access can't be used to get the base address of the PSP MMIO region that contains the PSP mailbox interface.

Modify driver to use SMN access also for Cezanne platforms (it is working there) in order to simplify codebase when adding support for new SoC versions.

Export amd_cache_northbridges() which was unexported by

e1907d3: "x86/amd_nb: Unexport amd_cache_northbridges()"

since function which registers i2c-designware-platdrv is a subsys_initcall that is executed before fs_initcall (when enumeration of NB descriptors occurs). Thus in order to use SMN accesses it's necessary to explicitly call amd_cache_northrbidges() from within this driver.

---
  arch/x86/include/asm/amd_nb.h               |   1 +
  arch/x86/kernel/amd_nb.c                    |   3 +-
  drivers/i2c/busses/i2c-designware-amdpsp.c  | 112 +++++++++++---------
  drivers/i2c/busses/i2c-designware-core.h    |   1 -
  drivers/i2c/busses/i2c-designware-platdrv.c |   1 -
  5 files changed, 66 insertions(+), 52 deletions(-)
arch/x86/include/asm/amd_nb.h +1 -0
index ed0eaf65c437..00d1a400b7a1 100644
@@ -16,6 +16,7 @@ extern const struct amd_nb_bus_dev_range amd_nb_bus_dev_ranges[];
 
  extern bool early_is_amd_nb(u32 value);
  extern struct resource *amd_get_mmconfig_range(struct resource *res);
+extern int amd_cache_northbridges(void);
  extern void amd_flush_garts(void);
  extern int amd_numa_init(void);
  extern int amd_get_subcaches(int);
arch/x86/kernel/amd_nb.c +1 -1
index 4266b64631a4..2077b6cfa8ad 100644
@@ -201,7 +201,7 @@ int amd_smn_write(u16 node, u32 address, u32 value)
  EXPORT_SYMBOL_GPL(amd_smn_write);
 
 
-static int amd_cache_northbridges(void)
+int amd_cache_northbridges(void)
  {
  	const struct pci_device_id *misc_ids = amd_nb_misc_ids;
  	const struct pci_device_id *link_ids = amd_nb_link_ids;

arch/x86/kernel/amd_nb.c static int amd_cache_northbridges(void) · line 307

Quoted 5 lines
arch/x86/kernel/amd_nb.c +1 -0
@@ -303,6 +303,7 @@ static int amd_cache_northbridges(void)
 
  	return 0;
  }
+EXPORT_SYMBOL_GPL(amd_cache_northbridges);
 

I feel changes to amd_nb.c/amd_nb.h should stand on their own patch in the series rather than being in this patch.

Quoted 21 lines
arch/x86/kernel/amd_nb.c +0 -0
  /*
   * Ignores subdevice/subvendor but as far as I can figure out
drivers/i2c/busses/i2c-designware-amdpsp.c +1 -2
index 8f36167bce62..1d467fc83f59 100644
@@ -8,12 +8,11 @@
  #include <linux/types.h>
  #include <linux/workqueue.h>
 
+#include <asm/amd_nb.h>
  #include <asm/msr.h>
 
  #include "i2c-designware-core.h"
 
-#define MSR_AMD_PSP_ADDR	0xc00110a2
-#define PSP_MBOX_OFFSET		0x10570
  #define PSP_CMD_TIMEOUT_US	(500 * USEC_PER_MSEC)
 
  #define PSP_I2C_RESERVATION_TIME_MS 100

drivers/i2c/busses/i2c-designware-amdpsp.c line 36

Quoted 7 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +4 -0
@@ -31,6 +30,10 @@
  #define PSP_MBOX_FIELDS_RECOVERY	BIT(30)
  #define PSP_MBOX_FIELDS_READY		BIT(31)
 
+#define PSP_MBOX_CMD_OFFSET		0x3810570
+#define PSP_MBOX_BUFFER_L_OFFSET	0x3810574
+#define PSP_MBOX_BUFFER_H_OFFSET	0x3810578
+

Just in case these offsets change for a future program, I think you should leave a comment here on the two APU programs they're valid for in case you need to special case it later.

Another approach is that you can have a lookup function and match some PCI device like the root device or the CPUID and then when another program comes along explicitly add it to this list.

This is what is done in drivers/platform/x86/amd/pmc.c for example.

Quoted 29 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +1 -11
  struct psp_req_buffer_hdr {
  	u32 total_size;
  	u32 status;
@@ -47,14 +50,8 @@ struct psp_i2c_req {
  	enum psp_i2c_req_type type;
  };
 
-struct psp_mbox {
-	u32 cmd_fields;
-	u64 i2c_req_addr;
-} __packed;
-
  static DEFINE_MUTEX(psp_i2c_access_mutex);
  static unsigned long psp_i2c_sem_acquired;
-static void __iomem *mbox_iomem;
  static u32 psp_i2c_access_count;
  static bool psp_i2c_mbox_fail;
  static struct device *psp_i2c_dev;
@@ -64,47 +61,43 @@ static struct device *psp_i2c_dev;
   * family of SoCs.
   */
 
-static int psp_get_mbox_addr(unsigned long *mbox_addr)
+static int psp_mbox_probe(void)
  {
-	unsigned long long psp_mmio;
-
-	if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
-		return -EIO;

drivers/i2c/busses/i2c-designware-amdpsp.c static struct device *psp_i2c_dev; · line 71

Quoted 10 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +6 -4
-
-	*mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
-
-	return 0;
+	/*
+	 * Explicitly initialize system management network interface here, since
+	 * usual init happens only after PCI subsystem is ready. This is too late
+	 * for I2C controller driver which may be executed earlier.
+	 */
+	return amd_cache_northbridges();

When a future program is added even if SMN addresses are the same and no special casing needed there is an implicit dependency on the PCI IDs being in arch/x86/kernel/amd_nb.c now. In order to make your life easier to debug in the future, suggest that you capture the return variable and emit a specific error message if amd_cache_northbridges() fails.

Quoted 167 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +51 -32
  }
 
-static int psp_mbox_probe(void)
+static int psp_smn_write(u32 smn_addr, u32 value)
  {
-	unsigned long mbox_addr;
-	int ret;
-
-	ret = psp_get_mbox_addr(&mbox_addr);
-	if (ret)
-		return ret;
-
-	mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
-	if (!mbox_iomem)
-		return -ENOMEM;
+	return amd_smn_write(0, smn_addr, value);
+}
 
-	return 0;
+static int psp_smn_read(u32 smn_addr, u32 *value)
+{
+	return amd_smn_read(0, smn_addr, value);
  }
 
  /* Recovery field should be equal 0 to start sending commands */
-static int psp_check_mbox_recovery(struct psp_mbox __iomem *mbox)
+static int psp_check_mbox_recovery(void)
  {
  	u32 tmp;
+	int status;
 
-	tmp = readl(&mbox->cmd_fields);
+	status = psp_smn_read(PSP_MBOX_CMD_OFFSET, &tmp);
+	if (status)
+		return status;
 
  	return FIELD_GET(PSP_MBOX_FIELDS_RECOVERY, tmp);
  }
 
-static int psp_wait_cmd(struct psp_mbox __iomem *mbox)
+static int psp_wait_cmd(void)
  {
  	u32 tmp, expected;
+	int ret, status;
 
  	/* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
  	expected = FIELD_PREP(PSP_MBOX_FIELDS_READY, 1);
@@ -113,30 +106,55 @@ static int psp_wait_cmd(struct psp_mbox __iomem *mbox)
  	 * Check for readiness of PSP mailbox in a tight loop in order to
  	 * process further as soon as command was consumed.
  	 */
-	return readl_poll_timeout(&mbox->cmd_fields, tmp, (tmp == expected),
-				  0, PSP_CMD_TIMEOUT_US);
+	ret = read_poll_timeout(psp_smn_read, status,
+				(status < 0) || (tmp == expected), 0,
+				PSP_CMD_TIMEOUT_US, 0, PSP_MBOX_CMD_OFFSET,
+				&tmp);
+	if (status)
+		ret = status;
+
+	return ret;
  }
 
  /* Status equal to 0 means that PSP succeed processing command */
-static u32 psp_check_mbox_sts(struct psp_mbox __iomem *mbox)
+static int psp_check_mbox_sts(void)
  {
  	u32 cmd_reg;
+	int status;
 
-	cmd_reg = readl(&mbox->cmd_fields);
+	status = psp_smn_read(PSP_MBOX_CMD_OFFSET, &cmd_reg);
+	if (status)
+		return status;
 
  	return FIELD_GET(PSP_MBOX_FIELDS_STS, cmd_reg);
  }
 
+static int psp_wr_mbox_buffer(phys_addr_t buf)
+{
+	u32 buf_addr_h = upper_32_bits(buf);
+	u32 buf_addr_l = lower_32_bits(buf);
+	int status;
+
+	status = psp_smn_write(PSP_MBOX_BUFFER_H_OFFSET, buf_addr_h);
+	if (status)
+		return status;
+
+	status = psp_smn_write(PSP_MBOX_BUFFER_L_OFFSET, buf_addr_l);
+	if (status)
+		return status;
+
+	return 0;
+}
+
  static int psp_send_cmd(struct psp_i2c_req *req)
  {
-	struct psp_mbox __iomem *mbox = mbox_iomem;
  	phys_addr_t req_addr;
  	u32 cmd_reg;
 
-	if (psp_check_mbox_recovery(mbox))
+	if (psp_check_mbox_recovery())
  		return -EIO;
 
-	if (psp_wait_cmd(mbox))
+	if (psp_wait_cmd())
  		return -EBUSY;
 
  	/*
@@ -145,16 +163,18 @@ static int psp_send_cmd(struct psp_i2c_req *req)
  	 * PSP. Use physical address of buffer, since PSP will map this region.
  	 */
  	req_addr = __psp_pa((void *)req);
-	writeq(req_addr, &mbox->i2c_req_addr);
+	if (psp_wr_mbox_buffer(req_addr))
+		return -EIO;
 
  	/* Write command register to trigger processing */
  	cmd_reg = FIELD_PREP(PSP_MBOX_FIELDS_CMD, PSP_I2C_REQ_BUS_CMD);
-	writel(cmd_reg, &mbox->cmd_fields);
+	if (psp_smn_write(PSP_MBOX_CMD_OFFSET, cmd_reg))
+		return -EIO;
 
-	if (psp_wait_cmd(mbox))
+	if (psp_wait_cmd())
  		return -ETIMEDOUT;
 
-	if (psp_check_mbox_sts(mbox))
+	if (psp_check_mbox_sts())
  		return -EIO;
 
  	return 0;
@@ -417,9 +437,3 @@ int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev)
 
  	return 0;
  }
-
-/* Unmap area used as a mailbox with PSP */
-void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev)
-{
-	iounmap(mbox_iomem);
-}
drivers/i2c/busses/i2c-designware-core.h +0 -1
index 70b80e710990..80dad6c8f321 100644
@@ -382,7 +382,6 @@ int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev);
 
  #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP)
  int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev);
-void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev);
  #endif
 
  int i2c_dw_validate_speed(struct dw_i2c_dev *dev);
drivers/i2c/busses/i2c-designware-platdrv.c +0 -1
index ba043b547393..99f54fe583e1 100644
@@ -214,7 +214,6 @@ static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
  #ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
  	{
  		.probe = i2c_dw_amdpsp_probe_lock_support,
-		.remove = i2c_dw_amdpsp_remove_lock_support,
  	},
  #endif
  	{}
#3 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Hi Mario,

(...)

Quoted 9 lines

arch/x86/kernel/amd_nb.c static int amd_cache_northbridges(void) · line 307

Quoted 6 lines
arch/x86/kernel/amd_nb.c +1 -0
@@ -303,6 +303,7 @@ static int amd_cache_northbridges(void)
 
      return 0;
  }
+EXPORT_SYMBOL_GPL(amd_cache_northbridges);
 

I feel changes to amd_nb.c/amd_nb.h should stand on their own patch in the series rather than being in this patch.

I was thinking about this initially, however I wanted to avoid a situation where we are exporting a symbol without actually using it (since this will be done in consecutive patch). It is also way easier for me to explain why I'm doing this in the context of i2c-designware-amdpsp.c changes.
That being said, if you think this should be a separate patch, that's fine with me - I don't have that strong feelings about it.

(...)

Quoted 13 lines

drivers/i2c/busses/i2c-designware-amdpsp.c line 36

Quoted 8 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +4 -0
@@ -31,6 +30,10 @@
  #define PSP_MBOX_FIELDS_RECOVERY    BIT(30)
  #define PSP_MBOX_FIELDS_READY               BIT(31)
 
+#define PSP_MBOX_CMD_OFFSET          0x3810570
+#define PSP_MBOX_BUFFER_L_OFFSET     0x3810574
+#define PSP_MBOX_BUFFER_H_OFFSET     0x3810578
+

Just in case these offsets change for a future program, I think you should leave a comment here on the two APU programs they're valid for in case you need to special case it later.

Quoted 5 lines

Another approach is that you can have a lookup function and match some PCI device like the root device or the CPUID and then when another program comes along explicitly add it to this list.

This is what is done in drivers/platform/x86/amd/pmc.c for example.

Yes, I see your point. I assumed that this will _not_ change and the new APU will just work out of the box - without a need to introduce a new ID to the table.
Bad thing with my approach is if this _will_ change, then it may not be that obvious for the developer what the issue actually is.

Considering your next comment - new PCI ID will anyway needs to be added to arch/x86/kernel/amd_nb.c - let's be consistent with this approach and I will introduce a list of supported CPU IDs.

Quoted 39 lines
Quoted 39 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +6 -15
  struct psp_req_buffer_hdr {
      u32 total_size;
      u32 status;
@@ -47,14 +50,8 @@ struct psp_i2c_req {
      enum psp_i2c_req_type type;
  };
 
-struct psp_mbox {
-     u32 cmd_fields;
-     u64 i2c_req_addr;
-} __packed;
-
  static DEFINE_MUTEX(psp_i2c_access_mutex);
  static unsigned long psp_i2c_sem_acquired;
-static void __iomem *mbox_iomem;
  static u32 psp_i2c_access_count;
  static bool psp_i2c_mbox_fail;
  static struct device *psp_i2c_dev;
@@ -64,47 +61,43 @@ static struct device *psp_i2c_dev;
   * family of SoCs.
   */
 
-static int psp_get_mbox_addr(unsigned long *mbox_addr)
+static int psp_mbox_probe(void)
  {
-     unsigned long long psp_mmio;
-
-     if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
-             return -EIO;
-
-     *mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
-
-     return 0;
+     /*
+      * Explicitly initialize system management network interface here, since
+      * usual init happens only after PCI subsystem is ready. This is too late
+      * for I2C controller driver which may be executed earlier.
+      */
Quoted 8 lines

drivers/i2c/busses/i2c-designware-amdpsp.c static struct device *psp_i2c_dev; · line 71

Quoted 1 line
drivers/i2c/busses/i2c-designware-amdpsp.c +1 -0
+     return amd_cache_northbridges();

When a future program is added even if SMN addresses are the same and no special casing needed there is an implicit dependency on the PCI IDs being in arch/x86/kernel/amd_nb.c now. In order to make your life easier to debug in the future, suggest that you capture the return variable and emit a specific error message if amd_cache_northbridges() fails.

Very good point, thanks!

Best Regards,
Jan

#4 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Quoted 1 line · Jan Dąbroś
Quoted 1 line
Quoted 1 line

+EXPORT_SYMBOL_GPL(amd_cache_northbridges);

Why is this being exported again?

It is called unconditionally as a fs_initcall()...

Signature · 6 lines
Regards/Gruss,
    Boris.

SUSE Software Solutions Germany GmbH
GF: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
(HRB 36809, AG Nürnberg)
#5 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Quoted 7 lines · Borislav Petkov
Quoted 1 line · Jan Dąbroś
Quoted 1 line
Quoted 1 line

+EXPORT_SYMBOL_GPL(amd_cache_northbridges);

Why is this being exported again?

It is called unconditionally as a fs_initcall()...

Jan mentioned this in the commit message:

Quoted 2 lines

The function which registers i2c-designware-platdrv is a
subsys_initcall that is executed before fs_initcall (when enumeration > of NB descriptors occurs).

So if it's not exported again, then it means that we somehow need to get i2c-designware-platdrv to register earlier too.

#6 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Quoted 8 lines · Mario

Jan mentioned this in the commit message:

Quoted 2 lines

The function which registers i2c-designware-platdrv is a subsys_initcall that is executed before fs_initcall (when enumeration > of

NB descriptors occurs).

So if it's not exported again, then it means that we somehow need to get i2c-designware-platdrv to register earlier too.

So I have this there:

/* This has to go after the PCI subsystem */
fs_initcall(init_amd_nbs);

as I need PCI. It itself does

arch_initcall(pci_arch_init);

so I guess init_amd_nbs() could be a subsys_initcall...

Or why is

subsys_initcall(dw_i2c_init_driver);

a subsys initcall in the first place?

Looking at

104522806a7d ("i2c: designware: dw_i2c_init_driver as subsys initcall")

I don't see a particular reason why it should be a subsys_initcall...

In any case, this should be fixed without an export which was crap in the first place.

Hm.

Signature · 6 lines
Regards/Gruss,
    Boris.

SUSE Software Solutions Germany GmbH
GF: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
(HRB 36809, AG Nürnberg)
#7 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Quoted 29 lines · Borislav Petkov
Quoted 8 lines · Mario

Jan mentioned this in the commit message:

Quoted 2 lines

The function which registers i2c-designware-platdrv is a subsys_initcall that is executed before fs_initcall (when enumeration > of

NB descriptors occurs).

So if it's not exported again, then it means that we somehow need to get i2c-designware-platdrv to register earlier too.

So I have this there:

/* This has to go after the PCI subsystem */
fs_initcall(init_amd_nbs);

as I need PCI. It itself does

arch_initcall(pci_arch_init);

so I guess init_amd_nbs() could be a subsys_initcall...

Or why is

subsys_initcall(dw_i2c_init_driver);

a subsys initcall in the first place?

Looking at

Quoted 8 lines

104522806a7d ("i2c: designware: dw_i2c_init_driver as subsys initcall")

I don't see a particular reason why it should be a subsys_initcall...

In any case, this should be fixed without an export which was crap in the first place.

Hm.

I'm speculating here, but IIRC the I2C controllers may serve PMICs on some platform that are required to be present earlier due to some ACPI code accessing them. This Hans de Goede can confirm or correct me.

Another case comes to my mind is that I2C framework wants to initialize I2C peripherals which were supplied via struct i2c_board_info on earlier stages. And again comes to the specifics of the certain peripherals that needs for power / reset / etc control, i.o.w. critical hardware for the platforms.

But it's still what I remember and I can be mistaken.

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#8 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

+Cc: Hans (mentioned your name and was under impression that you are in Cc list already)

Quoted 49 lines · Andy Shevchenko
Quoted 29 lines · Borislav Petkov
Quoted 8 lines · Mario

Jan mentioned this in the commit message:

Quoted 2 lines

The function which registers i2c-designware-platdrv is a subsys_initcall that is executed before fs_initcall (when enumeration > of

NB descriptors occurs).

So if it's not exported again, then it means that we somehow need to get i2c-designware-platdrv to register earlier too.

So I have this there:

/* This has to go after the PCI subsystem */
fs_initcall(init_amd_nbs);

as I need PCI. It itself does

arch_initcall(pci_arch_init);

so I guess init_amd_nbs() could be a subsys_initcall...

Or why is

subsys_initcall(dw_i2c_init_driver);

a subsys initcall in the first place?

Looking at

Quoted 8 lines

104522806a7d ("i2c: designware: dw_i2c_init_driver as subsys initcall")

I don't see a particular reason why it should be a subsys_initcall...

In any case, this should be fixed without an export which was crap in the first place.

Hm.

I'm speculating here, but IIRC the I2C controllers may serve PMICs on some platform that are required to be present earlier due to some ACPI code accessing them. This Hans de Goede can confirm or correct me.

Another case comes to my mind is that I2C framework wants to initialize I2C peripherals which were supplied via struct i2c_board_info on earlier stages. And again comes to the specifics of the certain peripherals that needs for power / reset / etc control, i.o.w. critical hardware for the platforms.

But it's still what I remember and I can be mistaken.

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#9 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Hi,

Quoted 45 lines · Andy Shevchenko

+Cc: Hans (mentioned your name and was under impression that you are in Cc list already)

Quoted 42 lines · Andy Shevchenko
Quoted 29 lines · Borislav Petkov
Quoted 8 lines · Mario

Jan mentioned this in the commit message:

Quoted 2 lines

The function which registers i2c-designware-platdrv is a subsys_initcall that is executed before fs_initcall (when enumeration > of

NB descriptors occurs).

So if it's not exported again, then it means that we somehow need to get i2c-designware-platdrv to register earlier too.

So I have this there:

/* This has to go after the PCI subsystem */
fs_initcall(init_amd_nbs);

as I need PCI. It itself does

arch_initcall(pci_arch_init);

so I guess init_amd_nbs() could be a subsys_initcall...

Or why is

subsys_initcall(dw_i2c_init_driver);

a subsys initcall in the first place?

Looking at

Quoted 8 lines

104522806a7d ("i2c: designware: dw_i2c_init_driver as subsys initcall")

I don't see a particular reason why it should be a subsys_initcall...

In any case, this should be fixed without an export which was crap in the first place.

Hm.

I'm speculating here, but IIRC the I2C controllers may serve PMICs on some platform that are required to be present earlier due to some ACPI code accessing them. This Hans de Goede can confirm or correct me.

Right, thank you for Cc-ing me. At least on X86 there are several platforms (and 100-s of device models) which use a PMIC connected to the i2c-designware controller and this PMIC gets poked directly from ACPI _S0 and _S3 (power on/off) methods. So the I2C bus driver needs to *bind* to the controller as soon as we find its description in ACPI, otherwise we get a whole bunch of failed ACPI OpRegion access errors as well as various actual really issues.

So please keep this as a subsys initcall.

Regards,

Hans

#10 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Quoted 6 lines · Hans de Goede

Right, thank you for Cc-ing me. At least on X86 there are several platforms (and 100-s of device models) which use a PMIC connected to the i2c-designware controller and this PMIC gets poked directly from ACPI _S0 and _S3 (power on/off) methods. So the I2C bus driver needs to *bind* to the controller as soon as we find its description in ACPI, otherwise we get a whole bunch of failed ACPI OpRegion access errors as well as various actual really issues.

Thanks for explaining - I couldn't find the reason why it has to be a subsys_initcall.

Quoted 1 line

So please keep this as a subsys initcall.

Which means, init_amd_nbs() would have to be sorted to run before dw_i2c_init_driver()...

Signature · 6 lines
Regards/Gruss,
    Boris.

SUSE Software Solutions Germany GmbH
GF: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
(HRB 36809, AG Nürnberg)
#11 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Hi,

Quoted 8 lines · Borislav Petkov
Quoted 6 lines · Hans de Goede

Right, thank you for Cc-ing me. At least on X86 there are several platforms (and 100-s of device models) which use a PMIC connected to the i2c-designware controller and this PMIC gets poked directly from ACPI _S0 and _S3 (power on/off) methods. So the I2C bus driver needs to *bind* to the controller as soon as we find its description in ACPI, otherwise we get a whole bunch of failed ACPI OpRegion access errors as well as various actual really issues.

Quoted 7 lines

Thanks for explaining - I couldn't find the reason why it has to be a subsys_initcall.

Quoted 1 line

So please keep this as a subsys initcall.

Which means, init_amd_nbs() would have to be sorted to run before dw_i2c_init_driver()...

Yes if possible. One solution might be to make it a arch_initcall_sync() which AFAIK runs after regular arch_initcall()-s but before subsys_initcall()-s.

Regards,

Hans

#12 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Hi Borislav,

czw., 22 wrz 2022 o 16:29 Hans de Goede <[email protected]> napisał(a):

Quoted 15 lines

Hi,

Quoted 11 lines · Borislav Petkov
Quoted 6 lines · Hans de Goede

Right, thank you for Cc-ing me. At least on X86 there are several platforms (and 100-s of device models) which use a PMIC connected to the i2c-designware controller and this PMIC gets poked directly from ACPI _S0 and _S3 (power on/off) methods. So the I2C bus driver needs to *bind* to the controller as soon as we find its description in ACPI, otherwise we get a whole bunch of failed ACPI OpRegion access errors as well as various actual really issues.

Thanks for explaining - I couldn't find the reason why it has to be a subsys_initcall.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

So please keep this as a subsys initcall.

Which means, init_amd_nbs() would have to be sorted to run before dw_i2c_init_driver()...

Yes if possible. One solution might be to make it a arch_initcall_sync() which AFAIK runs after regular arch_initcall()-s but before subsys_initcall()-s.

What do you think about this?

Best Regards,
Jan

#13 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Quoted 1 line · Jan Dąbroś

What do you think about this?

I don't mind as long as it is tested properly so that nothing breaks
- I can help out on my end as I have a lot of AMD hw - and then properly documented in a comment above it *why* it needs to be arch_initcall_sync().

Thx.

Signature · 6 lines
Regards/Gruss,
    Boris.

SUSE Software Solutions Germany GmbH
GF: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
(HRB 36809, AG Nürnberg)
#14 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Hi,

Sorry for a late reply. I did some research and experiments and right now got stuck with this problem.

It's not enough for running init_amd_nbs() to have only pci_arch_init() done. We need the pci bus to be created and registered with all devices found on the bus. We are traversing through them and trying to find northbridge VID/DID. Due to the above, we need to run init_amd_nbs() only after acpi_scan_init() that is invoked from acpi_init() which is registered as subsys_initcall. That's why the trick with switching init_amd_nbs() to arch_initcall_sync will not work.

So to summarize everything, I would like below order:

acpi_init() -> init_amd_nbs() -> dw_i2c_init_driver()
^--subsys_initcall ^--fs_initicall ^--subsys_initcall

but I don't have a clear idea how to achieve this in a clean way.

The only option seems to be to register init_amd_nbs() as subsys_initcall and force it to execute after acpi_init() and before dw_i2c_init_drvier(). However the only option (if I'm not mistaken) for forcing order on initcalls placed on the same level is to modify their order within Makefile, so that linker puts them in the "init" section with addresses in desired order. This doesn't seem to be an option for upstream.

Do you have any clue how to solve this problem?

Best Regards,
Jan

pon., 26 wrz 2022 o 16:45 Borislav Petkov <[email protected]> napisał(a):

Quoted 18 lines · Jan Dąbroś
Quoted 1 line

What do you think about this?

I don't mind as long as it is tested properly so that nothing breaks
- I can help out on my end as I have a lot of AMD hw - and then properly documented in a comment above it *why* it needs to be arch_initcall_sync().

Thx.

--
Regards/Gruss,
Boris.

SUSE Software Solutions Germany GmbH
GF: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman (HRB 36809, AG Nürnberg)

maintainer
#15 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Another forgotten thread... ;-\

+ Yazen.

Quoted 15 lines · Jan Dąbroś

So to summarize everything, I would like below order:

acpi_init() -> init_amd_nbs() -> dw_i2c_init_driver()
^--subsys_initcall ^--fs_initicall ^--subsys_initcall

but I don't have a clear idea how to achieve this in a clean way.

The only option seems to be to register init_amd_nbs() as subsys_initcall and force it to execute after acpi_init() and before dw_i2c_init_drvier(). However the only option (if I'm not mistaken) for forcing order on initcalls placed on the same level is to modify their order within Makefile, so that linker puts them in the "init" section with addresses in desired order. This doesn't seem to be an option for upstream.

Quoted 1 line

Do you have any clue how to solve this problem?

Make init_amd_nbs() arch_initcall_sync() so that it executes after PCI init.

By the time subsys_initcalls come, they'll have all the facilities they need, prepared for them...

Along with big fat comment why.

Btw, note to myself as I keep wondering about it each time: the sync calls come after the regular ones, in link order if you look at preprocessed linker script arch/x86/kernel/vmlinux.lds:

__initcall_start = .;
KEEP(*(.initcallearly.init)) __initcall0_start = .; KEEP(*(.initcall0.init)) KEEP(*(.initcall0s.init)) __initcall1_start = .; KEEP(*(.initcall1.init)) KEEP(*(.initcall1s.init)) __initcall2_start = .; KEEP(*(.initcall2.init)) KEEP(*(.initcall2s.init)) __initcall3_start = .; KEEP(*(.initcall3.init)) KEEP(*(.initcall3s.init)) __initcall4_start = .; KEEP(*(.initcall4.init)) KEEP(*(.initcall4s.init)) __initcall5_start = .; KEEP(*(.initcall5.init)) KEEP(*(.initcall5s.init)) __initcallrootfs_start = .; KEEP(*(.initcallrootfs.init)) KEEP(*(.initcallrootfss.init)) __initcall6_start = .; KEEP(*(.initcall6.init)) KEEP(*(.initcall6s.init)) __initcall7_start = .; KEEP(*(.initcall7.init)) KEEP(*(.initcall7s.init)) __initcall_end = .;

Mario, is that something that would work for what you wanna do too?

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#16 lore raw

RE: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

[Public]

+Grzegorz who Jan mentioned is taking over some of this. +Vijendar for sound/soc/amd

Quoted 62 lines

-----Original Message-----
From: Borislav Petkov <[email protected]>
Sent: Monday, January 9, 2023 05:12
To: Jan Dąbroś <[email protected]>; Limonciello, Mario <[email protected]>
Cc: Borislav Petkov <[email protected]>; Hans de Goede <[email protected]>; Andy Shevchenko <[email protected]>; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; M K, Muralidhara <[email protected]>; Chatradhi, Naveen Krishna <[email protected]>; Ghannam, Yazen <[email protected]>
Subject: Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Another forgotten thread... ;-\

+ Yazen.

Quoted 15 lines · Jan Dąbroś

So to summarize everything, I would like below order:

acpi_init() -> init_amd_nbs() -> dw_i2c_init_driver()
^--subsys_initcall ^--fs_initicall ^--subsys_initcall

but I don't have a clear idea how to achieve this in a clean way.

The only option seems to be to register init_amd_nbs() as subsys_initcall and force it to execute after acpi_init() and before dw_i2c_init_drvier(). However the only option (if I'm not mistaken) for forcing order on initcalls placed on the same level is to modify their order within Makefile, so that linker puts them in the "init" section with addresses in desired order. This doesn't seem to be an option for upstream.

Quoted 1 line

Do you have any clue how to solve this problem?

Make init_amd_nbs() arch_initcall_sync() so that it executes after PCI init.

By the time subsys_initcalls come, they'll have all the facilities they need, prepared for them...

Along with big fat comment why.

Btw, note to myself as I keep wondering about it each time: the sync calls come
after the regular ones, in link order if you look at preprocessed linker script arch/x86/kernel/vmlinux.lds:

__initcall_start = .;
KEEP(*(.initcallearly.init)) __initcall0_start = .; KEEP(*(.initcall0.init)) KEEP(*(.initcall0s.init)) __initcall1_start = .; KEEP(*(.initcall1.init)) KEEP(*(.initcall1s.init)) __initcall2_start = .; KEEP(*(.initcall2.init)) KEEP(*(.initcall2s.init)) __initcall3_start = .; KEEP(*(.initcall3.init)) KEEP(*(.initcall3s.init)) __initcall4_start = .; KEEP(*(.initcall4.init)) KEEP(*(.initcall4s.init)) __initcall5_start = .; KEEP(*(.initcall5.init)) KEEP(*(.initcall5s.init)) __initcallrootfs_start = .; KEEP(*(.initcallrootfs.init)) KEEP(*(.initcallrootfss.init)) __initcall6_start = .; KEEP(*(.initcall6.init)) KEEP(*(.initcall6s.init)) __initcall7_start = .; KEEP(*(.initcall7.init)) KEEP(*(.initcall7s.init)) __initcall_end = .;

Quoted 2 lines

Mario, is that something that would work for what you wanna do too?

Yeah I guess as long as the I2C init comes after arch_initcall_sync this should work.

If it works for Jan & Grzegorz, then I think we can also adjust the "effectively local copy" of amd_smn_write/amd_smn_read that got added into sound/soc/amd
to use this too now and it should work for what I wanted to do too.

Jan/Grzegorz can you please have a try on your system and comment?

Quoted 14 lines

Thx.

--
Regards/Gruss,
Boris.

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeo
ple.kernel.org%2Ftglx%2Fnotes-about- netiquette&data=05%7C01%7Cmario.limonciello%40amd.com%7C56e3280fc 1cd416bd7ae08daf23262d0%7C3dd8961fe4884e608e11a82d994e183d%7C0% 7C0%7C638088595520420341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4 wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7 C%7C%7C&sdata=9t7J4u3W0kxXu717gzIyBJpAvt6YtQOKqtK9AcjWOvg%3D&r eserved=0

#17 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

In reply to Borislav Petkov

Hi Borislav,

Quoted 1 line

Make init_amd_nbs() arch_initcall_sync() so that it executes after PCI init.

I described earlier in this thread why such option is not working - let me quote myself:

It's not enough for running init_amd_nbs() to have only pci_arch_init() done. We need the pci bus to be created and registered with all devices found on the bus. We are traversing through them and trying to find northbridge VID/DID. Due to the above, we need to run init_amd_nbs() only after acpi_scan_init() that is invoked from acpi_init() which is registered as subsys_initcall. That's why the trick with switching init_amd_nbs() to arch_initcall_sync will not work.

We have a kind of chicken-and-egg problem here. Or is there something I missed?

I wonder if there is upstreamable option to control order of the drivers' init by forcing link order?

Best Regards,
Jan

#18 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Quoted 16 lines · Jan Dąbroś

Hi Borislav,

Quoted 1 line

Make init_amd_nbs() arch_initcall_sync() so that it executes after PCI init.

I described earlier in this thread why such option is not working - let me quote myself:

It's not enough for running init_amd_nbs() to have only pci_arch_init() done. We need the pci bus to be created and registered with all devices found on the bus. We are traversing through them and trying to find northbridge VID/DID. Due to the above, we need to run init_amd_nbs() only after acpi_scan_init() that is invoked from acpi_init() which is registered as subsys_initcall. That's why the trick with switching init_amd_nbs() to arch_initcall_sync will not work.

Quoted 4 lines

We have a kind of chicken-and-egg problem here. Or is there something I missed?

I wonder if there is upstreamable option to control order of the drivers' init by forcing link order?

But what exactly do you need from North Bridge? Is it only its existence or do you need to have fully instantiated PCI device (if so, why?)?

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#19 lore raw

RE: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

[Public]

Quoted 33 lines

-----Original Message-----
From: Andy Shevchenko <[email protected]>
Sent: Monday, January 16, 2023 06:39
To: Jan Dąbroś <[email protected]>
Cc: Borislav Petkov <[email protected]>; Limonciello, Mario <[email protected]>; Borislav Petkov <[email protected]>; Hans de Goede <[email protected]>; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; M K, Muralidhara <[email protected]>; Chatradhi, Naveen Krishna <[email protected]>; Ghannam, Yazen <[email protected]>
Subject: Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Quoted 3 lines · Jan Dąbroś

Hi Borislav,

Quoted 1 line

Make init_amd_nbs() arch_initcall_sync() so that it executes after PCI

init.

Quoted 13 lines

I described earlier in this thread why such option is not working - let me quote myself:

It's not enough for running init_amd_nbs() to have only pci_arch_init() done. We need the pci bus to be created and registered with all devices found on the bus. We are traversing through them and trying to find northbridge VID/DID. Due to the above, we need to run init_amd_nbs() only after acpi_scan_init() that is invoked from acpi_init() which is registered as subsys_initcall. That's why the trick with switching init_amd_nbs() to arch_initcall_sync will not work.

Quoted 9 lines
Quoted 1 line

We have a kind of chicken-and-egg problem here. Or is there something I

missed?

Quoted 3 lines

I wonder if there is upstreamable option to control order of the drivers' init by forcing link order?

But what exactly do you need from North Bridge? Is it only its existence or do you need to have fully instantiated PCI device (if so, why?)?

There is a need to be able to write and read PCI config space.

#20 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Quoted 23 lines · Mario
Quoted 23 lines

From: Andy Shevchenko <[email protected]>
Sent: Monday, January 16, 2023 06:39
To: Jan Dąbroś <[email protected]>

Quoted 1 line · Jan Dąbroś
Quoted 1 line

Make init_amd_nbs() arch_initcall_sync() so that it executes after PCI

init.

Quoted 13 lines

I described earlier in this thread why such option is not working - let me quote myself:

It's not enough for running init_amd_nbs() to have only pci_arch_init() done. We need the pci bus to be created and registered with all devices found on the bus. We are traversing through them and trying to find northbridge VID/DID. Due to the above, we need to run init_amd_nbs() only after acpi_scan_init() that is invoked from acpi_init() which is registered as subsys_initcall. That's why the trick with switching init_amd_nbs() to arch_initcall_sync will not work.

Quoted 1 line

We have a kind of chicken-and-egg problem here. Or is there something I

missed?

Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

I wonder if there is upstreamable option to control order of the drivers' init by forcing link order?

But what exactly do you need from North Bridge? Is it only its existence or do you need to have fully instantiated PCI device (if so, why?)?

There is a need to be able to write and read PCI config space.

So, it's available even on early stages, are there some specifics why it can't be done using the respective APIs?

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#21 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Quoted 27 lines · Andy Shevchenko
Quoted 26 lines · Mario
Quoted 26 lines

From: Andy Shevchenko <[email protected]>
Sent: Monday, January 16, 2023 06:39
To: Jan Dąbroś <[email protected]>

Quoted 1 line · Jan Dąbroś
Quoted 1 line

Make init_amd_nbs() arch_initcall_sync() so that it executes after PCI

init.

Quoted 13 lines

I described earlier in this thread why such option is not working - let me quote myself:

It's not enough for running init_amd_nbs() to have only pci_arch_init() done. We need the pci bus to be created and registered with all devices found on the bus. We are traversing through them and trying to find northbridge VID/DID. Due to the above, we need to run init_amd_nbs() only after acpi_scan_init() that is invoked from acpi_init() which is registered as subsys_initcall. That's why the trick with switching init_amd_nbs() to arch_initcall_sync will not work.

Quoted 1 line

We have a kind of chicken-and-egg problem here. Or is there something I

missed?

Quoted 3 lines

I wonder if there is upstreamable option to control order of the drivers' init by forcing link order?

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

But what exactly do you need from North Bridge? Is it only its existence or do you need to have fully instantiated PCI device (if so, why?)?

There is a need to be able to write and read PCI config space.

So, it's available even on early stages, are there some specifics why it can't be done using the respective APIs?

I think I understood the problem with the above. You probably don't know where NB can be located in the topology and that's why you can't simply access a _fixed_ BDF. So you want to have the topology being scanned beforehand.

Signature · 2 lines
With Best Regards,
Andy Shevchenko
maintainer
#22 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

In reply to Jan Dąbroś
Quoted 3 lines · Jan Dąbroś

It's not enough for running init_amd_nbs() to have only pci_arch_init() done. We need the pci bus to be created and registered with all devices found on the bus.

And when is that done and ready? pci_scan_bus()?

Lemme see if I understand the ordering correctly:

1. PCI needs to be initialized and all devices on the bus registered

2. AMD NB needs to run and enumerate the NBs so that the SMN access which you need, can work.

3. acpi_scan_init()

4. i2c-designware-platdrv registration

Close?

Now, Mario, remind me again, pls, why can't they use the MSR to get the PSP MMIO base. It has changed but why?

Maybe we should talk offlist first.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#23 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

In reply to Jan Dabros
Quoted 11 lines · Jan Dabros

Due to a change in silicon compared to Cezanne, in future revisions MSR access can't be used to get the base address of the PSP MMIO region that contains the PSP mailbox interface.

Modify driver to use SMN access also for Cezanne platforms (it is working there) in order to simplify codebase when adding support for new SoC versions.

Export amd_cache_northbridges() which was unexported by

e1907d3: "x86/amd_nb: Unexport amd_cache_northbridges()"

Please, use standard format of referring to the commits in the history (basically the same as for Fixes tags).

Quoted 4 lines

since function which registers i2c-designware-platdrv is a subsys_initcall that is executed before fs_initcall (when enumeration of NB descriptors occurs). Thus in order to use SMN accesses it's necessary to explicitly call amd_cache_northrbidges() from within this driver.

Also it doesn't clarify if this commit a full revert of that (rebased for new kernel versions) or partial or functional?

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#24 lore raw

Re: [PATCH -next 1/2] i2c: designware: Switch from using MMIO access to SMN access

Hi Andy,

pon., 19 wrz 2022 o 15:59 Andy Shevchenko
<[email protected]> napisał(a):

Quoted 10 lines · Jan Dabros
Quoted 8 lines

Due to a change in silicon compared to Cezanne, in future revisions MSR access can't be used to get the base address of the PSP MMIO region that contains the PSP mailbox interface.

Modify driver to use SMN access also for Cezanne platforms (it is working there) in order to simplify codebase when adding support for new SoC versions.

Quoted 6 lines
Quoted 3 lines

Export amd_cache_northbridges() which was unexported by

e1907d3: "x86/amd_nb: Unexport amd_cache_northbridges()"

Please, use standard format of referring to the commits in the history (basically the same as for Fixes tags).

Sure.

Quoted 8 lines
Quoted 5 lines

since function which registers i2c-designware-platdrv is a subsys_initcall that is executed before fs_initcall (when enumeration of NB descriptors occurs). Thus in order to use SMN accesses it's necessary to explicitly call amd_cache_northrbidges() from within this driver.

Also it doesn't clarify if this commit a full revert of that (rebased for new kernel versions) or partial or functional?

This is a partial revert, I will mention this in the commit msg.

Best Regards,
Jan

author
#25 lore raw

PATCH 2/2 i2c: designware: Add support for new SoCs in AMDPSP driver

In reply to Jan Dabros

New AMD SoCs are using different algorithm for x86-PSP communication, thus need to modify I2C arbitration driver. Since possible future revisions should have follow new approach, mark functions used only for Cezanne with "czn_" prefix.

While at it, remove redundant check by modifying psp_wait_cmd() to only check for MBOX_READY bit, since MBOX_CMD field being zero is verified by czn_psp_check_mbox_sts() later on in the sequence.

---
 drivers/i2c/busses/i2c-designware-amdpsp.c | 145 ++++++++++++++-------
 1 file changed, 97 insertions(+), 48 deletions(-)
drivers/i2c/busses/i2c-designware-amdpsp.c +97 -48
index 1d467fc83f59..4395a2ae960a 100644
@@ -4,6 +4,7 @@
 #include <linux/bits.h>
 #include <linux/i2c.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/pci.h>
 #include <linux/psp-sev.h>
 #include <linux/types.h>
 #include <linux/workqueue.h>
@@ -30,9 +31,13 @@
 #define PSP_MBOX_FIELDS_RECOVERY	BIT(30)
 #define PSP_MBOX_FIELDS_READY		BIT(31)
 
-#define PSP_MBOX_CMD_OFFSET		0x3810570
-#define PSP_MBOX_BUFFER_L_OFFSET	0x3810574
-#define PSP_MBOX_BUFFER_H_OFFSET	0x3810578
+#define CZN_PSP_MBOX_CMD_OFFSET		0x3810570
+#define CZN_PSP_MBOX_BUFFER_L_OFFSET	0x3810574
+#define CZN_PSP_MBOX_BUFFER_H_OFFSET	0x3810578
+#define PSP_MBOX_CMD_OFFSET		0x3810A40
+#define PSP_MBOX_DOORBELL_OFFSET	0x3810A24
+
+#define AMD_CPU_ID_CZN			0x1630
 
 struct psp_req_buffer_hdr {
 	u32 total_size;
@@ -55,6 +60,7 @@ static unsigned long psp_i2c_sem_acquired;
 static u32 psp_i2c_access_count;
 static bool psp_i2c_mbox_fail;
 static struct device *psp_i2c_dev;
+static unsigned short cpu_id;
 
 /*
  * Implementation of PSP-x86 i2c-arbitration mailbox introduced for AMD Cezanne
@@ -63,6 +69,17 @@ static struct device *psp_i2c_dev;
 
 static int psp_mbox_probe(void)
 {
+	struct pci_dev *rdev;
+
+	rdev = pci_get_domain_bus_and_slot(0, 0, PCI_DEVFN(0, 0));
+	if (!rdev) {
+		dev_err(psp_i2c_dev, "Failed to get host bridge device\n");
+		return -ENODEV;
+	}
+
+	cpu_id = rdev->device;
+	pci_dev_put(rdev);
+
 	/*
 	 * Explicitly initialize system management network interface here, since
 	 * usual init happens only after PCI subsystem is ready. This is too late
@@ -81,80 +98,76 @@ static int psp_smn_read(u32 smn_addr, u32 *value)
 	return amd_smn_read(0, smn_addr, value);
 }
 
-/* Recovery field should be equal 0 to start sending commands */
-static int psp_check_mbox_recovery(void)
+static int psp_mbox_ready(u32 smn_addr)
 {
 	u32 tmp;
-	int status;
-
-	status = psp_smn_read(PSP_MBOX_CMD_OFFSET, &tmp);
-	if (status)
-		return status;
-
-	return FIELD_GET(PSP_MBOX_FIELDS_RECOVERY, tmp);
-}
-
-static int psp_wait_cmd(void)
-{
-	u32 tmp, expected;
 	int ret, status;
 
-	/* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
-	expected = FIELD_PREP(PSP_MBOX_FIELDS_READY, 1);
-
 	/*
 	 * Check for readiness of PSP mailbox in a tight loop in order to
 	 * process further as soon as command was consumed.
 	 */
 	ret = read_poll_timeout(psp_smn_read, status,
-				(status < 0) || (tmp == expected), 0,
-				PSP_CMD_TIMEOUT_US, 0, PSP_MBOX_CMD_OFFSET,
-				&tmp);
+				(status < 0) || (tmp & PSP_MBOX_FIELDS_READY),
+				0, PSP_CMD_TIMEOUT_US, 0, smn_addr, &tmp);
 	if (status)
 		ret = status;
 
 	return ret;
 }
 
+/* Recovery field should be equal 0 to start sending commands */
+static int czn_psp_check_mbox_recovery(void)
+{
+	u32 tmp;
+	int status;
+
+	status = psp_smn_read(CZN_PSP_MBOX_CMD_OFFSET, &tmp);
+	if (status)
+		return status;
+
+	return FIELD_GET(PSP_MBOX_FIELDS_RECOVERY, tmp);
+}
+
 /* Status equal to 0 means that PSP succeed processing command */
-static int psp_check_mbox_sts(void)
+static u32 czn_psp_check_mbox_sts(void)
 {
 	u32 cmd_reg;
 	int status;
 
-	status = psp_smn_read(PSP_MBOX_CMD_OFFSET, &cmd_reg);
+	status = psp_smn_read(CZN_PSP_MBOX_CMD_OFFSET, &cmd_reg);
 	if (status)
 		return status;
 
 	return FIELD_GET(PSP_MBOX_FIELDS_STS, cmd_reg);
 }
 
-static int psp_wr_mbox_buffer(phys_addr_t buf)
+static int czn_psp_wr_mbox_buffer(phys_addr_t buf)
 {
 	u32 buf_addr_h = upper_32_bits(buf);
 	u32 buf_addr_l = lower_32_bits(buf);
 	int status;
 
-	status = psp_smn_write(PSP_MBOX_BUFFER_H_OFFSET, buf_addr_h);
+	status = psp_smn_write(CZN_PSP_MBOX_BUFFER_H_OFFSET, buf_addr_h);
 	if (status)
 		return status;
 
-	status = psp_smn_write(PSP_MBOX_BUFFER_L_OFFSET, buf_addr_l);
+	status = psp_smn_write(CZN_PSP_MBOX_BUFFER_L_OFFSET, buf_addr_l);
 	if (status)
 		return status;
 
 	return 0;
 }
 
-static int psp_send_cmd(struct psp_i2c_req *req)
+static int czn_psp_send_cmd(struct psp_i2c_req *req)
 {
 	phys_addr_t req_addr;
 	u32 cmd_reg;
 
-	if (psp_check_mbox_recovery())
+	if (czn_psp_check_mbox_recovery())
 		return -EIO;
 
-	if (psp_wait_cmd())
+	if (psp_mbox_ready(CZN_PSP_MBOX_CMD_OFFSET))
 		return -EBUSY;
 
 	/*
@@ -163,18 +176,18 @@ static int psp_send_cmd(struct psp_i2c_req *req)
 	 * PSP. Use physical address of buffer, since PSP will map this region.
 	 */
 	req_addr = __psp_pa((void *)req);
-	if (psp_wr_mbox_buffer(req_addr))
+	if (czn_psp_wr_mbox_buffer(req_addr))
 		return -EIO;
 
 	/* Write command register to trigger processing */
 	cmd_reg = FIELD_PREP(PSP_MBOX_FIELDS_CMD, PSP_I2C_REQ_BUS_CMD);
-	if (psp_smn_write(PSP_MBOX_CMD_OFFSET, cmd_reg))
+	if (psp_smn_write(CZN_PSP_MBOX_CMD_OFFSET, cmd_reg))
 		return -EIO;
 
-	if (psp_wait_cmd())
+	if (psp_mbox_ready(CZN_PSP_MBOX_CMD_OFFSET))
 		return -ETIMEDOUT;
 
-	if (psp_check_mbox_sts())
+	if (czn_psp_check_mbox_sts())
 		return -EIO;
 
 	return 0;
@@ -185,8 +198,13 @@ static int check_i2c_req_sts(struct psp_i2c_req *req)
 {
 	u32 status;
 
-	/* Status field in command-response buffer is updated by PSP */
-	status = READ_ONCE(req->hdr.status);
+	if (req) {
+		/* Status field in command-response buffer is updated by PSP */
+		status = READ_ONCE(req->hdr.status);
+	} else {
+		status = psp_smn_read(PSP_MBOX_CMD_OFFSET, &status);
+		status &= ~PSP_MBOX_FIELDS_READY;
+	}
 
 	switch (status) {
 	case PSP_I2C_REQ_STS_OK:
@@ -199,8 +217,31 @@ static int check_i2c_req_sts(struct psp_i2c_req *req)
 	}
 }
 
-static int psp_send_check_i2c_req(struct psp_i2c_req *req)
+static int psp_send_cmd(enum psp_i2c_req_type i2c_req_type)
+{
+	int ret;
+
+	ret = psp_mbox_ready(PSP_MBOX_CMD_OFFSET);
+	if (ret)
+		return ret;
+
+	psp_smn_write(PSP_MBOX_CMD_OFFSET, i2c_req_type);
+
+	/* Ring the Doorbell for PSP by writing a non-zero value */
+	psp_smn_write(PSP_MBOX_DOORBELL_OFFSET, 0x1);
+
+	ret = psp_mbox_ready(PSP_MBOX_CMD_OFFSET);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int psp_send_check_i2c_req(struct psp_i2c_req *req,
+				  enum psp_i2c_req_type i2c_req_type)
 {
+	int ret;
+
 	/*
 	 * Errors in x86-PSP i2c-arbitration protocol may occur at two levels:
 	 * 1. mailbox communication - PSP is not operational or some IO errors
@@ -208,10 +249,15 @@ static int psp_send_check_i2c_req(struct psp_i2c_req *req)
 	 * 2. i2c-requests - PSP refuses to grant i2c arbitration to x86 for too
 	 * long.
 	 * In order to distinguish between these two in error handling code, all
-	 * errors on the first level (returned by psp_send_cmd) are shadowed by
+	 * errors on the first level (returned by *psp_send_cmd) are shadowed by
 	 * -EIO.
 	 */
-	if (psp_send_cmd(req))
+	if (req)
+		ret = czn_psp_send_cmd(req);
+	else
+		ret = psp_send_cmd(i2c_req_type);
+
+	if (ret)
 		return -EIO;
 
 	return check_i2c_req_sts(req);
@@ -219,24 +265,27 @@ static int psp_send_check_i2c_req(struct psp_i2c_req *req)
 
 static int psp_send_i2c_req(enum psp_i2c_req_type i2c_req_type)
 {
-	struct psp_i2c_req *req;
+	struct psp_i2c_req *req = NULL;
 	unsigned long start;
 	int status, ret;
 
-	/* Allocate command-response buffer */
-	req = kzalloc(sizeof(*req), GFP_KERNEL);
-	if (!req)
-		return -ENOMEM;
+	/* Allocate command-response buffer for Cezanne platforms */
+	if (cpu_id == AMD_CPU_ID_CZN) {
+		req = kzalloc(sizeof(*req), GFP_KERNEL);
+		if (!req)
+			return -ENOMEM;
 
-	req->hdr.total_size = sizeof(*req);
-	req->type = i2c_req_type;
+		req->hdr.total_size = sizeof(*req);
+		req->type = i2c_req_type;
+	}
 
 	start = jiffies;
 	ret = read_poll_timeout(psp_send_check_i2c_req, status,
 				(status != -EBUSY),
 				PSP_I2C_REQ_RETRY_DELAY_US,
 				PSP_I2C_REQ_RETRY_CNT * PSP_I2C_REQ_RETRY_DELAY_US,
-				0, req);
+				0, req, i2c_req_type);
+
 	if (ret) {
 		dev_err(psp_i2c_dev, "Timed out waiting for PSP to %s I2C bus\n",
 			(i2c_req_type == PSP_I2C_REQ_ACQUIRE) ?
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
#26 lore raw

Re: [PATCH -next 2/2] i2c: designware: Add support for new SoCs in AMDPSP driver

Quoted 8 lines · Jan Dabros

New AMD SoCs are using different algorithm for x86-PSP communication, thus need to modify I2C arbitration driver. Since possible future revisions should have follow new approach, mark functions used only for Cezanne with "czn_" prefix.

While at it, remove redundant check by modifying psp_wait_cmd() to only check for MBOX_READY bit, since MBOX_CMD field being zero is verified by czn_psp_check_mbox_sts() later on in the sequence.

...

drivers/i2c/busses/i2c-designware-amdpsp.c line 36

Quoted 6 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +3 -3
-#define PSP_MBOX_CMD_OFFSET		0x3810570
-#define PSP_MBOX_BUFFER_L_OFFSET	0x3810574
-#define PSP_MBOX_BUFFER_H_OFFSET	0x3810578
+#define CZN_PSP_MBOX_CMD_OFFSET		0x3810570
+#define CZN_PSP_MBOX_BUFFER_L_OFFSET	0x3810574
+#define CZN_PSP_MBOX_BUFFER_H_OFFSET	0x3810578

Can we avoid this renaming noise by putting the proper names in the first place (in the first patch)? Respectively move the corresponding commit message piece to there.

Or looking into the below maybe even split the renaming to a separate non-functional change?

...

drivers/i2c/busses/i2c-designware-amdpsp.c static int psp_send_check_i2c_req(struct psp_i2c_req *req)

Quoted 6 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +5 -0
+	if (req)
+		ret = czn_psp_send_cmd(req);
+	else
+		ret = psp_send_cmd(i2c_req_type);
 
+

Unnecessary blank line.

drivers/i2c/busses/i2c-designware-amdpsp.c static int psp_send_check_i2c_req(struct psp_i2c_req *req) · line 261

Quoted 2 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +1 -0
+	if (ret)
 		return -EIO;

Why you can't return the actual error code here?

...

drivers/i2c/busses/i2c-designware-amdpsp.c static int psp_send_check_i2c_req(struct psp_i2c_req *req) · line 288

Quoted 9 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +2 -1
 	start = jiffies;
 	ret = read_poll_timeout(psp_send_check_i2c_req, status,
 				(status != -EBUSY),
 				PSP_I2C_REQ_RETRY_DELAY_US,
 				PSP_I2C_REQ_RETRY_CNT * PSP_I2C_REQ_RETRY_DELAY_US,
-				0, req);
+				0, req, i2c_req_type);
 
+

Stray blank line addition.

Quoted 3 lines
drivers/i2c/busses/i2c-designware-amdpsp.c +0 -0
 	if (ret) {
 		dev_err(psp_i2c_dev, "Timed out waiting for PSP to %s I2C bus\n",
 			(i2c_req_type == PSP_I2C_REQ_ACQUIRE) ?
Signature · 2 lines
With Best Regards,
Andy Shevchenko
#27 lore raw

PATCH 2/2 i2c: designware: Add support for new SoCs in AMDPSP driver

In reply to Jan Dabros

- /* Status field in command-response buffer is updated by PSP */
- status = READ_ONCE(req->hdr.status);
+ if (req) {
+ /* Status field in command-response buffer is updated by PSP */
+ status = READ_ONCE(req->hdr.status);
+ } else {
+ status = psp_smn_read(PSP_MBOX_CMD_OFFSET, &status);
+ status &= ~PSP_MBOX_FIELDS_READY;
+ }

The value of the mbox cmd register is getting clobbered by the return value from psp_smn_read. This can cause bus arbitration to fail as the driver will think it can grab the bus when it's actually in use by the PSP.

#28 lore raw

Re: [PATCH -next 2/2] i2c: designware: Add support for new SoCs in AMDPSP driver

Quoted 10 lines · Mark Hasemeyer
drivers/i2c/busses/i2c-designware-amdpsp.c +7 -2
-	/* Status field in command-response buffer is updated by PSP */
-	status = READ_ONCE(req->hdr.status);
+	if (req) {
+		/* Status field in command-response buffer is updated by PSP */
+		status = READ_ONCE(req->hdr.status);
+	} else {
+		status = psp_smn_read(PSP_MBOX_CMD_OFFSET, &status);
+		status &= ~PSP_MBOX_FIELDS_READY;
+	}
Quoted 3 lines

The value of the mbox cmd register is getting clobbered by the return value from psp_smn_read. This can cause bus arbitration to fail as the driver will think it can grab the bus when it's actually in use by the PSP.

Mark - FYI this approach is being superseded by another patch series that instead uses MMIO from the CCP driver.

It would be better to comment on that series.

https://lore.kernel.org/linux-i2c/[email protected]/

#29 lore raw
In reply to Jan Dabros
Quoted 7 lines · Jan Dabros

This patchset comprises changes into i2c-designware-amdpsp.c module which aims to add support for new SoCs, while keep backward compatibility with Cezanne platforms.

Beside new algorithm introduced for the PSP-x86 communication, it also switches from MSR/MMIO access to SMN (System Management Network) since only the latter is working on both old new revisions of SoCs.

Which one is better performance/energy/etc wise? If it's not the SMN, can we keep better in place?

Signature · 2 lines
With Best Regards,
Andy Shevchenko