Skip to contentSkip to conversation

iavf: check that state transitions happen under lock

PATCH In review Stefan Assmann 3 participants 5 messages Oct 28, 2022 – Nov 9, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iavf: check that state transitions happen under lock

Add a check to make sure crit_lock is being held during every state transition and print a warning if that's not the case. For convenience a wrapper is added that helps pointing out where the locking is missing.

Make an exception for iavf_probe() as that is too early in the init process and generates a false positive report.

---
 drivers/net/ethernet/intel/iavf/iavf.h      | 23 +++++++++++++++------
 drivers/net/ethernet/intel/iavf/iavf_main.c |  2 +-
 2 files changed, 18 insertions(+), 7 deletions(-)
drivers/net/ethernet/intel/iavf/iavf.h +17 -6
index 3f6187c16424..28f41bbc9c86 100644
@@ -498,19 +498,30 @@ static inline const char *iavf_state_str(enum iavf_state_t state)
 	}
 }
 
-static inline void iavf_change_state(struct iavf_adapter *adapter,
-				     enum iavf_state_t state)
+static inline void __iavf_change_state(struct iavf_adapter *adapter,
+				       enum iavf_state_t state,
+				       const char *func,
+				       int line)
 {
 	if (adapter->state != state) {
 		adapter->last_state = adapter->state;
 		adapter->state = state;
 	}
-	dev_dbg(&adapter->pdev->dev,
-		"state transition from:%s to:%s\n",
-		iavf_state_str(adapter->last_state),
-		iavf_state_str(adapter->state));
+	if (mutex_is_locked(&adapter->crit_lock))
+		dev_dbg(&adapter->pdev->dev, "%s:%d state transition %s to %s\n",
+			func, line,
+			iavf_state_str(adapter->last_state),
+			iavf_state_str(adapter->state));
+	else
+		dev_warn(&adapter->pdev->dev, "%s:%d state transition %s to %s without locking!\n",
+			 func, line,
+			 iavf_state_str(adapter->last_state),
+			 iavf_state_str(adapter->state));
 }
 
+#define iavf_change_state(adapter, state) \
+	__iavf_change_state(adapter, state, __func__, __LINE__)
+
 int iavf_up(struct iavf_adapter *adapter);
 void iavf_down(struct iavf_adapter *adapter);
 int iavf_process_config(struct iavf_adapter *adapter);
drivers/net/ethernet/intel/iavf/iavf_main.c +1 -1
index 3fc572341781..bbc0c9f347a7 100644
@@ -4892,7 +4892,7 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 	hw->back = adapter;
 
 	adapter->msg_enable = BIT(DEFAULT_DEBUG_LEVEL_SHIFT) - 1;
-	iavf_change_state(adapter, __IAVF_STARTUP);
+	adapter->state = __IAVF_STARTUP;
 
 	/* Call save state here because it relies on the adapter struct. */
 	pci_save_state(pdev);
Signature · 1 line
2.37.3
Tested-by on v1 1/1
#1 lore raw

drivers/net/ethernet/intel/iavf/iavf.h

Quoted 30 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Stefan Assmann
Sent: Friday, October 28, 2022 3:45 PM
To: [email protected]
Cc: [email protected]; Piotrowski, Patryk <[email protected]>; [email protected]
Subject: [Intel-wired-lan] [PATCH net-next] iavf: check that state transitions happen under lock

Add a check to make sure crit_lock is being held during every state transition and print a warning if that's not the case. For convenience a wrapper is added that helps pointing out where the locking is missing.

Make an exception for iavf_probe() as that is too early in the init process and generates a false positive report.

---
 drivers/net/ethernet/intel/iavf/iavf.h      | 23 +++++++++++++++------
 drivers/net/ethernet/intel/iavf/iavf_main.c |  2 +-
 2 files changed, 18 insertions(+), 7 deletions(-)
drivers/net/ethernet/intel/iavf/iavf.h +0 -0
index 3f6187c16424..28f41bbc9c86 100644
#2 lore raw
In reply to Stefan Assmann
Quoted 27 lines · Stefan Assmann

Add a check to make sure crit_lock is being held during every state transition and print a warning if that's not the case. For convenience a wrapper is added that helps pointing out where the locking is missing.

Make an exception for iavf_probe() as that is too early in the init process and generates a false positive report.

---
 drivers/net/ethernet/intel/iavf/iavf.h      | 23 +++++++++++++++------
 drivers/net/ethernet/intel/iavf/iavf_main.c |  2 +-
 2 files changed, 18 insertions(+), 7 deletions(-)
drivers/net/ethernet/intel/iavf/iavf.h +4 -2
index 3f6187c16424..28f41bbc9c86 100644
@@ -498,19 +498,30 @@ static inline const char *iavf_state_str(enum iavf_state_t state)
 	}
 }
 
-static inline void iavf_change_state(struct iavf_adapter *adapter,
-				     enum iavf_state_t state)
+static inline void __iavf_change_state(struct iavf_adapter *adapter,
+				       enum iavf_state_t state,
+				       const char *func,
+				       int line)

drivers/net/ethernet/intel/iavf/iavf.h static inline const char *iavf_state_str(enum iavf_state_t state) · line 510

Quoted 10 lines
drivers/net/ethernet/intel/iavf/iavf.h +1 -4
 {
 	if (adapter->state != state) {
 		adapter->last_state = adapter->state;
 		adapter->state = state;
 	}
-	dev_dbg(&adapter->pdev->dev,
-		"state transition from:%s to:%s\n",
-		iavf_state_str(adapter->last_state),
-		iavf_state_str(adapter->state));
+	if (mutex_is_locked(&adapter->crit_lock))

Please use lockdep for that, and not reinvent it. In you case lockdep_assert_held(&adapter->crit_lock).

In addition, mutex_is_locked() doesn't check that this specific function is locked. It checks that this lock is used now.

Quoted 33 lines
drivers/net/ethernet/intel/iavf/iavf.h +12 -0
+		dev_dbg(&adapter->pdev->dev, "%s:%d state transition %s to %s\n",
+			func, line,
+			iavf_state_str(adapter->last_state),
+			iavf_state_str(adapter->state));
+	else
+		dev_warn(&adapter->pdev->dev, "%s:%d state transition %s to %s without locking!\n",
+			 func, line,
+			 iavf_state_str(adapter->last_state),
+			 iavf_state_str(adapter->state));
 }
 
+#define iavf_change_state(adapter, state) \
+	__iavf_change_state(adapter, state, __func__, __LINE__)
+
 int iavf_up(struct iavf_adapter *adapter);
 void iavf_down(struct iavf_adapter *adapter);
 int iavf_process_config(struct iavf_adapter *adapter);
drivers/net/ethernet/intel/iavf/iavf_main.c +1 -1
index 3fc572341781..bbc0c9f347a7 100644
@@ -4892,7 +4892,7 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 	hw->back = adapter;
 
 	adapter->msg_enable = BIT(DEFAULT_DEBUG_LEVEL_SHIFT) - 1;
-	iavf_change_state(adapter, __IAVF_STARTUP);
+	adapter->state = __IAVF_STARTUP;
 
 	/* Call save state here because it relies on the adapter struct. */
 	pci_save_state(pdev);
Signature · 1 line
2.37.3
#3 lore raw
Quoted 74 lines · Leon Romanovsky
Quoted 71 lines · Stefan Assmann

Add a check to make sure crit_lock is being held during every state

transition and print a warning if that's not the case. For convenience

a wrapper is added that helps pointing out where the locking is missing.

Make an exception for iavf_probe() as that is too early in the init

process and generates a false positive report.

---

 drivers/net/ethernet/intel/iavf/iavf.h      | 23 +++++++++++++++------
 drivers/net/ethernet/intel/iavf/iavf_main.c |  2 +-
 2 files changed, 18 insertions(+), 7 deletions(-)
drivers/net/ethernet/intel/iavf/iavf.h +0 -0

index 3f6187c16424..28f41bbc9c86 100644

--- a/drivers/net/ethernet/intel/iavf/iavf.h

+++ b/drivers/net/ethernet/intel/iavf/iavf.h

drivers/net/ethernet/intel/iavf/iavf.h +4 -6
@@ -498,19 +498,30 @@ static inline const char *iavf_state_str(enum iavf_state_t state)
 
 	}
 
 }
 
 
 
-static inline void iavf_change_state(struct iavf_adapter *adapter,
 
-				     enum iavf_state_t state)
 
+static inline void __iavf_change_state(struct iavf_adapter *adapter,
 
+				       enum iavf_state_t state,
 
+				       const char *func,
 
+				       int line)
 
 {
 
 	if (adapter->state != state) {
 
 		adapter->last_state = adapter->state;
 
 		adapter->state = state;
 
 	}
 
-	dev_dbg(&adapter->pdev->dev,
 
-		"state transition from:%s to:%s\n",
 
-		iavf_state_str(adapter->last_state),
 
-		iavf_state_str(adapter->state));
Quoted 7 lines

drivers/net/ethernet/intel/iavf/iavf.h static inline const char *iavf_state_str(enum iavf_state_t state) · line 510

Quoted 1 line
drivers/net/ethernet/intel/iavf/iavf.h +1 -0
+	if (mutex_is_locked(&adapter->crit_lock))

Please use lockdep for that, and not reinvent it.

In you case lockdep_assert_held(&adapter->crit_lock).

Lockdep is mostly enabled in debug kernel but I was hoping to get

warnings in production environments as well. As those transitions don't

happen too often it shouldn't hurt performance.

Quoted 3 lines

In addition, mutex_is_locked() doesn't check that this specific function

is locked. It checks that this lock is used now.

You are correct, this check only triggers if crit_lock is not locked at

all. It would be better to check the lock owner, but I couldn't find an

easy way to do that. Better than no check IMO but we can drop it if you

don't see a benefit in it.

Thanks for the comments!

Stefan

#4 lore raw
Quoted 100 lines · Stefan Assmann
Quoted 74 lines · Leon Romanovsky
Quoted 71 lines · Stefan Assmann

Add a check to make sure crit_lock is being held during every state

transition and print a warning if that's not the case. For convenience

a wrapper is added that helps pointing out where the locking is missing.

Make an exception for iavf_probe() as that is too early in the init

process and generates a false positive report.

---

 drivers/net/ethernet/intel/iavf/iavf.h      | 23 +++++++++++++++------
 drivers/net/ethernet/intel/iavf/iavf_main.c |  2 +-
 2 files changed, 18 insertions(+), 7 deletions(-)
drivers/net/ethernet/intel/iavf/iavf.h +0 -0

index 3f6187c16424..28f41bbc9c86 100644

--- a/drivers/net/ethernet/intel/iavf/iavf.h

+++ b/drivers/net/ethernet/intel/iavf/iavf.h

drivers/net/ethernet/intel/iavf/iavf.h +4 -6
@@ -498,19 +498,30 @@ static inline const char *iavf_state_str(enum iavf_state_t state)
 
 	}
 
 }
 
 
 
-static inline void iavf_change_state(struct iavf_adapter *adapter,
 
-				     enum iavf_state_t state)
 
+static inline void __iavf_change_state(struct iavf_adapter *adapter,
 
+				       enum iavf_state_t state,
 
+				       const char *func,
 
+				       int line)
 
 {
 
 	if (adapter->state != state) {
 
 		adapter->last_state = adapter->state;
 
 		adapter->state = state;
 
 	}
 
-	dev_dbg(&adapter->pdev->dev,
 
-		"state transition from:%s to:%s\n",
 
-		iavf_state_str(adapter->last_state),
 
-		iavf_state_str(adapter->state));
Quoted 7 lines

drivers/net/ethernet/intel/iavf/iavf.h static inline const char *iavf_state_str(enum iavf_state_t state) · line 510

Quoted 1 line
drivers/net/ethernet/intel/iavf/iavf.h +1 -0
+	if (mutex_is_locked(&adapter->crit_lock))

Please use lockdep for that, and not reinvent it.

In you case lockdep_assert_held(&adapter->crit_lock).

Lockdep is mostly enabled in debug kernel but I was hoping to get

warnings in production environments as well. As those transitions don't

happen too often it shouldn't hurt performance.

Quoted 3 lines

In addition, mutex_is_locked() doesn't check that this specific function

is locked. It checks that this lock is used now.

Quoted 7 lines

You are correct, this check only triggers if crit_lock is not locked at

all. It would be better to check the lock owner, but I couldn't find an

easy way to do that. Better than no check IMO but we can drop it if you

don't see a benefit in it.

Yes, please drop it. AFAIK, lockdep doesn't add much overhead while enabled.

Thanks

Quoted 9 lines

Thanks for the comments!

Stefan