Skip to contentSkip to conversation

habanalabs: print to kernel log when reset is finished

PATCH Superseded Oded Gabbay 2 participants 7 messages Aug 10, 2019 – Aug 11, 2019 Char and misc drivers

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 habanalabs: print to kernel log when reset is finished

Now that we don't print the queue testing messages, we need to print when the reset is finished so whoever looks at the kernel log will know the reset process was finished successfully and the driver is not stuck.

---
 drivers/misc/habanalabs/device.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/misc/habanalabs/device.c +2 -0
index 9a5926888b99..1fac808c2546 100644
@@ -907,6 +907,8 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
 	else
 		hdev->soft_reset_cnt++;
 
+	dev_info(hdev->dev, "Successfully finished resetting the device\n");
+
 	return 0;
 
 out_err:
Signature · 1 line
2.17.1
maintainer
#1 lore raw
Quoted 14 lines · Oded Gabbay

Now that we don't print the queue testing messages, we need to print when the reset is finished so whoever looks at the kernel log will know the reset process was finished successfully and the driver is not stuck.

---
 drivers/misc/habanalabs/device.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/misc/habanalabs/device.c +0 -0
index 9a5926888b99..1fac808c2546 100644

drivers/misc/habanalabs/device.c int hl_device_reset(struct hl_device *hdev, bool hard_reset, · line 910

Quoted 4 lines
drivers/misc/habanalabs/device.c +1 -0
@@ -907,6 +907,8 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
 	else
 		hdev->soft_reset_cnt++;
 
+	dev_info(hdev->dev, "Successfully finished resetting the device\n");

Really? For doing things "properly" there is no need to spam the kernel log. Only spit stuff out if an error happens.

thanks,

greg k-h

#2 lore raw
Quoted 19 lines · Greg KH
Quoted 17 lines · Oded Gabbay

Now that we don't print the queue testing messages, we need to print when the reset is finished so whoever looks at the kernel log will know the reset process was finished successfully and the driver is not stuck.

---
 drivers/misc/habanalabs/device.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/misc/habanalabs/device.c +0 -0
index 9a5926888b99..1fac808c2546 100644
@@ -907,6 +907,8 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
      else
              hdev->soft_reset_cnt++;
 
Quoted 8 lines

drivers/misc/habanalabs/device.c int hl_device_reset(struct hl_device *hdev, bool hard_reset, · line 910

Quoted 1 line
drivers/misc/habanalabs/device.c +1 -0
+     dev_info(hdev->dev, "Successfully finished resetting the device\n");

Really? For doing things "properly" there is no need to spam the kernel log. Only spit stuff out if an error happens.

thanks,

greg k-h

I beg to differ for two reasons:
1. Reset happens very rarely, if at all. So this message (that get printed after reset is done) will definitely not spam the kernel log.
2. When a reset starts we print an appropriate error message. I think it is expected by the user that we will also print if and when the reset has finished successfully. I really believe that lack of this printing might be deceiving for users.

Thanks,
Oded

maintainer
#3 lore raw
Quoted 29 lines · Oded Gabbay
Quoted 19 lines · Greg KH
Quoted 17 lines · Oded Gabbay

Now that we don't print the queue testing messages, we need to print when the reset is finished so whoever looks at the kernel log will know the reset process was finished successfully and the driver is not stuck.

---
 drivers/misc/habanalabs/device.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/misc/habanalabs/device.c +0 -0
index 9a5926888b99..1fac808c2546 100644
@@ -907,6 +907,8 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
      else
              hdev->soft_reset_cnt++;
 
Quoted 8 lines

drivers/misc/habanalabs/device.c int hl_device_reset(struct hl_device *hdev, bool hard_reset, · line 910

Quoted 1 line
drivers/misc/habanalabs/device.c +1 -0
+     dev_info(hdev->dev, "Successfully finished resetting the device\n");

Really? For doing things "properly" there is no need to spam the kernel log. Only spit stuff out if an error happens.

thanks,

greg k-h

Quoted 7 lines

I beg to differ for two reasons:
1. Reset happens very rarely, if at all. So this message (that get printed after reset is done) will definitely not spam the kernel log.
2. When a reset starts we print an appropriate error message. I think it is expected by the user that we will also print if and when the reset has finished successfully. I really believe that lack of this printing might be deceiving for users.

How is anyone going to parse the kernel log for anything to know if something happens?

How do you trigger a reset? Is it done by userspace? If so, just notify them then.

thanks,

greg k-h

#4 lore raw
Quoted 39 lines · Greg KH
Quoted 29 lines · Oded Gabbay
Quoted 19 lines · Greg KH
Quoted 17 lines · Oded Gabbay

Now that we don't print the queue testing messages, we need to print when the reset is finished so whoever looks at the kernel log will know the reset process was finished successfully and the driver is not stuck.

---
 drivers/misc/habanalabs/device.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/misc/habanalabs/device.c +0 -0
index 9a5926888b99..1fac808c2546 100644
@@ -907,6 +907,8 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
      else
              hdev->soft_reset_cnt++;
 
Quoted 8 lines

drivers/misc/habanalabs/device.c int hl_device_reset(struct hl_device *hdev, bool hard_reset, · line 910

Quoted 1 line
drivers/misc/habanalabs/device.c +1 -0
+     dev_info(hdev->dev, "Successfully finished resetting the device\n");

Really? For doing things "properly" there is no need to spam the kernel log. Only spit stuff out if an error happens.

thanks,

greg k-h

Quoted 7 lines

I beg to differ for two reasons:
1. Reset happens very rarely, if at all. So this message (that get printed after reset is done) will definitely not spam the kernel log.
2. When a reset starts we print an appropriate error message. I think it is expected by the user that we will also print if and when the reset has finished successfully. I really believe that lack of this printing might be deceiving for users.

Quoted 2 lines

How is anyone going to parse the kernel log for anything to know if something happens?

Actually our jenkins server parse the kernel log and look for bad messages from the driver...
But the main reason for this, IMO, is for developers (in userspace) that run with a terminal open on the kernel log to see if something bad happens while they run their applications.

Quoted 3 lines

How do you trigger a reset? Is it done by userspace? If so, just notify them then.

Reset can be triggered by two main reasons, but a userspace application is *not* one of them.
It can be due to a command submission not finishing in time or from a major error in the device (e.g. double ECC error).
That's why I think it is important to tell them the reset + re-initialization flow was finished successfully.
Of course all this information, let's call it operational status, can be acquired by the INFO IOCTL the driver provides, but it is *really* convenient, IMHO, to see these type of messages in the kernel log while they happen.

Quoted 4 lines

thanks,

greg k-h

maintainer
#5 lore raw
Quoted 53 lines · Oded Gabbay
Quoted 39 lines · Greg KH
Quoted 29 lines · Oded Gabbay
Quoted 19 lines · Greg KH
Quoted 17 lines · Oded Gabbay

Now that we don't print the queue testing messages, we need to print when the reset is finished so whoever looks at the kernel log will know the reset process was finished successfully and the driver is not stuck.

---
 drivers/misc/habanalabs/device.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/misc/habanalabs/device.c +0 -0
index 9a5926888b99..1fac808c2546 100644
@@ -907,6 +907,8 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
      else
              hdev->soft_reset_cnt++;
 
Quoted 8 lines

drivers/misc/habanalabs/device.c int hl_device_reset(struct hl_device *hdev, bool hard_reset, · line 910

Quoted 1 line
drivers/misc/habanalabs/device.c +1 -0
+     dev_info(hdev->dev, "Successfully finished resetting the device\n");

Really? For doing things "properly" there is no need to spam the kernel log. Only spit stuff out if an error happens.

thanks,

greg k-h

Quoted 7 lines

I beg to differ for two reasons:
1. Reset happens very rarely, if at all. So this message (that get printed after reset is done) will definitely not spam the kernel log.
2. When a reset starts we print an appropriate error message. I think it is expected by the user that we will also print if and when the reset has finished successfully. I really believe that lack of this printing might be deceiving for users.

Quoted 2 lines

How is anyone going to parse the kernel log for anything to know if something happens?

Actually our jenkins server parse the kernel log and look for bad messages from the driver...
But the main reason for this, IMO, is for developers (in userspace) that run with a terminal open on the kernel log to see if something bad happens while they run their applications.

Quoted 3 lines

How do you trigger a reset? Is it done by userspace? If so, just notify them then.

Quoted 10 lines

Reset can be triggered by two main reasons, but a userspace application is *not* one of them.
It can be due to a command submission not finishing in time or from a major error in the device (e.g. double ECC error).
That's why I think it is important to tell them the reset + re-initialization flow was finished successfully.
Of course all this information, let's call it operational status, can be acquired by the INFO IOCTL the driver provides, but it is *really* convenient, IMHO, to see these type of messages in the kernel log while they happen.

Ah, yes, if this is something that userspace can not trigger then it's ok. But you should then make it a 'dev_warn' to match up with the message level when the reset starts.

thanks,

greg k-h

#6 lore raw
Quoted 66 lines · Greg KH
Quoted 53 lines · Oded Gabbay
Quoted 39 lines · Greg KH
Quoted 29 lines · Oded Gabbay
Quoted 19 lines · Greg KH
Quoted 17 lines · Oded Gabbay

Now that we don't print the queue testing messages, we need to print when the reset is finished so whoever looks at the kernel log will know the reset process was finished successfully and the driver is not stuck.

---
 drivers/misc/habanalabs/device.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/misc/habanalabs/device.c +0 -0
index 9a5926888b99..1fac808c2546 100644
@@ -907,6 +907,8 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
      else
              hdev->soft_reset_cnt++;
 
Quoted 8 lines

drivers/misc/habanalabs/device.c int hl_device_reset(struct hl_device *hdev, bool hard_reset, · line 910

Quoted 1 line
drivers/misc/habanalabs/device.c +1 -0
+     dev_info(hdev->dev, "Successfully finished resetting the device\n");

Really? For doing things "properly" there is no need to spam the kernel log. Only spit stuff out if an error happens.

thanks,

greg k-h

Quoted 7 lines

I beg to differ for two reasons:
1. Reset happens very rarely, if at all. So this message (that get printed after reset is done) will definitely not spam the kernel log.
2. When a reset starts we print an appropriate error message. I think it is expected by the user that we will also print if and when the reset has finished successfully. I really believe that lack of this printing might be deceiving for users.

Quoted 2 lines

How is anyone going to parse the kernel log for anything to know if something happens?

Actually our jenkins server parse the kernel log and look for bad messages from the driver...
But the main reason for this, IMO, is for developers (in userspace) that run with a terminal open on the kernel log to see if something bad happens while they run their applications.

Quoted 3 lines

How do you trigger a reset? Is it done by userspace? If so, just notify them then.

Quoted 10 lines

Reset can be triggered by two main reasons, but a userspace application is *not* one of them.
It can be due to a command submission not finishing in time or from a major error in the device (e.g. double ECC error).
That's why I think it is important to tell them the reset + re-initialization flow was finished successfully.
Of course all this information, let's call it operational status, can be acquired by the INFO IOCTL the driver provides, but it is *really* convenient, IMHO, to see these type of messages in the kernel log while they happen.

Quoted 7 lines

Ah, yes, if this is something that userspace can not trigger then it's ok. But you should then make it a 'dev_warn' to match up with the message level when the reset starts.

thanks,

greg k-h

Sure, np.
Thanks,
Oded