Query: Best way to know if a watchdog is active (kicked)
Hi,
I am looking for the best way to know if a watchdog has been kicked and active.
I can see a way is to read timeout(WDIOC_GETTIMEOUT) and timeleft( WDIOC_GETTIMELEFT). If they do not match, it means that wdt is active.
But what if we tried to read timeleft just in time when watchdog daemon/or some other application had kicked it. May be we read timeleft twice at the interval of 1 sec.
Please let me know if there is any other alternative which could be a better way to know if watchdog is active? Or may be it would be good to implement an ioctl WDIOC_ACTIVE?
Thanks
~Pratyush
Guenter Roeck
maintainerQuoted 7 lines · Pratyush Anand
Hi,
I am looking for the best way to know if a watchdog has been kicked and active.
I can see a way is to read
timeout(WDIOC_GETTIMEOUT)andtimeleft( WDIOC_GETTIMELEFT). If they do not match, it means that wdt is active.
Quoted 8 lines
But what if we tried to read timeleft just in time when watchdog daemon/or some other application had kicked it. May be we read timeleft twice at the interval of 1 sec.
Please let me know if there is any other alternative which could be a better way to know if watchdog is active? Or may be it would be good to implement an ioctl
WDIOC_ACTIVE?
Normally the watchdog is active if the watchdog device is open, unless the application controlling it explicitly disabled it with WDIOC_SETOPTIONS. Therefore, the controlling application should always know the status. A different application can not open the watchdog device, so it won't be able to get its status using an ioctl anyway.
Why is that insufficient ?
Guenter
Hi Guenter,
Thanks a lot for your quick reply.
Quoted 17 lines · Guenter Roeck
Quoted 7 lines · Pratyush Anand
Hi,
I am looking for the best way to know if a watchdog has been kicked and active.
I can see a way is to read
timeout(WDIOC_GETTIMEOUT)andtimeleft( WDIOC_GETTIMELEFT). If they do not match, it means that wdt is active.Quoted 8 lines
But what if we tried to read timeleft just in time when watchdog daemon/or some other application had kicked it. May be we read timeleft twice at the interval of 1 sec.
Please let me know if there is any other alternative which could be a better way to know if watchdog is active? Or may be it would be good to implement an ioctl
WDIOC_ACTIVE?
Quoted 5 lines
Normally the watchdog is active if the watchdog device is open, unless the application controlling it explicitly disabled it with
WDIOC_SETOPTIONS. Therefore, the controlling application should always know the status. A different application can not open the watchdog device, so it won't be able to get its status using an ioctl anyway.
Yes, A different application can not open in parallel, but can open once the previous application has closed it. For example this is what I see:
--------------------------------------------------------------
# cat /dev/watchdog1 ; sleep 5; wdctl /dev/watchdog1 cat: /dev/watchdog1: Invalid argument wdctl: write failed: Invalid argument Device: /dev/watchdog1 Identity: iTCO_wdt [version 0] Timeout: 30 seconds Timeleft: 24 seconds FLAG DESCRIPTION STATUS BOOT-STATUS KEEPALIVEPING Keep alive ping reply 0 0 MAGICCLOSE Supports magic close char 0 0 SETTIMEOUT Set timeout (in seconds) 0 0 -------------------------------------------------------------- So, cat opened it and kicked it as well. But, it could not stop it as magic character "V" had not not received. Therefore, when wdctl opened and read Timeleft, it was different than Timeout.
Quoted 2 lines
Why is that insufficient ?
Well, let me explain the use case. Consider the situation when:
-- A system has activated its watchdog to take care of software hang. So, when software has hanged, wdt causes to reboot, else it is kicked again before timeout.
-- The same system has also activated kdump(kdump is a method to reboot to a minimal stable secondary kernel in case of kernel crash). Now when wdt was still active, there was a kernel crash and system booted to a secondary stable kernel which copies crash related data to a safe location. Since, wdt was active so before the desired process could complete in secondary kernel, hardware rebooted. -- So, the watchdog device need to be stoped in secondary kernel as early as possible. Loading of driver/module itself stops a kicked device. So, if there could be a way to know active wdt from kernel, then the two daemon (one which manages watchdog and other which manages kdump) can play independently, and kdump daemon can correctly program a kdump file system to load relevant watchdog module as early as possible.
-- Current distro implementations loads all the watchdog devices driver module in secondary kernel, which is not nice (secondary kdump kernel should be as minimal as possible).
~Pratyush
Hi, Pratyush
A comment from my understanding about the background..
Quoted 51 lines · Pratyush Anand
Hi Guenter,
Thanks a lot for your quick reply.
Quoted 17 lines · Guenter Roeck
Quoted 7 lines · Pratyush Anand
Hi,
I am looking for the best way to know if a watchdog has been kicked and active.
I can see a way is to read
timeout(WDIOC_GETTIMEOUT)andtimeleft( WDIOC_GETTIMELEFT). If they do not match, it means that wdt is active.Quoted 8 lines
But what if we tried to read timeleft just in time when watchdog daemon/or some other application had kicked it. May be we read timeleft twice at the interval of 1 sec.
Please let me know if there is any other alternative which could be a better way to know if watchdog is active? Or may be it would be good to implement an ioctl
WDIOC_ACTIVE?Quoted 5 lines
Normally the watchdog is active if the watchdog device is open, unless the application controlling it explicitly disabled it with
WDIOC_SETOPTIONS. Therefore, the controlling application should always know the status. A different application can not open the watchdog device, so it won't be able to get its status using an ioctl anyway.Yes, A different application can not open in parallel, but can open once the previous application has closed it. For example this is what I see:
--------------------------------------------------------------
# cat /dev/watchdog1 ; sleep 5; wdctl /dev/watchdog1 cat: /dev/watchdog1: Invalid argument wdctl: write failed: Invalid argument Device: /dev/watchdog1 Identity: iTCO_wdt [version 0] Timeout: 30 seconds Timeleft: 24 seconds FLAG DESCRIPTION STATUS BOOT-STATUS KEEPALIVEPING Keep alive ping reply 0 0 MAGICCLOSE Supports magic close char 0 0 SETTIMEOUT Set timeout (in seconds) 0 0 -------------------------------------------------------------- So, cat opened it and kicked it as well. But, it could not stop it as magic character "V" had not not received. Therefore, when wdctl opened and read Timeleft, it was different than Timeout.Quoted 2 lines
Why is that insufficient ?
Quoted 10 lines
Well, let me explain the use case. Consider the situation when:
-- A system has activated its watchdog to take care of software hang. So, when software has hanged, wdt causes to reboot, else it is kicked again before timeout.
-- The same system has also activated kdump(kdump is a method to reboot to a minimal stable secondary kernel in case of kernel crash). Now when wdt was still active, there was a kernel crash and system booted to a secondary stable kernel which copies crash related data to a safe location. Since, wdt was active so before the desired process could complete in secondary kernel, hardware rebooted. -- So, the watchdog device need to be stoped in secondary kernel as early as
Either stop it or continue kicking before timeout are fine.
Quoted 5 lines
possible. Loading of driver/module itself stops a kicked device. So, if there could be a way to know active wdt from kernel, then the two daemon (one which manages watchdog and other which manages kdump) can play independently, and kdump daemon can correctly program a kdump file system to load relevant watchdog module as early as possible.
Some drivers like iTCO_wdt can stop it during module loading. But I'm not sure all drivers work. At least under 'nowayout' mode.
So the better way (still is a best effort solution though) should be kicking it again before timeout.
Quoted 5 lines
-- Current distro implementations loads all the watchdog devices driver module in secondary kernel, which is not nice (secondary kdump kernel should be as minimal as possible).
~Pratyush
Thanks
Dave
Hi Dave,
Quoted 3 lines · Dave Young
Hi, Pratyush
A comment from my understanding about the background..
I agree with your understanding.
Thanks a lot for clarifying it more.
~Pratyush
Guenter Roeck
maintainerQuoted 22 lines · Pratyush Anand
Hi Guenter,
Thanks a lot for your quick reply.
Quoted 17 lines · Guenter Roeck
Quoted 15 lines · Pratyush Anand
Hi,
I am looking for the best way to know if a watchdog has been kicked and active.
I can see a way is to read
timeout(WDIOC_GETTIMEOUT)andtimeleft( WDIOC_GETTIMELEFT). If they do not match, it means that wdt is active.But what if we tried to read timeleft just in time when watchdog daemon/or some other application had kicked it. May be we read timeleft twice at the interval of 1 sec.
Please let me know if there is any other alternative which could be a better way to know if watchdog is active? Or may be it would be good to implement an ioctl
WDIOC_ACTIVE?
Quoted 9 lines
Quoted 5 lines
Normally the watchdog is active if the watchdog device is open, unless the application controlling it explicitly disabled it with
WDIOC_SETOPTIONS. Therefore, the controlling application should always know the status. A different application can not open the watchdog device, so it won't be able to get its status using an ioctl anyway.Yes, A different application can not open in parallel, but can open once the previous application has closed it. For example this is what I see:
Just by opening the watchdog it is going to be activated. So all you end up knowing is if the watchdog was active before, but in either case it will now be active. That doesn't seem to be very helpful.
Quoted 30 lines
--------------------------------------------------------------
# cat /dev/watchdog1 ; sleep 5; wdctl /dev/watchdog1 cat: /dev/watchdog1: Invalid argument wdctl: write failed: Invalid argument Device: /dev/watchdog1 Identity: iTCO_wdt [version 0] Timeout: 30 seconds Timeleft: 24 seconds FLAG DESCRIPTION STATUS BOOT-STATUS KEEPALIVEPING Keep alive ping reply 0 0 MAGICCLOSE Supports magic close char 0 0 SETTIMEOUT Set timeout (in seconds) 0 0 -------------------------------------------------------------- So, cat opened it and kicked it as well. But, it could not stop it as magic character "V" had not not received. Therefore, when wdctl opened and read Timeleft, it was different than Timeout.Quoted 2 lines
Why is that insufficient ?
Well, let me explain the use case. Consider the situation when:
-- A system has activated its watchdog to take care of software hang. So, when software has hanged, wdt causes to reboot, else it is kicked again before timeout.
-- The same system has also activated kdump(kdump is a method to reboot to a minimal stable secondary kernel in case of kernel crash). Now when wdt was still active, there was a kernel crash and system booted to a secondary stable kernel which copies crash related data to a safe location. Since, wdt was active so before the desired process could complete in secondary kernel, hardware rebooted. -- So, the watchdog device need to be stoped in secondary kernel as early as
Quoted 8 lines
possible. Loading of driver/module itself stops a kicked device. So, if there could be a way to know active wdt from kernel, then the two daemon (one which manages watchdog and other which manages kdump) can play independently, and kdump daemon can correctly program a kdump file system to load relevant watchdog module as early as possible.
-- Current distro implementations loads all the watchdog devices driver module in secondary kernel, which is not nice (secondary kdump kernel should be as minimal as possible).
If the watchdog was active in the original kernel, it needs to be activated in the crashdump kernel. I don't see a way around that. I can not comment on "loads all the watchdog device driver module in the secondary kernel". I would think it should be known which module to load.
One possible solution to your problem might be to have some sysfs attributes associated with watchdog devices, one of which would be its state. That has been on my mental task list for a while, but unfortunately I never found the time to implement it.
Guenter
Quoted 1 line · Guenter Roeck
On 08/17/2015 11:57 PM, Pratyush Anand wrote:
[...]
Quoted 5 lines
One possible solution to your problem might be to have some sysfs attributes associated with watchdog devices, one of which would be its state. That has been on my mental task list for a while, but unfortunately I never found the time to implement it.
Well, I will give an attempt to implement state as a sysfs attributes associated with watchdog devices, having two values ACTIVE and INACTIVE. Will send the modification for review.
We can always add latter on more attributes and more state values.
Thanks a lot for your suggestion.
~Pratyush
Guenter Roeck
maintainerQuoted 11 lines · Pratyush Anand
Quoted 1 line · Guenter Roeck
On 08/17/2015 11:57 PM, Pratyush Anand wrote:
[...]
Quoted 5 lines
One possible solution to your problem might be to have some sysfs attributes associated with watchdog devices, one of which would be its state. That has been on my mental task list for a while, but unfortunately I never found the time to implement it.
Quoted 5 lines
Well, I will give an attempt to implement state as a sysfs attributes associated with watchdog devices, having two values ACTIVE and INACTIVE. Will send the modification for review.
We can always add latter on more attributes and more state values.
Sounds good. We should have a 'name' attribute as well. We'll also need 'timeout', 'keepalive' to ping the watchdog, 'timeleft', and 'bootstatus'. Not sure what else.
The attributes should be implemented as class attributes, to ensure that they are created and removed automatically. That may require the watchdog class to be static instead of a pointer.
Thanks,
Guenter
Quoted 7 lines · Guenter Roeck
Sounds good. We should have a 'name' attribute as well. We'll also need 'timeout', 'keepalive' to ping the watchdog, 'timeleft', and 'bootstatus'. Not sure what else.
The attributes should be implemented as class attributes, to ensure that they are created and removed automatically. That may require the watchdog class to be static instead of a pointer.
Should n't attributes be implemented as device attribute as they are properties of each device. Under the class watchdog_class, we can have multiple device on a system, and these attributes would be different for different devices. So, what I am thinking is to create device in __watchdog_register_device using device_create_with_groups instead of device_create. Here we can link a attribute_group with all attributes you suggested.
~Pratyush
Guenter Roeck
maintainerQuoted 9 lines · Pratyush Anand
Quoted 7 lines · Guenter Roeck
Sounds good. We should have a 'name' attribute as well. We'll also need 'timeout', 'keepalive' to ping the watchdog, 'timeleft', and 'bootstatus'. Not sure what else.
The attributes should be implemented as class attributes, to ensure that they are created and removed automatically. That may require the watchdog class to be static instead of a pointer.
Quoted 7 lines
Should n't attributes be implemented as device attribute as they are properties of each device. Under the class
watchdog_class, we can have multiple device on a system, and these attributes would be different for different devices. So, what I am thinking is to create device in__watchdog_register_deviceusingdevice_create_with_groupsinstead ofdevice_create. Here we can link aattribute_groupwith all attributes you suggested.
Class attibutes are created automatically for each device registered which is a member of the class. They are attached to the device, not to the class. In other words, the infrastructure creates the attributes, the driver doesn't have to do it, and there is no race condition where the device exists but the attributes are not yet there.
Not sure what you mean with "different". Can you elaborate ?
Guenter
Quoted 17 lines · Guenter Roeck
Quoted 9 lines · Pratyush Anand
Quoted 7 lines · Guenter Roeck
Sounds good. We should have a 'name' attribute as well. We'll also need 'timeout', 'keepalive' to ping the watchdog, 'timeleft', and 'bootstatus'. Not sure what else.
The attributes should be implemented as class attributes, to ensure that they are created and removed automatically. That may require the watchdog class to be static instead of a pointer.
Quoted 7 lines
Should n't attributes be implemented as device attribute as they are properties of each device. Under the class
watchdog_class, we can have multiple device on a system, and these attributes would be different for different devices. So, what I am thinking is to create device in__watchdog_register_deviceusingdevice_create_with_groupsinstead ofdevice_create. Here we can link aattribute_groupwith all attributes you suggested.
Quoted 2 lines
Class attibutes are created automatically for each device registered which is a member of the class. They are attached to the device, not to the class.
Not sure, if we both are referring to same thing..
So, I did a basic testing. I changed "static struct class *watchdog_class;" to "tatic struct class watchdog_class" and other than .name and .owner I added .class_attrs and then further added one node "state" in this class_attrs.
I have two watchdog in my system, so I was expecting /sys/class/watchdog/watchdog0/state and /sys/class/watchdog/watchdog1/state. But I see that there is only /sys/class/watchdog/state with the above approach.
Quoted 3 lines
In other words, the infrastructure creates the attributes, the driver doesn't have to do it, and there is no race condition where the device exists but the attributes are not yet there.
Yes, I agree that individual driver does not need to do anything. watchdog_core.c and watchdog_dev.c will take care for each of the device registered with it. I am able to do that, but not using .class_attrs, rather using device_create_with_groups.
Quoted 2 lines
Not sure what you mean with "different". Can you elaborate ?
As I said above, with the "different" I meant that we should have state in each /sys/class/watchdog/watchdogn.
~Pratyush
Guenter Roeck
maintainerQuoted 18 lines · Pratyush Anand
Quoted 17 lines · Guenter Roeck
Quoted 16 lines · Pratyush Anand
Quoted 7 lines · Guenter Roeck
Sounds good. We should have a 'name' attribute as well. We'll also need 'timeout', 'keepalive' to ping the watchdog, 'timeleft', and 'bootstatus'. Not sure what else.
The attributes should be implemented as class attributes, to ensure that they are created and removed automatically. That may require the watchdog class to be static instead of a pointer.
Should n't attributes be implemented as device attribute as they are properties of each device. Under the class
watchdog_class, we can have multiple device on a system, and these attributes would be different for different devices. So, what I am thinking is to create device in__watchdog_register_deviceusingdevice_create_with_groupsinstead ofdevice_create. Here we can link aattribute_groupwith all attributes you suggested.
Quoted 8 lines
Quoted 2 lines
Class attibutes are created automatically for each device registered which is a member of the class. They are attached to the device, not to the class.
Not sure, if we both are referring to same thing..
So, I did a basic testing. I changed "static struct class*watchdog_class;" to "tatic struct classwatchdog_class" and other than .name and .owner I added .class_attrs and then further added one node "state" in thisclass_attrs.
If you use .dev_groups instead of .class_attrs it should work.
Guenter
Quoted 22 lines · Guenter Roeck
Quoted 21 lines · Pratyush Anand
Quoted 19 lines · Guenter Roeck
Quoted 9 lines · Pratyush Anand
Quoted 7 lines · Guenter Roeck
Sounds good. We should have a 'name' attribute as well. We'll also need 'timeout', 'keepalive' to ping the watchdog, 'timeleft', and 'bootstatus'. Not sure what else.
The attributes should be implemented as class attributes, to ensure that they are created and removed automatically. That may require the watchdog class to be static instead of a pointer.
Quoted 7 lines
Should n't attributes be implemented as device attribute as they are properties of each device. Under the class
watchdog_class, we can have multiple device on a system, and these attributes would be different for different devices. So, what I am thinking is to create device in__watchdog_register_deviceusingdevice_create_with_groupsinstead ofdevice_create. Here we can link aattribute_groupwith all attributes you suggested.Class attibutes are created automatically for each device registered which is a member of the class. They are attached to the device, not to the class.
Quoted 6 lines
Quoted 5 lines
Not sure, if we both are referring to same thing..
So, I did a basic testing. I changed "static struct class*watchdog_class;" to "tatic struct classwatchdog_class" and other than .name and .owner I added .class_attrs and then further added one node "state" in thisclass_attrs.If you use .dev_groups instead of .class_attrs it should work.
OK.. Got it..Thanks for this info. I see another issue with that. I will have to make watchdog_class as global. Let me send my RFC patch, which will be very soon and then may be we can see if some improvement can be done further.
~Pratyush