Skip to contentSkip to conversation

drivers: base: Introduce a new kernel parameter driver_sync_probe=

PATCH In review Yafang Shao 2 participants 11 messages Dec 6, 2023 – Dec 8, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drivers: base: Introduce a new kernel parameter driver_sync_probe=

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

To address this, a new kernel parameter 'driver_sync_probe=' is introduced to enforce synchronous probe behavior for specific drivers.

For instance, using the following kernel parameter:

driver_sync_probe=sd,nvme

The sd (SCSI) and nvme disks will undergo synchronous probing. This ensures that these disks maintain consistent identification behavior despite the default asynchronous probe, mitigating the issues experienced by applications reliant on fixed disk identification.

---
 .../admin-guide/kernel-parameters.txt         | 10 +++++
 drivers/base/dd.c                             | 41 ++++++++++++++-----
 2 files changed, 41 insertions(+), 10 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +10 -0
index 65731b060e3f..9b1a12b24f65 100644
@@ -1144,6 +1144,16 @@
 			match the *.
 			Format: <driver_name1>,<driver_name2>...
 
+	driver_sync_probe=  [KNL]
+			List of driver names to be probed synchronously. *
+			matches with all driver names. If * is specified, the
+			rest of the listed driver names are those that will NOT
+			match the *.
+			Format: <driver_name1>,<driver_name2>...
+
+			Note that 'driver_sync_probe=' takes precedence over
+			'driver_async_probe=' if both parameters are set.
+
 	drm.edid_firmware=[<connector>:]<file>[,[<connector>:]<file>]
 			Broken monitors, graphic adapters, KVMs and EDIDless
 			panels may send no or incorrect EDID data sets.
drivers/base/dd.c +31 -10

77 lines not on this page · raw message

Signature · 1 line
2.30.1 (Apple Git-130)
maintainer
#1 lore raw
Quoted 7 lines · Yafang Shao

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

Device names are never guaranteed to be stable, ALWAYS use a persistant names like a filesystem label or other ways. Look at /dev/disk/ for the needed ways to do this properly.

Quoted 2 lines

To address this, a new kernel parameter 'driver_sync_probe=' is introduced to enforce synchronous probe behavior for specific drivers.

This should be a per-bus thing, not a driver-specific thing as drivers for the same bus could have differing settings here which would cause a mess.

Please just revert the scsi bus functionality if you have had regressions here, it's not a driver-core thing to do.

thanks,

greg k-h

#2 lore raw
Quoted 10 lines · Greg KH
Quoted 7 lines · Yafang Shao

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

Quoted 3 lines

Device names are never guaranteed to be stable, ALWAYS use a persistant names like a filesystem label or other ways. Look at /dev/disk/ for the needed ways to do this properly.

The root disk is typically identified as /dev/sda or /dev/vda, right? This is because the root disk, which houses the operating system, cannot be removed or hotplugged. Therefore, it usually remains as the first disk in the system. With the synchronous probe, the root disk maintains a stable and consistent identification.

Quoted 10 lines
Quoted 3 lines

To address this, a new kernel parameter 'driver_sync_probe=' is introduced to enforce synchronous probe behavior for specific drivers.

This should be a per-bus thing, not a driver-specific thing as drivers for the same bus could have differing settings here which would cause a mess.

Please just revert the scsi bus functionality if you have had regressions here, it's not a driver-core thing to do.

Are you suggesting a reversal of the asynchronous probe code in the SCSI driver? While reverting to synchronous probing could ensure stability, it's worth noting that asynchronous probing can potentially shorten the reboot duration under specific conditions. Thus, there might be some resistance to reverting this change as it offers performance benefits in certain scenarios. That's why I prefer to introduce a kernel parameter for it.

Signature · 2 lines
Regards
Yafang
maintainer
#3 lore raw
Quoted 11 lines · Yafang Shao
Quoted 10 lines · Greg KH
Quoted 7 lines · Yafang Shao

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

Quoted 5 lines
Quoted 3 lines

Device names are never guaranteed to be stable, ALWAYS use a persistant names like a filesystem label or other ways. Look at /dev/disk/ for the needed ways to do this properly.

The root disk is typically identified as /dev/sda or /dev/vda, right?

Depends on your system. It can also be identified, in the proper way, as /dev/disk/by-uuid/eef0abc1-4039-4c3f-a123-81fc99999993 if you want (note, fake uuid, use your own disk uuid please.)

Why not do that? That's the most stable and recommended way of doing things.

Quoted 2 lines

This is because the root disk, which houses the operating system, cannot be removed or hotplugged.

Not true at all, happens for many systems (think about how systems that run their whole OS out of ram work...)

Quoted 12 lines

Therefore, it usually remains as the
first disk in the system. With the synchronous probe, the root disk maintains a stable and consistent identification.

Quoted 8 lines
Quoted 3 lines

To address this, a new kernel parameter 'driver_sync_probe=' is introduced to enforce synchronous probe behavior for specific drivers.

This should be a per-bus thing, not a driver-specific thing as drivers for the same bus could have differing settings here which would cause a mess.

Quoted 5 lines
Quoted 2 lines

Please just revert the scsi bus functionality if you have had regressions here, it's not a driver-core thing to do.

Are you suggesting a reversal of the asynchronous probe code in the SCSI driver?

For your broken scsi driver, yes.

Quoted 6 lines

While reverting to synchronous probing could ensure stability, it's worth noting that asynchronous probing can potentially shorten the reboot duration under specific conditions. Thus, there might be some resistance to reverting this change as it offers performance benefits in certain scenarios. That's why I prefer to introduce a kernel parameter for it.

I don't want to add a new parameter that we need to support for forever and add to the complexity of the system unless it is REALLY needed. Please work with the scsi developers to resolve the issue for your hardware, as it's been working for everyone else for well over a year now, right?

thanks,

greg k-h

#4 lore raw
Quoted 17 lines · Greg KH
Quoted 15 lines · Yafang Shao
Quoted 13 lines · Greg KH
Quoted 7 lines · Yafang Shao

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

Device names are never guaranteed to be stable, ALWAYS use a persistant names like a filesystem label or other ways. Look at /dev/disk/ for the needed ways to do this properly.

Quoted 8 lines
Quoted 1 line

The root disk is typically identified as /dev/sda or /dev/vda, right?

Depends on your system. It can also be identified, in the proper way, as /dev/disk/by-uuid/eef0abc1-4039-4c3f-a123-81fc99999993 if you want (note, fake uuid, use your own disk uuid please.)

Why not do that? That's the most stable and recommended way of doing things.

Adapting to this change isn't straightforward, especially for a large fleet of servers. Our monitoring system needs to accommodate and adjust accordingly.

Quoted 34 lines
Quoted 3 lines

This is because the root disk, which houses the operating system, cannot be removed or hotplugged.

Not true at all, happens for many systems (think about how systems that run their whole OS out of ram work...)

Quoted 12 lines

Therefore, it usually remains as the
first disk in the system. With the synchronous probe, the root disk maintains a stable and consistent identification.

Quoted 8 lines
Quoted 3 lines

To address this, a new kernel parameter 'driver_sync_probe=' is introduced to enforce synchronous probe behavior for specific drivers.

This should be a per-bus thing, not a driver-specific thing as drivers for the same bus could have differing settings here which would cause a mess.

Quoted 5 lines
Quoted 2 lines

Please just revert the scsi bus functionality if you have had regressions here, it's not a driver-core thing to do.

Are you suggesting a reversal of the asynchronous probe code in the SCSI driver?

For your broken scsi driver, yes.

Quoted 6 lines

While reverting to synchronous probing could ensure stability, it's worth noting that asynchronous probing can potentially shorten the reboot duration under specific conditions. Thus, there might be some resistance to reverting this change as it offers performance benefits in certain scenarios. That's why I prefer to introduce a kernel parameter for it.

Quoted 2 lines

I don't want to add a new parameter that we need to support for forever and add to the complexity of the system unless it is REALLY needed.

BTW, since there's already a 'driver_async_probe=', introducing another 'driver_sync_probe=' wouldn't significantly increase the maintenance overhead.

Quoted 3 lines

Please work with the scsi developers to resolve the issue for your hardware, as it's been working for everyone else for well over a year now, right?

The SCSI guys are added to this mail thread.

I'm uncertain whether it's possible to add SCSI kernel parameters selectively. If that's not feasible, we'll need to maintain the following modification in our local kernel:

drivers/scsi/sd.c +1 -1
index e934779..8148d12 100644
@@ -607,7 +607,7 @@ static void sd_set_flush_flag(struct scsi_disk *sdkp)
                .name           = "sd",
                .owner          = THIS_MODULE,
                .probe          = sd_probe,
-               .probe_type     = PROBE_PREFER_ASYNCHRONOUS,
+               .probe_type     = PROBE_PREFER_SYNCHRONOUS,
                .remove         = sd_remove,
                .shutdown       = sd_shutdown,
                .pm             = &sd_pm_ops,

--
Regards
Yafang

maintainer
#5 lore raw
Quoted 24 lines · Yafang Shao
Quoted 23 lines · Greg KH
Quoted 11 lines · Yafang Shao
Quoted 10 lines · Greg KH
Quoted 7 lines · Yafang Shao

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

Quoted 5 lines
Quoted 3 lines

Device names are never guaranteed to be stable, ALWAYS use a persistant names like a filesystem label or other ways. Look at /dev/disk/ for the needed ways to do this properly.

The root disk is typically identified as /dev/sda or /dev/vda, right?

Depends on your system. It can also be identified, in the proper way, as /dev/disk/by-uuid/eef0abc1-4039-4c3f-a123-81fc99999993 if you want (note, fake uuid, use your own disk uuid please.)

Quoted 6 lines
Quoted 2 lines

Why not do that? That's the most stable and recommended way of doing things.

Adapting to this change isn't straightforward, especially for a large fleet of servers. Our monitoring system needs to accommodate and adjust accordingly.

Agreed, that can be rough. But as this is an issue that was caused by a scsi core change, perhaps the scsi developers can describe why it's ok.

But really, device naming has ALWAYS been known to not be deterministic, which is why Pat and I did all the driver core work 20+ years ago so that you have the ability to properly name your devices in a way that is deterministic. Using the kernel name like sda is NOT using that functionality, so while it has been nice to see that it has been stable for you for a while, you are playing with fire here and will get burned one day when the firmware in your devices decide to change response times.

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

While reverting to synchronous probing could ensure stability, it's worth noting that asynchronous probing can potentially shorten the reboot duration under specific conditions. Thus, there might be some resistance to reverting this change as it offers performance benefits in certain scenarios. That's why I prefer to introduce a kernel parameter for it.

Quoted 6 lines
Quoted 2 lines

I don't want to add a new parameter that we need to support for forever and add to the complexity of the system unless it is REALLY needed.

BTW, since there's already a 'driver_async_probe=', introducing another 'driver_sync_probe=' wouldn't significantly increase the maintenance overhead.

Any new code adds maintenance overhead and complexity, so you have to justify it's existance especially when you are not going to be the one maintaining it :)

thanks,

greg k-h

#6 lore raw
Quoted 36 lines · Greg KH
Quoted 24 lines · Yafang Shao
Quoted 23 lines · Greg KH
Quoted 11 lines · Yafang Shao
Quoted 10 lines · Greg KH
Quoted 7 lines · Yafang Shao

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

Quoted 5 lines
Quoted 3 lines

Device names are never guaranteed to be stable, ALWAYS use a persistant names like a filesystem label or other ways. Look at /dev/disk/ for the needed ways to do this properly.

The root disk is typically identified as /dev/sda or /dev/vda, right?

Depends on your system. It can also be identified, in the proper way, as /dev/disk/by-uuid/eef0abc1-4039-4c3f-a123-81fc99999993 if you want (note, fake uuid, use your own disk uuid please.)

Quoted 6 lines
Quoted 2 lines

Why not do that? That's the most stable and recommended way of doing things.

Adapting to this change isn't straightforward, especially for a large fleet of servers. Our monitoring system needs to accommodate and adjust accordingly.

Agreed, that can be rough. But as this is an issue that was caused by a scsi core change, perhaps the scsi developers can describe why it's ok.

Quoted 8 lines

But really, device naming has ALWAYS been known to not be deterministic, which is why Pat and I did all the driver core work 20+ years ago so that you have the ability to properly name your devices in a way that is deterministic. Using the kernel name like sda is NOT using that functionality, so while it has been nice to see that it has been stable for you for a while, you are playing with fire here and will get burned one day when the firmware in your devices decide to change response times.

I agree that using UUID is a better approach. However, it's worth noting that the widely used IO monitoring tool 'iostat' faces challenges when working with UUIDs. This indicates that there's a significant amount of work ahead of us in this aspect.

Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Quoted 7 lines

While reverting to synchronous probing could ensure stability, it's worth noting that asynchronous probing can potentially shorten the reboot duration under specific conditions. Thus, there might be some resistance to reverting this change as it offers performance benefits in certain scenarios. That's why I prefer to introduce a kernel parameter for it.

I don't want to add a new parameter that we need to support for forever and add to the complexity of the system unless it is REALLY needed.

Quoted 7 lines
Quoted 3 lines

BTW, since there's already a 'driver_async_probe=', introducing another 'driver_sync_probe=' wouldn't significantly increase the maintenance overhead.

Any new code adds maintenance overhead and complexity, so you have to justify it's existance especially when you are not going to be the one maintaining it :)

Understood.

Signature · 2 lines
Regards
Yafang
maintainer
#7 lore raw
Quoted 46 lines · Yafang Shao
Quoted 36 lines · Greg KH
Quoted 24 lines · Yafang Shao
Quoted 23 lines · Greg KH
Quoted 11 lines · Yafang Shao
Quoted 10 lines · Greg KH
Quoted 7 lines · Yafang Shao

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

Quoted 5 lines
Quoted 3 lines

Device names are never guaranteed to be stable, ALWAYS use a persistant names like a filesystem label or other ways. Look at /dev/disk/ for the needed ways to do this properly.

The root disk is typically identified as /dev/sda or /dev/vda, right?

Depends on your system. It can also be identified, in the proper way, as /dev/disk/by-uuid/eef0abc1-4039-4c3f-a123-81fc99999993 if you want (note, fake uuid, use your own disk uuid please.)

Quoted 6 lines
Quoted 2 lines

Why not do that? That's the most stable and recommended way of doing things.

Adapting to this change isn't straightforward, especially for a large fleet of servers. Our monitoring system needs to accommodate and adjust accordingly.

Agreed, that can be rough. But as this is an issue that was caused by a scsi core change, perhaps the scsi developers can describe why it's ok.

Quoted 8 lines

But really, device naming has ALWAYS been known to not be deterministic, which is why Pat and I did all the driver core work 20+ years ago so that you have the ability to properly name your devices in a way that is deterministic. Using the kernel name like sda is NOT using that functionality, so while it has been nice to see that it has been stable for you for a while, you are playing with fire here and will get burned one day when the firmware in your devices decide to change response times.

Quoted 4 lines

I agree that using UUID is a better approach. However, it's worth noting that the widely used IO monitoring tool 'iostat' faces challenges when working with UUIDs. This indicates that there's a significant amount of work ahead of us in this aspect.

That indicates that iostat needs to be fixed as this has been an option that people rely on for 20+ years now. Or use a better tool :)

thanks,

greg k-h

#8 lore raw
Quoted 48 lines · Greg KH
Quoted 46 lines · Yafang Shao
Quoted 44 lines · Greg KH
Quoted 24 lines · Yafang Shao
Quoted 23 lines · Greg KH
Quoted 11 lines · Yafang Shao
Quoted 10 lines · Greg KH
Quoted 7 lines · Yafang Shao

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

Quoted 5 lines
Quoted 3 lines

Device names are never guaranteed to be stable, ALWAYS use a persistant names like a filesystem label or other ways. Look at /dev/disk/ for the needed ways to do this properly.

The root disk is typically identified as /dev/sda or /dev/vda, right?

Depends on your system. It can also be identified, in the proper way, as /dev/disk/by-uuid/eef0abc1-4039-4c3f-a123-81fc99999993 if you want (note, fake uuid, use your own disk uuid please.)

Quoted 6 lines
Quoted 2 lines

Why not do that? That's the most stable and recommended way of doing things.

Adapting to this change isn't straightforward, especially for a large fleet of servers. Our monitoring system needs to accommodate and adjust accordingly.

Agreed, that can be rough. But as this is an issue that was caused by a scsi core change, perhaps the scsi developers can describe why it's ok.

But really, device naming has ALWAYS been known to not be deterministic, which is why Pat and I did all the driver core work 20+ years ago so that you have the ability to properly name your devices in a way that is deterministic. Using the kernel name like sda is NOT using that functionality, so while it has been nice to see that it has been stable for you for a while, you are playing with fire here and will get burned one day when the firmware in your devices decide to change response times.

Quoted 7 lines
Quoted 4 lines

I agree that using UUID is a better approach. However, it's worth noting that the widely used IO monitoring tool 'iostat' faces challenges when working with UUIDs. This indicates that there's a significant amount of work ahead of us in this aspect.

That indicates that iostat needs to be fixed as this has been an option that people rely on for 20+ years now. Or use a better tool :)

The issue arises when a disk contains multiple partitions, such as /dev/sda1 and /dev/sda2. In this case, using 'iostat -j UUID' can only display 'sda' since only its partitions possess UUIDs. Uncertain how to address it yet.

Signature · 2 lines
Regards
Yafang
maintainer
#9 lore raw
Quoted 54 lines · Yafang Shao
Quoted 53 lines · Greg KH
Quoted 50 lines · Yafang Shao
Quoted 36 lines · Greg KH
Quoted 24 lines · Yafang Shao
Quoted 23 lines · Greg KH
Quoted 11 lines · Yafang Shao
Quoted 10 lines · Greg KH
Quoted 7 lines · Yafang Shao

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

Quoted 5 lines
Quoted 3 lines

Device names are never guaranteed to be stable, ALWAYS use a persistant names like a filesystem label or other ways. Look at /dev/disk/ for the needed ways to do this properly.

The root disk is typically identified as /dev/sda or /dev/vda, right?

Depends on your system. It can also be identified, in the proper way, as /dev/disk/by-uuid/eef0abc1-4039-4c3f-a123-81fc99999993 if you want (note, fake uuid, use your own disk uuid please.)

Quoted 6 lines
Quoted 2 lines

Why not do that? That's the most stable and recommended way of doing things.

Adapting to this change isn't straightforward, especially for a large fleet of servers. Our monitoring system needs to accommodate and adjust accordingly.

Agreed, that can be rough. But as this is an issue that was caused by a scsi core change, perhaps the scsi developers can describe why it's ok.

Quoted 8 lines

But really, device naming has ALWAYS been known to not be deterministic, which is why Pat and I did all the driver core work 20+ years ago so that you have the ability to properly name your devices in a way that is deterministic. Using the kernel name like sda is NOT using that functionality, so while it has been nice to see that it has been stable for you for a while, you are playing with fire here and will get burned one day when the firmware in your devices decide to change response times.

I agree that using UUID is a better approach. However, it's worth noting that the widely used IO monitoring tool 'iostat' faces challenges when working with UUIDs. This indicates that there's a significant amount of work ahead of us in this aspect.

Quoted 7 lines
Quoted 2 lines

That indicates that iostat needs to be fixed as this has been an option that people rely on for 20+ years now. Or use a better tool :)

The issue arises when a disk contains multiple partitions, such as /dev/sda1 and /dev/sda2. In this case, using 'iostat -j UUID' can only display 'sda' since only its partitions possess UUIDs. Uncertain how to address it yet.

Then use one of the other many other unique ids that are in /dev/disk/ today. You have loads of things to choose from:

	$ ls /dev/disk/
	by-diskseq  by-id  by-label  by-partlabel  by-partuuid  by-path  by-uuid

You have a plethera of choices here, use whatever works best for your systems. This is a userspace decision to make, not a kernel one, as this is a policy choice of yours.

good luck!

greg k-h

#10 lore raw
Quoted 64 lines · Greg KH
Quoted 54 lines · Yafang Shao
Quoted 53 lines · Greg KH
Quoted 50 lines · Yafang Shao
Quoted 36 lines · Greg KH
Quoted 24 lines · Yafang Shao
Quoted 23 lines · Greg KH
Quoted 11 lines · Yafang Shao
Quoted 10 lines · Greg KH
Quoted 7 lines · Yafang Shao

After upgrading our kernel from version 4.19 to 6.1, certain regressions occurred due to the driver's asynchronous probe behavior. Specifically, the SCSI driver transitioned to an asynchronous probe by default, resulting in a non-fixed root disk behavior. In the prior 4.19 kernel, the root disk was consistently identified as /dev/sda. However, with kernel 6.1, the root disk can be any of /dev/sdX, leading to issues for applications reliant on /dev/sda, notably impacting monitoring systems monitoring the root disk.

Quoted 5 lines
Quoted 3 lines

Device names are never guaranteed to be stable, ALWAYS use a persistant names like a filesystem label or other ways. Look at /dev/disk/ for the needed ways to do this properly.

The root disk is typically identified as /dev/sda or /dev/vda, right?

Depends on your system. It can also be identified, in the proper way, as /dev/disk/by-uuid/eef0abc1-4039-4c3f-a123-81fc99999993 if you want (note, fake uuid, use your own disk uuid please.)

Quoted 6 lines
Quoted 2 lines

Why not do that? That's the most stable and recommended way of doing things.

Adapting to this change isn't straightforward, especially for a large fleet of servers. Our monitoring system needs to accommodate and adjust accordingly.

Agreed, that can be rough. But as this is an issue that was caused by a scsi core change, perhaps the scsi developers can describe why it's ok.

Quoted 8 lines

But really, device naming has ALWAYS been known to not be deterministic, which is why Pat and I did all the driver core work 20+ years ago so that you have the ability to properly name your devices in a way that is deterministic. Using the kernel name like sda is NOT using that functionality, so while it has been nice to see that it has been stable for you for a while, you are playing with fire here and will get burned one day when the firmware in your devices decide to change response times.

I agree that using UUID is a better approach. However, it's worth noting that the widely used IO monitoring tool 'iostat' faces challenges when working with UUIDs. This indicates that there's a significant amount of work ahead of us in this aspect.

Quoted 7 lines
Quoted 2 lines

That indicates that iostat needs to be fixed as this has been an option that people rely on for 20+ years now. Or use a better tool :)

The issue arises when a disk contains multiple partitions, such as /dev/sda1 and /dev/sda2. In this case, using 'iostat -j UUID' can only display 'sda' since only its partitions possess UUIDs. Uncertain how to address it yet.

Quoted 9 lines

Then use one of the other many other unique ids that are in /dev/disk/ today. You have loads of things to choose from:

        $ ls /dev/disk/
        by-diskseq  by-id  by-label  by-partlabel  by-partuuid  by-path  by-uuid

You have a plethera of choices here, use whatever works best for your systems. This is a userspace decision to make, not a kernel one, as this is a policy choice of yours.

Indeed, there are alternative methods besides using UUIDs. This example serves to highlight that UUIDs might not cover all scenarios, similar to other IDs listed under /dev/disk/.

Signature · 2 lines
Regards
Yafang