Skip to contentSkip to conversation

enclosure: add support for enclosure services

PATCH In review James Bottomley 7 participants 30 messages Feb 3, 2008 – Feb 16, 2008

Subsystems (3)
lore
Share & tools
lore

James Bottomley

author maintainer
#0 lore raw

PATCH 1/1 enclosure: add support for enclosure services

The enclosure misc device is really just a library providing sysfs support for physical enclosure devices and their components.

---

See the additional ses patch for SCSI enclosure services users of this.

---
 drivers/misc/Kconfig      |   10 +
 drivers/misc/Makefile     |    1 +
 drivers/misc/enclosure.c  |  449 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/enclosure.h |  120 ++++++++++++
 4 files changed, 580 insertions(+), 0 deletions(-)
 create mode 100644 drivers/misc/enclosure.c
 create mode 100644 include/linux/enclosure.h
drivers/misc/Kconfig +10 -0
index b5e67c0..c6e5c09 100644
@@ -11,6 +11,7 @@ menuconfig MISC_DEVICES
 
 	  If you say N, all options in this submenu will be skipped and disabled.
 
+
 if MISC_DEVICES
 
 config IBM_ASM
@@ -232,4 +233,13 @@ config ATMEL_SSC
 
 	  If unsure, say N.
 
+config ENCLOSURE_SERVICES
+	tristate "Enclosure Services"
+	default n
+	help
+	  Provides support for intelligent enclosures (bays which
+	  contain storage devices).  You also need either a host
+	  driver (SCSI/ATA) which supports enclosures
+	  or a SCSI enclosure device (SES) to use these services.
+
 endif # MISC_DEVICES
drivers/misc/Makefile +1 -0
index 87f2685..de9f1f5 100644
@@ -17,3 +17,4 @@ obj-$(CONFIG_SONY_LAPTOP)	+= sony-laptop.o
 obj-$(CONFIG_THINKPAD_ACPI)	+= thinkpad_acpi.o
 obj-$(CONFIG_FUJITSU_LAPTOP)	+= fujitsu-laptop.o
 obj-$(CONFIG_EEPROM_93CX6)	+= eeprom_93cx6.o
+obj-$(CONFIG_ENCLOSURE_SERVICES) += enclosure.o
\ No newline at end of file
drivers/misc/enclosure.c new file +449 -0

449 lines not on this page · raw message

include/linux/enclosure.h new file +120 -0

120 lines not on this page · raw message

Signature · 1 line
1.5.3.8
#1 lore raw

Hi James.

Nitpicking only.

Sam

Quoted 22 lines

The enclosure misc device is really just a library providing sysfs support for physical enclosure devices and their components.

---

See the additional ses patch for SCSI enclosure services users of this.

---
 drivers/misc/Kconfig      |   10 +
 drivers/misc/Makefile     |    1 +
 drivers/misc/enclosure.c  |  449 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/enclosure.h |  120 ++++++++++++
 4 files changed, 580 insertions(+), 0 deletions(-)
 create mode 100644 drivers/misc/enclosure.c
 create mode 100644 include/linux/enclosure.h
drivers/misc/Kconfig +0 -0
index b5e67c0..c6e5c09 100644

drivers/misc/Kconfig menuconfig MISC_DEVICES · line 15

Quoted 5 lines
drivers/misc/Kconfig +1 -0
@@ -11,6 +11,7 @@ menuconfig MISC_DEVICES
 
 	  If you say N, all options in this submenu will be skipped and disabled.
 
+
 if MISC_DEVICES

Unrelated change.

drivers/misc/Kconfig config ATMEL_SSC · line 238

Quoted 9 lines
drivers/misc/Kconfig +3 -0
 config IBM_ASM
@@ -232,4 +233,13 @@ config ATMEL_SSC
 
 	  If unsure, say N.
 
+config ENCLOSURE_SERVICES
+	tristate "Enclosure Services"
+	default n

Not needed. n is default.

Quoted 12 lines
drivers/misc/Kconfig +6 -0
+	help
+	  Provides support for intelligent enclosures (bays which
+	  contain storage devices).  You also need either a host
+	  driver (SCSI/ATA) which supports enclosures
+	  or a SCSI enclosure device (SES) to use these services.
+
 endif # MISC_DEVICES
drivers/misc/Makefile +0 -0
index 87f2685..de9f1f5 100644

drivers/misc/Makefile obj-$(CONFIG_SONY_LAPTOP) += sony-laptop.o · line 20

Quoted 5 lines
drivers/misc/Makefile +1 -0
@@ -17,3 +17,4 @@ obj-$(CONFIG_SONY_LAPTOP)	+= sony-laptop.o
 obj-$(CONFIG_THINKPAD_ACPI)	+= thinkpad_acpi.o
 obj-$(CONFIG_FUJITSU_LAPTOP)	+= fujitsu-laptop.o
 obj-$(CONFIG_EEPROM_93CX6)	+= eeprom_93cx6.o
+obj-$(CONFIG_ENCLOSURE_SERVICES) += enclosure.o
\ No newline at end of file

Can we get one..

Quoted 55 lines
drivers/misc/enclosure.c new file +49 -0
new file mode 100644
index 0000000..e4683cd
@@ -0,0 +1,449 @@
+/*
+ * Enclosure Services
+ *
+ * Copyright (C) 2008 James Bottomley <[email protected]>
+ *
+**-----------------------------------------------------------------------------
+**
+**  This program is free software; you can redistribute it and/or
+**  modify it under the terms of the GNU General Public License
+**  version 2 as published by the Free Software Foundation.
+**
+**  This program is distributed in the hope that it will be useful,
+**  but WITHOUT ANY WARRANTY; without even the implied warranty of
+**  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+**  GNU General Public License for more details.
+**
+**  You should have received a copy of the GNU General Public License
+**  along with this program; if not, write to the Free Software
+**  Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+**
+**-----------------------------------------------------------------------------
+*/
+#include <linux/device.h>
+#include <linux/enclosure.h>
+#include <linux/err.h>
+#include <linux/list.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+
+static LIST_HEAD(container_list);
+static DEFINE_MUTEX(container_list_lock);
+static struct class enclosure_class;
+static struct class enclosure_component_class;
+
+/**
+ * enclosure_find - find an enclosure given a device
+ * @dev:	the device to find for
+ *
+ * Looks through the list of registered enclosures to see
+ * if it can find a match for a device.  Returns NULL if no
+ * enclosure is found.
+ */
+struct enclosure_device *enclosure_find(struct device *dev)
+{
+	struct enclosure_device *edev = NULL;
+
+	mutex_lock(&container_list_lock);
+	list_for_each_entry(edev, &container_list, node) {

drivers/misc/enclosure.c line 59

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+		if (edev->cdev.dev == dev) {
+			mutex_unlock(&container_list_lock);
+			return edev;
+		}
+	}
+	mutex_unlock(&container_list_lock);
+
+	return NULL;
+}
+EXPORT_SYMBOL(enclosure_find);

No GPL - but the other of your EXPORT's are GPL.

Quoted 33 lines
drivers/misc/enclosure.c new file +33 -0
+
+/**
+ * enclosure_for_each_device - calls a function for each enclosure
+ * @fn:		the function to call
+ * @data:	the data to pass to each call
+ *
+ * Loops over all the enclosures calling the function.
+ *
+ * Note, this function uses a mutex which will be held across calls to
+ * @fn, so it must have user context, and @fn should not sleep or
+ * otherwise cause the mutex to be held for indefinite periods
+ */
+int enclosure_for_each_device(int (*fn)(struct enclosure_device *, void *),
+			      void *data)
+{
+	int error = 0;
+	struct enclosure_device *edev;
+
+	mutex_lock(&container_list_lock);
+	list_for_each_entry(edev, &container_list, node) {
+		error = fn(edev, data);
+		if (error)
+			break;
+	}
+	mutex_unlock(&container_list_lock);
+
+	return error;
+}
+EXPORT_SYMBOL_GPL(enclosure_for_each_device);
+
+/**
+ * enclosure_register - register device as an enclosure
+ *

drivers/misc/enclosure.c line 102

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+ * @dev:	device containing the enclosure
+ * @components:	number of components in the enclosure
+ *
+ * This sets up the device for being an enclosure.  Note that @dev does
+ * not have to be a dedicated enclosure device.  It may be some other type
+ * of device that additionally responds to enclosure services
+ */
+struct enclosure_device *
+enclosure_register(struct device *dev, const char *name, int components,
+		   struct enclosure_component_callbacks *cb)

style purists would request you to put return type and function name on the same line...

drivers/misc/enclosure.c line 114

Quoted 12 lines
drivers/misc/enclosure.c new file +12 -0
+{
+	struct enclosure_device *edev =
+		kzalloc(sizeof(struct enclosure_device) +
+			sizeof(struct enclosure_component)*components,
+			GFP_KERNEL);
+	int err, i;
+
+	if (!edev)
+		return ERR_PTR(-ENOMEM);
+
+	if (!cb)
+		return ERR_PTR(-EINVAL);

We leak memory here - we never free edev.

Quoted 66 lines
drivers/misc/enclosure.c new file +66 -0
+
+	edev->components = components;
+
+	edev->cdev.class = &enclosure_class;
+	edev->cdev.dev = get_device(dev);
+	edev->cb = cb;
+	snprintf(edev->cdev.class_id, BUS_ID_SIZE, "%s", name);
+	err = class_device_register(&edev->cdev);
+	if (err)
+		goto err;
+
+	for (i = 0; i < components; i++)
+		edev->component[i].number = -1;
+
+	mutex_lock(&container_list_lock);
+	list_add_tail(&edev->node, &container_list);
+	mutex_unlock(&container_list_lock);
+
+	return edev;
+
+ err:
+	put_device(edev->cdev.dev);
+	kfree(edev);
+	return ERR_PTR(err);
+}
+EXPORT_SYMBOL_GPL(enclosure_register);
+
+static struct enclosure_component_callbacks enclosure_null_callbacks;
+
+/**
+ * enclosure_unregister - remove an enclosure
+ *
+ * @edev:	the registered enclosure to remove;
+ */
+void enclosure_unregister(struct enclosure_device *edev)
+{
+	int i;
+
+	if (!edev)
+		return;
+
+	mutex_lock(&container_list_lock);
+	list_del(&edev->node);
+	mutex_unlock(&container_list_lock);
+
+	for (i = 0; i < edev->components; i++)
+		if (edev->component[i].number != -1)
+			class_device_unregister(&edev->component[i].cdev);
+
+	/* prevent any callbacks into service user */
+	edev->cb = &enclosure_null_callbacks;
+	class_device_unregister(&edev->cdev);
+}
+EXPORT_SYMBOL_GPL(enclosure_unregister);
+
+static void enclosure_release(struct class_device *cdev)
+{
+	struct enclosure_device *edev = to_enclosure_device(cdev);
+
+	put_device(cdev->dev);
+	kfree(edev);
+}
+
+static void enclosure_component_release(struct class_device *cdev)
+{
+	if (cdev->dev)

drivers/misc/enclosure.c line 190

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+		put_device(cdev->dev);
+	class_device_put(cdev->parent);
+}
+
+struct enclosure_component *
+enclosure_component_register(struct enclosure_device *edev,
+			     unsigned int number,
+			     enum enclosure_component_type type,
+			     const char *name)
+{

Missing kernel-doc for this and the following exports.

Quoted 29 lines
drivers/misc/enclosure.c new file +29 -0
+	struct enclosure_component *ecomp;
+	struct class_device *cdev;
+	int err;
+
+	if (!edev || number >= edev->components)
+		return ERR_PTR(-EINVAL);
+
+	ecomp = &edev->component[number];
+
+	if (ecomp->number != -1)
+		return ERR_PTR(-EINVAL);
+
+	ecomp->type = type;
+	ecomp->number = number;
+	cdev = &ecomp->cdev;
+	cdev->parent = class_device_get(&edev->cdev);
+	cdev->class = &enclosure_component_class;
+	if (name)
+		snprintf(cdev->class_id, BUS_ID_SIZE, "%s", name);
+	else
+		snprintf(cdev->class_id, BUS_ID_SIZE, "%d", number);
+
+	err = class_device_register(cdev);
+	if (err)
+		ERR_PTR(err);
+
+	return ecomp;
+}
+EXPORT_SYMBOL_GPL(enclosure_component_register);
author maintainer
#2 lore raw
Quoted 5 lines · Sam Ravnborg

Hi James.

Nitpicking only.

Sam

Thanks for the review.

Quoted 4 lines
Quoted 2 lines

+
if MISC_DEVICES

Unrelated change.

Yes, removed it.

Quoted 4 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+config ENCLOSURE_SERVICES
+	tristate "Enclosure Services"
+	default n

Not needed. n is default.

It is now ... plus there are a few other entries in this file with default n.

Quoted 3 lines
Quoted 2 lines

+obj-$(CONFIG_ENCLOSURE_SERVICES) += enclosure.o
\ No newline at end of file

Can we get one..

added

Quoted 3 lines
Quoted 1 line

+EXPORT_SYMBOL(enclosure_find);

No GPL - but the other of your EXPORT's are GPL.

Done ... Greg gets annoyed about these.

Quoted 5 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+struct enclosure_device *
+enclosure_register(struct device *dev, const char *name, int components,
+		   struct enclosure_component_callbacks *cb)

style purists would request you to put return type and function name on the same line...

I tend to prefer the function name always at the beginning of the line. But even style purists have to agree it's better than trying to futilely squash all the arguments on separate lines because the return complex type is quite long.

Quoted 12 lines

+ struct enclosure_device *edev =

Review

Quoted 10 lines
(quoted diff) +10 -0
+		kzalloc(sizeof(struct enclosure_device) +
+			sizeof(struct enclosure_component)*components,
+			GFP_KERNEL);
+	int err, i;
+
+	if (!edev)
+		return ERR_PTR(-ENOMEM);
+
+	if (!cb)
+		return ERR_PTR(-EINVAL);

We leak memory here - we never free edev.

We do indeed .. fixed.

Quoted 7 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+struct enclosure_component *
+enclosure_component_register(struct enclosure_device *edev,
+			     unsigned int number,
+			     enum enclosure_component_type type,
+			     const char *name)
+{

Missing kernel-doc for this and the following exports.

Yes, added.

James

---

From: James Bottomley <[email protected]>
Date: Sun, 3 Feb 2008 15:40:56 -0600
Subject: [SCSI] enclosure: add support for enclosure services

The enclosure misc device is really just a library providing sysfs support for physical enclosure devices and their components.

---
 drivers/misc/Kconfig      |    9 +
 drivers/misc/Makefile     |    1 +
 drivers/misc/enclosure.c  |  486 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/enclosure.h |  120 +++++++++++
 4 files changed, 616 insertions(+), 0 deletions(-)
 create mode 100644 drivers/misc/enclosure.c
 create mode 100644 include/linux/enclosure.h
drivers/misc/Kconfig +9 -0
index b5e67c0..ca68480 100644
@@ -232,4 +232,13 @@ config ATMEL_SSC
 
 	  If unsure, say N.
 
+config ENCLOSURE_SERVICES
+	tristate "Enclosure Services"
+	default n
+	help
+	  Provides support for intelligent enclosures (bays which
+	  contain storage devices).  You also need either a host
+	  driver (SCSI/ATA) which supports enclosures
+	  or a SCSI enclosure device (SES) to use these services.
+
 endif # MISC_DEVICES
drivers/misc/Makefile +1 -0
index 87f2685..639c755 100644
@@ -17,3 +17,4 @@ obj-$(CONFIG_SONY_LAPTOP)	+= sony-laptop.o
 obj-$(CONFIG_THINKPAD_ACPI)	+= thinkpad_acpi.o
 obj-$(CONFIG_FUJITSU_LAPTOP)	+= fujitsu-laptop.o
 obj-$(CONFIG_EEPROM_93CX6)	+= eeprom_93cx6.o
+obj-$(CONFIG_ENCLOSURE_SERVICES) += enclosure.o
drivers/misc/enclosure.c new file +486 -0

486 lines not on this page · raw message

include/linux/enclosure.h new file +120 -0

120 lines not on this page · raw message

Signature · 1 line
1.5.3.8
#3 lore raw
Quoted 8 lines · On Sun, 03 Feb 2008 18:16:51 -0600 James Bottomley…

From: James Bottomley <[email protected]>
Date: Sun, 3 Feb 2008 15:40:56 -0600
Subject: [SCSI] enclosure: add support for enclosure services

The enclosure misc device is really just a library providing sysfs support for physical enclosure devices and their components.

Thanks for sending it out for review.

Quoted 6 lines
drivers/misc/enclosure.c new file +6 -0
+struct enclosure_device *enclosure_find(struct device *dev)
+{
+	struct enclosure_device *edev = NULL;
+
+	mutex_lock(&container_list_lock);
+	list_for_each_entry(edev, &container_list, node) {

drivers/misc/enclosure.c line 59

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+		if (edev->cdev.dev == dev) {
+			mutex_unlock(&container_list_lock);
+			return edev;
+		}
+	}
+	mutex_unlock(&container_list_lock);
+
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(enclosure_find);

This looks a little odd. We don't take a ref on the object after looking it up, so what prevents some other thread of control from freeing or otherwise altering the returned object while the caller is playing with it?

Quoted 9 lines

+/**
+ * enclosure_for_each_device - calls a function for each enclosure
+ * @fn: the function to call
+ * @data: the data to pass to each call
+ *
+ * Loops over all the enclosures calling the function.
+ *
+ * Note, this function uses a mutex which will be held across calls to
+ * @fn, so it must have user context, and @fn should not sleep or

Probably "non atomic context" would be more accurate.

fn() actually _can_ sleep.

Quoted 37 lines

+ * otherwise cause the mutex to be held for indefinite periods
+ */

drivers/misc/enclosure.c new file +35 -0
+int enclosure_for_each_device(int (*fn)(struct enclosure_device *, void *),
+			      void *data)
+{
+	int error = 0;
+	struct enclosure_device *edev;
+
+	mutex_lock(&container_list_lock);
+	list_for_each_entry(edev, &container_list, node) {
+		error = fn(edev, data);
+		if (error)
+			break;
+	}
+	mutex_unlock(&container_list_lock);
+
+	return error;
+}
+EXPORT_SYMBOL_GPL(enclosure_for_each_device);
+
+/**
+ * enclosure_register - register device as an enclosure
+ *
+ * @dev:	device containing the enclosure
+ * @components:	number of components in the enclosure
+ *
+ * This sets up the device for being an enclosure.  Note that @dev does
+ * not have to be a dedicated enclosure device.  It may be some other type
+ * of device that additionally responds to enclosure services
+ */
+struct enclosure_device *
+enclosure_register(struct device *dev, const char *name, int components,
+		   struct enclosure_component_callbacks *cb)
+{
+	struct enclosure_device *edev =
+		kzalloc(sizeof(struct enclosure_device) +
+			sizeof(struct enclosure_component)*components,

drivers/misc/enclosure.c line 116

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+			GFP_KERNEL);
+	int err, i;
+
+	if (!edev)
+		return ERR_PTR(-ENOMEM);
+
+	if (!cb) {
+		kfree(edev);
+		return ERR_PTR(-EINVAL);
+	}

It would be less fuss if this were to test cb before doing the kzalloc().

Can cb==NULL actually and legitimately happen?

Quoted 29 lines
drivers/misc/enclosure.c new file +29 -0
+	edev->components = components;
+
+	edev->cdev.class = &enclosure_class;
+	edev->cdev.dev = get_device(dev);
+	edev->cb = cb;
+	snprintf(edev->cdev.class_id, BUS_ID_SIZE, "%s", name);
+	err = class_device_register(&edev->cdev);
+	if (err)
+		goto err;
+
+	for (i = 0; i < components; i++)
+		edev->component[i].number = -1;
+
+	mutex_lock(&container_list_lock);
+	list_add_tail(&edev->node, &container_list);
+	mutex_unlock(&container_list_lock);
+
+	return edev;
+
+ err:
+	put_device(edev->cdev.dev);
+	kfree(edev);
+	return ERR_PTR(err);
+}
+EXPORT_SYMBOL_GPL(enclosure_register);
+
+static struct enclosure_component_callbacks enclosure_null_callbacks;
+
+/**

drivers/misc/enclosure.c line 156

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+ * enclosure_unregister - remove an enclosure
+ *
+ * @edev:	the registered enclosure to remove;
+ */
+void enclosure_unregister(struct enclosure_device *edev)
+{
+	int i;
+
+	if (!edev)
+		return;

Is this legal?

drivers/misc/enclosure.c line 160

Quoted 3 lines
drivers/misc/enclosure.c new file +3 -0
+	mutex_lock(&container_list_lock);
+	list_del(&edev->node);
+	mutex_unlock(&container_list_lock);

See, right now, someone who found this enclosure_device via enclosure_find() could still be playing with it?

Quoted 25 lines
drivers/misc/enclosure.c new file +25 -0
+	for (i = 0; i < edev->components; i++)
+		if (edev->component[i].number != -1)
+			class_device_unregister(&edev->component[i].cdev);
+
+	/* prevent any callbacks into service user */
+	edev->cb = &enclosure_null_callbacks;
+	class_device_unregister(&edev->cdev);
+}
+EXPORT_SYMBOL_GPL(enclosure_unregister);
+
+/**
+ * enclosure_component_register - add a particular component to an enclosure
+ * @edev:	the enclosure to add the component
+ * @num:	the device number
+ * @type:	the type of component being added
+ * @name:	an optional name to appear in sysfs (leave NULL if none)
+ *
+ * Registers the component.  The name is optional for enclosures that
+ * give their components a unique name.  If not, leave the field NULL
+ * and a name will be assigned.
+ *
+ * Returns a pointer to the enclosure component or an error.
+ */
+struct enclosure_component *
+enclosure_component_register(struct enclosure_device *edev,

drivers/misc/enclosure.c line 211

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+			     unsigned int number,
+			     enum enclosure_component_type type,
+			     const char *name)
+{
+	struct enclosure_component *ecomp;
+	struct class_device *cdev;
+	int err;
+
+	if (!edev || number >= edev->components)
+		return ERR_PTR(-EINVAL);

Is !edev possible and legitimate?

Quoted 4 lines
drivers/misc/enclosure.c new file +4 -0
+	ecomp = &edev->component[number];
+
+	if (ecomp->number != -1)
+		return ERR_PTR(-EINVAL);

drivers/misc/enclosure.c line 226

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+
+	ecomp->type = type;
+	ecomp->number = number;
+	cdev = &ecomp->cdev;
+	cdev->parent = class_device_get(&edev->cdev);
+	cdev->class = &enclosure_component_class;
+	if (name)
+		snprintf(cdev->class_id, BUS_ID_SIZE, "%s", name);
+	else
+		snprintf(cdev->class_id, BUS_ID_SIZE, "%d", number);

%u :)

Quoted 41 lines
drivers/misc/enclosure.c new file +41 -0
+
+	err = class_device_register(cdev);
+	if (err)
+		ERR_PTR(err);
+
+	return ecomp;
+}
+EXPORT_SYMBOL_GPL(enclosure_component_register);
+
+/**
+ * enclosure_add_device - add a device as being part of an enclosure
+ * @edev:	the enclosure device being added to.
+ * @num:	the number of the component
+ * @dev:	the device being added
+ *
+ * Declares a real device to reside in slot (or identifier) @num of an
+ * enclosure.  This will cause the relevant sysfs links to appear.
+ * This function may also be used to change a device associated with
+ * an enclosure without having to call enclosure_remove_device() in
+ * between.
+ *
+ * Returns zero on success or an error.
+ */
+int enclosure_add_device(struct enclosure_device *edev, int component,
+			 struct device *dev)
+{
+	struct class_device *cdev;
+
+	if (!edev || component >= edev->components)
+		return -EINVAL;
+
+	cdev = &edev->component[component].cdev;
+
+	class_device_del(cdev);
+	if (cdev->dev)
+		put_device(cdev->dev);
+	cdev->dev = get_device(dev);
+	return class_device_add(cdev);
+}
+EXPORT_SYMBOL_GPL(enclosure_add_device);
+

drivers/misc/enclosure.c line 302

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+/*
+ * sysfs pieces below
+ */
+
+static ssize_t enclosure_show_components(struct class_device *cdev, char *buf)
+{
+	struct enclosure_device *edev = to_enclosure_device(cdev);
+
+	return snprintf(buf, 40, "%d\n", edev->components);
+}

"40"?

Quoted 17 lines
drivers/misc/enclosure.c new file +17 -0
+static struct class_device_attribute enclosure_attrs[] = {
+	__ATTR(components, S_IRUGO, enclosure_show_components, NULL),
+	__ATTR_NULL
+};
+
+static struct class enclosure_class = {
+	.name			= "enclosure",
+	.owner			= THIS_MODULE,
+	.release		= enclosure_release,
+	.class_dev_attrs	= enclosure_attrs,
+};
+
+static char *enclosure_status [] = {
+	[ENCLOSURE_STATUS_UNSUPPORTED] = "unsupported",
+	[ENCLOSURE_STATUS_OK] = "OK",
+	[ENCLOSURE_STATUS_CRITICAL] = "critical",
+	[ENCLOSURE_STATUS_NON_CRITICAL] = "non-critical",

drivers/misc/enclosure.c line 330

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+	[ENCLOSURE_STATUS_UNRECOVERABLE] = "unrecoverable",
+	[ENCLOSURE_STATUS_NOT_INSTALLED] = "not installed",
+	[ENCLOSURE_STATUS_UNKNOWN] = "unknown",
+	[ENCLOSURE_STATUS_UNAVAILABLE] = "unavailable",
+};
+
+static char *enclosure_type [] = {
+	[ENCLOSURE_COMPONENT_DEVICE] = "device",
+	[ENCLOSURE_COMPONENT_ARRAY_DEVICE] = "array device",
+};

One could play with const here, if sufficiently keen.

drivers/misc/enclosure.c line 347

Quoted 6 lines
drivers/misc/enclosure.c new file +6 -0
+static ssize_t set_component_fault(struct class_device *cdev, const char *buf,
+				   size_t count)
+{
+	struct enclosure_device *edev = to_enclosure_device(cdev->parent);
+	struct enclosure_component *ecomp = to_enclosure_component(cdev);
+	int val = simple_strtoul(buf, NULL, 0);

hrm, we do this conversion about 1e99 times in the kernel and we have to go and pass three args where only one was needed. katoi()?

Quoted 18 lines
drivers/misc/enclosure.c new file +18 -0
+	if (edev->cb->set_fault)
+		edev->cb->set_fault(edev, ecomp, val);
+	return count;
+}
+
+static ssize_t get_component_status(struct class_device *cdev, char *buf)
+{
+	struct enclosure_device *edev = to_enclosure_device(cdev->parent);
+	struct enclosure_component *ecomp = to_enclosure_component(cdev);
+
+	if (edev->cb->get_status)
+		edev->cb->get_status(edev, ecomp);
+	return snprintf(buf, 40, "%s\n", enclosure_status[ecomp->status]);
+}
+
+static ssize_t set_component_status(struct class_device *cdev, const char *buf,
+				   size_t count)
+{

drivers/misc/enclosure.c line 376

Quoted 10 lines
drivers/misc/enclosure.c new file +10 -0
+	struct enclosure_device *edev = to_enclosure_device(cdev->parent);
+	struct enclosure_component *ecomp = to_enclosure_component(cdev);
+	int i;
+
+	for (i = 0; enclosure_status[i]; i++) {
+		if (strncmp(buf, enclosure_status[i],
+			    strlen(enclosure_status[i])) == 0 &&
+		    buf[strlen(enclosure_status[i])] == '\n')
+			break;
+	}

So if an application does

	write(fd, "foo", 3)

it won't work? Thye have to do

	write(fd, "foo\n", 4)

?

drivers/misc/enclosure.c

Quoted 12 lines
drivers/misc/enclosure.c new file +9 -0
+	if (enclosure_status[i] && edev->cb->set_status) {
+		edev->cb->set_status(edev, ecomp, i);
+		return count;
+	} else
+		return -EINVAL;
+}
+
 
...
 
+#define to_enclosure_device(x) container_of((x), struct enclosure_device, cdev)
+#define to_enclosure_component(x) container_of((x), struct enclosure_component, cdev)

These could be C functions...

Quoted 5 lines
include/linux/enclosure.h new file +5 -0
+struct enclosure_device *
+enclosure_register(struct device *, const char *, int,
+		   struct enclosure_component_callbacks *);
+void enclosure_unregister(struct enclosure_device *);
+struct enclosure_component *

include/linux/enclosure.h line 120

Quoted 10 lines
include/linux/enclosure.h new file +10 -0
+enclosure_component_register(struct enclosure_device *, unsigned int,
+				 enum enclosure_component_type, const char *);
+int enclosure_add_device(struct enclosure_device *enclosure, int component,
+			 struct device *dev);
+int enclosure_remove_device(struct enclosure_device *enclosure, int component);
+struct enclosure_device *enclosure_find(struct device *dev);
+int enclosure_for_each_device(int (*fn)(struct enclosure_device *, void *),
+			      void *data);
+
+#endif /* _LINUX_ENCLOSURE_H_ */

Nice looking driver.

author maintainer
#4 lore raw
Quoted 26 lines · Andrew Morton
Quoted 8 lines · On Sun, 03 Feb 2008 18:16:51 -0600 James Bottomley…

From: James Bottomley <[email protected]>
Date: Sun, 3 Feb 2008 15:40:56 -0600
Subject: [SCSI] enclosure: add support for enclosure services

The enclosure misc device is really just a library providing sysfs support for physical enclosure devices and their components.

Thanks for sending it out for review.

Quoted 12 lines
(quoted diff) +12 -0
+struct enclosure_device *enclosure_find(struct device *dev)
+{
+	struct enclosure_device *edev = NULL;
+
+	mutex_lock(&container_list_lock);
+	list_for_each_entry(edev, &container_list, node) {
+		if (edev->cdev.dev == dev) {
+			mutex_unlock(&container_list_lock);
+			return edev;
+		}
+	}
+	mutex_unlock(&container_list_lock);
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(enclosure_find);

This looks a little odd. We don't take a ref on the object after looking it up, so what prevents some other thread of control from freeing or otherwise altering the returned object while the caller is playing with it?

The use case is for enclosure destruction, so the free should never happen, but I take the point; I've added a class_device_get().

Quoted 10 lines
Quoted 9 lines

+/**
+ * enclosure_for_each_device - calls a function for each enclosure
+ * @fn: the function to call
+ * @data: the data to pass to each call
+ *
+ * Loops over all the enclosures calling the function.
+ *
+ * Note, this function uses a mutex which will be held across calls to
+ * @fn, so it must have user context, and @fn should not sleep or

Quoted 3 lines

Probably "non atomic context" would be more accurate.

fn() actually _can_ sleep.

"should" to me means you don't have to do this but ought to. I'll add a may (but should not).

Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+	if (!cb) {
+		kfree(edev);
+		return ERR_PTR(-EINVAL);
+	}

It would be less fuss if this were to test cb before doing the kzalloc().

Can cb==NULL actually and legitimately happen?

Not really ... I'll make it a BUG_ON.

Quoted 8 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+void enclosure_unregister(struct enclosure_device *edev)
+{
+	int i;
+
+	if (!edev)
+		return;

Is this legal?

No ... it'll oops on the null deref later ... I'll remove this.

Quoted 6 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+	mutex_lock(&container_list_lock);
+	list_del(&edev->node);
+	mutex_unlock(&container_list_lock);

See, right now, someone who found this enclosure_device via enclosure_find() could still be playing with it?

Yes, fixed.

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	if (!edev || number >= edev->components)
+		return ERR_PTR(-EINVAL);

Is !edev possible and legitimate?

It shouldn't be, no ... I can remove it.

Quoted 3 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+		snprintf(cdev->class_id, BUS_ID_SIZE, "%d", number);

%u :)

Nitpicker!

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	return snprintf(buf, 40, "%d\n", edev->components);
+}

"40"?

I just followed precedence ;-P

There doesn't seem to be a define for this maximum length, so 40 is the most commonly picked constant.

Quoted 6 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+static char *enclosure_type [] = {
+	[ENCLOSURE_COMPONENT_DEVICE] = "device",
+	[ENCLOSURE_COMPONENT_ARRAY_DEVICE] = "array device",
+};

One could play with const here, if sufficiently keen.

One will try to summon up the enthusiasm.

Quoted 9 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+static ssize_t set_component_fault(struct class_device *cdev, const char *buf,
+				   size_t count)
+{
+	struct enclosure_device *edev = to_enclosure_device(cdev->parent);
+	struct enclosure_component *ecomp = to_enclosure_component(cdev);
+	int val = simple_strtoul(buf, NULL, 0);

hrm, we do this conversion about 1e99 times in the kernel and we have to go and pass three args where only one was needed. katoi()?

Yes ... I'll add it to the todo list.

Quoted 9 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+	for (i = 0; enclosure_status[i]; i++) {
+		if (strncmp(buf, enclosure_status[i],
+			    strlen(enclosure_status[i])) == 0 &&
+		    buf[strlen(enclosure_status[i])] == '\n')
+			break;
+	}

So if an application does

Quoted 7 lines
	write(fd, "foo", 3)

it won't work? Thye have to do

	write(fd, "foo\n", 4)

?

No ... it's designed for echo; however, I'll add a check for '\0' which will catch the write case.

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+#define to_enclosure_device(x) container_of((x), struct enclosure_device, cdev)
+#define to_enclosure_component(x) container_of((x), struct enclosure_component, cdev)

These could be C functions...

OK ... I was just following precedence again, but I can make them inlines.

Quoted 1 line

Nice looking driver.

Thanks,

James

---

Here's the incremental diff.

drivers/misc/enclosure.c +14 -16
index 42e6e43..6fcb0e9 100644
@@ -39,7 +39,8 @@ static struct class enclosure_component_class;
  *
  * Looks through the list of registered enclosures to see
  * if it can find a match for a device.  Returns NULL if no
- * enclosure is found.
+ * enclosure is found. Obtains a reference to the enclosure class
+ * device which must be released with class_device_put().
  */
 struct enclosure_device *enclosure_find(struct device *dev)
 {
@@ -48,6 +49,7 @@ struct enclosure_device *enclosure_find(struct device *dev)
 	mutex_lock(&container_list_lock);
 	list_for_each_entry(edev, &container_list, node) {
 		if (edev->cdev.dev == dev) {
+			class_device_get(&edev->cdev);
 			mutex_unlock(&container_list_lock);
 			return edev;
 		}
@@ -66,8 +68,9 @@ EXPORT_SYMBOL_GPL(enclosure_find);
  * Loops over all the enclosures calling the function.
  *
  * Note, this function uses a mutex which will be held across calls to
- * @fn, so it must have user context, and @fn should not sleep or
- * otherwise cause the mutex to be held for indefinite periods
+ * @fn, so it must have non atomic context, and @fn may (although it
+ * should not) sleep or otherwise cause the mutex to be held for
+ * indefinite periods
  */
 int enclosure_for_each_device(int (*fn)(struct enclosure_device *, void *),
 			      void *data)
@@ -107,14 +110,11 @@ enclosure_register(struct device *dev, const char *name, int components,
 			GFP_KERNEL);
 	int err, i;
 
+	BUG_ON(!cb);
+
 	if (!edev)
 		return ERR_PTR(-ENOMEM);
 
-	if (!cb) {
-		kfree(edev);
-		return ERR_PTR(-EINVAL);
-	}
-
 	edev->components = components;
 
 	edev->cdev.class = &enclosure_class;
@@ -152,9 +152,6 @@ void enclosure_unregister(struct enclosure_device *edev)
 {
 	int i;
 
-	if (!edev)
-		return;
-
 	mutex_lock(&container_list_lock);
 	list_del(&edev->node);
 	mutex_unlock(&container_list_lock);
@@ -207,7 +204,7 @@ enclosure_component_register(struct enclosure_device *edev,
 	struct class_device *cdev;
 	int err;
 
-	if (!edev || number >= edev->components)
+	if (number >= edev->components)
 		return ERR_PTR(-EINVAL);
 
 	ecomp = &edev->component[number];
@@ -223,7 +220,7 @@ enclosure_component_register(struct enclosure_device *edev,
 	if (name)
 		snprintf(cdev->class_id, BUS_ID_SIZE, "%s", name);
 	else
-		snprintf(cdev->class_id, BUS_ID_SIZE, "%d", number);
+		snprintf(cdev->class_id, BUS_ID_SIZE, "%u", number);
 
 	err = class_device_register(cdev);
 	if (err)
@@ -313,7 +310,7 @@ static struct class enclosure_class = {
 	.class_dev_attrs	= enclosure_attrs,
 };
 
-static char *enclosure_status [] = {
+static const char *const enclosure_status [] = {
 	[ENCLOSURE_STATUS_UNSUPPORTED] = "unsupported",
 	[ENCLOSURE_STATUS_OK] = "OK",
 	[ENCLOSURE_STATUS_CRITICAL] = "critical",
@@ -324,7 +321,7 @@ static char *enclosure_status [] = {
 	[ENCLOSURE_STATUS_UNAVAILABLE] = "unavailable",
 };
 
-static char *enclosure_type [] = {
+static const char *const enclosure_type [] = {
 	[ENCLOSURE_COMPONENT_DEVICE] = "device",
 	[ENCLOSURE_COMPONENT_ARRAY_DEVICE] = "array device",
 };
@@ -371,7 +368,8 @@ static ssize_t set_component_status(struct class_device *cdev, const char *buf,
 	for (i = 0; enclosure_status[i]; i++) {
 		if (strncmp(buf, enclosure_status[i],
 			    strlen(enclosure_status[i])) == 0 &&
-		    buf[strlen(enclosure_status[i])] == '\n')
+		    (buf[strlen(enclosure_status[i])] == '\n' ||
+		     buf[strlen(enclosure_status[i])] == '\0'))
 			break;
 	}
 
drivers/scsi/ses.c +5 -2
index 2565584..54afb80 100644
@@ -420,9 +420,11 @@ static int ses_intf_add(struct class_device *cdev,
 
 	if (!scsi_device_enclosure(sdev)) {
 		/* not an enclosure, but might be in one */
-		edev = 	enclosure_find(&sdev->host->shost_gendev);
-		if (edev)
+		edev = 	enclosure_find(&sdev->host->shost_gendev); 
+		if (edev) {
 			ses_match_to_enclosure(edev, sdev);
+			class_device_put(&edev->cdev);
+		}
 		return -ENODEV;
 	}
 
@@ -634,6 +636,7 @@ static void ses_intf_remove(struct class_device *cdev,
 
 	kfree(edev->component[0].scratch);
 
+	class_device_put(&edev->cdev);
 	enclosure_unregister(edev);
 }
 
include/linux/enclosure.h +11 -2
index 622177a..a5978f1 100644
@@ -100,8 +100,17 @@ struct enclosure_device {
 	struct enclosure_component component[0];
 };
 
-#define to_enclosure_device(x) container_of((x), struct enclosure_device, cdev)
-#define to_enclosure_component(x) container_of((x), struct enclosure_component, cdev)
+static inline struct enclosure_device *
+to_enclosure_device(struct class_device *dev)
+{
+	return container_of(dev, struct enclosure_device, cdev);
+}
+
+static inline struct enclosure_component *
+to_enclosure_component(struct class_device *dev)
+{
+	return container_of(dev, struct enclosure_component, cdev);
+}
 
 struct enclosure_device *
 enclosure_register(struct device *, const char *, int,
#5 lore raw
In reply to James Bottomley
Quoted 4 lines · James Bottomley

The enclosure misc device is really just a library providing sysfs
support for physical enclosure devices and their components.

Who is the target audience/user of those facilities?
a) The kernel itself needing to read/write SES pages?
b) A user space application using sysfs to read/write SES pages?

At the moment SES device management is done via an application (user-space) and a user-space library used by the application and /dev/sgX to send SCSI commands to the SES device.

One could have a very good argument to not bloat the kernel with this but leave it to a user-space application and a library to do all this and communicate with the SES device via the kernel's /dev/sgX.

Luben

author maintainer
#6 lore raw
Quoted 9 lines · Luben Tuikov
Quoted 4 lines · James Bottomley

The enclosure misc device is really just a library providing sysfs
support for physical enclosure devices and their components.

Who is the target audience/user of those facilities?
a) The kernel itself needing to read/write SES pages?

That depends on the enclosure integration, but right at the moment, it doesn't

Quoted 2 lines

b) A user space application using sysfs to read/write SES pages?

Not an application so much as a user. The idea of sysfs is to allow users to get and set the information in addition to applications.

Quoted 4 lines

At the moment SES device management is done via an application (user-space) and a user-space library used by the application and /dev/sgX to send SCSI commands to the SES device.

I must have missed that when I was looking for implementations; what's the URL?

But, if we have non-scsi enclosures to integrate, that makes it harder for a user application because it has to know all the implementations. A sysfs framework on the other hand is a universal known thing for the user applications.

Quoted 4 lines

One could have a very good argument to not bloat the kernel with this but leave it to a user-space application and a library to do all this and communicate with the SES device via the kernel's /dev/sgX.

The same thing goes for other esoteric SCSI infrastructure pieces like cd changers. On the whole, given that ATA is asking for enclosure management in kernel, it makes sense to consolidate the infrastructure and a ses ULD is a very good test bed.

James

#7 lore raw
Quoted 6 lines · James Bottomley
Quoted 1 line
Quoted 1 line

The enclosure misc device is really just a

library providing

Quoted 4 lines
Quoted 3 lines

sysfs
support for physical enclosure devices and their
components.

Quoted 6 lines
Quoted 2 lines

Who is the target audience/user of those facilities?
a) The kernel itself needing to read/write SES pages?

That depends on the enclosure integration, but right at the moment, it
doesn't

Yes, I didn't suspect so.

Quoted 8 lines
Quoted 3 lines

b) A user space application using sysfs to read/write SES pages?

Not an application so much as a user. The idea of sysfs is to allow
users to get and set the information in addition to applications.

Exactly the same argument stands for a user-space application with a user-space library.

This is the classical case of where it is better to do this in user-space as opposed to the kernel.

The kernel provides capability to access the SES device. The user space application and library provide interpretation and control. Thus if the enclosure were upgraded, one doesn't need to upgrade their kernel in order to utilize the new capabilities of the SES device. Plus upgrading a user-space application is a lot easier than the kernel (and no reboot necessary).

Consider another thing: vendors would really like unprecedented access to the SES device in the enclosure so as your ses/enclosure code keeps state it would get out of sync when vendor user-space enclosure applications access (and modify) the SES device's pages.

You can test this yourself: submit a patch that removes SES /dev/sgX support; advertise your ses/class solution and watch the fun.

Quoted 8 lines
Quoted 4 lines

At the moment SES device management is done via an application (user-space) and a user-space library used by the application and /dev/sgX to send SCSI commands to the SES device.

I must have missed that when I was looking for
implementations; what's
the URL?

I'm not aware of any GPLed ones. That doesn't necessarily mean that the best course of action is to bloat the kernel. You can move your ses/enclosure stuff to a user space application library
and thus start a GPLed one.

Quoted 4 lines

But, if we have non-scsi enclosures to integrate, that makes it harder
for a user application because it has to know all the implementations.

So does the kernel. And as I pointed out above, it is a lot easier to upgrade a user-space application and library than it is to upgrade a new kernel and having to reboot the computer to run the new kernel.

Quoted 3 lines

A sysfs framework on the other hand is a universal known thing for the
user applications.

So would a user-space ses library, a la libses.so.

Quoted 6 lines
Quoted 4 lines

One could have a very good argument to not bloat the kernel with this but leave it to a user-space application and a library to do all this and communicate with the SES device via the kernel's

/dev/sgX.

Quoted 7 lines

The same thing goes for other esoteric SCSI infrastructure pieces like
cd changers. On the whole, given that ATA is asking for enclosure
management in kernel, it makes sense to consolidate the infrastructure
and a ses ULD is a very good test bed.

What is wrong with exporting the SES device as /dev/sgX and having a user-space application and library to do all this?

Luben

author maintainer
#8 lore raw
Quoted 31 lines · Luben Tuikov
Quoted 6 lines · James Bottomley
Quoted 1 line
Quoted 1 line

The enclosure misc device is really just a

library providing

Quoted 4 lines
Quoted 3 lines

sysfs
support for physical enclosure devices and their
components.

Quoted 6 lines
Quoted 2 lines

Who is the target audience/user of those facilities?
a) The kernel itself needing to read/write SES pages?

That depends on the enclosure integration, but right at the moment, it
doesn't

Yes, I didn't suspect so.

Quoted 8 lines
Quoted 3 lines

b) A user space application using sysfs to read/write SES pages?

Not an application so much as a user. The idea of sysfs is to allow
users to get and set the information in addition to applications.

Exactly the same argument stands for a user-space application with a user-space library.

This is the classical case of where it is better to do this in user-space as opposed to the kernel.

Quoted 8 lines

The kernel provides capability to access the SES device. The user space application and library provide interpretation and control. Thus if the enclosure were upgraded, one doesn't need to upgrade their kernel in order to utilize the new capabilities of the SES device. Plus upgrading a user-space application is a lot easier than the kernel (and no reboot necessary).

The implementation is modular, so it's remove and insert ...

Quoted 6 lines

Consider another thing: vendors would really like unprecedented access to the SES device in the enclosure so as your ses/enclosure code keeps state it would get out of sync when vendor user-space enclosure applications access (and modify) the SES device's pages.

The state model doesn't assume nothing else will alter the state.

Quoted 13 lines

You can test this yourself: submit a patch that removes SES /dev/sgX support; advertise your ses/class solution and watch the fun.

Quoted 8 lines
Quoted 4 lines

At the moment SES device management is done via an application (user-space) and a user-space library used by the application and /dev/sgX to send SCSI commands to the SES device.

I must have missed that when I was looking for
implementations; what's
the URL?

Quoted 5 lines

I'm not aware of any GPLed ones. That doesn't necessarily mean that the best course of action is to bloat the kernel. You can move your ses/enclosure stuff to a user space application library
and thus start a GPLed one.

Certainly ... patches welcome.

Quoted 9 lines
Quoted 4 lines

But, if we have non-scsi enclosures to integrate, that makes it harder
for a user application because it has to know all the implementations.

So does the kernel. And as I pointed out above, it is a lot easier to upgrade a user-space application and library than it is to upgrade a new kernel and having to reboot the computer to run the new kernel.

No, think again ... it's easy for SES based enclosures because they have a SCSI transport. We have no transport for SGPIO based enclosures nor for any of the other more esoteric ones.

That's not to say it can't be done, but it does mean that it can't be completely userspace.

Quoted 20 lines
Quoted 3 lines

A sysfs framework on the other hand is a universal known thing for the
user applications.

So would a user-space ses library, a la libses.so.

Quoted 6 lines
Quoted 4 lines

One could have a very good argument to not bloat the kernel with this but leave it to a user-space application and a library to do all this and communicate with the SES device via the kernel's

/dev/sgX.

Quoted 7 lines

The same thing goes for other esoteric SCSI infrastructure pieces like
cd changers. On the whole, given that ATA is asking for enclosure
management in kernel, it makes sense to consolidate the infrastructure
and a ses ULD is a very good test bed.

Quoted 3 lines

What is wrong with exporting the SES device as /dev/sgX and having a user-space application and library to do all this?

How do you transport the enclosure commands over /dev/sgX? Only SES has SCSI command encapsulation ... the rest won't even be SCSI targets ...

James

#9 lore raw
Quoted 50 lines · James Bottomley
Quoted 1 line · Luben Tuikov

--- On Mon, 2/4/08, James Bottomley

Quoted 1 line · <[email protected]> wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line

The enclosure misc device is really

just a

Quoted 3 lines
Quoted 3 lines

library providing

Quoted 2 lines
Quoted 2 lines

sysfs
support for physical enclosure devices

and their

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

components.

Who is the target audience/user of those

facilities?

Quoted 1 line
Quoted 1 line
Quoted 1 line

a) The kernel itself needing to read/write

SES pages?

Quoted 2 lines
Quoted 2 lines

That depends on the enclosure integration, but

right at the

Quoted 7 lines
Quoted 2 lines

moment, it
doesn't

Yes, I didn't suspect so.

Quoted 2 lines
Quoted 2 lines

b) A user space application using sysfs to

read/write

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

SES pages?

Not an application so much as a user. The idea

of sysfs is

Quoted 2 lines
Quoted 2 lines

to allow
users to get and set the information in addition

to

Quoted 8 lines
Quoted 1 line

applications.

Exactly the same argument stands for a user-space application with a user-space library.

This is the classical case of where it is better to do this in user-space as opposed to the kernel.

Quoted 8 lines

The kernel provides capability to access the SES device. The user space application and library provide interpretation and control. Thus if the enclosure were upgraded, one doesn't need to upgrade their kernel in order to utilize the new capabilities of the SES device. Plus upgrading a user-space application is a lot easier than the kernel (and no reboot necessary).

Quoted 2 lines

The implementation is modular, so it's remove and
insert ...

I guess the same could be said for STGT and SCST, right?

LOL, no seriously, this is unnecessary kernel bloat, or rather at the wrong place (see below).

Quoted 11 lines
Quoted 3 lines

Consider another thing: vendors would really like unprecedented access to the SES device in the

enclosure

Quoted 4 lines

so as your ses/enclosure code keeps state it would get out of sync when vendor user-space enclosure applications access (and modify) the SES device's pages.

The state model doesn't assume nothing else will alter the state.

But it would be trivial exercise to show that an inconsistent state can be had by modifying pages of the SES device directly from userspace bypassing your implementation.

Quoted 17 lines
Quoted 6 lines

You can test this yourself: submit a patch that removes SES /dev/sgX support; advertise your ses/class solution and watch the fun.

Quoted 1 line
Quoted 1 line

At the moment SES device management is done

via

Quoted 1 line
Quoted 1 line
Quoted 1 line

an application (user-space) and a user-space

library

Quoted 1 line
Quoted 1 line
Quoted 1 line

used by the application and /dev/sgX to send

SCSI

Quoted 6 lines
Quoted 5 lines
Quoted 1 line

commands to the SES device.

I must have missed that when I was looking for
implementations; what's
the URL?

Quoted 7 lines
Quoted 5 lines

I'm not aware of any GPLed ones. That doesn't necessarily mean that the best course of action is to bloat the kernel. You can move your ses/enclosure stuff to a user space application library
and thus start a GPLed one.

Certainly ... patches welcome.

I've non at the moment, plus I don't think you'd be the point of contact for a user-space SES library. Unless of course you've already started something up on sourceforge.

Really, such an effort already exists: it is called sg_ses(8).

Quoted 11 lines
Quoted 2 lines
Quoted 2 lines

But, if we have non-scsi enclosures to integrate,

that

Quoted 2 lines
Quoted 2 lines

makes it harder
for a user application because it has to know all

the

Quoted 4 lines
Quoted 1 line

implementations.

So does the kernel. And as I pointed out above, it is a lot easier to upgrade a user-space application

and

Quoted 8 lines
Quoted 2 lines

library than it is to upgrade a new kernel and having to reboot the computer to run the new kernel.

No, think again ... it's easy for SES based enclosures because they have
a SCSI transport. We have no transport for SGPIO based enclosures nor
for any of the other more esoteric ones.

Yes, for which the transport layer, implements the scsi device node for the SES device. It doesn't really matter if the SCSI commands sent to the SES device go over SGPIO or FC or SAS or Bluetooth or I2C, etc, the transport layer can implement that and present the /dev/sgX node.

Case in point: the protocol FW running on the ASIC provides this capability so really the LLDD would only see a the pure SCSI SES or processor device and register that with the kernel. At which point no new kernel bloat is required.

Your code doesn't quite do that at the moment as it actually goes further in to read and present SES pages. Ideally it would simply provide capability for transport layers to register a SCSI device of type SES, or processor.

Architecturally, the LLDD/transport layer would register the SGPIO device on one end with the SGPIO layer and on the other end as a SCSI SES/processpr device. After that sg_ses(8) or sglib, fits the bill for user space applications.

Quoted 3 lines

That's not to say it can't be done, but it does
mean that it can't be
completely userspace.

See previous paragraph.

Quoted 29 lines
Quoted 2 lines
Quoted 2 lines

A sysfs framework on the other hand is a

universal known

Quoted 6 lines
Quoted 2 lines

thing for the
user applications.

So would a user-space ses library, a la libses.so.

Quoted 1 line
Quoted 1 line

One could have a very good argument to not

bloat

Quoted 1 line
Quoted 1 line
Quoted 1 line

the kernel with this but leave it to a

user-space

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

application and a library to do all this and
communicate with the SES device via the

kernel's

Quoted 3 lines
Quoted 3 lines

/dev/sgX.

The same thing goes for other esoteric SCSI

infrastructure

Quoted 2 lines
Quoted 2 lines

pieces like
cd changers. On the whole, given that ATA is

asking for

Quoted 2 lines
Quoted 2 lines

enclosure
management in kernel, it makes sense to

consolidate the

Quoted 4 lines
Quoted 2 lines

infrastructure
and a ses ULD is a very good test bed.

What is wrong with exporting the SES device as

Quoted 8 lines

/dev/sgX

Quoted 2 lines

and having a user-space application and library to do all this?

How do you transport the enclosure commands over /dev/sgX? Only SES has
SCSI command encapsulation ... the rest won't even be SCSI targets ...

What is the protocol of those "rest" that you talk about?

At any rate, this capability lies in the kernel providing a _device node_ -- not quite what your patch is doing.

Luben

author maintainer
#10 lore raw
Quoted 51 lines · Luben Tuikov
Quoted 50 lines · James Bottomley
Quoted 1 line · Luben Tuikov

--- On Mon, 2/4/08, James Bottomley

Quoted 1 line · <[email protected]> wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line

The enclosure misc device is really

just a

Quoted 3 lines
Quoted 3 lines

library providing

Quoted 2 lines
Quoted 2 lines

sysfs
support for physical enclosure devices

and their

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

components.

Who is the target audience/user of those

facilities?

Quoted 1 line
Quoted 1 line
Quoted 1 line

a) The kernel itself needing to read/write

SES pages?

Quoted 2 lines
Quoted 2 lines

That depends on the enclosure integration, but

right at the

Quoted 7 lines
Quoted 2 lines

moment, it
doesn't

Yes, I didn't suspect so.

Quoted 2 lines
Quoted 2 lines

b) A user space application using sysfs to

read/write

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

SES pages?

Not an application so much as a user. The idea

of sysfs is

Quoted 2 lines
Quoted 2 lines

to allow
users to get and set the information in addition

to

Quoted 16 lines
Quoted 1 line

applications.

Exactly the same argument stands for a user-space application with a user-space library.

This is the classical case of where it is better to do this in user-space as opposed to the kernel.

The kernel provides capability to access the SES device. The user space application and library provide interpretation and control. Thus if the enclosure were upgraded, one doesn't need to upgrade their kernel in order to utilize the new capabilities of the SES device. Plus upgrading a user-space application is a lot easier than the kernel (and no reboot necessary).

Quoted 4 lines
Quoted 2 lines

The implementation is modular, so it's remove and
insert ...

I guess the same could be said for STGT and SCST, right?

You mean both of their kernel pieces are modular? That's correct.

Quoted 12 lines

LOL, no seriously, this is unnecessary kernel bloat, or rather at the wrong place (see below).

Quoted 9 lines
Quoted 3 lines

Consider another thing: vendors would really like unprecedented access to the SES device in the

enclosure

Quoted 4 lines

so as your ses/enclosure code keeps state it would get out of sync when vendor user-space enclosure applications access (and modify) the SES device's pages.

Quoted 7 lines
Quoted 2 lines

The state model doesn't assume nothing else will alter the state.

But it would be trivial exercise to show that an inconsistent state can be had by modifying pages of the SES device directly from userspace bypassing your implementation.

I don't think so ... if you actually look at the code, you'll see it doesn't really have persistent state for the enclosure.

Quoted 52 lines
Quoted 16 lines
Quoted 5 lines

You can test this yourself: submit a patch that removes SES /dev/sgX support; advertise your ses/class solution and watch the fun.

Quoted 1 line
Quoted 1 line

At the moment SES device management is done

via

Quoted 1 line
Quoted 1 line
Quoted 1 line

an application (user-space) and a user-space

library

Quoted 1 line
Quoted 1 line
Quoted 1 line

used by the application and /dev/sgX to send

SCSI

Quoted 6 lines
Quoted 5 lines
Quoted 1 line

commands to the SES device.

I must have missed that when I was looking for
implementations; what's
the URL?

Quoted 7 lines
Quoted 5 lines

I'm not aware of any GPLed ones. That doesn't necessarily mean that the best course of action is to bloat the kernel. You can move your ses/enclosure stuff to a user space application library
and thus start a GPLed one.

Certainly ... patches welcome.

I've non at the moment, plus I don't think you'd be the point of contact for a user-space SES library. Unless of course you've already started something up on sourceforge.

Really, such an effort already exists: it is called sg_ses(8).

Quoted 11 lines
Quoted 2 lines
Quoted 2 lines

But, if we have non-scsi enclosures to integrate,

that

Quoted 2 lines
Quoted 2 lines

makes it harder
for a user application because it has to know all

the

Quoted 4 lines
Quoted 1 line

implementations.

So does the kernel. And as I pointed out above, it is a lot easier to upgrade a user-space application

and

Quoted 8 lines
Quoted 2 lines

library than it is to upgrade a new kernel and having to reboot the computer to run the new kernel.

No, think again ... it's easy for SES based enclosures because they have
a SCSI transport. We have no transport for SGPIO based enclosures nor
for any of the other more esoteric ones.

Quoted 6 lines

Yes, for which the transport layer, implements the scsi device node for the SES device. It doesn't really matter if the SCSI commands sent to the SES device go over SGPIO or FC or SAS or Bluetooth or I2C, etc, the transport layer can implement that and present the /dev/sgX node.

But it does matter if the enclosure device doesn't speak SCSI. SGPIO isn't a SCSI protocol ... it's a general purpose serial bus protocol. It's pretty simple and register based, but it might (or might not) be accessible via a SCSI bridge.

Quoted 10 lines

Case in point: the protocol FW running on the ASIC provides this capability so really the LLDD would only see a the pure SCSI SES or processor device and register that with the kernel. At which point no new kernel bloat is required.

Your code doesn't quite do that at the moment as it actually goes further in to read and present SES pages. Ideally it would simply provide capability for transport layers to register a SCSI device of type SES, or processor.

Yes, it provides a glue between the enclosure services and the SES protocol.

Quoted 4 lines

Architecturally, the LLDD/transport layer would register the SGPIO device on one end with the SGPIO layer and on the other end as a SCSI SES/processpr device. After that sg_ses(8) or sglib, fits the bill for user space applications.

That's possible, but none of these layers exist yet ... although I think (assuming I can find a SGPIO enclosure) that SGPIO might be next.

Quoted 39 lines
Quoted 3 lines

That's not to say it can't be done, but it does
mean that it can't be
completely userspace.

See previous paragraph.

Quoted 33 lines
Quoted 2 lines
Quoted 2 lines

A sysfs framework on the other hand is a

universal known

Quoted 6 lines
Quoted 2 lines

thing for the
user applications.

So would a user-space ses library, a la libses.so.

Quoted 1 line
Quoted 1 line

One could have a very good argument to not

bloat

Quoted 1 line
Quoted 1 line
Quoted 1 line

the kernel with this but leave it to a

user-space

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

application and a library to do all this and
communicate with the SES device via the

kernel's

Quoted 3 lines
Quoted 3 lines

/dev/sgX.

The same thing goes for other esoteric SCSI

infrastructure

Quoted 2 lines
Quoted 2 lines

pieces like
cd changers. On the whole, given that ATA is

asking for

Quoted 2 lines
Quoted 2 lines

enclosure
management in kernel, it makes sense to

consolidate the

Quoted 4 lines
Quoted 2 lines

infrastructure
and a ses ULD is a very good test bed.

What is wrong with exporting the SES device as

/dev/sgX

Quoted 2 lines

and having a user-space application and library to do all this?

Quoted 6 lines
Quoted 4 lines

How do you transport the enclosure commands over /dev/sgX? Only SES has
SCSI command encapsulation ... the rest won't even be SCSI targets ...

What is the protocol of those "rest" that you talk about?

At the moment it looks to be SES, SGPIO and AHCI.

Quoted 2 lines

At any rate, this capability lies in the kernel providing a _device node_ -- not quite what your patch is doing.

So your idea is to provide a separate interface per enclosure in kernel? Sure ... like I said patches welcome. I just did a common in-kernel interface that abstracts common enclosure services.

James

#11 lore raw
Quoted 73 lines · James Bottomley
Quoted 1 line

--- On Mon, 2/4/08, James Bottomley

Quoted 1 line · <[email protected]> wrote:
Quoted 1 line

On Mon, 2008-02-04 at 18:01 -0800, Luben Tuikov

Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 1 line

--- On Mon, 2/4/08, James Bottomley

<[email protected]>

Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

The enclosure misc device is

really

Quoted 4 lines
Quoted 4 lines

just a

Quoted 3 lines
Quoted 3 lines

library providing

Quoted 2 lines
Quoted 2 lines

sysfs
support for physical

enclosure devices

Quoted 4 lines
Quoted 4 lines

and their

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

components.

Who is the target audience/user of

those

Quoted 2 lines
Quoted 2 lines

facilities?

Quoted 1 line
Quoted 1 line
Quoted 1 line

a) The kernel itself needing to

read/write

Quoted 3 lines
Quoted 3 lines

SES pages?

Quoted 2 lines
Quoted 2 lines

That depends on the enclosure

integration, but

Quoted 8 lines
Quoted 8 lines

right at the

Quoted 7 lines
Quoted 2 lines

moment, it
doesn't

Yes, I didn't suspect so.

Quoted 2 lines
Quoted 2 lines

b) A user space application using

sysfs to

Quoted 4 lines
Quoted 4 lines

read/write

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

SES pages?

Not an application so much as a user.

The idea

Quoted 3 lines
Quoted 3 lines

of sysfs is

Quoted 2 lines
Quoted 2 lines

to allow
users to get and set the information in

addition

Quoted 4 lines
Quoted 4 lines

to

Quoted 3 lines
Quoted 1 line

applications.

Exactly the same argument stands for a

user-space

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

application with a user-space library.

This is the classical case of where it is

better to

Quoted 1 line
Quoted 1 line
Quoted 1 line

do this in user-space as opposed to the

kernel.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

The kernel provides capability to access the

SES

Quoted 1 line
Quoted 1 line
Quoted 1 line

device. The user space application and

library

Quoted 1 line
Quoted 1 line
Quoted 1 line

provide interpretation and control. Thus if

the

Quoted 1 line
Quoted 1 line
Quoted 1 line

enclosure were upgraded, one doesn't

need to

Quoted 1 line
Quoted 1 line
Quoted 1 line

upgrade their kernel in order to utilize the

new

Quoted 1 line
Quoted 1 line
Quoted 1 line

capabilities of the SES device. Plus

upgrading

Quoted 1 line
Quoted 1 line
Quoted 1 line

a user-space application is a lot easier

than

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

the kernel (and no reboot necessary).

The implementation is modular, so it's remove

Quoted 8 lines

and

Quoted 3 lines
Quoted 1 line

insert ...

I guess the same could be said for STGT and SCST,

right?

You mean both of their kernel pieces are modular?
That's correct.

No, you know very well what I mean.

By the same logic you're preaching to include your solution part of the kernel, you can also apply to SCST.

Quoted 102 lines
Quoted 5 lines

LOL, no seriously, this is unnecessary kernel bloat, or rather at the wrong place (see below).

Quoted 2 lines
Quoted 2 lines

Consider another thing: vendors would really

like

Quoted 1 line
Quoted 1 line
Quoted 1 line

unprecedented access to the SES device in

the

Quoted 2 lines
Quoted 2 lines

enclosure

Quoted 1 line

so as your ses/enclosure code keeps state it

would

Quoted 1 line
Quoted 1 line
Quoted 1 line

get out of sync when vendor user-space

enclosure

Quoted 1 line
Quoted 1 line
Quoted 1 line

applications access (and modify) the SES

device's

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

pages.

The state model doesn't assume nothing else

will alter

Quoted 6 lines
Quoted 1 line

the state.

But it would be trivial exercise to show that an inconsistent state can be had by modifying pages of the SES device directly from userspace bypassing your implementation.

I don't think so ... if you actually look at the code, you'll see it
doesn't really have persistent state for the enclosure.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

You can test this yourself: submit a patch
that removes SES /dev/sgX support; advertise

your

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

ses/class solution and watch the fun.

Quoted 1 line
Quoted 1 line

At the moment SES device

management is done

Quoted 2 lines
Quoted 2 lines

via

Quoted 1 line
Quoted 1 line
Quoted 1 line

an application (user-space) and a

user-space

Quoted 2 lines
Quoted 2 lines

library

Quoted 1 line
Quoted 1 line
Quoted 1 line

used by the application and

/dev/sgX to send

Quoted 4 lines
Quoted 4 lines

SCSI

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

commands to the SES device.

I must have missed that when I was

looking for

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

implementations; what's
the URL?

I'm not aware of any GPLed ones. That

doesn't

Quoted 1 line
Quoted 1 line
Quoted 1 line

necessarily mean that the best course of

action is

Quoted 1 line
Quoted 1 line
Quoted 1 line

to bloat the kernel. You can move your

ses/enclosure

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

stuff to a user space application library
and thus start a GPLed one.

Certainly ... patches welcome.

I've non at the moment, plus I don't think

you'd be

Quoted 2 lines

the point of contact for a user-space SES library. Unless of course you've already started something

up

Quoted 7 lines

on sourceforge.

Really, such an effort already exists: it is called sg_ses(8).

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

But, if we have non-scsi enclosures to

integrate,

Quoted 3 lines
Quoted 3 lines

that

Quoted 2 lines
Quoted 2 lines

makes it harder
for a user application because it has

to know all

Quoted 4 lines
Quoted 4 lines

the

Quoted 3 lines
Quoted 1 line

implementations.

So does the kernel. And as I pointed out

above, it

Quoted 1 line
Quoted 1 line
Quoted 1 line

is a lot easier to upgrade a user-space

application

Quoted 2 lines
Quoted 2 lines

and

Quoted 1 line

library than it is to upgrade a new kernel

and having

Quoted 1 line
Quoted 1 line
Quoted 1 line

to reboot the computer to run the new

kernel.

Quoted 2 lines
Quoted 2 lines

No, think again ... it's easy for SES based

enclosures

Quoted 2 lines
Quoted 2 lines

because they have
a SCSI transport. We have no transport for SGPIO

based

Quoted 5 lines
Quoted 2 lines

enclosures nor
for any of the other more esoteric ones.

Yes, for which the transport layer, implements the scsi device node for the SES device. It doesn't

Quoted 8 lines

really

Quoted 4 lines

matter if the SCSI commands sent to the SES device go over SGPIO or FC or SAS or Bluetooth or I2C, etc, the transport layer can implement that and present the /dev/sgX node.

But it does matter if the enclosure device doesn't speak SCSI.

Enclosure management isn't as simple as you're portraying it here. The enclosure management device speaks either SES or SAF-TE. The transport protocol to access it could be SGPIO or I2C or...

Quoted 6 lines

SGPIO
isn't a SCSI protocol ... it's a general purpose serial bus protocol.
It's pretty simple and register based, but it might (or might not) be
accessible via a SCSI bridge.

I see. You've just discovered SGPIO -- good for you.

At any rate, I told you already that what is needed is not what you've provided but a _device node_ exported by the kernel, either a processor or enclosure type.

Quoted 10 lines
Quoted 7 lines

Case in point: the protocol FW running on the ASIC provides this capability so really the LLDD would only see a the pure SCSI SES or processor device and register that with the kernel. At which point no new kernel bloat is required.

Your code doesn't quite do that at the moment as

it

Quoted 1 line

actually goes further in to read and present SES

pages.

Quoted 8 lines
Quoted 1 line

Ideally it would simply provide capability for

transport

Quoted 1 line

layers to register a SCSI device of type SES, or

processor.

Yes, it provides a glue between the enclosure services and the SES
protocol.

You're jumping over layers here.

And that's not what's needed. What is needed is an instrumentation by the kernel to present the enclosure device as a device node to user space so that general utilities tools, like for example sg_ses(8) can work with it.

Your solution seems to be interpreting SES in the kernel and this is really unwieldy.

Just provide the device node to user space. You shouldn't have to interpret SES in the kernel as you're currently doing since the kernel itself has no use of it.

Quoted 5 lines
Quoted 1 line

Architecturally, the LLDD/transport layer would

register

Quoted 1 line

the SGPIO device on one end with the SGPIO layer and

on

Quoted 1 line

the other end as a SCSI SES/processpr device. After

Quoted 8 lines

that

Quoted 1 line

sg_ses(8) or sglib, fits the bill for user space

applications.

That's possible, but none of these layers exist yet ... although I think
(assuming I can find a SGPIO enclosure) that SGPIO might be next.

Vendors prefer to abstract the transport protocol in HW/FW so that to ULD the device is of type processor or enclosure.

This makes it easy for the vendor (in terms of support) and for the customer (in terms of usability).

Quoted 55 lines
Quoted 2 lines
Quoted 2 lines

That's not to say it can't be done, but

it does

Quoted 7 lines
Quoted 2 lines

mean that it can't be
completely userspace.

See previous paragraph.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

A sysfs framework on the other hand is

a

Quoted 5 lines
Quoted 5 lines

universal known

Quoted 4 lines
Quoted 2 lines

thing for the
user applications.

So would a user-space ses library, a la

libses.so.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

One could have a very good

argument to not

Quoted 2 lines
Quoted 2 lines

bloat

Quoted 1 line
Quoted 1 line
Quoted 1 line

the kernel with this but leave it

to a

Quoted 2 lines
Quoted 2 lines

user-space

Quoted 1 line
Quoted 1 line
Quoted 1 line

application and a library to do

all this and

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

communicate with the SES device

via the

Quoted 4 lines
Quoted 4 lines

kernel's

Quoted 3 lines
Quoted 3 lines

/dev/sgX.

The same thing goes for other esoteric

SCSI

Quoted 3 lines
Quoted 3 lines

infrastructure

Quoted 2 lines
Quoted 2 lines

pieces like
cd changers. On the whole, given that

ATA is

Quoted 8 lines
Quoted 8 lines

asking for

Quoted 2 lines
Quoted 2 lines

enclosure
management in kernel, it makes sense to

consolidate the

Quoted 4 lines
Quoted 2 lines

infrastructure
and a ses ULD is a very good test bed.

What is wrong with exporting the SES device

as

Quoted 2 lines
Quoted 2 lines

/dev/sgX

Quoted 1 line

and having a user-space application and

library to

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

do all this?

How do you transport the enclosure commands over

/dev/sgX?

Quoted 2 lines
Quoted 2 lines

Only SES has
SCSI command encapsulation ... the rest won't

Quoted 7 lines

even be

Quoted 3 lines
Quoted 1 line

SCSI targets ...

What is the protocol of those "rest" that

you talk about?

At the moment it looks to be SES, SGPIO and AHCI.

You should separate "what it talks" by "how it gets there".

Quoted 7 lines
Quoted 2 lines

At any rate, this capability lies in the kernel

providing

Quoted 1 line

a _device node_ -- not quite what your patch is doing.

So your idea is to provide a separate interface per enclosure in kernel?

No. My idea is to provide a device node of type processor or enclosure for each enclosure.

Expanders already do this by providing a virtual phy which is "connected" to the SES-2 device, so a device of type SES is discovered and registered with the kernel as any other device on the domain. Then the enclosure can be controlled via sg_ses(8) from user space.

Host adapters already do this in their FW, since it is there where they do the transport protocol abstraction as well.

You can help AHCI (if this really is needed of course) to achieve the same thing by providing code so that a LLDD/etc can register a processor/enclosure device accessible via the ASIC. At any rate the actual protocol to talk to the enclosure device is abstracted away by the HW solution (either in FW or HW).

At this moment I see all this effort in vain for reasons I've described above.

Quoted 3 lines

Sure ... like I said patches welcome. I just did a common in-kernel
interface that abstracts common enclosure services.

I know -- this is why and where I responded to your patches.

So I guess by what you just said, we're back at square one? ... is this really needed, who is the intended user/audience... why not just a device node to be controlled via sg_ses(8) from user space, as is already done for other enclosure devices... etc.

Luben

author maintainer
#12 lore raw
Quoted 11 lines · Luben Tuikov
Quoted 5 lines
Quoted 1 line

I guess the same could be said for STGT and SCST,

right?

You mean both of their kernel pieces are modular?
That's correct.

No, you know very well what I mean.

By the same logic you're preaching to include your solution part of the kernel, you can also apply to SCST.

Ah, but it's not ... the current patch is merely exporting an interface. The debate in STGT vs SCST is not whether to export an interface but where to draw the line.

You could also argue in the same vein that sd is redundant because a filesystem could talk directly to the device via /dev/sgX (in fact OSD based filesystems already do this). The argument is true, but misses the bigger picture that the interfaces exported by sd are more portable (apply to non-SCSI block devices) and easier to use.

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

Yes, for which the transport layer, implements the scsi device node for the SES device. It doesn't

really

Quoted 4 lines

matter if the SCSI commands sent to the SES device go over SGPIO or FC or SAS or Bluetooth or I2C, etc, the transport layer can implement that and present the /dev/sgX node.

Quoted 7 lines
Quoted 2 lines

But it does matter if the enclosure device doesn't speak SCSI.

Enclosure management isn't as simple as you're portraying it here. The enclosure management device speaks either SES or SAF-TE. The transport protocol to access it could be SGPIO or I2C or...

Look, just read the spec; SGPIO is a bus for driving enclosures ... it doesn't require SES or SAF-TE or even any SCSI protocol.

Quoted 7 lines
Quoted 6 lines

SGPIO
isn't a SCSI protocol ... it's a general purpose serial bus protocol.
It's pretty simple and register based, but it might (or might not) be
accessible via a SCSI bridge.

Quoted 6 lines

I see. You've just discovered SGPIO -- good for you.

At any rate, I told you already that what is needed is not what you've provided but a _device node_ exported by the kernel, either a processor or enclosure type.

Wrong ... we don't export non-SCSI devices as SCSI (with the single and rather annoying exception of ATA via SAT).

James

#13 lore raw
Quoted 14 lines · James Bottomley
Quoted 1 line
Quoted 1 line
Quoted 1 line

I guess the same could be said for STGT and

SCST,

Quoted 11 lines
Quoted 5 lines

right?

You mean both of their kernel pieces are modular?

That's correct.

No, you know very well what I mean.

By the same logic you're preaching to include your solution part of the kernel, you can also apply to SCST.

Quoted 5 lines

Ah, but it's not ... the current patch is merely exporting an interface.
The debate in STGT vs SCST is not whether to export an interface but
where to draw the line.

"draw the line" -- I see.
BTW, what is wrong with "exporting the interface"?

What is wrong if both implementations are in the kernel and then let the users and distros decide which one they like best and use more? It'll not be the fist time this has happened in the kernel. Both are actively maintained.

It seems highly arbitrary to say: "X is in the kernel, Y is not. If you want Y, just forget about it and fix X." Give people choice at config time.

This is off topic anyway.

Quoted 5 lines

You could also argue in the same vein that sd is redundant because a
filesystem could talk directly to the device via /dev/sgX (in fact OSD
based filesystems already do this).

Yes, I've mentioned this thing before on this list. Oh, maybe 3 years ago. This is why I had wanted for transport protocols to export ... (oh, let's not get this off topic).

(Apparently it takes 3 years...)

Quoted 5 lines

The argument is true,
but misses
the bigger picture that the interfaces exported by sd are more portable
(apply to non-SCSI block devices) and easier to use.

It isn't quite the same thing. It's like comparing apples to oranges.

Quoted 18 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Yes, for which the transport layer,

implements the

Quoted 1 line
Quoted 1 line
Quoted 1 line

scsi device node for the SES device. It

doesn't

Quoted 2 lines
Quoted 2 lines

really

Quoted 1 line

matter if the SCSI commands sent to the SES

device go

Quoted 1 line
Quoted 1 line
Quoted 1 line

over SGPIO or FC or SAS or Bluetooth or I2C,

etc, the

Quoted 1 line
Quoted 1 line
Quoted 1 line

transport layer can implement that and

present the

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

/dev/sgX node.

But it does matter if the enclosure device

doesn't

Quoted 2 lines
Quoted 1 line

speak SCSI.

Quoted 7 lines
Quoted 4 lines

Enclosure management isn't as simple as you're portraying it here. The enclosure management device speaks either SES or SAF-TE. The transport protocol to access it could be SGPIO or I2C or...

Look, just read the spec; SGPIO is a bus for driving enclosures ...

I thought Serial General Purpose Input Output (SGPIO) was a method to serialize general purpose IO signals.

Quoted 3 lines

it
doesn't require SES or SAF-TE or even any SCSI
protocol.

That's true. And this is why I mentioned a couple of emails ago to simply export a sgpio device node *IF* this is what is needed. Of course devices that use SGPIO abstract it away for their functional purpose, e.g. enclosures, LED, etc, and provide a more general way to control it -- highly hardware specific on one side.

Your abstraction currently deals with "SES" devices and I'd rather leave that to user-space. Alternatively, which I presume is what you're thinking, a HW specific core would be using your "abstraction" to provide some unified access to raw features, and that "unified access" isn't defined anywhere, and would likely not be. Alternatively that "unified access" is things like SES and SAF-TE, which is what vendors prefer to export, or they prefer to drive this directly via other means.

That is, I fail to see the kernel bloat, for things that aren't necessary in the kernel.

If you want your abstraction to fly, it first needs a common usage model to abstract, and the latter is missing _from the kernel_.

Unless I don't know the details and you've been asked to implement this for a single vendor's HW solution.

Quoted 12 lines
Quoted 2 lines
Quoted 2 lines

SGPIO
isn't a SCSI protocol ... it's a general

purpose

Quoted 2 lines
Quoted 2 lines

serial bus protocol.
It's pretty simple and register based, but it

might (or

Quoted 4 lines
Quoted 2 lines

might not) be
accessible via a SCSI bridge.

I see. You've just discovered SGPIO -- good for

you.

Quoted 1 line
Quoted 8 lines
Quoted 4 lines

At any rate, I told you already that what is needed is not what you've provided but a _device node_ exported by the kernel, either a processor or enclosure type.

Wrong ... we don't export non-SCSI devices as SCSI (with the single and
rather annoying exception of ATA via SAT).

I didn't say you should do that. I had already mentioned that vendors export such controls
as either enclosure or processor type devices, and this is why I told you that that is what needs to be exported, which incidentally is
a device node of that type.

Without a common usage model already in the kernel to abstract (e.g. sd for block device, since you brought that up) your abstraction seems redundant and arbitrary.

Your kernel code already uses READ DIAGNOSTIC, etc, and I'd rather leave that to user-space.

Luben

author maintainer
#14 lore raw
Quoted 11 lines · Luben Tuikov
Quoted 3 lines

Wrong ... we don't export non-SCSI devices as SCSI (with the single and
rather annoying exception of ATA via SAT).

I didn't say you should do that. I had already mentioned that vendors export such controls
as either enclosure or processor type devices, and this is why I told you that that is what needs to be exported, which incidentally is
a device node of that type.

Quoted 3 lines

Without a common usage model already in the kernel to abstract (e.g. sd for block device, since you brought that up) your abstraction seems redundant and arbitrary.

Exactly, so the first patch in this series (a while ago now) was a common usage model abstraction of enclosures, and the second was an
implementation in terms of SES. I will do one in terms of SGPIO as
well ... assuming I ever find a SGPIO enclosure ...

Quoted 2 lines

Your kernel code already uses READ DIAGNOSTIC, etc, and I'd rather leave that to user-space.

You can do it in user space as well. It's just a bit difficult to get information out of a SES enclosure without using it, and getting some of the information is a requirement of the abstraction.

James

#15 lore raw
Quoted 12 lines · James Bottomley
Quoted 1 line
Quoted 1 line

Wrong ... we don't export non-SCSI devices as

SCSI

Quoted 10 lines
Quoted 2 lines

(with the single and
rather annoying exception of ATA via SAT).

I didn't say you should do that. I had already mentioned that vendors export such controls as either enclosure or processor type devices, and this is why I told you that that is what needs to be exported, which incidentally is a device node of that type.

Quoted 7 lines
Quoted 2 lines

Without a common usage model already in the kernel to abstract (e.g. sd for block device, since you

brought

Quoted 1 line

that up) your abstraction seems redundant and

arbitrary.

Exactly, so the first patch in this series (a while ago

^^^^^^^^^^^

See last paragraph.

Quoted 6 lines

now) was a
common usage model abstraction of enclosures, and the second was an
implementation in terms of SES. I will do one in terms of
SGPIO as
well ... assuming I ever find a SGPIO enclosure ...

The vendor would've abstracted that away most commonly using SES.

Quoted 9 lines
Quoted 3 lines

Your kernel code already uses READ DIAGNOSTIC, etc, and I'd rather leave that to user-space.

You can do it in user space as well. It's just a bit difficult to get
information out of a SES enclosure without using it, and getting some of
the information is a requirement of the abstraction.

You missed my point. Your abstraction is redundant and arbitrary -- it is not based on any known, in-practice, usage model, already in place that needs a better, common way of doing XYZ, and therefore needs an abstraction.

Luben

#16 lore raw
In reply to Luben Tuikov
Quoted 97 lines · On Mon, 4 Feb 2008 18:01:36 -0800 (PST) Luben Tuikov <[email protected]> wrote:

--- On Mon, 2/4/08, James Bottomley

Quoted 6 lines · <[email protected]> wrote:
Quoted 1 line
Quoted 1 line

The enclosure misc device is really just a

library providing

Quoted 4 lines
Quoted 3 lines

sysfs
support for physical enclosure devices and their
components.

Quoted 6 lines
Quoted 2 lines

Who is the target audience/user of those facilities?
a) The kernel itself needing to read/write SES pages?

That depends on the enclosure integration, but right at the moment, it
doesn't

Yes, I didn't suspect so.

Quoted 8 lines
Quoted 3 lines

b) A user space application using sysfs to read/write SES pages?

Not an application so much as a user. The idea of sysfs is to allow
users to get and set the information in addition to applications.

Exactly the same argument stands for a user-space application with a user-space library.

This is the classical case of where it is better to do this in user-space as opposed to the kernel.

The kernel provides capability to access the SES device. The user space application and library provide interpretation and control. Thus if the enclosure were upgraded, one doesn't need to upgrade their kernel in order to utilize the new capabilities of the SES device. Plus upgrading a user-space application is a lot easier than the kernel (and no reboot necessary).

Consider another thing: vendors would really like unprecedented access to the SES device in the enclosure so as your ses/enclosure code keeps state it would get out of sync when vendor user-space enclosure applications access (and modify) the SES device's pages.

You can test this yourself: submit a patch that removes SES /dev/sgX support; advertise your ses/class solution and watch the fun.

Quoted 8 lines
Quoted 4 lines

At the moment SES device management is done via an application (user-space) and a user-space library used by the application and /dev/sgX to send SCSI commands to the SES device.

I must have missed that when I was looking for
implementations; what's
the URL?

I'm not aware of any GPLed ones. That doesn't necessarily mean that the best course of action is to bloat the kernel. You can move your ses/enclosure stuff to a user space application library
and thus start a GPLed one.

Quoted 4 lines

But, if we have non-scsi enclosures to integrate, that makes it harder
for a user application because it has to know all the implementations.

So does the kernel. And as I pointed out above, it is a lot easier to upgrade a user-space application and library than it is to upgrade a new kernel and having to reboot the computer to run the new kernel.

Quoted 3 lines

A sysfs framework on the other hand is a universal known thing for the
user applications.

So would a user-space ses library, a la libses.so.

Quoted 6 lines
Quoted 4 lines

One could have a very good argument to not bloat the kernel with this but leave it to a user-space application and a library to do all this and communicate with the SES device via the kernel's

/dev/sgX.

Quoted 7 lines

The same thing goes for other esoteric SCSI infrastructure pieces like
cd changers. On the whole, given that ATA is asking for enclosure
management in kernel, it makes sense to consolidate the infrastructure
and a ses ULD is a very good test bed.

Quoted 6 lines

What is wrong with exporting the SES device as /dev/sgX and having a user-space application and library to do all this?

Luben

Hi,
I apologize for taking so long to review this patch. I obviously agree wholeheartedly with Luben. The problem I ran into while trying to design an enclosure management interface for the SATA devices is that there is all this vendor defined stuff. For example, for the AHCI LED protocol, the only "defined" LED is 'activity'. For LED2 and LED3 it is up to hardware vendors to define these. For SGPIO there's all kinds of ways for hw vendors to customize. I felt that it was going to be a maintainance nightmare to have to keep track of various vendors enclosure implementations in the ahci driver, and that it'd be better to just have user space libraries take care of that. Plus, that way a vendor doesn't have to get a patch into the kernel to get their new spiffy wizzy bang blinky lights working (think of how long it takes something to even get into a vendor kernel, which is what these guys care about...). So I'm still not sold on having an enclosure abstraction in the kernel - at least for the SATA controllers.

Kristen

author maintainer
#17 lore raw
Quoted 7 lines · Kristen Carlson Accardi

I apologize for taking so long to review this patch. I obviously agree wholeheartedly with Luben. The problem I ran into while trying to design an enclosure management interface for the SATA devices is that there is all this vendor defined stuff. For example, for the AHCI LED protocol, the only "defined" LED is 'activity'. For LED2 and LED3 it is up to hardware vendors to define these. For SGPIO there's all kinds of ways for hw vendors to customize. I felt that it was going to be a

Quoted 8 lines

maintainance nightmare to have to keep track of various vendors enclosure implementations in the ahci driver, and that it'd be better to just have user space libraries take care of that. Plus, that way a vendor doesn't have to get a patch into the kernel to get their new spiffy wizzy bang blinky lights working (think of how long it takes something to even get into a vendor kernel, which is what these guys care about...). So I'm still not sold on having an enclosure abstraction in the kernel - at least for the SATA controllers.

Correct me if I'm wrong, but didn't the original AHCI enclosure patch expose activity LEDs via sysfs?

I'm not saying there aren't a lot of non standard pieces that need to be activated by direct commands or other user activated protocol. I am saying there are a lot of standard pieces that we could do with showing in a uniform manner.

The pieces I think are absolutely standard are

1. Actual enclosure presence (is this device in an enclosure)
2. Activity LED, this seems to be a feature of every enclosure.

I also think the following are reasonably standard (based on the fact that most enclosure standards recommend but don't require this):

3. Locate LED (for locating the device). Even if you only have an activity LED, this is usually done by flashing the activity LED in a well defined pattern.
4. Fault. this is the least standardised of the lot, but does seem to be present in about every enclosure implementation.

All I've done is standardise these four pieces ... the services actually take into account that it might not be possible to do certain of these (like fault).

James

#18 lore raw
Quoted 18 lines · On Tue, 12 Feb 2008 12:45:35 -0600 James Bottomley…
Quoted 8 lines · Kristen Carlson Accardi

I apologize for taking so long to review this patch. I obviously agree wholeheartedly with Luben. The problem I ran into while trying to design an enclosure management interface for the SATA devices is that there is all this vendor defined stuff. For example, for the AHCI LED protocol, the only "defined" LED is 'activity'. For LED2 and LED3 it is up to hardware vendors to define these. For SGPIO there's all kinds of ways for hw vendors to customize. I felt that it was going to be a maintainance

Quoted 8 lines

nightmare to have to keep track of various vendors enclosure implementations in the ahci driver, and that it'd be better to just have user space libraries take care of that. Plus, that way a vendor doesn't have to get a patch into the kernel to get their new spiffy wizzy bang blinky lights working (think of how long it takes something to even get into a vendor kernel, which is what these guys care about...). So I'm still not sold on having an enclosure abstraction in the kernel - at least for the SATA controllers.

Quoted 2 lines

Correct me if I'm wrong, but didn't the original AHCI enclosure patch expose activity LEDs via sysfs?

You are sort of wrong. we exposed a sysfs entry to enable sofware controlled activity LED, then the driver was responsible for turning it on and off. (blech, I know, but some vendors want this feature).

Quoted 20 lines

I'm not saying there aren't a lot of non standard pieces that need to be activated by direct commands or other user activated protocol. I am saying there are a lot of standard pieces that we could do with showing in a uniform manner.

The pieces I think are absolutely standard are

1. Actual enclosure presence (is this device in an enclosure)
2. Activity LED, this seems to be a feature of every enclosure.

I also think the following are reasonably standard (based on the fact that most enclosure standards recommend but don't require this):

3. Locate LED (for locating the device). Even if you only have an activity LED, this is usually done by flashing the activity LED in a well defined pattern.
4. Fault. this is the least standardised of the lot, but does seem to be present in about every enclosure implementation.

Quoted 7 lines

All I've done is standardise these four pieces ... the services actually take into account that it might not be possible to do certain of these (like fault).

James

I understand what you are trying to do - I guess I just doubt the value you've added by doing this. I think that there's going to be so much customization that system vendors will want to add, that they are going to wind up adding a custom library regardless, so standardising those few things won't buy us anything.

author maintainer
#19 lore raw
Quoted 5 lines · Kristen Carlson Accardi

I understand what you are trying to do - I guess I just doubt the value you've added by doing this. I think that there's going to be so much customization that system vendors will want to add, that they are going to wind up adding a custom library regardless, so standardising those few things won't buy us anything.

It depends ... if you actually have a use for the customisations, yes. If you just want the basics of who (what's in the enclousure), what (activity) and where (locate) then I think it solves your problem almost entirely.

So, entirely as a straw horse, tell me what else your enclosures provide that I haven't listed in the four points. The SES standards too provide a huge range of things that no-one ever seems to implement (temperature, power, fan speeds etc).

I think the users of enclosures fall int these categories

85% just want to know where their device actually is (i.e. that sdc is in enclosure slot 5)
50% like watching the activity lights
30% want to be able to have a visual locate function
20% want a visual failure indication (the other 80% rely on some OS notification instead)

When you add up the overlapping needs, you get about 90% of people happy with the basics that the enclosure services provide. Could there be more ... sure; should there be more ... I don't think so ... that's what value add the user libraries can provide.

James

#20 lore raw
Quoted 26 lines · On Tue, 12 Feb 2008 13:28:15 -0600 James Bottomley…
Quoted 5 lines · Kristen Carlson Accardi

I understand what you are trying to do - I guess I just doubt the value you've added by doing this. I think that there's going to be so much customization that system vendors will want to add, that they are going to wind up adding a custom library regardless, so standardising those few things won't buy us anything.

It depends ... if you actually have a use for the customisations, yes. If you just want the basics of who (what's in the enclousure), what (activity) and where (locate) then I think it solves your problem almost entirely.

So, entirely as a straw horse, tell me what else your enclosures provide that I haven't listed in the four points. The SES standards too provide a huge range of things that no-one ever seems to implement (temperature, power, fan speeds etc).

I think the users of enclosures fall int these categories

85% just want to know where their device actually is (i.e. that sdc is in enclosure slot 5)
50% like watching the activity lights
30% want to be able to have a visual locate function
20% want a visual failure indication (the other 80% rely on some OS notification instead)

Quoted 8 lines

When you add up the overlapping needs, you get about 90% of people happy with the basics that the enclosure services provide. Could there be more ... sure; should there be more ... I don't think so ... that's what value add the user libraries can provide.

James

I don't think I'm arguing whether or not your solution may work, what I am arguing is really a more philosophical point. Not "can we do it this way", but "should we do it way". I am of the opinion that management belongs in userspace. I also am of the opinion that if you can successfully accomplish something in user space, you should. I also believe that even if you provide this basic interface, all system vendors are going to provide libraries on top of that to customize it, so you've not added much value to just a simple message passing interface.

So, I'm happy to defer to Jeff's judgement call here - I just want to do what's right for our customers and get an enclosure management interface for SATA exposed, preferrably in time for the 2.6.26 merge window. If he prefers your design, I'll disagree, but commit to his decision and try to get this to work for SATA. If he'd rather see something along the lines of what I proposed, then since it is 100% self contained in the SATA subsystem, it shouldn't impact whatever you want to do in the SCSI subsystem.

Jeff?

author maintainer
#21 lore raw
Quoted 9 lines · Kristen Carlson Accardi

I don't think I'm arguing whether or not your solution may work, what I am arguing is really a more philosophical point. Not "can we do it this way", but "should we do it way". I am of the opinion that management belongs in userspace. I also am of the opinion that if you can successfully accomplish something in user space, you should. I also believe that even if you provide this basic interface, all system vendors are going to provide libraries on top of that to customize it, so you've not added much value to just a simple message passing interface.

I'm not necessarily arguing against that. However, what you're providing is slightly more than just a userspace tap into the enclosure. You're adding a file to display and control the enclosure state (sw_activity). This constitutes an ad-hoc sysfs interface. I'm not telling you not to do it, but I am pleading that if we have to have all these sysfs interfaces, lets at least do it in a uniform way.

Enclosures are such nasty beasts, that even the job of getting a tap into them is problematic, so if we have a different tap infrastructure for every different enclosure type and connection it's still going to be pretty unmanageable to a userspace interface.

Quoted 8 lines

So, I'm happy to defer to Jeff's judgement call here - I just want to do what's right for our customers and get an enclosure management interface for SATA exposed, preferrably in time for the 2.6.26 merge window. If he prefers your design, I'll disagree, but commit to his decision and try to get this to work for SATA. If he'd rather see something along the lines of what I proposed, then since it is 100% self contained in the SATA subsystem, it shouldn't impact whatever you want to do in the SCSI subsystem.

James

#22 lore raw
In reply to Kristen Carlson Accardi
Quoted 34 lines · Kristen Carlson Accardi
Quoted 31 lines · On Tue, 12 Feb 2008 13:28:15 -0600 James Bottomley…
Quoted 5 lines · Kristen Carlson Accardi

I understand what you are trying to do - I guess I just doubt the value you've added by doing this. I think that there's going to be so much customization that system vendors will want to add, that they are going to wind up adding a custom library regardless, so standardising those few things won't buy us anything.

It depends ... if you actually have a use for the customisations, yes. If you just want the basics of who (what's in the enclousure), what (activity) and where (locate) then I think it solves your problem almost entirely.

So, entirely as a straw horse, tell me what else your enclosures provide that I haven't listed in the four points. The SES standards too provide a huge range of things that no-one ever seems to implement (temperature, power, fan speeds etc).

I think the users of enclosures fall int these categories

85% just want to know where their device actually is (i.e. that sdc is in enclosure slot 5)
50% like watching the activity lights
30% want to be able to have a visual locate function
20% want a visual failure indication (the other 80% rely on some OS notification instead)

When you add up the overlapping needs, you get about 90% of people happy with the basics that the enclosure services provide. Could there be more ... sure; should there be more ... I don't think so ... that's what value add the user libraries can provide.

Quoted 7 lines
Quoted 3 lines

James

I don't think I'm arguing whether or not your solution may work, what I am arguing is really a more philosophical point. Not "can we do it this way", but "should we do it way". I am of the opinion that

Hw abstraction is still kernel's job. That's why we have leds exported in sysfs... let vendors have their libraries, but lets put the 'everyone does these' stuff in kernel.

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#23 lore raw
In reply to James Bottomley
Quoted 35 lines · --- On Tue, 2/12/08, James Bottomley [email protected]>…
Quoted 1 line

I apologize for taking so long to review this patch.

I obviously agree

Quoted 1 line

wholeheartedly with Luben. The problem I ran into

while trying to

Quoted 1 line

design an enclosure management interface for the SATA

devices is that

Quoted 1 line

there is all this vendor defined stuff. For example,

for the AHCI LED

Quoted 1 line

protocol, the only "defined" LED is

'activity'. For LED2 and LED3 it

Quoted 1 line

is up to hardware vendors to define these. For SGPIO

there's all kinds

Quoted 1 line

of ways for hw vendors to customize. I felt that it

was going to be a

Quoted 1 line

maintainance nightmare to have to keep track of

various vendors

Quoted 1 line

enclosure implementations in the ahci driver, and that

it'd be better

Quoted 1 line

to just have user space libraries take care of that.

Plus, that way a

Quoted 1 line

vendor doesn't have to get a patch into the kernel

to get their new

Quoted 1 line

spiffy wizzy bang blinky lights working (think of how

long it takes

Quoted 1 line

something to even get into a vendor kernel, which is

what these guys

Quoted 1 line

care about...). So I'm still not sold on having

an enclosure

Quoted 1 line

abstraction in the kernel - at least for the SATA

controllers.

Correct me if I'm wrong, but didn't the original
AHCI enclosure patch
expose activity LEDs via sysfs?

Quoted 7 lines

I'm not saying there aren't a lot of non standard pieces that need to be
activated by direct commands or other user activated protocol. I am
saying there are a lot of standard pieces that we could do with showing
in a uniform manner.

Which is already the case without the SES kernel bloat. Case in point, the excellent user-space application "lsscsi" would clearly show which device is SES.

And the excellent user-space application "sg_ses" could control an SES device.

Quoted 4 lines

The pieces I think are absolutely standard are

1. Actual enclosure presence (is this device in an enclosure)

"lsscsi"

Quoted 2 lines

2. Activity LED, this seems to be a feature of every enclosure.

So that means that it needs a kernel representation? If this indeed were the case, for every "feature" of every type of device (not only SCSI) then the kernel itself would be insurmountably big.

Quoted 14 lines

I also think the following are reasonably standard (based on the fact
that most enclosure standards recommend but don't require this):

3. Locate LED (for locating the device). Even if you only have an
activity LED, this is usually done by flashing the activity LED in a
well defined pattern.
4. Fault. this is the least standardised of the lot, but does seem to
be present in about every enclosure implementation.

Quoted 5 lines

All I've done is standardise these four pieces ... the services actually
take into account that it might not be possible to do certain of these
(like fault).

And none of this means that it needs a kernel representation.

1. You're not "standardizing" any known, in-practice, kernel representation, that is already in practice and thusly needs a kernel representation.

2. The kernel itself is not using nor needing this "representation" in order to function properly (the kernel).

Leaving control of SES devices to user-space makes both the kernel and the vendors happy. All the kernel needs to do is expose the SES device to user-space as it currently does. It makes it so much easier both to vendors and to the kernel to stay out of unnecessary representations.

Vendors may choose to distribute their own applications to control their hardware, as long as the kernel exposes an SES device and provides functionality, as opposed to policy of any kind.

Luben

#24 lore raw

The keep-it-in-user-space arguments seem fairly compelling to me. Especially as we've pushed whole i/o subsystems out to user space (iscsi, stgt, talked about fcoe, a lot of dm control, etc).

The functionality seems to align with Doug's sg/lsscsi utility chain as well. Granted, the new utility would have to be designed in such as way that it can incorporate vendor "hardware handlers". But, if James has a somewhat common implementation already for a kernel implementation, I'm sure that can be the starting point for lsscsi.

So, the main question I believe is being asked is:
- Do we need to represent this via the object/sysfs tree or can an outside utility be depended upon to show it ?

Note that I am not supporting:
"Vendors may choose to distribute their own applications". For this to become truly useful, there needs to be a common tool/method that presents common features in a common manner, regardless of whether it is in kernel or not.

-- james s

Quoted 69 lines · Luben Tuikov

Which is already the case without the SES kernel bloat. Case in point, the excellent user-space application "lsscsi" would clearly show which device is SES.

And the excellent user-space application "sg_ses" could control an SES device.

Quoted 4 lines

The pieces I think are absolutely standard are

1. Actual enclosure presence (is this device in an enclosure)

"lsscsi"

Quoted 2 lines

2. Activity LED, this seems to be a feature of every enclosure.

So that means that it needs a kernel representation? If this indeed were the case, for every "feature" of every type of device (not only SCSI) then the kernel itself would be insurmountably big.

Quoted 14 lines

I also think the following are reasonably standard (based on the fact
that most enclosure standards recommend but don't require this):

3. Locate LED (for locating the device). Even if you only have an
activity LED, this is usually done by flashing the activity LED in a
well defined pattern.
4. Fault. this is the least standardised of the lot, but does seem to
be present in about every enclosure implementation.

Quoted 5 lines

All I've done is standardise these four pieces ... the services actually
take into account that it might not be possible to do certain of these
(like fault).

And none of this means that it needs a kernel representation.

1. You're not "standardizing" any known, in-practice, kernel representation, that is already in practice and thusly needs a kernel representation.

2. The kernel itself is not using nor needing this "representation" in order to function properly (the kernel).

Leaving control of SES devices to user-space makes both the kernel and the vendors happy. All the kernel needs to do is expose the SES device to user-space as it currently does. It makes it so much easier both to vendors and to the kernel to stay out of unnecessary representations.

Vendors may choose to distribute their own applications to control their hardware, as long as the kernel exposes an SES device and provides functionality, as opposed to policy of any kind.

Luben

-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author maintainer
#25 lore raw
Quoted 3 lines · James Smart

The keep-it-in-user-space arguments seem fairly compelling to me. Especially as we've pushed whole i/o subsystems out to user space (iscsi, stgt, talked about fcoe, a lot of dm control, etc).

And to me too.

Quoted 10 lines

The functionality seems to align with Doug's sg/lsscsi utility chain as well. Granted, the new utility would have to be designed in such as way that it can incorporate vendor "hardware handlers". But, if James has a somewhat common implementation already for a kernel implementation, I'm sure that can be the starting point for lsscsi.

So, the main question I believe is being asked is:
- Do we need to represent this via the object/sysfs tree or can an outside utility be depended upon to show it ?

Quoted 5 lines

Note that I am not supporting:
"Vendors may choose to distribute their own applications". For this to become truly useful, there needs to be a common tool/method that presents common features in a common manner, regardless of whether it is in kernel or not.

I don't disagree with that, but the fact is that there isn't such a
tool. It's also a fact that the enterprise is reasonably unhappy with
the lack of an enclosure management infrastructure, since it's something they got on all the other unix systems.

I think a minimal infrastructure in-kernel does just about everything the enterprise wants ... and since it's stateless, they can always use direct connect tools in addition.

However, I'm happy to be proven wrong ... anyone on this thread is welcome to come up with a userland enclosure infrastructure. Once it does everything the in-kernel one does (which is really about the minimal possible set), I'll be glad to erase the in-kernel one.

James

#26 lore raw
Quoted 4 lines · James Bottomley

I don't disagree with that, but the fact is that there isn't such a
tool. It's also a fact that the enterprise is reasonably unhappy with
the lack of an enclosure management infrastructure, since it's something they got on all the other unix systems.

I don't disagree.

Quoted 8 lines

I think a minimal infrastructure in-kernel does just about everything the enterprise wants ... and since it's stateless, they can always use direct connect tools in addition.

However, I'm happy to be proven wrong ... anyone on this thread is welcome to come up with a userland enclosure infrastructure. Once it does everything the in-kernel one does (which is really about the minimal possible set), I'll be glad to erase the in-kernel one.

yeah, but... putting something new in, only to pull it later, is a bad paradigm for adding new mgmt interfaces. Believe me, I've felt users pain in the reverse flow : driver-specific stuff that then has to migrate to upstream interfaces, complicated by different pull points by different distros. You can migrate a management interface, but can you really remove/pull one out ?

Isn't it better to let the lack of an interface give motivation to create the "right" interface, once the "right way" is determined - which is what I
thought we were discussing ? or is this simply that there is no motivation
until something exists, that people don't like, thus they become motivated ?

-- james s

author maintainer
#27 lore raw
Quoted 17 lines · James Smart
Quoted 4 lines · James Bottomley

I don't disagree with that, but the fact is that there isn't such a
tool. It's also a fact that the enterprise is reasonably unhappy with
the lack of an enclosure management infrastructure, since it's something they got on all the other unix systems.

I don't disagree.

Quoted 8 lines

I think a minimal infrastructure in-kernel does just about everything the enterprise wants ... and since it's stateless, they can always use direct connect tools in addition.

However, I'm happy to be proven wrong ... anyone on this thread is welcome to come up with a userland enclosure infrastructure. Once it does everything the in-kernel one does (which is really about the minimal possible set), I'll be glad to erase the in-kernel one.

Quoted 5 lines

yeah, but... putting something new in, only to pull it later, is a bad paradigm for adding new mgmt interfaces. Believe me, I've felt users pain in the reverse flow : driver-specific stuff that then has to migrate to upstream interfaces, complicated by different pull points by different distros. You can migrate a management interface, but can you really remove/pull one out ?

That depends on the result. I agree that migration will be a pain, so I suppose I set the bar a bit low; the user tool needs to be a bit more compelling; plus I'll manage the interface transition ... if there is one; there's no such tool yet.

Quoted 4 lines

Isn't it better to let the lack of an interface give motivation to create the "right" interface, once the "right way" is determined - which is what I
thought we were discussing ? or is this simply that there is no motivation
until something exists, that people don't like, thus they become motivated ?

Well ... I did learn the latter from Andrew, so I thought I'd try it. It's certainly true that the enclosure problem has been an issue for over a decade, so there doesn't seem to be anything motivating anyone to solve it. I wouldn't have bothered except that I could see ad-hoc in-kernel sysfs solutions beginning to appear. At least this way they can all present a unified interface.

James

#28 lore raw
Quoted 2 lines · James Bottomley

... I wouldn't have bothered except that I could see ad-hoc in-kernel sysfs solutions beginning to appear.

If this is true, and if no one quickly volunteers to do the utility, then I agree with what you are doing.

-- james s

#29 lore raw
In reply to Kristen Carlson Accardi
Quoted 23 lines · Kristen Carlson Accardi

Hi,
I apologize for taking so long to review this patch. I obviously agree
wholeheartedly with Luben. The problem I ran into while trying to
design an enclosure management interface for the SATA devices is that
there is all this vendor defined stuff. For example, for the AHCI LED
protocol, the only "defined" LED is
'activity'. For LED2 and LED3 it
is up to hardware vendors to define these. For SGPIO there's all kinds
of ways for hw vendors to customize. I felt that it was going to be a
maintainance nightmare to have to keep track of various vendors
enclosure implementations in the ahci driver, and that it'd be better
to just have user space libraries take care of that. Plus, that way a
vendor doesn't have to get a patch into the kernel to get their new

Quoted 8 lines

spiffy wizzy bang blinky lights working (think of how long it takes
something to even get into a vendor kernel, which is what these guys
care about...). So I'm still not sold on having an enclosure
abstraction in the kernel - at least for the SATA controllers.

And I agree wholeheartedly with Kristen.

Luben