sg3_utils Hybrid Information Log utility

RFC PATCH New Jason B. Akers 3 participants 8 messages Oct 29, 2014 – Mar 24, 2015 libata

lore

Share & tools

lore

Jason B. Akers

author

2014-10-29 19:57 UTC #1 lore raw

RFC PATCH 1/1 sg3_utils: Added hybrid information log utility

From: Kapil Karkra <[email protected]>

A solid state hybrid drive (SSHD) is a hard drive with a small embedded NAND. Host drivers can manage this NAND using the commands defined by SATA 3.2 standard. One SATA command that provides visibility into the SSHD's cache behavior is get hybrid log command. This augmentation allows users to issue the command as follows:

sg_sat_get_hybrid_log /dev/sdc
assuming /dev/sdc is an SSHD.

patch applies to the following sourcebase from where to clone:
https://github.com/hreinecke/sg3_utils.git

Signed-off-by Kapil Karkra <[email protected]>

Signed-off-by Jason B. Akers <[email protected]>

---
 src/Makefile.am             |   21 ++
 src/sg_sat_get_hybrid_log.c |  419 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 434 insertions(+), 6 deletions(-)
 create mode 100644 src/sg_sat_get_hybrid_log.c

src/Makefile.am +15 -6

index de14608..3f8e052 100644
@@ -14,7 +14,8 @@ bin_PROGRAMS = \
 	sg_raw sg_rbuf sg_rdac sg_read sg_readcap sg_read_block_limits \
 	sg_read_buffer sg_read_long sg_reassign sg_referrals \
 	sg_requests sg_reset sg_rmsn sg_rtpg sg_safte sg_sat_identify \
-	sg_sat_phy_event sg_sat_set_features sg_scan sg_senddiag sg_ses \
+	sg_sat_phy_event sg_sat_set_features sg_sat_get_hybrid_log \
+	sg_scan sg_senddiag sg_ses \
 	sg_start sg_stpg sg_sync sg_test_rwbuf sg_turs sg_unmap sg_verify \
 	sg_vpd sg_write_buffer sg_write_long sg_write_same sg_wr_mode \
 	sg_xcopy sg_copy_results

@@ -36,7 +37,8 @@ bin_PROGRAMS = \
 	sg_prevent sg_raw sg_rdac sg_readcap sg_read_block_limits \
 	sg_read_buffer sg_read_long sg_reassign sg_referrals sg_requests \
 	sg_rmsn sg_rtpg sg_safte sg_sat_identify sg_sat_phy_event \
-	sg_sat_set_features sg_scan sg_senddiag sg_ses sg_start sg_stpg \
+	sg_sat_set_features sg_sat_get_hybrid_log \
+	sg_scan sg_senddiag sg_ses sg_start sg_stpg \
 	sg_sync sg_turs sg_unmap sg_verify sg_vpd sg_write_buffer \
 	sg_write_long sg_write_same sg_wr_mode

@@ -57,7 +59,8 @@ bin_PROGRAMS = \
 	sg_prevent sg_raw sg_rdac sg_readcap sg_read_block_limits \
 	sg_read_buffer sg_read_long sg_reassign sg_referrals sg_requests \
 	sg_rmsn sg_rtpg sg_safte sg_sat_identify sg_sat_phy_event \
-	sg_sat_set_features sg_scan sg_senddiag sg_ses sg_start sg_stpg \
+	sg_sat_set_features sg_sat_get_hybrid_log sg_scan sg_senddiag sg_ses \
+	sg_start sg_stpg \
 	sg_sync sg_turs sg_unmap sg_verify sg_vpd sg_write_buffer \
 	sg_write_long sg_write_same sg_wr_mode

@@ -78,7 +81,8 @@ bin_PROGRAMS = \
 	sg_prevent sg_raw sg_rdac sg_readcap sg_read_block_limits \
 	sg_read_buffer sg_read_long sg_reassign sg_referrals sg_requests \
 	sg_rmsn sg_rtpg sg_safte sg_sat_identify sg_sat_phy_event \
-	sg_sat_set_features sg_senddiag sg_ses sg_start sg_stpg \
+	sg_sat_set_features sg_sat_get_hybrid_log \
+	sg_senddiag sg_ses sg_start sg_stpg \
 	sg_sync sg_turs sg_unmap sg_verify sg_vpd sg_write_buffer \
 	sg_write_long sg_write_same sg_wr_mode

@@ -93,7 +97,8 @@ bin_PROGRAMS = \
 	sg_prevent sg_raw sg_rdac sg_readcap sg_read_block_limits \
 	sg_read_buffer sg_read_long sg_reassign sg_referrals sg_requests \
 	sg_rmsn sg_rtpg sg_safte sg_sat_identify sg_sat_phy_event \
-	sg_sat_set_features sg_senddiag sg_ses sg_start sg_stpg \
+	sg_sat_set_features sg_sat_get_hybrid_log \
+	sg_senddiag sg_ses sg_start sg_stpg \
 	sg_sync sg_turs sg_unmap sg_verify sg_vpd sg_write_buffer \
 	sg_write_long sg_write_same sg_wr_mode

@@ -108,7 +113,8 @@ bin_PROGRAMS = \
 	sg_prevent sg_raw sg_rdac sg_readcap sg_read_block_limits \
 	sg_read_buffer sg_read_long sg_reassign sg_referrals sg_requests \
 	sg_rmsn sg_rtpg sg_safte sg_sat_identify sg_sat_phy_event \
-	sg_sat_set_features sg_senddiag sg_ses sg_start sg_stpg \
+	sg_sat_set_features sg_sat_get_hybrid_log sg_senddiag \
+	sg_ses sg_start sg_stpg \
 	sg_sync sg_turs sg_unmap sg_verify sg_vpd sg_write_buffer \
 	sg_write_long sg_write_same sg_wr_mode

@@ -229,6 +235,9 @@ sg_sat_phy_event_LDADD = ../lib/libsgutils2.la @os_libs@
 sg_sat_set_features_SOURCES = sg_sat_set_features.c
 sg_sat_set_features_LDADD = ../lib/libsgutils2.la @os_libs@
 
+sg_sat_get_hybrid_log_SOURCES = sg_sat_get_hybrid_log.c
+sg_sat_get_hybrid_log_LDADD = ../lib/libsgutils2.la @os_libs@
+
 sg_scan_SOURCES = sg_scan.c
 sg_scan_LDADD = ../lib/libsgutils2.la @os_libs@

src/sg_sat_get_hybrid_log.c new file +419 -0

419 lines not on this page · raw message

Phillip Susi

2014-11-25 00:33 UTC #2 lore raw

Re: [RFC PATCH] sg3_utils: Added hybrid information log utility

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Quoted 9 lines · Jason B. Akers

From: Kapil Karkra <[email protected]>
A solid state hybrid drive (SSHD) is a hard drive with a small embedded NAND. Host drivers can manage this NAND using the commands defined by SATA 3.2 standard. One SATA command that provides visibility into the SSHD's cache behavior is get hybrid log command. This augmentation allows users to issue the command as follows:

Quoted 4 lines

sg_sat_get_hybrid_log /dev/sdc assuming /dev/sdc is an SSHD.
patch applies to the following sourcebase from where to clone:
https://github.com/hreinecke/sg3_utils.git

I cloned that repo and the patch does not apply cleanly. git am -3 says the referenced blobs are not found and it has conflicts in src/Makefile.am. I manually fixed that up and then it fails to link due to unresolved externals.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCgAGBQJUc85UAAoJENRVrw2cjl5RR3YIAJOF0fcTkF5pM0OIA87hK8k8 ujwflBTy66ZbPzXMD97cTGYQ6gPp3UCeT/ZcElBfrGGAuD5u/ETXMJY7lqAZgYtK bELPv6oLhz3ya2IO442sUlRYO4dVfwhsnak+2rLiQkXCy4WKSJ1EYiMIP/VHj8ne 5e8VYkuc0KamKjvT4v6EwJ7tM0blfJbYgmFGruIxOSgCAo94Zp9thAcctb9PbRex HnPTcnw5cI1sCoLf5fJafhc9w/1H45WjJTnGF7zs4hq/GGsYCwryvI0ovwAzU9ST R3jRvE+70oiyQDsj9EBak63hRYIAd8QzQ3TkCjaM5E1XHSU7vTnxvxAjKb4ZyF4= =a8Vh
-----END PGP SIGNATURE-----

Akers, Jason B

author

2014-11-25 19:14 UTC #3 lore raw

RE: [RFC PATCH] sg3_utils: Added hybrid information log utility

Hi Phillip,
It turns out that this patch was based on an old github repository that doesn't appear to be updated. Doug Gilbert reached out after the initial RFC and directed us to his page (http://sg.danny.cz/sg/) where he has updated sg3_utils code.

As we continue to work through the feedback from our SSHD kernel patches, I'm going to follow up with Doug again on sg3_utils.

Quoted 46 lines

-----Original Message-----
From: Phillip Susi [mailto:[email protected]]
Sent: Monday, November 24, 2014 4:34 PM
To: Akers, Jason B; [email protected]
Cc: Karkra, Kapil; Williams, Dan J; [email protected]; [email protected]
Subject: Re: [RFC PATCH] sg3_utils: Added hybrid information log utility
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Quoted 4 lines · Jason B. Akers
From: Kapil Karkra <[email protected]>
A solid state hybrid drive (SSHD) is a hard drive with a small embedded NAND. Host drivers can manage this NAND using the
commands
Quoted 9 lines
defined by SATA 3.2 standard. One SATA command that provides visibility into the SSHD's cache behavior is get hybrid log command. This augmentation allows users to issue the command as follows:
sg_sat_get_hybrid_log /dev/sdc assuming /dev/sdc is an SSHD.
patch applies to the following sourcebase from where to clone:
https://github.com/hreinecke/sg3_utils.git
I cloned that repo and the patch does not apply cleanly. git am -3 says the referenced blobs are not found and it has conflicts in src/Makefile.am. I manually fixed that up and then it fails to link due to unresolved externals.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBCgAGBQJUc85UAAoJENRVrw2cjl5RR3YIAJOF0fcTkF5pM0OIA87hK8k 8 ujwflBTy66ZbPzXMD97cTGYQ6gPp3UCeT/ZcElBfrGGAuD5u/ETXMJY7lqAZgYt K bELPv6oLhz3ya2IO442sUlRYO4dVfwhsnak+2rLiQkXCy4WKSJ1EYiMIP/VHj8ne 5e8VYkuc0KamKjvT4v6EwJ7tM0blfJbYgmFGruIxOSgCAo94Zp9thAcctb9PbRex HnPTcnw5cI1sCoLf5fJafhc9w/1H45WjJTnGF7zs4hq/GGsYCwryvI0ovwAzU9ST R3jRvE+70oiyQDsj9EBak63hRYIAd8QzQ3TkCjaM5E1XHSU7vTnxvxAjKb4ZyF4= =a8Vh
-----END PGP SIGNATURE-----

Douglas Gilbert

2014-11-25 20:28 UTC #4 lore raw

Re: [RFC PATCH] sg3_utils: Added hybrid information log utility

Quoted 4 lines · Jason B

Hi Phillip,
It turns out that this patch was based on an old github repository that doesn't appear to be updated. Doug Gilbert reached out after the initial RFC and directed us to his page (http://sg.danny.cz/sg/) where he has updated sg3_utils code.
As we continue to work through the feedback from our SSHD kernel patches, I'm going to follow up with Doug again on sg3_utils.

Hi,
As discussed earlier in this thread adding a new utility for each log page like sg_sat_get_hybrid_log would lead to hundreds of utilities. There are now these groups of storage related pages (with user space utilities to decode them shown in parentheses):
- SCSI VPD pages (sg_vpd (sdparm and legacy sg_inq))
- SCSI mode pages (sdparm and sg_modes)
- SCSI log pages (sg_logs)
- ATA GP log pages (from 1.40 sg_sat_read_gplog)
- ATA SMART log pages (smartctl)

At the start of this thread the proposed patch was very close to the release of sg3_utils version 1.40 and Hannes Reinecke contributed a more general approach:
sg_sat_read_gplog which I did incorporate. Currently it only decodes pages to hex.

Please look at that utility (i.e. sg_sat_read_gplog) as the basis for adding ATA GP log page decoding such as the SATA hybrid information page. Please send patches against its source.

Doug Gilbert

Quoted 46 lines

Quoted 46 lines
-----Original Message-----
From: Phillip Susi [mailto:[email protected]]
Sent: Monday, November 24, 2014 4:34 PM
To: Akers, Jason B; [email protected]
Cc: Karkra, Kapil; Williams, Dan J; [email protected]; [email protected]
Subject: Re: [RFC PATCH] sg3_utils: Added hybrid information log utility
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Quoted 4 lines · Jason B. Akers
From: Kapil Karkra <[email protected]>
A solid state hybrid drive (SSHD) is a hard drive with a small embedded NAND. Host drivers can manage this NAND using the
commands
Quoted 9 lines
defined by SATA 3.2 standard. One SATA command that provides visibility into the SSHD's cache behavior is get hybrid log command. This augmentation allows users to issue the command as follows:
sg_sat_get_hybrid_log /dev/sdc assuming /dev/sdc is an SSHD.
patch applies to the following sourcebase from where to clone:
https://github.com/hreinecke/sg3_utils.git
I cloned that repo and the patch does not apply cleanly. git am -3 says the referenced blobs are not found and it has conflicts in src/Makefile.am. I manually fixed that up and then it fails to link due to unresolved externals.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBCgAGBQJUc85UAAoJENRVrw2cjl5RR3YIAJOF0fcTkF5pM0OIA87hK8k 8 ujwflBTy66ZbPzXMD97cTGYQ6gPp3UCeT/ZcElBfrGGAuD5u/ETXMJY7lqAZgYt K bELPv6oLhz3ya2IO442sUlRYO4dVfwhsnak+2rLiQkXCy4WKSJ1EYiMIP/VHj8ne 5e8VYkuc0KamKjvT4v6EwJ7tM0blfJbYgmFGruIxOSgCAo94Zp9thAcctb9PbRex HnPTcnw5cI1sCoLf5fJafhc9w/1H45WjJTnGF7zs4hq/GGsYCwryvI0ovwAzU9ST R3jRvE+70oiyQDsj9EBak63hRYIAd8QzQ3TkCjaM5E1XHSU7vTnxvxAjKb4ZyF4= =a8Vh
-----END PGP SIGNATURE-----

Phillip Susi

2014-12-16 00:47 UTC #5 lore raw

Re: [RFC PATCH] sg3_utils: Added hybrid information log utility

In reply to Akers, Jason B

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Quoted 7 lines · Jason B

Hi Phillip, It turns out that this patch was based on an old github repository that doesn't appear to be updated. Doug Gilbert reached out after the initial RFC and directed us to his page
(http://sg.danny.cz/sg/) where he has updated sg3_utils code.
As we continue to work through the feedback from our SSHD kernel patches, I'm going to follow up with Doug again on sg3_utils.

So is there a set of patches that applies to a release from that web site? Also is the source only released as tarballs? There is no source repo?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCgAGBQJUj4DKAAoJENRVrw2cjl5RwEwH/147w+N+z9r3J70AQrLdXgtN mRF2VLEK/v0l7rpnnyiKyL7Trjc3/YXY9tLf3feC225l9UL6K73moLKGWMI+iRU2 xvCWIJKHQyf8E/j6qFqid0G9V/IQNG9vXAZSEmrTcvT5NaEJ0U3kSCKe5w9xEYEE 6SLYTF+FRHkXceloFvW/8PAwj/2K4g/KsZShIS07kdzAh+fIoZM6qQP3XbEK5v1T x+zuFZtIVzVglQmQWF6wAn5pBjHGNLCA1Yw1+Lx3tuJCoVv1B5ze1k0FOnWu6xOQ bjeOQ0cgZOjYGjgywhoJn5hgKxOhfXsNsIhSSqA/SNky4vMOV5gRXKjA2WgHu6g= =7jVR
-----END PGP SIGNATURE-----

Phillip Susi

2015-03-24 02:27 UTC #6 lore raw

Re: [RFC PATCH] sg3_utils: Added hybrid information log utility

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Quoted 7 lines · Phillip Susi

Quoted 5 lines · Jason B
Hi Phillip, It turns out that this patch was based on an old github repository that doesn't appear to be updated. Doug Gilbert reached out after the initial RFC and directed us to his page (http://sg.danny.cz/sg/) where he has updated sg3_utils code.

Quoted 6 lines

Quoted 2 lines
As we continue to work through the feedback from our SSHD kernel
patches, I'm going to follow up with Doug again on sg3_utils.
So is there a set of patches that applies to a release from that web site? Also is the source only released as tarballs? There is no source repo?

Hello. I am still beta testing an SSHD drive and would very much like to get some more information out of its logs and/or manipulate the cache pinned working set. Is there somewhere I can get these patched utilities yet?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCgAGBQJVEMt2AAoJENRVrw2cjl5RR7AH/iOPMYRoT1/gdpdVlbIfCsWw kP9MnaWVTcqT3O1rgKbudFuMqcJGD/ScnZOIsowdnJkKxwWyGUq7w863GFtaPOul o4DBWqLYPYFbmbsgs/meiS3I2qyFrUNGYL0Db8H2mQ2bbaXg59hJ4gO00eSvf8Zm gMOu3njcs3REEfu/QDi2pfJXXlWMAQfgUUomxdGb1HHmnQqR7YGAdakfRXOXfNHk Feu2QtdZTLZPkBYkcNDM22yPDSYOb6LbpbuNTYFUfPfHp8X8aejAV8zdtZx5aH7C ZNA3DEejFqlOltT1fcgTTSb0U0WV/ZcvfTH4g9FCUfx5HOnTI5Oz86He1OthbR0= =YMQp
-----END PGP SIGNATURE-----

Douglas Gilbert

2015-03-24 19:08 UTC #7 lore raw

Re: [RFC PATCH] sg3_utils: Added hybrid information log utility

Quoted 14 lines · Phillip Susi

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Quoted 10 lines · Phillip Susi
Quoted 8 lines · Jason B
Hi Phillip, It turns out that this patch was based on an old github repository that doesn't appear to be updated. Doug Gilbert reached out after the initial RFC and directed us to his page (http://sg.danny.cz/sg/) where he has updated sg3_utils code.
As we continue to work through the feedback from our SSHD kernel
patches, I'm going to follow up with Doug again on sg3_utils.

Quoted 8 lines

Quoted 3 lines
So is there a set of patches that applies to a release from that web site? Also is the source only released as tarballs? There is no source repo?
Hello. I am still beta testing an SSHD drive and would very much like to get some more information out of its logs and/or manipulate the cache pinned working set. Is there somewhere I can get these patched utilities yet?

As discussed in another thread on the subject, the sg_sat_read_gplog utility was added in the most recent release of the package:
http://sg.danny.cz/sg/p/sg3_utils-1.40.tar.xz

as a framework for adding ATA and SATA general purpose log pages. This approach is consistent with the way the package has one utility for all SCSI logs pages: sg_logs.

So if you or anyone else wants support for ATA log pages in sg3_utils, could you please send me patches against the sg_sat_read_gplog utility.

Doug Gilbert