MSL4048 with LTO6 6250 - wrong drive type

[CTurtle]

Hello all,

This is a copy of a thread I opened on reddit's r/DataHoarder, just hedging my bets here as I am really out of my league with this one.

I have gotten some sweet deals on ebay that may be turning sour, and I really need your insight to assert where my problem lays; I got myself an HP MSL4048 Tape Library without drives which seems to work fine and is running firmware version 9.60/3.02e which I believe is the last one from a cursory search on HP's site.

I separately got myself an LTO-6 drive, HP branded and with a few IDs on it "spare no. 721592-001", "AQ288D#103", "C5F90A".

Now, when trying out the tape drive in the library, I get a note that "Drive 1: Disabled - Exception 48", and there's a log entry saying "Wrong drive fitted".

There isn't too much information about that from my googling around, though it seems that this error stems from using a drive on a library that was factory prepared to operate on a different brand library, though it seems odd this would bear the HP brand and be for another library.

I do not have any other tape drive to test the operation of the library and I am waiting on the seller to tell me what method they used to test this to advertise it as "tested working". But, in the mean time, since I don't have any prior knowledge with tapes and such, I wanted to ask if there's anything I can do to further test this, as well as maybe force update the tape firmware. Right now it won't let me update the drive as it is disabled, and nothing comes up in the SAS interface, not the tape nor the library which I guess is expected as the drive is powered off once it failed to be initialize.

I generated a support ticket and while it seems to be mostly binary data, the plain text part does read "HP Ultrium 6-SCSI HUJ42813R2 33HW 018.421" and I find no mention to any other brand.

Finally, there are some other errors that show up in that support ticket, though I don't see these in the event log I downloaded:

HE: drive communication error

HUJ42813R2 HE: drive communication error

HUJ42813R2 HE: drive logged out

HE: invalid rotation range

LTC015065 HE: slider blocked



So, yeah, any help would be greatly appreciated

 

[Terry Kennedy]

Now, when trying out the tape drive in the library, I get a note that "Drive 1: Disabled - Exception 48", and there's a log entry saying "Wrong drive fitted".

Welcome to the weird and wacky world of HP. Up until Tandberg got out of the LTO drive business (after LTO-6, I think), HP used Tandberg drives in HP libraries (which were OEM'd IBM TS3200 units). I guess they were willing to pay IBM enough for custom firmware to support non-IBM drives.

You can switch a library between most personalities by changing the library controller card (bottom right slot) and telling the library to use the config in the controller card, not the library. That won't work for the HP personality as it has a different operator control panel than any of the other TS3200 personalities. Just HP doing things the (new) "HP Way".

Right now it won't let me update the drive as it is disabled, and nothing comes up in the SAS interface, not the tape nor the library which I guess is expected as the drive is powered off once it failed to be initialize.

Normally, the library talks to the host via a separate LUN on the SAS cable of the primary drive (or drives, if the library is partitioned). There's an Ethernet port (not to be confused with the serial port) on the back panel of the library controller card.

LTC015065 HE: slider blocked

You do have this mounted in a rack with the correct rails, right? These don't like being run as tabletop drives without some tricks - the bottom of the library bows up and interferes with the motion of the picker assembly.

If you carefully lift up the library and look at the bottom, you'll see a narrow indentation on both the left and right side, running from front to back. You'll need to stick rubber feet on the indented part (4 feet, front and rear left and right) to keep the large center portion elevated off the surface you have it on.

 

[CTurtle]

Welcome to the weird and wacky world of HP. Up until Tandberg got out of the LTO drive business (after LTO-6, I think), HP used Tandberg drives in HP libraries (which were OEM'd IBM TS3200 units). I guess they were willing to pay IBM enough for custom firmware to support non-IBM drives.

Well, thank you, it is nice to be here In all seriousness I did do my due diligence before deciding to embark on this journey, but I still get amazed at how thickly user-unfriendly anything "business hardware" is. And HP is really up there in the making things harder for no reason department.

You can switch a library between most personalities by changing the library controller card (bottom right slot) and telling the library to use the config in the controller card, not the library. That won't work for the HP personality as it has a different operator control panel than any of the other TS3200 personalities. Just HP doing things the (new) "HP Way".

At some point I stumbled across some controller firmwares to address just that, getting the Dell personality to work with the MSL hardware which is exactly the same in all ways... except for the controller UI. I cannot find that again, though.

Normally, the library talks to the host via a separate LUN on the SAS cable of the primary drive (or drives, if the library is partitioned). There's an Ethernet port (not to be confused with the serial port) on the back panel of the library controller card.

You do have this mounted in a rack with the correct rails, right? These don't like being run as tabletop drives without some tricks - the bottom of the library bows up and interferes with the motion of the picker assembly.

If you carefully lift up the library and look at the bottom, you'll see a narrow indentation on both the left and right side, running from front to back. You'll need to stick rubber feet on the indented part (4 feet, front and rear left and right) to keep the large center portion elevated off the surface you have it on.

I do have it on a rack with the original rails, the robot picker moves tapes to any position in any magazine, which is why that error is a bit awkward. My guess is that it is a mishap from communicating with the drive that triggers it, as it only happens if the drive is connected, and always happens in tandem with the invalid rotation range error.

Physically, if I try to manually power on the drive (using either the web ui or the menu on the library), there's a fan spool up, spool down, a minute passes without noises and it errors out. I don't think the picker is trying to move at all at this point.

I just wish I could pinpoint somehow if the drive has some actual issue or if it is just a misconfiguration, i.e. I'm trying to use a drive meant for another system. And of course knowing for which system in that latter option would be great.

 

[Terry Kennedy]

Well, thank you, it is nice to be here In all seriousness I did do my due diligence before deciding to embark on this journey, but I still get amazed at how thickly user-unfriendly anything "business hardware" is. And HP is really up there in the making things harder for no reason department.

Yup. And dont get me started about HP's policies on printer ink.

At some point I stumbled across some controller firmwares to address just that, getting the Dell personality to work with the MSL hardware which is exactly the same in all ways... except for the controller UI. I cannot find that again, though.

There was some discussion about that here on STH a few years ago.

AFAIK, you can't cross-flash the library controller card via the Web UI or SAS. It may be possible to do it with the serial diagnostic port on the LCC. I've found it easier to just swap LCCs.

I do have it on a rack with the original rails, the robot picker moves tapes to any position in any magazine, which is why that error is a bit awkward. My guess is that it is a mishap from communicating with the drive that triggers it, as it only happens if the drive is connected, and always happens in tandem with the invalid rotation range error.

I think rotation range is also a picker issue.

In at least the Dell and IBM personalities, the library will inventory on power-up, even when no drives are installed. You need barcode labels on your tapes if you want it to complete in a reasonable timeframe, though. The magazines have a "nobody home" barcode for empty slots, but if there's an un-coded tape the picker will try multiple times to read the missing label, varying the focus. Once it gives up looking for a barcode, it will then prod the slot with a stick to see if there's a tape in there. Really!

I have some info about printing your own barcode labels here.

Physically, if I try to manually power on the drive (using either the web ui or the menu on the library), there's a fan spool up, spool down, a minute passes without noises and it errors out. I don't think the picker is trying to move at all at this point.

I just wish I could pinpoint somehow if the drive has some actual issue or if it is just a misconfiguration, i.e. I'm trying to use a drive meant for another system. And of course knowing for which system in that latter option would be great.

Hard to say. Normally the library drives get cannibalized and turned into standalone drives (usually still without faceplates) although I'm sure there's a bit of going the other way, too. If it was anyone but HP, I'd say to contact them and ask.

 

[CTurtle]

There was some discussion about that here on STH a few years ago.

AFAIK, you can't cross-flash the library controller card via the Web UI or SAS. It may be possible to do it with the serial diagnostic port on the LCC. I've found it easier to just swap LCCs.

That might have been my source. I wouldn't mind just swapping LCCs, but the fact remains that I wouldn't know which flavor to use in replacement for the HP one, and as far as I understand it, and you pointed it out, this HP unit I have will not be compatible with any other flavor without some firmware tweaking. Whether that means not working at all or just losing the front panel interface I'm not sure.

I'm happy to use any method required to flash the LCC if that would help, my plan would then be to buy another one (maybe from another vendor) and use that to play and experiment. I've been known to solder directly to MCUs to grab a serial line...

I think rotation range is also a picker issue.

Well, that would definitely change things a bit. I have been running under the assumption all errors are drive related because the UI says the drive is disabled, but the library is working fine. This nagging feeling I'm going down a rabbit hole that is going to consume my next few months, though...

Oh well.

 

[CTurtle]

Just confirming what I had previously asserted, I removed the drive, finally decided to update the date/time which helped know which logs are from this last effort, reset the library to defaults which reboots it and the logs show exactly one warning, the "no drive installed" that one expects.

No other error whatsoever. Looking at the complete "Standard Trace" with details, there's a log of back and forth with the robotic picker, but everything looks like a success.

Perhaps the drive is physically impacting the robot movement somehow? I have tried multiple slots with no differing results, but I'm grasping at straws right now.

 

[Terry Kennedy]

Just confirming what I had previously asserted, I removed the drive, finally decided to update the date/time which helped know which logs are from this last effort, reset the library to defaults which reboots it and the logs show exactly one warning, the "no drive installed" that one expects.

No other error whatsoever. Looking at the complete "Standard Trace" with details, there's a log of back and forth with the robotic picker, but everything looks like a success.

Perhaps the drive is physically impacting the robot movement somehow? I have tried multiple slots with no differing results, but I'm grasping at straws right now.

That sounds like you're getting somewhere. Does the picker cycle when you power the library up? On a Dell or IBM personality, I believe it says "Accessing", accompanied by obvious picker sounds.

I don't know how much money you have invested in this library, but there's an LTO4-HH SAS drive on eBay for $114 w/ free US shipping: HP AK383A LTO4 HH SAS TAPE DRIVE for 1/8 G2 MSL2024 MSL4048 MSL8096 467729-001 | eBay
The usual caveats apply - not my listing, I don't know the seller, etc.

 

[CTurtle]

Today I opened up the library and watched it do the power up without drives, the picker does a whole cycle, checks the whole range of movement and scans all slots, no problems or errors reported.
I then inserted the drive, booted up again, same thing, no issues. The drive's led blinks at first, stays solid for a bit and then turns off. There's a fan spool up and then down, and even with the drive "off" it still spins the fan, albeit slowly.

I have taken the full logs and will parse them later to see if there's any clue there, but my cursory glance at them shows nothing of note, not even the "wrong drive type" I would expect as after booting the library fully it shows the "disabled - exception 48" as its status... not quite sure why that would not show in the logs.

As for the tape drive, I have already ordered another LTO6 one, from another seller that asserted it will work on the MSL4048, and I'm still waiting on the original drive seller to say something about how they tested the one I have. I am in Europe, though, and since I ordered from a US based seller it'll take a while to arrive.

It is mesmerizing to look at the robot picker do its thing, and it seems to my untrained eye to be working great, the only caveat being that all axis movements make a "zzzz" sound from the motors spinning their gears, except the vertical movement, which does sound a healthy "zzzz" when going up, but makes a different "rrrrr" sound when going down. It does its thing without hesitation, so I assume it is nothing to worry about.

 

[Terry Kennedy]

As for the tape drive, I have already ordered another LTO6 one, from another seller that asserted it will work on the MSL4048, and I'm still waiting on the original drive seller to say something about how they tested the one I have. I am in Europe, though, and since I ordered from a US based seller it'll take a while to arrive.

HPE PartSurfer knows about your C5F90A drive part number. But it says it is "OEM Automation LTO6HH SAS Drive", which sounds fishy. HP is known for blocking otherwise-identical products from moving between categories.

If I put spare part number 706824-001 into PartSurfer, I get "1/8 G2 LTO-6 Ultrium 6250 SAS tape drive - 6.25TB compressed capacity, 1.44TB/hr compressed transfer rates, Linear Tape File System (LTFS), and AES 256-bit hardware encryption" and when I click on products it is used in, I get a list of HP libraries (see attached image).

It is mesmerizing to look at the robot picker do its thing, and it seems to my untrained eye to be working great, the only caveat being that all axis movements make a "zzzz" sound from the motors spinning their gears, except the vertical movement, which does sound a healthy "zzzz" when going up, but makes a different "rrrrr" sound when going down. It does its thing without hesitation, so I assume it is nothing to worry about.

In one direction it is working against gravity, in the other it has gravity assist, so it is probably fine. However, the picker elevator gear is a problem area on these libraries - it is a plastic gear press-fit onto a splined shaft, and it can break from wear. Fortunately, it isn't firmware-locked to any particular variant, and you can even scrounge the gear + shaft from a 2RU library picker and put it in the 4RU picker. Or you can just replace the entire picker.

BTW, I think the library picker motion errors that were in the error log previously were caused by someone starting the library with the shipping lock in place. The picker really should have had a microswitch to detect when the shipping lock was engaged. Lots of the shipping locks get lost - the little notch strip on the back of the library is where the lock is supposed to be stored, but anyone working in the rack can knock the lock out of the slot and it will fall to the bottom of the rack and be lost. Fortunately, a paper clip bent into a "T" shape and taped down works fine as a shipping lock.

 

[CTurtle]

HPE PartSurfer knows about your C5F90A drive part number. But it says it is "OEM Automation LTO6HH SAS Drive", which sounds fishy. HP is known for blocking otherwise-identical products from moving between categories.

If I put spare part number 706824-001 into PartSurfer, I get "1/8 G2 LTO-6 Ultrium 6250 SAS tape drive - 6.25TB compressed capacity, 1.44TB/hr compressed transfer rates, Linear Tape File System (LTFS), and AES 256-bit hardware encryption" and when I click on products it is used in, I get a list of HP libraries (see attached image).

Not quite sure what the "2" is after one of the MSL4048 lines, but from looking at this it feels like it should work.

In one direction it is working against gravity, in the other it has gravity assist, so it is probably fine. However, the picker elevator gear is a problem area on these libraries - it is a plastic gear press-fit onto a splined shaft, and it can break from wear. Fortunately, it isn't firmware-locked to any particular variant, and you can even scrounge the gear + shaft from a 2RU library picker and put it in the 4RU picker. Or you can just replace the entire picker.

BTW, I think the library picker motion errors that were in the error log previously were caused by someone starting the library with the shipping lock in place. The picker really should have had a microswitch to detect when the shipping lock was engaged. Lots of the shipping locks get lost - the little notch strip on the back of the library is where the lock is supposed to be stored, but anyone working in the rack can knock the lock out of the slot and it will fall to the bottom of the rack and be lost. Fortunately, a paper clip bent into a "T" shape and taped down works fine as a shipping lock.

I did remove the lock before my initial powerup, and it was correctly placed during shipping. With that I would not expect to be getting these errors, although the one thing different today is that I removed the library from the rack and used a set of "universal rack mounts" to prop up the the sides just that slight bit off the table. I was using the original mounts in my rack and no unit below the library. I do have stuff immediately above though it is not a tight fit, it slides freely in and out without effort.

Just in case there's something obvious I'm overlooking, here's the complete bootup log with the drive in the first slot. Given the last command sent to the drive was drive enable, and it got a positive reply (see @ 15:11:51), I'm really not sure why it would show the exception 48.

Oh, just as an added node, this was all done without any SAS connection (today, previous attempts had it), so maybe that's why?

24.01.22 15:16:27.55  Libctrl task Low-Level Trace
     Raw data: 31 33 32 34 00 07 50 01 43 80 27 2C 5B 40 CF 00 00 00 00 00
     Raw data: 00 00 00 00 00 AF
24.01.22 15:16:27.55  Libctrl task Low-Level Trace
     Raw data: 10 44 45 43 38 31 32 30 31 54 31 00 00 00 04 04 04 04 04 04
     Raw data: 04 04 04 C0 A8 0B 56 C0 A8 0B 01 FF FF FF 00 AC 16 2D 45 FC
     Raw data: EE 74 75 73 65 72 00 00 00 00 00 74 61 64 6D 69 6E 00 00 00
     Raw data: 00 74 73 75 70 70 6F 72 74 00 00 61 67 73 69 6C 76 65 72 00
     Raw data: 00 78 79 7A 7A 79 31 30 36 36 00 78 79 7A 7A 79 31 34 39 32
     Raw data: 00 30 30 30 30 30 30 30 30 00 34 32 33 31
24.01.22 15:16:27.53  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:16:22.91  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:16:22.59  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x02 - get initialisation state
24.01.22 15:16:22.59  Robotic task Command - Robotic command
     Command code: 0x02 - get initialisation state
24.01.22 15:16:22.34  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:16:21.19  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:16:21.05  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:16:18.93  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:16:18.81  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:16:17.87  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:16:17.73  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:16:11.08  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:16:10.72  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:16:02.12  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:16:01.98  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:15:52.50  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:15:52.36  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:15:36.83  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:15:36.48  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x1E - move home
24.01.22 15:15:26.69  Robotic task Command - Robotic command
     Command code: 0x1E - move home
24.01.22 15:15:26.55  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x03 - inventory scan
24.01.22 15:15:03.70  Libctrl task Event - Library event
     Library manager event code: 0x08 - login
24.01.22 15:13:01.22  Robotic task Command - Robotic command
     Command code: 0x03 - inventory scan
24.01.22 15:13:01.10  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x07 - get sensor status
24.01.22 15:13:00.98  Robotic task Command - Robotic command
     Command code: 0x07 - get sensor status
24.01.22 15:13:00.59  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x09 - mailslot scan
24.01.22 15:12:54.06  Trace module Event - Memory report:
     Free: System memory: 18318060; OCP memory: 40288
24.01.22 15:12:54.06  Robotic task Command - Robotic command
     Command code: 0x09 - mailslot scan
24.01.22 15:12:53.67  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x09 - mailslot scan
24.01.22 15:12:41.74  Robotic task Command - Robotic command
     Command code: 0x09 - mailslot scan
24.01.22 15:12:41.35  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x01 - mechanical initialisation
24.01.22 15:11:51.44  Robotic task Command - Robotic command
     Command code: 0x01 - mechanical initialisation
24.01.22 15:11:51.37  Tape drive task Response - Drive response
     Return status: 0x00 - command succeeded
24.01.22 15:11:51.04  Tape drive task Command - Drive command
     Command code: 0x05 - drive enable
     Drive number: 0x00, 0
24.01.22 15:11:50.05  Tape drive task Response - Drive response
     Return status: 0x00 - command succeeded
     Command code: 0x07 - destroy drive-object
24.01.22 15:11:50.05  Tape drive task Command - Drive command
     Command code: 0x07 - destroy drive-object
     Drive number: 0x00, 0
24.01.22 15:11:50.04  Tape drive task Response - Drive response
     Return status: 0x00 - command succeeded
     Command code: 0x3F - disconnect
24.01.22 15:11:50.04  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x3F - drive disconnect command
24.01.22 15:11:49.72  Tape drive task Command - Drive command
     Command code: 0x3F - disconnect
     Drive number: 0x00, 0
24.01.22 15:11:49.72  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x42 - read new CDB
24.01.22 15:11:49.44  Tape drive task Response - Drive response
     Return status: 0x00 - command succeeded
     Command code: 0x13 - get configuration
24.01.22 15:11:49.44  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x13 - get drive configuration
24.01.22 15:11:49.42  Tape drive task Command - Drive command
     Command code: 0x13 - get configuration
     Drive number: 0x00, 0
24.01.22 15:11:49.41  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x11 - get drive info
24.01.22 15:11:49.36  Tape drive task Command - Drive command
     Command code: 0x11 - get info
     Drive number: 0x00, 0
24.01.22 15:11:49.35  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x41 - report library status to LUN
24.01.22 15:11:49.34  Tape drive task Response - Drive response
     Return status: 0x00 - command succeeded
     Command code: 0x12 - set configuration
24.01.22 15:11:49.34  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x12 - set drive configuration
24.01.22 15:11:49.29  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x45 - send SCSI status
24.01.22 15:11:49.28  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x43 - send SCSI data
24.01.22 15:11:49.27  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x42 - read new CDB
24.01.22 15:11:49.27  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x60 - attention
24.01.22 15:11:49.27  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x45 - send SCSI status
24.01.22 15:11:49.27  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x43 - send SCSI data
24.01.22 15:11:49.26  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x42 - read new CDB
24.01.22 15:11:49.26  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x60 - attention
24.01.22 15:11:49.25  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x45 - send SCSI status
24.01.22 15:11:49.25  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x43 - send SCSI data
24.01.22 15:11:49.24  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x42 - read new CDB
24.01.22 15:11:49.24  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x60 - attention
24.01.22 15:11:49.23  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x45 - send SCSI status
24.01.22 15:11:49.23  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x43 - send SCSI data
24.01.22 15:11:49.22  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x42 - read new CDB
24.01.22 15:11:49.22  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x60 - attention
24.01.22 15:11:49.22  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x45 - send SCSI status
24.01.22 15:11:49.21  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x43 - send SCSI data
24.01.22 15:11:49.20  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x42 - read new CDB
24.01.22 15:11:49.20  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x60 - attention
24.01.22 15:11:49.19  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x45 - send SCSI status
24.01.22 15:11:49.19  CDB interpreter task Response - SCSI check condition
     Virtual library: 0x00, 0
     Host ID: 0x00 - external host
     Control path: 0x00, 0
     SCSI Status: 0x02 - CHECK CONDITION
     Sense Key = 0x02 - NOT READY
     ASC = 0x04, 4
     ASCQ = 0x01, 1
24.01.22 15:11:49.19  CDB interpreter task Command - PREVENT / ALLOW MEDIUM REMOVAL
     Virtual library: 0x00, 0
     Host ID: 0x00 - external host
     Control path: 0x00, 0
     Prevent flag: 0x00 - Unlocked
24.01.22 15:11:49.18  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x42 - read new CDB
24.01.22 15:11:49.18  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x60 - attention
24.01.22 15:11:49.18  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x45 - send SCSI status
24.01.22 15:11:49.17  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x42 - read new CDB
24.01.22 15:11:49.17  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x60 - attention
24.01.22 15:11:49.04  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x45 - send SCSI status
24.01.22 15:11:49.03  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x42 - read new CDB
24.01.22 15:11:49.03  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x60 - attention
24.01.22 15:11:48.65  Tape drive task Command - Drive command
     Command code: 0x12 - set configuration
     Drive number: 0x00, 0
24.01.22 15:11:48.60  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x92, 146
24.01.22 15:11:48.60  Drive communication (SDCI) Event - Drive status
     Drive number: 0x00, 0
     Drive status flags: completely unloaded
     Extended drive status flags: data compression enabled & drive initialization complete
     Drive activity: 0x00 - idle
     Drive error: 0x00 - no error
24.01.22 15:11:47.58  SCSI task Event - Global event
     Global error code: 0x70 - AC: SCSI message error
24.01.22 15:11:17.58  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x40 - enable surrogate SCSI
24.01.22 15:11:17.57  Tape drive task Response - Drive response
     Return status: 0x00 - command succeeded
     Command code: 0x91 - set IP configuration
24.01.22 15:11:17.54  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x91, 145
24.01.22 15:11:17.52  Tape drive task Command - Drive command
     Command code: 0x91 - set IP configuration
     Drive number: 0x00, 0
24.01.22 15:11:17.48  Tape drive task Command - Drive command
     Command code: 0x90 - get IP configuration
     Drive number: 0x00, 0
24.01.22 15:11:17.48  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x11 - get drive info
24.01.22 15:11:17.42  Tape drive task Command - Drive command
     Command code: 0x11 - get info
     Drive number: 0x00, 0
24.01.22 15:11:17.42  Tape drive task Response - Drive response
     Return status: 0x00 - command succeeded
     Command code: 0x19 - wake up
24.01.22 15:11:17.42  Drive communication (SDCI) Trace - SDCI event
     Drive number: 0x00, 0
     SDCI command: 0x19 - drive wakeup
24.01.22 15:11:17.08  Drive communication (SDCI) Event - SDCI debug trace
     SDCI debug trace
24.01.22 15:11:05.89  Network Event
     Network event: 0x01 - net device opened
24.01.22 15:11:05.05  Tape drive task Command - Drive command
     Command code: 0x19 - wake up
     Drive number: 0x00, 0
24.01.22 15:11:05.05  Tape drive task Response - Drive response
     Return status: 0x00 - command succeeded
     Command code: 0x06 - create drive object
24.01.22 15:11:05.05  Tape drive task Command - Drive command
     Command code: 0x06 - create drive object
     Drive number: 0x00, 0
24.01.22 15:11:05.04  Tape drive task Response - Drive response
     Return status: 0x00 - command succeeded
     Command code: 0x1A - clear data
24.01.22 15:11:05.04  Tape drive task Command - Drive command
     Command code: 0x1A - clear data
     Drive number: 0x00, 0
24.01.22 15:11:04.39  Libctrl task Command - Library controller command
     Current command: 0x11 - initialise
24.01.22 15:11:04.38  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x53 - check robotic hardware version
24.01.22 15:11:04.16  Robotic task Command - Robotic command
     Command code: 0x53 - check robotic hardware version
24.01.22 15:11:03.43  CDB interpreter task Event - SCSI CDB Event
     CDB event: 0x01 - SCSI hard reset
24.01.22 15:11:03.43  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:11:02.75  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:11:01.81  Tape drive task Event - Global event
     Global error code: 0x01 - NR: drive cleaning requested
24.01.22 15:11:01.46  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:10:59.38  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:10:59.26  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:10:58.86  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:10:58.74  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:10:58.00  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:10:57.88  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:10:56.98  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:10:56.86  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:10:54.78  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:10:54.65  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:10:52.16  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:10:52.03  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:10:48.11  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:10:47.99  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:10:42.13  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:10:42.02  Libctrl task Low-Level Trace
     Raw data: 31 33 32 34 00 07 50 01 43 80 27 2C 5B 40 CF 00 00 00 00 00
     Raw data: 00 00 00 00 00 AF
24.01.22 15:10:42.01  Libctrl task Low-Level Trace
     Raw data: 10 44 45 43 38 31 32 30 31 54 31 00 00 00 04 04 04 04 04 04
     Raw data: 04 04 04 C0 A8 0B 56 C0 A8 0B 01 FF FF FF 00 AC 16 2D 45 FC
     Raw data: EE 74 75 73 65 72 00 00 00 00 00 74 61 64 6D 69 6E 00 00 00
     Raw data: 00 74 73 75 70 70 6F 72 74 00 00 61 67 73 69 6C 76 65 72 00
     Raw data: 00 78 79 7A 7A 79 31 30 36 36 00 78 79 7A 7A 79 31 34 39 32
     Raw data: 00 30 30 30 30 30 30 30 30 00 34 32 33 31
24.01.22 15:10:42.01  Robotic task Response - Robotic response
     Return status: 0x00 - command succeeded
     Command code: 0x50 - save/restore DB
24.01.22 15:10:38.80  Robotic task Command - Robotic command
     Command code: 0x50 - save/restore DB
24.01.22 15:10:35.07  Libctrl task Event - Library event
     Library manager event code: 0x00 - power-on or reset

 

[Terry Kennedy]

Not quite sure what the "2" is after one of the MSL4048 lines, but from looking at this it feels like it should work.

Note that your drive is a 721592-001 which comes up as an OEM drive. The 706824-001 part number I found is just an example of the "correct" type of drive.

G2 is Generation 2, HP-speak. I don't think there was a G1 of the MSL4048.

IBM EoL'd the original Type 3573 library and replaced it with the Type 6173. Although the firmware is different, the functions are similar.

I did remove the lock before my initial powerup, and it was correctly placed during shipping. With that I would not expect to be getting these errors, although the one thing different today is that I removed the library from the rack and used a set of "universal rack mounts" to prop up the the sides just that slight bit off the table. I was using the original mounts in my rack and no unit below the library. I do have stuff immediately above though it is not a tight fit, it slides freely in and out without effort.

Those errors may have occurred quite some time ago under a previous owner.

Just in case there's something obvious I'm overlooking, here's the complete bootup log with the drive in the first slot. Given the last command sent to the drive was drive enable, and it got a positive reply (see @ 15:11:51), I'm really not sure why it would show the exception 48.

I'm not sure. Try to find a recent copy of HP Library & Tape Tools (LTT) as it may provide a useful decode. IBM's tool (ITDT) won't work with an HP-flavor library.

Your raw trace data decodes (roughly) to text as:

1324PC',[@ϯDEC81201T1À¨VÀ¨ÿÿÿ¬-Eüîtusertadmintsupportagsilverxyzzy1066xyzzy1492000000004231

Looks like some ident info and passwords.

 

[CTurtle]

Note that your drive is a 721592-001 which comes up as an OEM drive. The 706824-001 part number I found is just an example of the "correct" type of drive.

I just double checked this last one I ordered, it is the 706824-001, so that's good. To be honest, at this point I would be happy to find out where the drive I have would work, so I can sell it without misleading anyone.

This does bring me back to the plan B discussed above, buying a different brand LCC, but I cannot for the life of me find the post about firmwares across brands for these libraries. I wonder if buying a Dell one would at least give me access to the web ui even if the buttons and screen would fail to work...

Your raw trace data decodes (roughly) to text as:

1324PC',[@ϯDEC81201T1À¨VÀ¨ÿÿÿ¬-Eüîtusertadmintsupportagsilverxyzzy1066xyzzy1492000000004231

Looks like some ident info and passwords.

I had seen that on the support ticket I downloaded, and 00000000 is the admin pass, so I tried 4231, 1492, 1066, xyzzy1066, xyzzy1492 and agsilver as the service password but they all failed. Not that I know why I would want the service level access, but I guess "because I own the thing" would be reason enough...

And after I wrote the above paragraph I decided to do a quick googling, 42311324 is the support password (nice obfuscation attempt ). Lets see if this buys me a little more fun while waiting for the new drive.

 

[CTurtle]

I'm not sure. Try to find a recent copy of HP Library & Tape Tools (LTT) as it may provide a useful decode. IBM's tool (ITDT) won't work with an HP-flavor library.

And I forgot about this one in the previous post. I do have version 6.5 (which I believe is the latest) of LTT, but the only thing I can do with it is scan hardware, since the tape is powered down I cannot access the library through the SAS connection.

 

[Terry Kennedy]

I just double checked this last one I ordered, it is the 706824-001, so that's good. To be honest, at this point I would be happy to find out where the drive I have would work, so I can sell it without misleading anyone.

Hopefully all of this will work out with the new drive.

The drive may be from a StorageTek SL24 or SL48 library - they're the only other variants I know of that use the HP-style operator control panel.

I had an HP drive that wouldn't work in Dell or IBM libraries. I bought a bezel for it on eBay (you need to find a while to get an inexpensive one, and even longer if you want one that has the correct model number) and put it in an external SAS case as a standalone drive. Some drives (I've heard this about IBM library drives) need either a switch setting or configuration over SAS to enable standalone mode. I'd be concerned you would be spending more money that might not provide anything useful. You could try running the drive off the Molex connector in a PC and the appropriate data cable to test before you spring for a bezel and enclosure.

This does bring me back to the plan B discussed above, buying a different brand LCC, but I cannot for the life of me find the post about firmwares across brands for these libraries. I wonder if buying a Dell one would at least give me access to the web ui even if the buttons and screen would fail to work...

I think this is the post you were thinking of. Given that the HP control panel is different, I'd say this is a last resort. You may not be able to configure the network if the LCC you get has a static IP configured, given that the front panel will probably not work correctly.

I had seen that on the support ticket I downloaded, and 00000000 is the admin pass, so I tried 4231, 1492, 1066, xyzzy1066, xyzzy1492 and agsilver as the service password but they all failed. Not that I know why I would want the service level access, but I guess "because I own the thing" would be reason enough...

On the Dell-flavor libraries, it is service/ser001.

 

[CTurtle]

Hopefully all of this will work out with the new drive.

Fingers crossed, otherwise I'll likely scrap the whole plan. I'm an "old techy" person, a coder by trade, but I'm always on the hunt for new things to try, and the only time I interacted with tape libraries before was in the University of Dundee, in Scotland. The IT head there was a colleague and I followed him around every now and then as he did the odd tasks he had to do. Once the good folks from IBM where doing some maintenance on the tape backup system and he had to go there to meet them, I tagged along.
Now, that was a slightly different beast of a setup, I remember a huge room with a few cages filled with tape libraries, we're talking each cage the size of a large walk-in closet, each with its robot picking tapes here, dropping them there, each library the size of a person, that was something to gaze at. So now I needed some long term backup system, albeit a few M-Disc, well, discs would suffice, I found a great deal on the 4048, so, down the rabbit hole we go

The drive may be from a StorageTek SL24 or SL48 library - they're the only other variants I know of that use the HP-style operator control panel.

I did another search on the 721592-001, and I got one ebay listing showing my exact drive labels (well, not the serial number and revision) but calling it a fujitsu drive, and noting the brand as Overland Data... go figure. I actually found a number of these listed as HP Fujitsu, but none specifies the library range they are compatible with.

I had an HP drive that wouldn't work in Dell or IBM libraries. I bought a bezel for it on eBay (you need to find a while to get an inexpensive one, and even longer if you want one that has the correct model number) and put it in an external SAS case as a standalone drive. Some drives (I've heard this about IBM library drives) need either a switch setting or configuration over SAS to enable standalone mode. I'd be concerned you would be spending more money that might not provide anything useful. You could try running the drive off the Molex connector in a PC and the appropriate data cable to test before you spring for a bezel and enclosure.

It's not a bad idea, reusing this drive as a standalone unit if possible. Either way, I was going to ask if it wasn't frowned upon trying to connect the drive to external power through the molex connector (assuming the pin keying is the same) and not connect any of the other cables that talk to the library, so I could at least assert I can see the drive through the SAS interface and have a play with it using LTT. You obviously saw the question coming and pre-emptively answered it... nice

I think this is the post you were thinking of. Given that the HP control panel is different, I'd say this is a last resort. You may not be able to configure the network if the LCC you get has a static IP configured, given that the front panel will probably not work correctly.

You are a master at this... that is exactly what I was looking for.

 

[jd456]

In one direction it is working against gravity, in the other it has gravity assist, so it is probably fine. However, the picker elevator gear is a problem area on these libraries - it is a plastic gear press-fit onto a splined shaft, and it can break from wear. Fortunately, it isn't firmware-locked to any particular variant, and you can even scrounge the gear + shaft from a 2RU library picker and put it in the 4RU picker. Or you can just replace the entire picker.

I had this break on mine. The small nylong gear split on one side and locked the veritcal picker in place. I measured the gears and found a module of ~ 0.5 with 12 tooth fit. I puchased 30 small nyon gears for a few £:

122A, 12 Teeth, 0.5 Modulus, 7mm Outer diameter, 5mm Thickness, 2mm Hole Diameter

And drilled out the cwenter until reaching ~ 2.7 mm. Pressed these back onto the shaft, re assembled libary and it works great.

 

[CTurtle]

Well, the seller of the original drive says it was pulled from an MSL2024, which I guess means it should work with my MSL4048, so something is malfunctioning. Not ideal...
I will try to test the drive by itself asap, maybe they just got confused with the provenance.

 

[CTurtle]

Quick update, I was happily surprised that the seller for the drive I could not get working, the 721592-001 one, accepted a return and full refund.
I also received the other drive and it is detected as valid on the library, and I can see both the tape drive and the library detected on the server. Now I need to learn how this whole setup works so I can properly test it.

 

[Terry Kennedy]

Quick update, I was happily surprised that the seller for the drive I could not get working, the 721592-001 one, accepted a return and full refund.
I also received the other drive and it is detected as valid on the library, and I can see both the tape drive and the library detected on the server. Now I need to learn how this whole setup works so I can properly test it.

Looks like that all worked out very well for you.

If you're using Linux or one of the *BSDs, take a look at: The RAIDzilla II upgrade project (scroll down to "Backups") for the script I use for backups.

 

[CTurtle]

Looks like that all worked out very well for you.

If you're using Linux or one of the *BSDs, take a look at: The RAIDzilla II upgrade project (scroll down to "Backups") for the script I use for backups.

It did work out great, given the hiccups in the process.

I'm now 2 days in testing and learning, and it works, but there are so many unknowns still, it is crazy... Because the idea is to integrate this tape library into scripted actions I am trying to go as bare metal as possible, so no "backup manager software". And being on Linux I've been fiddling with mt and mtx, and I can backup and restore, but the amount of state information I get back seems insufficient and sometimes incoherent.
Also, doing plain tar -cvzf to tape I was getting speeds as low as 15MB/s, turns out using gzip was limiting the overall speed due to single core performance.
Without software side compression I was getting 60MB/s, which is good but still not good enough so mbuffer got thrown in the mix, and now I can use the full tape's performance but only read back if extracting the tape data with "dd" before piping to tar. So manual tweaks to block sizes is where I am at currently, I didn't even get a chance to play with multivolume tape changing at this point, but I'm learning

I will read up on the approaches you are taking on that script, so thank you.