Thank you. I purchased another matching LSI SAS 9201-16e HBA card. I’m huge fan of SAS over SATA drives just for overall reliability and resiliency. Like you eluded to they just last longer and perform better over a longer period of time. While not always the largest possible drives available they aren’t too far behind.
EMC KTN-STL3 15 bay chassis
- Thread starter nthu9280
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Oh know it’s a SATA hater! Who knew?
For the record, I have a bunch of helium-filled SATA drives and many plain jane SATAs. Many are past 7 years of continuous running now in several machines. Although these are all HGST, or are WD labelled but are HGST inside. All my crappy Toshiba piece of shit drives all died long ago, most still in warranty, that company could make a sata hater out of anyone. But still I’d rather have a fresh new SATA drive and baby it along, instead of a used SAS with a million hours that was pulled as end of life from a data center. Well. As long as they’re not Toshibas.
All my HGSTs had About 23k - 25k/ hours or less on them. I bought 10 14tb EXOS that had just about 1000 hours. I am sure someones startup went tits up.
I don’t believe I’ve ever had one HGST drive fail.I still have a very old SATA 2 single TB from back when they were still Hitachi. I run it still, as a backup for my kid’s PC. Mainly just to see when the thing will die. But it may outlive me.
All my stuff is HGST now, everything else died and i noticed they did not. I bought a few direct but most were from a lot of drive shucking that I did, from before I got into decom enterprise storage and was still stringing sata cables. WD had HGST white label drives wrapped in their plastic.
That’s really why I don’t use SAS: when I was first buying drives, I didn’t know much about them. I do have five 8TB data center pulls from ebay that are HGST SAS, set aside for my first ZFS adventure. Soon. Maybe I will love them too.
I never had anything like an Exos from Toshiba but I did have four of their 6Tb, two were called “Gold” as I recall. Best of their consumer grade crap at the time, but still consumer grade crap. Every single one died a few months before the warranty was up. They fought me on the claims for three for no valid reason, and in the end I got a small prorated refund. They refused to replace. I didn't even bother filing for the fourth drive when it died too, it wasn't worth it. I’ll never buy a Toshiba drive again though. Hope you have better luck with them.
Curious. Your Toshiba experience is opposite of the high praise Level1 Wendell has recently sung about his Toshibas. Not saying one is right or wrong. Probably an argument for going what numbers Backblaze is reporting every year. That said, I also like HGST best. And if one drive dies, I just replace because it is so rare. HGST's population bathtub curve has a very nice because glacially slow rising slope on the outgoing side.
Hey Fiberton does this graphic represent what you were describing for my cabling for 3 arrays using 2 x LSI SAS 9201-16e cards for multipathing for all SAS drives?
Attachments
I've just picked up a EMC KTN-STL3 but for the life of me I can't get it to work, I'm using a Dell Perc H200E flashed with the LSI SAS2008 bios in IT mode (Firmware Version: 20.00.07.00) which picks up all the drives my NetApp shelf
I've tried plugging into all 4 ports individually directly to the SAS card using the two different keyed connectors but it will not see the drives in sg_scan under Windows or Ubuntu.
I've powered both PSUs just in case, tried 3 SAS drives in different sections of the shelf just in case, when I plug the mini SAS cable in I get the blue LED to signify it's connected but no disks are recognised by the PC, the shelf and channel IDs remain flashing even after 30 minutes which may or may not be significant if there's only one shelf used
The only thing that I've noticed particularly odd is the drives don't appear to be being powered up, I can't here them on power up, there's no vibration, when I pull a drive out it doesn't feel like there's movement and there's no heat in the drive
Has anyone experienced similar? Does anyone know if there's a factory reset switch somewhere?
I've tried plugging into all 4 ports individually directly to the SAS card using the two different keyed connectors but it will not see the drives in sg_scan under Windows or Ubuntu.
I've powered both PSUs just in case, tried 3 SAS drives in different sections of the shelf just in case, when I plug the mini SAS cable in I get the blue LED to signify it's connected but no disks are recognised by the PC, the shelf and channel IDs remain flashing even after 30 minutes which may or may not be significant if there's only one shelf used
The only thing that I've noticed particularly odd is the drives don't appear to be being powered up, I can't here them on power up, there's no vibration, when I pull a drive out it doesn't feel like there's movement and there's no heat in the drive
Has anyone experienced similar? Does anyone know if there's a factory reset switch somewhere?
Attachments
there are dud ones out there - possibly the drives aren't seated fully, the interposers are dead (you are using interposers right? You haven't shown a picture of a drive in the sled with the interposer fitted) or there's a board failure in the circuit somewhere.
Just of note, straining to look at the top of the drive picture third along you provided, if you don't have an interposer fitted, the drive will be air gapped to the backplane so won't start. Even then, the backplane has FC connectors....
edit: ignore the flashing ID lights - they're only picked up by the SAN it's supposed to be connected to.
edit2: i don't know if windows will see any drive formatted other than 512byte per sector. Unsure about sg-utils but i believe it should - i've only really used camcontrol under bsd. This leads me back to points above about the drives not actually being attached to the backplane or else something significant being fried.
Just of note, straining to look at the top of the drive picture third along you provided, if you don't have an interposer fitted, the drive will be air gapped to the backplane so won't start. Even then, the backplane has FC connectors....
edit: ignore the flashing ID lights - they're only picked up by the SAN it's supposed to be connected to.
edit2: i don't know if windows will see any drive formatted other than 512byte per sector. Unsure about sg-utils but i believe it should - i've only really used camcontrol under bsd. This leads me back to points above about the drives not actually being attached to the backplane or else something significant being fried.
Last edited:
Yes. That is exactly what I was describing. Sorry for late reply. I was never tagged on your reply.
Last edited:
The 4 ports on the back are 2 that connect to the HBA ( Inner ) and 2 ( Outer ) that are used for daisy chaining. Unplug the diskshelf. Put the drives in the diskshelf, then plug the power in. Then boot the server up. Channel ID stuff will blink forever. Just ignore it. YOu need to be plugged into the circle not the diamond. Diamond is for daisy chaining.
Hey folks, I got two of these disk shelves years ago and had success using the 30x 1TB drives that came with them once I changed the sector size.
Recently I decided change one to use larger 8TB SATA disks but I'm running into an issue where the lsblk is showing all the 8TB drives as 2TB. I've tried multiple HBAs with the following results:
LSI7404EP-LC - All show 2TB
QLE2562 - All show 2TB
QLE8362 - No drives are detected by the HBA. (Proxmox throws this error at boot "qla2xxx ... Unable to read SFP data ... ")
I updated both QLogic cards with the latest firmware I could find on marvell.com but still facing the same issue.
Can anyone enlighten me as to what I'm doing wrong? Am I picking the wrong HBA?
Configuration:
Proxmox 8.1
(1x) KTN-STL3 shelf
(15x) EMC 250-076-900D SATA to FC Fiber Channel Interposer
(2x) EMC 303-127-000A 0T987N 4GB FIBRE CHANNEL CONTROLLER CARD
(2x) EMC DAE 071-000-553 400 Watt Power Supply
(1x) EMC 038-003-503 2.1M SFP To HSSDC2 4GB Fiber Channel Cable
Note: Only the top controller of the shelf is connected via the PRI port to the HBA.
Recently I decided change one to use larger 8TB SATA disks but I'm running into an issue where the lsblk is showing all the 8TB drives as 2TB. I've tried multiple HBAs with the following results:
LSI7404EP-LC - All show 2TB
QLE2562 - All show 2TB
QLE8362 - No drives are detected by the HBA. (Proxmox throws this error at boot "qla2xxx ... Unable to read SFP data ... ")
I updated both QLogic cards with the latest firmware I could find on marvell.com but still facing the same issue.
Can anyone enlighten me as to what I'm doing wrong? Am I picking the wrong HBA?
Configuration:
Proxmox 8.1
(1x) KTN-STL3 shelf
(15x) EMC 250-076-900D SATA to FC Fiber Channel Interposer
(2x) EMC 303-127-000A 0T987N 4GB FIBRE CHANNEL CONTROLLER CARD
(2x) EMC DAE 071-000-553 400 Watt Power Supply
(1x) EMC 038-003-503 2.1M SFP To HSSDC2 4GB Fiber Channel Cable
Note: Only the top controller of the shelf is connected via the PRI port to the HBA.
Last edited:
I don't know F/C however that 2TB sounds suspiciously like the original SAS1 drive limitation of 2TB. So my guess is that your EMC 303-127-000A 0T987N which handles the F/C and I presume has some kind of SAS controller on it for the disk may be SAS1? (or its firmware is limited by max number of blocks/LBN.)
KTN-STL3 use SAS controllers
KTN-STL4 use fiberchannel controllers.
You can put sas controllers into that unit if I recall.
Had one with sas controllers filled with 14tb at one point worked perfectly. Not sure what the limits on the Fiber channel controllers. The Fiber channel controllers could be limited to 2GB but not sure why that would be or if that is even true. Sas controllers do not have any issues far as I know. I run 6 KTN-STL3s. Could be anything with the fiber channel. Sas II just works. Hope it works out.
Strange my STL3 had the FC setup and not the SAS. Oh well, I got them both along with the EMC cabinet for $50 so I can't complain.
So what cables and HBA would I need after switching to the SAS controllers to utilize the max speed of the shelf?
Those controllers are interchangeable. You can use a cheap
LSI 9286-8e Something like this on ebay. LSI 9286-8e = LSI HBA SAS 9207-8e PCI-E 3.0 w/ LSI P20 IT Mode ZFS FreeNAS F/H | eBay For cables you can use something https://www.amazon.com/gp/product/B01KH9OO9G Few of these and you are golden. If you want max speed you would want a quad card. I use a quad card 9400-16e and the cables I used are 10Gtek SFF-8644 to Mini SAS SFF-8088 . This is if you are using two units I use this card and have two sets of daisy chains. 3 units each.
LSI 9286-8e Something like this on ebay. LSI 9286-8e = LSI HBA SAS 9207-8e PCI-E 3.0 w/ LSI P20 IT Mode ZFS FreeNAS F/H | eBay For cables you can use something https://www.amazon.com/gp/product/B01KH9OO9G Few of these and you are golden. If you want max speed you would want a quad card. I use a quad card 9400-16e and the cables I used are 10Gtek SFF-8644 to Mini SAS SFF-8088 . This is if you are using two units I use this card and have two sets of daisy chains. 3 units each.
Attachments
Last edited:
I only have 2 shelves. Can you explain the optimal topology and cabling?
I'm not quite understanding the need for daisy chaining. Couldn't I just use a dual port card, connecting each to the top controller of each shelf and keep them separate?
I also do not think I'd use the second lower controller as the last time I did that each drive appeared twice. Unless of course there's a way to double up the speed instead of having the redundancy.