Xeon D-1541 Quanta storage servers on ebay

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
C

cookiesowns

Active Member
Feb 12, 2016
235
86
28
30
tjk said:
So, I ordered a few of these to play with, 2 of the 4 had physical damage and can't be racked, waiting on the vendor for replacement. Shipping box and packaging was fine, so I assume they were damaged when removed and not noticed when they packed them up for shipment.

They came with both mezz slots filled, one with a 2x 10gb sfp and one with a 3008 LSI mezz card. I replaced the LSI mezz with a 3108 series mezz, easy swap.

I was concerned if these would be serviceable (replace a HDD) without shutting them down, and to my surprise, they are! You can pull the front of the system out that has the HDD's in them while the unit is running, which allows you to service/swap HDD's, without taking the unit down or out of service...risky but doable.

The 12x 3.5" are SAS only, the 4x front 2.5" are SATA or SAS, and the rear 2x 2.5" are SATA only. The 3008/3108 LSI raid supports the 12x 3.5" HDD's and the front 4x 2.5" SSD's. The rear 2.5" are NOT supported by the LSI raid mezz. There is a PCH bios setting for these rear slots. Not sure if they are fake raid or what, but I haven't found anywhere to setup raid on the rear 2x 2.5" SSD's yet.

Some pics and a short video showing the system and the ability to "slide out" the 12x 3.5" portion of the server - https://photos.app.goo.gl/ftqiaSWYK3WzXctaA

It looks like these were part of some large CEPH or other distributed storage setup at some large cloud provider, they have labels on them with various StorageX notations, storage127, storage4, etc.
Click to expand...
Did you test SATA drives in the 3.5" bay, and they don't get detected?
 
BLinux

BLinux

cat lover server enthusiast
Jul 7, 2016
2,748
1,121
113
artofserver.com
tjk said:
Seagate, wd, hgst. Sas only even the docs say that.
Click to expand...
that's interesting... is that because of the expander chip they use? The SAS3008 controller certainly can handle SATA, but I assume the backplane has a SAS-(3? 2?) expander (8 SAS lanes vs 12x 3.5" bays)

most SAS systems can usually handle SATA...
 
tjk

tjk

Active Member
Mar 3, 2013
494
201
43
BLinux said:
that's interesting... is that because of the expander chip they use? The SAS3008 controller certainly can handle SATA, but I assume the backplane has a SAS-(3? 2?) expander (8 SAS lanes vs 12x 3.5" bays)

most SAS systems can usually handle SATA...
Click to expand...
Not sure, I dropped in the 3108 ocp mezz. The system is 12g sas. The Sata drives I tested lit up as red and the raid bios didn't see them for the 3.5 bays.

The 4x 2.5 bays I tested sas and sata ssds and it saw both of them.
 
C

cookiesowns

Active Member
Feb 12, 2016
235
86
28
30
tjk said:
Not sure, I dropped in the 3108 ocp mezz. The system is 12g sas. The Sata drives I tested lit up as red and the raid bios didn't see them for the 3.5 bays.

The 4x 2.5 bays I tested sas and sata ssds and it saw both of them.
Click to expand...
It's a firmware thing 100%. The physical hardware can support it, but depending on firmware, along with the firmware version... SATA tunneling over wideport SAS or also called databolt could be wonky or disabled.
 
T

tekmunki

New Member
Aug 4, 2019
15
18
3
tjk said:
Seagate, wd, hgst. Sas only even the docs say that.
Click to expand...
I got a unit on Friday and just finished setting it up. The 4 SATA drives I had laying around from 2012 3xWD blacks and some random Samsung SATA2 drive do seem to work with the 3008 card but not with every boot. Some times I've had a reboot where it will not detect a random drive but a follow-up reboot seems to get it to pick up all 4 SATA drives. I've set up a RAIDZ with the blacks and the Samsung as a hot spare and they seem to work normally. I'm gonna try updating the bios and firmwares tonight to see if that helps at all.

I have yet to put the 8x 12GB SAS Seagate ST6000NM0074's I got from that Ebay deal in as I'm waiting on an M.2 and some 7mm SSD's to fill the front 4 bays for cache/log drives. For $425 shipped I'm super impressed with what this thing offers in a 1U 12x 3.5" trays, 6x 2.5" trays, 1xM.2 slot, 4xDDR4 slots, 4x10gb sfp+ ports, 8c Xeon-D, and dual PSU. Not to mention it comes with rails.

The ICX-6450-48P I have mounted above is louder by far than the quanta when it's running but with the door closed I can't hear the whole rack at all. Without any tweaks using some Finisar sfp+ modules and OM4 patch fiber from fs I'm getting full 10gb speeds. I put two EVO DC SSDs in the back two trays and FreeNAS had no issue raiding them for the boot drive.







From the built in IPMI:


All in all I'm super impressed with these 1U units with 4x32GB 2400mhz DDR4 and the 4x3.5" HDDs 2x2.5" SSDs drives it's pulling ~60w with only one PSU plugged into the UPS when idle. I'm thinking of picking up 2 more to play around with at work.
 
Last edited:
  • Like
Reactions: BLinux
T

tekmunki

New Member
Aug 4, 2019
15
18
3
BLinux said:
@tekmunki can you post some pictures of how you have it mounted in your rack? Just curious how far out they stick out the back?
Click to expand...
Sure no problem there's a Dell r730 mounted above if you need a scale I think those are 26" long? I have a Startech 25U adjustable rack standard 19" wide I've included a pick of what I have it set to as far as the adjustment rails.



 
  • Like
Reactions: Blinky 42, semidetached and BLinux
BLinux

BLinux

cat lover server enthusiast
Jul 7, 2016
2,748
1,121
113
artofserver.com
Schoondoggy said:
The specs show SAS and SATA support. The CCL shows SATA drives. There were issues with the expander on these chassis. There is updated firmware for the expander and the SAS controller, under the download menu:
QuantaGrid SD1Q-1ULH | 1U Storage Server | QCT.io
Click to expand...
Do you know how that SAS expander firmware is applied? There's a .bin and .fw file. I didn't see any instructions in the zip file...
 
tjk

tjk

Active Member
Mar 3, 2013
494
201
43
Firmware update:
1. Use "fdl" command to update FW through console port.
2. Use "MegaCli" tool to update FW through Raid Card.
(Please download tool from Avago website. And refer S2PH firmware download SOP.)
Click to expand...
According to the readme, you need to install the megacli linux tools to apply the expander firmware.
 
F

frsp_1

Member
Apr 13, 2018
58
70
18
New York, NY
I've gotten my replacement unit and done some testing over the weekend. A couple of things to note:
  • SATA (3,6) and SAS(3,6,12) drives work in the 3.5" bays after firmware update.
  • couldn't test the 2.5" bays in the front as I don't have any 7mm 2.5" SAS disks...
  • it's already been addressed but the bays are all hot swap and the tray remains active when extended
  • MB works fine with 2133Mhz or 2400Mhz DDR4 (ECC and non-ECC). It also will take mixed size DDR but will throw an error at the BIOS screen
  • Don't update to the most recent BIOS before updating the SAS expander (still haven't worked out how to do that)
  • I'm running the following firmware:
    • BMC version 3.31.00
    • BIOS version S1Q_3A12.02
    • SAS controller (QS-3008) versions:
      Quanta SAS3008 Mezz FW Version : 1B10
      LSI Code Base Version : Phase16.0-16.00.08.00
      LSI BIOS Version: 8.37.00.00
      LSI UEFI Version: 18.00.00.00
    • SAS expander firmware version 0520
  • the latest BIOS (3A14) release notes have this gem hidden in them:
    • "- Before update BIOS to 3A13 or later version, Check BMC FW is newer than 3.31 and Expander FW is newer than 1.08
      System may hang on POST if above FW version not compatiable."
  • SFP+ ports all work fine with DAC cables.
  • The rails have a small bracket on the back that is held on with four screws which needs to be removed to fit in a standard size rack. The server will project a few inches past the rear rails but it seems to work fine (I don't have rear or front doors on my rack).
  • default BMC user/pass is admin/admin
  • BMC uses an AST2400 with the awful Java remote KVM (similar to Asrock Rack and Super Micro boards of the same era) so it doesn't work on most every stock OS without a lot of fiddling.
If someone figures out how to flash the expander firmware, please let me know as I would like to update to the latest BIOS.
 
  • Like
Reactions: nedimzukic2, scobar, Blinky 42 and 4 others
F

frsp_1

Member
Apr 13, 2018
58
70
18
New York, NY
Also, this nugget from the SAS expander firmware release notes:

Firmware update:
1. Use "fdl" command to update FW through console port.
2. Use "MegaCli" tool to update FW through Raid Card.
(Please download tool from Avago website. And refer S2PH firmware download SOP.)

Not sure what console port they mean, there is a mini-USB port on the motherboard marked "serial" but I couldn't get anything to work with it and didn't feel like getting the scope out to see what it was doing electrically.
Also, can't find any documentation for the MegaCli firmware download SOP...
 
BLinux

BLinux

cat lover server enthusiast
Jul 7, 2016
2,748
1,121
113
artofserver.com
frsp_1 said:
  • the latest BIOS (3A14) release notes have this gem hidden in them:
    • "- Before update BIOS to 3A13 or later version, Check BMC FW is newer than 3.31 and Expander FW is newer than 1.08
      System may hang on POST if above FW version not compatiable."
Click to expand...
Thank you for all the updates and awesome information! :) I quote the above to emphasize the importance of it.... hopefully anyone else here will see this point. Or, since you're the OP, can you put that in the first post in bright red or something?
 
F

frsp_1

Member
Apr 13, 2018
58
70
18
New York, NY
BLinux said:
Thank you for all the updates and awesome information! :) I quote the above to emphasize the importance of it.... hopefully anyone else here will see this point. Or, since you're the OP, can you put that in the first post in bright red or something?
Click to expand...
Great idea! Guess I should have led with that one, huh?
Thanks.
 
You must log in or register to reply here.
Top Bottom