EXPIRED UK/EU - C240 M5 26sff - 40gbe

[ano]

is it stock occulink to front as well? manual is not saying anything about what cable they use

the front did not look like occulink

and they are using a pcie switch for nvme for morethan X drives, no bifurcation :|

I got single nvme working in dualsupermicro m.2 pci gen3 card at least, gen4,entire server hangs, and it doesnt detect both drives, regardless

 

[turbo]

Thanks mrpasc for posting the Cisco server installation doc, I didn't realize there were -HD versions of the RAID and SAS cards that makes sense.

you can toggle JBOD mode on the drive/controller, it seems to be working decent with ZFS from the labbing today

you can connect the cables to normal HBA, (front is non expander) but there is i2c magic happening

You're using the UCSC-RAID-M5 controller successfully? I do see the doc mentions JBOD mode, but I usually try to avoid RAID controllers for ZFS.

Anybody have a clue what the P/N is for the "Mini-storage M.2 SSD carrier" referenced? Since I'm missing the rear NVME backplane I'm thinking this might be the cheapest way to add some NVME storage to the box?

 

[mrpasc]

UCS-MSTOR-M2=
caveat: the socket has only x2 PCIE connection (was initially used with similar carrier for two SD cards, the M.2 option came after ESXI 7.0 was announced )
So it’s intended as boot device, not for real NVME storage

 

[mrpasc]

You're using the UCSC-RAID-M5 controller successfully? I do see the doc mentions JBOD mode, but I usually try to avoid RAID controllers for ZFS.

Not being @ano but can answer: you can either just set the attached disks into JBOD mode or the whole controller. Later needs to have the battery pack removed, otherwise the option to change „controller personality“ stays grayed out.
There is an dedicated „HBA“ available for M5 as well but never seen in the wild. UCSC-SAS-M5HD=

 

[ano]

for like VSAN 6/7 theeese things would have been nice, m2 boot, nvme cache in rear, lots of storage etc

makeing ceph lab and misc labs with them.

getting nvme working properly seems a tad bit expensive, can just buy h12 and stuff like that instead, but its ending up like a dirt cheap lab with sata/sas devices and 6132 cpus.


fan mode "accoustic" is also key btw. even there it cooles m.2 nvme to 35c.. rear mounted nvme m.2on pcie...

 

[turbo]

UCS-MSTOR-M2=
caveat: the socket has only x2 PCIE connection (was initially used with similar carrier for two SD cards, the M.2 option came after ESXI 7.0 was announced )
So it’s intended as boot device, not for real NVME storage

Thanks! I found that P/N around but I wasn't sure if it was only for the UCS blade servers. That's a shame though that it's 2x. Will a pair of Optane 900P PCI cards work in the server? I'm not sure if it will barf with non-Cisco certified HW. I'd rather not use USB for boot but I don't really want to spend hundreds on adapter boards either.

250GB of NVME for is enough for what I'm hoping to use it for, then stuffing it with SATA SSDs

 

[mrpasc]

I do run both my C220 M5 as well the C240 M5 with Optane 900p. It works.

 

[michel333]

Got mine today. No time to start it, just checked everything is there. Pretty happy with it!

 

[Cruzader]

Ive gotten as far as unpack 3 units and still missing the 4th, i suspect i will end up being refunded for the last unit with how its looking atm.

 

[michel333]

I have some 1.6 TB sas ssd ibm micron formatted as 528 byte sector size... probably because of T10-pi standard.

Tried to format to 512 bytes using sg_format, not working because of error. I guess I need an hba or megaraid that supports T10-pi standard, or a pcie ibm power megaraid... not sure.
I bought this sas ssd pretty cheap, I don't think that Cisco raid adapter will work with these drives but I'll give it a try.
I also ordered a pcie ibm power raid adapter 57b4 but I am not sure it is compatible with non ibm servers...
I checked some info on forums ... I will be grateful if you guys have some knowledge about it!

 

[Cruzader]

Tried to format to 512 bytes using sg_format, not working because of error. I guess I need an hba or megaraid that supports T10-pi standard, or a pcie ibm power megaraid... not sure.

I have a supermicro board with a LSI3008 chip in IT mode just for sg_format.
Tested 10+ hbas i had and none of them would read the drive at a low enough level to do it before the 3008 could, so just kept it for that.

 

[michel333]

I have a supermicro board with a LSI3008 chip in IT mode just for sg_format.
Tested 10+ hbas i had and none of them would read the drive at a low enough level to do it before the 3008 could, so just kept it for that.

I have a avago megaraid sas 9341-4i, on the forum people successfully flashed it to it mode (hba), no time to try this yet.

 

[michel333]

OMG.... This is loud!

 

[mrpasc]

UCS C-Series M5 Server Components Relation to Fan Policy and Fan Noise

This document describes the Unified Computing Server (UCS) C-Series M5 server components relation to Fan Policy and Fan Noise.

www.cisco.com

 

[michel333]

UCS C-Series M5 Server Components Relation to Fan Policy and Fan Noise

This document describes the Unified Computing Server (UCS) C-Series M5 server components relation to Fan Policy and Fan Noise.

www.cisco.com

Nice, thanks!

 

[ano]

its still very loud, louder than any 2U I've ever used (when you try to quiet it)

 

[michel333]

I try to enter icmi, F8, but it is asking for current password

 

[michel333]

I meant CIMC setup, f8, for setting fans, it has a current password...

 

[Layla]

its still very loud, louder than any 2U I've ever used (when you try to quiet it)

Agreed - anyone know if these things fan speeds can be remotely controlled via IPMI like SuperMicro servers can be?

 

[ano]

it has a serial console, we used that to reset the cimc password etc

@Layla not found a a way yet, the fan themselfs are HUUUUUUGE,double length style. but even in acoustic mode, they push so much air you can benchmount m.2 in pcie slot...