AVOID - 4U, 2x Node, 4x E5 V3/V4, 56x LFF SAS3 3.5" bay - $299 - CISCO UCS C3260

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

eduncan911

The New James Dean
Jul 27, 2015
648
506
93
eduncan911.com
Regarding "supported" these things will run anything. But for Intel NVMe drives you need certain firmware levels to hit that metric.
What exact specs/connectors are we looking for for NVMe drives? (I just had a dizzying crash course on OCU Link 1 and 2, MiniSAS and SlimSAS, NVMe PCIe and U.2 drives, etc. Etc.

Does this machine support U.2 drives as simple Drop In!? And I'm guessing only PCIe 3.0 versions, since the Intel C612 chipset is only 3.0.

Here I was, thinking only SAS3 12Gbps was the fastest we could install. And that it supports standard SATA drives. And now I learned what U.2 is, and NVMe, etc and now I need to know if I can setup an U.2 ZFS ZIL as that would be sweet...
 
  • Like
Reactions: Samir and jtaj

oddball

Active Member
May 18, 2018
206
121
43
42
I agree with this as well, as I've only seen the full options in LSI controllers via the CIMC.



Are the Linux drivers really necessary? That's a Yikes, if 3 GB for a Linux driver.

Last question we are all trying to answer:

Does the MegaRAID LSI controller present the true "raw" drive to the OS, when the controller is set to Pass-through mode? No RAID0 Volumes.

This is killing me and a few others from buying these machines and setting them up for ZFS.
You can set the disk to JBOD mode and it's passed straight through to the OS. You can mix them as well, we have some with RAID on a few drives and then JBOD for others. No issues.
 

Digital Spaceport

New Member
Apr 17, 2022
11
10
3
Austin, Tx
digitalspaceport.com
Is there a "Way" for a homelab pleeb like me to get CIMC?

I'd ditch the BIOS modification directly. You want to use CIMC to manage the BIOS options and setup. It will overrule whatever is set in the BIOS anyways.

I've installed windows on a LOT of UCS servers with NVMe. As others have noted you need to be in UEFI mode, your error is showing you're in legacy boot mode.

In the UEFI shell list the devices, you should be able to see your NVMe drive. UEFI is similar to a lspci shell, you can probe devices.

In 100% of the cases where I had issues the hardware was detected fine but the software wasn't setup correctly to see things.

CIMC unlocks a lot of other BIOS settings that you won't have access to. You can change boot order, set the boot loader path, change network configs etc. This is how you setup to boot from the bottom SSD's as well.

You can download drivers from the Cisco support portal. They are free without a support contract, you want the c-series stand alone managed servers. They have a driver pack for Windows, Linux, VMWare, Xen, they're about 3GB each. There is also a firmware pack that's a few gigs as well.
 
  • Like
Reactions: Samir

oddball

Active Member
May 18, 2018
206
121
43
42
In the server node there is one or two U.2 drive connections. If you install the expander node you get another two U.2 drives and two PCI slots.

In theory you could have four U.2 drives and then 2 HHHL NVMe cards as well for six total NVMe drives.

Everything else is 12Gbps. There are some really fast SAS SSD's out there that RAIDed can scream.
 

oddball

Active Member
May 18, 2018
206
121
43
42
Is there a "Way" for a homelab pleeb like me to get CIMC?
Yeah, it's built into the server. Just plug the management port into a switch and go to the DHCP assigned IP, if there's a static IP you can reset it. Default password is something like "admin" "Cisco1234".

Management is on the SOIC module. You can manage multiple of these with fabric interconnects. Management software is pre-installed on there as well, you can't remove it.
 

oddball

Active Member
May 18, 2018
206
121
43
42
Regarding drivers. Cisco bundles all of their drivers together. So it's all supported hardware in a giant zip for all versions of Linux they support.
 

jtaj

Member
Jul 13, 2021
74
38
18
CIMC unlocks a lot of other BIOS settings that you won't have access to. You can change boot order, set the boot loader path, change network configs etc. This is how you setup to boot from the bottom SSD's as well
how can I use the CIMC for other bios settings or even access them?

In terms of noise, if you have 'supported' hardware and the firmware is up to date these are very quiet. I had an NVMe drive with out of date firmware and it caused the fans to ramp.
If you have CIMC 4.2 you can modify the fan level and set it to low.
I took out the NVMe later on with just 1 node, 1 SIOC and 1 CPU so I don't see what would be the "unsupported" hardware. but if theres a fan setting then I'd love to have access to that. is CIMC 4.2 a firmware and how do I check what version I have?

also since you mentioned NVMe SSD, I'd assume thats the M4 node and above, M3 node also just as loud and doesn't have NVMe drive so maybe it's just the setting?
 
  • Like
Reactions: Samir

oddball

Active Member
May 18, 2018
206
121
43
42
Seems like you need to start over with this.

On the SIOC there is a 1GbE ethernet port. Plug that into a switch that has dhcp, make sure the link comes up. Then use your browser to hit the IP assigned. If the username/password aren't default then boot the server with the kvm. When it gets to the screen with the Cisco logo and the hardware details for the node press F8. This is the CIMC network setup. You can fully reset to factory default here, or you can just set the IP to dhcp. I'd reset to default.

It'll take 3-4min to reset. Once it's reset hit up the IP your dhcp server assigned in a browser. Plug in the default username and password.

This is CIMC, Cisco Integrated Management Console... you can control every aspect of the machine via this. All of the bios settings are in here, there are more bios settings than what the bios control will give you. You can control storage, raid, power management, every aspect of the machine from this web console.

Here is a fun fact. If you plug in a serial port there is a CLI to the CIMC as well. I've had to use it one or two times to reset hardware details when upgrading CPU generations. You can do everything from the cli that you can do from the browser.
 

jtaj

Member
Jul 13, 2021
74
38
18
@oddball so thats basically what Ive been looking for all along. I'd have to learn how to use CIMC. I am just upset that BIOS is so lacking in this regards seems half assed effort from cisco. I'll give those a try.
 
  • Like
Reactions: Samir

oddball

Active Member
May 18, 2018
206
121
43
42
These are loaded with custom BIOS's and to get to all the advanced features you need to use their tool to access it.

The way these are supposed to be managed is with something called UCS Manager. It's a central management platform where you can template out bios profiles and boot profiles and then apply them dynamically to machines. It's really helpful when managing a fleet of servers. It also means you never have to do anything with the actual server, just the management interface/appliance.
 
  • Love
  • Like
Reactions: Samir and jtaj

jtaj

Member
Jul 13, 2021
74
38
18
These are loaded with custom BIOS's and to get to all the advanced features you need to use their tool to access it.

The way these are supposed to be managed is with something called UCS Manager. It's a central management platform where you can template out bios profiles and boot profiles and then apply them dynamically to machines. It's really helpful when managing a fleet of servers. It also means you never have to do anything with the actual server, just the management interface/appliance.
if I may get some more knowledge from you. I set the password to CIMC prior and this time loggin in by just typing the password but it simply just blackscreen with nothing loading showing. do you know what the issue may be? also if I were need to reset the password, how can I go about this or would reset CMOS battery be enough?
 
  • Like
Reactions: Samir

oddball

Active Member
May 18, 2018
206
121
43
42
Strange, I've never seen that. What happens when you connect via the serial port? Can you navigate around? Maybe reset via serial?
 
  • Like
Reactions: Samir

jtaj

Member
Jul 13, 2021
74
38
18
Strange, I've never seen that. What happens when you connect via the serial port? Can you navigate around? Maybe reset via serial?
i dont have serial connection. only KVM to VGA but finally got time to try out the DHCP thing. as you can probably tell I have always been installing via USB rather than manage them through network.

the dhcp thing, do you have a step by step of it? im not entirely sure what I need to do here.
 
Last edited:
  • Like
Reactions: Samir

Digital Spaceport

New Member
Apr 17, 2022
11
10
3
Austin, Tx
digitalspaceport.com
Can you tell me if these come with the heatsinks? I have a friend who ordered one and we will be working on it to get to the bottom of these answers but might be delayed if we need to get heatsinks.

i dont have serial connection. only KVM to VGA but finally got time to try out the DHCP thing. as you can probably tell I have always been installing via USB rather than manage them through network.

the dhcp thing, do you have a step by step of it? im not entirely sure what I need to do here.
 
  • Like
Reactions: Samir and jtaj

jtaj

Member
Jul 13, 2021
74
38
18
Can you tell me if these come with the heatsinks? I have a friend who ordered one and we will be working on it to get to the bottom of these answers but might be delayed if we need to get heatsinks.
I did not buy them from ebay but from some local people rid off of their system so I am not sure. but looks like listing says it comes with heatsink.
 
Last edited:
  • Like
Reactions: Samir

TTEG33

New Member
Apr 23, 2022
2
2
1
Full caddy chassis price dropped to 500, unfortunately the shipment to Canada is 995 according to the seller
 
  • Like
Reactions: Samir