Finally got everything racked/stacked how I like it. Well, not really, since everything changes weekly - but its much better than it was! Ordered a self-assembly rack from StarCase.com sized to fit a small space in my garage. Didn't really plan for the new switch (Cisco SG500X-48 - 48Gbe + 4 10Gbe SFP) and that C6100 Patrick found for us. But its all worked out real nice.
A few pics:
The whole thing tucked away in its little corner. It looks crooked because it is. Well, it isn't, but there is a small rise in the concrete floor near the wall and when you park it, it lists. Its all level and square when you pull it out, really, it is!
And pulled out to access the back:
Cabling into the house, etc, hides in this little cabinet above it.
24 port pp to outlets inside the house, fiber pp to outlets at my desk, 24 port pp to rack. Also a small POE switch to power a few things (2 WiFi APs, outside cameras).
Lower half has Comcast cable modem, Mikrotik router and HD Home Run 3-tuner cable TV tuner for MythTV
Lower part of the rack - servers (bottom to top)
APC SmartUPS 1500 RT (2U gap below it is for extra APC battery shelf when I find the right deal)
Norco 4220 with 20x2TB Hitachi, X8SIA-F, X3450, 32GB ECC, 3x M1015, Intel 10GB X520-DA2. Bare metal Solaris 11.1 ZFS server.
Dell C6100 w/4 nodes, each with 2xL5638, 48GB ECC & 10Gbe (currently Mellanox, but Intel mezz cards on the way - thanks Justin!). All lit up as 4-node Proxmox VE cluster, but no VMs built yet.
I-Star USA 4U case with 11x 3.5" hotswaps, 2x 2.5" hotswaps & 600w PSU. Currently no electronics. Will be backup server soon.
Norco RPC-270. Currently empty, nothing, just taking up space and pretending to be a shelf (used to house ESXi server).
Upper part of rack - Network (bottom to top)
Avocent 17" rackmount KVM (there is a 16 port Trendnet KVM switch in the back of rack). All console ports connected, including all 4 nodes of the C6100
Juniper EX2500 24 port 10Gbe SFP+ switch (yum!). Loud and power hungry, but 24 ports 10Gbe - love & hate live together here. Haven't cabled all the C6100 10Gbe - waiting on the Intel Mezz cards first.
Cisco SG500X-48. Was going to retire the Juniper and just use the 4x 10Gbe on this baby 'till the 6100 came along...
24 port pp cabled to pp in closet. This keeps the "house" wire demarced from the rack. Connected by the fat bundle of cables (the smaller bundle in the white split inner-duct is the 6 fiber pairs from the upper pp in the closet)
A shot with the Avocent opened up.
I'm a big believer in remote management via IPMI and/or vPRO/AMT - but sometimes you just can't live without a real console! Besides - I got it cheap because the left rail was damaged and it wouldn't slide back in (amazing - somebody gave away a $400 KVM shelf for <$100 over something that could be fixed with large needle nose pliers, 2 minutes and a couple of grunts).
And the hot-swaps inside the I-Star 4U.
I have 12 2TB Hitachi's to fill it with, but looking for a new MB/CPU. Might buy 3TB or 4TB drives if the prices would get anywhere near pre-flood levels...but I might be waiting a really long time for that to happen!
Its a decent setup. Also has two rackmount outlet strips mounted in back, one near the bottom and one near the top (stupid ones - no monitoring). The Trendnet KVM switch is mounted near the top on the back side (and the ugly bundles of cables you can see in the side view are the KVM cables). I'm not completely happy with now I have the power run. I need to run data cables on both sides in back so I've got power and data running parallel on the back post - a no-no in "proper" cabling. I also need to figure out some kind of swing-arm for the Norco and I-Star cases as they need to slide out sometimes and the cables get all tangled & stressed. But other than that I am happy with how it turned out.
Biggest problem is power - the entire rack draws about 400w nominal idle load from the SmartUPS and goes up towards 850w when I have things running (ugh!). Might need to invest in solar before I buy any more toys!
A few pics:
The whole thing tucked away in its little corner. It looks crooked because it is. Well, it isn't, but there is a small rise in the concrete floor near the wall and when you park it, it lists. Its all level and square when you pull it out, really, it is!
And pulled out to access the back:
Cabling into the house, etc, hides in this little cabinet above it.
24 port pp to outlets inside the house, fiber pp to outlets at my desk, 24 port pp to rack. Also a small POE switch to power a few things (2 WiFi APs, outside cameras).
Lower half has Comcast cable modem, Mikrotik router and HD Home Run 3-tuner cable TV tuner for MythTV
Lower part of the rack - servers (bottom to top)
APC SmartUPS 1500 RT (2U gap below it is for extra APC battery shelf when I find the right deal)
Norco 4220 with 20x2TB Hitachi, X8SIA-F, X3450, 32GB ECC, 3x M1015, Intel 10GB X520-DA2. Bare metal Solaris 11.1 ZFS server.
Dell C6100 w/4 nodes, each with 2xL5638, 48GB ECC & 10Gbe (currently Mellanox, but Intel mezz cards on the way - thanks Justin!). All lit up as 4-node Proxmox VE cluster, but no VMs built yet.
I-Star USA 4U case with 11x 3.5" hotswaps, 2x 2.5" hotswaps & 600w PSU. Currently no electronics. Will be backup server soon.
Norco RPC-270. Currently empty, nothing, just taking up space and pretending to be a shelf (used to house ESXi server).
Upper part of rack - Network (bottom to top)
Avocent 17" rackmount KVM (there is a 16 port Trendnet KVM switch in the back of rack). All console ports connected, including all 4 nodes of the C6100
Juniper EX2500 24 port 10Gbe SFP+ switch (yum!). Loud and power hungry, but 24 ports 10Gbe - love & hate live together here. Haven't cabled all the C6100 10Gbe - waiting on the Intel Mezz cards first.
Cisco SG500X-48. Was going to retire the Juniper and just use the 4x 10Gbe on this baby 'till the 6100 came along...
24 port pp cabled to pp in closet. This keeps the "house" wire demarced from the rack. Connected by the fat bundle of cables (the smaller bundle in the white split inner-duct is the 6 fiber pairs from the upper pp in the closet)
A shot with the Avocent opened up.
I'm a big believer in remote management via IPMI and/or vPRO/AMT - but sometimes you just can't live without a real console! Besides - I got it cheap because the left rail was damaged and it wouldn't slide back in (amazing - somebody gave away a $400 KVM shelf for <$100 over something that could be fixed with large needle nose pliers, 2 minutes and a couple of grunts).
And the hot-swaps inside the I-Star 4U.
I have 12 2TB Hitachi's to fill it with, but looking for a new MB/CPU. Might buy 3TB or 4TB drives if the prices would get anywhere near pre-flood levels...but I might be waiting a really long time for that to happen!
Its a decent setup. Also has two rackmount outlet strips mounted in back, one near the bottom and one near the top (stupid ones - no monitoring). The Trendnet KVM switch is mounted near the top on the back side (and the ugly bundles of cables you can see in the side view are the KVM cables). I'm not completely happy with now I have the power run. I need to run data cables on both sides in back so I've got power and data running parallel on the back post - a no-no in "proper" cabling. I also need to figure out some kind of swing-arm for the Norco and I-Star cases as they need to slide out sometimes and the cables get all tangled & stressed. But other than that I am happy with how it turned out.
Biggest problem is power - the entire rack draws about 400w nominal idle load from the SmartUPS and goes up towards 850w when I have things running (ugh!). Might need to invest in solar before I buy any more toys!
Last edited: