Colo tomorrow! Tips/ tricks
- Thread starter Patrick
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Things I always bring to make sure I don't have to go to the colo/datacenter twice:
1. Extra screwdrivers
2. Extra patch cables (unless facility provided) (And I bring some anyways)
3. Laptop w/4G card and CD/dvd burner. Laptop can test external connectivity before you walk away from it.
4. Blank CD-R/DVD-R media, and thumbdrives
5. Noise isolating or noise cancelling headphones if I'm going to be there for a while, otherwise earplugs
6. Wallet/Drivers License so they let me in.
With the exception of the BGP configuration, is there any reason you can't have pFsense + vpn setup and tested before you get on site?
1. Extra screwdrivers
2. Extra patch cables (unless facility provided) (And I bring some anyways)
3. Laptop w/4G card and CD/dvd burner. Laptop can test external connectivity before you walk away from it.
4. Blank CD-R/DVD-R media, and thumbdrives
5. Noise isolating or noise cancelling headphones if I'm going to be there for a while, otherwise earplugs
6. Wallet/Drivers License so they let me in.
With the exception of the BGP configuration, is there any reason you can't have pFsense + vpn setup and tested before you get on site?
Fiberhub is awesome. They had everything racked for me when I got there and provided shorter cables! Awesome! Used the phone for 4G but probably could have used a second charged battery (I had two at 30% in my bag.)
Forgot blank CD/ DVD. The crash cart/ kvm cart had a USB CD rom though and everything wired nicely into a USB hub. The Dell C6100's only have two USB ports per node and no internal port so good thing!
Ended up using Yurbuds which helped.
I had the drives with me. Did have the base pfsense installed on the two drives so basically hooked up a cart and got the box up and Steven from Rack911 working on the setup.
Forgot blank CD/ DVD. The crash cart/ kvm cart had a USB CD rom though and everything wired nicely into a USB hub. The Dell C6100's only have two USB ports per node and no internal port so good thing!
Ended up using Yurbuds which helped.
I had the drives with me. Did have the base pfsense installed on the two drives so basically hooked up a cart and got the box up and Steven from Rack911 working on the setup.
Their head of operations stopped by when I did the install and helped big time. Four hands tamed the beast! I made the mistake of doing the mockup in a half rack that I could only have easy front access to. That meant I sent cables that were too long since I was planning to be able to pull the servers out. With the cold spare chassis, easy front/ rear access, and the C6100's having hot swap everything, it was super easy to work on. I actually did a CPU/ memory swap on the pfsense nodes when I got there. Two screws and they popped right out.
Looks really good, I hope your main site article discusses the layout of the physical and logical networking. I am assuming the cat5e colors have a meaning, if so particularly interested in the red cable connecting the two switches.
My best guess on the cable color coding:
- Green = admin/IPMI
- Pale-blue = "black" side of firewall (safe side)
- Red = "Red" side of firewall (exposed). Red cable between nodes connects HA pair of pfsense firewalls.
(note there also appears to be a red inter-switch link between the HP switches too)
Did I get it right?
- Green = admin/IPMI
- Pale-blue = "black" side of firewall (safe side)
- Red = "Red" side of firewall (exposed). Red cable between nodes connects HA pair of pfsense firewalls.
(note there also appears to be a red inter-switch link between the HP switches too)
Did I get it right?
Last edited:
Yea that was my guess my on the reds as well. Hence wondering why there is red inter-switch link....me thinks there should be no red on those switches.
Very close. 100% correct on green and blue. Red has the two uplink ports to different DC routers. The red between the two pfsense boxes is a carp link. The red between the two switches is the path for the switches to talk. I loosely defined red as "non server to switch" interfaces.
Hey Patrick,
As I've been playing around getting a baseline on the fans, when the speed up, etc, I noticed that the C6100's cooling is REALLY degraded when you run it with one or more of the sleds removed. You have a nice low-resistance pathway through the empty node for air to get out without doing any good. Seems to lower the overall positive pressure in the back half of the chassis and the temps go up - which means the fans speed up, power consumption goes up (those fans use a LOT of power - 200+ watts just for the fans when they reach full speed). You might want to consider storing cold sleds in those empty holes, or possibly mocking up some kind of blank.
As I've been playing around getting a baseline on the fans, when the speed up, etc, I noticed that the C6100's cooling is REALLY degraded when you run it with one or more of the sleds removed. You have a nice low-resistance pathway through the empty node for air to get out without doing any good. Seems to lower the overall positive pressure in the back half of the chassis and the temps go up - which means the fans speed up, power consumption goes up (those fans use a LOT of power - 200+ watts just for the fans when they reach full speed). You might want to consider storing cold sleds in those empty holes, or possibly mocking up some kind of blank.
Picture #1 of the colo setup. More to follow (and main site post this week.)
And the annotated version so you can navigate what's going on:
