Questions About Moving to 10gbe

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Roman2179

Member
Sep 23, 2013
49
14
8
Decisions, decisions. I kind of want a Microtik because it's silent but four 10gbe ports would be ideal. Seang86s, how loud is your D-link switch?

Since the T420-CR is a dual port card, I could probably just connect the 2nd ESXi server directly and connect the other port to the Microtik. To the same point, I could connect both ESXi servers directly and hold out for a switch with four 10gbe ports.
 
Last edited:

Baddreams

New Member
Mar 15, 2011
21
7
3
For 10 GbE - Chelsio cards seem to get the most attention from FreeBSD developers. I've used almost every brand card, and do prefer the Chelsio.

As a cheap plug, I do have a T420 for sale on my other thread.
Thats because the FreeBSD developers are reviewing code submitted by Chelsio, they do the same with Mellanox. Intel, I think, just sets up their own BSD environment, gets everything happy and says "here ya go".
 

neo

Well-Known Member
Mar 18, 2015
672
363
63
Thats because the FreeBSD developers are reviewing code submitted by Chelsio, they do the same with Mellanox. Intel, I think, just sets up their own BSD environment, gets everything happy and says "here ya go".
If you look across the pfSense, FreeNAS, and FreeBSD forums the majority all recommend Chelsio cards.

Furthermore, current version of pfSense and FreeNAS do not support Mellanox out of the box. Which are the top two most popular FreeBSD appliances.

Additionally, Chelsio is a server partner of IX Systems (the creators of FreeNAS).
Server Partners | Chelsio Communications
IX Systems also sells preconfigured systems and integrates Chelsio cards.

When PfSense sells preconfigured systems for 10 GbE NICs they solely offer Chelsio.
C2758 1U Rack Mount Network Firewall Router Appliance
 
Last edited:

Roman2179

Member
Sep 23, 2013
49
14
8
For the sake of doing some research, any one know which SFP+ modules work with the Microtik CRS226? Well, anything other than S+85DLC03D modules.
 

neo

Well-Known Member
Mar 18, 2015
672
363
63
For the sake of doing some research, any one know which SFP+ modules work with the Microtik CRS226? Well, anything other than S+85DLC03D modules.
I can't speak for transceivers, but for my Mikrotik CRS226 I use Dell (Amphenol rebrands) and Cisco DAC cables. I actually have a few extras if you are considering SFP+ DAC cables.

I'm also considering selling my CRS226, for what it's worth.
 

neo

Well-Known Member
Mar 18, 2015
672
363
63
I bought a Dell PowerConnect 7024P (w/ optional 10 GbE uplinks) to replace it. I still use the Mikrotik since it's so simple and efficient as a backup but I wouldn't mind selling it.
 

Entz

Active Member
Apr 25, 2013
269
62
28
Canada Eh?
For the sake of doing some research, any one know which SFP+ modules work with the Microtik CRS226? Well, anything other than S+85DLC03D modules.
I have run both Mikrotik and Brocade compatible (Fiberstore) in mine. Much like the ConnectX-2 cards they don't seem to really care.

I would definitely look into DACs. Can save some money that way, especially if both ends do not care (no vendor lock-in). i.e. Mikrotik S+DA0001 works great at < 30$ shipped.
 

Roman2179

Member
Sep 23, 2013
49
14
8
I have run both Mikrotik and Brocade compatible (Fiberstore) in mine. Much like the ConnectX-2 cards they don't seem to really care.

I would definitely look into DACs. Can save some money that way, especially if both ends do not care (no vendor lock-in). i.e. Mikrotik S+DA0001 works great at < 30$ shipped.

Do you happen to have model numbers or links to the ones you used? DACs may not work for me since I have a combination of Chelsio and Connectx-2 cards and I already have optics for them.
 

Entz

Active Member
Apr 25, 2013
269
62
28
Canada Eh?
Do you happen to have model numbers or links to the ones you used? DACs may not work for me since I have a combination of Chelsio and Connectx-2 cards and I already have optics for them.
I have used
Mikrotik S+85DLC03D (transceiver) and S+DA0001 DAC
Brocade (OEM) 57-0000075-01 (Expensive as hell)
Brocade (Fiberstore) SKU0142A2 Brocade 10G-SFPP-SR 10GBASE-SR SFP+ Module-$18

All my cable is Fiberstore OM3 50/125 LC/LC LC-LC Duplex 10G OM3 50/125 Mulitmode Fiber Patch Cord | Fiberstore
 
  • Like
Reactions: T_Minus

Roman2179

Member
Sep 23, 2013
49
14
8
As a quick update, I did end up getting the MikroTik CRS226. At this point I have a CRS226, Chelsio T420, a couple of Connectx-2 ENs, a bunch of transceivers and some fiber coming. Once all of that arrives I will start moving my network and servers over to the new hardware. I think I am going to attach the FreeNAS server and the first ESXi server to the switch. The second ESXi server will be connected directly to the FreeNAS server after I cut down the Connectx-2 card.

Anything I should be aware as I make the change over to the new hardware? Should I do firmware updates on everything before I start deploying it all?

I should probably make build thread with my change over, huh?

Thanks everyone!
 
  • Like
Reactions: T_Minus

Entz

Active Member
Apr 25, 2013
269
62
28
Canada Eh?
Build threads are always good

I have a very similar setup. CRS226 , File server in the first port, ESXi 1 in the second port. Then I have my second ESXi box connected directly to the fileserver and have the connections bridged. So that way both ESXi servers get 10GBe to the fileserver and extremely good speeds to each other ( >8 gbit) / rest of lan.
 

Roman2179

Member
Sep 23, 2013
49
14
8
What are you using as your file server? I am guessing with some messing around I can bridge the two 10gbe connections in FreeNAS.
 

Entz

Active Member
Apr 25, 2013
269
62
28
Canada Eh?
Mine is running Ubuntu 14.04 LTS (using bridge-utils). Server is a E3-1230v1 with a Connect2-EN card in it.
 

Roman2179

Member
Sep 23, 2013
49
14
8
Everyone will get a good show out of me stumbling with my changeover since my network experience is almost not existent. I am moving away from pfsense and going to an Ubnt EdgeRouter Lite. I like pfsense but right now I have it virtualized on the second ESXi box and at this point I don't need all of the functionality that pfsense has to offer. Plus I am almost always on the road, so if anything happens, it's much easier to have someone reset the router than trying to explain how to restart the VM.

The idea is to create a few VLANs and just have a single switch for everything as opposed to the couple of switches I am using right now. I want to get rid of as much cabling as possible. I have tried to set up VLANs before on my zyxel switch previously and failed miserably. This should be an interesting undertaking and I fully plan on reseting configs on more than one occasion. How hard is to configure VLANs on the MikroTik switch? Right now I have a total of three subnets: 1) LAN 2) Management/IPMI and 3) Domain Controller. I think I will keep it the same in that sense but I finally have a chances to reorganize my IP addresses, which I have been putting off for a while.