Movin' on up...to 40GbE

NSKA

Active Member
Nov 17, 2015
213
26
28
26
I never made an FS thread since the high-end network doesn't seem to sell well here.

I was matching eBay prices (most of which didn't include that 10G card) at 1200.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,258
1,710
113
CA
With the price of ConnectX3 cards capable of 40Gbe and switches hitting $1500 or less for good # of ports of 40Gbe (some all)...

Is there any reason to even consider buying more 10Gbe cards? I mean they're $60 for single port now, no more $25-30$, and dual port that were $60-80 are now $130+ again...

Why not just go 40Gbe all over for a home lab, small business, etc... it seems as long as you're not going past 12-16 ports (cheaper switches) you could go ALL 40Gbe?? Heck, a 40Gbe only switch + LB6M would cover a lot of ground, and be cheap cheap. Those of us who got the DACs for 40Gb and 56Gb should be sitting even more affordable!!

Just thinking out loud, looking for error in my thinking... ?
 

i386

Well-Known Member
Mar 18, 2016
2,673
789
113
32
Germany
@T_Minus which cards do you mean? Intel or Chelsio?
Because I see still a bunch of 10gbe mellanox cards for <$20 each.

40gbe is great but the switches are all designed for datacenters where noise is not a concern. My family would kill me if had such a screaming monster in the house.
 

pyro_

Active Member
Oct 4, 2013
747
165
43
The other big downside for most 40gbe switches is the power draw. You can get 10gbe switches that idle around 15w or so. Plan on closer to 100+w for most 40gbe switches at this point. Even with these junipers I am seeing 70-80w power draw at idle. Luckily for me it will replace two of my switches in use so that makes the power draw a bit more acceptable
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,258
1,710
113
CA
More power is inevitable at this stage technologically for higher throughput so that's to be expected... like FusionIO, NVME, etc... faster comes at some cost :) somewhere.

Good info on power draw, and sound, and cost, etc... still doable :) imho.

@i386 in regards to the Intel NICs
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,205
5,291
113
For most of the 48x 10GbE + 4x or 6x 40Gbe switches power is comparable to the 32 port 40GbE versions.

For lower port counts, idle power is much better, of course.
 
  • Like
Reactions: T_Minus

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,258
1,710
113
CA
@Patrick exactly... so why not go 32 port 40Gbe? I know that's what I'm trying to find a deal on :) something in the 24-32 port range.

$25-30 is what I paid for single port Intel 10Gig SFP+ based NICs
$30-50 or so for Mellanox connectx2 10G
Now the Intel are $50+ 2 port $100+

Or $60-70 for 2port 40G -- seems like a no thought , that's the way to go... trying to see if I'm missing something.


It's not like 24x 10G ports are a silent switch ie: LB6M

Now what would be badass would be 30x 40GBE and 2-8x 10G so you could connect some LB6M/s.
Or, I guess just find another 10G switch that has 40Gbe and stack with 1 or possibly 2 ports... still would ROCK!!!!


I wish those ConnectX5 100Gbe were not $1k+ ha ha!!

Less of a concern for me, personally, on the capacity/size I'm working with... but ya know, why not? HAHA GOALS!!
 

pyro_

Active Member
Oct 4, 2013
747
165
43
There is both cli and web GUI though depending on what j-web GUI you get the web GUI could be limited. Unfortunately the limited version is what I have on mine from what I understand
 

CookiesLikeWhoa

Active Member
Sep 7, 2016
112
26
28
32
Thanks for the information! I'm a little hesitant to jump in on the CLI interface for switches. Been using Netgear switches with their WEB GUI which has made things very simple.
 

whitey

Moderator
Jun 30, 2014
2,774
869
113
39
My EX4300 just arrived from UPS. Looks good and the seller did a fantastic packing job (gotta luv overzealous packing jobs!!!)

Awaiting my JPSU-350-AC-AFO PSU from China now, 4-6 day DHL ship times they say. We shall see. Wish those 4 port 10G EX-UM-4X4SFP modules weren't OUTRAGEOUS lol. Gonna have to wait those out just like I did the switch and be patient on hunting a deal!
 

Evan

Well-Known Member
Jan 6, 2016
3,287
567
113
Little off topic, Cisco switch like the 93180 are 48 port 10/25G SFP+, what cards are 25G ??
I see lots of 10/40 but 25 is kind of new right but I don't recall ever seeing a 25G card...
 

Evan

Well-Known Member
Jan 6, 2016
3,287
567
113
Because they are currently shipped to hyperscalers and not to "normal" enterprises.
25 Is the New 10, 50 Is the new 40, 100 Is the New Amazing | Mellanox Technologies Blog
Fair enough I guess, Using 40/100 for uplinks already, just never saw much 25 stuff, I thought it may have been likes the 2.5/5 stuff that never took off despite it being a good idea, dead now as 10 should be standard on anything except it's not and still so many 1G ports but then again the power cost of 10g copper is rather unattractive unless you need the bandwidth.
 

nighthawk43

New Member
Nov 17, 2016
4
0
1
34
How hard are these switches to setup for basic use? I don't need to setup anything fancy like VLANs.
Do they require licensing or subscriptions, or can I just guy one off of eBay and plug it in?

Sorry if these are obvious questions, I'm not much of a networking guy.
 

i386

Well-Known Member
Mar 18, 2016
2,673
789
113
32
Germany
Most of the switches can be used immediately, only the fancier stuff (vlans, dcb,pfc and so on) need additional configurations.
Licenses/supscriptions depend on the switch manufacturer. Juniper switches for example requiere licenses for layer 3+ protocols, other need supscriptions for firmware updates. You should look that kind of information up before buying a switch.