Another Cheap 10Gb Switch Option (Netgear 4 x SFP+ and more)!

Discussion in 'Networking' started by Emulsifide, Jan 17, 2015.

  1. Emulsifide

    Emulsifide Active Member

    Joined:
    Dec 1, 2014
    Messages:
    201
    Likes Received:
    82
    After being completely disappointed with the Quanta LB4G switch that @h0tw1r3 and I purchased from the same eBay seller, I scoured eBay for other options and came across the Netgear GSM7300s series of switches. The series has 2 x 10Gb SFP+ connections on the front of the switch with 2 module bays on the back that can support the following options:
    • AX742 (24 Gigabit Stacking Kit, comes with 2 modules and one stacking cable)
    • AX741 (10 Gigabit XFP adapter module)
    • AX743 (10 Gigabit SFP+ adapter module)
    Depending on your needs, using a single switch with 2 AX743 modules nets you 4 x SFP+ ports or by stacking the switches, you get a 24Gb backplane limit (per stacking kit for a total of 48Gb backplane fabric if you use both module ports for stacking only) to tie together up to 8 switches. This will net you a max total of 16 SFP+ ports OR 24 SFP+ ports if you drop the backplane fabric to 24Gb. Netgear also claims a total of 144Gb of fabric for a single switch (GSM7328Sv2), which is perfect if you only need 4 x SFP+ ports. Regardless of how you set it up, there's plenty of different options.

    For my setup, I'm trying to interconnect 3 x ESXi nodes (for vSAN), an OmniOS ZFS file server, and my main desktop rig. Given I only care about having the three ESXi nodes talk to one another for I/O purposes, the stacking traffic limit of 24Gb doesn't really hinder what I'm doing since the file server and the desktop rig can be on the same switch. I bought the following switching equipment:
    • 2 x Netgear ProSafe GSM7328Sv2 switches: $239.99 each from eBay
    • 1 x AX742 kit: $125.00 from eBay
    • 2 x AX743 SFP+ modules with included genuine Netgear AXM761 SFP+ transceivers: $75.00 each from eBay
    • 4 x Fiberstore.com 10GBASE-SR SFP+ transceivers: $18.00 each
    • 6 x Monoprice 10Gb Fiber Optic Cable, LC/LC, Multi Mode, Duplex - 2 Meter (50/125 Type) - Aqua cables: $10.76 each
    The grand total comes to $891.54 for 48 gigabit ports and 6 10Gb ports. Setting aside the gigabit ports, that comes to $148.59 per port that's ready to interface with another LC connection equipped NIC.

    Now for the actual switches:

    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]

    As you can see, the switches aren't terribly loud (the reading is with a single switch on). Mr. Kill-O-Watt has an even better story to share by stating that each individual switch is only sipping a blistering 39 watts of energy with a single SFP+ transceiver active in one of the front slots and a stacking module installed. Not bad!

    As it stands, I'm still waiting on the AX743 modules and fiberstore transceivers to show up before I can go ahead and integrate this solution. I did manage to find out that the switches are compatible with Brocade 57-0000075-01 transceivers installed in the front SFP+ slots. Using Brocade CNA 1020 interfaces connected to two of my ESXi nodes with Brocade 57-0000075-01 on all four interconnects (CNA and switch slots), the traffic passing between the two switches (so one CNA is connected to one switch), I'm getting roughly 6.5Gb/sec using iperf in Ubuntu 14.04.1 LTS with virtual connections to flat vSwitches and MTU 1500 across the board on everything. Not terrible considering I haven't tuned anything and jumbo frames are not involved.

    To conclude (for now), if you all you need is less than 4 SFP+ ports, this can be an ideal solution for you if you find the right price on the parts you need. Using my pricing example, a single switch 4 x SFP+ setup would have cost $469.03. I haven't verified if my switches are eligible or not (given I'm not the original owner), but Netgear does advertise a limited lifetime warranty with this model as well. Regardless, the price is right and the power level is nice and low. Obtaining the parts from eBay seems to be somewhat of a waiting game at the moment, but I passed up nabbing another switch that went for $56.24 (before an obscene $67.45 shipping charge was tacked onto it by the seller):

    Netgear ProSafe GSM7328S V2 24 Ports External Switch Managed Stackable | eBay

    I'll update more when I get the rest of my parts in.
     
    #1
    ELit3, Chuckleb, Jeggs101 and 4 others like this.
  2. Marsh

    Marsh Moderator

    Joined:
    May 12, 2013
    Messages:
    1,880
    Likes Received:
    851
    Thanks for the write up, I am sure that you are saving me lots of money and headache.
     
    #2
  3. Atomicslave

    Atomicslave Member

    Joined:
    Dec 3, 2014
    Messages:
    41
    Likes Received:
    11
    Yea this sounds like a good fit for my expanding my setup.
     
    #3
  4. coolrunnings82

    coolrunnings82 Active Member

    Joined:
    Mar 26, 2012
    Messages:
    389
    Likes Received:
    84
    Very interested in this. Thanks for the write-up. I eagerly await your results. Also interested to see if there's a 48-port version that's affordable...
     
    #4
  5. Emulsifide

    Emulsifide Active Member

    Joined:
    Dec 1, 2014
    Messages:
    201
    Likes Received:
    82
    There is (the GSM7352Sv2), but it's a little harder to find. I should also mention, version 1 of the hardware would work as well, but it doesn't come with any built-in 10Gb ports. Instead, they come with 4 x module ports that you can populate however you'd like.

    Work was closed today due to MLK day. I should have my fiberstore transceivers tomorrow. The SFP+ modules are taking forever for the seller to ship.
     
    #5
  6. Hank C

    Hank C Active Member

    Joined:
    Jun 16, 2014
    Messages:
    642
    Likes Received:
    66
    can't find this switch with similar price on ebay...
     
    #6
  7. Atomicslave

    Atomicslave Member

    Joined:
    Dec 3, 2014
    Messages:
    41
    Likes Received:
    11
    Yea they all seem way more expensive
     
    #7
  8. Emulsifide

    Emulsifide Active Member

    Joined:
    Dec 1, 2014
    Messages:
    201
    Likes Received:
    82
    Search with a more generic search like "netgear gsm". A lot of people are only putting a partial model number into the auction title.

    Here's one at $75 that's due up in 33 minutes:

    Netgear ProSafe GSM7328S 24 Port Gigabit Ethernet Network Switch | eBay

    I was going to grab it, but I really don't need more at the moment. Somebody get it!!
     
    #8
  9. Hank C

    Hank C Active Member

    Joined:
    Jun 16, 2014
    Messages:
    642
    Likes Received:
    66
    thx!
     
    #9
  10. Hank C

    Hank C Active Member

    Joined:
    Jun 16, 2014
    Messages:
    642
    Likes Received:
    66
  11. Emulsifide

    Emulsifide Active Member

    Joined:
    Dec 1, 2014
    Messages:
    201
    Likes Received:
    82
    Depends. That one comes with the stacking modules, but does NOT have 10Gb as built-in SFP+ ports. Like I said above, V1 hardware doesn't come with them (only V2 like the 24 port switch I just posted up).
     
    #11
  12. Emulsifide

    Emulsifide Active Member

    Joined:
    Dec 1, 2014
    Messages:
    201
    Likes Received:
    82
    Did you get it? It shot up to about what I paid for mine.
     
    #12
  13. mrkrad

    mrkrad Well-Known Member

    Joined:
    Oct 13, 2012
    Messages:
    1,226
    Likes Received:
    48
    #13
  14. Hank C

    Hank C Active Member

    Joined:
    Jun 16, 2014
    Messages:
    642
    Likes Received:
    66
    i did not get it. someone took it on the last 2 second
     
    #14
  15. Emulsifide

    Emulsifide Active Member

    Joined:
    Dec 1, 2014
    Messages:
    201
    Likes Received:
    82

    Good switch, but there's no way it's going to stay at that price. The 2910al will sell around $300-$500 on its own. That SFP+ module sells for a solid $400 on eBay as well. I would expect this switch to go for $800-900 when it's all over and in that configuration you 2 10Gb capable ports. The switch also pulls significantly more power (idle of 64W and max of 105W) when the Netgear I found maxes at 66W.

    Regardless, I'd appreciate it if we kept the discussion in this thread to be Netgear GSM73xx only. I'm trying to compile as much as I can about this particular switch series into this thread as an easy reference. I think it would be AWESOME if everyone who went down a different path that what's already been written about created their own thread with similar information on different switches. We could then create a sticky at the top of this forum that has links in it (along with pricing information) to convey the different options to people.

    I got my fiberstore.com transceivers today, so I'll be migrating my vSAN over to it tonight after testing all the links. I'll post up iperf info as I go.

    [​IMG]
     
    #15
    Last edited: Jan 20, 2015
    Patrick and F1ydave like this.
  16. Emulsifide

    Emulsifide Active Member

    Joined:
    Dec 1, 2014
    Messages:
    201
    Likes Received:
    82
    Alrighty. I installed the transceivers as soon as I walked in the door. Here's how they show up on the switch:

    Code:
    (GSM7328Sv2) #show fiber-ports optical-transceiver-info all
    
                             Link Link                                 Nominal
                           Length Length                                   Bit
                             50um 62.5um                                  Rate
    Port     Vendor Name      [m] [m]  Serial Number    Part Number     [Mbps] Rev  Compliance
    -------- ---------------- --- ---- ---------------- ---------------- ----- ---- ----------------
    1/0/25   FiberStore        30   30 FS50113E0064     SFP-10G85-3M-NE  10300 1.0        10GBase-SR
    1/0/26   FiberStore        30   30 FS50113E0062     SFP-10G85-3M-NE  10300 1.0        10GBase-SR
    2/0/25   FiberStore        30   30 FS50113E0063     SFP-10G85-3M-NE  10300 1.0        10GBase-SR
    
    (GSM7328Sv2) #show fiber-ports optical-transceiver all
    
                                        Output    Input
    Port      Temp  Voltage  Current     Power    Power   TX     LOS
               [C]   [Volt]     [mA]     [dBm]    [dBm]   Fault
    --------  ----  -------  -------   -------  -------   -----  ---
    1/0/25    16.3    3.200      5.7    -2.886   -2.132   No     No
    1/0/26    18.3    3.240      5.7    -2.607   -2.273   No     No
    2/0/25    22.3    3.222      5.1    -3.139   -2.227   No     No
    
     Temp - Internally measured transceiver temperatures.
     Voltage - Internally measured supply voltage.
     Current - Measured TX bias current.
     Output Power - Measured optical output power relative to 1mW.
     Input Power - Measured optical power received relative to 1mW.
     TX Fault - Transmitter fault.
     LOS - Loss of signal.
    
    
    What I find interesting about this is that I thought fiberstore.com was supposed to program the transceivers to mimic the OEM Netgear transceiver part number that I gave them. Whatever. Either it's doing it, or these switches don't care what brand transceiver is connected to them (since I also tried the Brocade 57-0000075-01 transceivers I have laying around).

    The transition to doing vSAN over the links is going to have to wait. I tried adding them to my vSAN distributed switch and I was getting dropped datastore errors when trying some migrations. I think it mostly had to do with the fact that the two sets of NICS (my gigabit and 10 gigabit) are currently interconnected to two separate physical switches that have no connection to one another. Once I get the switches mounted (which will be after getting the SFP+ modules I still have in the mail), I'll do the transition and do some benchmarking. In the meantime, I'll see if I can pass some other types of traffic across the links just to play around and get some numbers.
     
    #16
  17. Entz

    Entz Active Member

    Joined:
    Apr 25, 2013
    Messages:
    269
    Likes Received:
    62
    My brocade ones showed as Fiberstore as well.

    Are you mainly working with the switch via the command line or have you used the web gui at all? I wasn't overly pleased with the one on my older 724 but those are low end by comparison.
     
    #17
  18. Emulsifide

    Emulsifide Active Member

    Joined:
    Dec 1, 2014
    Messages:
    201
    Likes Received:
    82
    I'm using both command line and the GUI. I also own a Netgear GS724T, which has the absolute worst GUI on the planet. The GSM73XXSV2 GUI is MUCH BETTER by comparison. Many more features are available to use, but command line still has the most functionality.
     
    #18
    Entz likes this.
  19. Entz

    Entz Active Member

    Joined:
    Apr 25, 2013
    Messages:
    269
    Likes Received:
    62
    awesome. Thanks!
     
    #19
  20. Emulsifide

    Emulsifide Active Member

    Joined:
    Dec 1, 2014
    Messages:
    201
    Likes Received:
    82
    Huzzah!! The SFP+ modules finally came in today:

    [​IMG]

    Tonight should be fun. I'm hoping to have the switches mounted and the majority of my network moved over. I still need to test the vSAN I/O stuff a little more before I'm confident with the transition. My next test for that is to plug the vSAN switch into the Netgear on an isolated VLAN that includes the 10Gb links. After everything is up and communicating on the same subnet, some vMotion tests should confirm that everything is working properly. I'll then deactivate the 1Gb links and do the same testing so that I can confirm 100% that the 10Gb links are solid. After that, the 1Gb links will be put back into action as a HA failover. If I have enough time, the file server will get a Brocade CNA 1020 makeover and be added into the mix as well.
     
    #20
    Patrick likes this.
Similar Threads: Another Cheap
Forum Title Date
Networking Yet another PfSense Question - Building a box ( quiet/1u/passive'ish ) Jul 13, 2018
Networking Infiniband PCIe card preventing boot in one server but not another May 12, 2018
Networking Another Intel Fake? Jul 21, 2017
Networking HP 1810G-24. Questions about adding another switch Mar 5, 2017
Networking Sorry another 10gb setup question Oct 26, 2016

Share This Page