12 sfp+ + 4 10gbe rj45 - Ubiquiti Edgeswitch 16xg

Terry Kennedy

Well-Known Member
Jun 25, 2015
1,068
508
113
New York City
www.glaver.org
The ES-16-XG does not recognize my Nebula DAC cables but does recognize my Cisco DAC cables. This is a real bummer because I have 8 of the Nubula cables for all my servers and only 2 of the Cisco cables to "stack" this switch to the ES-48-LITE (once I get that setup). If I have to get all new cables I may just go with fiber transceivers/patch cables from FS.com this time around. Can anyone confirm which specific transceivers that work with EdgeSwitches?
Generally, the router / switch manufacturers that don't force you to use vendor-locked pluggables say something like "We don't specifically exclude any parts, but we have only tested with the following parts: ..."

That's pretty reasonable, but not much of a condolence when you're trying to get something working. Incompatibilities can arise from:
  • Interactions with vendor proprietary "magic footprints". These are generally computed based on some of the SEEPROM fields and stored in one of the reserved areas, so this should not be a problem if the part accurately describes itself (I've seen all sorts of screwball problems, including DWDM parts that report the wrong channel and so on).
  • Parts that aren't compatible regardless of branding. For example, Dell has a dual-speed 1/10GbE SFP+ part that is used in one product line. It isn't recognized anywhere else, even by other Dell products. This part requires some hardware support that isn't present in most routers / switches. That's also one of the issues with copper SFPs - some of them do speed conversion internally and always talk to the router / switch innards at GigE speed, while others expect the router / switch to handle 10/100/1000 speeds itself.
  • Parts that aren't one of the types the hardware / firmware knows about. For example, a lot of the lower-end products only know about short or long range optics or passive direct attach cables. If you plug in something like a BiDi part (bidirectional on a single fiber), the device will probably go "I don't know what this is, so I can't use it". Likewise if you plug a SONET part into a switch that is only expecting Ethernet.
  • Parts that are not listed in any relevant standard. It is possible for a switch / router manufacturer to come up with something non-standard that will work in their equipment but nowhere else. I haven't seen much of this, and most of it has been oddball variants of direct attach for things like active / standby failover.
 
  • Like
Reactions: ignx

Terry Kennedy

Well-Known Member
Jun 25, 2015
1,068
508
113
New York City
www.glaver.org
quick test of dell dacs,the ones i got" Dell 10Gb 2M SFP+ Cable 1JTTN "from ebay ,10 gbp per dac. So centos server has copper intel x540, windows intel 520 doubleport with dell dac. centos SSd was 300gb dc s3500 so isnt a speed demon.
Pretty much all 10GbE hardware should be capable of "wire speed", given reasonable hardware with a decent networking stack. This is /dev/zero to /dev/null between a pair of Xeon E5620 boxes, using X540-T1 cards and a Dell 8024 switch:
Code:
(0:1) srchost:~terry# bbcp -P 2 -s 8 /dev/zero desthost:/dev/null
bbcp: Creating /dev/null/zero
bbcp: 160620 06:06:45  0% done; 1.2 GB/s
bbcp: 160620 06:06:47  0% done; 1.2 GB/s
bbcp: 160620 06:06:49  0% done; 1.2 GB/s
bbcp: 160620 06:06:51  0% done; 1.2 GB/s
bbcp: 160620 06:06:53  0% done; 1.2 GB/s
^C
Sending real data is also pretty fast - this is a ZFS send / receive in a zrep wrapper with bbcp as the transport (same hardware as above). Spinning rust (no SSDs) on either end:
Code:
(0:2) srchost:/sysprog/terry# zrep init storage/data desthost storage/data
Setting properties on storage/data
Creating snapshot storage/data@zrep_000000
Sending initial replication stream to desthost:storage/data
bbcp: Creating zfs
bbcp: 160619 22:26:28  not done; 748.5 MB/s
bbcp: 160619 22:26:58  not done; 727.1 MB/s
bbcp: 160619 22:27:28  not done; 738.4 MB/s
bbcp: 160619 22:27:58  not done; 739.2 MB/s
bbcp: 160619 22:28:28  not done; 738.8 MB/s
bbcp: 160619 22:28:58  not done; 739.7 MB/s
bbcp: 160619 22:29:28  not done; 739.6 MB/s
bbcp: 160619 22:29:58  not done; 739.3 MB/s
bbcp: 160619 22:30:28  not done; 736.9 MB/s
...
 

ignx

New Member
Aug 10, 2016
9
13
3
37
I just put together a pc from spare parts on a testbench ( msi x99 gaming motherboard with a xeon) so I could with flash firmware to all nics with ease before starting anything else . I realise that dragging large files between windows isn`t a good way to demonstrate anything beside the fact that both ends can communicate with speed higher than 1G and that was the purpose of my post. I should have replied directly to IamSpartacus .With some nice deals around , I mean vendor branded 10G nics there can be some problems but there is always that urge to get some cheap while the stock lasts.I found myself in that situation (I guess I am not the only one) when I bought few nics with dacs and transceivers bundled together so plugging a transceiver that I got with a nic costed me 10s ,less than hours of research on a product than doesnt have many reviews yet (if any) and will have few firmware updates. But that`s what you get when you buy first batch of hardware with stock firmware.
 
  • Like
Reactions: Patrick

Terry Kennedy

Well-Known Member
Jun 25, 2015
1,068
508
113
New York City
www.glaver.org
Never heard about bbcp before, works great on Linux, not so great on FreeBSD
It works fine on FreeBSD as long as you use the 20120520 version. Newer versions have all sorts of bizarre problems which seem to boil down to there not being an assigned maintainer who also uses the port. Thus, some patches got in which (at a minimum) do all sorts of strange things with the IPv6 stack.

I go into this in some more detail in my RAIDzilla II upgrade article.

The sample outputs from my earlier post are from FreeBSD, by the way.
 
  • Like
Reactions: BackupProphet
Aug 17, 2016
32
1
8
31
very interested in the fan model numbers and specs. also power consumption running at all SFP+ interfaces at max loads\speeds.
but that initial reading for idle power consumption is very good. there does not seem to be much love with Finisar transceivers from what i have gathered.

is this something that could be corrected with future firmware updates from Ubiquiti?
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,539
979
113
artofserver.com
very interested in the fan model numbers and specs. also power consumption running at all SFP+ interfaces at max loads\speeds.
but that initial reading for idle power consumption is very good. there does not seem to be much love with Finisar transceivers from what i have gathered.

is this something that could be corrected with future firmware updates from Ubiquiti?
I'm very interested in the answers to the above too....
 

GCM

Active Member
Aug 24, 2015
137
43
28
I doubt they'll adopt them, finisar transceivers have been historically bad with Ubiquiti in terms of support.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,313
559
113
Just found these being sold on eBay for anyone interested since it seems the compatible DACs are quite limited right now and these are confirmed working. Buyer accepted BO of $25 per when I bought 8. Listing only has 7 left but the description says quantity of 100 available and it's a new listing so might have more not being listed.
 
  • Like
Reactions: GCM and Patrick
Aug 17, 2016
32
1
8
31
Just got mine today.
No issues with build quality. even got the rubber boots for unused SFP+ ports
Quality Check date of 06/17/16
Fans are quiet but still emit a muffled whine . nothing that closing a door does not fix for near total silence. (x1 1G RJ45 port active only so far)
UPS says that device is only drawing <10 watts currently. device seems very power optimized. which is very good to see.
 

Jeff Robertson

Active Member
Oct 18, 2016
421
113
43
Chico, CA
Has anyone tried replacing the internal fans with the 40mm noctuas? I'm hoping to get this switch but I'm going to be sitting about 8' from it so silence is a must.
 
Aug 17, 2016
32
1
8
31
they look like standard 3 pin fans. you should have no issues replacing them. mine are acceptable noise as they are locked in another room.
just remember the 10G base T ports will suck a lot of power per port and will create more heat vs SFP+.
 

Netwerkz101

Active Member
Dec 27, 2015
298
75
28

Jeff Robertson

Active Member
Oct 18, 2016
421
113
43
Chico, CA
Just verified it's still an active deal. You have to have an account then sign up for their beta account (free, takes about two seconds). This is what I see when I log in:

Capture.JPG

I ordered one last night just in case they took it down quickly. For that price it's worth it to have some cable headaches.