Mix 10G 2.5G, slow speed, high Retr

[dag]

Let us know what you find out, including brand/model, esp. since the AQS107 is next to impossible to find at this point.

Some user on Reddit reported some success with hifiber, but he didn’t perform a full blown iperf test, I’ll try to get one on Amazon and will report (I have a feeling I’m going to receive yet another 88x3310-based module though, crossing fingers).

 

[spektykles]

Let us know what you find out, including brand/model, esp. since the AQS107 is next to impossible to find at this point.

Some user on Reddit reported some success with hifiber, but he didn’t perform a full blown iperf test, I’ll try to get one on Amazon and will report (I have a feeling I’m going to receive yet another 88x3310-based module though, crossing fingers).

I found the one using AQR113C. Direct from OEM. The rest are just rebrands from this. Unfortunately I dont have enough equipment on hand (no 10G NIC) to test the whole iperf range. It only available in China, no one outside selling it. Well cant compete with your customers, i guess...

 

[spektykles]

Also should I avoid switches that using those faulty Marvell chip? A lot of them using it, very rare used the Aquantia chip.

 

[dag]

Well, I'll be damned. I just got the HiFiber one, tested it, and lo and behold, here is what I get:

*** 2.5G both ways *simultaneously*
* RJ45-> Switch
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-60.00 sec 16.3 GBytes 2.34 Gbits/sec sender
[ 4] 0.00-60.00 sec 16.3 GBytes 2.34 Gbits/sec receiver
* Switch -> RJ45
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-60.00 sec 16.4 GBytes 2.35 Gbits/sec sender
[ 4] 0.00-60.00 sec 16.4 GBytes 2.35 Gbits/sec receiver

*** 5G both ways *simultaneously*
* RJ45-> Switch
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-60.00 sec 32.8 GBytes 4.70 Gbits/sec sender
[ 4] 0.00-60.00 sec 32.8 GBytes 4.70 Gbits/sec receiver
* Switch -> RJ45
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-60.00 sec 32.8 GBytes 4.70 Gbits/sec sender
[ 4] 0.00-60.00 sec 32.8 GBytes 4.70 Gbits/sec receiver

1G and 10G also work as expected, but that's the boring part. Interestingly, my Aruba switch took it without protesting, while the AQS107 simply never worked on it.

It's clearly not a 88x3310, the results are just way too good. And if it's the AQC113C, it does indeed support pause frames:

	Interface	Received Packets w/o Error	Received Packets with Error	Broadcast Received Packets	Transmitted Packets	Collisions	Transmitted Pause Frames	Received Pause Frames
	1/15	66750808	21	38	66096619	0	0	118399011

	Interface	Received Packets w/o Error	Received Packets with Error	Broadcast Received Packets	Transmitted Packets	Collisions	Transmitted Pause Frames	Received Pause Frames
	1/1	66082426	0	30	66667801	0	13024495	0

Looks like we have a winner here.

 

[spektykles]

Well, I'll be damned. I just got the HiFiber one, tested it, and lo and behold, here is what I get:

*** 2.5G both ways *simultaneously*
* RJ45-> Switch
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-60.00 sec 16.3 GBytes 2.34 Gbits/sec sender
[ 4] 0.00-60.00 sec 16.3 GBytes 2.34 Gbits/sec receiver
* Switch -> RJ45
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-60.00 sec 16.4 GBytes 2.35 Gbits/sec sender
[ 4] 0.00-60.00 sec 16.4 GBytes 2.35 Gbits/sec receiver

*** 5G both ways *simultaneously*
* RJ45-> Switch
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-60.00 sec 32.8 GBytes 4.70 Gbits/sec sender
[ 4] 0.00-60.00 sec 32.8 GBytes 4.70 Gbits/sec receiver
* Switch -> RJ45
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-60.00 sec 32.8 GBytes 4.70 Gbits/sec sender
[ 4] 0.00-60.00 sec 32.8 GBytes 4.70 Gbits/sec receiver

1G and 10G also work as expected, but that's the boring part. Interestingly, my Aruba switch took it without protesting, while the AQS107 simply never worked on it.

It's clearly not a 88x3310, the results are just way too good. And if it's the AQC113C, it does indeed support pause frames:

	Interface	Received Packets w/o Error	Received Packets with Error	Broadcast Received Packets	Transmitted Packets	Collisions	Transmitted Pause Frames	Received Pause Frames
	1/15	66750808	21	38	66096619	0	0	118399011

	Interface	Received Packets w/o Error	Received Packets with Error	Broadcast Received Packets	Transmitted Packets	Collisions	Transmitted Pause Frames	Received Pause Frames
	1/1	66082426	0	30	66667801	0	13024495	0

Looks like we have a winner here.

A teardown is expected from you

 

[dag]

You know me too well. Confirmed.

 

[Stephan]

If I hover over "www.hfiber.com" in the PDF https://m.media-amazon.com/images/I/91Yg9RX-HEL.pdf linked from Amazon product page for the ASF-10G-T (Marvell), it wants me to go to www.10gtek.com. So hifiber is 10gtek in USA? ASF-10G2-T not found in EU yet. 5 left on Aliexpress... ^^

 

[dag]

Yeah I suspect they're the same company, and spektykles is correct, they all seem to be made by the same OEM anyway. You may or may not get an ASF-10G2-T when you buy the ASF-10G-T, seems to be hit or miss, you don't really have an option as far as I can tell. I don't think they really understand what they're selling to be honest, all they do is slap a sticker on something built by someone else.

 

[Stephan]

Here is a supplier out of China called "HiLink", not "H!Fiber": Rj45 10gbase Transceiver 10g Copper Sfp+ Aqr113c - Buy Rj45 10g Copper Sfp+ Aqr113c,10g Copper Sfp+ Aqr113c,10g Copper Sfp+ Aqr113c Product on Alibaba.com Has AQR113C in the product title so this might be the real deal. Both out of Shenzhen. HiFiber is a distributor of 10GTek registered in Hong Kong. Confusing...

 

[mach3.2]

I found the one using AQR113C. Direct from OEM. The rest are just rebrands from this. Unfortunately I dont have enough equipment on hand (no 10G NIC) to test the whole iperf range. It only available in China, no one outside selling it. Well cant compete with your customers, i guess...

Found these on Taobao, they are almost half price as compared to 10GbE SFP+ transceivers on aliexpress/amazon. Claims to have DDM too, which is lacking on most other transceivers.

https://m.tb.cn/h.UISuJSj

 

[dag]

I did a bit more research and found the original datasheet for the AQR113, circa 2018, before Marvell acquired Aquantia. It clearly says the 113 supports the same rate adaptation mechanism as the 107 (via PAUSE frames).

I find it interesting Marvell chose to remove this kind of details in subsequent revisions, but I’m not surprised, Marvell didn’t buy Aquantia because of the tech per se, it was totally overlapping with what Marvell was already doing anyway, they really wanted their patents, and they really, really wanted to kill a threatening, fast growing and innovative competitor in the networking space.

I find it even more interesting Marvell still makes the chip, but these seem to be somewhat old stock, if you look at the pic of the chip I posted it shows “2145”, as in 45th week of 2021. I can’t easily find stock for the chip (not the SFP), even on the gray market, so it might be a short run until they run out.

AQR113 original datasheet

 

[dag]

Found these on Taobao, they are almost half price as compared to 10GbE SFP+ transceivers on aliexpress/amazon. Claims to have DDM too, which is lacking on most other transceivers.

https://m.tb.cn/h.UISuJSj

They seem to be 88x3310-based, stay away

 

[mach3.2]

They seem to be 88x3310-based, stay away

There's an "Aquantia" option which claims to be using AQR113C. The part number (GL-9630TQ PLUS) matches the image I initially quoted too.

 

[dag]

There's an "Aquantia" option which claims to be using AQR113C. The part number (GL-9630TQ PLUS) matches the image I initially quoted too.

Fair enough, I didn’t realize. If you give it a shot, please report back.

 

[spektykles]

I did a bit more research and found the original datasheet for the AQR113, circa 2018, before Marvell acquired Aquantia. It clearly says the 113 supports the same rate adaptation mechanism as the 107 (via PAUSE frames).

I find it interesting Marvell chose to remove this kind of details in subsequent revisions, but I’m not surprised, Marvell didn’t buy Aquantia because of the tech per se, it was totally overlapping with what Marvell was already doing anyway, they really wanted their patents, and they really, really wanted to kill a threatening, fast growing and innovative competitor in the networking space.

I find it even more interesting Marvell still makes the chip, but these seem to be somewhat old stock, if you look at the pic of the chip I posted it shows “2145”, as in 45th week of 2021. I can’t easily find stock for the chip (not the SFP), even on the gray market, so it might be a short run until they run out.

AQR113 original datasheet

Thanks for the datasheet you dugged out somewhere. Very useful
Your module is using an older revision of the chip (B0). There is B1 now, I dont know what changed.
Marvell still making these right now, no worry about running out. Shame that Aquantia didnt last longer, we could really have something awesome by now.

 

[spektykles]

The fact that Aquantia so ahead of the game they using 14nm for the chip, while 88X3310 and Broadcom still stuck at ancient 28nm

 

[spektykles]

Also should I avoid switches that using those faulty Marvell chip? A lot of them using it, very rare used the Aquantia chip.

Also this question too

 

[spektykles]

Also should I avoid switches that using those faulty Marvell chip? A lot of them using it, very rare used the Aquantia chip.

Answered my own question. Aquantia chip in switches are available but very rare (pic are AQR813 from Zyxel switch). Same specs as AQR113, Aquantia designed the same "core" then just multiplied it based on how many ports needed.

 

[mach3.2]

Fair enough, I didn’t realize. If you give it a shot, please report back.

Just put some dough down for it, will report back once I'm able to test it.

 

[gabberpocky]

A question - are you guys able to reproduce this issue by overloading the affected port with traffic from multiple gigabit hosts, rather than a single 10G host?

I only have a single 10G transceiver at this point, which appears to be working exactly as expected when connected to a 2.5G i225. My switch (Engenius) doesn't support 2.5G so views it as a 10G link, while the transceiver negotiates 2.5G with the NIC. I'm trying to make a decision between adding another one or adding a 2.5G switch, and I don't want to add more transceivers if I'm going to start experiencing issues once both ends of the link are above gigabit.

I've tried to reproduce the issue described in this thread by flooding the 2.5G server with simultaneous traffic from 3 to 4 gigabit hosts but I'm not able to. iperf in both directions will max out at around 2.35Gbps when the totals are summed. Plenty of retransmits start occurring once the third and fourth hosts join and the link becomes saturated but I never see the kind of speed losses you guys are posting, and retransmits are normal on any saturated link.

Have I got lucky with my particular combination of switch/SFP+ module or is it just not possible to reproduce the issue without a 10G host in the mix?