Can't get 10G speeds on main subnet

[arcadeperfect]

Hi

I have a nas with 10GbaseT and a PC with a Mellanox SFP+ as well as on board 2.5G. Connected via a CRS305-1G-4S+IN switch. Using a consumer TP-Link router (although I have a box to set up PFSense or OPNSense on).

If I have the 10G nics on the same subnet as the rest of the home network, I can't get more than 6.5GB/s. If I give them their own subnet, they push 9GB/s or so.

Is that normal / a limitation of the switch? Since both machines have 2 nics it's not a problem to keep it that way.

The problem is I have / will have other devices with just 1 x 10G nic, which need access to all the various crap on the wifi network + internet + fast pipe to NAS.

Anything I can do to speed it up?

Thanks for any advice.

 

[Scott Laird]

How much traffic is the CRS305 trying to send to your tp-link router when everything's on one LAN and you're running tests? Is it more than a couple Mbps?

Try turning off flow control on the switch and see if that helps? Or, if it defaults to off, turn it on? It's probably not the right solution, but it'll give you some information.

CRS305s are *really* short on CPU, and Mikrotik gives you a lot of options that will make the switch fall back to using the CPU instead of the hardware switch chip. Anything that does any routing or filtering is going to cause performance to nosedive, but that shouldn't happen when just acting like a perfectly normal switch. Does Mikrotik's CPU/RAM/etc monitoring show anything?

 

[prdtabim]

Hi

I have a nas with 10GbaseT and a PC with a Mellanox SFP+ as well as on board 2.5G. Connected via a CRS305-1G-4S+IN switch. Using a consumer TP-Link router (although I have a box to set up PFSense or OPNSense on).

If I have the 10G nics on the same subnet as the rest of the home network, I can't get more than 6.5GB/s. If I give them their own subnet, they push 9GB/s or so.

Is that normal / a limitation of the switch? Since both machines have 2 nics it's not a problem to keep it that way.

The problem is I have / will have other devices with just 1 x 10G nic, which need access to all the various crap on the wifi network + internet + fast pipe to NAS.

Anything I can do to speed it up?

Thanks for any advice.

The Mikrotik switches uses HW offloading to avoid the use of the SoC. Only the 1st bridge created is HW offloaded.
See port isolation item here Manual:Switch Chip Features - MikroTik Wiki
This will allow to isolate ports in the same bridge and use the HW offload.