Expanding My Network Suggestions

[Tubz]

Time to expand and splash a "little cash", so here is the plan, anything in red text is in place.



So the plan is ..

Proxmox box to talk to the servers on the normal 192.x.x.x network
Servers to talk to each other on a 10.x.x.x network.
Servers also talk to AP on 192.168.x.x. network, as 1G is enough to stream media to the house.

Am I making life hard for myself, and should just get from ebay and if so what do you recommend, servers only have PCIe v3 8x slots, Prozmox has mixture of PCIe v4 8x/16x.

SFP+ 8+ port unmanaged switch
4x2 port Mellanox cards, 2 for Proxmox and one each for server
6 cables to team to switch between Proxmox and servers and two cables with an RJ45s to team back to unmanaged switch
Even a cheap managed switch to replace everything, supporting 2.5/10G RJ45 and SFP+?
Then drop the 1G links to AP

Hope I haven't confused you.

 

[bugacha]

whats your question?

 

[Tubz]

Should I just get a switch off Ebay that can handle 2.5G/10G RJ45 and SFP, or get a seperate SFP+ switch and recommend the SFP+ cards and cables, and the cable to link SFP+ to RJ45, furthest distance is an arms length from server to existing switch ?

 

[louie1961]

Your network diagram seems confusing to me. Not to say its somehow wrong, I just don't understand what you are trying to accomplish. My network is a lot "flatter". Rather than separating out network segments with physical hardware, I have 6 VLANs defined in pfSense. My switch is a managed switch, so I can configure any port for any one VLAN or in the case of the WAP and Proxmox nodes, I configure the ports as trunked ports to pass all VLANs. I keep all of my home lab stuff on basically one VLAN, so there is no inter-VLAN routing to slow me down.

 

[louie1961]

Should I just get a switch off Ebay that can handle 2.5G/10G RJ45 and SFP, or get a seperate SFP+ switch and recommend the SFP+ cards and cables, and the cable to link SFP+ to RJ45, furthest distance is an arms length from server to existing switch ?

My switch is this one. Works great for me. Twenty four 2.5GB ports and six SFP+ ports

https://www.amazon.com/dp/B0CCDD4CLC?th=1

 

[anakronox]

I agree with louie1961. Seriously, save yourself the headaches and go for a managed switch that can handle VLANs. Since you seem to have an OPNsense router, it'll support trunking and inter-VLAN routing if you set it up properly. It's best not to rely on intermediate devices like the servers and AP to handle that - let your router actually do it's job and route!