LGA2011 or LGA1155 for 10Gbps Router + VPN + Suricata + IPS + DPI + etc

[eduncan911]

I'm on the fence here and need some advice. I have a collection of LGA2011 v3 and v4 12C and 14C CPUs and DDR4 ECC ram in boxes. Some matched CPUs are even L (low powered) series.

• I want to build a home Linux server, and expand to threat detection, DPI, etc.
• I only have LGA2011 hardware.
• I am looking to buy a cheap 1U LGA1155 Xeon V3 though (and sell my LGA2011 gear).

What I am unsure of is if an LGA1155 quad-core Xeon 1275 V3 will handle such requirements. Or, do I need the heft from some of the LGA2011 gear?

Xen/ESXi host with pass-through of most devices to a domU.

OpenVPN, Suricata and other tools, DPI tools (not sure which), etc.

All logs and metrics will be pushed to a remote server, so nothing stored locally.

Right now, the L3 switch handles all 10 Gbps VLANs ACLs. However, I'd like to start monitoring this for one or two VLANs eventually. Which means, I'd need to route 10 Gbps on this same box (preferred) instead of the switch.

Thoughts?

 

[RTM]

So... 10G routing is not exactly trivial, at the very least you will need to figure out if you want stateful packet inspection firewalling, if not you could use a solution like TNSR to do the packet routing.

The big deal is the IDS/IPS, I doubt you can do that at 10G level with either platforms.

Unless you really want to do IPS, why not consider keeping your L3 switch as a router, install security onion on whatever hardware you have and set it up to listen to some of the traffic in your network.

I assume your internet connection is not 10g, so you could consider deploying IDS around it?

 

[eduncan911]

Thanks for the reply. To outline the topology:

1Gbps up/dn Internet connection
^- Connected to USG Pro 4 router via 2x 1Gbps (1x all VLAN tagging, 1x untagged)
^- Connected to an L3 Switch w/10Gbps SPF+ and 1Gbps RJ45

Currently the switch is handling the 10Gbps InterLAN VLAN routing between subnets w/ACLs (default deny all). Pretty simple/standard stuff, just two servers, desktop, and a laptop. However, I run about 15+ IoT devices on other VLANs, and want to add a bunch more. I want the ability to monitor the IoT VLANs that I do connect to the internet.

I want to replace the USG Pro 4 with a Linux box to do DPI/IPS on the 1Gbps connection, and keep the speed of 1Gbps.

So yeah, I needed to re-word the OP better:

• I would like to do IPS via Suricata and other inspection tooling on a 1Gbps up/dn fiber connection, and block/act on alerts.
• Future proofing: I would like the ability to DPI/IPS across InterLAN via 10Gbps connections in the future.

The first one, Suricata over 1 Gbps can be done with a moderate box. I typically run 4 vCPU machines in AWS for that and get about 900Mbps at work. Though, it's usually only 1-way (inbound).

As far as InterLAN 10Gbps DPI/IPS tooling, that would require removing the InterLAN from the L3 switch and passing it to the router to let it see the traffic. But at 10Gbps, I am not sure what kind of hardware is needed for that, for internal.

---

RE: Security Onion, I haven't heard of this. I popped onto their website and will read more. Can you give me an elevator pitch of what it might do here?

You did give me an idea of traffic cloning. The AVI Networks we've setup at work does Traffic Cloning, where it can clone the decrypted TLS traffic and send it to a 3rd party machine, like an DPI inspector/logger. But, this is an reactive/alerting approach instead of a proactive approach. Also, I am unsure of how to do via the Linux kernel and open source tooling, and the traffic cloning requires a big CPU for a large amount of connections.

 

[kapone]

First things first. Take 2011 vs 1155 out of the equation.

1155 is way more than enough. You mentioned e3-1275 v3, which btw is socket 1150, but that will work as well. Or a 1270 v2. Either way.

Your problem is...interesting.

- I'd keep the L3 routing on the switch.
- If you're not already doing a router-on-a-stick, I'd go that way.
- Assuming you do the above, clone the traffic on the transit port to the router. That way, you can monitor the outgoing traffic without affecting the throughput to the inter webs.
- Then comes the complex part. How you'd integrate any monitoring/alerting logic from the cloned port back to the router to set policies becomes complex. But it's a clean approach.
- Your monitoring server on the cloned port will need to be ...well...not even sure what can handle monitoring/inspection at 10gbps, without spending a good amount of money. That's your call.

 

[MichalPL]

Few years ago we build a "software switch" for 100GbE on 2011 (Xeon 1620 - 4 core HT overclocked to 4.0GHz, ~16 DDR3 ECC 1600MHz) and measured speed was about 7-8Gbytes/s (~75% of the 100GbE).
Cards if I remember correctly: Mellanox Connect-X 4 100GbE Dual (PCIE 3.0 x16), Mellanox Connect-X 3 40GbE Dual (PCIE 3.0 x8), Mellanox Connect-X 3 40GbE Dual(PCIE 3.0 x8), Mellanox Connect-X 3 10GbE Dual(PCIE 2.0 x4).

The big difference between 2011 and 1055 is PCIe lanes (40 vs 16), for 10GbE traffic it can be even chipset, but better to connect directly to CPU, and than 1055 processors have one slot x16 or 2 PCIE slots x8 only, rest is motherboard chipset (I don't remember if 2.0 or 3.0).