First real network setup. Looking for confirmation that I'm doing it correctly.

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Oldhome7

Member
Feb 9, 2020
71
23
8
I'm finally upgrading from where, I assume, most of us started, just the standard store bought routers. I've always ran a half a step farther and used custom firmware, ie. Tomato, DD-WRT, OpenWRT.

I recently purchased an Dell R510 with an X5660 (6C/12T) and 8Gb ram, that after trying to figure out the best routing/firewall platform I settled on pfSense. Now, the main questions/concerns and reason for this thread. I'm planning on taking everything offline to put that box in place, reconfigure the WRT3200ACM as a WAP, relocate a few devices, and change a few ethernet cables around.

The setup as planned would be: Cable modem -> pfSense -> Linksys SE2800 8 port GB dumb switch with main PC and Media server connected -> WRT3200ACM in media room as AP with game consoles and tv connected. Hopefully it's obvious that "->" = hard line. I'm thinking about adding an older Linksys E2500 as another AP, for better coverage on the opposite side of the house from the WRT3200ACM, after the SE2800, since that switch is pretty central in the house.

Does this sound sensible? Any input at all would be greatly appreciated. Thanks.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
A Dell R510 w\x5660 is a lot of power for a router\pfsense, any reason you went for that vs a tiny appliance that's 4"x4"x2" and uses 10x less power?
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
The problem with SFF which are indeed tiny, but pfSense is crap with USB NICs (at least in my experience and I tried like 10 of them)
You could run only on 1 nic with a managed layer2 switch, but it requires a more complex setup.
Intel NICs are solid, but for specific Suricate capture features, a more specializes NIC is suggested.
This desktop PC sold for $150 (not great listing) - has 4gb ram, 1tb HD, and more importantly Core i3-9100 which has similar PassMark performance as X5660 does. Ram is cheap to expand just as to add a 1gig low profile nic card.
 

Oldhome7

Member
Feb 9, 2020
71
23
8
Mostly because I found it locally for $25. It didn't come with any drives, but I had a spare 120g ssd laying around. Honestly, power isn't too much of a concern to me at the moment, this is mostly a learning experience to begin with but, so far I'm clocking that thing at about 80-90w usage. I'd imagine it will go up a little when actually put into service.

My old WRT is generally pegged at 60+% CPU usage. I didn't figure an Atom would be much improvement.

Also, I picked up a 47u hp cabinet that only has about 15u occupied and I have a problem with empty spaces and also buying "computer crap" as the wife says.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
The problem with SFF which are indeed tiny, but pfSense is crap with USB NICs (at least in my experience and I tried like 10 of them)
You could run only on 1 nic with a managed layer2 switch, but it requires a more complex setup.
Intel NICs are solid, but for specific Suricate capture features, a more specializes NIC is suggested.
This desktop PC sold for $150 (not great listing) - has 4gb ram, 1tb HD, and more importantly Core i3-9100 which has similar PassMark performance as X5660 does. Ram is cheap to expand just as to add a 1gig low profile nic card.
I don't do anything special with pfsense but I've never had any issue with my mini setup.

This device has served me well for many years now: Protectli: Trusted Firewall Appliances with Firmware Protection
" 4 Intel® Gigabit Ethernet NIC ports "


Comparing WRT to Atom? Yeah, it wouldn't be 60% that's for sure.
 

Oldhome7

Member
Feb 9, 2020
71
23
8
The problem with SFF which are indeed tiny, but pfSense is crap with USB NICs (at least in my experience and I tried like 10 of them)
You could run only on 1 nic with a managed layer2 switch, but it requires a more complex setup.
Intel NICs are solid, but for specific Suricate capture features, a more specializes NIC is suggested.
This desktop PC sold for $150 (not great listing) - has 4gb ram, 1tb HD, and more importantly Core i3-9100 which has similar PassMark performance as X5660 does. Ram is cheap to expand just as to add a 1gig low profile nic card.
This R510 fortunately has a dual 1g broadcom setup. Plus, the added benefit of a dedicated ipmi port.
 

Oldhome7

Member
Feb 9, 2020
71
23
8
Comparing WRT to Atom? Yeah, it wouldn't be 60% that's for sure.
Again, I have a buying problem. And for what I paid for it, I'm not too worried. I'm pretty sure the same comparison could be made between the Atom and Xeon.
 

RTM

Well-Known Member
Jan 26, 2014
956
359
63
Ignoring the fact that using an old R510 is a little extreme power wise, the planned network idea looks sane.
Perhaps a little basic for my liking, but I suppose that could be a good thing ;)

Could I give you some ideas on how to overengineer it?
Sure... but I am not seeing any requirements listed (what are you looking to achieve with this network?), nor information on the internet connection (bandwidth).

You may want to give us some this information, else I believe there is a significant risk that the discussion will just go of on a tangent.
 

Oldhome7

Member
Feb 9, 2020
71
23
8
There's not much to the requirements as of yet, again this is just a learning experience to get my feet wet on things that aren't just plug and play. I managed to setup Squid, pfBlockerNG, and a few other things after browsing here and a couple other places for guidance.

It's just a basic home network with the usual online gaming, p2p traffic, and streaming. The p2p is what really showed that the Linksys couldn't handle heavy usage. When it starts to get into the higher connection counts and cpu/ram usages, as show in the dd-wrt gui, it randomly starts dropping wireless clients. Lately it's even started to hand out bogus dhcp addresses, which in turn has required me to set a static ip on everything.

It's currently a 150/10 connection but I'm planning on stepping up to the ~300/10 plan unless my current provider gives me a deal on gig.

Just curious, how advanced does it have to be for you to like it?
 

RTM

Well-Known Member
Jan 26, 2014
956
359
63
There's not much to the requirements as of yet, again this is just a learning experience to get my feet wet on things that aren't just plug and play. I managed to setup Squid, pfBlockerNG, and a few other things after browsing here and a couple other places for guidance.

It's just a basic home network with the usual online gaming, p2p traffic, and streaming. The p2p is what really showed that the Linksys couldn't handle heavy usage. When it starts to get into the higher connection counts and cpu/ram usages, as show in the dd-wrt gui, it randomly starts dropping wireless clients. Lately it's even started to hand out bogus dhcp addresses, which in turn has required me to set a static ip on everything.

It's currently a 150/10 connection but I'm planning on stepping up to the ~300/10 plan unless my current provider gives me a deal on gig.

Just curious, how advanced does it have to be for you to like it?
First of all, you don't need that much hardware to cover a 150 or even 300 Mbps connection, so unless electricity is very cheap where you are at, I would suggest you find something newer and less power consuming.

I would probably add a managed switch to your network, and configure some VLANs to separate the devices on the network (network segmentation is the keyword here). If your devices need to talk to each other, I would do it via a Layer 3 switch, if not (or only in limited amount) a Layer 2 switch should be fine.

The primary motivation for doing this is improved security.
Among others it allows you to limit access to various management interfaces (firewall, AP's, switch) to only the machine you use for management.

Since you are using wifi routers with 3rd party firmware, I would configure those, so that the management interface is separated out on a VLAN or separate interface, and individual SSID's on VLANs, so you can configure them separately (one could be a guest network, another could be for mobile devices etc).
 

Oldhome7

Member
Feb 9, 2020
71
23
8
Electricity is 8/11 cents (winter/summer rates) here and I generate about 60% of my own anyways. So that averages out to 3/4 cents, if my math is correct.

I do have a 24 port managed Cisco 100mb switch in storage, but wouldn't the routers count as managed L3 "switches"? I have a few of those that are gigabit. Where would you suggest that get inserted?

Now VLANs and network segmentation is where you start speaking Greek to me. I'd like to keep this fairly open to talk to each other since there is generally a lot of file sharing across the network, mostly client to media server but occasionally client to client. The individual segments for certain wireless clients would be nice, especially with all this remote learning and my kids bringing home school chromebooks.
 

kapone

Well-Known Member
May 23, 2015
1,095
642
113
I recently re-did my pfSense from virtual to physical and used a teeny tiny board. It's a Gigabyte B75-TN thin-mini ITX motherboard. Has an mSATA slot onboard, a pci-e 3.0 x4 slot (which is where my 10g Nic is) and takes direct 12v/19v DC :)

The power consumption (i5-3570s, 2x4GB SO-DIMM RAM, mSATA SSD, passive heatsink , Mellanox CX3 10g Nic) is excellent. ~14w at idle with a platinum PSU.

This is an ITX board with an ancient i5. Runs ~14w like I said and handles full symmetric gigabit with pfBlockerNG and a few other packages. A Dell 510 is....well, way way overkill. Hell, if you really wanna use that thing, I'd go with a Type 1 hypervisor and install pfsense, your media server, maybe a file server and a bunch of other VMs on it.

Edit - the 14w power consumption is WITH a Mellanox CX3 10g card. Without it, it runs at less than 10w idle.
 

Oldhome7

Member
Feb 9, 2020
71
23
8
Can we get past the fact that the 510 is overkill? I got it for cheap and electricity is cheap. We're beating a dead horse here.

Now hypervisor is way over my head, I wouldn't know where to begin. I've played with VMs like virtualbox and such in Windows but that was years ago and only to play old W98 games and such.

My media server I'd like to keep as is in its own box. It serves double duty as my workstation for 3d modeling also.
 

kapone

Well-Known Member
May 23, 2015
1,095
642
113
Well, then install pfSense on the 510 and go to town?

In terms of "doing it right", only you can decide if your network design works for you or not. There is no right or wrong answer.
 

Oldhome7

Member
Feb 9, 2020
71
23
8
pfSense is already on the 510. It's the layout that I came here to get insight on. RTM mentioned VLANs and segregation which I'd like but I don't know where to start with that.
 

kapone

Well-Known Member
May 23, 2015
1,095
642
113
My rule of thumb about Layer 3/managed switches, VLANs, segregation etc etc is...not sure how to say this politely (and I do mean this in a good way, not trying to put you down or anything)...if you know how to set it up correctly. These concepts and their implementations across various vendors are very very complex, which is why network admins are paid what they are paid.

You're looking at implementing all of this, just at a smaller scale, but the knowledge remains the same.

So, that being said, if you really want to implement them, start reading/learning. Not gonna be quick, you're looking at weeks to months of learning.
 

Oldhome7

Member
Feb 9, 2020
71
23
8
I don't mind reading and learning. But most of what I find are posts that say do this, enter these settings, just generic cookie cutter copy and paste things. If you happen to have any recommendations on reading material, that would be appreciated.
 

RTM

Well-Known Member
Jan 26, 2014
956
359
63
Electricity is 8/11 cents (winter/summer rates) here and I generate about 60% of my own anyways. So that averages out to 3/4 cents, if my math is correct.

I do have a 24 port managed Cisco 100mb switch in storage, but wouldn't the routers count as managed L3 "switches"? I have a few of those that are gigabit. Where would you suggest that get inserted?

Now VLANs and network segmentation is where you start speaking Greek to me. I'd like to keep this fairly open to talk to each other since there is generally a lot of file sharing across the network, mostly client to media server but occasionally client to client. The individual segments for certain wireless clients would be nice, especially with all this remote learning and my kids bringing home school chromebooks.
Wow that is really inexpensive, I am jealous ;)

Keep the old Cisco switch in storage, I see no reason to start using that.
I consider the main reason why people use L3 switches, as opposed to L2 switches (that support VLANs), to be that they need to route more packets between local machines on different VLANs than what the firewall (main router) is able to (be it because of interfaces or straight up CPU performance). Given that your pfSense machine is clearly overkill and you have no 10G, it should be plenty capable.

You could definitely think of the wifi routers as L3 switches (L3 switches are routers), at least assuming that the 3rd party software will allow you to do so. I think the easiest (although it will probably take a bit of configuration) thing to do, would be to configure the AP that the chromebooks will connect to, to not allow access to anything in the primary network as provided by pfSense.

In order to do that you should connect the wifi router that the chromebooks will connect to the switch on the WAN/internet/whatever port of the router, the configuration should be largely be default but you want to ensure that it locally uses a different range than what the pfsense uses.
Say the pfsense uses the range 192.168.0.1/24 (means: 192.168.0.1-255, subnet: 255.255.255.0), then this wifi router should use a different range, like 192.168.1.1/24. Of course if you want to have different types of computers on the wifi AP's, you should configure the AP to have multiple SSID's (wifi names), each with the own range.
Keep in mind that with this configuration, you may get the same issues with performance due to P2P on computers connect to this AP.

Assuming you don't need to connect the chromebooks to the other AP, you could configure it to work essentially in bridge mode. The lazy way to do this, is to disable DHCP in its configuration and connect one of the LAN ports on the AP to the SE2800. Before you do this, you may want to configure the static IP-address of the AP to be in the same network range as the pfsense machine. Again assuming that pfsense uses 192.168.0.1/24, you could configure the AP to use 192.168.0.2/24. If the wifi routers software supports a simple bridge mode, then use that.

If you need the chromebooks to be able to connect to either of the AP's, AND you need/want to do P2P over wifi AND you want to segregate the chromebooks from the other devices, you may be able to do it via some kind of hybrid configuration, but honestly I believe at this point you should look into how to do VLAN's.
 

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
I don't have any sources, but learning about this stuff doesn't have to be too bad. I think of a VLAN as an isolated network. It's like if you have 2 switches that are not connected to each other. Now you have 2 networks. If you want machines to talk to each other on different networks, you need them to be on different subnets and have a router between them. A VLAN just lets you use one switch to do it.

There are two kinds of VLAN ports. One is usually called a trunk. It lets you send multiple VLANS over the same port. The devices connected must be able to understand trunked setups or the packets may just get dropped. The other type is an access port. All traffic will be on that VLAN, and no other VLANs traffic will ever pass through it. The switch itself enforces this. On a trunked port, you also set the default VLAN, which is the VLAN traffic without tags will be on. Start with a simple 2 VLAN setup. If you have a managed switch you aren't using, or some spare ports, you can do this without messing up your existing network.

As an example...

VLAN 1 - 10.0.0.0/24, normal internal network.
VLAN 2 - 192.168.1.0/24, guest network.

Port 1 - Trunk, VLAN 1/2 allowed, Default 1. Connect to the router.
Port 2 - Trunk, VLAN 1/2 allowed, Default 1. Connect to the AP.
Port 3 - Access, VLAN 1.
Port 4 - Access, VLAN 2.

Configure the AP to run the guest network on VLAN2. The router will have 2 interfaces in software. One for each VLAN, with DHCP and such all set up, and a firewall rule blocking traffic between the VLANs. If pfSense, there is an interface type for VLANS. You tell it which interface you want it on, and the VLAN number, and it creates a new interface for you to assign rules to etc..

To start with, perhaps ignore the AP and router and just test with static IP. Plug a machine into port 3 and set an address etc.. 10.0.0.1/24. Plug another machine into port 4, and set it to 10.0.0.2/24. You should not be able to ping each other. It should be as if they are on different switches. That's the VLAN working. Now, set up the router and you should be able to get DHCP on each machine, with different ranges to match the router. Now you also have a working trunk port and router. If the router has internet access, each VLAN should be able to hit the internet. And maybe each other, depending on the router.

Then you can add an AP to provide wifi on them. If your AP doesn't support VLANS, you could also use access ports and 2 APs. It's up to you.

The switch can also act as the router. That's what the Layer 3 stuff does. There are pros and cons to doing it in the switch, I decided I was already familiar with setting firewall rules on the router box, so I put it all there. Downside is that if the firewall is down, all traffic between VLANS is too. They can still communicate with other devices in the same VLAN, but not between them. They are also limited to 1Gbps as that's the link speed to the firewall. I have almost no inter-VLAN traffic, so it's not a big issue for me.
 

Oldhome7

Member
Feb 9, 2020
71
23
8
That's a lot to wrap my head around, thanks for that though.

With the custom firmware I can change quite a lot compared to the factory stuff. I think the most relevant to this would be disabling the WAN port so I can add it to the switch, changing the operating mode from gateway to router, and possibly enabling dynamic routing and extra SSIDs.

From the chromebooks point, if I'm understanding correctly, I could just hang an AP off the SE2800, or wherever, with a different subnet and it'll be segregated? And if I want to add range, just hang another AP in that same second subnet? Hey, that means I can send all my guests to that also lol.

Most everything is hardwired anyways so the P2P traffic doesn't touch the wireless. Most of the wireless is just phones, tablets, a few laptops (not including the aforementioned chromebooks), and IoT devices.

Edit: Forgot to add, I should probably disable NAT on those APs as well.
 
Last edited: