FreeNas: How to Bridge Nics like a Switch and DHCP-serve them

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

RageBone

Active Member
Jul 11, 2017
617
159
43
In the wake of the Jellyfish "it just works out of the box etc BS", scenario is as follows.

You have a FreeNas box with multiple Nic-ports that you are directly connecting to the client devices.
Possibly a 10Gbit Rj45 or in my case, Mellanox CX3s set to 40GbE.

For maximum comfort, all clients should be able to reach the NAS under the same IP.
That is Behavior of a network-switch with the exception that a separate one, is a bit much when the NAS can be made to behave the same.
For instance additional NICs for the NAS-switch could cost less then a respective network-switch.
This doesn't take up additional space and might be lower in power-consumption.
One apparent downside to this solution is that all clients can now also reach each other through the NAS, and all of this puts more load on the NAS, but for me that isn't of concern.

Why should you need to manually configure the clients IP?
Automatic configuration of IP addresses via DHCP it is!
In addition, PXE enables the clients to boot from the NAS if desired and configured.

Lets begin with the configuration of FreeNas as a "Switch".
The Bridge Interface.
Since FreeNas is an appliance and scripts can't simply be put and executed on it, the following are my System-Tunables.
Format is variable, value, type. comment and enable are up to you.

Code:
cloned_interfaces, bridge9, rc
ifconfig_bridge9_aliases, inet IP/Mask, rc
ifconfig_bridge9, addm Nic1 addm Nic2 .... addm NicX , rc
It might be necessary to UP all the devices, just in case you run into issues where things aren't ping-able. For example
Code:
ifconfig_NicX, up, rc
....
bridge9 because FreeNas itself creates bridges and taps "on the fly" for VMs and other stuff, so i chose that number to not be in direct conflict.

FreeNas sadly isn't jet able to create Bridges in the UI, or list them, or use their IP as a selection anywhere.
Shares need to be bound to 0.0.0.0 to also be reachable by the bridge.

The DHCP Server
In short,
FreeNas doesn't come with some kernel-modules needed for DHCP to work, Jails won't therefore work.
So create a new VM with what ever OS you like, FreeBsd in my case. There is sadly no way around a VM that i know of.
Add a Tap device to the cloned_interfaces and addm parts of the Tunables "tap9" for example and add it to the VM.
The order of devices of the FreeNas dialog will be the same as inside the VM, and tap devices are easily addable there.

Install your dhcpserver of choice, dnsmasq in my case, and configure it for your IP range on the tap interface.
Add a static IP to the passed Tap device inside of the VM, start and enable autostart of the dhcpserver inside of the VM, and of the VM and you are Done.
You should now have everything working.

Did i miss anything?
And yes, I'm not the first to write about this, but i thought it is time to share this little setup that seems so simple, still is way to complicated and not enough supported by FreeNas and UI, and maybe keep others from wasting time on BS.
Like getting DHCP working inside a Jail.
 
  • Like
Reactions: scline and gb00s

scline

Member
Apr 7, 2016
92
33
18
36
Are there any performance drops in networking bandwidth when bridging vs routed interfaces? I know many platforms don't "switch" at line rate when lacking specialized ASIC's or NIC's.
 

RageBone

Active Member
Jul 11, 2017
617
159
43
i have noticed that my duallport cx3 40GbE card will drop to about 11Gbits in any case when it can normally do about 28Gbits with my particular setup.
It has been a while that i looked into this and debugging was or is kinda hard so i can't really say anything else.
 

ziggygt

Member
Jul 23, 2019
62
10
8
support for bridging is now native in Freenas 11.3
I cannot get it to save. I can assign the two the ports of my Intel X520 to a bridge and IP address 192.168.0.62. And I can use that port to log into the server and it seems to work. When I go into the network section to check that the bridge is installed it is not. Does this automatially reset the bridge? why does it work that way?
 
Last edited:

RageBone

Active Member
Jul 11, 2017
617
159
43
the new "test" and "save" changes functionality that seems to be new in 11.3 still gets me sometimes.
It would at least maybe explain your issues.

The attached picture shows my current config, so it is definitively working.

Had something break on another Freenas machine though, testing the changes caused an "error 13, device is busy" and the bridge did disappear afterward. No clue what broke in that case, it worked on it before and i couldn't get it to work on that one again.
I'd recommend a reinstall in that case.
 

Attachments

ziggygt

Member
Jul 23, 2019
62
10
8
i was able to get this to work by logging in to the server on a separate subnet from another machine on the network. I could then configure the bridge and the connection was still around after the testing to be able to save the configuration. I can only get this to work on my mellanox cards.
 

RageBone

Active Member
Jul 11, 2017
617
159
43
@ziggygt it has been a while for me since i dabbled in this topic.

Would you mind elaborating on your setup?
Bridging wad become supported by the Freenas UI at the time.
I am not aware if that changed with Truenas.
 

ziggygt

Member
Jul 23, 2019
62
10
8
Here is an explanation at this link. Bridge between network interfaces | Page 2 | TrueNAS Community
Basically, I could not get the bridge configuration saved because when you make the change from one of the interfaces that will be in the bridge the GUI disconnects and you cannot press save. So the bridge works until the test times out or the system is rebooted. I logged into the server with an interface that will not be part of the bridge. to do this i logged into the server at ex:192.168.1.60 fixed address from a different computer. Set up the bridge on the server. that node is unaffected by the new config so it is possible to save the state. The bridge is set up on a different subnet ex; 192.168.0.60.
 

ziggygt

Member
Jul 23, 2019
62
10
8
I thought my experience comparing a bridge in TRUENAS and 4 Mellanox 10Gb dual port cards as a switch verses a TP-Link TL-SX3008F 8 port 10Gb switch might be useful to anyone viewing this thread.

Summary:
- Performance of bridge is almost as good as the switch (in my tests)
- Bridge takes less than 3% of CPU when transferring data
- If you are not already running the server you can pay for the new consumers switches with the power savings in about a year. 200watts vs 20watts
- Microtik and TP-Link 10Gb switches are quiet. Server or Surplus switches are noisy.