Proxmox - Separate Network WITH IP-Adress for interconnect

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Markus

Member
Oct 25, 2015
78
19
8
Hello,

I just started to build up my 3-node-Cluster and have one question regarding a "separate" Management-Network.

I want to have a separate Mgmt-Network, a separate Network for cluster interconnect and several networks for the VMs.

So as far as I know this could not be achieved in an easy way.
  • vmbr0 is always the Management-Network and got an IP.
    • IP -Address is given so I can access the server on ports 22, 8006, 5900 etc
  • vmbr1 should be the Cluster-Interconnect
    • IP-Adress is given (because 1 need at least 3 participants - node1, node2, node3
    • But: As all Services listen on *, the webinterface and vnc are also accessible on this interface
  • vmbr2-x should be the Bridges for the VMs
    • No IP-Adresses - Just VLAN-Tagging on the bridges...
So now the questions.
Is this the way to got? Just keep the Services on the interconnect?
Enable the firewall and just let through the ports for the cluster (ssh, corosync, ceph...)?

How this is handled?

Regards
Markus
 

TangoWhiskey9

Active Member
Jun 28, 2013
402
59
28
i usually do an internal and an external NIC. Internal keeps storage and management. External is for VMs communicating to the outside world. The issue is that whatever you install with, I think you need to keep as the management IP.
 

Markus

Member
Oct 25, 2015
78
19
8
But that's the thing.

If you don't apply any further configuration and your external NIC also got an IP-Address on the host you can access the management services (Webinterface, SSH) on this interface as well.

Do you apply a firewall-config or something like this?
Regards
Markus
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
One thing that you might want to try is to use a private IP/ network during Proxmox setup. I always use a private IP address range for that which requires VPN'ing into. I keep a separate storage network and then a third network for VM traffic that goes to the firewall to the outside world.
 

Markus

Member
Oct 25, 2015
78
19
8
@g33k: I don't think that there is a big difference. As my labs are most of the times an "nearly" exact repica of the production site I need to implement such a setup in both worlds.

@Patrick : As I am in a LAN I only got internal / private IP-Adresses.
How do you excactly implement the separate storage network (I mean: Please speak in config files to me :)).

Regards
Markus
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
The storage network what I basically do is create a second IP address (no gateway) that I add to a 10GbE NIC. I then setup Ceph to use that network.
 

Markus

Member
Oct 25, 2015
78
19
8
Ok. But you don't do any further measures to prevent the binding of all services on those interfaces, don't you?

So the webinterface and for example the spiceproxy are also listening on this interface?
Regards
Markus
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
Ok. But you don't do any further measures to prevent the binding of all services on those interfaces, don't you?

So the webinterface and for example the spiceproxy are also listening on this interface?
Regards
Markus
I have not had the need to yet. Then again, I am running all the VMs on the cluster and all hardware behind the firewall so I might work harder if I were building a public hosting cluster.