Proxmox networking guidance

Discussion in 'Linux Admins, Storage and Virtualization' started by Cape, Feb 11, 2019 at 12:14 PM.

  1. Cape

    Cape New Member

    Joined:
    Oct 28, 2015
    Messages:
    29
    Likes Received:
    1
    Hey,
    I'm setting up Proxmox for the first time, after being a long-time ESXi user, and am somewhat confused on how to best configure networking.
    I've decided to go with open vSwitch, and what I think I want to do is this (certainly up for debate if there's better ideas):

    eno1/vmbr1 - Management net, connected to (untagged) VLAN 255
    * mgmt1 - OVS_IntPort with proxmox management IP
    * Any VMs which expose management interfaces will have a vNIC here
    eno2/vmbr2 - WAN net, connected to (untagged) VLAN 1000
    * VMs which expose directly to WAN. Initially will only be pfsense, but there are plans that could include others.
    eno3/vmbr3 - "Internal" net, connected to trunked VLAN port for client lan, etc
    * Most VMs
    eno4/vmbr4 - Storage?
    * If/when there are multiple nodes this will be used for storage

    I'm hitting a road block pretty much right away when setting up pfsense, initially passing it NICs on the WAN and management bridges. Setting an IP on the management interface, I'm unable to reach it from other machines. The PVE host can reach it, though.
    So, two questions:
    1. Am I doing it wrong? :) Basically, this is translated as best as I could from how I'd do an ESXi setup, and it might not make sense here.
    2. If not conceptually wrong, why can't I reach my VM? I tried enabling ip forwarding/routing on the PVE host, which didn't help (although I would expect this not to be needed, OVS should handle that, right?
     
    #1
  2. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,835
    Likes Received:
    812
    I ***think*** you still need to do E1000 NICs on the pfSense NICs with Proxmox. FreeBSD is well...…..

    If that isn't it, then there is some other little setting like that. I can't remember exactly, but I think it's the E1000 NIC issue why you can't get to the pfSense NIC.
     
    #2
  3. Cape

    Cape New Member

    Joined:
    Oct 28, 2015
    Messages:
    29
    Likes Received:
    1
    #3
  4. Cape

    Cape New Member

    Joined:
    Oct 28, 2015
    Messages:
    29
    Likes Received:
    1
    Nope, E1000 does not work either :( Thanks for the suggestion though! Any idea what other setting could be relevant? Basically running with just standard everything on that VM right now.
     
    #4
  5. vudu

    vudu New Member

    Joined:
    Dec 30, 2017
    Messages:
    20
    Likes Received:
    4
  6. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,835
    Likes Received:
    812
    That's it. Sorry I brain farted. E1000 is a different one.
     
    #6
  7. niekbergboer

    niekbergboer Member

    Joined:
    Jun 21, 2016
    Messages:
    99
    Likes Received:
    32
    I run a virtualized pfSense using VirtIO, and indeed, you have to disable offloading or you'll see a lot of packet loss.
     
    #7
  8. Cape

    Cape New Member

    Joined:
    Oct 28, 2015
    Messages:
    29
    Likes Received:
    1
    Ah, cool. I'll try that tonight!
    But this is causing packet loss? I seem to be unable to even route to the VM, though. Should it be that severe packet loss?
     
    #8
  9. Cape

    Cape New Member

    Joined:
    Oct 28, 2015
    Messages:
    29
    Likes Received:
    1
    Did some more digging now. I cannot access the ui, so I don't really have a way to disable offload...
    I can however do tcpdump via the console. So I have had tcpdumps on my laptop, the proxmox host, and on the pfsense VM. I can see ping/curl etc coming in to pfsense (ie showing up in all three tcpdumps), but pfsense doesn't seem to respond. Even on the pfsense box, I don't see any replies in the dump.
    Outbound access from pfSense works, though...

    Could it be so simple as pf installing with everything blocked for some reason? I've tried resetting interfaces etc without effect
     
    #9
  10. Cape

    Cape New Member

    Joined:
    Oct 28, 2015
    Messages:
    29
    Likes Received:
    1
    This is beginning to look the case. Started up a Linux VM and was able to reach both in and out... Will try an older pfsense image and see if there's a problem with the latest
     
    #10
  11. Cape

    Cape New Member

    Joined:
    Oct 28, 2015
    Messages:
    29
    Likes Received:
    1
    *Finally* found the problem... My own doing, of course.
    Since I'm in the migration process from ESXi, I've set up proxmox side-by-side with my old lab, which still has the pfsense acting as router/gateway for the mangement network. When setting up a new pfSense install, it seems it expects it to be the gateway of all networks (fair enough), and thus doesn't add a gateway on the LAN/management interface. So thus it can't reply to my pings etc which comes from a different subnet...

    Facepalms have been had. Sorry for wasting your time, and appreciate the suggestions!
     
    #11
    vudu likes this.
  12. vudu

    vudu New Member

    Joined:
    Dec 30, 2017
    Messages:
    20
    Likes Received:
    4
    Glad you got it sorted and sorry I hadn't been keeping a closer eye on this. The gateway issues you mentioned have bit me many times when I have built a PFSense box on our LAN and then moved it to a customer LAN with different subnet.
     
    #12
  13. Cape

    Cape New Member

    Joined:
    Oct 28, 2015
    Messages:
    29
    Likes Received:
    1
    Btw, I did have to go back an disable the offload when I finally got everything working. Speed tests with ~1 Mbps instead of 800+ Mbps...
    So thanks for pointing that out in advance :)
     
    #13
Similar Threads: Proxmox networking
Forum Title Date
Linux Admins, Storage and Virtualization pfSense on Proxmox What Does Networking Look Like? Jul 22, 2018
Linux Admins, Storage and Virtualization Proxmox Clustering / Networking Resources (coming from ESXi) Mar 1, 2016
Linux Admins, Storage and Virtualization Disable proxmox host ipv6 dhcp on WAN port Today at 6:30 AM
Linux Admins, Storage and Virtualization Proxmox network conundrum, help needed and appreciated Feb 9, 2019
Linux Admins, Storage and Virtualization Poor Low Que Depth Performance with Windows 10 Guest under Proxmox KVM 5.3-8 Feb 1, 2019

Share This Page