Hi all.
I'm sure I'm missing something obvious, but I'm a hardware guy trying to get into a software world. So this may or may not be a super easy fix.
I currently run a flat network, 192.168.0.0/24, which I am working to replace entirely with a vlan'd up network, coming soon when my other renovations give me wall access to run everything wired.
I have my 3 ICXs that I'll be using around the place, and am trying to build a network to get some practice with VLANs and routing and firewalling in the mean time.
In doing so, I set up the 7150-24 to take the place of the older netgear switch that was linking everything at my desk to my Wireless bridge. Straightforward, all ports untagged VLAN 1, with a VE using 192.168.0.152, set statically on the switch (and outside my current DHCP reservation pool)
I've added a new VLAN, 22, and tagged it on port 8, and connected that to port 1 on my 7250, and gave them both VEs in the 10.0.0.0/24 range, again statically (7150 is 10.0.0.2, 7250 is 10.0.0.1). I then added a static route to my desktop for 10.0.0.0/24 to go via 192.168.0.152
So far so good, I can ssh into the 7250 at 10.0.0.1, interVLAN routing works like a charm, since the 7150 has no ACLs or anything. I put away the 7250's Serial Cable.
So I go ahead and set up a few VLANs and VEs on the 7250, tag and untag a few ports with different VLANs and get comfortable with the syntax. Time for something new.
I connect up my little SuperMicro 1U server's IPMI port to the 7250, on a port that is untagged VLAN 22. Use it's BIOS to set a static IP, 10.0.0.4/24. Reboot, and bam, I can remote in from my desktop. Now in the IPMI web interface, I change it so it's ethernet port is tagging everything VLAN 22, and then go back to the 7250 and remove the untagged 22 and tag it 22 instead. Success, can still log into the IPMI from my desktop.
So I go ahead and install Proxmox using the IPMI's virtual media function. I connect a DAC between the SFP+ port on the 7250 and the SFP+ port on the 1U, and using the IPMI's KVM, set a static IP, 10.0.0.5/24.
Now from the Desktop, I can both ssh into Proxmox, as well as access it's web interface, both at 10.0.0.5, no problems. As I go about configuring it though, I find that Proxmox cannot connect to the internet. Oh, it's DNS server is itself. Cool, just change that to my DNS server on 192.168.0.200, easy.
Nope, still can't resolve domains. Can't ping the DNS server either, huh.
Right, static routes. Configure a static route to 192.168.0.0/24 via 10.0.0.2, but no change, still can't reach my DNS server.
But I can ping my desktop, which is 192.168.0.2, which I guess makes sense, since the SSH connection to it would tell each device along the path to it where it came from. And the DNS server is on the other side of the Wireless bridge, so that's gonna be a bit tricky to route to, perhaps.
So I try to ping my NUC, which is directly connected to the wireless bridge (192.168.0.203) via a VLAN 1 port, at 192.168.0.10, and just get time outs. So the Wireless bridge is the problem here? Nope. I SSH from the NUC to the IPMI of the 1U, and now proxmox can ping the NUC but still NOT the wireless bridge.
I've tried setting a static route on the 7150, as "0.0.0.0/0 via 192.168.0.203", "192.168.0.0/24 via 192.168.0.203", and "0.0.0.0/0 via 192.168.0.201", but no change. The 7150 can ping the gateway (192.168.0.201) and the DNS server fine, but Proxmox behind it gets told there is no route.
So.... what little thing did I miss?
Network topology
Gateway (192.168.0.201) and DNS Server (192.168.0.200)
Wired to
Wireless Router (and DHCP Server) (192.168.0.1)
Wireless to
Wireless Bridge (192.168.0.203)
Wired to
NUC (192.168.0.10) and 7150 (VLAN 1: 192.168.0.152, Static IP; VLAN 22: 10.0.0.2, Static IP)
7150, VLAN 1 untagged
Wired to
Desktop (192.168.0.2)
7150, VLAN 22 tagged
Wired to
7250 (10.0.0.1)
Wired to
IPMI (10.0.0.4, tagged VLAN 22) and Proxmox (10.0.0.5, untagged VLAN 22)
I'm sure I'm missing something obvious, but I'm a hardware guy trying to get into a software world. So this may or may not be a super easy fix.
I currently run a flat network, 192.168.0.0/24, which I am working to replace entirely with a vlan'd up network, coming soon when my other renovations give me wall access to run everything wired.
I have my 3 ICXs that I'll be using around the place, and am trying to build a network to get some practice with VLANs and routing and firewalling in the mean time.
In doing so, I set up the 7150-24 to take the place of the older netgear switch that was linking everything at my desk to my Wireless bridge. Straightforward, all ports untagged VLAN 1, with a VE using 192.168.0.152, set statically on the switch (and outside my current DHCP reservation pool)
I've added a new VLAN, 22, and tagged it on port 8, and connected that to port 1 on my 7250, and gave them both VEs in the 10.0.0.0/24 range, again statically (7150 is 10.0.0.2, 7250 is 10.0.0.1). I then added a static route to my desktop for 10.0.0.0/24 to go via 192.168.0.152
So far so good, I can ssh into the 7250 at 10.0.0.1, interVLAN routing works like a charm, since the 7150 has no ACLs or anything. I put away the 7250's Serial Cable.
So I go ahead and set up a few VLANs and VEs on the 7250, tag and untag a few ports with different VLANs and get comfortable with the syntax. Time for something new.
I connect up my little SuperMicro 1U server's IPMI port to the 7250, on a port that is untagged VLAN 22. Use it's BIOS to set a static IP, 10.0.0.4/24. Reboot, and bam, I can remote in from my desktop. Now in the IPMI web interface, I change it so it's ethernet port is tagging everything VLAN 22, and then go back to the 7250 and remove the untagged 22 and tag it 22 instead. Success, can still log into the IPMI from my desktop.
So I go ahead and install Proxmox using the IPMI's virtual media function. I connect a DAC between the SFP+ port on the 7250 and the SFP+ port on the 1U, and using the IPMI's KVM, set a static IP, 10.0.0.5/24.
Now from the Desktop, I can both ssh into Proxmox, as well as access it's web interface, both at 10.0.0.5, no problems. As I go about configuring it though, I find that Proxmox cannot connect to the internet. Oh, it's DNS server is itself. Cool, just change that to my DNS server on 192.168.0.200, easy.
Nope, still can't resolve domains. Can't ping the DNS server either, huh.
Right, static routes. Configure a static route to 192.168.0.0/24 via 10.0.0.2, but no change, still can't reach my DNS server.
But I can ping my desktop, which is 192.168.0.2, which I guess makes sense, since the SSH connection to it would tell each device along the path to it where it came from. And the DNS server is on the other side of the Wireless bridge, so that's gonna be a bit tricky to route to, perhaps.
So I try to ping my NUC, which is directly connected to the wireless bridge (192.168.0.203) via a VLAN 1 port, at 192.168.0.10, and just get time outs. So the Wireless bridge is the problem here? Nope. I SSH from the NUC to the IPMI of the 1U, and now proxmox can ping the NUC but still NOT the wireless bridge.
I've tried setting a static route on the 7150, as "0.0.0.0/0 via 192.168.0.203", "192.168.0.0/24 via 192.168.0.203", and "0.0.0.0/0 via 192.168.0.201", but no change. The 7150 can ping the gateway (192.168.0.201) and the DNS server fine, but Proxmox behind it gets told there is no route.
So.... what little thing did I miss?
Network topology
Gateway (192.168.0.201) and DNS Server (192.168.0.200)
Wired to
Wireless Router (and DHCP Server) (192.168.0.1)
Wireless to
Wireless Bridge (192.168.0.203)
Wired to
NUC (192.168.0.10) and 7150 (VLAN 1: 192.168.0.152, Static IP; VLAN 22: 10.0.0.2, Static IP)
7150, VLAN 1 untagged
Wired to
Desktop (192.168.0.2)
7150, VLAN 22 tagged
Wired to
7250 (10.0.0.1)
Wired to
IPMI (10.0.0.4, tagged VLAN 22) and Proxmox (10.0.0.5, untagged VLAN 22)
Code:
Current configuration:
!
ver 08.0.95hT213
!
stack unit 1
module 1 icx7150-24-port-management-module
module 2 icx7150-2-copper-port-2g-module
module 3 icx7150-4-sfp-plus-port-40g-module
!
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 22 by port
tagged ethe 1/1/8
router-interface ve 22
!
!
!
!
!
!
!
!
!
!
!
!
!
!
aaa authentication web-server default local
aaa authentication login default local
enable aaa console
ip dhcp-client disable
!
no telnet server
username super password .....
!
!
!
!
clock timezone gmt GMT+10
!
manager disable
!
!
manager port-list 987
!
!
!
!
!
!
!
!
!
interface ethernet 1/3/1
speed-duplex 10G-full
!
interface ethernet 1/3/2
speed-duplex 10G-full
!
interface ethernet 1/3/3
speed-duplex 10G-full
!
interface ethernet 1/3/4
speed-duplex 10G-full
!
interface ve 1
ip address 192.168.0.152 255.255.255.0
!
interface ve 22
ip address 10.0.0.2 255.255.255.0
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
end
Code:
Total number of IP routes: 2
Type Codes - B:BGP D:Connected O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP Codes - i:iBGP e:eBGP
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2
STATIC Codes - v:Inter-VRF
Destination Gateway Port Cost Type Uptime
1 10.0.0.0/24 DIRECT ve 22 0/0 D 2h34m
2 192.168.0.0/24 DIRECT ve 1 0/0 D 95d0h
Code:
Current configuration:
!
ver 08.0.95hT213
!
stack unit 1
module 1 icx7250-24p-poe-port-management-module
module 2 icx7250-sfp-plus-8port-80g-module
!
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 22 by port
tagged ethe 1/1/1 ethe 1/1/24 ethe 1/2/1 ethe 1/2/3
untagged ethe 1/2/2
router-interface ve 22
!
vlan 50 by port
tagged ethe 1/2/1 to 1/2/3
router-interface ve 50
!
vlan 60 by port
tagged ethe 1/2/1 to 1/2/3
router-interface ve 60
!
vlan 70 by port
tagged ethe 1/2/1 to 1/2/3
router-interface ve 70
!
vlan 80 by port
tagged ethe 1/2/1 to 1/2/3
router-interface ve 80
!
vlan 107 by port
tagged ethe 1/2/1 to 1/2/3
router-interface ve 107
!
!
!
!
!
!
!
!
!
!
!
!
!
!
optical-monitor
optical-monitor non-ruckus-optic-enable
aaa authentication web-server default local
aaa authentication login default local
enable aaa console
ip dhcp-client disable
!
no telnet server
username super password .....
!
!
!
!
!
manager disable
!
!
manager port-list 987
!
!
!
!
!
!
!
!
!
interface ethernet 1/1/1
no inline power
!
interface ethernet 1/1/2
no inline power
!
interface ethernet 1/1/3
no inline power
!
interface ethernet 1/1/4
no inline power
!
interface ethernet 1/1/5
no inline power
!
interface ethernet 1/1/6
no inline power
!
interface ethernet 1/1/7
no inline power
!
interface ethernet 1/1/8
no inline power
!
interface ethernet 1/1/9
no inline power
!
interface ethernet 1/1/10
no inline power
!
interface ethernet 1/1/11
no inline power
!
interface ethernet 1/1/12
no inline power
!
interface ethernet 1/1/13
no inline power
!
interface ethernet 1/1/14
no inline power
!
interface ethernet 1/1/15
no inline power
!
interface ethernet 1/1/16
no inline power
!
interface ethernet 1/1/17
no inline power
!
interface ethernet 1/1/18
no inline power
!
interface ethernet 1/1/19
no inline power
!
interface ethernet 1/1/20
no inline power
!
interface ethernet 1/1/21
no inline power
!
interface ethernet 1/1/22
no inline power
!
interface ethernet 1/1/23
no inline power
!
interface ethernet 1/1/24
no inline power
!
interface ethernet 1/2/1
speed-duplex 10G-full
!
interface ethernet 1/2/2
no optical-monitor
speed-duplex 10G-full
!
interface ethernet 1/2/3
speed-duplex 10G-full
!
interface ethernet 1/2/4
speed-duplex 10G-full
!
interface ethernet 1/2/5
speed-duplex 10G-full
!
interface ethernet 1/2/6
speed-duplex 10G-full
!
interface ethernet 1/2/7
speed-duplex 10G-full
!
interface ethernet 1/2/8
speed-duplex 10G-full
!
interface ve 1
ip address 192.168.0.151 255.255.255.0
!
interface ve 22
ip address 10.0.0.1 255.255.255.0
!
interface ve 50
!
interface ve 60
!
interface ve 70
!
interface ve 80
!
interface ve 107
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
end
Code:
Total number of IP routes: 2
Type Codes - B:BGP D:Connected O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP Codes - i:iBGP e:eBGP
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2
STATIC Codes - v:Inter-VRF
Destination Gateway Port Cost Type Uptime
1 10.0.0.0/24 DIRECT ve 22 0/0 D 2h33m
2 192.168.0.0/24 DIRECT ve 1 0/0 D 2h33m
Code:
auto lo
iface lo inet loopback
iface eno7 inet manual
iface eno1 inet manual
iface eno2 inet manual
iface eno3 inet manual
iface eno4 inet manual
iface eno5 inet manual
iface eno6 inet manual
iface eno8 inet manual
auto vmbr0
iface vmbr0 inet static
address 10.0.0.5/24
gateway 10.0.0.2
bridge-ports eno7
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
Code:
default via 10.0.0.2 dev vmbr0 proto kernel onlink
10.0.0.0/24 dev vmbr0 proto kernel scope link src 10.0.0.5