Hmmm, it appears this Brocade thread is ALL over the place anyway, so I'll continue to respond in turn here. I hope a mod can start splicing this thread up into various new threads and keep the Brocade ICX-specific stuff in here.
Yes hidden gems and treasures. I concur with what
@Blue)(Fusion is recommending on ACL's.
@svtkobra7 Unless you're in a hurry I'd pause and take a step back for a moment. Based on what you've posted you have at least 1 underlying IP challenge. I also do disagree with using gi-normous subnets. (that's probably not fatal but...)... However that is up to you.
Warning may be TLDR.
<#Disclaimer>
What I've written here is an opinion and is not necessarily the right or best way for you - that's up to you to decide. Just offering ideas, food for thought etc.
</Disclaimer>
This is all probably overkill and may be extreme - but a stripped down version of this thought process is still IMO prudent. KISS is important too, over engineering / designing can also lead to challenges - but there's a balance here. If you think about it ahead of time and plan and maintain simplicity - adding functional areas, services, networks later etc is a lot easier. Not unlike an addition to a house if the house was designed for additions in the first place.
so:
Plan - you've already done good work mapping out storage, mgmt planes, and the like.
What services are "mission critical" in your environment? Don't forget WAF, SAF, PAF, RAF... Nothing like a good Saturday evening home engineering session that is met with a sudden silence and then yells from other rooms when Netflix no longer work-ie instead of Wookie-ing.
Does this network support household mission critical services?
If no - continue on to the networking section.
If yes - think carefully about whether to segregate those services entirely or build them into this infrastructure and protect them.
Here's a starter list and is by no means complete - really just something to get the noodle churning.
In no particular order:
LAN Services
DHCP (
@Blue)(Fusion uses Pfsense, there are other options: switch based? dedicated server based with switch helper? etc.
DNS (are you running internal DNS?, 1 zone, many zones? if many think about how you will map zones to subnets)
Syslog
Internet (well depends on DHCP, DNS, transit, NAT etc. etc.) ?
Internal Backup of desktops laptops and the like?
Peer to Peer access and/or storage?
Shared Central Storage? (what forms? NFS, CIFS, Bonjour)
Monitoring of your infrastructure?
Printers?
VoIP?
IoT things (Ring, Nest, SCADA, 3d printers, Cameras)
WiFi (multiple SSID's mapping into the above?)
MFG ecosystems (for example, Apple)
Internet to your network VPN?
Classes of Users
Pinholing services (please don't)
Technology test beds (are you testing new (to you and/or industry) technologies) and if so who/what needs access to it?
Are you going to fulfill any of these services via VM's? One service per VM?
Once you've got your beginning set of services start thinking about the network and inter-relationship between the services
Networking
Consider whether you want to segregate any of the following (I think you do based on your previous posts):
IPMI Network? (mgmt plane)
vmware mgmt network? (mgmt plane)
storage
NFS 4.1
vSAN (are you going to play with this? do you want it on a separate network? plan now if you do - implement later if you want)
CIFS?
IOT (or cameras, or NEST, or RING etc.)
WAN
LAN <-> Internet
VPN <-> Internet <-> Your Network
Are you going to be doing IPv6? If so how and where.
Virtualization:
vMotion or other vendor equiv.?
vSAN or other vendor equiv. ?
mgmt planes?
prviate within virtual host network?
Physical interconnects, meta layers and guest system presentation.
only going to say this once for your VM hosts: trunk and tag, trunk and tag, trunk and tag
Even with with things like FreeNAS: consider trunk and tag.
leverage meta layers within your solutions to minimize impact of lower level changes.
Link AGG / connection protection - Build your network first and it may be simpler / easier to get working with single connections and add link agg or the like later.
IP-Wise (only talking about ipv4 here - ipv6 whole 'nother discussion)
Think about the subnet ranges you are going to use:
Is 10/8 right? 172.16/16? 172.27/16? 192.168/16? (please don't use 192.168.1-.3) You have a ton of options and the right one for you may be shaped by your answers to some of the previous questions.
Do you need to worry about IP subnet conflicts from outside coming in (VPN)?
Do you need to worry about IP subnet conflicts from inside going out to other networks (VPN)?
What classes of systems/clients need access to what?
You've already picked what I consider a nice convention, if possible use subnet numbering to match VLAN ID (your storage network for example)
Are there any other conventions to use? This can lead to limiting the number of VLANs but unless you are doing boundary condition testing 2 - 254 vlans may be sufficient. You can do more and still match vlan ID to subnet by extending into the next octet of the subnet.
What size subnets do you need? Do you want to add the complexity and brain cycles thinking about supernetting and/or classless subnetting while troubleshooting?
Access Control
@Blue)(Fusion has excellent points.
How do the various points that you map out fit into the capabilities of the ICX 6610 - what can you take advantage of and what do you need to learn to do so?
Last questions:
There's a bit of focus in your posts about virtualization, how many of your critical services are depending on it and if that part of your infrastructure fails, how fast can you implement a fall back or work-around and what is that plan?
Do any of the design/implementation decisions you are making create additional complexity for that plan? If so should that complexity be simplified?
trade-offs there are always trade-offs.
itr
