E3-1220L V2 w\S3500 mirror SSD + pFsense = around 17w power each. Cheap and easy. You can get entire chassis\mobo\CPU for $100-200 ready to run, just add SSD. Even less power but more up front $ is the mini pfSense appliance that are like 5"x5" but cost $200-$300 depending on specs, they are lower than 17w and smaller than 1U. You could do this with an Intel Atom but by the time you build that out you're talking even more $$ but they're lower spec, and more powerful.
I will cross-post your suggestions to the
Low Power / HA - Router options. It seems like the HA router setup might be easier than I expected and I was already given some pretty decent advice I completely overlooked like 'Why not just use a normal WAP and flash DDRT?" Idk why I didn't think of that. I flashed at least 2 or 3 WRT-54G's in my day.
I think to hit the final goal I will need to pinch every watt. So that's why I like the 2.5 watt pi router. I'm also not sure what I need to run on the routers and how much CPU that will consume. My normal home router, I usually only modify the DNS servers, turn on DNSmasq, change the DHCP range, add a couple of static leases. Thats it.
In this project I'm guessing it will about the same but I want to have wireguard VPN (somewhere on the network), maybe will need DynamicDNS and I was thinking to have some VLANs. Nothing crazy like packet shaping or IDS. Maybe some logging and some metrics from the routers? Basic stuff like access logs and some CPU/Memory stuff, uptime etc.
Now I have only read about VLAN and never really used them. I was thinking of having a couple of segments like wifi, guest/wifi, IOT, Managment, and then whatever I would call the "rack" maybe "core". With just some simple route rules like don't allow guest/wifi to access management devices. I'm not sure the CPU hit for that or if the switches I use will be L3 and can handle the routing.
For switch do you really need HA? Or could you just load the same config on a 2nd, keep it powered down unless you need it then power it on? This is what I do at my home for router and switch, then you save power all the time and can swap if needed.
I think this will also be a common theme. What can be fault-tolerant vs highly available? It seems even I have mixed up the terms according to my favorite
article on the subject. The plan is no single point of failure (at least in the core network). I would prefer fault-tolerant over highly available. If the VPN is on the routers and I can hit the network with a switch being down and I can WoL the switch? That might be something I look at. But if I can just get 2 regular switches and then plug one switch into the other then as long as the switch is like 20 or 30 watts (mikrotik), it might be worth it.
The hardest thing about this project is the dependencies from one layer to the next. What I choose for the routers will greatly influence every other part of the build. I'm actually better with routers than switches for whatever reason. I will definitely be making a sub-thread for switch discussions. I'm not even sure what I need. I think like 12-24 ports should be fine for the main rack? Then I can dangle a switch in the house for WIFI and home entertainment devices? But what about the security cameras and stuff? POE? This is where some investigations and POC's might help. Is a single POE switch more energy-efficient than using DC wall outlets to power the cameras? What about when the Cat5 run is 100+ feet? These little things will end up mattering the most I suspect.
For Server\Hypervisor it's hard to beat the AMD right now for power\performance ratio. 5600x or you can save $ with previous gen. ASROCK Rack motherboard is the big $ item here, and the RAM isn't cheap, but neither is other UDIMM (I guess RDIMM isn't a deal right now either June\2021). Another option E3 v3 will cost MUCH LESS to build, we're talking $150 for CPU+Motherboard and maybe 4 or 8GB if you can find a deal. Less performance, but if you don't have high demand needs you could build 4x of these for nearly the same as one 5600x. If you're going to load them with 32GB (or more in the AMD) well RAM cost is almost the same right now so you're spending that either way.
I will definitely get a thread up soon about this. It's the hardest part of the build for sure. I think it will also require the most creativity lol, I'm even thinking about gutting laptops for nodes etc. I wan't to first figure out the hypervisor. So expect the software thread first. I'm roughly budgeting 200w for core networking. That leaves about 400ish for servers. Lets say 125w's each average load. That's not an easy ask. Throw in 32GB of Ram and 10ish TB of storage per node and it's really getting out of hand. I'm evening trying to figure out what uses the least electricity, LC fiber or passive copper lol.
Once I know the hypervisor then I can better plan the nodes. Pretty sure for Hyper-V failover clusters on S2D it requires 4 nodes minimum. Kubernetes is 3 nodes but can't do live migrations or anything cool. That usually means doing crazy stuff in software to make stateful vm's to work correctly. Not sure about the others because I never touched them. I tried ESXi but always had trouble getting the trial license and stuff to work. Windows Hyper-V has been the easiest so far with the most features so it's what I would like to use.
We are "off grid" power for weeks at a time in winter for over the last 10 years, and I run my entire house on Generator during that time including all networking, cameras, office PC + house stuff (fridges and freezers) and we use around 2G of gas per-day. There's absolutely no way you can do this on diesel it isn't affordable, while a diesel generator will last much longer than a gas you're going to spend way way more than replacing that gas generator on diesel operating cost, for that I would spend the $$ on more solar (and I have). I have 1000s of hours on my original Honda EU2000i and it starts up first pull every time, my second one starts up on the second pull. Solar is where the $$ should go not a $4000 or $8000 diesel generator IMO get a used\deal gas honda 3000i or 6500 if you need 220. If $ was no object then yes, I would have a top of the line diesel backup and 5000G tank don't get me wrong, but at this stage I would (and am) starting to build my solar system. I've run my Honda Eu2000i for 100hrs straight only stopping to put fuel in to keep animals alive in snow storm providing them heat, I really am impressed what these can do and how they hold up.
Will definitely need an emergency generator for those severe storm events. I was originally thinking an RV generator. Something like
this is the standard. They typically run on LP/NG and I was thinking of getting one of those huge tanks installed. Not sure about how fuel-efficient these things are compared to gas/diesel etc but it will all need to be looked at in time. Also where I buy land will also play a huge role in what the energy requirements for the property will be and how much naturally can be harnessed and what kinda backup power will be used. At least with a huge LP tank I could get a gas stove and heat the house in an emergency.
I won't unfortunately be at the house to start the generator, so I'm looking for something with a little bit of brains that I can wire a microcontroller into and control myself. That way I can say if batteries is less than 30% run the generator every other hour until the batteries are above 80%. WIll eventually cover all the power stuff in this thread or separately but I do appreciate all the feedback.
Thanks