Where To start ?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

lost1130

New Member
Dec 8, 2018
10
0
1
None of this will add to the value of your home. Wiring up your home with ports/cables might, but even then it's highly unlikely to do much more than what a coat of paint would.
how would this ad no value to the house full home auto lights kitchen everything will run on 10 gbe lines so lots of room, not a cheap task to do if you want it in a home already done
 

kapone

Well-Known Member
May 23, 2015
1,095
642
113
Ok cool, so I can answer most of that actually,
multiple servers are ending goal starting server will be an epic duel or quad chip 10 -16 core chip to start brand not sure of might make it myself
i have a 3-year window to launch running wires is all right now cat 6a in every room might do 7 if it becomes more available price wise
50tb for server 50tb in nas (but I want the big one that is able to do peta for nas and for server or 150+ drives)
CPU on server 32+ starting server going rack so it open-ended there
streaming music and movies to family and friends connected to my network not only on my home be able to keep all my finances extra safe control my internet connections and home automation and security I have now made host Minecraft up to 10 streams at one time 4k editing
I have all this running at home, although I don't call myself Tony Stark.

- dual Storage heads
- 72 3.5" bays (with 12-14TB drives, that's 1PB of storage)
- 100 cores @3.2GHz and 1TB of RAM spread over 8 servers
- All on a 10G network with the storage at 40g
- Takes up one 42U rack.
- 4x 802.11ac APs spread around the house.

I'll happily ship everything to you at a low low price of ~$250-300K, including the consulting cost of building this. I take PayPal, credit cards, cash, wire transfers. Wanna bite?
 

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
If the "now" goal is 50tb, you could go with a dual Epyc board, Supermicro 24 bay case with expander backplane, and a HBA. Use Proxmox or EXSI w/passthrough for the HBA and a FreeNAS VM. Create VMs for the other services as needed. Your IOPS is going to be a limiting factor with multi streams, using mirrors would help. 5 paired 10TB drives, so 10x10TB for 50TB usable. That leaves plenty of bays for expansion, if you need more speed, you can increase the number of spindles. You can later add an external port HBA for storage expansion boxes. Buying a 150+ disk storage box then not populating it is a waste, but sure, go get one if you want to. Or fill it with smaller drives, for now, and upgrade as needed. If you really want PB level storage later, max out the RAM. You will need it to keep performance up with VMs running and lots of storage.

For keeping your private data safe, you might want to consider keeping it on a dedicated server or perhaps VM on an isolated network/vlan. Perhaps even shutting down the server/VM when not in use. Put the public facing stuff on it's own isolated network, along with a guest wifi if you want to let guests use wifi.

Even with mirrors/RAID, you will have drives die. Make sure you know how to deal with that and have a backup plan in place. The only way I know to really backup that much data is another big HDD array, but it can be lower performance. An enterprise tape robot is another option, though it comes with a similar price tag.
 
  • Like
Reactions: lost1130

lost1130

New Member
Dec 8, 2018
10
0
1
If the "now" goal is 50tb, you could go with a dual Epyc board, Supermicro 24 bay case with expander backplane, and a HBA. Use Proxmox or EXSI w/passthrough for the HBA and a FreeNAS VM. Create VMs for the other services as needed. Your IOPS is going to be a limiting factor with multi streams, using mirrors would help. 5 paired 10TB drives, so 10x10TB for 50TB usable. That leaves plenty of bays for expansion, if you need more speed, you can increase the number of spindles. You can later add an external port HBA for storage expansion boxes. Buying a 150+ disk storage box then not populating it is a waste, but sure, go get one if you want to. Or fill it with smaller drives, for now, and upgrade as needed. If you really want PB level storage later, max out the RAM. You will need it to keep performance up with VMs running and lots of storage.

For keeping your private data safe, you might want to consider keeping it on a dedicated server or perhaps VM on an isolated network/vlan. Perhaps even shutting down the server/VM when not in use. Put the public facing stuff on it's own isolated network, along with a guest wifi if you want to let guests use wifi.

Even with mirrors/RAID, you will have drives die. Make sure you know how to deal with that and have a backup plan in place. The only way I know to really backup that much data is another big HDD array, but it can be lower performance. An enterprise tape robot is another option, though it comes with a similar price tag.
Ok so have an enterprise tape duel tape already have a amd 1950x workstation that I run everything on now but only consumer grade level equipment router and switch what my biggest concern is not the overkill on equipment it's the I didn't get big enough and have to buy again dont mind getting more equipment but replacing it altogether cause I wanted to be cheap
 

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
Ok so have an enterprise tape duel tape already have a amd 1950x workstation that I run everything on now but only consumer grade level equipment router and switch what my biggest concern is not the overkill on equipment it's the I didn't get big enough and have to buy again dont mind getting more equipment but replacing it altogether cause I wanted to be cheap

For the network, a DIY pfSense is hard to beat for the router. If your internet connection is fast and you want full VPN, you might need a fairly fast CPU, but there are options for that. Ideally, this is a dedicated box. An attacker can't break out of a VM sandbox if there isn't one to break. That's not to say there are never security issues with the software, it just presents a smaller attack surface. You could drop this in a 1U in your rack for convenience.

For wifi, Ubiquiti has some nice options, there are others as well. You can even just use consumer wifi gear and put them into AP mode, though you lose the single configuration point. And many won't do VLAN etc..

For the switch, check out the thread in the networking forum here for the ICX6610. https://forums.servethehome.com/index.php?threads/brocade-icx6450-icx6610-etc.21107

That should keep you going for a while. Connect the server to that switch with 10Gbe (or 40), everything else as needed. Use VLANs for segmentation. You are unlikely to outgrow that setup anytime soon. Same with a dual Epyc, lots of CPU there.
 

capn_pineapple

Active Member
Aug 28, 2013
356
80
28
how would this ad no value to the house full home auto lights kitchen everything will run on 10 gbe lines so lots of room, not a cheap task to do if you want it in a home already done
I'm not trying to annoy you btw, just being realistic regarding the average punter looking to buy a house.
What you're doing is putting in a very high-end, very niche networking solution which for the vast vast majority of people is not something they will use, have the knowledge of what it does, or have the equipment to take advantage of it.

Unless you know how to specifically sell your house to someone working in the IT industry, this is mostly wasted effort which would be better spent on stone bench-tops, solar panels with a battery bank, or updating the joinery around the house. Smart-home stuff may add some value to the property, but you'll need to go to an industrial installer to make it "just work" without having to troubleshoot it all the time, as well as getting a support contract for the place.

If you're going to spend $50,000-$100,000 on server/networking systems to run this infrastructure and sell that with the house, you're not going to get a return on your investment of the same value, IT equipment is too expensive for most people just trying to purchase either a primary residence or an investment to rent out, most people don't understand how/why computers are so expensive, just get the IT person to make it work, not to mention, that the real estate agents selling the properties will be thoroughly confused as to how to market it, they have enough trouble with the ipads they use for their emails.
 
  • Like
Reactions: Blinky 42

WANg

Well-Known Member
Jun 10, 2018
1,302
967
113
46
New York, NY
If you're going to spend $50,000-$100,000 on server/networking systems to run this infrastructure and sell that with the house, you're not going to get a return on your investment of the same value, IT equipment is too expensive for most people just trying to purchase either a primary residence or an investment to rent out, most people don't understand how/why computers are so expensive, just get the IT person to make it work, not to mention, that the real estate agents selling the properties will be thoroughly confused as to how to market it, they have enough trouble with the ipads they use for their emails.
Why would you turn your house into a data center and expect the next owner to pay for it? If the buyer had the technical knowhow they will want something more modern and all the home automation will be ripped out and replaced with whatever they want, anyways - by the time you manage to sell the place all those stuff would be obsolete. If I am the potential buyer I would negotiate the price down simply because of all this extra cable and silicon junk I'll have to pay the contractor to rip out and dispose of.

Just ask me about all those CAT5 wiring in Manhattan condos that NO ONE uses, or all those idiotic X10 devices that were embedded within from the 80s and 90s bringing no benefits to stakeholders whatsoever. Me and the missus went to an apartment showing 18 months ago where the seller was showing off an X10 based setup that can control his bedroom lighting...from the comfort and convenience of his Commodore 64. The guy was real cool, mind you (I am a bit of a Commadore fanboy). but I am ripping those freaking things off the wall if I ended up buying the place, never mind the 1980s woodgrain controller vibe. The whole thing was clumsy looking and totally not "me".
 
Last edited:

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
I do agree that selling gear like that with a home is not likely to be a good investment. People that want that sort of thing are likely to be particular about the gear and want it installed to their specs, and will want support. Even solar panels and such don't often add their cost unless you get a buyer that knows the details, and even then they would be smart enough to play dumb and try to get a deal. If you are trying to flip properties, you are better off to take marketing lessons from Apple than IBM. IBM might have better gear, but Apple knows how to sell it. It's about looks, not necessarily ability. If you're goal is to make money selling later, put the money into making the house pretty, not techy. If you know some realtors, have a chat with them about it.

If you're doing it because you want to and you enjoy doing it, by all means. But I wouldn't call it an investment.
 

capn_pineapple

Active Member
Aug 28, 2013
356
80
28
Boiling everything down. This is all a financial liability, not an investment. It doesn't provide an ROI, nor will it ever.
This stuff requires maintenance, tweaking, balancing, human intervention, a working understanding of relatively high end virtualisation and networking infrastructure.

Having the house cabled up is fine, but you only need a few things hard wired and that's just for things like WAP's (realistically you might need 3 of), the rest should be networked within a single rack.

A smart-home installer setting up a system like what you've asked for is going to set you back $10k-$20k at a minimum, if you 100% want to go down that route. The support contract is also going to cost you another $1k-$2k per annum. All of which costs an arm and a leg more than what I can setup for $1k-$2k, and "Alexa/Hey Google turn the kitchen lights on/off/green".

If it were me... The server/networking infrastructure would be mine, not for resale with the house, it would be done as cheap as possible and sized for what I need at the time, not specced out for 5-10 years down the track when all the kit I buy today is completely out of date. So that's down to perhaps $12k in expenses total, storage/compute/network/smarthome, all brand new but it goes with me. That $12k is a far more palatable solution instead of your $100k+ solution that an investor doesn't understand and therefore doesn't want to pay for.
 

lost1130

New Member
Dec 8, 2018
10
0
1
Why would you turn your house into a data center and expect the next owner to pay for it? If the buyer had the technical knowhow they will want something more modern and all the home automation will be ripped out and replaced with whatever they want, anyways - by the time you manage to sell the place all those stuff would be obsolete. If I am the potential buyer I would negotiate the price down simply because of all this extra cable and silicon junk I'll have to pay the contractor to rip out and dispose of.

Just ask me about all those CAT5 wiring in Manhattan condos that NO ONE uses, or all those idiotic X10 devices that were embedded within from the 80s and 90s bringing no benefits to stakeholders whatsoever. Me and the missus went to an apartment showing 18 months ago where the seller was showing off an X10 based setup that can control his bedroom lighting...from the comfort and convenience of his Commodore 64. The guy was real cool, mind you (I am a bit of a Commadore fanboy). but I am ripping those freaking things off the wall if I ended up buying the place, never mind the 1980s woodgrain controller vibe. The whole thing was clumsy looking and totally not "me".
I really don't think 10 gbe will be needing to be replaced in a home anytime soon and doing it for my self but it will add value to the home I'm building this home for me and my family don't have plans of selling it
 

capn_pineapple

Active Member
Aug 28, 2013
356
80
28
I really don't think 10 gbe will be needing to be replaced in a home anytime soon and doing it for my self but it will add value to the home I'm building this home for me and my family don't have plans of selling it
Ok, that's a different kettle of fish. You'll be able to get by with either a single or dual Epyc 7371 machine, it will easily have the Cores and frequency you'll be looking for. Plus plenty of PCIe lanes for Storage & networking cards.
 
  • Like
Reactions: lost1130

WANg

Well-Known Member
Jun 10, 2018
1,302
967
113
46
New York, NY
Having the house cabled up is fine, but you only need a few things hard wired and that's just for things like WAP's (realistically you might need 3 of), the rest should be networked within a single rack.

A smart-home installer setting up a system like what you've asked for is going to set you back $10k-$20k at a minimum, if you 100% want to go down that route. The support contract is also going to cost you another $1k-$2k per annum. All of which costs an arm and a leg more than what I can setup for $1k-$2k, and "Alexa/Hey Google turn the kitchen lights on/off/green".

If it were me... The server/networking infrastructure would be mine, not for resale with the house, it would be done as cheap as possible and sized for what I need at the time, not specced out for 5-10 years down the track when all the kit I buy today is completely out of date. So that's down to perhaps $12k in expenses total, storage/compute/network/smarthome, all brand new but it goes with me. That $12k is a far more palatable solution instead of your $100k+ solution that an investor doesn't understand and therefore doesn't want to pay for.
Oh, even wiring up the house with copper CAT-whatever in the conventional sense is not entirely worth it. Seriously, now. 10GbE copper? Some of us are already on 40Gbit/100Gbit networks at home with fiber, media converters and POE injectors (if needed). To a true nerd it should be drop ceilings, removable conduits, and the media can/should always be easily swapped out down the line. If I have to knock things down to re-cable it, then it's simply not going in, it's just extra crap to rip off the wall later.

As for the rack? Buy a good one like an APC netshelter, and make sure you have a good circuit in the house backing it up. If you are running a stack of machines off a 13 Amp house circuit, well, that's just dumb. Actually, scratch that. Your house is not, and should not ever be, a data center for hosting a business. A typical house do not have multiple power feeds, stand-by power, multiple carrier tenancy, disaster recovery planning, physical security, insurance provisions and liability to cover for extreme events or intrusions. If you do, you are overpaying for all of it. Even if you run a business out of your house the most you can ask for is ADP alarm monitoring and a pair of redundant fiber (or MetroE) to your cage in the data center. Your house can be the development node or the place where you keep the backup library, but "going live" from there is just a very bad idea.

Also, considering the core count wars between Intel and AMD brewing up, whatever recommendations anyone can push out here will be obsolete by the time you actually get around to buying it. The same goes for networking and storage - and considering the cringey call I had with Synology 3 weeks back regarding a NAS we had (they do not offer paid support contracts - it's more of a best-efforts incident response), they are NOT what I would consider to be viable enterprise storage.
 
Last edited:

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
Oh, I don't know... I'm still happy I ran CAT5e and RG6 to every room, I use a fair bit of it too. Sure, I can't run 10GBe on it, well, maybe really short runs, but we know how that goes.. There are still a LOT of things that don't really use 1Gbe, some even connect at 100Mb. I do wish I had dropped fiber in for my office, but that's more because I like to play with it than that I really need it for anything I'm doing. When I did it >10 years back, fiber and CAT6 were stupid expensive. If I were building now, I'd use CAT6 or CAT7. And probably some fiber drops along with places like my office running open conduit. Anything you put in WILL be obsolete eventually, but that doesn't mean it's not useful for some things. I mean, a phone run isn't going to need even 100Mb if it's even digital. Currently, my wiring has analog phone, 100Mbe, 1Gbe, audio, and RS-485 running on it. I've run IR repeaters, HDMI, and a few other things on it using converters.

For power to the rack, do make sure to run a dedicated circuit. Or three. I specced one, but the idiot installing decided I didn't need it and I didn't catch it in time. I ended up running a new one later, but it was in the same room with the breaker box, super easy if somewhat tedious to run. If I have to add more, I'll run conduit so I can expand easily later, but I don't think it will be needed anytime soon.
 

capn_pineapple

Active Member
Aug 28, 2013
356
80
28
Oh, even wiring up the house with copper CAT-whatever in the conventional sense is not entirely worth it. Seriously, now. 10GbE copper?
I'd be very happy to have a single or double drop in every non-wetzone room. As it is, I'm piping audio/video around my home on wifi / powerline (renting) which isn't cutting it for my personal Plex setup with housemates in their own rooms without being transcoded down (though I am running 4k endpoints).

Cheap as chips Cat5e is certainly capable of allowing wirespeed 10GbE in the home (up to 50m) with 6a being not hugely more expensive per roll. Even if you don't run ethernet over it, thats at least 4 twisted pairs for other uses (ELV sensors / phones etc)

Ideally just having a few pre-roped conduit and brush covers would be fantastic and provide flexibility in rooms to run other things like coax, HDMI, or Audio cables.

Would it make a difference at re-sale? No. Would it make my life easier? Most certainly.
I will caveat that though:
For a new build it makes sense to do all this stuff because you can plan for it and spec it into the build cost at a much more palatable price point, where you hit issues is with existing and especially older buildings.

I would say the exact same thing when it comes to other home based infrastructure, such as solar panels / battery systems, ceiling fans / HVAC, a pool
 

Blinky 42

Active Member
Aug 6, 2015
615
232
43
48
PA, USA
If starting from scratch with building a house or had $ and time to burn running cables before moving in:
I would only run copper ethernet to:
* Points in the house that make sense to have POE devices. AP's, Phones, Cameras etc.
* Where you might put TVs etc (with Coax and Ethernet)
Those are more probable to be used long term as-is. Do home-runs to some central point that you can put a switch next to the patch panel and cable it up nice. If you have a huge estate or multiple buildings at the new place (nice!) then possibly do home runs to a little patch panel+ switch in the top of closet in each logical area.

Between the switch(es) + areas where you would put servers, run fiber - even better conduit so you can pull more/diff fiber down the road. You will outgrow 10Gb networking if you are talking about 100s of TB of storage. Unless you have the equipment today don't spend money on new 10G stuff for anything that isn't multi-gig POE to run new Wifi AP's.

The other important point to consider if starting from scratch is put proper power and cooling in if you have that level of design freedom. If you build a mini data center somewhere in your place, then the impact on the rest of the home can be minimized (nose & heat). And when you do sell your place, it is just one small area that potential buyers will look at and say "Look a soundpoof room the kids can play in that has cameras already... hmmmm.. "

But more realistically, ALL of the investment you do for custom cables, power, servers etc is just to suit your own needs. Don't expect that any of it will add value to the home unless you are going to sell it in < 5 years. What you deploy will be so out of date by the time you sell, as others have detailed above. Spending extra funds on a generator or solar system that can help offset or protect your computer empire + add value to the home down the line is a much more probable ROI vs. trying to unload old hardware to a future buyer who will just need to pay more to scrap it.

And at some level of scale it is cheaper to put massive infrastructure in a colo vs build out in your home. Easier to expand to larger space without re-arranging your home and annoying the family, and if you need to far easier to shrink or drop it totally and your home is not impacted at all.

I would also suggest going somewhat slow if there are so many unknowns still - build out a server for whatever you want to get going first. When you figure that out and want to grow - sell it on ebay, and upgrade to something that meets your needs better as you evolve. Your interests will ebb and flow over time as your available time to put into it all.
 
  • Like
Reactions: T_Minus

WANg

Well-Known Member
Jun 10, 2018
1,302
967
113
46
New York, NY
If starting from scratch with building a house or had $ and time to burn running cables before moving in:
I would only run copper ethernet to:
* Points in the house that make sense to have POE devices. AP's, Phones, Cameras etc.
* Where you might put TVs etc (with Coax and Ethernet)
Those are more probable to be used long term as-is. Do home-runs to some central point that you can put a switch next to the patch panel and cable it up nice. If you have a huge estate or multiple buildings at the new place (nice!) then possibly do home runs to a little patch panel+ switch in the top of closet in each logical area.

Between the switch(es) + areas where you would put servers, run fiber - even better conduit so you can pull more/diff fiber down the road. You will outgrow 10Gb networking if you are talking about 100s of TB of storage. Unless you have the equipment today don't spend money on new 10G stuff for anything that isn't multi-gig POE to run new Wifi AP's.

The other important point to consider if starting from scratch is put proper power and cooling in if you have that level of design freedom. If you build a mini data center somewhere in your place, then the impact on the rest of the home can be minimized (nose & heat). And when you do sell your place, it is just one small area that potential buyers will look at and say "Look a soundpoof room the kids can play in that has cameras already... hmmmm.. "

But more realistically, ALL of the investment you do for custom cables, power, servers etc is just to suit your own needs. Don't expect that any of it will add value to the home unless you are going to sell it in < 5 years. What you deploy will be so out of date by the time you sell, as others have detailed above. Spending extra funds on a generator or solar system that can help offset or protect your computer empire + add value to the home down the line is a much more probable ROI vs. trying to unload old hardware to a future buyer who will just need to pay more to scrap it.

And at some level of scale it is cheaper to put massive infrastructure in a colo vs build out in your home. Easier to expand to larger space without re-arranging your home and annoying the family, and if you need to far easier to shrink or drop it totally and your home is not impacted at all.

I would also suggest going somewhat slow if there are so many unknowns still - build out a server for whatever you want to get going first. When you figure that out and want to grow - sell it on ebay, and upgrade to something that meets your needs better as you evolve. Your interests will ebb and flow over time as your available time to put into it all.
If you are really planning to be the next Tony Stark (whatever the hell that means), CAT7 is a dead end. Serious 10Gb networking uses fiber (no one trusts pre-existing copper, pulling out existing CAT5/6 to replace it with CAT7 is expensive, and rolling it out now doesn't give you any more margin for growth, since you can't do 40/100 on them), and intra-rack it's mostly DAC cables anyways. Unless you are green-fielding a rollout, It's another reason to just pretend that you are required to go fiber all the way, and put in conduits instead. As for the entire POE thing, standard POE is, what, 13 or 26 watts max, with 55 and 95 coming "eventually"? So yeah, throwing tech money as a home improvement investment? It's absurd.

I should mention that one of my buddies just bought a new-build place in the Durham research triangle - place came pre-wired for CAT5e, and the second he showed me how the constructors terminated the entire thing, I just had to laugh. They wired it into a "smart block" that acts as a hub. Yeah, a hub. I had to clue him into pulling the twisted pairs out of the block, crimping them into standard jacks, and them run them into a switch. Of course, this being a bunch of builders they put the smart block inside of a walk-in closet, which has no ventilation whatsoever. At the end of the day I think he gave up on running much of his stuff through the setup and got some Eero 802.11ac access points.
 
Last edited:
  • Like
Reactions: SwanRonson

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
Well, yes, if you run the lines to a crap location or have them installed by idiots, bad things happen. I tried to help a guy who had an electrician run network wires in his new home. They were daisy chained CAT5 like the old phone wiring. He was not pleased when I explained that the wiring setup they installed was not usable as an ethernet network. This was made worse by the fact that it happened before wifi was really a thing. We managed to get a few rooms working with it, and ran new cable through some unfinished areas, but it was a painful experience. Enough that I wired my own home while under construction to ensure it was done properly. I knew it would be outdated eventually, but I'll still take a switched 1gb CAT5 over even the latest wifi. And really high speed stuff is going to live in the rack anyway.

Hindsight being what it is, I do wish I had put some conduit in the areas not accessible from the attic, but it's good enough to be very useful as it is. Knowing me, I would do that and find I really want fiber to a place right between two conduits with no other access. Just how it goes.

I'd have seriously considered a drop panel ceiling, but that would never pass muster with the wife.