Design for a cloud service?

[Johan45]

Hi guys and gals!

So I have been thinking on removing my main storage device so it doesn't is exposed to the internet. And in the making of this, I thought to set up a small service for my family (they have been asking for a something like dropbox). I have been looking for something small and power efficient like a 1U atom server, that seems to be able to run a nextcloud and have ha few 100 users (redudency if someone else want storage). But how about the storage... I need something bigger then the 1U server to host drives, right? 500GB of SSD won't get me far. I dont want Amazon or google, I want to own the data and the network myself. So a 1U server, managing the services and the users, and another server to host the disks?

And that connection would then be on a VLAN, not exposing the fileserver directly to internet. The best scenario would be to run something similar to digital ocean, every users has their own container/droplet whatever, and then a quota on the fileserver.

Is their any solutions for this already out their? Or how should i proceed?

 

[EffrafaxOfWug]

Assuming when you say "cloud service which I control but not exposed directly to the internet" I assume you actually mean "my own server sitting on a network that myself and other people can access remotely through a VPN"?

Before you start thinking about the server, probably the most important part is governing how you're going to manage ingress - you might want to use whatever VPN solutions your router might provide, or think about deploying a pfsense box, or a new router like a mikrotik or ubiquiti - and ideally this'll be in place and working before you do a server build. Personally I have a pfsense box that's accessible via an IPSec client and that gets you access to one of several internal VLANs (although I don't run anything like owncloud).

A hundred users though...? What sort of network and data capacity are we talking about here? What's your current storage device and networking hardware?

 

[Johan45]

Thansk for your reply! I have been looking into NFS and it seems like an option to have one server managing users, and then mount their home dirs on a second server with NFS. Or have a the same setup with VMs, lower coast. It will not be 100 users, i have friends an family around 50 that i want to give disk space to. 1000/1000 internet connection and if i dont go VM i would go with 10gbe lan, wont be necessary but still have some network redundancy if back ups should be done frequently. I dont think 50 people will hit the server simontauniusly

 

[Blinky 42]

If you are really going to have more than a dozen users and want/need to control what they have access to in terms of files other than a big central pool of files everyone can have either read-only or read/write access to, then I would get a game plan in place for managing the users, quota, what they have access to on shared storage and ideally the VPN connections / VM access all in one shot to keep your life easy.

That choice may influence the rest of your design, so you may want to evaluate the options there.

If you want to go the whole mile and do a full VMs per user type setup, you may want to investigate Openstack etc.

 

[Johan45]

If you are really going to have more than a dozen users and want/need to control what they have access to in terms of files other than a big central pool of files everyone can have either read-only or read/write access to, then I would get a game plan in place for managing the users, quota, what they have access to on shared storage and ideally the VPN connections / VM access all in one shot to keep your life easy.

That choice may influence the rest of your design, so you may want to evaluate the options there.

If you want to go the whole mile and do a full VMs per user type setup, you may want to investigate Openstack etc.

Thanks for your reply!

I figure to have something simple (no pun intended), to run own cloud / next cloud and let more advance users use rsync, scp and a shell. No shared files what so ever. And I don't believe that VPN is what I really was after.

Go one VM/user would be probably bo a really cool solution and keep the data integrity! Like digitalocean, but that seems really hard and to configure that baby would really be a hassle I imagine, really enterprise stuff regarding hardware.

 

[Johan45]

It might be a SAS expander I want? No need for NFS or a second server, am I right?

 

[Blinky 42]

Well if you just want users to have a home directory with storage most any server layout will work.
You don't *need* to split the storage server from the rest, it could be all in one box.
NFS will work fine but with multiple servers if you want to manage the users and sync the IDs between the boxes for permissions and quotas to work properly.

You can also run iSCSI on the 2nd server and export block devices to the front-end box, or an ex pander if you just need more drives than can fit in the front end box.

 

[Johan45]

Well if you just want users to have a home directory with storage most any server layout will work.
You don't *need* to split the storage server from the rest, it could be all in one box.
NFS will work fine but with multiple servers if you want to manage the users and sync the IDs between the boxes for permissions and quotas to work properly.

You can also run iSCSI on the 2nd server and export block devices to the front-end box, or an ex pander if you just need more drives than can fit in the front end box.

Thanks for your reply! I have been watching some youtube videos and it seems to be something called "storage shelf". And that is might the thing I'm looking for.

I would like to run a 1u server, and add storage to that box. And correct me if I'm wrong, storage shelf is a case just containing disk, power supply and backplane. Then I can connect that into the server? This seems to be a nice way to do it, very scalable system. The only problem I can see is the filesystem.