Proxmox and DRBD

[SNayFoo]

I played with DRBD, (and other software) to make a shared-nothing cluster using two Intel Modular Servers a really long time ago.

I just recently read that DRBD has plugins for Proxmox, and I wanted a new project for fun at home.

So, I was planning on building two proxmox boxes using DRBD to mirror the storage between them.

Is this something folks would be interested in this?
I have a standalone proxmox box and two hyper-v boxes I'll move VMs from to this project once done.

My hardware:
Servers:
2x Intel Xeon E5-2680 v3 12C/24T 2.2GHz-3.5GHz
128GB RAM (for now)
2x 480GB Datacenter SSD ZFS mirrored for boot
10x 6TB SAS HDD in 5x ZFS Mirrors
2x 1TB Datacenter NVME drives in ZFS Mirror
2x 10GB Intel SFP+ NICs (1 for replication, 1 for data bridge for VMs)
2x 1GB Intel NICs (1 for MGMT, 1 maybe for extra heartbeat)
Intel Arc A310 Graphics card (for jellyfin decoding)

Networking:
2x Microtik crs309-1g-8s+inv(1 for replication/storage traffic, 1 for Data)
1 8x 2.5GB, 2X SFP+ dumb switch (to connect the data microtik to LAN)

There are other things and servers on the network, hence why the two microtiks.

If no interest, I'm still excited. If interested, I'll try to note progress and anything of interest.

In anyrate, thanks for the forums and site folks, been reading for over a decade, but never made an account, until I wanted to buy some drives.

--SNay

 

[SNayFoo]

So, to apologise for a lack of a reply. This is still going. I had a few reality checks of stupid things.

First, I need a witness/quorom device. (Unsure how I forgot this, not my first 2-server cluster). I have a third mostly identical box, so this isn't an issue. In many ways, it is beneficial, since the third box has all the VM's I want to move anyway.

Next, I somehow forgot fundamental math, and ordered the same number of mini-GBICs for the fibre runs I'd need. Um, duh, you need a mini-GBIC on both sides.

Also, I figured we should have a few LC-LC patch cables in our misc fibre bin at my office. Nope, LC to SC or ST, sure. But no LC-LC.

Also, all the equipment was scattered around my house. So, I used this time to clean up, move everything into the (literally empty and occupying floor space in my basement) 42U rack I have, move everything to the work area and cable and wire everything (semi?) intelligently.

I'm awaiting 8 more mini-GBICs and one more LC-LC cable. I need to bring the existing Proxmox box into version 9.0.

This should all happen relatively soon, and I'll write out what I do as I do it. I'd rather write out and own my "duh" moments, like lacking a witness/quorom box on a 2-node cluster.

Also, while this *might* have professional implications down the line (read: I might be able to sell the knowledge I get from this), I"m not doing this for any real reason other than this: It seems like fun. I've played with all of the technologies involved in one form or another. Also, there's no kill like overkill for storage, and I want to do this for S&G purposes.

Be well all!
--SNay

 

[Greg_E]

Do you NEED a third full host or can you just make it a witness node? It would only serve to vote in quorum so maybe three nodes is not the best case for this, but a question the same. I think you need at least 4 nodes in Harvester to be able to have a witness node that can vote, but I'd need to go back and read that again. General practice they suggest building in odd numbers of nodes to meet quorum, 3 minimum, then 5.

 

[SNayFoo]

From everything I've read it really just needs to be a witness, but I happened to have the third box anyway that has the VM's on it I want to move, so handy enough, but I could have just put a small box to do the same.

--SNay