Proxmox VE "noob" build Ceph question

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
A

awedio

Active Member
Feb 24, 2012
776
225
43
I'm looking to get my feet wet in the Proxmox world...

Chassis: SuperMicro 2027TR-H72RF
CPU: Xeon 2x E5-2620 per node
RAM: 128GB per node
SSD o/s: 2x SuperMicro SSD-DM064-PHI per node
SSD Ceph: 6x Samsung SM863 or Intel S3710 per node
Networking: 1x Mellanox ConnectX-3 dual port 56g + 2x 1gig on board nics per node

1) Install Proxmox on 2x SMicro SATADom in ZFS RAID1

2) What to do for the Ceph storage, how to set this up?
 
Last edited:
MiniKnight

MiniKnight

Well-Known Member
Mar 30, 2012
3,072
973
113
NYC
Nice server.

Do you have 10/40 gig E for networking? If not, I'm going to strongly recommend before doing SSD ceph.

1) you can do that from the installer. ServeTheHome had a guide maybe years ago but I recall seeing it here.

2) Ceph Server - Proxmox VE has everything you need to get Ceph online. Then you can use it like a normal shared data store.

Seriously, 10GbE min for the Ceph network. You can do 1GbE for the clustering and outbound but storage you'll need 10GbE.
 
A

awedio

Active Member
Feb 24, 2012
776
225
43
MiniKnight said:
Nice server.

Do you have 10/40 gig E for networking? If not, I'm going to strongly recommend before doing SSD ceph.

1) you can do that from the installer. ServeTheHome had a guide maybe years ago but I recall seeing it here.

2) Ceph Server - Proxmox VE has everything you need to get Ceph online. Then you can use it like a normal shared data store.

Seriously, 10GbE min for the Ceph network. You can do 1GbE for the clustering and outbound but storage you'll need 10GbE.
Click to expand...
1) I have 40/56 gig E for networking. (edited orig post)

2) There's journal, mons, osd etc. How does one divy that up between the 6x SSDs on each node?
 
MiniKnight

MiniKnight

Well-Known Member
Mar 30, 2012
3,072
973
113
NYC
With all SATA SSD's I'm not sure how much you'd gain with a journal. 4 nodes maybe just have all of them be monitors so if one fails you have 3?
 
MiniKnight

MiniKnight

Well-Known Member
Mar 30, 2012
3,072
973
113
NYC
I don't know how much a SAS SSD would improve or hurt performance over what you have. You're caching 6 SSDs with 1 slightly faster SSD but adding another tier to the storage.

Try without a journal?
 
You must log in or register to reply here.
Top Bottom