Proxmox VE 5 with Intel Atom C3000 Series Denverton

Piers DD

New Member
Aug 29, 2017
7
0
1
45
Any idea how Ceph might perform on the Atom C3000?
3-node PVE with, say a pair of OSD SSDs each.

Piers
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,906
4,868
113
Probably C3758 or C3858.
The Supermicro 5019A-FTN4 and E300-9A look like they could make interesting edge converged cluster nodes.
I am sketching next-gen STH hosting cluster architecture and the plan is Atom C3000 series for the Ceph cluster nodes.
 

PigLover

Moderator
Jan 26, 2011
2,964
1,271
113
I am sketching next-gen STH hosting cluster architecture and the plan is Atom C3000 series for the Ceph cluster nodes.
I've thought about that too. One issue for me remains SFP+. As of now, the only SM Denverton board with SFP+ is the A2SDi-H-TP4F. The others with 10Gig are Base-T, and that requires more of an infrastructure change. This is a really nice board, but 16 cores (and the $$$) are overkill for a Ceph node.

I really do wish they had at least a couple of lower core-count boards with SFP+. I guess you can always shove Mellanox cards onto the other ones, but that seems to defeat a big part of the goal with this platf0rm.
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,906
4,868
113
I've thought about that too. One issue for me remains SFP+. As of now, the only SM Denverton board with SFP+ is the A2SDi-H-TP4F. The others with 10Gig are Base-T, and that requires more of an infrastructure change. This is a really nice board, but 16 cores (and the $$$) are overkill for a Ceph node.

I really do wish they had at least a couple of lower core-count boards with SFP+. I guess you can always shove Mellanox cards onto the other ones, but that seems to defeat a big part of the goal with this platf0rm.
I agree. The Gigabyte board we have is SFP+. The Tyan is RJ-45. Both are 16C as well.

I think from a product positioning standpoint, if you see the C3000 series as an edge device, then you do base-T instead of SFP+
 

PigLover

Moderator
Jan 26, 2011
2,964
1,271
113
I agree. The Gigabyte board we have is SFP+. The Tyan is RJ-45. Both are 16C as well.

I think from a product positioning standpoint, if you see the C3000 series as an edge device, then you do base-T instead of SFP+
I know a pretty big organization with edge compute plans at somewhere between 1,000 and 60,000 locations (depending on how you define "edge"). This company won't deploy 10gbase-T. :)
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,906
4,868
113
I know a pretty big organization with edge compute plans at somewhere between 1,000 and 60,000 locations (depending on how you define "edge"). This company won't deploy 10gbase-T. :)
Okay, you got me. I think a lot of these chips and boards end up as networking or NVR boxes or similar at smaller shops.

I do prefer SFP+ since if you are going for low power, the SFP+ switches and NICs are still more efficient. Even in a small cluster it only takes a few SFP+ v. 10Gbase-T links to add another Denverton in the same power envelope.
 

Piers DD

New Member
Aug 29, 2017
7
0
1
45
Hey Patrick,
Have you progressed any further with your C3000 converged cluster ideas?
Piers
 

Piers DD

New Member
Aug 29, 2017
7
0
1
45
Hi Patrick,
Any thoughts as to, Atom C3858 vs Xeon-D 2146NT, for the following small/fast cluster?

4x nodes, PVE 5.3, ea with
- 4x SATA SSD OSDs for RBD & CephFS
- 2x 10GBe, 2x 1GBe
Fairly light compute workload
Redundant corosync networks to mitigate split brain potential w/ even number of nodes.

Does the Atom have the headroom in your opinion for a 16x (or 20x) SATA SSD OSD cluster?

TIA,

Piers
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,906
4,868
113
@Piers DD I would probably go D-2146NT if it is in the budget and not much more. Single threaded performance helps and when OSDs fail, you will like that feature. You also get the option to run more NVMe with Xeon D.
 
  • Like
Reactions: T_Minus and Evan

Evan

Well-Known Member
Jan 6, 2016
3,065
512
113
Dont know the ceph specific answer but the d-2146 would be double the c3858 performance, less real cores but more threads and full cores not light cores and with that a 50% in max clock speed thrown in.

Now cost is a different topic, the Xeon-D costs more for sure.