Proxmox homelab advice requested - low power usage, mid performance - Which TinyMiniMicro?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Marsh

Moderator
May 12, 2013
2,642
1,496
113
Few years ago,
I brought a lot of IBM 49Y4242 49Y4241 Intel I340-T4 Quad Port Ethernet for $15 to $20 each.
I am still using it for now. It is not I350 , but it works for homelab.

At this time, I won't buy any more 1gb nic, at lease 2.5gbe nic. Avoid Intel I225v, look for Intel I226 model of 2.5gbe nic

For 10gb NIC been using and buying
Supermicro AOC-STGN-I2S 10 Gigabit Ethernet Adapter
 
  • Like
Reactions: T_Minus

Marsh

Moderator
May 12, 2013
2,642
1,496
113
At this time, I won't buy any more 1gb nic, at lease 2.5gbe nic. Avoid Intel I225v, look for Intel I226 model of 2.5gbe nic
What I mean is , there lots of realtek 2.5gbe cards on Amazon / Ebay, price cheaper than Intel 1gb NIC.
Even USB Realtek 2.5 nic works OK.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
What I mean is , there lots of realtek 2.5gbe cards on Amazon / Ebay, price cheaper than Intel 1gb NIC.
Even USB Realtek 2.5 nic works OK.
Which realtek 2.5 NIC are you using and which are they working on?
Proxmox? Truenas scale?Core?
Windows?

Curious :)
 

bigfellasdad

Member
Apr 10, 2018
74
40
18
67
Which realtek 2.5 NIC are you using and which are they working on?
Proxmox? Truenas scale?Core?
Windows?

Curious :)
My pcie dual 2.5gb is this one. https://www.amazon.co.uk/dp/B07XJ8CMQX?psc=1&ref=ppx_yo2ov_dt_b_product_details
USB adapters, 2 of these https://www.amazon.co.uk/gp/product/B0856V9FDH/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1
For the USB adapters they work in proxmox, openwrt, pfsense and of course windows. I had to disable powertop settings for the usb dongles as they didnt wake up when needed, fine after changing that, I dont have that issue with the pcie card.
 
  • Like
Reactions: denis_tmm and Marsh

bigfellasdad

Member
Apr 10, 2018
74
40
18
67
4 x Dell 7070 I5-9500T , 32gb ram mini cluster + 1 Lenovo M720q ( I5-8500 ) + 8 port switch
Proxmox 7.2 idle
All Dell 7070 mini + M720q + switch = 14-18w
Hi, im really intrigued on how your rig pulls so little power. My qnap QSW-1105-5T alone pulls 6w with only 3 connections, are you reading the power with the dell's sleeping, with WoL enabled?
 

Parallax

Active Member
Nov 8, 2020
417
207
43
London, UK
I looked at that too initially but i couldnt find a low profile bracket for it, would you replace the onboard nic for that?
I have the 2.5GbE m.2 A/E key NIC on order. I think I should be able to punch out one of the rear ports of my M75q-2 and screw it in there. Since there's no riser in these I'm not losing out on anything and I should be able to fit the SSD and caddy still.
 
  • Like
Reactions: bigfellasdad

Marsh

Moderator
May 12, 2013
2,642
1,496
113
Hi, im really intrigued on how your rig pulls so little power. My qnap QSW-1105-5T alone pulls 6w with only 3 connections, are you reading the power with the dell's sleeping, with WoL enabled?

Me too, I agree with you.
I had to get a camera to take couple of pictures , one picture shown 14w, another picture shown 18w.

I do not believed it is true measurement due to the ultra cheap kill-a-watt meter,
I don't think the meter is accurate in the low end range.

May be true power consumption is 5w to 7w more for the whole cluster. (19w to 25w).
 
  • Like
Reactions: bigfellasdad

denis_tmm

New Member
Jan 24, 2023
20
4
3
Thank you for all your NIC suggestions. I'm going through them later on. For now I have now bought the Dell Optiplex 5060 SFF i5-8500 for 170€. Not quite as cheap as in the US but it is OK for me. :) I will get now 2x16GB of RAM and SSD + M2 SSD storage + PCIx 1x2,5G NIC.

Any suggestions for the SSD stuff regarding power usage?
 
  • Like
Reactions: Marsh

denis_tmm

New Member
Jan 24, 2023
20
4
3
Ok one question from the guy that hasn't done hardware upgrades for some years. :)

The "Dell Optiplex 5060 SFF" does have a M.2 slot with Gen3 PCIe x4 NVMe support. (like e.g. the https://de.transcend-info.com/product/internal-ssd/mte110s-112s)

From what I have read are the NVMe SSDs not very power efficient. Is this right? Is it possible to use the M.2 SATA drives (which can be pretty power efficient) in such a NVMe slot?

I wanted to have the Proxmox installation on the M.2 slot and the data for the VMs/Containers on one or two "normal" SATA3 SSDs.

Or would you go another approach in this machine from a power usage standpoint?
 

denis_tmm

New Member
Jan 24, 2023
20
4
3
And I'd like to add one more question. Would you rather go with "standard" ext4 when using only one single SATA3 disk? Or would you create a LVM? Or what would be your approach when just using one single NVMe (WD Red) for the system + one SATA3 SSD (WD Red as well) for the VMs/containers.

Backups will be done via NFS for the moment to my Synology NAS that is already sitting around.
 

bigfellasdad

Member
Apr 10, 2018
74
40
18
67
I have the 2.5GbE m.2 A/E key NIC on order. I think I should be able to punch out one of the rear ports of my M75q-2 and screw it in there. Since there's no riser in these I'm not losing out on anything and I should be able to fit the SSD and caddy still.
If you could let me know your findings on this please.
 

zer0sum

Well-Known Member
Mar 8, 2013
849
473
63
I would aim for a Lenovo m720q as they can be found for as low as ~$80 (without cpu) or ~$130 (i3/i5) off Ebay in the US.
I'm not sure on Europe, but at that price even including shipping to you it would be really cheap :)

It can take 64G ram, 1 x nvme, 1 x sata and has a low profile PCIe slot that you can add a quad 1G or dual 10G network card.
The built in lan port supports vPro so you get poor man IPMI.

I don't think they can be beaten really :D
 

Parallax

Active Member
Nov 8, 2020
417
207
43
London, UK
If you could let me know your findings on this please.
Very straightforward, my biggest problem was the m.2 screw head got stripped trying to get the WiFi card out. After sorting that out I just put in the NIC card, punched out one of the panels in the rear baffle, and the LAN port screwed in to the space nicely. The whole thing looks good and works well.

It's Realtek which some people (and ESXi) don't like, but I couldn't see an Intel alternative on Aliexpress and Intel's 2.5GbE chips haven't been great successes either.
 

Parallax

Active Member
Nov 8, 2020
417
207
43
London, UK
And I'd like to add one more question. Would you rather go with "standard" ext4 when using only one single SATA3 disk? Or would you create a LVM? Or what would be your approach when just using one single NVMe (WD Red) for the system + one SATA3 SSD (WD Red as well) for the VMs/containers.
I do LVMs because there's flexibility to move it to a new drive and grow the partitions pretty easily.
 
  • Like
Reactions: bigfellasdad

bigfellasdad

Member
Apr 10, 2018
74
40
18
67
Very straightforward, my biggest problem was the m.2 screw head got stripped trying to get the WiFi card out. After sorting that out I just put in the NIC card, punched out one of the panels in the rear baffle, and the LAN port screwed in to the space nicely. The whole thing looks good and works well.

It's Realtek which some people (and ESXi) don't like, but I couldn't see an Intel alternative on Aliexpress and Intel's 2.5GbE chips haven't been great successes either.
Thanks very much, i think ill order one or two then. I have had no problems with the usb realtek nics, other than in debian/proxmox using the wrong kernel module, causing half duplex (easy fix). The mPCIe option looks much cleaner though. thanks again
 

denis_tmm

New Member
Jan 24, 2023
20
4
3
@Marsh

Does your Dell 5060 SFF run smoothly at the moment. I have done an update in the last week. And I'm getting these kinds of kernel panics.

Code:
Feb 28 07:44:19 pve kernel: BUG: unable to handle page fault for address: ffffffff8e88a940
Feb 28 07:44:19 pve kernel: #PF: supervisor write access in kernel mode
Feb 28 07:44:19 pve kernel: #PF: error_code(0x0002) - not-present page
Feb 28 07:44:19 pve kernel: PGD 6d8415067 P4D 6d8415067 PUD 6d8416063 PMD 10eb8a063 PTE 800ffff927775062
Feb 28 07:44:19 pve kernel: Oops: 0002 [#1] SMP PTI
Feb 28 07:44:19 pve kernel: CPU: 4 PID: 678197 Comm: z_rd_int Tainted: P           O      5.15.85-1-pve #1
Feb 28 07:44:19 pve kernel: Hardware name: Dell Inc. OptiPlex 5060/0654JC, BIOS 1.7.1 07/02/2020
Feb 28 07:44:19 pve kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x1e1/0x240
Feb 28 07:44:19 pve kernel: Code: 41 89 ce 44 0f b7 e8 41 83 ee 01 49 c1 e5 05 4d 63 f6 49 81 c5 40 19 03 00 49 81 fe ff 1f 00 00 77 49 4e 03 2c f5 e0 0a ec 8d <4d> 89 65 00 41 8b 44 24 08 85 c0 75 0b f3 90 41 8b 44 24 08 85 c0
Feb 28 07:44:19 pve kernel: RSP: 0018:ffffa2552192bc28 EFLAGS: 00010282
Feb 28 07:44:19 pve kernel: RAX: 0000000000000000 RBX: ffff8f41b47bf0c0 RCX: 0000000000000010
Feb 28 07:44:19 pve kernel: RDX: 0000000000140000 RSI: 0000000000140000 RDI: ffff8f41b47bf0c0
Feb 28 07:44:19 pve kernel: RBP: ffffa2552192bc50 R08: 0000000000000001 R09: 9ae16a3b2f90404f
Feb 28 07:44:19 pve kernel: R10: 0000000000000000 R11: ffff8f4155cc2000 R12: ffff8f469c531940
Feb 28 07:44:19 pve kernel: R13: ffffffff8e88a940 R14: 000000000000000f R15: 0000000000140000
Feb 28 07:44:19 pve kernel: FS:  0000000000000000(0000) GS:ffff8f469c500000(0000) knlGS:0000000000000000
Feb 28 07:44:19 pve kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Feb 28 07:44:19 pve kernel: CR2: ffffffff8e88a940 CR3: 00000006d8410004 CR4: 00000000003726e0
Feb 28 07:44:19 pve kernel: Call Trace: (the whole file is attached)
I haven't changed anything. But what I see is something like that in the syslog: Device: /dev/sdb [SAT], SMART Usage Attribute: 190 Drive_Temperature changed from 62 to 63

But when looking at hddtemp they do report me 36 and 38 degrees celcius. (two Intel SSD D3-S4510 480GB 2.5" SATA)

These are the other temperatures:

Code:
coretemp-isa-0000
Adapter: ISA adapter
Package id 0:  +33.0°C  (high = +82.0°C, crit = +100.0°C)
Core 0:        +33.0°C  (high = +82.0°C, crit = +100.0°C)
Core 1:        +30.0°C  (high = +82.0°C, crit = +100.0°C)
Core 2:        +29.0°C  (high = +82.0°C, crit = +100.0°C)
Core 3:        +30.0°C  (high = +82.0°C, crit = +100.0°C)
Core 4:        +30.0°C  (high = +82.0°C, crit = +100.0°C)
Core 5:        +30.0°C  (high = +82.0°C, crit = +100.0°C)

acpitz-acpi-0
Adapter: ACPI interface
temp1:        +27.8°C  (crit = +119.0°C)

pch_cannonlake-virtual-0
Adapter: Virtual device
temp1:        +39.0°C
Perhaps you go an idea what could have happened.
 

Attachments