Lenovo Thinkcentre/ThinkStation Tiny (Project TinyMiniMicro) Reference Thread

[AlexGee]

nice, wonder if you have the heat issue. And what are you installing on that lovely device, @alexge

My new Proxmox Hypervisor
M720q with 9500T
2x 32GB RAM 2666
Proxmox SSD: either 850 Pro SATA or M.2 2230
4TB Lexar NM790 for data
NIC: either X520-DA2 10GBit SFP+ (existing), or X550-T2 (searching: one of the few which should be able to handle also 2.5GBit)

proxmox:

• OPNSense
• PiHole
• Cloudflared
• Omada Controller
• Home Assistant
• TrueNAS Scale (to use 4TB NVME as NFS storage for Nextcloud & Kavita)
• Nextcloud
• PhotoPrism (pictures stored on my Fileserver)
• Jellyfin (Movies, Music stored on my Fileserver)
• Kavita
• Paperless-ngx

 

[truongnht]

@AlexGee Nice setup, did you manage to replace Google Photo with PhotoPrism?

 

[shtirlic]

Hi, here's some backstory:

I purchased the m720q some time ago, and it was working great until October when I decided to buy the Inspur X540-T2 PCI-E 10G.

Notice the unusual pads at the end? Without any clue that these pads are the actual x1 interface with a 12v rail, I inserted it. After powering on, it destroyed my motherboard/PCH or perhaps just some capacitors. Later, I purchased a simple CPU (Pentium) to test, but there was no success, so I ordered the 920q motherboard as a replacement.

The question is: Are there any clues as to which components (capacitors, voltage regulator) were damaged by the 12v rail?

I have a simple voltage tester and a cheap Saleae-compatible logic analyzer.

PS
My riser is and I was thinking that after the x8 part everything is not actually connected to the rails, but actually it's wrong

x16 (electrical x8)

01AJ940

 

[shtirlic]

Some tips if someone wants to passthrough the integrated video card to the Proxmox VM for video output on m720q/m920q:

kernel cmdline

BOOT_IMAGE=/boot/vmlinuz-6.5.11-7-pve root=/dev/mapper/pve-root ro quiet audit=0 textonly initcall_blacklist=sysfb_init hpet=disable intel_iommu=on iommu=pt nvme_core.default_ps_max_latency_us=0

cat /etc/modprobe.d/blacklist.conf

install i915 /bin/false
blacklist i915

cat /etc/modprobe.d/iommu.conf

options vfio_iommu_type1 allow_unsafe_interrupts=1

libreELEC VM settings

 

[joeribl]

Hi, here's some backstory:

I purchased the m720q some time ago, and it was working great until October when I decided to buy the Inspur X540-T2 PCI-E 10G.

Notice the unusual pads at the end? Without any clue that these pads are the actual x1 interface with a 12v rail, I inserted it. After powering on, it destroyed my motherboard/PCH or perhaps just some capacitors. Later, I purchased a simple CPU (Pentium) to test, but there was no success, so I ordered the 920q motherboard as a replacement.

The question is: Are there any clues as to which components (capacitors, voltage regulator) were damaged by the 12v rail?

I have a simple voltage tester and a cheap Saleae-compatible logic analyzer.

PS
My riser is and I was thinking that after the x8 part everything is not actually connected to the rails, but actually it's wrong

x16 (electrical x8)

01AJ940

View attachment 33817

Some sellers sell this card, with the last two pads physically cut off. You can also find people in Google who tape the pads to get it working.

 

[shtirlic]

Some sellers sell this card, with the last two pads physically cut off. You can also find people in Google who tape the pads to get it working.

Yeah I saw this, but went ahead without cutting/taping because I was assuming that the riser pads after x8 are not connected to anything.

 

[Jakob]

The question is: Are there any clues as to which components (capacitors, voltage regulator) were damaged by the 12v rail?

I have a simple voltage tester and a cheap Saleae-compatible logic analyzer.

The schematic and boardview of the M920q/M720q are floating around the web (Can you share boardview and schematics? · Issue #3 · badger707/m920q-dual-NVME here for example).
A good place to start would be to measure all the supply rails connected to the PCIe connector (at least 12V, 3V3, 3V3_Aux).
With the system turned off you should see 3.3V on the 3V3_Aux rail and the others off.
When pressing the power button you should see the other two starting as well.

The schematic also contains a power sequencing diagram (sadly with potato quality) which can help you figure out where to start if both other rails are missing (the next will probably not turn on if the one before it did not start correctly in the sequence. At least with my back-feeding ConnectX-4 it behaved that way).

 

[darrend23]

Hi all, a lot of useful information in this thread.
I’ve just bought 3 of the below, my original plan was a proxmox cluster but now I’m thinking about using 1 as an unraid nas as I like how easy u raid is to use, just a shame I now have 3 and may only need the 1, what the most amount of storage anyone has managed to fit in one of these?



3x m720q - i5-8400 - 16gb ram - 240gb ssd - empty nvme slot
I found them at £100 each, hence why I bought all 3

Thanks Daz

 

[SyntacticSugarDev]

Newbie to this forum. Been watching this thread ever since I got my hands on a m720q to build a OPNSense box. Looking for a 4-port 2.5 gbe intel nic for the m720q. So far i've found the Qnap 4 port 2.5 nic. But looking for other options as well. Came across a bunch of these IOCrest 4 port nics that supposedly have i225 chips.



Looks like they are specifically being advertised to work in the lenovo m-series minis. I haven't been able to find much info or reviews on these nics anywhere else. Anyone here have any specific experience with these nics in their routers?

 

[SyntacticSugarDev]

So I have finally setup a m720q tiny to replace an old R210ii, with my ATT fiber line.

I wanted to share this info for anyone who wants to move to a m720q tiny who has 2.5/5/10gb AT&T fiber line.

I have AT&T 2.5gb fiber.

The ATT BWG320 is the fiber modem I have to use, which has a 5GB rj45 port.

I tried to use the SuperMicro AOC-STGN-i2s NIC by I cannot get it to negotiate a 5GB connection.

pfsense recognizes the NIC, but I just cannot get a connection between it and the modem.

i have tried multiple transceivers, and still cannot get it to work.

I Assume the problem is with the x520 chipset negotiating a 5GB connection, with a transceiver - if anyone knows one that works please let me know.

i then tried to fit in my x710-T2L - which is rj45 10/5/2.5 GB.

As I pulled it out I forgot what a LONG boy this is.

so I was initially crest fallen, assuming it would not fit..

upon trying to jam it in the m720q tiny, the only thing blocking it was this thin piece of metal at the front of the chassis, which luckily is secured by one tiny screw, and popped out very easily!

This piece of metal I removed:

https://cdn.discordapp.com/attachments/1164621435784925299/1193375069586587698/PXL_20240107_020519763.jpg?ex=65ac7c24&is=659a0724&hm=516c90a375fffe95ef23d1f0e2fc95380c2caabeb1db323f7944edd9c4ef18b4&

After removing this, the i710-t2l FIT like a dream.

I don't have a rear plastic baffle as it appears nobody has done this, but no big deal as the card is so long it rests on the back of the far side front USB ports. It's literally a perfect fit. It's rock solid, doesn't wobble at all, and the case still closes.

Here's a pic of the huge NIC installed:

https://media.discordapp.net/attachments/1164621435784925299/1193374164468371506/PXL_20240107_0152368462.jpg?ex=65ac7b4c&is=659a064c&hm=dced1c3193b0ef358e07ce43d943730e09b51ebff401697380932a9c3b66c40e&

The case closes up, and it works like a dream!

Bonus over the Super micro AOC-STGN-i2s NIC, is the X710-t2l runs a lot cooler, and has a heatsink twice the size.

Side question, in the above image I highlighted what I think is an antenna in purple. Is it safe to remove that?

It was resting in the metal piece I removed, and is just kinda loose now. It appears to be secured under the WiFi card with a single screw.

Anyway, wanted to share for anyone who wants to move to a m720q tiny who has 2.5/5/10gb AT&T fiber.

I'm in the same boat as you. I have AT&T 2 gbps symmetrical service. Have a SuperMicro AOC-STGN-i2s sitting around. So was thinking of using that with a new OPNSense box if I can't find a decent 4 port 2.5 nic that doesn't have the buggy issues with i225. Before you dumped the AOC-STGN- i2s - did you look at possibly getting a SPF+ module that will negotiate 2.5/5 gb with the BGW320? I saw some comments on Amazon review that the 10Gtek and Wiitek tranceivers seem to able to negotiate 2.5/5 gb. I'm going to order the Wiitek SFP module and see if it will work between the AOC_STGN-i2s and the 5 gb port on AT&T's gateway.

 

[elseless]

That's genuinely surprising to me. I feel like even casually supporting things like this leads to more problems, especially if this spec was never intended to be supported.

I say this as someone who also has to push back on unusual asks in my day job...

That said, here we go. Not expecting a whole lot (two nvme slots, a SATA port, and a pci-e slot that can do way more with storage still not enough for you?). Well, no.

Thanks a lot for asking them about it. From that Lenovo tech's response, it sounds like they don't ever plan to support booting directly from the 2230 A+E slot. I just bought a P350 Tiny for Proxmox, and, like many others, want to keep both 2280 slots available as separate storage devices. I'll also be installing a ConnectX-4 Lx in the big PCIe slot.

I've seen posts where people manage to stuff a SATA SSD underneath the big PCIe card, but that gets a little messy, as in order to fit, the SSD PCB needs to be removed from its metal/plastic housing and wrapped in thin non-conductive tape/pads. For me, a better solution is to put rEFInd (Clover also works) on a tiny USB drive plugged into the back, and then use that to trampoline into Proxmox on the 2230.

So now I have a WD SN740 and A+E to M.2 adapter on order from AliExpress (knock on wood). Even attached to PCIe Gen. 3 x 2 (as they mentioned), it should still be 4X faster than SATA.

 

[joeribl]

Thanks a lot for asking them about it. From that Lenovo tech's response, it sounds like they don't ever plan to support booting directly from the 2230 A+E slot. I just bought a P350 Tiny for Proxmox, and, like many others, want to keep both 2280 slots available as separate storage devices. I'll also be installing a ConnectX-4 Lx in the big PCIe slot.

I've seen posts where people manage to stuff a SATA SSD underneath the big PCIe card, but that gets a little messy, as in order to fit, the SSD PCB needs to be removed from its metal/plastic housing and wrapped in thin non-conductive tape/pads. For me, a better solution is to put rEFInd (Clover also works) on a tiny USB drive plugged into the back, and then use that to trampoline into Proxmox on the 2230.

So now I have a WD SN740 and A+E to M.2 adapter on order from AliExpress (knock on wood). Even attached to PCIe Gen. 3 x 2 (as they mentioned), it should still be 4X faster than SATA.

Beware that several people had problems with the ConnectX-4. Someone posted a hack some days ago, to get it working. But it requires soldering.

 

[elseless]

Thanks, I totally missed that! I see Jakob's post on the previous page now. I'll test it out on my end too but almost certain it'll fail due to that weird back-voltage issue. Kind of an odd engineering choice by Mellanox, but whatever. Will likely just go with ConnectX-3 instead of the ConnectX-4 + solder mod. That card and its driver (mlx4) are both getting a bit long in the tooth, but I've not had any problems with them on Proxmox 8 w/kernel v. 6.5.

EDIT: ConnectX-4 Lx (MCX4121A-ACAT) hardware revisions AG and newer appear to have fixed the back-voltage issue, and do not require the diode solder mod (see posts below).

 

[andrea87]

Time to join the thread. After my past woes with chinese mini PCs, and with a clean possibility of getting a proper FTTH connection in the future, I've decided to build myself a dedicated Proxmox/OPNsense box with one of those.

Found off ebay a very basic M720q with a g5400t, then added some ram, a pcie riser and a dual 2.5g nic based on the i226-v b4 chip.



Plenty of those on Ebay and Aliexpress, price is usually in the 40-45$ range. They look nice, have actual i226-v chips on them, the only downside is that they run off a pcie bridge - asm1182e - which limits the whole card to a single lane of pcie gen2. With the two nics connected with an ethernet cable, and each one assigned to a different network bridge in proxmox, running a bidirectional iperf between two containers I get max 1.43gbit/s throughput.

Here are some iperf results for anyone interested:

Spoiler: Iperf3 Bidirectional

root@test-ct-1:~# iperf3 -c 192.168.64.11 --bidir -i 1 -t 10
Connecting to host 192.168.64.11, port 5201
[ 5] local 192.168.64.10 port 49042 connected to 192.168.64.11 port 5201
[ 7] local 192.168.64.10 port 49054 connected to 192.168.64.11 port 5201
[ ID][Role] Interval Transfer Bitrate Retr Cwnd
[ 5][TX-C] 0.00-1.00 sec 167 MBytes 1.40 Gbits/sec 0 1.41 MBytes
[ 7][RX-C] 0.00-1.00 sec 170 MBytes 1.43 Gbits/sec
[ 5][TX-C] 1.00-2.00 sec 169 MBytes 1.42 Gbits/sec 0 1.41 MBytes
[ 7][RX-C] 1.00-2.00 sec 168 MBytes 1.41 Gbits/sec
[ 5][TX-C] 2.00-3.00 sec 170 MBytes 1.43 Gbits/sec 0 1.41 MBytes
[ 7][RX-C] 2.00-3.00 sec 168 MBytes 1.41 Gbits/sec
[ 5][TX-C] 3.00-4.00 sec 170 MBytes 1.43 Gbits/sec 0 1.41 MBytes
[ 7][RX-C] 3.00-4.00 sec 169 MBytes 1.41 Gbits/sec
[ 5][TX-C] 4.00-5.00 sec 169 MBytes 1.42 Gbits/sec 0 1.41 MBytes
[ 7][RX-C] 4.00-5.00 sec 168 MBytes 1.41 Gbits/sec
[ 5][TX-C] 5.00-6.00 sec 170 MBytes 1.43 Gbits/sec 0 1.48 MBytes
[ 7][RX-C] 5.00-6.00 sec 169 MBytes 1.41 Gbits/sec
[ 5][TX-C] 6.00-7.00 sec 169 MBytes 1.42 Gbits/sec 0 1.48 MBytes
[ 7][RX-C] 6.00-7.00 sec 169 MBytes 1.41 Gbits/sec
[ 5][TX-C] 7.00-8.00 sec 170 MBytes 1.43 Gbits/sec 0 1.55 MBytes
[ 7][RX-C] 7.00-8.00 sec 169 MBytes 1.41 Gbits/sec
[ 5][TX-C] 8.00-9.00 sec 170 MBytes 1.43 Gbits/sec 0 1.64 MBytes
[ 7][RX-C] 8.00-9.00 sec 168 MBytes 1.41 Gbits/sec
[ 5][TX-C] 9.00-10.00 sec 169 MBytes 1.42 Gbits/sec 0 1.72 MBytes
[ 7][RX-C] 9.00-10.00 sec 168 MBytes 1.41 Gbits/sec

[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.00 sec 1.65 GBytes 1.42 Gbits/sec 0 sender
[ 5][TX-C] 0.00-10.00 sec 1.65 GBytes 1.42 Gbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 1.65 GBytes 1.42 Gbits/sec 5 sender
[ 7][RX-C] 0.00-10.00 sec 1.65 GBytes 1.41 Gbits/sec receiver
iperf Done.

Spoiler: Iperf3 - 2500 / 500

root@test-ct-1:~# iperf3 -c 192.168.64.11 -i 1 -t 10 -b 2500M
Connecting to host 192.168.64.11, port 5201
[ 5] local 192.168.64.10 port 39248 connected to 192.168.64.11 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 244 MBytes 2.05 Gbits/sec 0 660 KBytes
[ 5] 1.00-2.00 sec 242 MBytes 2.03 Gbits/sec 0 718 KBytes
[ 5] 2.00-3.00 sec 242 MBytes 2.03 Gbits/sec 0 877 KBytes
[ 5] 3.00-4.00 sec 242 MBytes 2.03 Gbits/sec 0 1.04 MBytes
[ 5] 4.00-5.00 sec 236 MBytes 1.98 Gbits/sec 35 854 KBytes
[ 5] 5.00-6.00 sec 242 MBytes 2.03 Gbits/sec 0 854 KBytes
[ 5] 6.00-7.00 sec 242 MBytes 2.03 Gbits/sec 0 854 KBytes
[ 5] 7.00-8.00 sec 242 MBytes 2.03 Gbits/sec 0 857 KBytes
[ 5] 8.00-9.00 sec 242 MBytes 2.03 Gbits/sec 0 857 KBytes
[ 5] 9.00-10.00 sec 250 MBytes 2.10 Gbits/sec 0 919 KBytes

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.37 GBytes 2.03 Gbits/sec 35 sender
[ 5] 0.00-10.00 sec 2.37 GBytes 2.03 Gbits/sec receiver

iperf Done.

root@test-ct-2:~# iperf3 -c 192.168.64.10 -i 1 -t 10 -b 500M
Connecting to host 192.168.64.10, port 5201
[ 5] local 192.168.64.11 port 57736 connected to 192.168.64.10 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 59.6 MBytes 500 Mbits/sec 0 257 KBytes
[ 5] 1.00-2.00 sec 59.6 MBytes 500 Mbits/sec 0 270 KBytes
[ 5] 2.00-3.00 sec 59.6 MBytes 500 Mbits/sec 0 303 KBytes
[ 5] 3.00-4.00 sec 59.5 MBytes 499 Mbits/sec 0 389 KBytes
[ 5] 4.00-5.00 sec 59.6 MBytes 500 Mbits/sec 0 409 KBytes
[ 5] 5.00-6.00 sec 59.6 MBytes 500 Mbits/sec 0 515 KBytes
[ 5] 6.00-7.00 sec 59.6 MBytes 500 Mbits/sec 0 515 KBytes
[ 5] 7.00-8.00 sec 59.6 MBytes 500 Mbits/sec 0 515 KBytes
[ 5] 8.00-9.00 sec 59.6 MBytes 500 Mbits/sec 0 515 KBytes
[ 5] 9.00-10.00 sec 59.5 MBytes 499 Mbits/sec 0 540 KBytes

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 596 MBytes 500 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 596 MBytes 500 Mbits/sec receiver

iperf Done.

Spoiler: Iperf3 - 2000 / 700

root@test-ct-1:~# iperf3 -c 192.168.64.11 -i 1 -t 10 -b 2000M
Connecting to host 192.168.64.11, port 5201
[ 5] local 192.168.64.10 port 35700 connected to 192.168.64.11 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 231 MBytes 1.94 Gbits/sec 0 631 KBytes
[ 5] 1.00-2.00 sec 227 MBytes 1.90 Gbits/sec 0 631 KBytes
[ 5] 2.00-3.00 sec 227 MBytes 1.90 Gbits/sec 0 631 KBytes
[ 5] 3.00-4.00 sec 220 MBytes 1.84 Gbits/sec 0 1.09 MBytes
[ 5] 4.00-5.00 sec 226 MBytes 1.90 Gbits/sec 0 1.14 MBytes
[ 5] 5.00-6.00 sec 226 MBytes 1.90 Gbits/sec 0 1.20 MBytes
[ 5] 6.00-7.00 sec 227 MBytes 1.90 Gbits/sec 0 1.20 MBytes
[ 5] 7.00-8.00 sec 226 MBytes 1.89 Gbits/sec 0 1.32 MBytes
[ 5] 8.00-9.00 sec 227 MBytes 1.90 Gbits/sec 0 1.69 MBytes
[ 5] 9.00-10.00 sec 227 MBytes 1.90 Gbits/sec 0 1.96 MBytes

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.21 GBytes 1.90 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.21 GBytes 1.90 Gbits/sec receiver

iperf Done.

root@test-ct-2:~# iperf3 -c 192.168.64.10 -i 1 -t 10 -b 700M
Connecting to host 192.168.64.10, port 5201
[ 5] local 192.168.64.11 port 43742 connected to 192.168.64.10 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 83.4 MBytes 699 Mbits/sec 0 205 KBytes
[ 5] 1.00-2.00 sec 83.5 MBytes 700 Mbits/sec 0 205 KBytes
[ 5] 2.00-3.00 sec 83.5 MBytes 700 Mbits/sec 0 216 KBytes
[ 5] 3.00-4.00 sec 83.4 MBytes 699 Mbits/sec 0 496 KBytes
[ 5] 4.00-5.00 sec 83.5 MBytes 700 Mbits/sec 0 523 KBytes
[ 5] 5.00-6.00 sec 83.4 MBytes 699 Mbits/sec 0 550 KBytes
[ 5] 6.00-7.00 sec 83.5 MBytes 700 Mbits/sec 0 550 KBytes
[ 5] 7.00-8.00 sec 83.4 MBytes 699 Mbits/sec 0 588 KBytes
[ 5] 8.00-9.00 sec 83.4 MBytes 697 Mbits/sec 0 618 KBytes
[ 5] 9.00-10.00 sec 83.6 MBytes 704 Mbits/sec 0 618 KBytes

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 834 MBytes 700 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 834 MBytes 700 Mbits/sec receiver

iperf Done.

Spoiler: Iperf3 2500/250

root@test-ct-1:~# iperf3 -c 192.168.64.11 -i 1 -t 10 -b 2500M
Connecting to host 192.168.64.11, port 5201
[ 5] local 192.168.64.10 port 39204 connected to 192.168.64.11 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 267 MBytes 2.24 Gbits/sec 0 725 KBytes
[ 5] 1.00-2.00 sec 260 MBytes 2.19 Gbits/sec 0 882 KBytes
[ 5] 2.00-3.00 sec 261 MBytes 2.19 Gbits/sec 0 939 KBytes
[ 5] 3.00-4.00 sec 261 MBytes 2.19 Gbits/sec 0 1.11 MBytes
[ 5] 4.00-5.00 sec 261 MBytes 2.19 Gbits/sec 0 1.11 MBytes
[ 5] 5.00-6.00 sec 261 MBytes 2.19 Gbits/sec 0 1.56 MBytes
[ 5] 6.00-7.00 sec 261 MBytes 2.19 Gbits/sec 0 1.81 MBytes
[ 5] 7.00-8.00 sec 261 MBytes 2.19 Gbits/sec 0 1.81 MBytes
[ 5] 8.00-9.00 sec 261 MBytes 2.19 Gbits/sec 0 1.81 MBytes
[ 5] 9.00-10.00 sec 261 MBytes 2.19 Gbits/sec 0 1.81 MBytes

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.56 GBytes 2.20 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.55 GBytes 2.19 Gbits/sec receiver

iperf Done.

root@test-ct-2:~# iperf3 -c 192.168.64.10 -i 1 -t 10 -b 250M
Connecting to host 192.168.64.10, port 5201
[ 5] local 192.168.64.11 port 45550 connected to 192.168.64.10 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 29.9 MBytes 251 Mbits/sec 0 134 KBytes
[ 5] 1.00-2.00 sec 29.8 MBytes 250 Mbits/sec 0 134 KBytes
[ 5] 2.00-3.00 sec 29.9 MBytes 251 Mbits/sec 0 134 KBytes
[ 5] 3.00-4.00 sec 29.8 MBytes 250 Mbits/sec 1 134 KBytes
[ 5] 4.00-5.00 sec 29.8 MBytes 250 Mbits/sec 0 134 KBytes
[ 5] 5.00-6.00 sec 29.9 MBytes 251 Mbits/sec 0 160 KBytes
[ 5] 6.00-7.00 sec 29.8 MBytes 250 Mbits/sec 0 160 KBytes
[ 5] 7.00-8.00 sec 29.9 MBytes 251 Mbits/sec 0 160 KBytes
[ 5] 8.00-9.00 sec 29.8 MBytes 250 Mbits/sec 0 160 KBytes
[ 5] 9.00-10.00 sec 29.8 MBytes 250 Mbits/sec 0 160 KBytes

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 298 MBytes 250 Mbits/sec 1 sender
[ 5] 0.00-10.00 sec 298 MBytes 250 Mbits/sec receiver

iperf Done.

The performance results (with virtIO, through two debian LXC containers) are not bad, and shouldn't provide too big restrictions on any 2.5/1gb GPON fiber. Currently I do have OPNsense running with VirtIO networking for ease of backup / migration, in the future I might just pcie passthrough the whole card to it as the mini pc will be essentially dedicated to internet connection, routing and such with no other major vms running.

Power and Thermals.

The box, at idle with just the hypervisor running (governor conservative and acpi-cpufreq driver) and the main nic, draws off the plug about 3.5-4W. With the card installed, the power draw raises significantly to about 8-9W at idle with the two nics up at 2.5gb speed. With no additional active airflow and the case closed the card's heatsink reach easily 60-65°C. As both the pcie bridge and the two nics are connected to it with 4mm thermal pads, I expect their temperature to have been at least 10-15°C above. It's a bit too hot for me to run them 24/7.

As the card is fairly small, there is enough space for a second fan. Instead of adding any, I've decided to cut away a small piece of the main fan's air guide to allow some airflow out to the side. Temporarily, I've added a couple pieces of masking tape to divert the air torwards the NIC. Results so far have been very good, the heatsink temp doesn't go over 40-45°C. I've set the fan control in the bios to something akin to "better cooling" instead of "better acoustic performance". The mini PC lives inside a closed 9U network rack with a single fan pushing air out of the top and ~27°C inside air temp.



The card is held with a custom 3d printed bracket, if anyone is interested I'll post the stl file somewhere.

 

[PnoT]

Does anyone know of a baffle that will work for the SuperMicro AOC-STG-I2 (dual port 1g) card on the M920x?

 

[Richard Berg]

Dumb questions for people turning these boxes into 10gbe routers:

• If you buy a single port NIC, won’t you have to give up on WOL & vPro? Surely you want the fast port facing the LAN, which means the onboard port becomes the WAN, which won’t be reachable from your management/controller apps.
• Ok so you buy dual port, and cable one of them to your WAN…how? A SFP+ to RJ45 transceiver runs $50 used, which is like double the cost of the NIC itself.
• How do you measure real-world throughput? I ran some basic speed tests (Google, Ookla, Verizon) and got the same results whether cabling my PC directly to the ONT or sticking a 12-year-old consumer router in front: about 940 down / 840 up both ways. But this may not be reflective of routing performance when there are hundreds of TCP connections open in both directions?

 

[jlist]

Hello all, I've been checking this thread for a while and I currently have a M75Q and P330, it's been very helpful. I only wish that the thread was further separated (maybe one thread for 2-3 generations?) so the discussions are more specific. With 75 pages of information, it seems like this has been more popular than originally anticipated.

I currently have a GPU installed in the P330 and it seems like there's enough space to cram in a small SATA SSD, but the cable isn't long enough to snake around the GPU. I see that there's a longer cable originally for Lenovo laptops (5B40S22058, available on ebay and aliexpress), but I can't find any anecdotes of it working out.

Has anyone else found a longer SATA cable which is compatible with a Lenovo Tiny?

 

[elseless]

A few measurements later:
With a CX3 connected, all three Voltages (3V3_Aux, 3V3 and 12V) are present. With the CX4 only 3V3_Aux is at 3.3V. 3V3 is at ~100mV and 12V is at ~500mV.
As the T600 the system came with draws a lot of power on the 12V rail I'd guess that the card is drawing too much power on the 3V3 rail during startup.

Other things I noticed:
- The CX4 does not seem to use the 3V3 rail at all? There is a footprint on the card for a polyfuse on the 3V3 rail that is not populated. The "too much power draw on 3V3 rail" is therefore not the culprit.
- Taping over *all* pins except ground, 3V3, 3V3_Aux and 12V has the same result so definitely a power supply thing.
- Sadly the power supply part of the P350 tiny and M920Q are wildly different and I only have the schematic for the latter so debugging is a bit harder than expected. Seeing that the M920Q apparently has the same issues though the schematic of the M920Q might help. The 12V regulator looks *very* similar. The layout is different but the rough position on the board and the schematic seem to be almost identical.

UPDATE:
I removed the polyfuse on the CX4's 12V line and connected an external 12V supply. This is working fine and the card is detected and works.
The two potential solutions I'm currently thinking of are either to add a DCDC from the DC input's 20V to 12V or to try some kind of filter as I suspect some weird interaction between the DCDC that generates the 12V and the DCDCs on the CX4.

UPDATE2:
It works! The culprit is, that the CX4 has some form of back-flow from the 3V3_Aux supply to the 12V supply (~1.8V) which seems to confuse the DCDC on the P350 Tiny.
Replacing the polyfuse on the 12V line of the CX4 with polyfuse and one of these cheap "ideal diode" boards from Aliexpress makes the card work

I have a bit of good news... @Jakob: what hardware revision is your ConnectX-4 Lx? And it's a Mellanox proper card, not Dell/HP/Lenovo/etc. OEM? (I know that Dell in particular has a CX-4 Lx with a quite different PCB design than the official Mellanox one -- on the Dell one, the SFP28 cages are much closer together.)

Right now, I own one Mellanox MCX4121A-ACAT Rev. AG, and one Rev. AK. In my P350 w/170W adapter (which also came stock with the NVIDIA T600), both the AG and the AK boot up just fine and are fully recognized by the driver. So it seems that at least later revisions (Rev. AG on, possibly earlier) of official Mellanox cards are immune to the problem. See attached for a shot of the back of their PCBs; perhaps there's some obvious difference between those and yours?

 

[Dunadan]

My new Proxmox Hypervisor
M720q with 9500T
2x 32GB RAM 2666
Proxmox SSD: either 850 Pro SATA or M.2 2230
4TB Lexar NM790 for data
NIC: either X520-DA2 10GBit SFP+ (existing), or X550-T2 (searching: one of the few which should be able to handle also 2.5GBit)

proxmox:

• OPNSense
• PiHole
• Cloudflared
• Omada Controller
• Home Assistant
• TrueNAS Scale (to use 4TB NVME as NFS storage for Nextcloud & Kavita)
• Nextcloud
• PhotoPrism (pictures stored on my Fileserver)
• Jellyfin (Movies, Music stored on my Fileserver)
• Kavita
• Paperless-ngx

Hello
What RAM Sticks are you using?

 

[Jakob]

I have a bit of good news... @Jakob: what hardware revision is your ConnectX-4 Lx? And it's a Mellanox proper card, not Dell/HP/Lenovo/etc. OEM? (I know that Dell in particular has a CX-4 Lx with a quite different PCB design than the official Mellanox one -- on the Dell one, the SFP28 cages are much closer together.)

Right now, I own one Mellanox MCX4121A-ACAT Rev. AG, and one Rev. AK. In my P350 w/170W adapter (which also came stock with the NVIDIA T600), both the AG and the AK boot up just fine and are fully recognized by the driver. So it seems that at least later revisions (Rev. AG on, possibly earlier) of official Mellanox cards are immune to the problem. See attached for a shot of the back of their PCBs; perhaps there's some obvious difference between those and yours?

My two are both MCX4121A-ACAT Rev AF.

The AG revision you posted does not seem to have the 3V3_AUX Fuse populated. If you don't need any WoL functionality it might be viable to just remove this fuse on the AF revision too (without this fuse WoL and any other things that need to work when the system is off will certainly not work).
Apart from that fuse my board looks identical to the AG rev board you posted (same board revision PCB001141_00 and identical component placement apart from that one fuse).

The AK rev of yours seems very similar as well but it has the 3V3_AUX fuse populated and has a different PCB revision. They might have changed something on the front side around the power delivery stuff to actually prevent this backflow.

Thanks for testing and sharing!